What the score actually proves.

Five checks, one explicit contract

A run passes only when all five checks pass: the expected disposition, the tool-call budget, the read-only boundary, evidence supporting the answer, and the optional expected numeric value. Every attempted tool invocation counts; a blocked policy decision does not.

Answer evidence must point to a successful read of the requested tool, with matching arguments, metric and time window, at least one result row, a finite numeric value and a named source. The cited value must equal the answer. These checks are tailored to scalar metric-query workflows.

The example policies are deterministic

The direct policy accepts the first successful reply. The guarded policy applies a fixed read-only gate, timeout retry limit and evidence validation. Neither policy reads the expected contract while producing its output. Both use synthetic tool replies and a fixture clock; no language model participates.

Passing these eight authored cases is not evidence of general Agent quality. Free-form reasoning, semantic SQL correctness, source authenticity, concurrent spans and actual authorization enforcement require other checks. The “source” field is an assertion in a record, not an attestation.

Pairing, timing and missing information

Case-level comparison requires matching requests, contracts and timing modes. The workbench reports unmatched cases separately. All fixture durations are sums of declared tool-response times. Recorded durations are provided by the importer, include any gaps, and are not independently verified. Mixed timing modes are not added into a displayed duration.

There is no token-cost estimate, model ranking or statistical significance claim. The example suite is small and deliberately selected to reveal specific failure modes.

Format and processing limits

Browser imports are limited to 2 MB, 200 runs and 100 sequential steps per run. Step IDs must be unique within a run; policy/case pairs must be unique within a bundle. Overlapping, reversed or out-of-range step timings are rejected. Results are recomputed from records rather than trusted from a saved “pass” field.

Files are processed in this tab. The site has no upload endpoint, analytics SDK or model integration. The downloaded kit and browser share the same evaluation engine. Close the tab to discard imported data; use export to keep a result.

Sources and next steps

Langfuse datasets provide a useful reference for collecting test cases. OpenTelemetry’s tracing API is background for spans and events; this Lab implements a smaller sequential native format. See the downloadable format reference before writing an adapter.