Open measurement infrastructure for agent-facing software
Know what your agent usage metrics actually count.
Separate logical operations from attempts, preserve retry and cost evidence, and show what your telemetry can — and cannot — prove.
Read the Specification →01 — The problem
The software consumer changed.
The measurement stack didn’t.
Three attempts of one operation are not three billable operations — unless the metering policy says so.
Agent systems collapse these states into vague “tool usage.”
AgentMeasure keeps them separate.
Agents hammer your endpoint; retries inflate every count.
Price by operation, not by noiseBilling rides on usage numbers — with CI and synthetic traffic inside them.
Meters that survive an auditEvery runtime counts differently, so comparisons are folklore.
Same semantics across five runtimes02 — Claim discipline
Similar events. Different facts.
Attempt≠Operation
Attempts are execution facts. Operations are logical intents. Retries are multiple attempts of one operation — not “2 operations with 50% success.”
Returned≠Consumed
The tool returned a result. Did the caller actually use it? Returned is observable; consumed needs evidence. Unknown is a valid, first-class answer.
Observed≠Inferred
A selection is not a preference — it can be made by the model, a router, a policy, a workflow, or the user. AgentMeasure records what happened before inferring why.
03 — The measurement model
From reach to value.
Metric families, not a universal KPI — the same semantics map onto search, booking, and compute.
Effect confirmation — planned
Defined does not mean observable in every runtime. Semantics are spec status; observation depends on the harness — see the observability matrix ↓
04 — How it works
Provider-side measurement.
No agent-side install.
No agent-side install
Third-party agents never need AgentMeasure. Measurement happens where the capability is delivered.
Provider-side measurement
The SDK wraps your tool handlers and emits canonical observations — one schema, six payload classes.
Not on the critical path
Non-blocking, durable spool with loss accounting. If the SDK fails, your server doesn’t.
Your data can stay local
JSONL on your machine, analytics generated locally. Hosted ingestion is explicit opt-in, later.
MCP is the first reference surface, not a requirement. Closed-source software is fine.
Telemetry records events. AgentMeasure defines what those events mean.
05 — Available today
What works today.
Shipped, with tests, conformance vectors, and published profiles.
Numbers track releases and the roadmap.
06 — Try it
See the model in two minutes.
What the demo proves
- ● Canonical observations — one portable schema
- ● Operation / attempt correlation, fail-closed
- ● Caller attribution — correlated · declared · unknown
- ● Local analytics without any cloud
- ● Deterministic replay — reproducible numbers
07 — External provider trial
Find out what your agent traffic actually is.
7 days. One capability. Your data stays on your machine.
How many calls are actually one logical operation?
What does one successful operation really cost?
How much “production” traffic is CI, synthetic, or unknown?
You get: a Measurement Report plus a 45-minute review call — findings as business conclusions, each with its evidence class.
Three providers, one capability each. If the method breaks on your data, we publish that — the discipline already applied to six public claims in Benchmark Run #001.
08 — Open standard
One measurement language
across agent runtimes.
A public record of observation blind spots — what each runtime can and cannot observe.
| Runtime | Operation | Attempt | Consumption | Delegation |
|---|---|---|---|---|
| Claude Code | ● | ● | ◐ | ◐ |
| Codex | ● | ● | ○ | ○ |
| DeepSeek Harness | ● | ● | ○ | ● |
| Pydantic AI | ● | ● | ○ | ○ |
| OTel GenAI | ◐ | ● | ○ | ○ |
Harness profiles, Draft 0.4.4 — a fixed questionnaire per runtime, with the mapping rules and blind spots in writing. Read the profiles →
09 — Explore Lab
Measure usage. Then measure causality.
What happened?
Observation · correlation · evidence-graded facts
Did the change actually improve it?
Preregistered experiments · effect sizes · honest nulls
10 — Long-term thesis
Software is becoming callable.
Before capabilities can be priced, compared, or purchased by agents,
they need a shared measurement layer.
That layer is the foundation for Capability as a Service — a direction, not a product sold today. The standard defines economic facts; money movement stays with payment rails.