The refund that never issued. The records it never wrote. The account it updated on the wrong customer. Your trace shows all green, because a trace only records what the agent says about itself. Sigma checks every claimed action against your real systems, and shows you what actually happened.
Two things stand between your agent and trouble in production: your traces and your guardrails. Both look at the action before or as it happens. Neither looks at your system afterward to confirm it worked. That blind spot is where silent failures live.
It records the call and the 200 that came back, not whether Stripe, Zendesk, or your database actually changed.
The write can be dropped, ignored, or applied to the wrong record and still return success.
"Done, refund issued" comes from the same run that made the mistake. If it hallucinated the action, it hallucinates the success too.
Good teams gate risky tools with amount thresholds, approval steps, and eligibility checks. They stop the obvious bad request, going in. They cannot confirm a good request actually landed, or catch a bad one shaped to look fine. Here is what passes a well built guardrail and still breaks.
passes: a $40 refund is correctly auto-approved. The gate did its job.
breaks: the call returns 200, the write silently drops. Money never moves.
passes: the agent meant $24.70, wrote $247. A valid number under $500, so every check clears.
breaks: it lands. The customer is overpaid ten times over.
passes: each refund is individually valid and under threshold, so each is approved.
breaks: a retry fires a second refund for the same intent. Two land.
passes: the refund is approved and lands cleanly. The gate did its job.
breaks: the request was cancel-and-refund. The agent refunded, never fired the cancel, and reported both done. The subscription renews and bills the customer again.
Observability, evals, and guardrails all act before or during the run. Sigma is the layer that acts after, against reality. It does not replace them. It closes the gap they leave.
| Layer | When it acts | What it sees |
|---|---|---|
| Observability | during the run | what the agent reported about itself |
| Evals | before production | performance against answers you already have |
| Guardrails | before an action | whether a request breaks a rule |
| Sigma | after a run | whether it actually happened in your real systems |
That verified layer is the foundation. On top of it, Sigma gives every team a cockpit view of their agents: reliability, cost per real outcome, experience, and policy adherence.
Point Sigma at your agent's traces. It sees every action the agent claims to take.
Connect where the work actually lands: billing, support, CRM, ERP, your own database, your code tools. If you run an approval gate, connect that too and we will show you what it let through. If your team touches it, it probably fits.
One config says which tools write, and what counts as a real outcome for each agent. Your definition, your source of truth. Sensible defaults ship for common systems.
Sigma matches each claimed action to reality and gives it a verdict, with a confidence: confirmed when your system shows it directly, inferred when your system does not expose it directly.
A top-line health view, down to each agent, into each run. Sample from a support and billing deployment, 24 hours. The runs below flag what is worth a look.
Cost per outcome divides real spend (model plus tool) by real successes, never by attempts. It is $0.13 across every run, and $0.19 where you have written explicit criteria, because a defined success is a higher bar met less often.
- run map -Every run sits on two axes at once: whether it had to write to a real system, and whether you wrote a rule for success. The four cells are the same runs, so transactional and defined are two edges of one grid, not two totals.
Defined work is mostly transactional (the tall cyan block): teams write rules for the actions that move money and records. Inferred is almost entirely informational (the wide slate block), the questions nobody wrote a rule for. Open any run below to see the verdict behind a cell.
Before any number means anything, you get an honest map of your own coverage: which systems we can read, which we cannot, where success is explicitly defined and where it is inferred. No guessing, no quiet gaps. We walk you through yours on a call.
I am building Sigma solo. It is early: parts of the core are built and tested against tau-bench, a public agent benchmark, and I am now looking for the first design partners running real agents in production.
If that is you, I would rather learn how you verify your agents today than pitch you. No polished product yet, just a sharp problem and working pieces.
I read every one of these myself.