Every published number has a harness behind it.
Aegis publishes two latency figures: 2.9 ms p95 for the prompt and response rails, and 14 ms p95 for full tool & MCP inspection against a 50 ms budget. This page documents exactly how those numbers are produced — the workload, the percentile arithmetic, the hardware, and what is deliberately excluded — so a claim you cannot check becomes one you can.
Two figures, two harnesses, one rule.
Both loads are defined once and used twice. On every commit the suite runs them and
asserts each against its budget — the published 50 ms gate for tool inspection,
and for the rails an internal bound well inside the 300 ms budget they answer to.
That is a regression guard, and deliberately a coarse one: it catches a rail that has
turned pathological, not a figure that has drifted a few percent. The numbers published
on this page come from somewhere stricter. They are produced on demand by
make benchmark-baseline as the median of 5
repetitions on an idle machine, frozen into a committed artifact, and substituted into
this page at build time — so a published figure moves when someone regenerates the
evidence, and cannot drift quietly in between.
2.9 ms — the prompt & response rails
Measures one full pass of the deterministic input and output rails: 3000 iterations over a rotating mix of a benign request, a prompt-injection attempt, and a large (multi-kilobyte) input. The timer wraps only the engine calls — no HTTP, no serialization, no upstream model. This is the overhead added to a turn that carries no tool calls.
14 ms — full tool & MCP inspection
Measures the execution rail — name policy, least-privilege constraints, dangerous-action detection, argument scanning, and tool-result validation — under a deliberately pessimistic setup: a 55-glob enterprise policy with 8 constraint groups, a payload mix that exercises every branch of the precedence chain, and a 32 KiB nested-JSON argument in one sample out of seven.
50 ms — the number that is a promise
Absolute milliseconds vary with hardware; the budget does not. The published gate — tool inspection under 50 ms p95 — is asserted by the test suite on every commit, at the worst-case payload size. That payload measures 19.29 ms p95, a 2.5× margin against the 50 ms budget; the run it is read from is in the next section. If your hardware is slower than ours, that margin is what absorbs the difference.
The output the two figures above are read from.
The harness writes a JSON artifact and the site is built from it. Every number in this table is substituted at build time, so a figure on this page cannot disagree with the measurement it came from — and the site does not build at all if the artifact is missing, unparseable, or short a figure.
| Workload | Iterations | p50 | p95 | p99 |
|---|---|---|---|---|
| Prompt & response rails | 3000 | 0.06 ms | 2.85 ms | 2.96 ms |
| Tool & MCP inspection | 2000 | 0.02 ms | 14.36 ms | 14.47 ms |
| Tool inspection, worst-case payload | 200 | 18.92 ms | 19.29 ms | 19.98 ms |
Run 2026-08-29 on Apple M4 Pro (14-core, macOS 26.5), CPython 3.11.16, coverage instrumentation off — instrumentation inflates the same workload measurably, so a run carrying it is not publishable. Each figure is the median of 5 independent repetitions rather than one run: the first artifact this harness produced was a single reading that happened to be the lowest of nine taken that afternoon, and it sat just under a rounding boundary, so one run decided what this page said. Repetitions are taken on an otherwise idle machine, and the harness refuses to measure a busy one — the gate margin is 50 divided by a tail figure, so load enters a published number as a divisor rather than as noise. At the precision each figure is published to, every repetition read 2.8–2.9 ms for the rails and 14 ms for the tool guard; each one is recorded individually in the artifact, at full precision, rather than summarised away.
The rails p50 and p95 sit far apart on purpose. That workload rotates three inputs and one of them is a 5416-character body, so the p95 is a statement about the large sample and the p50 about the two small ones. A rails figure quoted without the payload it was measured on cannot be checked by anyone, and an uncheckable figure is the kind that goes stale quietly — which is what the last row of the history table below records.
make benchmark
# Runs these three workloads in-process on your own machine: no gateway, no
# model, no network, no database. Prints p50/p95/p99 per workload and diffs
# them against the committed baseline this page is built from.The details that make a percentile honest.
A p95 can hide a lot: what was sampled, how the rank was computed, what the machine was doing, and whether the slow tail was trimmed. Here is each choice, stated.
Timing
A monotonic high-resolution timer (time.perf_counter()) wraps the engine
calls and nothing else. Every iteration is recorded — no warm-up discards, no outlier
trimming. The slow tail is part of the number.
Percentile definition
Nearest-rank on the sorted sample: p95 is the value at rank ⌈0.95 · n⌉. At small n this makes p95 equal the maximum, which is correct — a percentile that reads below an observed value is the misleading kind.
Hardware & environment
One machine: Apple M4 Pro (14-core, macOS 26.5), running CPython 3.11.16 — a single process,
standard library, no accelerator. Every figure on this page was measured there and
nowhere else, and the artifact names it, so each number reads as "on this hardware"
rather than as a universal; the second environment is your own, which is what
make benchmark is for. Published figures come from uninstrumented runs:
the suite normally runs under coverage tooling, which inflates the same workload
measurably, so gate assertions leave headroom for it.
Pessimistic by construction
The tool-inspection workload is worse than production on purpose: every allow-glob is a potential match attempt, constraints add a path pass over every argument leaf, and the oversized sample is an order of magnitude past a realistic call. Payloads beyond the scan bound are refused, not truncated — truncation would be a bypass, so it is not available as a performance optimization either.
End-to-end, separately
A separate concurrent load harness fires thousands of requests at a running gateway over HTTP and reports throughput and latency percentiles with the upstream model stubbed out. It answers a different question — gateway overhead including transport — and is deliberately not the source of the per-rail figures.
What is excluded, and why
Your model's inference time, the network hop to your gateway, and your tools' execution time. Aegis runs in-process in your own infrastructure, so there is no vendor round-trip to include — which is the point of comparison with network-hop guardrail APIs, and the reason the honest comparison is budget against budget.
A latency figure is a liability the moment it goes stale.
Detectors are code, and code grows. Every time added coverage has cost measurable latency, the published number changed with it — in public, with the reason. This table is the complete history.
| Date | What changed | Measured effect | Published figure |
|---|---|---|---|
| Aug 2026 | Thirteen non-English detector language packs added to the input rails | 1.19× on the rails benchmark, same machine, with and without | Rails 1.4 → 1.7 (superseded) |
| Aug 2026 | Detector coverage for reverse shells, download-and-execute, credential stores and destructive cloud-CLI actions — the cost of nine closed bypasses from the internal red team | Mixed-workload tool inspection roughly 7.6 → 13 on the same harness; the 50 ms gate re-measured and still held | Tool guard 7.6 → 13 (superseded) |
| Aug 2026 | A defect in our own percentile arithmetic: p95 was computed one rank low whenever the rank fell on a fraction, so the p95 of two samples returned the lower one | Found by exercising the deployed artifact, not by the suite; fixed at the definition above and regression-locked | Definition corrected, published |
| Aug 2026 |
The rails figure re-measured for the first time since commit
7a62038 added nine detectors to the input rails. That commit
re-measured tool inspection only, so the rails base — 1.4 — carried
forward untouched; the 1.19× multilingual ratio in the row above was then
applied to it, and 1.4 × 1.19 was published as 1.7. Neither step was wrong
on its own. Applying a correctly measured ratio to a stale base is the
defect, and closing it is why the harness behind the table above
exists.
|
Rails 2.85 ms p95 against 1.7 published — roughly two thirds higher, and the ratio is written that way because a decimal typed here would go stale at the next regeneration; tool inspection 14.36 ms against 13. Every latency figure on this site is now substituted at build time from the committed artifact, so a hand-typed one cannot exist. Separately, in the same pass, four hand-rolled copies of the percentile arithmetic were reduced to one shared definition: its isolated effect on the rails p95 was −0.009%, and downward, so it accounts for none of the gap in either direction. That is a different expression in a different file from the defect two rows above, and it erred the other way: the benchmark copies read one rank high wherever the rank fell exactly on a whole number, so the old figures were marginally pessimistic rather than flattering. The gate margin moves with them: the superseded ≥ 3 named no measurement a reader could check, and it is now 50 divided by the worst-case p95 in the table above, which is itself measured on a tool name the least-privilege stage actually matches — the previous worst case named one no constraint covered, so it returned before the constraint pass and measured less work than the mixed workload above it. | Rails 1.7 → 2.9 ms p95, tool guard 13 → 14 ms p95, gate margin ≥ 3 → 2.5× |
The build now refuses to ship a page carrying a retired figure: the site's audit step scans the built HTML for every number this table has ever replaced and fails on a match. A stale benchmark is treated as a build error, not a copywriting oversight. The stronger half is upstream of that guard — a live figure is no longer typed into a page at all, so the failure in the last row now needs a regenerated measurement to happen, not a missed find-and-replace.
Run the same probes against your own policy.
Design partners get the benchmark harness with the product — the same
make benchmark target that produced the table above, run against your own
policy on your own hardware. The operator console ships a red-team runner as well,
evaluating our adversarial corpus against your policy in-process, generating no
traffic.