Platform
Monitors
A judge model reviewing captured traffic — async, on your own keys.
A monitor points a judge model at a sample of your captured traffic and asks one question per exchange: does this response violate your instructions? Hallucinated citations, policy violations, prompt-injection compliance, leaked data — you write the rubric, the judge flags what a human should review.
How it runs
- Async, never inline. Judging happens seconds after the response, off the request path. A monitor can never add latency or block traffic.
- On your own keys, at zero markup. Judge calls route through your gateway like any traffic — budgeted, rate-limited, logged with
source=monitorso the spend is visible in analytics. - Deterministically sampled. You pick the sample rate per monitor; the same request always makes the same sampling decision.
What it costs
The levers: sampling rate, a cheap judge model, input truncation (~4k tokens per judgment), and a daily budget that auto-pauses the monitor until midnight UTC. At 100,000 requests a month with 10% sampling, a mini-class judge model costs roughly $2–10 per month of your own provider spend.
Actions
| Action | Default | What happens |
|---|---|---|
| Flag | Always on | The finding lands in the review queue with severity, categories, and the judge's reasoning. |
| Notify | Opt-in | Org admins get an email — at most one per cooldown window; repeat findings batch into it. |
| Disable key | Opt-in + severity gate | The offending API key is disabled, admins are emailed, the action is audit-logged, and re-enabling is one click on the API Keys page. |
The judge is hardened against prompt injection — captured content is presented as data, and verdicts arrive through a forced tool call — but no judge is perfect. Treat auto-disable as a circuit breaker for severe cases, not a content filter.