How we test a model before release
Every model we build passes the same release gates before it reaches users: task quality, grounding, safety, privacy, latency, cost and drift.
- Publication

On this page
Every model we build passes the same gates before it reaches your users. This note sets out what each gate measures, how it is measured and when it runs. The gates are drawn on our AI page, under Evaluation.
1. The gates
- Task quality: answers correct on your test set, scored against a labelled set you approve.
- Grounding: every answer cites your sources, each sentence traced to a retrieved passage.
- Safety: harmful and off-policy requests refused, tested with red-team prompts and jailbreak suites.
- Privacy: no personal data leaves its boundary, with redaction tested on seeded records.
- Latency: fast enough for the task, with response time measured under load.
- Cost: affordable at your volume, modelled at the volume you expect.
- Drift: still right next month, with live inputs compared with the test set.
2. When they run
- Build the test set: labelled examples of your task, approved by you.
- Set the thresholds: one for each gate, agreed with you before work starts.
- Run on every build: task quality, grounding and privacy, as the code changes.
- Run before release: safety, latency and cost, on the release candidate.
- Watch in production: live inputs compared with the test set, so drift is caught.
3. What a release candidate needs
A release candidate that misses a threshold waits until it passes. Each gate's method is written down and runs the same way every time. For safety, a fixed suite of adversarial prompts, plus new ones written for your domain, runs against the candidate. Any unsafe answer blocks the release and is listed in the report you receive, which gives each gate's result for every release candidate.
Related research. More from our Lab on AI.
How we build an AI agent, in five steps
An agent is software that takes actions in your systems. We build it in five steps, inside limits held outside the model, with a person where it matters.
Which AI model should we use?
The one that does your task best at a cost you can carry. We score candidate models on your evaluation set and choose by quality, cost and response time.
What the Etheon Lab works on
The Etheon Lab studies what makes AI software safe to ship: evaluation, guardrails, observability, safe releases and governance.