OpenAI compatible API. Attested gateway. Public status.

TrustedRouter Evals Guide

Run model, provider, privacy, latency, and cost evals through one OpenAI compatible API.

Verify gateway

1 URLbase_url migration

100smodels and routes

0prompt logs by default

Run evals through TrustedRouter

model switching

Use TrustedRouter when an agent or eval runner needs to compare models, providers, privacy tiers, and cost without rewriting client code.

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["TRUSTEDROUTER_API_KEY"],
    base_url="https://api.trustedrouter.com/v1",
)

def ask(model: str, prompt: str) -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=300,
    )
    return response.choices[0].message.content or ""

for model in ["trustedrouter/zdr", "trustedrouter/auto", "trustedrouter/cheap"]:
    print(model, ask(model, "Recommend 5 real albums for fans of Talk Talk."))

What to record

metadata only

Model and provider route selected.
Latency, time to first token, and token counts.
Cost in integer microdollars, not floats.
Evaluator score and blind judge notes.
Request ID for debugging. Do not store prompt/output unless the eval owner explicitly opts in.

Default test pools

safe starts

trustedrouter/zdr for legal, medical, financial, and private customer data.
trustedrouter/e2e when the eval needs an end-to-end encrypted provider route.
trustedrouter/auto for availability and broad fallback testing.
trustedrouter/cheap for wide sweeps where cost matters more than top quality.

Cost discipline

before large runs

Start each eval with a small model set and a hard token cap. Check the public leaderboard and model pages before widening the run.

Leaderboard Models Providers Agent setup