Self-hosted LLMOps stack · Verified August 16, 2026

TrustedRouter vs TensorZero

TensorZero was an Apache-2.0 LLMOps stack — Rust gateway, observability, evaluations, and a real optimization loop — until the project was archived on June 12, 2026 and left unmaintained. TrustedRouter is a maintained, source-available managed gateway with attested execution and no durable prompt logs on realtime inference.

Compare details

8decision dimensions

7official sources

1base URL to try TR

Choose TensorZero when

Its operating model is the feature.

Choose TensorZero only if you are prepared to own a fork: the Apache-2.0 stack still runs, all data stays in your own database, and its optimization loop (SFT, DPO, GEPA, gateway-run A/B tests) has no drop-in replacement in any plain gateway, TrustedRouter included. The repository has been archived and read-only since June 12, 2026, so security patches and new provider support are on you.

Choose TrustedRouter when

Privacy needs evidence.

Choose TrustedRouter for a maintained gateway: one API over 550+ model routes across ~49 providers (as of 2026-08-16), realtime inference with no durable prompt or output logs, privacy-tier routing (zdr, e2e, eu), and a gateway build you can verify against live attestation on three clouds.

DimensionTensorZeroTrustedRouter

DeploymentSelf-hosted Docker/Helm stack; repo archived and read-only since June 12, 2026Hosted control plane, source-available (BUSL-1.1), with an attested API path

API surfaceNative /inference (functions, variants, episodes) plus an OpenAI-compatible endpointOpenAI Chat Completions and Responses plus Anthropic Messages

Model access~19 provider integrations on your keys; frozen June 2026 bar the generic OpenAI path500+ models across ~50 providers: prepaid, BYOK, and direct

RoutingRetries, fallbacks, caching, rate limits, adaptive A/B tests, best-of-N/mixture-of-NProvider fallback plus auto, cheap, fast, free, ZDR, E2E, and EU routes with composable privacy and jurisdiction preferences

ObservabilityInferences and feedback in your ClickHouse or Postgres; optional, can be disabledMetadata analytics and opt-in external broadcast

Prompt contentStays in your DB; pseudonymous usage analytics on by default until disabledNo durable prompt or output logs on realtime inference; batch is opt-in encrypted retention

VerificationApache-2.0 source you audit, build, and run; no attestation layerLive gateway attestation on three clouds, bound to published source and release evidence

BillingFree (Apache-2.0); your infrastructure plus direct provider billing, no markupPrepaid at provider price + 5.5% ($0.01/M floor), or BYOK

Start with the wind-down

TensorZero's GitHub repository was archived by its owner on June 12, 2026 and is read-only. The homepage now carries a single notice: still available on GitHub, no longer maintained. Co-founder and CEO Gabriel Bianconi wrote on Hacker News that the team wound the project down and is returning remaining capital to investors, after raising a $7.3M seed in 2024 and spending less than half of it. The final release, 2026.6.0, shipped on June 4, 2026 — eight days before the archive. The docs site is gone; its URLs now redirect to raw files inside the archived repository.

That changes what this page compares: a maintained managed gateway on one side, and an Apache-2.0 codebase that still runs but that nobody maintains on the other.

What TensorZero got right

TensorZero shipped a full self-hosted LLMOps stack: a Rust gateway with a unified API across roughly 19 documented provider integrations, observability that stored inferences and feedback in your own ClickHouse or Postgres, built-in evaluations, optimization recipes — supervised fine-tuning, DPO, GEPA prompt optimization, dynamic in-context learning, best-of-N and mixture-of-N — and gateway-run experimentation with adaptive A/B tests that hold variant assignment consistent within a multi-step episode.

The engineering was serious. The team published a benchmark showing 0.94 ms p99 gateway overhead at 10,000 QPS on a single c7i.xlarge; the caveats (self-run, mock provider, observability disabled) are in their own notes. Adoption was real: 11,723 GitHub stars (checked August 2026), and a self-reported README claim of fueling about 1% of global LLM API spend. The structured data model was the genuine differentiator among open-source gateways: inferences plus downstream feedback in your database made dataset curation, fine-tuning, prompt optimization, and gateway-run A/B testing one continuous loop. No plain proxy — TrustedRouter included — replaces that loop.

The cost of running archived software

Apache-2.0 survives the archive: forking and self-hosting remain fully permitted. What you take on is everything upstream used to do — security patches, dependency CVEs, and provider-API drift, with the provider catalog frozen at June 2026. New providers and model APIs go through the generic OpenAI-compatible integration or your own fork. Two specifics deserve attention. Pseudonymous usage analytics is on by default (disable_pseudonymous_usage_analytics defaults to false) and still tries to send usage data to the wound-down company's endpoint, so set it to true. And Autopilot, the planned managed layer, never left private beta.

Where TrustedRouter differs

Someone operates it. TrustedRouter is a hosted gateway: one base URL, OpenAI Chat Completions and Responses plus Anthropic-style Messages, and a maintained catalog of 550+ model routes across roughly 49 providers as of August 16, 2026 (live list). Provider onboarding, failover, and API drift are our problem, not a fork you carry.

The trust model is proof, not possession. TensorZero's privacy answer is possession: prompts stay on your machines, in your database, and observability can be switched off entirely. That is a strong answer, and self-hosting is the right call when policy requires it. TrustedRouter is hosted, so we owe evidence instead: the gateway runs inside TEEs on GCP Confidential Space, AWS Nitro Enclaves, and Azure Confidential Containers, with live attestation endpoints and a verifier script published at trust.trustedrouter.com that check the running build against published source digests. Realtime inference keeps no durable prompt or output logs; we retain operational metadata (request ids, model, token counts, latency, cost, region, API-key hash). The gateway and control plane are source-available under BUSL-1.1, converting to Apache-2.0 four years after each release. That is not OSI open source, and we do not call it that; it does mean every line that touches your prompt is public and inspectable.

Privacy is routable. TensorZero routed for reliability and experiments. TrustedRouter also routes on privacy posture: trustedrouter/zdr restricts to providers with contractual or policy zero-data-retention commitments, trustedrouter/e2e to confidential-compute providers with provider-side end-to-end encryption (currently Tinfoil and Phala), trustedrouter/eu to an EU-focused provider order, and provider.min_privacy composes with any model id.

Pricing, with bases stated

The bases differ, so no single number settles it. TensorZero has no fees at all: Apache-2.0 software, your infrastructure (gateway, ClickHouse or Postgres, UI, optionally Valkey), and direct provider billing on your own keys. TrustedRouter prepaid inference bills the provider's token price plus 5.5%, with a $0.01 per million token floor; video is the provider quote plus 20%; there are no seat or subscription fees (pricing). BYOK is supported, and the pricing page does not publish a separate BYOK fee. If your ops time is free and your team is willing to maintain a fork, TensorZero is cheaper. If it is not, the 5.5% buys the operations, the catalog, and the attestation above.

What we do not claim

Our attestation covers the gateway, not the model providers behind it. Downstream handling is policy and contract, tracked per provider in our catalog tiers — except e2e routes, where the provider's own confidential-compute and encryption mechanisms apply, and those are the provider's claims, not our attestation. Our repositories are young: public since late April 2026, with benchmark history from June 2026. We publish a DPA, BAA, and subprocessor list, but no SOC 2 or HIPAA certification today (security). And nothing here replaces TensorZero's optimization loop: we have no fine-tuning recipes, no feedback API, no gateway-run A/B tests. We support small evals; that is a fraction of the flywheel.

Migration in practice

If your application used TensorZero's OpenAI-compatible endpoint, migration is a base-URL and key swap, plus deleting tensorzero:: extra-body parameters and mapping function names to concrete model ids. If it used the native /inference API, prompt templates and schemas move out of gateway TOML into your application code, and calls are rewritten as chat completions. Historical data is the easy part — it already sits in your ClickHouse or Postgres in documented schemas. The hard part is honest: episodes, feedback, and the optimization recipes have no TrustedRouter equivalent, so teams that depend on them either keep the archived stack running for that workflow or rebuild it on other tooling.

Migration shape

Start with one real request.

Apps on TensorZero's OpenAI-compatible endpoint swap the base URL and key, drop tensorzero:: extra-body parameters, and map function names to concrete model ids. Native /inference callers move prompt templates out of gateway TOML into application code. Episodes, feedback, and the SFT/DPO/GEPA optimization recipes have no TrustedRouter equivalent; historical data exports from your own ClickHouse or Postgres with standard SQL.

Keep the first test small, stream the response, and compare output, latency, provider selection, and billed usage before moving production traffic.

Agent setup Run a small eval

TrustedRouter sideOpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="https://api.trustedrouter.com/v1",
    api_key="sk-tr-v1-...",
)

with client.chat.completions.create(
    model="trustedrouter/zdr",
    messages=[{"role": "user", "content": "Reply PONG"}],
    stream=True,
) as response:
    for chunk in response:
        print(chunk.choices[0].delta.content or "", end="")

Official evidence

Sources checked August 16, 2026

Report a change

TensorZero homepage wind-down noticehttps://www.tensorzero.com/
TensorZero GitHub repository (archived June 12, 2026)https://github.com/tensorzero/tensorzero
TensorZero CEO wind-down statement on Hacker Newshttps://news.ycombinator.com/item?id=48518120
TensorZero gateway benchmarks (repo docs)https://raw.githubusercontent.com/tensorzero/tensorzero/main/docs/gateway/benchmarks.mdx
TensorZero configuration reference: usage-analytics defaulthttps://raw.githubusercontent.com/tensorzero/tensorzero/main/docs/gateway/configuration-reference.mdx
TrustedRouter pricing: 5.5% fee and $0.01/M floorhttps://trustedrouter.com/pricing
TrustedRouter trust page: attestation endpoints and verifierhttps://trust.trustedrouter.com
TrustedRouter monthly benchmark reportsStable first-party route evidence
TrustedRouter live trust recordRelease and attestation evidence

All gateways

TrustedRouter vs OpenRouterHosted model marketplace
TrustedRouter vs Vercel AI GatewayHosted AI gateway
TrustedRouter vs LiteLLMSelf-hosted AI gateway
TrustedRouter vs Cloudflare AI GatewayHosted edge AI gateway

Questions

Can we keep running TensorZero in production?

Mechanically, yes. Apache-2.0 permits forking and self-hosting indefinitely, and the final 2026.6.0 release (June 4, 2026) still works. But the repository has been archived and read-only since June 12, 2026: no security patches, no new provider integrations, no support. Budget for maintaining a fork, and set disable_pseudonymous_usage_analytics to true — the default-on analytics still tries to send usage data to the wound-down company's endpoint.

Does TrustedRouter replace TensorZero's optimization and experimentation loop?

No. TensorZero's structured data model, feedback API, fine-tuning and prompt-optimization recipes (SFT, DPO, GEPA, DICL), and gateway-run adaptive A/B tests have no TrustedRouter equivalent. We offer routing, failover, and small-scale evals (/docs/evals). Teams that depend on that loop should plan to keep the archived stack for it or rebuild the workflow on other tooling.

What does each one cost?

TensorZero is free software: you pay for your own infrastructure (gateway, ClickHouse or Postgres, UI, optional Valkey) and bill providers directly on your keys, with no markup. TrustedRouter prepaid inference is the provider's token price plus 5.5% with a $0.01 per million token floor; BYOK is supported but no separate BYOK fee is published. The bases differ — self-run infrastructure and ops time versus a managed fee on provider cost — so compare against your own operations budget.