Recorded before it is dispatched
Every call the hosted gateway admits is written to the audit ledger before it goes to your provider. If the ledger cannot take the write, the call does not go out.
How the ledger worksOPEN SOURCE INFRASTRUCTURE / AI AGENTS / BUILT EU-FIRST
The flight recorder
for AI agents.
Every call the hosted gateway admits, recorded in the ledger before it is dispatched. Every ledger record, verifiable.
No credit card · Apache-2.0 core
Self-host · your keys, 0% markup
Built EU-first · gateway and primary databases in Germany
SHA-256 Hash chainEd25519 Signed Merkle root191 ProvidersTwo Rust services — gateway and ingest — with ClickHouse and NATS.
205 providers, one canonical wire format
OpenAI
Anthropic
Google
Mistral
Llama
Bedrock
Groq
205 providers
What the flight recorder does
Verifiable without trusting us
SHA-256 hash-chained, Ed25519-signed and anchored to Sigstore's public log. Three open verifiers, in Rust, Python and TypeScript, check it offline.
Security postureEvery attempt, and the model that answered
Each retry and failover hop is on the trace with its status and timing, and every call records the model you asked for next to the one the provider says served it.
What a trace shows2 ms added at p50, on your own keys
Measured with LiteLLM's open AIGatewayBench, full table published. 205 providers through one endpoint, your keys, 0% markup.
The benchmark, method and caveatsTHE REQUEST PATH
From admitted call
to verifiable record.
Hover or tap a step to see it recorded.
THE AUDIT LEDGERHash-chained
Model callsModel · tokens · cost
Tool callsRecorded metadata
Signed Merkle rootEd25519
A record you can inspect. A chain you can verify offline.
The problem
Agents fail without crashing.
State drift compounds
A 200 doesn't mean it worked
Every tool call is code running on your behalf
How it works
Record. Replay. Prove.
Two Rust services. Open core, Apache 2.0.
Record
Metadata on every call by default; full prompt and response capture once it is enabled for your workspace — always on your own provider keys.
Replay
Inspect the recorded calls to find what broke. Replay reads stored spans; it does not re-execute models or tools.
Prove
Hash-chained, signed, verifiable offline.
Built on the record
Failures, loops and cost spikes, read from the record.
Each one reads what was recorded. The analysis runs when you read, not on the request path of a model call.
failed run → test case
From a failed trace to a regression fixture.
The incident panel on each trace shows the failing signal — an error status, a bad annotation, or an outcome you recorded as a failure — and what changed since the last comparable good run of the same agent within the past week: requested and served model, tool definitions, environment and release. Every explanation is marked supported or uncertain, with its evidence.
Export the captured input as a dataset file or a Promptfoo config. Record a trace's outcome yourself, in the panel or with POST /v1/outcomes — an outcome is never inferred from a 200.
Limits: expected output is left empty and you choose the assertion. The fixture is the recorded call; no tool is executed and there is no live re-run. Export needs prompt capture enabled for your workspace, a plan that includes datasets, and a role that can manage datasets. Finding a good run to compare needs an agent name or identity on the trace.
loops and rescues
Repeated tool calls, and the requests failover or retry saved.
A loop is the same tool called with the same arguments 3 or more times within 5 minutes, inside one session or trace. It is found when you read, from stored spans, and it works with prompt capture off: arguments are compared by a fingerprint keyed to your workspace, not by stored text. Each loop lists its repeated spans and what the repeats cost.
The dashboard also counts requests that hit a failed provider attempt and were then answered, split into failover and retry, with the median time spent in the failed attempts.
Limits: detection only — it never touches a request. Calls are grouped by session (x-conversation-id) or trace, so calls sent with neither cannot be compared across requests. Repeat spend counts only calls with a recorded cost. Only tool calls recorded after loop detection shipped carry the fingerprint; a self-hosted gateway fingerprints only when TRACELANE_APIKEY_PEPPER is set. Rescue counts start on 2026-09-19.
spend spikes
When spend jumps, see who carried it.
Spend per workspace by hour or day, with a spike flagged when a bucket costs at least 3× the median of the previous week of hours, or 2.5× the median of the previous two weeks of days, and meets a minimum dollar amount. Select a spike to see its largest contributors by model, key, provider, environment, service, user or tag, each with its cost and its share of the bucket.
Limits: these are the largest contributors, not a proven cause. Calls with no recorded cost are counted separately, never treated as $0. Excess over a contributor's own baseline is shown for model, key, environment and service only, contributor rows do not link to traces yet, and spikes are flagged on the chart — no alerts are sent. Until there is enough history it says so and flags nothing. Environment, service and tag need those labels sent.
Measured, not claimed
2 ms added at p50, 4 ms at p95, 5 ms at p99.
Gateway overhead on top of the model call, with span capture on — measured with LiteLLM's open AIGatewayBench harness at a pinned commit, plus three published patches of ours (an OpenAI-route scenario and two mock-upstream timing fixes). Measured again with the production topology — a Postgres control plane, API-key authentication and the acknowledged audit-ledger write on every request — and within the same table.
| Gateway | Version | p50 | p95 | p99 | Peak RSS |
|---|---|---|---|---|---|
| Tracelane | 1f3af9ad | 2 ms | 4 ms | 5 ms | 14 MB |
| Bifrost | v2.0.0 | 2 ms | 2 ms | 3 ms | 101 MB |
| Portkey (OSS) | 1.15.2 | 3 ms (run 1: 4 ms) | 5 ms (run 1: 7 ms) | 8 ms (run 1: 9 ms) | 177 MB (run 1; run 2 not sampled) |
| LiteLLM Python | 1.100.0 | 14 ms (run 1: 15 ms) | 34 ms (run 1: 36 ms) | 47 ms (run 1: 52 ms) | 323 MB |
| LiteLLM Rust | not measured | not measured | not measured | not measured | not measured |
Peak memory (RSS), gateway process only
Tracelane 14 MBBifrost 101 MBPortkey (OSS) 177 MB (run 1; run 2 not sampled)LiteLLM Python 323 MB
Method and caveats
Direct-to-mock baseline (already subtracted from every figure above): 1 / 2 / 3 ms p50/p95/p99.
Host: one Hetzner CCX23 (4 vCPU, 16 GB). 16 concurrent clients, 30 seconds, two runs.
† Bifrost via npx -y @maximhq/bifrost, both runs — its version string is only
emitted on first install, so run 2's CSV records it as "unknown"; v2.0.0 is from run 1.
‡ LiteLLM Rust: as of 2026-09-07 no public build exists — the harness's own notes build it
from a private branch on LiteLLM's machine. We'll run it when one is published.
bash bench/gateway/aigatewaybench/run.sh Full method, caveats and the raw CSVs How we compare
Thirteen claims, sourced.
Against each vendor's own docs and pricing pages, fetched 2026-09-07. A dash means we didn't find it documented, not that the vendor lacks it.
| Capability | Tracelane | LiteLLM | Portkey | Langfuse | LangSmith |
|---|---|---|---|---|---|
| Measured gateway overhead (p50) | ✓ 2 ms † | ◐ Python proxy 14–15 ms, same bench; Rust build not public | ◐ 3–4 ms, same bench run | — no gateway | ◐ gateway not benchmarkable; no self-hostable runtime documented |
| Rust hot path | ✓ | — Python proxy measured; Rust gateway announced, no public build | — TypeScript/Node.js | — no gateway | — no gateway |
| BYOK, 0% token markup | ✓ | ✓ | ✓ | — | ◐ lists an LLM Gateway; BYOK not documented |
| OTel-native ingest | ✓ | ◐ partial | ◐ partial | ✓ | ◐ partial |
| Multi-agent swimlanes | ✓ | — | — | ✓ | ✓ |
| Sessions | ✓ per-user identity (x-tracelane-user-id) | — | — | ✓ | ✓ |
| Eval loop — datasets, experiments, online evals, annotation queues, CI gate | ✓ | — | — | ✓ | ✓ |
| Prompt management + playground | ✓ | — | ✓ | ✓ | ✓ |
| MCP server for traces | ✓ on npm, read-only, tenant-scoped | — | — | — | — |
| Claude Code flight recorder | ✓ | — | — | — | — |
| Tamper-evident ledger, offline-verifiable | ✓ | — | — | — | — |
| Self-host, single Compose file | ✓ | ✓ | ✓ | ✓ + Kubernetes | ◐ Enterprise BYOC |
| License | ✓ Apache 2.0 | ◐ MIT (core) | ✓ MIT | ✓ MIT | — none for OSS; Enterprise self-hosted SKU is proprietary |
shipped and documented · supported with a real limit · not documented on the vendor's own page as of 2026-09-07
† Benchmark: 2 ms p50 / 4 ms p95 / 5 ms p99 added latency on LiteLLM's own AIGatewayBench harness (4 vCPU, two runs, capture on) — full method and raw CSVs.
Not yet: SOC 2 report, configurable data residency, HIPAA/BAA, documented SAML SSO, contractual SLA, documented Slack/PagerDuty alert integrations.
The audit ledger
Every run. On the record. Provably.
SHA-256 hash-chained, Ed25519-signed — verify your ledger offline with the open verifier.
The ledger covers gateway-admitted calls and their recorded guardrail verdicts; SDK/OTLP-only spans are captured, not chained; the trace ledger badge shows whether that trace has a ledger record.
# verify any trace offline — no trust in us required
$ tlane verify ./audit.ndjson --tenant-pubkey <b64>
✓ tlane verify: PASS
✓ hash_chain_valid: true · signatures_valid: true
✓ anchors_included: 12 (Sigstore Rekor v2) EU AI Act, India DPDP and HIPAA — what the ledger does and does not cover
eu ai act
Article 12: the tamper-evident ledger, signing and anchoring are on every tier today; the Article 12 export ships on Enterprise and does not yet carry per-record inclusion proofs or a signed completeness attestation. Annex III enforcement provisionally deferred to Dec 2, 2027 (Digital Omnibus political agreement, May 7, 2026; pending formal adoption — details).
india dpdp
Tracelane enforces no per-tenant storage region today; traces and the ledger are stored in the EU (Germany), with the cold tier in EU-jurisdiction object storage. The audit ledger has no expiry clock — it is kept for the life of the ledger.
hipaa
No signed BAA today, so do not route PHI through the hosted gateway. Prompt and response text is not written to your traces unless we enable capture for your workspace; the response cache holds cacheable responses in memory for up to 7 days, and durably when your workspace has an embedding key. Inline redaction (Team and above) masks structured identifiers only — card numbers, SSNs, emails, E.164 phone numbers and secret-shaped strings — never names, addresses or free-text clinical detail. Provider keys are AES-256-GCM encrypted at rest, bound to your workspace and provider, under a rotatable master key.
Migration
Moving off a tool that changed hands or changed its terms?
One command rewrites your config. Nothing leaves your infrastructure.
# rewrites HELICONE_* env vars, base URLs and custom
# headers across your source tree — printed as a diff,
# dry-run by default
$ tlane migrate helicone# reads litellm_config.yaml and prints your providers,
# routing and callbacks translated into Tracelane terms for review
$ tlane import-litellmPricing
Pricing.
Six meters, priced the same way on every paid tier. Self-host free forever.
OSS self-host
$0
- Full Rust gateway · 199 catalogued OpenAI-compatible providers, plus native adapters
- ClickHouse + NATS JetStream storage on your own node
- Hash-chained audit log with Ed25519 batch signatures — Rekor anchoring and the durable, restart-safe ledger need the Postgres control plane
- Community support on GitHub Discussions
Free
$0
- 1 GB ingest included / mo
- 60 RPM
- No credit card · never cut off by billing state
Builder
$29/mo
- 20 GB ingest included / mo
- 600 RPM
recommended
Team
$229/mo
- 100 GB ingest included / mo
- 6,000 RPM
- WorkOS SSO
Business
$799/mo
- 200 GB ingest included / mo
- 60,000 RPM
- WorkOS SSO
Enterprise
from $2,499/mo
- custom ingest included / mo
- Uncapped RPM
- WorkOS SSO
- 7-year ledger retention (included)
Every workspace starts on Free — no card. Upgrade to a paid plan inside the app when you need it.
Security posture
Built for procurement reviews.
release integrity
Sigstore Cosign keyless signing with attested build provenance. Trusted Publishing OIDC — no long-lived registry tokens.
sbom
CycloneDX SBOM published with every release, verifiable from GitHub release artifacts.
vuln scanning
gitleaks and a high-severity dependency audit on every push; cargo-deny in the pre-push gate; OSV-Scanner, Grype and cargo-deny nightly and a Syft SBOM weekly on the public repository.
pre-flight policy
Policy checks run inline at the gateway before the call leaves. Today: schema-violation + tool-definition-drift on the Anthropic Messages route, fail-open. More on a published roadmap.
transport
TLS 1.3 from the edge to the origin and on the internal ingest leg, which is mTLS via SPIFFE/SPIRE; the public edge accepts TLS 1.2. SSRF defense — RFC1918, link-local and CGNAT denied.
standards
OTel GenAI semconv — tracking the pre-execution judgment / negative-proof proposal (#3244).
FAQ
Frequently asked.
What is Tracelane?
An open-source (Apache 2.0) flight recorder for AI agents: a universal Rust LLM gateway, full-fidelity OpenTelemetry observability, and a tamper-evident audit ledger. Each ledger event is SHA-256 hash-chained and, every 100 events per tenant, the Merkle root is Ed25519-signed, so any gateway call's record can be verified offline. OTLP/SDK traces are not yet in the chain.
How is it different from Helicone, LangSmith or Langfuse?
Tracelane unifies the gateway, observability, pre-flight policy and a tamper-evident audit ledger in one stack: a Rust gateway and a Rust ingest service. Licensing: Tracelane is Apache 2.0 core with a published self-host path; Helicone's AI gateway is GPL-3.0, Arize Phoenix is ELv2 (no SaaS resale), LangSmith is closed-source. Langfuse has no gateway; Helicone and LangSmith ship theirs separately from their observability products. The audit ledger is hash-chained and Ed25519-signed.
What about LangSmith Engine?
LangSmith Engine (announced May 2026) clusters production failures after they happen and drafts pull requests to fix them, inside the LangSmith trace surface; Tracelane's pre-flight policy checks flag failure modes at the gateway and its ingest is neutral OTel with 0% BYOK markup across 205 providers. Different layer of the agent reliability stack.
Is Tracelane really open source?
Yes. The core (Rust gateway, ingest workers, observability backend, pre-flight layer, CLI, SDKs, audit verifiers) is Apache 2.0. The AFT-1 spec under /spec/aft-1 is CC0. Paid features are gated by the workspace_entitlements table at the application layer, not by a license-string compare.
Which providers are supported?
205 providers today — 6 native adapters (Anthropic, Google Gemini, Google Vertex, AWS Bedrock, Azure OpenAI, Cohere) plus 199 catalogued OpenAI-compatible providers including OpenAI, Together, Fireworks, Groq, OpenRouter, DeepSeek, xAI and Mistral; self-hosters can point any catalog entry at their own base URL. One canonical wire format across all of them.
Does Tracelane support EU AI Act Article 12?
The Tamper-Evident Agent Ledger (SHA-256 hash chain, Ed25519-signed Merkle root, Rekor anchoring, self-verification) is included on every tier, free and paid — no add-on to buy. Enterprise includes a seven-year ledger window and the ledger export (/v1/audit/export); the Article 12 evidence pack — per-record inclusion proofs and a signed completeness attestation — is on the roadmap. Tracelane enforces no per-tenant storage region, signs no BAA, and makes no regulatory-conformance claim. EU AI Act Annex III obligations were originally scheduled for Aug 2, 2026; the Digital Omnibus political agreement (May 7, 2026) provisionally deferred enforcement to Dec 2, 2027, pending formal adoption (details). The ledger ships now so regulated buyers can build audit history before binding enforcement.
What's the latency overhead?
Measured, not targeted: 2ms p50, 4ms p95, 5ms p99 of added latency (excluding provider time), taken with LiteLLM's open AIGatewayBench harness on a 4 vCPU host with capture on, two runs, alongside Bifrost, Portkey and LiteLLM — see the full breakdown above or the benchmark page for method, caveats and the raw CSVs. The pre-flight policy layer is still an engineering target (p99 <50ms on commodity CPU).
Can I move over from Helicone or LiteLLM?
Yes: tlane migrate helicone rewrites your Helicone config as a dry-run diff, and tlane import-litellm translates a litellm_config.yaml for review.
Can I self-host?
Yes. One Docker Compose file (infra/self-host/docker-compose.yml) runs the gateway, ingest, NATS and ClickHouse; a Helm chart is on the roadmap. Tracelane's own production — ClickHouse with an R2 cold tier — runs on one Hetzner node. The self-host Compose runs ClickHouse on local disk; an S3/R2 cold tier is a by-hand step. See the self-hosting guide.
Is Tracelane available now?
Yes — V1 is live. Start free at app.tracelane.dev, or self-host the Apache 2.0 core.
Stay close
Get product updates.
One email when something ships — no drip, no list sharing. Reply to any message and you are removed.