How Tracelane compares

Four tables — gateway, observability, evals, trust — built the same way as our gateway-overhead benchmark: every number and every claim is sourced, and a — is not a loss, it’s a “we didn’t find it,” which is not the same thing.

Method

Every competitor cell that asserts a capability links to a numbered source, fetched from the vendor’s own page — docs site, pricing page, or public GitHub repository — on 2026-09-07. No third-party blogs, no re-published “comparison” pages from a fourth party. A bare “Not documented” or ”—” is a cell where we found nothing to cite.
Every Tracelane cell cites a code path, a public doc (/pricing, /benchmarks) or a dated production proof — never a spec or a marketing draft.
A cell marked “Not documented” means we did not find that capability documented on the vendor’s own site as of the fetch date — it does not mean the vendor lacks it. Vendors change their products continuously; if something below is stale, open an issue and we’ll fix it.
“Yes” for Tracelane always means shipped and either deployed to production or reproducible from the public repo — not planned, not “in the design doc.” Where a capability is partial or gated by plan, the cell says so and the FAQ or a footnote explains the boundary.

Gateway

Capability	Tracelane	LiteLLM	Portkey	Bifrost	Helicone	Langfuse	LangSmith	Braintrust	Arize Phoenix
Hot-path runtime	Rust ^[T1]	Python proxy (the build measured in /benchmarks); a “Rust core” is announced on GitHub with no public build as of 2026-09-07 ^[1] ^[6]	TypeScript/Node.js ^[2]	Go ^[3]	Rust (`Helicone/ai-gateway` repository) ^[22]	N/A — no gateway	N/A — no self-hostable gateway runtime documented ^[5]	N/A — no gateway	N/A — no gateway
Measured gateway overhead (p50)	2 ms — AIGatewayBench, CCX23, 2026-09-07 ^[T2]	Python proxy: 14–15 ms p50, same bench run ^[T2]; the announced Rust gateway has no public build, and its self-reported “8 ms P95 at 1k RPS” is not independently measured ^[6]	3–4 ms p50, same bench run ^[T2]	2 ms p50, same bench run ^[T2]	Not published ^[4]	—	—	—	—
Providers supported	191 ^[T3]	140+ upstream vendors / ~1,892 models (its own count) ^[7]	Claims 1,600+ LLMs ^[8]; source code lists ~75 provider modules ^[9] — see “Portkey’s provider count” below	23+ upstream vendors (its own count) ^[3]	Multiple — count not documented ^[4]	—	—	—	—
BYOK, 0% token markup	Yes ^[T3]	Yes — per-key `api_key` in config ^[10]	Yes — virtual keys with your own provider keys ^[8]	Yes — virtual keys ^[3]	Yes — routes through your keys ^[4]	—	Partial — lists an “LLM Gateway” to “control agent model calls”; BYOK not documented on that page ^[5]	—	—
Response caching	Exact-match caching in production. Embedding-similarity caching is built but not enabled on the hosted product ^[T4]	Yes — exact + semantic, 8+ backends ^[11]	Yes — simple + semantic ^[12]	Yes — semantic caching ^[3]	Partial — caching listed as a feature; exact vs. semantic not documented ^[4]	Not documented	Not documented	Not documented	Not documented
Rate limits + budgets per key	Yes — per-key RPM and budget, 429 on overage ^[T3]	Yes — per key/team/user RPM + TPM ^[10]	Yes — hourly/daily/per-minute on requests or tokens ^[12]	Yes — budget management ^[3]	Yes — rate limiting feature ^[4]	Not documented	Not documented	Not documented	Not documented
Guardrails on the hot path	Yes — R1–R8, deterministic rule-based rails inline at the gateway (cost/budget, secrets+PII redaction, tool pinning, lethal-trifecta, format, system-prompt-leak, topic/competitor, prompt-injection); every verdict is written to the audit ledger. R1/R3/R4/R8 free on every deployment incl. OSS self-host; R2/R5/R6/R7 (data-governance + quality) on Team and above ^[T5]	Yes — PII masking, prompt-injection detection, secret detection ^[1]	Yes — “50+ AI Guardrails” ^[2]	Partial — enterprise only, custom plugin ^[3]	Not documented ^[4]	Not documented	Not documented	Not documented	Not documented
Failover / routing	Partial — model→provider map fails closed (400 `unroutable_model`, no silent default); multi-key routing built ^[T6]	Yes — fallback chains, context-window fallback, cooldown ^[13]	Yes — automatic fallbacks, circuit breaker ^[2]	Yes — automatic failover, load balancing ^[3]	Not documented ^[4]	Not documented	Not documented	Not documented	Not documented
OpenAI-compatible endpoint	Yes — `/v1/chat/completions` ^[T1]	Yes ^[1]	Yes ^[2]	Yes ^[3]	Yes ^[4]	—	Partial — gateway is managed ^[5]	—	—
Anthropic-native `/v1/messages`	Yes — proven on production with a real Claude Code session ^[T7]	Partial — routes Anthropic models, but via OpenAI-shaped format ^[10]	Not documented ^[2]	Not documented ^[3]	Yes — direct integration listed ^[4]	—	—	—	—

Observability

Capability	Tracelane	LiteLLM	Portkey	Bifrost	Helicone	Langfuse	LangSmith	Braintrust	Arize Phoenix
OTel-native ingest (OTLP)	Yes — the OTLP decode path preserves `parent_span_id` by design ^[T8]	Partial — emits OTel spans for guardrail violations ^[1]; native ingest not confirmed	Partial — OTel emission, not primary ingest ^[8]	Partial — Prometheus + distributed tracing ^[3]	Not documented ^[4]	Yes — OTel-native, primary wire format ^[14]	Partial — proprietary format primary ^[5]	Partial — proprietary format primary ^[15]	Yes — native OTel ^[16]
Full-fidelity capture (no default sampling)	Yes — no default sampling; capture on in the published benchmark ^[T2]	Not documented	Not documented	Not documented	Not documented	Yes — all LLM and non-LLM calls ^[14]	Yes — “full visibility” ^[5]	Not documented	Not documented
Trace tree + tool spans	Yes ^[T9]	Not documented	Not documented	Not documented	Not documented	Yes — traces include tool calls ^[14]	Yes — step-by-step agent view ^[5]	Yes — tracing with spans ^[15]	Yes — every step incl. tool calls ^[16]
Sessions / threads	Yes — `/sessions`, keyed on `gen_ai.conversation.id` ^[T10]	Not documented	Not documented	Not documented	Yes — sessions listed ^[4]	Yes — multi-turn sessions ^[14]	Yes ^[5]	Yes — “Loop thread” ^[15]	Not documented
Multi-agent swimlanes / agent graph	Yes ^[T9]	Not documented	Not documented	Not documented	Not documented	Yes — agent graphs ^[14]	Yes — agent step view ^[5]	Not documented	Not documented
Full-text trace search	Yes ^[T9]	Not documented	Not documented	Not documented	Partial — HQL query language ^[4]	Not documented ^[14]	Yes — production monitoring + search ^[5]	Not documented	Not documented
Cost per request / model	Yes ^[T11]	Yes — per key/team/user tracking ^[1]	Yes — usage analytics ^[2]	Yes — budget management ^[3]	Not documented	Yes ^[14]	Yes ^[5]	Yes ^[15]	Not documented
In-app AI assistant over your traces	Yes — “Ask Tara”, a natural-language query surface over your own traces, deployed to production ^[T12]	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented
MCP server for trace access	Yes — `@tracelanedev/mcp` (published on npm, `0.3.0`), read-only and tenant-scoped: it reads through the gateway’s own `/v1/*` routes with the caller’s own bearer token ^[T13]	Not documented	Not documented	Yes — MCP for external tool integration ^[3]	Not documented	Not documented	Not documented	Not documented	Not documented
Claude Code / coding-agent integration	Yes — two paths: an OTLP flight recorder that decodes Claude Code’s native span shape, and an Anthropic-native `/v1/messages` proxy, both proven on production with real Claude Code sessions ^[T14]	Not documented — routing Claude models is not a coding-agent integration	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented
Shareable trace links	Yes — a public `/s/<token>` page carrying the audit-ledger badge ^[T15]	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented

Evals

Capability	Tracelane	LiteLLM	Portkey	Bifrost	Helicone	Langfuse	LangSmith	Braintrust	Arize Phoenix
Datasets	Yes — proven on production ^[T16]	Not documented	Not documented	Not documented	Yes — listed ^[4]	Yes ^[14]	Yes ^[5]	Yes ^[15]	Yes ^[16]
Experiments	Yes — proven on production ^[T16]	Not documented	Not documented	Not documented	Not documented	Yes ^[14]	Yes ^[5]	Yes ^[15]	Yes ^[16]
LLM-as-judge online evals with sampling	Yes — 11.11% sampling against a 10% configured rate, proven on production ^[T17]	Not documented	Not documented	Not documented	Not documented	Yes — LLM-as-judge evaluators ^[14]	Yes — online evaluations ^[5]	Yes ^[15]	Yes — LLM-as-judge ^[16]
Annotation / review queues	Yes — proven on production ^[T18]	Not documented	Not documented	Not documented	Not documented	Yes — annotation queues ^[14]	Yes — annotation queues ^[5]	Not documented	Yes — human review ^[16]
CI eval gate (GitHub Action / CLI)	Yes — `@tracelanedev/cli` (published on npm, `0.3.0`), proven on production ^[T19]	Not documented	Not documented	Not documented	Not documented	Partial — webhooks + GitHub Actions for alerts ^[14]	Not documented	Not documented	Not documented
Prompt management + playground	Yes ^[T20]	Not documented	Yes — prompt templates across tiers ^[12]	Not documented	Yes — prompts listed as a feature ^[4]	Yes — versioning + deployment without code changes ^[14]	Yes ^[5]	Yes — playground ^[15]	Yes — Prompt IDE ^[16]

Trust

Capability	Tracelane	LiteLLM	Portkey	Bifrost	Helicone	Langfuse	LangSmith	Braintrust	Arize Phoenix
Tamper-evident / hash-chained audit ledger	Yes — Merkle-batched hash chain + per-tenant Ed25519 signing (keys held by Tracelane; the Rekor anchor is the part we cannot forge), included on every tier ^[T21]	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented
Third-party verifiable offline	Yes — Sigstore Rekor v2 inclusion proof on anchored batches, best-effort; `tlane verify --tenant-pubkey` needs no call back to us ^[T21]	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented	Not documented
Self-host in one binary / Compose	Yes — single-node Docker Compose, ClickHouse-backed ^[T3]	Yes — Docker Compose ^[1]	Yes — `npx @portkey-ai/gateway` or Docker ^[2]	Yes — Docker/self-host ^[3]	Yes — hosted or self-host ^[22]	Yes — Docker Compose + Kubernetes ^[14]	Partial — Enterprise BYOC/self-hosted SKU ^[5]	Partial — Enterprise on-premise ^[15]	Yes — Docker, Kubernetes ^[16]
Open-source license	Apache 2.0 ^[T3]	MIT ^[1]	MIT ^[2]	Apache 2.0 ^[3]	Apache-2.0 (`Helicone/helicone`); GPL-3.0 (`Helicone/ai-gateway`) ^[22]	MIT ^[14]	None for OSS; Enterprise self-hosted SKU is proprietary ^[5]	Not confirmed	ELv2 (not Apache/MIT) ^[16]
SOC 2 / ISO 27001 (published report)	No ^[T22]	Not documented ^[7]	Yes — SOC2 + ISO27001 + GDPR + HIPAA; reports via paid plans ^[17]	Not documented ^[3]	Yes — SOC2 Type II at Team tier ($799/mo) ^[18]	Yes — SOC2 Type II + ISO27001 at Pro tier ( $199/mo){/* pricing-guard: allow$ 199 competitor-price */} ^[19]	Partial — Trust Center referenced, no detail ^[5]	Yes — SOC2 Type II, all tiers ^[20]	Not documented ^[16]
Data residency / regions	No customer-configurable data residency is documented on our pricing page as of 2026-09-07 ^[T22]	Self-host anywhere; no hosted multi-region ^[1]	VPC hosting for Enterprise ^[17]	Not documented ^[3]	Not documented ^[4]	US, EU, JP (vendor abbreviation) ^[19]	US or EU cloud; BYOC for custom ^[21]	Not documented ^[20]	Not documented ^[16]
Free tier	1 GB ingest / 0.25 GB hot / 100 series / 10 scan-units per mo, 60 RPM, 1 seat, no commercial use, rule-based, observe-first predictive layer ^[T3]	Free core; cloud pricing not public ^[7]	10K logs/mo, 3-day retention, 1 user ^[12]	Not documented ^[3]	10K requests/mo, 1 GB, 1 seat ^[18]	50K units/mo, 30-day data access, 2 users ^[19]	5K traces/mo, 14-day retention, 1 seat ^[21]	$0 +$ 10 credits/mo, 1 GB data, 14-day retention ^[20]	2 Phoenix Cloud instances free ^[16]
Pricing model summary	Six usage meters (ingest/hot/series/query/cold/evals) + tier price; $29 Builder /$ 229 Team / $799 Business /$ 2,499+ Enterprise ^[T3]	Free core; Enterprise sales-only ^[7]	$49/mo Production; Enterprise custom{/* pricing-guard: allow$ 49 competitor-price */} ^[12]	Not published ^[3]	$79/mo Pro;$ 799/mo Team ^[18]	$29/mo Core;$ 199/mo Pro; $2,499/mo Enterprise{/* pricing-guard: allow$ 199 competitor-price */} ^[19]	$39/seat/mo Plus; Enterprise custom{/* pricing-guard: allow$ 39 competitor-price */} ^[21]	$249/mo Pro; Enterprise custom{/* pricing-guard: allow$ 249 competitor-price */} ^[20]	Cloud pricing not public ^[16]

Portkey’s provider count

Portkey’s homepage says “1,600+ LLMs.” Its own GitHub repository’s src/providers directory, read the same day, lists roughly 75 provider modules — one module (e.g. openrouter) can route to many models, which partly explains the gap between a provider count and a model count. But Portkey’s own pages are not internally consistent: across different pages we found three different figures — 250+, 1,600+, and 3,000+ — for what should be one number. We’re not asserting which is “the” real count; we’re reporting that the vendor’s own marketing disagrees with itself. Read their repository yourself before treating any of the three as ground truth: github.com/portkey-ai/gateway/tree/main/src/providers. For our own “205 providers” claim: it is the count of crates/gateway/providers.tsv rows (185) plus the 6 native adapters, held to that file by a guard in CI — count it yourself.

Honest gaps

Things competitors publish that we do not have. In rough order of how likely they are to block a deal on day one:

SOC 2 Type II report (published). Portkey, Helicone (Team+), Langfuse (Pro+), Braintrust (all tiers) and LangSmith (Trust Center) all have or reference a published SOC 2. Not yet.
Data residency / multi-region. Langfuse offers US, EU, JP (its own abbreviation); LangSmith offers US or EU; Portkey and Helicone offer VPC hosting on Enterprise. Tracelane is single-region and residency is not customer-configurable. Not yet.
HIPAA / BAA. Documented by Portkey, Langfuse, Helicone, Braintrust and LangSmith, each on their higher tiers. Not yet.
SAML SSO, documented as a self-serve feature. Tracelane authenticates through WorkOS, but SAML is not documented as an available feature on any public Tracelane page today. Not yet.
Contractual SLA / uptime guarantee. apps/docs/pricing.mdx states plainly: “Tracelane does not offer a contractual uptime SLA or service credits.” Langfuse and Braintrust document one on their Enterprise tiers. Not yet.
Slack / PagerDuty alert integrations, documented. LangSmith documents webhooks + PagerDuty; Langfuse documents Slack/GitHub Actions/webhooks; Helicone documents alerts. Tracelane’s burn-rate alerts on paid plans are not documented as integrating with a specific paging tool. Not yet.

None of these are silent gaps — every one is stated as a limit in a public Tracelane doc (mostly pricing) rather than only showing up when you ask.

Sources

Tracelane (code-verified)

[T1] crates/gateway/src/admission.rs + crates/gateway/src/server/chat.rs — the Rust gateway hot path
[T2] /benchmarks — AIGatewayBench CCX23, 2026-09-07: Tracelane 2/4/5 ms, Portkey 3–4/5–7/8–9 ms, Bifrost 2/2/3 ms, LiteLLM Python 14–15/34–36/47–52 ms
[T3] /pricing — 205 providers, BYOK 0%, plan limits, Apache 2.0, Docker Compose, free-tier limits
[T4] exact-match caching built and proven; the semantic tier is built but not enabled on the hosted product
[T5] crates/gateway/src/guardrail/rails/ (r1_cost.rs … r8_injection.rs) + /pricing FAQ (“What inline guardrails run on every request?”) for the free-vs-Team+ split
[T6] crates/gateway/src/providers/mod.rs — model→provider routing fails closed to unroutable_model
[T7] Anthropic-native POST /v1/messages, deployed and proven on production 2026-09-06 with a real Claude Code session (3 proxied spans, tokens and cost present)
[T8] crates/shared/src/otlp/decode.rs — the parent_span_id must survive comment and the OTLP decode path
[T9] multi-agent swimlanes and full-text trace search, both shipped and deployed 2026-09-06
[T10] /v1/sessions and /v1/sessions/{id}/traces handlers in crates/gateway/src/server.rs
[T11] crates/gateway/src/spend.rs — cost-per-request tracking
[T12] “Ask Tara,” the in-app natural-language query surface, shipped and deployed 2026-09-06; distinct from the MCP server (below)
[T13] apps/mcp/src/reader.ts, GatewayReader reads /v1/* with the caller’s own bearer, tenant isolation proven against production ClickHouse; @tracelanedev/[email protected] published to npm 2026-09-07, confirmed by npm view @tracelanedev/mcp version
[T14] the Claude Code flight recorder (OTLP decoder aliases + cost backfill, deployed and proven 2026-09-06) and the Anthropic-native proxy (see [T7])
[T15] the public /s/<token> page, built and deployed 2026-09-06
[T16] crates/gateway/src/dataset_routes.rs and crates/gateway/src/experiment_routes.rs — datasets and experiments, proven on production
[T17] online evals with sampling, proven on production 2026-08-29, achieved 11.11% against a configured 10% rate
[T18] annotation queues, proven on production 2026-08-29
[T19] the CI eval gate and @tracelanedev/cli, proven on production 2026-08-30; @tracelanedev/[email protected] published to npm 2026-09-07
[T20] prompt management, playground and trace search, shipped 2026-09-06
[T21] /pricing audit-ledger description and /audit-ledger — Merkle-batched hash chain, Ed25519 per-tenant signing (live), best-effort Sigstore Rekor v2 inclusion proof on anchored batches
[T22] /pricing — no SOC 2 / ISO 27001 mentioned, no SLA clause; single-region; retention is set by plan and is not customer-configurable

Competitors (all fetched 2026-09-07 from vendors’ own pages)

github.com/BerriAI/litellm — “Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format”; MIT license
github.com/portkey-ai/gateway — “routes to 1,600+ LLMs across 45+ [vendors]”; “50+ AI Guardrails”; MIT license
github.com/maximhq/bifrost — “high-performance AI gateway unifying access to 23+ [vendors]”; Apache 2.0
helicone.ai — “10,000 free requests, 1 GB storage, 1 seat, 1 organization”
langchain.com/langsmith — “Control agent model calls” under the LLM Gateway heading
github.com/BerriAI/litellm — “8ms P95 latency at 1k RPS” (self-reported, not independently measured)
litellm.ai — “One OpenAI-compatible API to 140+ [vendors] and 1,800+ models”
portkey.ai — “Portkey lets you access 1,600+ LLMs via a unified API”
github.com/portkey-ai/gateway/tree/main/src/providers — directory listing, ~75 provider-module entries counted by hand; see “Portkey’s provider count” above
docs.litellm.ai/docs/proxy/configs — BYOK via litellm_params.api_key; per-deployment RPM/TPM
docs.litellm.ai/docs/caching/all_caches — “similarity threshold for cache hits, 0 == no similarity, 1 = exact matches”; 8+ backend options
portkey.ai/pricing — “ $49/month Production"{/* pricing-guard: allow$ 49 competitor-price */}; “100k logs/month”; semantic caching; “10k logs free / 3-day retention”; SOC2 + ISO27001
docs.litellm.ai/docs/proxy/reliability — “If a call fails after num_retries, LiteLLM falls back to another model group”
langfuse.com — OTel-native; “US, EU, or Japan”; free 50K units / 2 users / 30-day; annotation queues; agent graphs
braintrust.dev — “Tracing, evaluation, and storage infrastructure”; SOC2 Type II all tiers; “ $249/mo Pro"{/* pricing-guard: allow$ 249 competitor-price */}; “14-day retention Starter”
arize.com/phoenix — “ELv2 licensed”; “native OpenTelemetry support”; “2 Phoenix Cloud instances for free”; Docker/Kubernetes self-host
portkey.ai/pricing — “SOC2, ISO27001, GDPR, HIPAA Compliance Certificates”; VPC hosting Enterprise
helicone.ai/pricing — “SOC-2 Type II at Team tier ($799/mo)”; “10,000 free requests, 1 seat”
langfuse.com/pricing — “SOC2 Type II + ISO27001 reports at Pro ( $199/mo)"{/* pricing-guard: allow$ 199 competitor-price */}; “US, EU, or Japan”; “Hobby: 50K units/30-day/2 users”
braintrust.dev/pricing — “SOC 2 Type II” (all tiers); “ $249/mo Pro"{/* pricing-guard: allow$ 249 $10 competitor-price */}; "14-day retention Starter"; "$ 0 + $10 credits/mo free”
langchain.com/pricing — “ $39/seat/mo Plus"{/* pricing-guard: allow$ 39 competitor-price */}; “5K traces/mo free”; “14-day retention base”; “US or EU cloud”
github.com/Helicone/helicone and github.com/Helicone/ai-gateway — per-repository LICENSE files; self-hosting documented in the README

Matrix compiled 2026-09-07. Every competitor source above was fetched that day from an official vendor page — a docs site, a pricing page, or a public GitHub repository. If anything here is inaccurate or out of date, please open an issue and we’ll correct it.

Get started

SDKs

Integrations

Concepts

Features

Tools

Audit & compliance

Operations

Migrations

Comparisons

How Tracelane compares

Method

Gateway

Observability

Evals

Trust

Portkey’s provider count

Honest gaps

Sources

Tracelane (code-verified)

Competitors (all fetched 2026-09-07 from vendors’ own pages)

​Method

​Gateway

​Observability

​Evals

​Trust

​Portkey’s provider count

​Honest gaps

​Sources

​Tracelane (code-verified)

​Competitors (all fetched 2026-09-07 from vendors’ own pages)

Method

Gateway

Observability

Evals

Trust

Portkey’s provider count

Honest gaps

Sources

Tracelane (code-verified)

Competitors (all fetched 2026-09-07 from vendors’ own pages)