tlane CLI is the operator-side companion to the Tracelane gateway.
Commands
tlane init
Scaffolds Tracelane into the project in the working directory. It writes files
and installs the SDK — four steps, each one skippable:
What it will not do.
.env is only ever appended to — an existing
TRACELANE_API_KEY is never rewritten, --force included. An existing config or
bootstrap is kept unless you pass --force. A failed install exits non-zero and
prints the command to re-run; the scaffolded files stay.
How far the bootstrap gets on its own. On Python it emits
init(..., auto_instrument=True), which really does wrap installed openai,
anthropic, litellm and claude_code with no further edit. Everything else —
including every TypeScript adapter — wraps an object only you can construct, so
the bootstrap imports the right instrument* function and emits the exact
one-line call next to it.
--endpoint is the OTLP endpoint spans are exported to. On Tracelane Cloud that
is https://gateway.tracelane.dev, with a tlane_… key carrying the ingest
scope. Self-hosting, it is the ingest receiver you run. The default is
http://localhost:4318 — a local receiver — so on Cloud, pass --endpoint
explicitly.
tlane trace <traceId>
Fetch and render a trace. Fetches the trace directly from the Tracelane API
using TRACELANE_API_KEY (tenant-scoped, same auth as the dashboard).
tlane eval run / tlane eval list
The CI gate. eval run runs a frozen dataset (or an inline case list)
against a prompt version through the gateway and exits non-zero when the pass
score falls below the --threshold floor, so a change that drops below your bar cannot merge. eval list
shows recent eval runs in your workspace.
--env (default staging), so no UUID has
to live in your workflow file. Pass --version-id to pin one.
The suite file holds the assertions, which belong in your repo next to the
prompt and versioned with it:
assertions must be non-empty. A run that asserts nothing scores every case as
passed, so the gate would report 100% on the day the prompt breaks; the CLI
refuses it with exit 2 before spending a provider call.
The comparison, stated: higher is better and --threshold is a floor —
--threshold 0.8 means “fail if the mean score is below 0.8”. A tie passes
(score == threshold exits 0), because a gate that fails on exactly the number
you set is a gate nobody can configure. --threshold is a fraction in [0,1],
so a bare 80 is rejected rather than read as 8000%.
It thresholds the mean SCORE, not the pass rate. For contains,
exact_match and json_schema a case scores exactly 1.0 or 0.0, so the mean
is the pass rate. For an LLM judge the score is continuous, and there the two
differ: a judge scoring 0.68 against a 0.70 rule and one scoring 0.02 are the
same “failed” and very different results.
Errored cases are excluded from the mean, not scored as zero. One provider
429 in a twenty-case run is not a quality problem. They are bounded separately by
--max-error-rate (default 0.10); above it, and for a run where every case
errored, the verdict is could not evaluate (exit 3) — never a pass and
and never a statement about your prompt at all.
This gate asserts a FLOOR on a single run — it does not detect regressions.
There is no baseline, no history and no comparison: the same shape as a coverage
threshold, which nobody considers broken for lacking a previous run. A run
scoring 0.9 today and 0.85 tomorrow clears a 0.8 floor both times, and calling
the second “no regression” would be a claim nothing checked. Comparing a run
against an earlier one is not supported.
Running the Tracelane repository’s own conformance suite is a separate thing and
is not part of this command: from a checkout, use pnpm eval:run --suite=all.
Changed in
0.3.0. tlane eval run used to shell out to the repo’s vitest
runner and needed a checkout. It is now the gateway-backed CI gate described
above. The old --suite flag is still recognised and exits 2 with a pointer to
pnpm eval:run, rather than failing as an unknown option.tlane prompt list | show | promote | rollback | diff
Front-end for Prompt Promotion.
Every subcommand takes the prompt name as a positional argument, plus the
shared --gateway <url> / --token <bearer> (defaulting to
$TRACELANE_GATEWAY_URL and $TRACELANE_TOKEN).
promote and rollback both exit 2 without --version-id; diff requires
both --from-env and --to-env. The gate flag is --eval-run <uuid> — an
eval-run id, not an eval name.
tlane verify <ledger.ndjson>
Verify a tamper-evident audit ledger. Re-runs the per-tenant SHA-256 hash chain, and
— when you supply the trusted key — checks the Ed25519 per-tenant signatures and any
resolved Sigstore Rekor v2 inclusion proof. Divergences print with their seq (first 10).
--tenant-pubkey is what makes the run mean something. Without it the verifier checks the hash chain only: signature and Rekor-anchor verification never run, so a forged anchor would not be caught. The CLI therefore exits non-zero with INCOMPLETE whenever a ledger contains anchor records and no trusted key was supplied. Get the key out-of-band from Settings → Audit signing key, or GET /v1/audit/pubkey — not from the export itself.
All three verifiers (Rust, Python, TypeScript) agree verdict-for-verdict on the shared
conformance vectors in evals/audit-ledger/.
tlane export --pack <name>
Generate a documentation pack: static markdown templates describing how
Tracelane logs, which models it routes to, and how it handles data, plus a
manifest.json, emitted as a ZIP.
It reads no ledger and makes no network calls, so it contains none of your
data. For your actual ledger, export it from /v1/audit/export and run
tlane verify.
--pack is required. --output-dir is the staging directory for the pack files
(default ./compliance-pack); the ZIP is written to the working directory
unless you pass --no-zip, which keeps the directory instead. There is no
positional pack argument and no --output flag.
tlane migrate helicone
Rewrite a project’s Helicone wiring in place. Scans the project root for env
files and source files carrying Helicone references, prints a diff, and — with
--apply — writes the rewritten files after an interactive confirm.
--apply, --dir <path> (default: cwd) and --endpoint <url>.
tlane import-litellm --config <path>
Translate a LiteLLM model_list to Tracelane gateway routing. The config path
is a flag, not a positional argument — a path passed positionally is ignored and
the default litellm_config.yaml is read instead.
tlane replay <traceId>
Read-only trace inspection — fetches a recorded trace’s spans and renders them
step-by-step. It does not re-execute models or tools. For a fresh evaluation,
use an eval run; recorded-trace inspection is not a new execution.
Exit codes
3 fails your CI job on purpose, and there is no flag that turns it into 0.
“The gate could not check” and “the gate checked and was satisfied” are
different facts, and a step that goes green on the first silently disarms the
gate for your whole repo.