0006 - Two-phase BYOK-then-PTB attempt routing

• Status: Accepted
• Date: 2026-07-03
• Deciders: nexus core team
• Tags: gateway, routing, byok, ptb, billing

Tenants that configure their own provider keys (BYOK) expect requests to run on their credentials and their provider bill. Operators that enable Pass-Through Billing (PTB) lend platform credentials to tenants and recover cost from an organization wallet. Mixing the two silently is unacceptable: a tenant request must never ride operator credentials without an explicit opt-in, and a PTB request must never let the wallet go negative. The gateway needs one routing plan per request that keeps the billing boundary explicit at every fallback position.

• BYOK traffic must be unaffected by whether PTB is deployed or enabled.
• PTB spend must be reserved before the upstream call, not reconciled after the fact (see ADR-0016 Postgres escrow).
• Every attempt in the fallback chain must carry its own billing identity for audit and analytics attribution.
• Retry behavior inside an attempt is a separate concern (see ADR-0017 retry policy).

• Two-phase attempt list: BYOK attempts first, PTB attempts appended when enabled — the billing boundary is decided once, at plan-build time.
• PTB-first or a single cost-interleaved list — cheapest route wins regardless of whose credentials it uses; rejected because it produces surprise operator spend for tenants who configured their own keys.
• Single-phase list with a per-attempt billing decision in the executor — rejected because it pushes wallet logic into the hot execution loop and blurs audit attribution per attempt.
• No PTB at all — rejected; operators running managed deployments need a way to onboard tenants without provider keys.

Chosen option: two-phase attempt list, BYOK first, PTB appended only when explicitly enabled.

AttemptBuilder in services/gateway/src/routing/builder.rs builds the plan. For each candidate endpoint it enumerates a BYOK attempt from the tenant’s stored provider key first, then appends a PTB attempt on platform credentials only when NEXUS_GATEWAY_PTB=1 and the endpoint allows PTB. Each provider key carries a byok_only flag from the control plane, propagated onto the attempt. When one canonical model fans out across providers with no provider pin, attempts are ranked by per-token cost; caller-authored comma-separated route lists and nexus-fallback header routes preserve the caller’s order.

Each Attempt in services/gateway/src/routing/types.rs carries auth_type (byok or ptb), needs_escrow, escrow_budget_usd, and a source string suffixed with the phase. PTB attempt budgets are computed at build time as a worst-case cost estimate with a configurable safety margin.

The executor in services/gateway/src/routing/executor.rs runs attempts in order, retrying or failing over per attempt. Before a PTB upstream call it reserves the budget through the escrow service; on success it commits the escrow at measured cost, and on failure it cancels the reservation. Spend caps evaluated in services/gateway/src/spend_cap.rs gate requests independently of the escrow lifecycle.

• Tenants with provider keys never spend operator money unless the operator has enabled PTB and the BYOK attempts are exhausted.
• The attempt trail and request events record auth_type per attempt, so audit and analytics can attribute every upstream call to a billing identity.
• Worst-case cost estimation runs on the hot path for every PTB attempt so the escrow reserve can precede the upstream call.
• PTB routing is disabled entirely unless NEXUS_GATEWAY_PTB=1; a deployment without the flag behaves as a pure BYOK gateway.