Skip to content
Nexus Docs
↑↓Navigate↵SelectescClose

0027. Store responses for retrieval by ID

Synced from the Nexus repository

Last updated on:
  • Status: Proposed
  • Date: 2026-09-02
  • Deciders: nexus core team
  • Tags: gateway, openai, storage, retention, classification, authorization

Context and Problem Statement

Section titled “Context and Problem Statement”

The OpenAI-compatible gateway accepts POST /v1/responses, but it does not expose GET /v1/responses/{id}. A retrieval route needs an authoritative association between the identifier returned from response creation, the response content, and the Nexus scope that owns it.

Nexus routes requests across provider and credential boundaries. An upstream provider may not support response retrieval, a fallback may move creation to a different provider, and a tenant may later rotate or remove the BYOK credential used for creation. An upstream response identifier alone therefore cannot provide stable Nexus retrieval semantics.

Response content can contain prompts, generated text, tool calls, and other tenant data at the request’s classification. Persisting it creates a security and data-lifecycle obligation distinct from always-on analytics metadata and audit evidence. The current traceability boundary in ../architecture/traceability-auditability.md does not persist full request or response bodies by default; full payload capture is policy-governed.

Decision Drivers

Section titled “Decision Drivers”
  • A successful retrieval must not depend on the original provider, credential, or gateway process still being available.
  • The identifier returned by Nexus must resolve to at most one response within the authenticated tenant, organization, and project.
  • Cross-scope probes must not reveal whether a response exists.
  • Stored response content must retain its classification and pass authorization and policy evaluation on every read.
  • Retention expiry and explicit deletion must remove the response from the read path and arrange deletion of its content.
  • The retrieval path needs predictable read-after-write behavior and cannot depend on eventual ingestion into analytics storage.
  • Response storage must remain separate from audit records and always-on analytics metadata.

Considered Options

Section titled “Considered Options”
  • Nexus-managed response storage — Nexus issues the public response ID and stores scoped metadata plus response content. This gives provider-independent retrieval, consistent authorization, and a lifecycle Nexus can enforce, at the cost of storing sensitive tenant content and operating a new durable resource.
  • Provider-backed retrieval without Nexus-managed content storage — Nexus stores or reconstructs enough routing information to call the original provider’s retrieval API. This avoids duplicating response content, but fails when a provider lacks retrieval, a credential is rotated or revoked, or routing used a provider-specific translation. It also makes retention and deletion depend on provider behavior and still requires Nexus to persist sensitive provider linkage.
  • Reuse tenant analytics or trace-body storage as the response store — This avoids a separate write path, but analytics ingestion is asynchronous and trace bodies are optional. Neither provides the authoritative, read-after-write lifecycle required by an API resource. Analytics and trace retention may also differ from response-resource retention.
  • Do not support GET /v1/responses/{id} — This avoids new persistence and privacy obligations, but leaves a gap in the OpenAI-compatible surface and prevents clients from recovering a response after creation.

Decision Outcome

Section titled “Decision Outcome”

Chosen option: Nexus-managed response storage.

Supporting GET /v1/responses/{id} requires Nexus to persist retrievable responses as first-class, project-scoped resources. This ADR does not add that route or any storage implementation.

Identity and creation

Section titled “Identity and creation”

Nexus assigns an opaque, globally unique Nexus response ID when accepting a storable POST /v1/responses request. That ID is the id returned on the OpenAI-compatible response and is the only identifier accepted by the retrieval route. Any upstream response ID is internal routing metadata and is never sufficient to authorize or locate a response.

A response metadata record is keyed by the Nexus response ID and includes, at minimum, tenant_id, org_id, project_id, creation time, expiry time, deletion state, classification label, payload state, content hash, and an object-store locator. The complete OpenAI-compatible response object is stored as content through the existing Nexus object-store abstraction rather than inline in Postgres or ClickHouse.

The metadata record and content become visible as one logical resource. Creation does not report a stored response as successful until its content is durable and its metadata is readable, so an immediate retrieval does not depend on asynchronous analytics ingestion. If persistence fails, the gateway must not claim that the response was stored; partial metadata or content is cleaned up without exposing a retrievable resource.

Storage eligibility follows the store setting on response creation and tenant policy. A response for which storage is disabled or denied is returned to the caller but has no retrievable Nexus resource, and a later lookup returns the same not-found response used for an unknown or inaccessible ID. Policy may force storage off or require redaction, but cannot silently broaden content capture beyond the request’s storage setting.

For streaming creation, Nexus derives the retrievable response object from the terminal response state rather than storing the SSE byte stream as the resource representation. An interrupted or failed stream is not exposed as a successfully stored response unless the OpenAI-compatible contract defines a terminal response object that Nexus can persist with its actual status.

Retrieval and tenant isolation

Section titled “Retrieval and tenant isolation”

GET /v1/responses/{id} authenticates through the existing OpenAI edge and derives tenant, organization, and project scope from the verified Nexus principal. The metadata lookup always constrains all four values: response ID, tenant ID, organization ID, and project ID. Object keys and locators are likewise scoped, and the gateway never accepts an object locator or provider identifier from the caller.

An unknown, expired, deleted, or out-of-scope ID returns the same OpenAI-compatible 404 shape. This prevents the route from becoming a cross-tenant existence oracle. Successful reads return the stored OpenAI-compatible response representation; they do not issue a new provider request, create usage, or charge the tenant again.

Retrieval performs a policy decision for the authenticated principal and a response.read action before reading content. The resource attributes include its authoritative tenant, organization, project, classification, and payload state. The gateway applies all returned obligations and fails closed on unknown obligations. Possession of a response ID is not authorization.

Classification and content handling

Section titled “Classification and content handling”

The response record snapshots the resolved classification label from the creation request. It does not recompute that label from a later default. Stored output is governed at least as strictly as its associated request, including when provider output contains data that was not present in the prompt.

The classification label remains tenant data and does not enter Tier 2 operator telemetry. Response content is never written to audit events, logs, Postgres metadata fields, or ClickHouse analytics rows. Audit records describe lifecycle and access outcomes using identifiers and safe metadata, while the response bytes remain in the object store. Redaction obligations are applied before persistence when policy permits a redacted stored representation; the metadata records that payload state so retrieval cannot imply that omitted content is complete.

Retention and deletion

Section titled “Retention and deletion”

Every stored response has an expires_at value fixed at creation from the effective tenant or project retention policy. Changing a default retention policy does not silently extend existing response lifetimes. A policy may shorten retention or prohibit storage for higher classifications. An expired response is excluded from reads immediately even if physical object deletion has not completed.

Explicit response deletion marks the scoped metadata record deleted before content removal is attempted. After that state change, reads return 404. Object deletion is retried asynchronously and is idempotent so a transient object-store failure cannot restore logical access. Retention expiry uses the same logical-delete and physical-cleanup path. Backup and object-store lifecycle settings must bound residual copies consistently with the declared retention period.

Legal hold is not implemented in Nexus today, as recorded in ../glossary.md. This design does not imply a legal-hold override; adding one requires a separate decision before it can prevent expiry or deletion.

Architectural placement

Section titled “Architectural placement”

The OpenAI edge owns creation and retrieval semantics and invokes a response repository rather than querying analytics services. Postgres holds authoritative resource metadata and lifecycle state because the read path requires scoped point lookup and consistent visibility. The existing object-store abstraction holds response bytes, using tenant, organization, and project prefixes and integrity-checked locators.

The asynchronous request analytics pipeline continues to receive operational metadata and, when tracing is enabled and policy permits it, its independently governed trace payloads. It is not the source of truth for response retrieval. The audit pipeline records response storage, retrieval, expiry, and deletion events without recording response content and remains distinct from request analytics.

Consequences

Section titled “Consequences”
  • Nexus can provide stable response retrieval across provider routing, fallback, credential rotation, and gateway restarts.
  • A Nexus response ID has consistent meaning and does not expose provider identifiers or make provider availability part of the read path.
  • Read-after-write requires a synchronous durable-storage step on storable response creation, increasing creation latency and adding a failure mode after provider completion.
  • Nexus assumes responsibility for sensitive response content, including encryption and access controls at the storage layer, policy enforcement, retention sweeps, deletion retries, backup lifecycle, and capacity planning.
  • Storage and retrieval add object-store and Postgres cost even when the upstream provider already retains the same response.
  • Scope-constrained lookup and uniform 404 behavior reduce cross-tenant disclosure, but response IDs must still be treated as sensitive correlation identifiers.
  • Classification is preserved with the resource, and reads can become unauthorized after policy or principal permissions change even though the response still exists.
  • Response-resource retention is independent of analytics, tracing, and audit retention; deleting a response does not erase required non-content billing or audit evidence.
  • Responses created without permitted storage cannot be retrieved later, and Nexus cannot reconstruct them from analytics metadata.
  • Implementation requires coordinated schema, object-storage, policy-action, gateway, audit, cleanup, and end-to-end test changes; none are included in this ADR.

More Information

Section titled “More Information”