Architecture Overview

This page is the high-level picture of nexus as it exists in this repository today, with pointers to deeper docs. Per-component detail lives in services.md; schemas in data-model.md; HTTP and gRPC shapes in wire-contracts.md. File and vector-store ownership is described in artifact-storage.md.

What nexus is (today)

nexus is a self-hostable AI gateway and local control-plane toolkit. A developer can:

• bootstrap organizations, projects, Nexus API keys, and BYOK provider keys via the nexus CLI against the control-plane Postgres database (NEXUS_DATABASE_URL);
• run nexus ops serve (or the nexus-gateway binary) to obtain an OpenAI-compatible edge (POST /v1/chat/completions and the other routes in services.md) that forwards to upstream providers using the registered BYOK secret;
• emit structured audit lines locally (audit.log) and metrics/health endpoints consistent with other Rust services in the workspace.

The Docker Compose stack in deploy/compose/docker-compose.yml (merged with deploy/compose/docker-compose.dev.yml when you run just dev from the repository root) brings up backing stores (Postgres, ClickHouse, MinIO for local S3-compatible object storage, NATS, and Valkey as the Redis-compatible cache), the Rust service binaries, and the console Next.js app. The dev overlay bind-mounts the repo, builds Rust dev images for linux/amd64 by default (aligned with the base stack and typical Linux x86_64 hosts; on Apple Silicon set DOCKER_PLATFORM=linux/arm64 in .env), and uses a one-shot cargo-build (cargo build --workspace --bins with sccache as RUSTC_WRAPPER) to populate shared CARGO_HOME, RUSTUP_HOME, CARGO_TARGET_DIR, and SCCACHE_DIR volumes before any service runs cargo watch — so the heavy compile happens once and the watchers do incremental work against the same target/. The console runs next dev on the same bind-mount. Use just dev-release for the release-style images only (CI parity with linux/amd64 musl/distroless).

System diagram (implemented paths)

Solid lines are wired in code today.

flowchart LR
  client[LLM_client_or_SDK]
  browser[Browser]
  cli[nexus_CLI]

  subgraph edge [Edge]
    gateway[gateway_axum]
  end

  subgraph services [Compose_services]
    cpHttp[control_plane_REST_gRPC]
    authHttp[auth_REST_gRPC]
    policyHttp[policy_PDP]
    traceIngestSvc[trace_ingest_OTLP]
    ingestSvc[ingest]
    auditSvc[audit]
    queryHttp[query_HTTP_gRPC]
    consoleApp[console_Nextjs]
  end

  subgraph stores [Compose_data_plane]
    pg[(Postgres)]
    ch[(ClickHouse)]
    minio[(MinIO)]
    nats{{NATS_JetStream}}
    redis[(Valkey_Redis_compatible)]
  end

  client -->|OpenAI_HTTP| gateway
  gateway -->|verify_keys_fetch_BYOK| cpHttp
  gateway --> pg
  gateway -->|nexus.requests| nats
  gateway -->|nexus.audit.events.tenant| nats
  gateway --> redis

  cli --> cpHttp
  cpHttp --> pg
  authHttp --> pg
  cpHttp -->|nexus.audit.events.tenant| nats

  client -->|OTLP_traces| traceIngestSvc
  traceIngestSvc -->|nexus.traces| nats
  nats --> ingestSvc
  ingestSvc --> ch
  ingestSvc --> minio
  ingestSvc -->|nexus.audit.events.tenant| nats
  nats --> auditSvc
  auditSvc --> pg
  auditSvc --> minio

  queryHttp --> ch
  queryHttp --> minio
  queryHttp --> nats

  browser --> consoleApp
  consoleApp --> authHttp
  consoleApp --> cpHttp
  consoleApp --> queryHttp

Service summary (status today)

Service	Role today	Detail
gateway	OpenAI-compatible BYOK edge (chat, responses, embeddings, images, models, files, vector stores) with attempt-based routing, optional Redis-compatible cache + rate limit, audit + analytics emission	services.md
control-plane	Org/project/membership/API-key/provider-key/model-registry REST + gRPC backed by Postgres	services.md
auth	Email/password users + sessions, HMAC JWT, console session cookie, API-key verify; Postgres backed	services.md
policy	Request-time PDP: classification-ceiling enforcement, a decision cache, and policy.decision audit events; evaluated in-process by the gateway or over gRPC	services.md
ingest	JetStream nexus.requests.* consumer → ClickHouse rows + MinIO bodies + audit fan-out	services.md
audit	Tenant-scoped audit-event consumer, per-tenant Merkle batches, signed anchors, archive writer, verifier/read RPCs	services.md
trace-ingest	Public OTLP/HTTP intake for tenant-visible agentic traces, key-verified, fanned out on nexus.traces.*	services.md
query	ClickHouse + object-store read API (requests, sessions, traces, usage, live SSE) consumed by console	services.md
console	Tenant Next.js app for auth, control-plane management, request/session/trace inspection, usage dashboard, audit viewer	services.md

Data stores

Store	Purpose in Compose today
Postgres	Backing store for control-plane (orgs, projects, API keys, provider keys, model registry, wallet/escrow), auth (users, sessions), and audit (sealed events, Merkle roots). The gateway and CLI reach the control-plane database through the in-process ControlPlane handle.
ClickHouse	ingest writes the requests table; query reads via HTTP and gRPC. Schema in services/ingest/migrations/clickhouse/.
S3-compatible object storage	Local Compose uses MinIO so ingest can write zstd-compressed request/response bodies and audit can write audit archive manifests and compressed batch event files. Production deployments should provide an S3-compatible service with Object Lock/WORM semantics. query produces V4-presigned GET URLs for body fetch; in dev a filesystem fallback is served via /_bodies/*. The gateway holds no credentials for the request-body or audit-archive buckets; it does hold artifact-bucket credentials for file/vector-store routes (see artifact-storage.md).
NATS JetStream	gateway publishes per-request analytics envelopes on nexus.requests.<org_id>.<request_id> (inline ≤ 256 KB or chunked); audit events flow on nexus.audit.events.<tenant_id>. ingest runs the durable ingest_requests pull consumer; audit runs the durable audit pull consumer; query subscribes for live SSE. The gateway reloads its registry snapshot by a 60-second Postgres poll, not over NATS.
Redis-compatible cache	Local Compose uses Valkey. The gateway uses this endpoint for response caching (nexus-cache), per-API-key rate limiting, and disallow-list lookups.

Trust boundaries

1. Client ↔ gateway. TLS is the operator’s responsibility in production; local dev often uses plain HTTP. The gateway verifies the Nexus API key before calling upstream.
2. Gateway ↔ providers. BYOK secrets are loaded through ControlPlane + KmsProvider; upstream calls use reqwest with TLS verification to the provider.
3. Operator ↔ tenant data. In a self-hosted deployment the operator controls the databases, object storage, and the local state directory (KMS wrapping key, dev audit sink); there is no additional hosted separation between operator and tenant data.
4. nexus ↔ vortex-* shared crates. Shared across Vortex products under ../../vortex-common-crates/contracts/shared-crates.md; changes require review per that contract.

Telemetry tiers

nexus splits observability into two pipes in the contracts (../../vortex-common-crates/contracts/telemetry.md):

• Tier 1 — tenant-visible analytics: the ingest service writes ClickHouse tables and the console serves usage/request views over the query service.
• Tier 2 — operator OTLP traces/metrics through the OpenTelemetry Collector; the dev compose stack ships Grafana/Tempo/Loki/Prometheus for local inspection.

The gateway emits Prometheus-style counters and histograms (nexus_gateway_attempt_total, nexus_gateway_attempts_per_request, nexus_gateway_fallback_invocations_total, nexus_gateway_rate_limited_total, nexus_gateway_cache_hits_total, etc.) via metrics macros.

Full payload trace capture is separate from both tiers. Audit events and tenant analytics metadata are always emitted; request/response bodies and full tool-call payloads are stored only when tracing is enabled. See traceability-auditability.md.

Gateway Scope Boundary

The gateway owns request authentication, model/route resolution against the in-memory registry snapshot, provider-specific request/response translation, upstream transport, per-tenant rate limit and cache hooks, artifact file/vector-store routes, in-process vector indexing/search, and audit/analytics emission. It does not own:

• Provider-side schema validation. The compatibility layer in services/gateway/src/compat.rs only validates fields the gateway must branch on (for example stream: bool); everything else is forwarded to the upstream and any provider-shaped error is returned to the client unchanged.
• Policy decision logic. The gateway evaluates gateway.request queries through the policy engine (in-process, or via the policy service over gRPC) and applies the returned decision and obligations; the decision logic itself lives in services/policy.
• Hosted retrieval injection. File and vector-store routes are tenant-scoped compatibility resources backed by control-plane Postgres metadata, the configured artifact object store, and the pgvector chunk index (see artifact-storage.md).
• Tenant analytics materialization, body archival, or query. Those live in ingest and query.

Cache keys are scoped by (tenant, org, project, api-key subject, endpoint_key, auth_type, body) to prevent cross-tenant collision. Attempt order is preserved as authored by the caller: only canonical-model fan-out across providers is cost-sorted, comma-separated routes and nexus-fallback headers retain caller order.

How this page evolves

Update this file and services.md whenever a merged MR changes observable architecture (new routes, new bus consumers, new UI surfaces). Keep diagrams consistent with what the code actually does.