Skip to content

CI/CD Pipeline

This is the contributor reference for the Nexus GitLab pipeline and the local commands that reproduce its checks. Run commands from the repository root unless a command says otherwise.

Merge-request pipelines are the fast feedback gate for code review. They run the checks that do not require a running Nexus stack or live provider credentials:

  • Documentation reference lint: bash scripts/check-adr-references.sh.
  • Rust formatting and linting: just fmt-check and just lint.
  • Rust tests: just test-workspace.
  • Dependency policy: just deny, just licenses-rust, just licenses-node, and just licenses-images.
  • TypeScript workspace checks: just node-lint and just node-build.
  • Security gates: gitleaks for secrets, checkov for infrastructure-as-code, and semgrep for console TypeScript/Next.js rules.

The checked-in pipeline may split these into separate jobs, but every job should wrap one of the local commands above or print the exact tool invocation it runs.

Default-branch pipelines repeat the merge-request gates and build SHA-tagged container images for every published runtime image. Images are pushed to the GitLab Container Registry under:

registry.gitlab.com/vortexaq/nexus/vnexus-<image>:sha-<short-sha>
registry.gitlab.com/vortexaq/nexus/vnexus-<image>:main

The image tier generates a CycloneDX SBOM with syft and scans the image/SBOM pair with grype. Rust images are built with cargo auditable build, which embeds the crate dependency list in the binary; syft inventories those crates from the static binary, so image SBOMs and grype scans cover Rust dependencies, not just OS packages. SBOM generation fails for a Rust image whose SBOM lacks cargo components, which guards against a build regression that would silently produce dependency-blind scans. The scan job also fails if the grype advisory database is older than the configured window (GRYPE_DB_MAX_AGE_DAYS, default 7 days), so a stale or no-longer-published database schema cannot produce clean-looking results. The scan gate fails on High or Critical vulnerabilities unless an exception is checked in with a justification, expiry or review date, and security review. Default-branch images are build artifacts for validation and deployment testing; they are not release artifacts unless they are produced by the tag release flow.

Protected tags that match v* run the release tier. The release tier checks that the tag points at default-branch history, rebuilds images from the tagged commit, generates SBOMs, scans them, signs each image with the Nexus cosign release key, attaches SBOM, provenance, and vulnerability-scan attestations, verifies the signatures and decoded predicate contents in a separate job, and then publishes the GitLab Release.

The release job publishes only after verification succeeds for every image. A tag pipeline that cannot prove the image signature, SBOM attestation, provenance attestation, and scan-result attestation fails closed.

Reproduce the GitLab lint and test jobs without a running stack or provider credentials:

Terminal window
just ci-tests

Run the full offline policy gate, including dependency and artifact license checks:

Terminal window
just checks

just checks runs Rust formatting, clippy, workspace tests, dependency/license checks, image license boundary checks, and documentation reference linting. Use focused commands when reproducing one failing job:

Terminal window
just fmt-check
just lint
just test-workspace
just deny
just licenses-rust
just licenses-node
just licenses-images
just node-lint
just node-build
bash scripts/check-adr-references.sh

Security tools can be reproduced with the same open-source CLIs used by CI:

Terminal window
gitleaks detect --source . --config .gitleaks.toml --redact --verbose --log-opts="--all"
checkov --config-file deploy/pipeline/scan/checkov.yaml
semgrep scan --error --metrics=off --config p/typescript --config p/nextjs apps/console packages

If a security job reports a finding, fix the underlying issue when possible. Suppressions and exceptions must include the reason, the scope, the reviewer, and the next review date.

The release flow is:

  1. Build each runtime image with Docker BuildKit/buildx.
  2. Push immutable image tags and record the digest.
  3. Generate a CycloneDX JSON SBOM for each image with syft.
  4. Scan each image and SBOM with grype.
  5. Sign each image digest with the Nexus cosign release key.
  6. Attach SBOM, SLSA-style provenance, and vulnerability-scan predicates as cosign attestations.
  7. Verify every signature and attestation before the GitLab Release is created.
  8. Generate an LLM-assisted human release report from verified release notes, image metadata, SBOMs, scan reports, and provenance predicates.
  9. Publish the GitLab Release with the report, image digests, artifact links, and verification commands.

Consumer verification commands live in Releases and Verification.

The image and release tiers require self-hosted GitLab runners with:

  • Docker executor or equivalent container build isolation.
  • Docker BuildKit/buildx.
  • Access to the GitLab Container Registry for registry.gitlab.com/vortexaq/nexus.
  • Persistent cache volumes for Cargo, pnpm, and BuildKit registry cache.
  • Enough CPU and disk for concurrent Rust image builds; 8 CPU and a persistent cache volume are the practical baseline.
  • Protected cosign key variables: COSIGN_PRIVATE_KEY_FILE, COSIGN_PUBLIC_KEY_FILE, and COSIGN_PASSWORD.
  • Release-report CI variables for an OpenAI-compatible LLM endpoint: LLM_RELEASE_REPORT_BASE_URL, LLM_RELEASE_REPORT_MODEL, and protected/masked LLM_RELEASE_REPORT_API_KEY.

The encrypted signing key is provided to release jobs as a protected GitLab CI file variable; the public verification key is checked in at deploy/pipeline/signing/cosign.pub. Release signatures and attestations are recorded in Rekor. The LLM report is generated only after verification succeeds; deterministic artifact links and per-CVE vulnerability findings are produced by CI and included in dist/pipeline/release/evidence.json and dist/pipeline/release/report.md. The report includes a security decisions section based on the checked-in threat model, compliance map, telemetry tiering rules, and ADR excerpts. The report job also renders dist/pipeline/release/report.pdf, a self-contained evidence document described in Releases and Verification; PDF generation is best-effort and never blocks the Markdown report.

The following suites are intentionally not part of the default CI gate because they require a running stack, live provider credentials, browser interaction, or API spend:

  • just test
  • just test-compat
  • just test-gateway-cert
  • just compare
  • just test-adopter-e2e
  • just test-adopter-ptb-e2e
  • just node-test
  • just proto-lint
  • Billing reconciliation scripts under test-suite/billing

Run these when the change touches the corresponding gateway, provider, analytics, billing, or adopter workflow. Record the commands and relevant artifact paths in the merge-request test plan.

The compose-backed Rust integration target is available as:

Terminal window
just compose-it

Use it when a change needs a local stack exercise but does not need live model-provider calls.