Documentation Sync Automation

This documentation site receives automated sync merge requests from the internal Nexus code repository. External contributions become eligible after the team reviews and merges them into the internal repository. The sync requests review from the Nexus author and configured docs maintainers. Publication happens only after the docs merge request reaches main. Required approvals are enforced by GitLab project settings, not by the sync scripts.

Overview

Goal: Keep the docs site in sync with documentation in the Nexus codebase, with human approval on every change and without ever disturbing the curated sidebar order.

How it works:

1. Developer merges documentation changes to nexus main
2. Nexus pipeline triggers a cross-project pipeline in this repository
3. The sync job clones Nexus, detects changed docs, and applies them to a sync/* branch
4. A merge request is opened, with the Nexus author and the docs maintainer requested as reviewers
5. Both approve (or adjust) the MR; merging it publishes via the normal build/deploy pipeline

Architecture

Sync flow and checks

Documentation sync flow from Nexus to publication

1. NexusMerge to main or push a protected v* tagThe Nexus pipeline must pass before its dispatch job runs.
2. ↓
3. Sender gateIs this the canonical internal Nexus project and an eligible ref?The public mirror and merge request pipelines cannot dispatch a sync.
4. ↓
5. Receiver gatesValidate pipeline source, trigger type, and commit identifierThe receiver always clones the fixed internal Nexus repository.
6. ↓
7. Merge path

   Incremental documentation sync

   1. Verify the requested commit belongs to canonical Nexus main.
   2. Diff from the last reviewed sync state and filter to documentation paths.
   3. Exit without an MR for a code-only or no-op change.
   4. Update mapped pages and propose placement for new pages.
   Release path

   Frozen version snapshot

   1. Require a safe semantic version and resolve the tag from Nexus.
   2. Verify the tag commit belongs to canonical main.
   3. Fail on broken relative links or missing assets.
   4. Update versions/manifest.json.
8. ↓
9. Push gateDid content actually change?No change: exit. Otherwise: push (or refresh) the sync branch and open an MR.
10. ↓
11. Human gateThe Nexus author and a docs maintainer both review the MRBoth approve, revise, or reject every proposed documentation change.
12. ↓
13. PublicationMerging to main runs validation, build, and deploymentThe sync jobs never publish directly.

The receiver applies these controls before publication:

Control	Enforcement point	Result when the check fails
Canonical sender	Nexus docs-sync-dispatch rules	Only the canonical internal Nexus project can dispatch; the public mirror is skipped
Pipeline source	.gitlab-ci.yml workflow and job rules	Sync jobs run only for downstream pipelines created with GitLab’s native trigger: keyword
Trigger routing	.gitlab-ci.yml job rules	Only merge and release values select a sync job
Fixed repository	clone_nexus()	Trigger variables cannot change the internal repository, host, or default branch
Clone scope	clone_nexus()	Clones only main (--single-branch) with full history, not other branches — full history is required so ancestry checks and catch-up diffs can walk arbitrarily far back
Commit verification	clone_nexus()	The supplied full commit SHA must exist and belong to canonical Nexus main
Catch-up state	state.json and merge.sh	Each docs-bearing sync starts at the last reviewed source commit; rewritten history fails closed
Documentation scope	is_site_docs_path() for live pages; is_docs_path() for snapshots	Live imports exclude assets and selected repository-only or curated sources; changes outside the selected scope create no sync merge request
Release checks	release.sh	The semantic-version tag must resolve to the verified commit; broken relative links or assets fail snapshot creation
Human review	GitLab merge request protections	Sync automation cannot publish directly; publication starts only after the review merge request reaches main

Pipeline Sequence

When a developer merges to Nexus main:

1. Nexus Pipeline — runs tests/builds; final job docs-sync-dispatch triggers this repo
2. Nexus-Docs Triggered Pipeline — runs docs-sync-merge only (build/deploy are skipped in triggered pipelines). The job pushes (or refreshes) the sync/nexus-main branch and opens a review MR
3. Human review — the Nexus author and a docs maintainer both review the MR; placement suggestions for new pages are part of the diff
4. Merge — merging the MR lands on main, which runs the normal build → deploy pipeline and publishes the site

The Path Map: Who Owns What

The heart of the sync is scripts/docs-sync/path-map.tsv, a tab-separated mapping of Nexus repo paths to curated site pages:

docs/architecture/data-model.md  concepts/data-models.md
docs/runbooks/audit-out-of-band.md  security/audit-archive-export.md
docs/dev/ci.md  contributing/ci-cd-pipeline.md
CHANGELOG.md  reference/changelog.md

This map determines ownership:

File type	Owner	Sync behavior
Mapped pages (in path-map.tsv)	Nexus	Body replaced on every Nexus edit; curated frontmatter (title/description) preserved; no sidebar change
Docs-side pages (not in the map, e.g. this page)	This repo	Never touched by the sync — no Nexus change can modify or delete them
Holding-area files (sync-pending/)	Nexus	Unpublished copies awaiting a placement decision; deleted if deleted in Nexus

Mapped pages are Nexus-owned

If you hand-edit a page listed in path-map.tsv, the next sync merge request for that file will overwrite your body edits. Frontmatter survives. Either make the edit in Nexus or follow Reclaiming a page from Nexus ownership to make it a curated docs-side page.

Sidebar Order Is Never Reshuffled

The sidebar is manually curated in astro.config.mjs. The sync never reorders existing entries:

• Edits to mapped files touch only the page body — zero sidebar changes
• New pages get a sidebar entry appended to the end of the suggested section (or nested group); you can move it anywhere in the MR
• Deletions remove only that page’s entry, leaving everything else in place

Sync Behavior

Changed Files (Mapped)

When docs/architecture/data-model.md changes in Nexus:

1. The sync finds its mapping → concepts/data-models.md
2. The page’s title and description are preserved; lastReviewed is refreshed and a missing owner defaults to Engineering
3. The body is replaced with the Nexus content (the duplicate # Heading is stripped)
4. The change appears in the MR under “Updated in place (existing sidebar order unchanged)”

New Files (Unmapped) — Auto-Suggested Placement

When a brand-new doc appears in Nexus, the sync applies a placement suggestion directly in the MR diff:

1. Section is inferred from the Nexus path:

   Nexus path	Suggested destination	Sidebar group
   docs/decisions/*	concepts/	Architecture Decisions (ADRs)
   docs/architecture/*	concepts/	Architecture
   docs/reference/*	reference/	Technical reference
   docs/security/*	security/	Security and compliance
   docs/runbooks/*, docs/ops/*	operations/	Configure and operate
   docs/dev/integrations/*	contributing/	Connect applications
   docs/dev/*	contributing/	Contributing
   docs/perf/*	contributing/	Performance baselines
   docs/process/*, docs/community/*, docs/legal/*	contributing/	Community and maintenance
   docs/guides/*, docs/getting-started/*	getting-started/	Run and deploy Nexus

2. The page is created there with proper frontmatter (title from the first heading; ADR numeric prefixes like 0022- are dropped from the filename to match site conventions)

3. A sidebar entry is appended to the end of that section/group in astro.config.mjs

4. The new mapping is recorded in path-map.tsv so future edits update the page in place

All of this is in the MR diff. As a reviewer you can:

• Approve — merge as-is
• Adjust — move the file, reposition the sidebar entry, and update the path-map.tsv line to match; future syncs follow your decision
• Reject: Delete the page, its sidebar entry, and its path-map.tsv line, then exclude the source in is_site_docs_path() to prevent reimport.

New Files With No Good Suggestion — Holding Area

Files the sync cannot place confidently, such as an unmapped service README or a filename that would collide with an existing page, are parked under sync-pending/<original path>. This directory is outside the published content collection, search index, and sitemap. The merge request flags them under New pages needing a placement decision. To publish one, move it into src/content/docs/, add its sidebar entry, and record its source and destination in scripts/docs-sync/path-map.tsv.

Live-page sync accepts Markdown files only. Repository entry points, agent instructions, and selected duplicate or package-local READMEs are excluded by is_site_docs_path() in scripts/docs-sync/lib.sh. Release snapshots retain their separate documentation-and-assets selection.

After placement, scripts/docs-sync/finalize.py translates repository-relative links to mapped site URLs. References without a site page point to the public source repository. Code examples are preserved. scripts/docs-sync/link-map.tsv records additional links to curated pages without making those pages body-sync targets. The same finalization step checks duplicate routes and sidebar destinations and updates public/llms.txt for added or deleted pages.

Deletions

When a mapped file is deleted in Nexus, the sync deletes the site page and removes its sidebar entry. Deletions of unmapped Nexus files with no site counterpart are skipped with a log line. Docs-side pages can never be deleted by the sync.

Renames

Git rename detection turns a rename into delete-old + add-new. The old page and sidebar entry are removed; the new path goes through new-file placement. No automatic redirect is created — add one manually if the old URL matters.

Versioned Snapshots (Release Tags)

When a protected v* tag is pushed to Nexus:

1. The complete doc set at the tagged commit is written to versions/<tag>/
2. versions/manifest.json is updated
3. All relative links are validated — broken links fail the job before any MR is opened (submodule and out-of-snapshot links are warnings)
4. A review MR is opened on release/docs-<tag>

Latest/current docs are unaffected; they’re owned by the merge path.

Document Metadata

All documentation pages include metadata in their frontmatter to track ownership and review status:

---
title: Page Title
description: Page description
owner: Product
lastReviewed: 2026-08-05
---

Metadata Fields

Field	Purpose	Values
owner	Team responsible for maintaining this documentation	Product, Engineering, Design
lastReviewed	Date this page was last reviewed for accuracy	ISO date format (YYYY-MM-DD)

Updating Metadata

When syncing from Nexus:

• New pages created by the sync automatically get owner: Engineering and lastReviewed set to today’s date (UTC)
• Existing mapped pages have their lastReviewed date automatically updated to today’s date (UTC) when content changes
• If an existing page doesn’t have an owner field, it’s automatically set to Engineering

For docs-side pages (not in path-map.tsv):

• Owner remains Product (or whatever the page specifies)
• Update lastReviewed manually whenever you make substantial content changes

Display:

• Metadata appears at the top of each page (below the description)
• Format: “Owner: Product Last reviewed: August 5, 2026” (formatted with the page viewer’s local timezone)
• The date is stored in UTC in frontmatter but displayed in the user’s browser timezone

Setup & Configuration

Credentials and review policy live in this repository. The Nexus dispatch job targets this project directly. Using the job-token authentication fallback also requires inbound job-token access configured on Nexus.

Required Nexus-Side Change

Nexus’s .gitlab-ci.yml docs-sync-dispatch job must:

• Hardcode trigger: project: vortexaq/vortexai/nexus-docs (not read a DOCS_SYNC_PROJECT_PATH variable — this project no longer accepts a caller-supplied path anyway)
• Restrict its rules to the canonical internal host and project so the public mirror cannot dispatch a sync.
• Forward TRIGGER_SOURCE (merge or release), NEXUS_COMMIT_SHA, RELEASE_TAG, and NEXUS_ACTOR (the sender’s $GITLAB_USER_LOGIN).

This change already ships on the Nexus side as of its ci(docs-sync): target nexus-docs directly and ci(docs-sync): restore NEXUS_ACTOR for reviewer assignment commits.

Required CI/CD Variables

Set in Settings → CI/CD → Variables. Mask credentials and protect them for the protected docs main branch where the receiver runs.

Variable	Scope needed	Description
NEXUS_GIT_HOST	n/a	Git host of the canonical Nexus project. Kept out of tracked source and out of MR/doc output since this repository is public; set once, mark Protected, and restrict “Minimum role to use pipeline variables” to Maintainer+ so a manually-triggered pipeline can’t override it
DOCS_SYNC_BOT_TOKEN	api	Project access token from this project (Settings → Access tokens, role Developer+). Pushes sync branches, opens MRs, resolves reviewer usernames
NEXUS_REPO_READ_TOKEN	read_repository	Optional when job-token access is configured. Otherwise, supply a token whose owner can read the Nexus repository.

Clone authentication

Git-over-HTTPS uses HTTP Basic authentication. The script uses oauth2 with NEXUS_REPO_READ_TOKEN, or gitlab-ci-token with the automatically supplied CI_JOB_TOKEN when the read token is unset. The fallback requires this docs project on Nexus’s inbound job-token allowlist and Nexus read access for the triggering user. GitLab’s API token headers do not authenticate the clone.

Recommended Variables

Variable	Purpose
DOCS_MAINTAINER	Comma-separated GitLab usernames always requested as MR reviewers, alongside the Nexus author who made the change; enforce the two-approval requirement itself in project settings

Permission Requirements

• Nexus mergers need Developer+ membership in this project: the native trigger: keyword runs with the triggering user’s permissions. Manage this in this repo’s Settings → Members.
• The DOCS_SYNC_BOT_TOKEN bot user only needs to create branches and MRs (no push-to-main rights required — everything goes through MRs)
• Protect main, disallow direct bot pushes, and require a successful pipeline before merge. Configure the approval rule to require sign-off from both the Nexus author (NEXUS_ACTOR, requested automatically) and a docs maintainer — requesting reviewers alone does not enforce that; the project’s approval rule does.

Testing & Debugging

Reading a Sync Job Log

1. Merge a doc change in Nexus or retry the Nexus dispatch job as described below.
2. Check the docs-sync-merge job log:

   docs-sync trigger received:
     TRIGGER_SOURCE=merge
     NEXUS_COMMIT_SHA=abc1234...
   cloning canonical vortexaq/vortexai/nexus (main only) at abc1234
   diffing verified range def5678..abc1234
   documentation changes:
   M  docs/decisions/backlog.md
   A  docs/decisions/0022-registry-state-from-signed-artifacts.md
   pushed sync/nexus-main
   docs sync merge request opened: <url>

3. Review the opened (or refreshed) sync/nexus-main merge request; nothing publishes until it’s merged

Rerunning a sync

To pick up a fix merged into docs main:

1. Open the Nexus repository’s pipeline for the source commit.
2. Retry docs-sync-dispatch in the sync stage. This creates a fresh downstream docs pipeline.
3. Open the new downstream pipeline and confirm its docs commit contains the fix.

The retry button on an existing downstream docs pipeline reruns its original checkout. It does not pick up a newer docs commit. Run pipeline in the docs project also does not select the sync jobs: they require CI_PIPELINE_SOURCE=pipeline from the cross-project trigger.

If a pending sync/nexus-main branch already records the same Nexus commit, the script leaves that branch unchanged. Review or edit that existing branch rather than expecting a retry to regenerate it. A newer source commit may refresh and force-push the branch from docs main; merge placement edits before dispatching another sync to preserve them.

Troubleshooting

Symptom	Check
Trigger never arrives	Does the Nexus merger have Developer+ in this project? Is the hardened Nexus docs-sync-dispatch job merged and green?
Clone of Nexus fails	Check token availability, expiry, scope, and Nexus read access. For the job-token fallback, check the Nexus inbound allowlist and triggering user’s access.
Commit verification fails	Confirm the full SHA exists on the internal Nexus main; public-mirror and feature-branch commits are intentionally rejected
State ancestry fails	Internal Nexus history was rewritten or state.json was edited incorrectly; reconcile manually rather than bypassing the check
MR creation fails (401/403)	Does DOCS_SYNC_BOT_TOKEN have api scope? Is it expired?
Reviewer not assigned	Username must exist and be a member here; fallback actor strings like Jane Doe <jane@…> are skipped by design (check job log)
Sync MR has odd placement	Adjust in the MR: move file + sidebar entry + fix the path-map.tsv line
Same merge request keeps updating	Newer source commits refresh sync/nexus-main; an already-represented commit exits without refreshing it.

Maintaining the Automation

Updating Path Patterns

The shared documentation selection lives in scripts/docs-sync/lib.sh. Live pages additionally pass through is_site_docs_path(), which limits imports to Markdown and excludes repository-only or duplicate pages:

DOCS_INCLUDE_PATTERNS='
docs/*
*.md
'

DOCS_EXCLUDE_PATTERNS='
__snapshots__/*
*/__snapshots__/*
*.test.*
node_modules/*
*/node_modules/*
.gitlab/*
'

Updating Placement Suggestions

Section inference lives in suggest_section() and sidebar_group_for_nexus_path() in scripts/docs-sync/lib.sh. Edit these as the Nexus docs layout or the site’s sidebar structure evolves.

The sidebar separates Product documentation (console workflows, team access, reporting, and administration guidance) from Developer documentation (integration, deployment, operations, architecture, security, contribution, and reference). Automatic source placement targets developer groups. Product guides are curated here, with contextual links to technical instructions. File paths do not determine audience: client integration pages retain their contributing/ URLs but appear under Connect applications.

Reclaiming a Page from Nexus Ownership

Delete its line from scripts/docs-sync/path-map.tsv and add the source to the exclusions in is_site_docs_path(). This keeps future edits from recreating a second copy. A link-map.tsv entry can still direct links from other synced pages to the curated page.

Testing, Wire Contracts, and the Claude Code session checklist are curated this way. Changes to their Nexus source documents require manual reconciliation so that reader-facing structure and instructions stay current without losing the site’s formatting.

Synchronization State

scripts/docs-sync/state.json records the latest internal Nexus commit represented by a merged sync MR. A code-only Nexus merge creates no MR; the next documentation-bearing merge catches up from the recorded commit. The first run without state performs a full reconciliation.

Disabling Sync Temporarily

Set each receiver job’s rules to - when: never in .gitlab-ci.yml. Removing its rules entirely does not disable a job.

Security Notes

• Fixed source and ancestry: the receiver always clones the internal Nexus repository and rejects commits outside canonical main. Trigger variables cannot redirect the read token to another repository.
• Internal host not published: this site is public, so the Nexus git host lives only in a protected CI/CD variable, never in tracked source or in generated MR/doc output (commits and tags are referenced by id, not by link).
• External contributions: the public GitLab mirror never dispatches syncs. Its contributions become eligible only after internal review and merge to canonical Nexus main.
• Two-person sign-off: nothing reaches main without approval from both the Nexus author who made the change (NEXUS_ACTOR, forwarded by the sender and requested as a reviewer) and a docs maintainer (DOCS_MAINTAINER). This is enforced by the nexus-docs approval and protected-branch rules configured by a maintainer — requesting reviewers is routing, not enforcement.
• Every sync MR carries the verified source commit, previous synchronized commit, and the Nexus author.
• The bot token only needs branch/MR rights — it cannot push to protected main
• Release snapshots are link-validated before an MR is opened

Related Documentation

Docs Sync Quick Reference

The cheat sheet version of this page: commands, paths, and review steps at a glance.

Open the cheat sheet →

CI/CD Pipeline

The Nexus source repository’s build checks, security gates, and release flow.

See the pipeline →