Usage-review measurement

Run from the Nexus checkout with the local Compose stack already running:

export NEXUS_USAGE_COOKIE_FILE=/absolute/path/to/console-cookies.txt
./scripts/measure-usage-review.sh --scope workspace --window 7d
./scripts/measure-usage-review.sh \
  --seed /tmp/nexus-stress-usage/seed.json \
  --scope all --window all \
  --as-of 2026-10-01T12:00:00Z --timezone America/New_York \
  --cookie-file "$NEXUS_USAGE_COOKIE_FILE" \
  --base-url http://localhost:14449 \
  --warmups 1 --repeats 3 --out-dir /tmp/nexus-usage-review

Dependencies

• Python 3.9+ with IANA timezone data; Python dependencies are standard-library only.
• Docker Compose and the existing clickhouse service with clickhouse-client, using the local Compose vortex database/user credentials.
• A completed seed.json with workspace_id, inventory, and the owned org IDs produced by stress-workspace-usage.py.
• A Netscape cookie export from a logged-in console session with access to the seed workspace and organizations. COOKIE_FILE is also accepted as an environment default. Cookies must match the base origin; redirects are rejected. Cookie values are never printed or retained in the report.
• system.query_log enabled for all query-service HTTP statements, with terminal records, full query text, and no sampling or duration filter. Inspection uses SELECT, EXPLAIN indexes = 1, and bounded SYSTEM FLUSH LOGS; it does not change global settings or write seed data.

Keep other analytics clients idle during capture. The harness checks HTTP queries already in flight and rejects foreign or duplicate statements in the capture interval. Background HTTP queries, including gateway spend-cap snapshots, can therefore make a run fail correlation. The two interval boundaries come from ClickHouse’s microsecond clock, not the host clock. Query-log polling defaults to five flush/read attempts spaced by 0.5 seconds; tune --log-polls and --poll-interval if logging is delayed. Inspection commands have a 30-second client deadline and bounded server resource settings.

--as-of defaults to the current instant, frozen once for the run. All presets resolve to inclusive console local-day bounds ending in .999, including DST changes and the previous calendar quarter. all selects the five presets, not all stored history. The BFF’s existing validation still applies: an old frozen MTD anchor can be rejected by the server. Workspace measurements request the summary plus the first 50 organization rows (six statements); org/project measurements use the four dashboard panels and a separate current-month MTD request. Org/project select the first numbered seed organization and its default project. --overview-diagnostic adds the separate overview BFF call for those scopes and includes it in their measured totals.

Evidence and interpretation

usage-scale.json retains warmups, measured repeats, each HTTP status/payload size/actual wall time, raw query-log events, deduplicated terminal statements, exceptions, correlation failures, and source/host metadata. usage-scale.md is the compact measured grid. Exit status is nonzero for failed or incomplete measurements. Authentication, ambiguous/incomplete logs, active server queries, and inspection failures stop the run; terminal unambiguous query failures can proceed to the next cell. A failure during warmup is retained and never presented as a successful measured cell.

Per HTTP call, memory is the maximum statement memory, read bytes and statement durations are sums, and wall time is measured around the HTTP exchange. The cell’s memory is the maximum across measured repeats; read bytes and payload bytes sum those repeats; wall time is the median of each repeat’s serial HTTP wall-time sum. Inspection and polling are excluded from HTTP wall time. Normalized per-request/per-token ratios are descriptive, not memory prediction coefficients.

Physical snapshots cover all seven tables across the database: active parts, levels, rows, disk/compressed/uncompressed bytes, per-row storage ratios, and active merges/mutations, before and after the grid. Per-cell scoped censuses distinguish physical rows from logical keys and occupied rollup buckets. Raw request tokens include prompt, completion, and image tokens, without FINAL. Monthly logical rows use the current calendar month; raw and rollup window statistics use request creation time. Replay counts and EXPLAIN evidence describe the raw discovery scans even when replay counts are zero. Census reads themselves affect cache warmth and have their own bounded inspection settings.

The 1B/10B/100B tokens/day scenarios each describe 30 days. They retain an explicit observed topology, a fixed-dimension occupancy bound, and a replicated-topology growth bound, with physical-state references and separate rollup/replay, raw overview, monthly, and spend-cap FINAL families. They use a successful nonempty workspace 30-day cell if available, otherwise the last successful nonempty workspace cell. No numerical memory bound is claimed from a single topology/merge-state observation: unsupported values are null, with reasons. Raw scan row scenarios assume the measured tokens/request; they do not imply token-linear aggregate memory. Spend-cap FINAL is a separate global UTC-month gRPC query and is explicitly unmeasured by this BFF harness.

Offline checks

bash -n scripts/measure-usage-review.sh
shellcheck scripts/measure-usage-review.sh
python3 -m py_compile scripts/measure_usage_review.py scripts/test_measure_usage_review.py
python3 -m unittest discover -s scripts -p 'test_measure_usage_review.py' -v
./scripts/measure-usage-review.sh --help

Tests mock HTTP, subprocess, and ClickHouse interactions; they do not require or mutate a running database.