With #20161 the Release-Green HARD items run with OMNIROUTE_GATE_MODE=pr and the strict comparison moves to the release:refresh-derived --check drift item, so the skip text now says: the release PR, pushes to main, local runs.
Contract with the phase-4/5 session (2026-11-14):
- ui-keys: syncUiKeys is called with pruneExtra (keys en.json dropped leave the other catalogs)
and rewrittenSinceRef (English values rewritten since the last release go back to translation;
--rewritten-since=<ref>, default origin/main, skipped when unresolvable). The backend translates
only when OMNIROUTE_TRANSLATION_* is configured, never in --check. The CLI fallback passes
--rewritten-since when the script supports it.
- e2e-timings: new step that runs syncE2eTimings (phase 5) when the tree ships it; skipped
otherwise.
- merge-train.sh: i18n:check-keys, i18n:check-keys:cli and i18n:check-ratio run in pr mode — the
train judges the catalogs its PRs changed against the merge base, and a key that exists only in
en.json is completed by the release refresh instead of reddening the train.
The .env loader ran at module load, so release:refresh-derived and the tests that import syncUiKeys() got the operator's .env (DATA_DIR, OMNIROUTE_TRANSLATION_*, ...) in their own process. loadDotEnv() is now exported and called by the CLI and by syncUiKeys() only on the translate path, before backendConfig(); the translate concurrency default is read after it, so a value in the .env still applies there.
Owner decision 2026-11-14: check:pr-derived-paths starts with --warn and, in GitHub Actions, adds one ::warning annotation per violation so the advisory result is visible on the PR. A one-line PR drops --warn after the one-time cleanup of the open PRs (plan section 8).
i18n-mirror-refresh.yml runs check:docs-all with OMNIROUTE_GATE_MODE="pr", byte-identical to the same line in feat/derived-p1-gate-modes (#20161), so the two branches merge without a conflict.
The locale-parity helpers (localeParityOptions, localeParityIsReleaseOnly, placeholderParityDeferred) and the mirror-freshness helper (announceMirrorPlan) skip in a feature PR what the release refresh rewrites. countSkips() cannot see them, so the gate now counts each helper (imports excluded) and names it with its effect in the report-only block. The helpers' own unit tests are left out of the list. Report-only, as approved by the owner on 2026-11-14.
check-test-masking.test.ts quotes releaseOnlyUnlessTouched( as fixture text, so the new
report-only block listed the gate's own test. It now skips the self-test family, the same
exclusion the tautology counters already apply; covered by a new test.
(cherry picked from commit e78c2de871)
Review finding: the README SVG / e2e-timings tests moved from t.skip() to
{ skip: releaseOnlyUnlessTouched(...) }, which countSkips() does not see (it matches only
.skip( and { skip: true }), so check:test-masking answered a plain OK for 13 assertions that
a PR run can skip.
countModeConditionalSkips() counts releaseOnlyUnlessTouched( calls per changed/renamed test
file; a net increase is printed on its own report-only block naming each file and where the
edited-artifact comparison runs in a PR (check:generated-artifacts). It does not fail the
gate: whether a mode-conditional skip should block is the owner's decision. countSkips()
and every existing signal are unchanged.
(cherry picked from commit c20f9d67cb)
check:docs-counts drops AGENTS.md, llm.txt, the services guide and the skills from the derived
count checks (README.md and the docs keep them) and adds one STRICT invariant per
agent-instruction file — AGENTS.md, llm.txt, the bs fork (Bosnian patterns), the services guide,
agentSkills.ts, skills/README.md and every skills/*/SKILL.md — that fails any provider,
migration, MCP-tool, scope, strategy, scoring-factor or mode-pack count there, in both gate modes.
It reuses the gate's own claim patterns, now named constants.
Tests: #15159 DOC-01 now proves the instruction files point to the strategy registry and carry
no count or inline list (the AUTO-COMBO.md table is still checked against the code); the skill
provider-count test proves descriptions and generated skills are count-free and that the
invariant rejects a count; the provider-count coverage test checks the skill surfaces through
the invariant. QUALITY_GATES.md documents it; changelog fragment.
The npm description carried the live provider total, so every provider PR
rewrote the same line and 25 open PRs conflicted on it alone. Use a
count-free wording (also in flake.nix, which had drifted to 160+) and turn
the docs-counts check into a STRICT guard that rejects an exact provider
count in the description instead of requiring the live one.
Adversarial review of this branch, each finding reproduced before the fix:
- Coverage floor in pr mode: the locator read any four slash-separated numbers, so editing the
provider breaker's HTTP status list in AGENTS.md (`408/500/502/503/504`) failed as "edited a
managed value". The diff and the rewrite plan now use only tuples that can be percentages;
the release validator is unchanged.
- Merge train: its detached worktree cannot tell a writer branch (the nightly refresh PR, a
translation wave) from a feature, so a train boarding one went red on legitimate rewrites. The
gate honours OMNIROUTE_GATE_BATCH=1 (skip the managed-value diff, like merge_group) and the
train declares itself a batch.
- workflow_dispatch on a feature branch ran the docs jobs in strict release mode, red by design:
those steps now set OMNIROUTE_GATE_MODE=pr when the dispatch is not on main or release/v*.
- i18n-mirror-refresh (schedule) ran check:docs-all in release mode and stopped opening the
mirror PR whenever the base was behind the code: its docs check runs in pr mode (same line as
the phase-4 PR, coordinated with that session).
- Nightly refresh: a run with no diff now closes the robot's stale refresh PR (a hand-pushed
tip is only reported).
- Refresh: the plan lists the README mirrors' database row before README.md, so mirrors were
rewritten first, the relay found nothing to carry and 66 locales turned into drifted targets.
Sources now go first, a mirror the plan rewrote counts its rewrites as carried (local digits
included), and a mirror is certified in sync only when every change reached it; a locale that
was already stale keeps its source hash.
- The snapshot test judged release-only checks (provider reference vs live modules) against the
snapshot, which a provider PR that regenerates PROVIDER_REFERENCE.md would break; it now judges
the derived and invariant checks only, with a floor on how many it judges.
check-test-masking.test.ts quotes releaseOnlyUnlessTouched( as fixture text, so the new
report-only block listed the gate's own test. It now skips the self-test family, the same
exclusion the tautology counters already apply; covered by a new test.
The contribution golden path still told feature PRs to keep the en/pt-BR
CLI catalogs aligned and put i18n:sync-ui:dry and a release-mode value-drift
run in the local loop, which reports every key the cycle added and pushes
the author to edit locales. The feature PR template asked for strings in
"the i18n catalogs", and the translate-new-keys.sh header said a branch that
adds keys must translate them before its PR opens.
Now: a feature PR edits en.json only and previews CI with --mode=pr; the
template asks for en.json only; translate-new-keys.sh names its callers
(release refresh, translation waves). I18N.md and QUALITY_GATES.md document
the changed-catalog judgment of pr mode, the placeholder-rename case (prefer
a new key) and localeParityMode.ts as the only sanctioned mode-conditional
skip; the ci.yml i18n UI Coverage comments and step names say the same.
Review finding: the README SVG / e2e-timings tests moved from t.skip() to
{ skip: releaseOnlyUnlessTouched(...) }, which countSkips() does not see (it matches only
.skip( and { skip: true }), so check:test-masking answered a plain OK for 13 assertions that
a PR run can skip.
countModeConditionalSkips() counts releaseOnlyUnlessTouched( calls per changed/renamed test
file; a net increase is printed on its own report-only block naming each file and where the
edited-artifact comparison runs in a PR (check:generated-artifacts). It does not fail the
gate: whether a mode-conditional skip should block is the owner's decision. countSkips()
and every existing signal are unchanged.
Review finding: in PR CI the README SVG byte comparison and the e2e-timings completeness
never ran, not even for an SVG the PR edited. test-unit checks out a shallow merge ref with
no base, so resolvePrDiffBase() returns null and releaseOnlyUnlessTouched() skipped
(reason=base-unresolved). A PR allowed by the generated family's companions (generator +
regenerated SVG) passed with a wrong or hand-edited SVG.
- scripts/check/check-generated-artifacts.mjs (npm run check:generated-artifacts): the 12
README diagrams plus the e2e timings seed. release = every artifact strict (what the unit
tests check); pr = only the artifacts the change edits; pr without a diff base = every
artifact strict (fail-closed). Meant for a job with fetch-depth: 0 (the lint job, next to
check:agent-skills-sync); wiring it there is requested from the job's owner.
- resolveDerivedScope() gains onUnresolved: "skip" (default, unit tests) | "strict".
- The unit tests' base-unresolved skip reason now names the check that owns the comparison.
- gate-manifest: new alias, separately-invoked.
Two gaps of the programmatic API the release refresh calls:
- a locale code outside the set it would process (typo, missing from
config/i18n.json, no catalog file, or en) was dropped silently and the
run reported nothing to do; it now throws before writing anything;
- a dry run left out the src/i18n/global-error-messages/*.json files a real
run writes (2 listed vs 5 written in the review fixture). The compact
catalogs are now rendered in memory (renderGlobalErrorCatalog, extracted
from generate-global-error-messages.mjs, whose output is unchanged), so
dry and real runs report the same changedFiles.
- With --json, the pr-mode SKIP / scope lines go to stderr and a skip prints a JSON object
({skipped, mode, reason, detail}) on stdout, so --json output is always JSON again.
- classifyTouchedSkills judged any non-SKILL.md file under skills/<id>/ as generated. The
generator writes only SKILL.md (outside the custom block) and references/**; a PR that adds
an example or a note beside SKILL.md is no longer judged against the generator.
Both are covered by new tests that fail on the previous commit.
syncE2eTimings rewrote the _meta note (its \u2014 escape became a raw em dash) and
counted a new spec's LOC as split('\n').length, one more than the seed's wc -l. It now
escapes non-ASCII like the committed file and counts newlines; the existing test's
expected LOC goes from 4 to 3 for a three-line fixture because 3 is the seed's value
(the old 4 encoded the off-by-one). Two new tests pin the byte-for-byte behaviour.
- ci.yml docs-sync-strict and quality.yml docs-gates ("Docs accuracy"): checkout with
fetch-depth 2 and BASE_SHA (PR or merge-group base) on `npm run check:docs-all`, so the
counts gate runs in pr mode on feature PRs against the base the merge commit sits on.
- Release-Green: every HARD item runs with OMNIROUTE_GATE_MODE=pr unless the environment
already sets one (they answer "did the base break?"); a new DRIFT item `derived-refresh`
runs `npm run release:refresh-derived -- --check`, the strict view, which reports without
opening the base-red issue. Between releases the base is behind the live code by design.
- nightly-derived-refresh.yml: once a day on the highest release/vX.Y.Z (skipped under
release-freeze), runs the refresh with --profile=nightly and keeps ONE PR open from the
robot branch chore/derived-refresh (same lease-pinned rebuild as the UI Kit watch). Never
merges; flags agent-instruction surfaces for the owner. Needs the DERIVED_REFRESH_PR_TOKEN
secret to open the PR; without it only the branch is pushed.
- merge-train.sh: the counts gate judges the train in pr mode against the train base.
The pr-mode advisory reused the release instruction (run quality:ratchet --update and
commit the tightened baseline in this PR), contradicting the line above it. The
measurement and the instruction are now separate: release mode prints exactly the old
line, pr mode says the improvement is banked by the release refresh / nightly.
The pr-mode SKIP of the completeness gates also covered the PRs that still
edit locale catalogs: translation waves (docs/i18n-*, chore/i18n-*), the
release refresh (chore/derived-*) and translation-only PRs, which
check:pr-derived-paths allows. A wave that deleted a key from ko.json or
overwrote a translation with __MISSING__ failed before and passed after.
check-key-completeness (UI and --catalog=cli) now SKIPs in pr mode only when
the PR changed no locale catalog of that family. Otherwise each changed
catalog is judged against itself at the merge base
(scripts/i18n/lib/touched-catalogs.mjs): no key it had may be lost, no key
en.json lacks may appear, no real translation may become a __MISSING__
marker or the English text (outside untranslatable-keys.json), and no value
the PR changed may break the placeholders/tags of an English value the PR
did not touch (ICU scanner in scripts/i18n/lib/icu-names.mjs, same grammar
as the placeholder-parity test). A catalog new in the PR must carry the
whole en.json key set. Release mode is unchanged.
The ratio and CLI-parity SKIP lines now say where a changed catalog is
judged. The support list of i18n-release-mode-gates.test.ts gains the two
new libs, which the copied check-key-completeness imports.
Every number the counts gate checks inside the README SVGs (providers, MCP tools, routing
strategies, pools, free-forever providers, catalog entries, CLI integrations, scoring factors,
the free-tier budget card) comes from docs/reference/derived-facts.json through
scripts/docs/readmeFacts.mjs instead of a literal kept by hand in each generator. Curated copy
(150+, 15-95%, $0) stays literal. Each generator still produces the committed SVG byte for byte.
One entry point recomputes every derived value, in order: provider reference, facts snapshot,
managed numbers (planDerivedRewrites, relayed into the docs/i18n mirrors with the
bump-version-claims rule for .i18n-state.json), README diagrams, generated skills, UI key
catalogs and the llm.txt mirrors (the phase-4 sync functions when exported, the current CLIs
otherwise). Changelog aggregation is opt-in (--with-changelog, release profile only): the
release keeps `npm run release:reconcile` as the canonical path.
--check writes nothing (every step runs on an in-memory overlay), exits 1 when a step would
change a file and lists the rewrites that need a human (manual), exit 2 on an infrastructure
error. --profile=nightly never touches the changelog or version claims. Every subprocess gets a
throwaway DATA_DIR. On this tree the full --check reports zero changes and zero manual items.
The counts gate now has two modes, resolved by scripts/check/lib/gateMode.mjs
(--mode, OMNIROUTE_GATE_MODE, then the Actions event):
- release (release PR, release skill, local default): strict equality against the live code,
exactly as before. On this tree the report is identical to the previous gate except for the
mode header and two new STRICT sections (routing-strategy counts in prose), all green.
- pr (feature PRs, merge_group, omni-queue): no tsx, no facts. Every derived check exposes an
extractor with the span of each value; the gate diffs the PR's changed files against the
merge base and fails only what the PR itself did: writing a managed value the base did not
carry, or dropping a required claim. It never fails because the base moved. Content
invariants that need no facts run in both modes; fact-only checks (provider reference vs
live modules, mode-pack names, manifest free tier) are release-only. Writer branches
(chore/derived-*, release/v*, docs/i18n-*, chore/i18n-*) may edit managed values.
- Exit codes: 0 ok, 1 drift or a PR violation, 2 infrastructure (facts not collected, or the
PR diff base cannot be resolved in CI), with a message saying it is not documentation drift.
- planDerivedRewrites({facts, readFile}) returns the exact rewrites that bring every
rewritable STRICT claim to the facts, and lists the rest as manual. It is what
`npm run release:refresh-derived` applies.
- scripts/release/facts.mjs: collectFacts/readCodeFactsResult classify a failed probe
(timeout, spawn, exit, parse) as an infrastructure error, never as documentation drift.
The 180 s hard timeout that turned a loaded host into a false red is now
OMNIROUTE_FACTS_TIMEOUT_MS (default 600 s). readCodeFacts() and the test seams keep
their old shape.
- The inline `node -e` probe becomes a real module (scripts/release/lib/code-facts-probe.mjs),
run under tsx in a subprocess. It forces a throwaway DATA_DIR before any import, so loading
the MCP tool modules can never open or migrate the operator's SQLite file.
- `npm run facts [-- --key=a.b] [--snapshot]` prints every derived number as JSON
(exit 2 on an infrastructure error).
- docs/reference/derived-facts.json: the facts as of the last refresh, serialized in a stable,
Prettier-identical layout. Generated artifacts and unit tests are judged against it; only
the release refresh and the nightly refresh PR write it.
tap.testFiles was a hand-kept list of 781 entries (775 unique) and 46 open
PRs touched it. stryker.conf.json becomes stryker.conf.mjs (Stryker 10
auto-discovers it), which computes the list at load time: every unit test
that imports a module in `mutate`, the same rule check:mutation-test-coverage
--strict enforces, now shared in scripts/quality/mutation-test-files.mjs.
The 109 former entries the import rule does not select (barrel/indirect
coverage) stay pinned, so the resolved list equals the old set exactly.
All other options keep their values.
The drift gate and mutation-radiography load the config the way Stryker
does (dynamic import of the default export). stryker.disablebail.json
becomes stryker.disablebail.mjs and spreads the base config: Stryker 10
does not implement "extends", so the redundancy run had none of it.
The direct-run guard added with the syncLlmMirrors() export compared the
module URL (a real path) with argv[1] as invoked, so a symlinked checkout
turned the CLI into a silent no-op that exited 0. The old script always ran.
The guard now resolves argv[1] with realpathSync first; covered by a test.
Two review findings on the PR-mode split of check-docs-sync:
- The docs-sync-strict job checks out at depth 1, so the diff base never
resolves there and a hand edit to one docs/i18n/<locale>/llm.txt passed
as SKIP. PR mode now runs a check that needs no base: every llm.txt
mirror must exist, keep its separator and carry either the root body or
the body most mirrors share. A base that moved leaves them all equally
stale and passes; a single hand-edited mirror fails. Release mode is
unchanged (same lines and exit code as the script before the split).
- The pre-commit hook ran the script without a mode, i.e. release mode
locally. After a feature PR that edits only the root llm.txt merged,
every later commit on that base would be blocked until the wave synced
the mirrors. The hook now passes --mode=pr: a commit is judged on what
it does; mirrors it writes are still judged strictly.
check-quality-ratchet.mjs --require-tighten failed a PR whose metric improved beyond
tightenSlack unless the PR also edited quality-baseline.json, so two improving PRs
conflicted on the baseline. Banking belongs to the release refresh / nightly (plan
2026-11-13, phase 5).
In pr gate mode the un-banked improvement is printed as information and the run
passes; a regression fails exactly as before. Release mode (push, schedule, the
release PR, local runs) is byte-identical to the previous script on the 12-case matrix
(with/without _policy x improved/regressed/equal x with/without --require-tighten),
and the velocity-phase advisory (_policy.requireTighten=false) is unchanged.
The existing tests now pin OMNIROUTE_GATE_MODE=release in their spawn helper: they
describe release behaviour, and a CI pull_request run would otherwise resolve to pr
mode. No assertion was removed or relaxed; four tests cover the pr/release split.
Owner decision 2026-11-13 (plan "derived numbers and texts resolved only at release",
phase 4): a feature PR adds or rewrites keys in en.json and touches no other locale, so
the gates that compare every locale with en.json judge the base, not the change.
With scripts/check/lib/gateMode.mjs, these gates print one
"SKIP reason=pr-mode — <mode>: <why>; the strict check runs in release mode (...)" line
and exit 0 in pr mode (feature PR, merge_group, omni-queue), and run unchanged in release
mode (release PR, pushes to main, workflow_dispatch, local runs):
- check-key-completeness (UI and --catalog=cli)
- check-new-key-coverage (also --json)
- check-ui-value-drift (also --json)
- check-translation-ratio (the gate path only; --report, --json and --update still run)
- check-cli-i18n: the t()-key check runs in both modes; pt-BR sections and zh-CN/zh-TW
parity skip in pr mode.
Release mode was compared against the 01258da1 scripts on this tree and on a tree with an
en.json-only key plus a rewritten value: same stdout/stderr and exit code for every gate
(UI, CLI, new-key, value drift, ratio, cli-i18n).
The shared line lives in scripts/i18n/lib/release-mode.mjs. The new test runs each real
script in a throw-away git repo: both modes, the event-derived mode, and the release PR
(BASE_REF=main) judging every key of the cycle until syncUiKeys completes the catalogs.
Kept in both modes: check-ui-keys-coverage (65 % floor) and check:ui-icu.