mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-11-16 17:09:09 +00:00
* feat(ci): automate ratchet shrink-banking on the release branch (#8584)
The quality ratchet is only half automatic, and it is the wrong half. Raising a
cap is a manual JSON edit that takes ten seconds and is the fastest way to unblock
a red PR. Lowering one requires someone to run `--update` and commit the result —
and no workflow does: grepping `.github/workflows/` for `--update` finds only
wiki-sync.yml (unrelated) and ci.yml's check-quality-ratchet.mjs --require-tighten
(a different script against a different metric).
Measured on release/v3.8.49 at 4053e2314: 18 frozen files already at or under the
800-line new-file cap, the worst at 132x (src/shared/validation/schemas.ts, 19
lines carrying a 2,523 cap); the complexity ceiling walked 1794 -> 2169 across ~37
rebaseline notes with exactly one decrease (-1); "tighten via --update next cycle"
written 31 times and honoured once in six weeks. A cap that outlives the code that
earned it converts every completed decomposition into a growth allowance for
whoever edits that file next.
New job `bank-ratchet-shrinks` in nightly-release-green.yml measures the active
release branch, runs the two shrink-only `--update` paths, and opens ONE
always-current PR with the result. Schedule/dispatch only, deliberately not on
push: banking has no latency requirement, while a per-merge run would rebuild the
PR branch during merge campaigns and pay for a full ESLint walk each time.
Detection stays on push (release-green); only banking is batched. The job never
pushes to release/* — a human merges, so a bad measurement cannot land unreviewed.
scripts/quality/verify-ratchet-bank.mjs is the hard guarantee that the automation
can only ever write downward. It diffs the post-`--update` tree against HEAD and
aborts the job before a commit exists — opening no PR — unless every change is a
frozen/testFrozen entry lowered or removed, `count` lowered, or
cognitiveComplexity.value lowered. Raising a number, adding an entry, changing
cap/testCap, or deleting/rewriting a `_rebaseline_*` note all fail. A bot that
could raise a cap would be strictly worse than the status quo.
Verified both directions against the real baselines: --update + verifier reports
77 lowered / 14 removed / nothing raised, exit 0; hand-raising chatCore.ts to 9999
and cap to 1200 is rejected with exit 1. 22 unit tests cover each way the
automation could go wrong.
No product code changes.
* chore(skills): sync cli-backup-sync SKILL.md after rebase onto tip
This commit is contained in:
142
.github/workflows/nightly-release-green.yml
vendored
142
.github/workflows/nightly-release-green.yml
vendored
@@ -301,3 +301,145 @@ jobs:
|
||||
main-green.json
|
||||
main-green.log
|
||||
if-no-files-found: ignore
|
||||
|
||||
# ── Banking lane (#8584) ──────────────────────────────────────────────────
|
||||
# The ratchet is asymmetric: RAISING a cap is a ten-second manual JSON edit made
|
||||
# under merge pressure, LOWERING one requires someone to run `--update` and commit
|
||||
# — which no workflow does. Grep `.github/workflows/` for `--update`: only
|
||||
# wiki-sync.yml (unrelated) and ci.yml's check-quality-ratchet.mjs --require-tighten
|
||||
# (a different script, a different metric). So a cap outlives the code that earned
|
||||
# it and every completed decomposition silently becomes a growth allowance for
|
||||
# whoever touches the file next. Measured on 2026-07-25: 18 frozen files already at
|
||||
# or under the 800-line new-file cap, up to 132x (schemas.ts, 19 lines / 2523 cap),
|
||||
# and 31 unfulfilled "tighten via --update next cycle" notes honoured exactly once
|
||||
# (-1 unit) in six weeks.
|
||||
#
|
||||
# This job closes that loop by making the DOWNWARD direction as automatic as the
|
||||
# upward one is easy. It measures the active release branch, runs the shrink-only
|
||||
# `--update` paths, and opens ONE always-current PR with the result. It never
|
||||
# pushes to release/* — a human still merges, so a bad measurement cannot land
|
||||
# unreviewed. verify-ratchet-bank.mjs is the hard guarantee that the automation can
|
||||
# only ever write in the shrink direction; if anything was raised, added, or a
|
||||
# rebaseline note was touched, the job aborts and opens nothing.
|
||||
#
|
||||
# Schedule/dispatch only, deliberately NOT on push: banking has no latency
|
||||
# requirement (a shrink banked within 8h is fine) and a per-merge run would rebuild
|
||||
# the PR branch repeatedly during merge campaigns while paying for a full ESLint
|
||||
# walk each time. Detection stays on push (release-green above); banking is batched.
|
||||
bank-ratchet-shrinks:
|
||||
name: Bank ratchet shrinks
|
||||
if: ${{ github.event_name != 'push' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Resolve active release branch
|
||||
id: branch
|
||||
env:
|
||||
INPUT_BRANCH: ${{ github.event.inputs.branch }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -n "${INPUT_BRANCH:-}" ]; then
|
||||
TARGET="$INPUT_BRANCH"
|
||||
else
|
||||
TARGET=$(git for-each-ref --format='%(refname:short)' 'refs/remotes/origin/release/v*' \
|
||||
| sed 's#origin/##' \
|
||||
| sort -t/ -k2 -V \
|
||||
| tail -1)
|
||||
fi
|
||||
if [ -z "$TARGET" ]; then echo "No release/v* branch found"; exit 1; fi
|
||||
# Same strict guard as the validation job — blocks ref/command injection
|
||||
# through the workflow_dispatch input.
|
||||
if ! printf '%s' "$TARGET" | grep -qE '^release/v[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "Refusing non-canonical branch name: $TARGET"; exit 1
|
||||
fi
|
||||
echo "target=$TARGET" >> "$GITHUB_OUTPUT"
|
||||
echo "bank_branch=chore/bank-ratchet-${TARGET#release/}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Checkout the release branch
|
||||
env:
|
||||
TARGET: ${{ steps.branch.outputs.target }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git checkout "$TARGET"
|
||||
git log -1 --oneline
|
||||
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: "24"
|
||||
cache: npm
|
||||
|
||||
- uses: ./.github/actions/npm-ci-retry
|
||||
|
||||
- name: Ratchet the baselines down
|
||||
run: |
|
||||
# Both --update paths are shrink-only by construction (file-size writes only
|
||||
# on `improvements`, complexity-ratchets only when `.improved`), and both exit
|
||||
# non-zero while the branch is over baseline — which is exactly when there is
|
||||
# nothing to bank. Their exit code is not the signal; the verifier below is.
|
||||
set +e
|
||||
node scripts/check/check-file-size.mjs --update
|
||||
node scripts/check/check-complexity-ratchets.mjs --update
|
||||
exit 0
|
||||
|
||||
- name: Verify the write only went downward
|
||||
id: verify
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# Exits 1 if ANYTHING was raised/added or a rebaseline note was touched.
|
||||
# `set -e` then aborts the job before a commit exists — no PR is opened.
|
||||
node scripts/quality/verify-ratchet-bank.mjs > bank-summary.md
|
||||
if [ -n "$(git status --porcelain config/quality/)" ]; then
|
||||
echo "changed=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "changed=false" >> "$GITHUB_OUTPUT"
|
||||
echo "Nothing to bank — baselines already match the code."
|
||||
fi
|
||||
|
||||
- name: Open / update the banking PR
|
||||
if: steps.verify.outputs.changed == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
TARGET: ${{ steps.branch.outputs.target }}
|
||||
BANK_BRANCH: ${{ steps.branch.outputs.bank_branch }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -B "$BANK_BRANCH"
|
||||
git add config/quality/
|
||||
git commit -m "chore(quality): bank ratchet shrinks measured on ${TARGET}"
|
||||
git push --force origin "$BANK_BRANCH"
|
||||
|
||||
{
|
||||
echo "Automated banking of quality-ratchet **shrinks** already present in"
|
||||
echo "\`${TARGET}\` — the downward half of the ratchet, which nothing else runs (#8584)."
|
||||
echo ""
|
||||
echo "Produced by \`check:file-size --update\` + \`check:complexity-ratchets --update\`,"
|
||||
echo "then verified by \`scripts/quality/verify-ratchet-bank.mjs\`: **nothing was raised,"
|
||||
echo "nothing was added, no rebaseline note was touched** — the job aborts without"
|
||||
echo "opening a PR if any of those is violated."
|
||||
echo ""
|
||||
echo "No product code changes. Merging retires growth allowances that the code no"
|
||||
echo "longer needs; not merging leaves them available to whoever edits those files next."
|
||||
echo ""
|
||||
cat bank-summary.md
|
||||
echo ""
|
||||
echo "**Run:** ${RUN_URL}"
|
||||
} > pr-body.md
|
||||
|
||||
EXISTING=$(gh pr list --repo "$GITHUB_REPOSITORY" --head "$BANK_BRANCH" \
|
||||
--state open --json number --jq '.[0].number' 2>/dev/null || echo "")
|
||||
if [ -n "$EXISTING" ]; then
|
||||
gh pr edit "$EXISTING" --repo "$GITHUB_REPOSITORY" --body-file pr-body.md
|
||||
echo "Updated existing PR #$EXISTING"
|
||||
else
|
||||
gh pr create --repo "$GITHUB_REPOSITORY" --base "$TARGET" --head "$BANK_BRANCH" \
|
||||
--title "chore(quality): bank ratchet shrinks (${TARGET})" --body-file pr-body.md
|
||||
fi
|
||||
|
||||
1
changelog.d/features/8612-auto-bank-ratchet-shrinks.md
Normal file
1
changelog.d/features/8612-auto-bank-ratchet-shrinks.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(ci):** automate ratchet shrink-banking so file-size/complexity caps stop outliving their files after successful shrinks (#8584) ([#8612](https://github.com/diegosouzapw/OmniRoute/pull/8612)) — thanks @MumuTW
|
||||
@@ -259,6 +259,53 @@ justification entry. Never push combination drift onto a contributor PR, and nev
|
||||
rebaseline per-PR (that hides real regressions). Discriminate first: reproduce the
|
||||
red against the pure tip in a probe worktree before assuming your PR caused it.
|
||||
|
||||
## Banking Ratchet Shrinks — the downward direction (#8584)
|
||||
|
||||
The ratchet is only half automatic, and it is the wrong half. **Raising** a cap is a
|
||||
manual JSON edit that takes ten seconds and is the fastest way to unblock a red PR.
|
||||
**Lowering** one requires someone to run `--update` and commit the result — and until
|
||||
the `bank-ratchet-shrinks` job landed, no workflow ran it. The measured consequence
|
||||
(2026-07-25): 18 frozen files already at or under the 800-line new-file cap, the worst
|
||||
at 132× (`src/shared/validation/schemas.ts`, 19 lines carrying a 2,523 cap); the
|
||||
complexity ceiling walked `1794 → 2169` across ~37 rebaseline notes with exactly one
|
||||
decrease (−1); and "tighten via `--update` next cycle" written 31 times and honoured
|
||||
once. A cap that outlives the code that earned it silently converts every completed
|
||||
decomposition into a growth allowance for whoever edits the file next.
|
||||
|
||||
`nightly-release-green.yml` → job **`bank-ratchet-shrinks`** closes that loop:
|
||||
|
||||
| | |
|
||||
| -------- | ------------------------------------------------------------------------------------------------------ |
|
||||
| Runs on | `schedule` (3×/day) + `workflow_dispatch` — deliberately **not** `push` |
|
||||
| Measures | the highest `release/vX.Y.Z`, same resolution + injection guard as `release-green` |
|
||||
| Writes | `check:file-size --update` and `check:complexity-ratchets --update` (both shrink-only by construction) |
|
||||
| Verifies | `npm run check:ratchet-bank` (`scripts/quality/verify-ratchet-bank.mjs`) |
|
||||
| Ships | one always-current PR against the release branch — force-updated, never spammed |
|
||||
|
||||
Banking is batched rather than per-push because it has no latency requirement (a shrink
|
||||
banked within 8h is fine) while a per-merge run would rebuild the PR branch repeatedly
|
||||
during merge campaigns and pay for a full ESLint walk each time. Detection stays on
|
||||
push (`release-green`); only banking is batched.
|
||||
|
||||
### The safety verifier
|
||||
|
||||
The job writes to the baselines unattended, so `verify-ratchet-bank.mjs` is what makes
|
||||
that acceptable. It diffs the post-`--update` tree against `HEAD` and **aborts the job
|
||||
before any commit exists** — opening no PR — unless every change is one of:
|
||||
|
||||
- a `frozen` / `testFrozen` numeric entry **lowered** or **removed**
|
||||
- `complexity-baseline.json` → `count` **lowered**
|
||||
- `quality-baseline.json` → `metrics.cognitiveComplexity.value` **lowered**
|
||||
|
||||
Anything else fails: raising a number, adding an entry, changing `cap`/`testCap`, or
|
||||
deleting/rewriting a `_rebaseline_*` note (those notes are the audit trail for why each
|
||||
ceiling exists and are stored inside the same `frozen` object as the file entries).
|
||||
A bot that could raise a cap would be strictly worse than the status quo. Regression
|
||||
guard: `tests/unit/verify-ratchet-bank.test.ts`.
|
||||
|
||||
The job never pushes to `release/*` — a human merges the PR, so a bad measurement
|
||||
cannot land unreviewed.
|
||||
|
||||
## Allowlist Policy
|
||||
|
||||
Every gate that cannot fail on pre-existing violations uses a frozen allowlist
|
||||
|
||||
@@ -178,6 +178,7 @@
|
||||
"check:cognitive-complexity": "node scripts/check/check-cognitive-complexity.mjs",
|
||||
"check:complexity-ratchets": "node scripts/check/check-complexity-ratchets.mjs",
|
||||
"check:release-green": "node scripts/quality/validate-release-green.mjs",
|
||||
"check:ratchet-bank": "node scripts/quality/verify-ratchet-bank.mjs",
|
||||
"check:type-coverage": "node scripts/check/check-type-coverage.mjs",
|
||||
"check:lockfile": "node scripts/check/check-lockfile.mjs",
|
||||
"check:bundle-size": "node scripts/check/check-bundle-size.mjs",
|
||||
|
||||
245
scripts/quality/verify-ratchet-bank.mjs
Normal file
245
scripts/quality/verify-ratchet-bank.mjs
Normal file
@@ -0,0 +1,245 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Safety verifier for the automated ratchet-banking lane (#8584 proposal 2).
|
||||
*
|
||||
* WHY THIS EXISTS: the ratchet is asymmetric. Raising a cap is a ten-second manual
|
||||
* JSON edit made under merge pressure; lowering one requires someone to run
|
||||
* `--update` and commit, which no workflow does. The result is a high-water mark,
|
||||
* not a ratchet — caps outlive the shrinks that earned them (#8584 measured 18 of
|
||||
* them, up to 132x the file's actual size).
|
||||
*
|
||||
* The automation that fixes that has to write to the baselines, so it needs a hard
|
||||
* guarantee it can only ever write in the shrink direction. This script is that
|
||||
* guarantee: it diffs the post-`--update` working tree against HEAD and FAILS if
|
||||
* anything moved the wrong way. A bot that could raise a cap would be strictly worse
|
||||
* than the status quo, so the workflow aborts (opens no PR) on any problem here.
|
||||
*
|
||||
* Allowed, and nothing else:
|
||||
* file-size-baseline.json — a frozen/testFrozen numeric entry LOWERED or REMOVED
|
||||
* complexity-baseline.json — `count` LOWERED
|
||||
* quality-baseline.json — `metrics.cognitiveComplexity.value` LOWERED
|
||||
*
|
||||
* Explicitly rejected: raising any number, adding any entry, changing cap/testCap,
|
||||
* and deleting or rewriting a `_rebaseline_*` / `_comment` note (the notes are the
|
||||
* audit trail for why each ceiling exists — see the 37 of them in complexity-baseline).
|
||||
*
|
||||
* Usage: node scripts/quality/verify-ratchet-bank.mjs [--ref HEAD]
|
||||
* stdout — markdown summary of what would be banked (for the PR body)
|
||||
* stderr — human log
|
||||
* exit 0 — safe (see summary for whether anything changed at all)
|
||||
* exit 1 — UNSAFE, the caller must not commit
|
||||
*/
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const ROOT = process.cwd();
|
||||
|
||||
export const BANKED_BASELINES = {
|
||||
fileSize: "config/quality/file-size-baseline.json",
|
||||
complexity: "config/quality/complexity-baseline.json",
|
||||
quality: "config/quality/quality-baseline.json",
|
||||
};
|
||||
|
||||
/** Deep structural equality over JSON-shaped values. */
|
||||
function jsonEqual(a, b) {
|
||||
return JSON.stringify(a) === JSON.stringify(b);
|
||||
}
|
||||
|
||||
/**
|
||||
* Verifies one frozen map (`frozen` / `testFrozen`). Numeric entries may only be
|
||||
* lowered or removed; string entries are `_rebaseline_*` notes stored inside the
|
||||
* same object and must survive verbatim.
|
||||
* @returns {{problems: string[], removed: string[], lowered: [string, number, number][]}}
|
||||
*/
|
||||
export function verifyFrozenMap(before = {}, after = {}, label = "frozen") {
|
||||
const problems = [];
|
||||
const removed = [];
|
||||
const lowered = [];
|
||||
|
||||
for (const [key, prev] of Object.entries(before)) {
|
||||
const has = Object.prototype.hasOwnProperty.call(after, key);
|
||||
if (typeof prev !== "number") {
|
||||
// note key
|
||||
if (!has) problems.push(`${label}: note "${key}" was deleted — notes must be preserved`);
|
||||
else if (after[key] !== prev)
|
||||
problems.push(`${label}: note "${key}" was rewritten — notes must be preserved verbatim`);
|
||||
continue;
|
||||
}
|
||||
if (!has) {
|
||||
removed.push(key); // banked: dropped below the cap and left the baseline
|
||||
continue;
|
||||
}
|
||||
const next = after[key];
|
||||
if (typeof next !== "number") problems.push(`${label}: ${key} is no longer a number`);
|
||||
else if (next > prev) problems.push(`${label}: ${key} RAISED ${prev} → ${next}`);
|
||||
else if (next < prev) lowered.push([key, prev, next]);
|
||||
}
|
||||
|
||||
for (const key of Object.keys(after)) {
|
||||
if (!Object.prototype.hasOwnProperty.call(before, key))
|
||||
problems.push(`${label}: ${key} was ADDED — banking may not introduce entries`);
|
||||
}
|
||||
|
||||
return { problems, removed, lowered };
|
||||
}
|
||||
|
||||
/** Every top-level key except `except` must be byte-identical. */
|
||||
function verifyRestUntouched(before, after, except, file) {
|
||||
const problems = [];
|
||||
const keys = new Set([...Object.keys(before), ...Object.keys(after)]);
|
||||
for (const key of keys) {
|
||||
if (except.has(key)) continue;
|
||||
if (!jsonEqual(before[key], after[key]))
|
||||
problems.push(`${file}: "${key}" changed — only the ratcheted values may move`);
|
||||
}
|
||||
return problems;
|
||||
}
|
||||
|
||||
/** @returns {{problems: string[], removed: string[], lowered: [string, number, number][]}} */
|
||||
export function verifyFileSizeBaseline(before, after) {
|
||||
const src = verifyFrozenMap(before.frozen, after.frozen, "frozen");
|
||||
const tst = verifyFrozenMap(before.testFrozen, after.testFrozen, "testFrozen");
|
||||
return {
|
||||
problems: [
|
||||
...verifyRestUntouched(before, after, new Set(["frozen", "testFrozen"]), "file-size"),
|
||||
...src.problems,
|
||||
...tst.problems,
|
||||
],
|
||||
removed: [...src.removed, ...tst.removed],
|
||||
lowered: [...src.lowered, ...tst.lowered],
|
||||
};
|
||||
}
|
||||
|
||||
/** Lowers a single numeric field, everything else frozen. */
|
||||
function verifyScalar(before, after, prev, next, label, file, except) {
|
||||
const problems = verifyRestUntouched(before, after, except, file);
|
||||
const lowered = [];
|
||||
if (typeof next !== "number") problems.push(`${file}: ${label} is missing or not a number`);
|
||||
else if (next > prev) problems.push(`${file}: ${label} RAISED ${prev} → ${next}`);
|
||||
else if (next < prev) lowered.push([label, prev, next]);
|
||||
return { problems, removed: [], lowered };
|
||||
}
|
||||
|
||||
export function verifyComplexityBaseline(before, after) {
|
||||
return verifyScalar(
|
||||
before,
|
||||
after,
|
||||
before.count,
|
||||
after.count,
|
||||
"count",
|
||||
"complexity",
|
||||
new Set(["count"])
|
||||
);
|
||||
}
|
||||
|
||||
export function verifyQualityBaseline(before, after) {
|
||||
const prev = before.metrics?.cognitiveComplexity?.value;
|
||||
const next = after.metrics?.cognitiveComplexity?.value;
|
||||
// Only metrics.cognitiveComplexity.value may move; compare the rest of `metrics`
|
||||
// by swapping in the old value and requiring a byte-identical result.
|
||||
const patched = JSON.parse(JSON.stringify(after));
|
||||
if (patched.metrics?.cognitiveComplexity) patched.metrics.cognitiveComplexity.value = prev;
|
||||
const problems = jsonEqual(before, patched)
|
||||
? []
|
||||
: ["quality: something other than metrics.cognitiveComplexity.value changed"];
|
||||
const lowered = [];
|
||||
if (typeof next !== "number") problems.push("quality: cognitiveComplexity.value is not a number");
|
||||
else if (next > prev) problems.push(`quality: cognitiveComplexity RAISED ${prev} → ${next}`);
|
||||
else if (next < prev) lowered.push(["cognitiveComplexity", prev, next]);
|
||||
return { problems, removed: [], lowered };
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{fileSize?: {before: object, after: object}, complexity?: ..., quality?: ...}} pairs
|
||||
* @returns {{ok: boolean, changed: boolean, problems: string[], removed: string[], lowered: [string, number, number][]}}
|
||||
*/
|
||||
export function verifyRatchetBank(pairs) {
|
||||
const verifiers = {
|
||||
fileSize: verifyFileSizeBaseline,
|
||||
complexity: verifyComplexityBaseline,
|
||||
quality: verifyQualityBaseline,
|
||||
};
|
||||
const problems = [];
|
||||
const removed = [];
|
||||
const lowered = [];
|
||||
for (const [kind, verify] of Object.entries(verifiers)) {
|
||||
const pair = pairs[kind];
|
||||
if (!pair) continue;
|
||||
const r = verify(pair.before, pair.after);
|
||||
problems.push(...r.problems);
|
||||
removed.push(...r.removed);
|
||||
lowered.push(...r.lowered);
|
||||
}
|
||||
return {
|
||||
ok: problems.length === 0,
|
||||
changed: removed.length > 0 || lowered.length > 0,
|
||||
problems,
|
||||
removed,
|
||||
lowered,
|
||||
};
|
||||
}
|
||||
|
||||
export function formatBankSummary(result) {
|
||||
if (!result.changed) return "Nothing to bank — every baseline already matches the code.\n";
|
||||
const lines = [];
|
||||
if (result.lowered.length) {
|
||||
lines.push(
|
||||
`### Lowered (${result.lowered.length})`,
|
||||
"",
|
||||
"| entry | was | now |",
|
||||
"| --- | --- | --- |"
|
||||
);
|
||||
for (const [key, prev, next] of result.lowered)
|
||||
lines.push(`| \`${key}\` | ${prev} | ${next} |`);
|
||||
lines.push("");
|
||||
}
|
||||
if (result.removed.length) {
|
||||
lines.push(
|
||||
`### Removed — now at or under the cap (${result.removed.length})`,
|
||||
"",
|
||||
...result.removed.map((f) => `- \`${f}\``),
|
||||
""
|
||||
);
|
||||
}
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function readJson(file) {
|
||||
return JSON.parse(fs.readFileSync(path.join(ROOT, file), "utf8"));
|
||||
}
|
||||
|
||||
function readJsonAtRef(ref, file) {
|
||||
// execFileSync (no shell) — `ref`/`file` are never interpolated into a command string.
|
||||
return JSON.parse(execFileSync("git", ["show", `${ref}:${file}`], { encoding: "utf8" }));
|
||||
}
|
||||
|
||||
function main() {
|
||||
const i = process.argv.indexOf("--ref");
|
||||
const ref = i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : "HEAD";
|
||||
|
||||
const pairs = {};
|
||||
for (const [kind, file] of Object.entries(BANKED_BASELINES)) {
|
||||
pairs[kind] = { before: readJsonAtRef(ref, file), after: readJson(file) };
|
||||
}
|
||||
|
||||
const result = verifyRatchetBank(pairs);
|
||||
process.stdout.write(formatBankSummary(result));
|
||||
|
||||
if (!result.ok) {
|
||||
console.error(
|
||||
`[verify-ratchet-bank] UNSAFE — ${result.problems.length} problem(s); refusing to bank:\n` +
|
||||
result.problems.map((p) => " ✗ " + p).join("\n") +
|
||||
"\n → the banking lane may only lower or remove. Investigate before committing."
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
console.error(
|
||||
result.changed
|
||||
? `[verify-ratchet-bank] OK — ${result.lowered.length} lowered, ${result.removed.length} removed, nothing raised.`
|
||||
: "[verify-ratchet-bank] OK — no baseline movement."
|
||||
);
|
||||
}
|
||||
|
||||
if (import.meta.url === pathToFileURL(process.argv[1] || "").href) main();
|
||||
224
tests/unit/verify-ratchet-bank.test.ts
Normal file
224
tests/unit/verify-ratchet-bank.test.ts
Normal file
@@ -0,0 +1,224 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert";
|
||||
import {
|
||||
formatBankSummary,
|
||||
verifyComplexityBaseline,
|
||||
verifyFileSizeBaseline,
|
||||
verifyFrozenMap,
|
||||
verifyQualityBaseline,
|
||||
verifyRatchetBank,
|
||||
} from "../../scripts/quality/verify-ratchet-bank.mjs";
|
||||
|
||||
// The banking lane writes to the quality baselines unattended, so the only thing
|
||||
// standing between it and a bot that can RAISE a cap is this verifier. Each test
|
||||
// below is one way the automation could go wrong.
|
||||
|
||||
// --- frozen map (file-size `frozen` / `testFrozen`) ---
|
||||
|
||||
test("a lowered entry is banked as lowered", () => {
|
||||
const r = verifyFrozenMap({ "a.ts": 1000 }, { "a.ts": 900 });
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.lowered, [["a.ts", 1000, 900]]);
|
||||
assert.deepEqual(r.removed, []);
|
||||
});
|
||||
|
||||
test("an entry that dropped out of the baseline is banked as removed", () => {
|
||||
const r = verifyFrozenMap({ "a.ts": 1000, "b.ts": 900 }, { "b.ts": 900 });
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.removed, ["a.ts"]);
|
||||
assert.deepEqual(r.lowered, []);
|
||||
});
|
||||
|
||||
test("an unchanged entry is neither lowered nor removed", () => {
|
||||
const r = verifyFrozenMap({ "a.ts": 1000 }, { "a.ts": 1000 });
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.removed, []);
|
||||
assert.deepEqual(r.lowered, []);
|
||||
});
|
||||
|
||||
test("RAISING an entry is rejected — this is the failure mode the gate exists for", () => {
|
||||
const r = verifyFrozenMap({ "a.ts": 1000 }, { "a.ts": 1001 });
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /RAISED 1000 → 1001/);
|
||||
});
|
||||
|
||||
test("ADDING an entry is rejected — banking may not enroll new files", () => {
|
||||
const r = verifyFrozenMap({ "a.ts": 1000 }, { "a.ts": 1000, "new.ts": 5000 });
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /new\.ts.*ADDED/);
|
||||
});
|
||||
|
||||
test("deleting a _rebaseline_ note stored inside frozen is rejected", () => {
|
||||
const r = verifyFrozenMap({ _rebaseline_x: "why", "a.ts": 10 }, { "a.ts": 10 });
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /_rebaseline_x.*deleted/);
|
||||
assert.deepEqual(r.removed, [], "a note is not a banked removal");
|
||||
});
|
||||
|
||||
test("rewriting a note is rejected — the notes are the audit trail", () => {
|
||||
const r = verifyFrozenMap({ _rebaseline_x: "why" }, { _rebaseline_x: "different" });
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /rewritten/);
|
||||
});
|
||||
|
||||
// --- file-size baseline as a whole ---
|
||||
|
||||
const fsBefore = () => ({
|
||||
cap: 800,
|
||||
testCap: 800,
|
||||
_comment: "keep me",
|
||||
frozen: { "a.ts": 1000, _rebaseline_1: "note" },
|
||||
testFrozen: { "t.test.ts": 900 },
|
||||
});
|
||||
|
||||
test("file-size: a shrink in frozen and testFrozen is banked", () => {
|
||||
const after = fsBefore();
|
||||
after.frozen["a.ts"] = 850;
|
||||
delete after.testFrozen["t.test.ts"]; // fell under testCap
|
||||
const r = verifyFileSizeBaseline(fsBefore(), after);
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.lowered, [["a.ts", 1000, 850]]);
|
||||
assert.deepEqual(r.removed, ["t.test.ts"]);
|
||||
});
|
||||
|
||||
test("file-size: raising the cap is rejected", () => {
|
||||
const after = fsBefore();
|
||||
after.cap = 1200;
|
||||
const r = verifyFileSizeBaseline(fsBefore(), after);
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /"cap" changed/);
|
||||
});
|
||||
|
||||
test("file-size: changing testCap is rejected", () => {
|
||||
const after = fsBefore();
|
||||
after.testCap = 1200;
|
||||
const r = verifyFileSizeBaseline(fsBefore(), after);
|
||||
assert.match(r.problems[0], /"testCap" changed/);
|
||||
});
|
||||
|
||||
test("file-size: touching an unrelated top-level key is rejected", () => {
|
||||
const after = fsBefore();
|
||||
after._comment = "rewritten";
|
||||
const r = verifyFileSizeBaseline(fsBefore(), after);
|
||||
assert.match(r.problems[0], /"_comment" changed/);
|
||||
});
|
||||
|
||||
// --- complexity baseline ---
|
||||
|
||||
const cxBefore = () => ({ _comment: "c", count: 2169, _rebaseline_a: "note" });
|
||||
|
||||
test("complexity: lowering count is banked", () => {
|
||||
const after = cxBefore();
|
||||
after.count = 2100;
|
||||
const r = verifyComplexityBaseline(cxBefore(), after);
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.lowered, [["count", 2169, 2100]]);
|
||||
});
|
||||
|
||||
test("complexity: raising count is rejected", () => {
|
||||
const after = cxBefore();
|
||||
after.count = 2200;
|
||||
const r = verifyComplexityBaseline(cxBefore(), after);
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /count RAISED 2169 → 2200/);
|
||||
});
|
||||
|
||||
test("complexity: adding a rebaseline note is rejected — the bot never justifies a ceiling", () => {
|
||||
const after = cxBefore();
|
||||
after.count = 2100;
|
||||
after._rebaseline_new = "bot said so";
|
||||
const r = verifyComplexityBaseline(cxBefore(), after);
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /_rebaseline_new/);
|
||||
});
|
||||
|
||||
// --- quality baseline (cognitive complexity lives nested) ---
|
||||
|
||||
const qBefore = () => ({
|
||||
metrics: {
|
||||
eslintWarnings: { value: 345 },
|
||||
cognitiveComplexity: { value: 956, _rebaseline_a: "note" },
|
||||
},
|
||||
});
|
||||
|
||||
test("quality: lowering cognitiveComplexity.value is banked", () => {
|
||||
const after = qBefore();
|
||||
after.metrics.cognitiveComplexity.value = 900;
|
||||
const r = verifyQualityBaseline(qBefore(), after);
|
||||
assert.deepEqual(r.problems, []);
|
||||
assert.deepEqual(r.lowered, [["cognitiveComplexity", 956, 900]]);
|
||||
});
|
||||
|
||||
test("quality: raising cognitiveComplexity.value is rejected", () => {
|
||||
const after = qBefore();
|
||||
after.metrics.cognitiveComplexity.value = 1000;
|
||||
const r = verifyQualityBaseline(qBefore(), after);
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /RAISED 956 → 1000/);
|
||||
});
|
||||
|
||||
test("quality: moving a DIFFERENT metric is rejected even when cognitive went down", () => {
|
||||
const after = qBefore();
|
||||
after.metrics.cognitiveComplexity.value = 900;
|
||||
after.metrics.eslintWarnings.value = 400;
|
||||
const r = verifyQualityBaseline(qBefore(), after);
|
||||
assert.equal(r.problems.length, 1);
|
||||
assert.match(r.problems[0], /other than metrics\.cognitiveComplexity\.value/);
|
||||
});
|
||||
|
||||
// --- aggregator ---
|
||||
|
||||
test("verifyRatchetBank reports ok+changed when every file only shrank", () => {
|
||||
const fsAfter = fsBefore();
|
||||
fsAfter.frozen["a.ts"] = 850;
|
||||
const cxAfter = cxBefore();
|
||||
cxAfter.count = 2100;
|
||||
const r = verifyRatchetBank({
|
||||
fileSize: { before: fsBefore(), after: fsAfter },
|
||||
complexity: { before: cxBefore(), after: cxAfter },
|
||||
quality: { before: qBefore(), after: qBefore() },
|
||||
});
|
||||
assert.equal(r.ok, true);
|
||||
assert.equal(r.changed, true);
|
||||
assert.equal(r.lowered.length, 2);
|
||||
});
|
||||
|
||||
test("verifyRatchetBank is not ok if ANY file moved the wrong way", () => {
|
||||
const fsAfter = fsBefore();
|
||||
fsAfter.frozen["a.ts"] = 850; // legitimate shrink
|
||||
const cxAfter = cxBefore();
|
||||
cxAfter.count = 2200; // but complexity was raised
|
||||
const r = verifyRatchetBank({
|
||||
fileSize: { before: fsBefore(), after: fsAfter },
|
||||
complexity: { before: cxBefore(), after: cxAfter },
|
||||
});
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.problems.length, 1);
|
||||
});
|
||||
|
||||
test("verifyRatchetBank reports ok+unchanged when nothing moved", () => {
|
||||
const r = verifyRatchetBank({
|
||||
fileSize: { before: fsBefore(), after: fsBefore() },
|
||||
complexity: { before: cxBefore(), after: cxBefore() },
|
||||
quality: { before: qBefore(), after: qBefore() },
|
||||
});
|
||||
assert.equal(r.ok, true);
|
||||
assert.equal(r.changed, false);
|
||||
});
|
||||
|
||||
// --- summary rendering (this becomes the PR body) ---
|
||||
|
||||
test("formatBankSummary says nothing to bank when unchanged", () => {
|
||||
const out = formatBankSummary({ changed: false, lowered: [], removed: [] });
|
||||
assert.match(out, /Nothing to bank/);
|
||||
});
|
||||
|
||||
test("formatBankSummary lists both lowered and removed entries", () => {
|
||||
const out = formatBankSummary({
|
||||
changed: true,
|
||||
lowered: [["a.ts", 1000, 850]],
|
||||
removed: ["schemas.ts"],
|
||||
});
|
||||
assert.match(out, /\| `a\.ts` \| 1000 \| 850 \|/);
|
||||
assert.match(out, /- `schemas\.ts`/);
|
||||
});
|
||||
Reference in New Issue
Block a user