The tests that timed out under load stay well inside their timeouts #138

Merged

agra merged 8 commits from task/3595-under-load-npm-test-cases-of-1-1-5-s-hit into main

2026-10-24 06:42:13 +03:00

agra commented

2026-10-24 05:24:26 +03:00

Owner

Tracker: #3595

Under load, npm test cases that take about 0.5–1 s alone hit vitest's default 5,000 ms per-test timeout, so the fast and full gates failed with no assertion failing. This PR profiles each test that has timed out under load (hud-css text contrast, hud-mock's scene loop and the two tests its timeout broke, craft's two helipad landing sweeps, collapses' no-scale test and wrecks' Medium shards test), plus lobby-mock's screen loop, which cascades the same way, and lobby-css's contrast loop, which costs what hud-css's does. It cuts each one's cost, or the size of each test, without changing what the tests pin. The global timeout is unchanged, and only test files change.

What changes

collapses.test.ts, the no-scale test. Its debris watcher decomposed the matrix of all 256 pool slots on every frame. It now reads only the slots the pool draws (mesh.count, or none while the mesh is hidden). Slots it was tracking that fall past the count still count as having left. That takes it from 558 to 242 ms alone. The rest of its cost is replaying 318 building falls and 24 structure deaths frame by frame: 62 ms for the collapse matrices, 163 ms for the looks' debris and 20 ms for the structures'. So it now has its own 15 s timeout, with its measured cost stated beside it: 0.25 s alone, and up to 4.6 s with two more npm test runs on the machine.
wrecks.test.ts, the Medium shards test. It made one expect per shard per frame, about 170k. It now takes the fastest spin over each wreck's 3 s and asserts that once: 359 → 128 ms alone.
hud-css.test.ts and lobby-css.test.ts, text contrast. Each of the 28 HUD scenes and the 7 lobby screens is its own test (it.each), named for its scene or screen. A HUD scene takes 22–73 ms, against 746 ms for the single loop. The cost is happy-dom matching each stylesheet rule once per element. Memoising the computed-style reads in inks gained nothing (679 vs 672–697 ms), because happy-dom already caches each element's style.
hud-mock.test.ts and lobby-mock.test.ts, the scene and screen checks. Each scene (29) and screen (6) is its own test, with its expectation keyed by name. A HUD scene takes 11–19 ms. The first case of each file pays the mock's cold import: 234 ms for the HUD and 183 ms for the lobby. A timeout in one case no longer leaves a loop replacing document.body under the tests after it.
craft.test.ts, the two helipad landing sweeps. Each gunship's start is its own test (it.each over PAD_STARTS). That gives 8 starts for the two-gunship sweep, at 93–163 ms each against 940 ms, and 5 for the lone-gunship sweep, at 63–156 ms each against 449 ms. The two-gunship sweep runs 544 fields and 210k ticks at about 4.5 µs a tick. helipadField() setup takes 0.35 ms, 12% of the sweep, so sharing it would not pay.

Under load (runner m5, 18 cores)

Main d7b69a7, extracted with git archive, and the head alternated, each running npm test under two loads:

18 busy loops: 18 yes > /dev/null loops (one-minute load 22–39). 3 rounds per arm at f013ba6, then 2 at the final 496ca08.
Two more suites: two more npm test runs of main beside the measured one (load 24–35). 3 rounds per arm at cb205fa, then 2 at 496ca08.

Slowest case of each test, in s (✗ = "Test timed out in 5000ms"):

	main, 18 loops	head, 18 loops	main, two more suites	head, two more suites
rounds passed	1 of 3	5 of 5	0 of 3 (7–8 failures each)	5 of 5
craft, two gunships	3.4–7.2 ✗	1.1–1.4	13.9–14.2 ✗	2.4–2.7
craft, lone gunship	2.9–4.3	1.0	7.4–8.5 ✗	2.1–2.6
hud-css contrast	3.6–6.3 ✗	0.4–0.6	12.6–13.7 ✗	0.8–1.2
hud-mock scenes	2.2–4.5	1.3	5.1 ✗ (and 2 AssertionErrors after it)	2.0–3.2
collapses no-scale	3.2–4.9	1.6–1.9	8.3–8.7 ✗	3.4–4.6 (15 s timeout)
wrecks Medium shards	1.7–3.1	0.7–0.9	4.3–5.7 ✗	1.3–2.0
lobby-mock screens	1.1–2.1	0.6–0.8 *	2.5–3.8	1.9–2.3
lobby-css contrast	1.0–1.4	0.6–0.7 *	2.9–3.3	0.9–1.2 *

* From the rounds after the split: lobby-mock's split is cb205fa, after f013ba6's 18-loop rounds, and lobby-css's is 496ca08.

With --testTimeout=150, main fails both scene loops and four tests after them, on AssertionErrors: expected [ null, null ] to deeply equal [ 'Monument', 'Growth ×2' ], expected [] to deeply equal [ … ] and expected null not to be null. Head fails only lobby-mock's first screen, on its timeout.

These default-timeout tests are untouched. None of them timed out in any round:

With 18 busy loops they took 1.2–1.8 s, at most 0.37 of 5 s: net's economy-256 and churn-256 staging, units' mesh budget, game's monument refill, cities' town founding and rocks.
With two more suites they took up to 3.0–3.5 s each. They are the next to watch if a runner's load grows past that.

What the tests still pin

Collapses: two mutants of DebrisPool.advance, in git archive copies of main and the head. One shrinks a piece as it fades (width * (1 - 0.5 * leaving)), the other drops a piece at half its fade (leaving >= 0.5). Each fails the no-scale test on both arms, with the same 330 failures byte for byte.
Wrecks: a mutant that spins shards 4× faster (shape.spin * 4) fails the shards test on both arms: main at 7.21 rad/s, head at 7.96 rad/s, against 2.5.
The split tests: each case runs the old loop body's own assertions, and every scene, screen and start of the old loops is a case. Each expectation record holds exactly its list's 29 scenes and 6 screens.

The changelog and in-game help are unchanged: nothing a player meets changes.

Gates

fast (run 463's gate #1, tracker gate run 257) at cb205fa: passed, npm test 966/966 in 24.5 s.
full (run 463's gate #2, tracker gate run 258) at 496ca08: passed, npm test 972/972 in 26.5 s and test:phone 180/180 in 190 s.

Tracker: [#3595](https://sak.swipelab.com/tasks/3595) Under load, `npm test` cases that take about 0.5–1 s alone hit vitest's default 5,000 ms per-test timeout, so the `fast` and `full` gates failed with no assertion failing. This PR profiles each test that has timed out under load (hud-css text contrast, hud-mock's scene loop and the two tests its timeout broke, craft's two helipad landing sweeps, collapses' no-scale test and wrecks' Medium shards test), plus lobby-mock's screen loop, which cascades the same way, and lobby-css's contrast loop, which costs what hud-css's does. It cuts each one's cost, or the size of each test, without changing what the tests pin. The global timeout is unchanged, and only test files change. ## What changes - **collapses.test.ts, the no-scale test.** Its debris watcher decomposed the matrix of all 256 pool slots on every frame. It now reads only the slots the pool draws (`mesh.count`, or none while the mesh is hidden). Slots it was tracking that fall past the count still count as having left. That takes it from 558 to 242 ms alone. The rest of its cost is replaying 318 building falls and 24 structure deaths frame by frame: 62 ms for the collapse matrices, 163 ms for the looks' debris and 20 ms for the structures'. So it now has its own 15 s timeout, with its measured cost stated beside it: 0.25 s alone, and up to 4.6 s with two more `npm test` runs on the machine. - **wrecks.test.ts, the Medium shards test.** It made one `expect` per shard per frame, about 170k. It now takes the fastest spin over each wreck's 3 s and asserts that once: 359 → 128 ms alone. - **hud-css.test.ts and lobby-css.test.ts, text contrast.** Each of the 28 HUD scenes and the 7 lobby screens is its own test (`it.each`), named for its scene or screen. A HUD scene takes 22–73 ms, against 746 ms for the single loop. The cost is happy-dom matching each stylesheet rule once per element. Memoising the computed-style reads in `inks` gained nothing (679 vs 672–697 ms), because happy-dom already caches each element's style. - **hud-mock.test.ts and lobby-mock.test.ts, the scene and screen checks.** Each scene (29) and screen (6) is its own test, with its expectation keyed by name. A HUD scene takes 11–19 ms. The first case of each file pays the mock's cold import: 234 ms for the HUD and 183 ms for the lobby. A timeout in one case no longer leaves a loop replacing `document.body` under the tests after it. - **craft.test.ts, the two helipad landing sweeps.** Each gunship's start is its own test (`it.each` over `PAD_STARTS`). That gives 8 starts for the two-gunship sweep, at 93–163 ms each against 940 ms, and 5 for the lone-gunship sweep, at 63–156 ms each against 449 ms. The two-gunship sweep runs 544 fields and 210k ticks at about 4.5 µs a tick. `helipadField()` setup takes 0.35 ms, 12% of the sweep, so sharing it would not pay. ## Under load (runner m5, 18 cores) Main d7b69a7, extracted with `git archive`, and the head alternated, each running `npm test` under two loads: - **18 busy loops:** 18 `yes > /dev/null` loops (one-minute load 22–39). 3 rounds per arm at f013ba6, then 2 at the final 496ca08. - **Two more suites:** two more `npm test` runs of main beside the measured one (load 24–35). 3 rounds per arm at cb205fa, then 2 at 496ca08. Slowest case of each test, in s (✗ = "Test timed out in 5000ms"): | | main, 18 loops | head, 18 loops | main, two more suites | head, two more suites | |---|---|---|---|---| | rounds passed | 1 of 3 | 5 of 5 | 0 of 3 (7–8 failures each) | 5 of 5 | | craft, two gunships | 3.4–7.2 ✗ | 1.1–1.4 | 13.9–14.2 ✗ | 2.4–2.7 | | craft, lone gunship | 2.9–4.3 | 1.0 | 7.4–8.5 ✗ | 2.1–2.6 | | hud-css contrast | 3.6–6.3 ✗ | 0.4–0.6 | 12.6–13.7 ✗ | 0.8–1.2 | | hud-mock scenes | 2.2–4.5 | 1.3 | 5.1 ✗ (and 2 AssertionErrors after it) | 2.0–3.2 | | collapses no-scale | 3.2–4.9 | 1.6–1.9 | 8.3–8.7 ✗ | 3.4–4.6 (15 s timeout) | | wrecks Medium shards | 1.7–3.1 | 0.7–0.9 | 4.3–5.7 ✗ | 1.3–2.0 | | lobby-mock screens | 1.1–2.1 | 0.6–0.8 * | 2.5–3.8 | 1.9–2.3 | | lobby-css contrast | 1.0–1.4 | 0.6–0.7 * | 2.9–3.3 | 0.9–1.2 * | \* From the rounds after the split: lobby-mock's split is cb205fa, after f013ba6's 18-loop rounds, and lobby-css's is 496ca08. With `--testTimeout=150`, main fails both scene loops and four tests after them, on AssertionErrors: `expected [ null, null ] to deeply equal [ 'Monument', 'Growth ×2' ]`, `expected [] to deeply equal [ … ]` and `expected null not to be null`. Head fails only lobby-mock's first screen, on its timeout. These default-timeout tests are untouched. None of them timed out in any round: - With 18 busy loops they took 1.2–1.8 s, at most 0.37 of 5 s: net's economy-256 and churn-256 staging, units' mesh budget, game's monument refill, cities' town founding and rocks. - With two more suites they took up to 3.0–3.5 s each. They are the next to watch if a runner's load grows past that. ## What the tests still pin - **Collapses:** two mutants of `DebrisPool.advance`, in `git archive` copies of main and the head. One shrinks a piece as it fades (`width * (1 - 0.5 * leaving)`), the other drops a piece at half its fade (`leaving >= 0.5`). Each fails the no-scale test on both arms, with the same 330 failures byte for byte. - **Wrecks:** a mutant that spins shards 4× faster (`shape.spin * 4`) fails the shards test on both arms: main at 7.21 rad/s, head at 7.96 rad/s, against 2.5. - **The split tests:** each case runs the old loop body's own assertions, and every scene, screen and start of the old loops is a case. Each expectation record holds exactly its list's 29 scenes and 6 screens. The changelog and in-game help are unchanged: nothing a player meets changes. ## Gates - `fast` (run 463's gate #1, tracker gate run 257) at cb205fa: passed, `npm test` 966/966 in 24.5 s. - `full` (run 463's gate #2, tracker gate run 258) at 496ca08: passed, `npm test` 972/972 in 26.5 s and `test:phone` 180/180 in 190 s.

agra added 5 commits

2026-10-24 05:24:26 +03:00

The collapses no-scale test reads only the debris slots the pool draws, so it decomposes a handful of matrices a frame instead of all 256 (#3595 ) df2ba27fd1

The Medium shards test bounds the fastest spin of each wreck's shards with one assertion instead of one per shard and frame (#3595 ) 204e4d05fe

hud.css's contrast check runs each HUD scene as its own test (#3595 ) d1bf4ca318

The HUD mock's scene check runs each scene as its own test (#3595 ) 4748bc0056

The two helipad landing sweeps run each gunship's start as its own test (#3595 ) f013ba633a

agra changed title from ~~Heavy tests stay well inside their timeouts under load~~ to The tests that timed out under load stay well inside their timeouts

2026-10-24 05:54:18 +03:00

agra added 3 commits

2026-10-24 05:54:19 +03:00

The lobby mock's screen check runs each screen as its own test (#3595 ) cb205fa9ac

The collapses no-scale test has a 15 s timeout, with its cost alone and beside two more test runs stated beside it (#3595 ) b070f86fb2

lobby.css's contrast check runs each lobby screen as its own test (#3595 ) 496ca082cb

agra merged commit d4a6158633 into main

2026-10-24 06:42:13 +03:00

agra referenced this pull request from a commit

2026-10-24 06:42:13 +03:00

Merge pull request 'The tests that timed out under load stay well inside their timeouts' (#138) from task/3595-under-load-npm-test-cases-of-1-1-5-s-hit into main