test: group examples into per-category folders

Move examples/*.sx and their expected/ snapshots into per-category
subfolders (examples/<category>/...). Folder = leading filename token,
with ffi-objc/ffi-jni kept whole; filenames are unchanged. The corpus
runner and LSP sweep now discover each category's expected/ dir, while
issues/ stays flat. Example 1058's repo-root-relative companion import
is made file-relative. Path strings embedded in 164 snapshots were
regenerated (path-only changes). Test-layout docs in CLAUDE.md updated.
This commit is contained in:
agra
2026-06-21 14:41:34 +03:00
parent 6d1409bc1f
commit 66bdc70bf1
3357 changed files with 456 additions and 363 deletions

View File

@@ -452,24 +452,33 @@ Examples and pinned issue repros use the `XXXX-category-test-name` scheme — a
4-digit number in per-category 100-blocks: `basic` 00xx, `types` 01xx, `generics`
02xx, `closures` 03xx, `protocols` 04xx, `packs` 05xx, `comptime` 06xx, `modules`
07xx, `memory` 08xx, `optionals` 09xx, `errors` 10xx, `diagnostics` 11xx, `ffi`
12xx, `ffi-objc` 13xx, `ffi-jni` 14xx, `vectors` 15xx, `platform` 16xx.
12xx, `ffi-objc` 13xx, `ffi-jni` 14xx, `vectors` 15xx, `platform` 16xx. (Newer
categories have grown past 16xx — `atomics` 17xx, `concurrency` 18xx — and some
share 16xx; the **category is the leading name token**, not the number block.)
Expected output lives in an `expected/` directory **next to the test file**,
split into three streams (no more merged `2>&1`) plus an optional IR snapshot:
`examples/` is organized into **per-category subfolders** — the folder name is
the leading token of the filename (`ffi-objc`/`ffi-jni` kept whole). The full
`XXXX-category-...` filename is unchanged; the folder just groups it. Each
category folder has its own `expected/` directory holding the snapshots, split
into three streams (no more merged `2>&1`) plus an optional IR snapshot:
```
<root>/XXXX-category-name.sx
<root>/expected/XXXX-category-name.exit # process exit code
<root>/expected/XXXX-category-name.stdout # normalized stdout
<root>/expected/XXXX-category-name.stderr # normalized stderr
<root>/expected/XXXX-category-name.ir # optional `sx ir` snapshot
<root>/expected/XXXX-category-name.build # optional JSON build/run directives
examples/<category>/XXXX-category-name.sx
examples/<category>/expected/XXXX-category-name.exit # process exit code
examples/<category>/expected/XXXX-category-name.stdout # normalized stdout
examples/<category>/expected/XXXX-category-name.stderr # normalized stderr
examples/<category>/expected/XXXX-category-name.ir # optional `sx ir` snapshot
examples/<category>/expected/XXXX-category-name.build # optional JSON build/run directives
```
A test is any `<name>.sx` with an `expected/<name>.exit` marker. The runner
scans two roots: `examples/` (the feature suite) and `issues/` (pinned bug
repros). Multi-file tests keep companions (`.c`/`.h`, imported `.sx`, fixture
dirs) under the same `XXXX-` prefix.
`issues/` stays **flat** (`issues/<name>.sx` + `issues/expected/<name>.exit`).
A test is any `<name>.sx` with a sibling `expected/<name>.exit` marker. The
runner scans two roots — `examples/` (the feature suite, recursing one level
into category folders) and `issues/` (pinned bug repros) — discovering every
`expected/` directory under each. Multi-file tests keep companions (`.c`/`.h`,
imported `.sx`, fixture dirs) under the same `XXXX-` prefix **in the same
category folder**, and reference them with file-relative imports (e.g.
`#import "XXXX-foo/lib.sx"`), never a repo-root-relative `examples/...` path.
The optional `<name>.build` JSON sidecar carries per-example directives
(unknown keys are a hard error — never silently ignored):
@@ -498,18 +507,18 @@ The optional `<name>.build` JSON sidecar carries per-example directives
Safe workflow:
1. Fix the code until `zig build test` passes against the **existing** snapshots.
2. Only run `zig build test -Dupdate-goldens` when you've intentionally changed output (new feature, new test, changed formatting).
3. After regenerating, review the diff (`git diff examples/expected/ issues/expected/`) to confirm no error messages or empty output were captured.
3. After regenerating, review the diff (`git diff examples/ issues/expected/`) to confirm no error messages or empty output were captured.
**Scope a regen to specific examples with `-Dname`.** A *full* `-Dupdate-goldens`
re-runs and rewrites all ~690 snapshots, so a single flaky/host-divergent example
(AOT links, cross-arch `target` examples, anything that intermittently fails) can
silently clobber a good snapshot. To capture just the example(s) you added, pass
their full repo-relative `.sx` path(s), comma-separated — this rewrites ONLY those
and touches nothing else:
their full repo-relative `.sx` path(s) — now including the category folder —
comma-separated; this rewrites ONLY those and touches nothing else:
```sh
zig build test -Dname=examples/0625-comptime-weld-struct-field.sx -Dupdate-goldens
zig build test -Dname=examples/0625-foo.sx,examples/0626-bar.sx # verify just these
zig build test -Dname=examples/comptime/0625-comptime-weld-struct-field.sx -Dupdate-goldens
zig build test -Dname=examples/comptime/0625-foo.sx,examples/types/0126-bar.sx # verify just these
```
`-Dname` also busts the test-run cache (the corpus enumerates `.sx`/`expected/`
@@ -521,11 +530,11 @@ compiler source — forces a fresh run.
There is no monolithic smoke file — each feature is its own focused example.
1. Create `examples/XXXX-<category>-<name>.sx` (next free number in the matching
category block).
2. Run it: `./zig-out/bin/sx run examples/XXXX-<category>-<name>.sx`
1. Create `examples/<category>/XXXX-<category>-<name>.sx` (next free number in
the matching category block, in that category's folder).
2. Run it: `./zig-out/bin/sx run examples/<category>/XXXX-<category>-<name>.sx`
3. Seed the marker and capture expected output:
`: > examples/expected/XXXX-<category>-<name>.exit` then
`: > examples/<category>/expected/XXXX-<category>-<name>.exit` then
`zig build test -Dupdate-goldens`
4. Verify all tests still pass: `zig build test`
@@ -533,9 +542,9 @@ There is no monolithic smoke file — each feature is its own focused example.
| File | Purpose |
|------|---------|
| `examples/XXXX-category-name.sx` | Focused feature example — one feature per file. |
| `examples/expected/XXXX-category-name.{exit,stdout,stderr}` | Expected exit code + the two output streams. Regenerate with `zig build test -Dupdate-goldens`. |
| `examples/expected/XXXX-category-name.ir` | Optional `sx ir` snapshot — present only where lowering shape is locked. |
| `examples/<category>/XXXX-category-name.sx` | Focused feature example — one feature per file, in its category folder. |
| `examples/<category>/expected/XXXX-category-name.{exit,stdout,stderr}` | Expected exit code + the two output streams. Regenerate with `zig build test -Dupdate-goldens`. |
| `examples/<category>/expected/XXXX-category-name.ir` | Optional `sx ir` snapshot — present only where lowering shape is locked. |
| `issues/NNNN-slug.md` | Open-issue / bug-report writeup (mark RESOLVED in a banner when fixed; the `.md` stays). |
| `issues/NNNN-slug.sx` (+ `issues/NNNN-slug/`) | The issue's minimal repro, co-located with the `.md`. A repro with an `issues/expected/NNNN-slug.exit` marker runs in the suite; unpinned ones don't. |
| `src/corpus_run.test.zig` | The corpus runner inside `zig build test` — spawns `sx` per example, diffs stdout/stderr/exit (+ optional IR); regenerates snapshots under `-Dupdate-goldens`. |
@@ -557,10 +566,12 @@ All Zig unit tests live in separate `*.test.zig` files alongside the source they
### Creating a new standalone test
1. Create `examples/XXXX-<category>-<name>.sx` (focused example) **or**, for an
open bug, `issues/NNNN-slug.{md,sx}` (repro co-located with the writeup).
1. Create `examples/<category>/XXXX-<category>-<name>.sx` (focused example)
**or**, for an open bug, `issues/NNNN-slug.{md,sx}` (repro co-located with
the writeup).
2. Run it: `./zig-out/bin/sx run <path>.sx`
3. Seed the marker (`: > <root>/expected/<name>.exit`) and capture expected:
3. Seed the marker (`: > <dir>/expected/<name>.exit`, where `<dir>` is the
example's category folder or `issues/`) and capture expected:
`zig build test -Dupdate-goldens`
4. Verify: `zig build test`
@@ -569,8 +580,8 @@ All Zig unit tests live in separate `*.test.zig` files alongside the source they
When a bug filed under `issues/NNNN-slug.{md,sx}` is fixed:
1. Move the repro into the feature suite as a regression test:
`git mv issues/NNNN-slug.sx examples/XXXX-<category>-<name>.sx`.
2. Seed `examples/expected/XXXX-<category>-<name>.exit`, capture with
`git mv issues/NNNN-slug.sx examples/<category>/XXXX-<category>-<name>.sx`.
2. Seed `examples/<category>/expected/XXXX-<category>-<name>.exit`, capture with
`zig build test -Dupdate-goldens`, and review the diff.
3. Tighten the example's comment header to describe the feature (keep a one-line
`Regression (issue NNNN)` note for provenance).

Some files were not shown because too many files have changed in this diff Show More