test: run example corpus in zig build test; sx ir → stdout
`zig build test` now runs the full examples/ + issues/ regression corpus alongside the Zig unit tests, driven by a pure-Zig test (src/corpus_run.test.zig) — no shell script in the build path. It spawns the installed `sx` per example (subprocess-isolated, per-run timeout), diffs stdout/stderr/exit and optional `sx ir` snapshots, and fails the build on any mismatch. The file list is enumerated at runtime, so new examples are covered with no test edit. - `sx ir` / `ir-dump` now write to stdout (fd 1) instead of stderr, so the dumps can be piped/redirected. - `zig build test -Dupdate-goldens` regenerates snapshots in-build, byte-identical to the legacy `run_examples.sh --update`; on mismatch the runner prints how to regenerate. - run_examples.sh kept (still used by tools/verify-step.sh) and made portable to a bare macOS: timeout/gtimeout fallback, bash 3.2-safe empty-array handling. - CLAUDE.md: document the new workflow.
This commit is contained in:
@@ -31,6 +31,28 @@ if [[ "${1:-}" == "--update" ]]; then
|
||||
UPDATE=1
|
||||
fi
|
||||
|
||||
# Per-test wall-clock guard. GNU `timeout` (or `gtimeout` from Homebrew
|
||||
# coreutils) kills a hung test after $TIMEOUT seconds. Neither ships on a
|
||||
# bare macOS, so degrade gracefully: when no timeout binary is found, run the
|
||||
# command directly (a hang then blocks the suite, but the suite still works).
|
||||
TIMEOUT_CMD=()
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
TIMEOUT_CMD=(timeout "$TIMEOUT")
|
||||
elif command -v gtimeout >/dev/null 2>&1; then
|
||||
TIMEOUT_CMD=(gtimeout "$TIMEOUT")
|
||||
fi
|
||||
|
||||
# Run a command under the timeout wrapper if one is available, else directly.
|
||||
# The length check (not "${arr[@]}") keeps this safe under bash 3.2 + `set -u`,
|
||||
# where expanding an empty array trips "unbound variable".
|
||||
run_sx() {
|
||||
if [[ ${#TIMEOUT_CMD[@]} -gt 0 ]]; then
|
||||
"${TIMEOUT_CMD[@]}" "$@"
|
||||
else
|
||||
"$@"
|
||||
fi
|
||||
}
|
||||
|
||||
# Normalize stdout/stderr for snapshot diffing. Applied identically to both
|
||||
# expected and actual, so it can only reconcile location/host noise — never
|
||||
# desync an otherwise-matching pair. The path rule collapses any absolute
|
||||
@@ -75,7 +97,7 @@ for root in "${ROOTS[@]}"; do
|
||||
fi
|
||||
|
||||
printf " %-48s" "$name"
|
||||
actual_out=$(timeout "$TIMEOUT" "$SX" run "$sx_file" 2>"$TMP_ERR" | normalize)
|
||||
actual_out=$(run_sx "$SX" run "$sx_file" 2>"$TMP_ERR" | normalize)
|
||||
actual_exit=${PIPESTATUS[0]}
|
||||
actual_err=$(normalize < "$TMP_ERR")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user