`zig build test` now runs the full examples/ + issues/ regression corpus
alongside the Zig unit tests, driven by a pure-Zig test
(src/corpus_run.test.zig) — no shell script in the build path. It spawns
the installed `sx` per example (subprocess-isolated, per-run timeout),
diffs stdout/stderr/exit and optional `sx ir` snapshots, and fails the
build on any mismatch. The file list is enumerated at runtime, so new
examples are covered with no test edit.
- `sx ir` / `ir-dump` now write to stdout (fd 1) instead of stderr, so
the dumps can be piped/redirected.
- `zig build test -Dupdate-goldens` regenerates snapshots in-build,
byte-identical to the legacy `run_examples.sh --update`; on mismatch
the runner prints how to regenerate.
- run_examples.sh kept (still used by tools/verify-step.sh) and made
portable to a bare macOS: timeout/gtimeout fallback, bash 3.2-safe
empty-array handling.
- CLAUDE.md: document the new workflow.
THREADSAFE=0 was correct when sx had no threads; with std.thread (S6)
and std.http's pooled dispatch (S7b), concurrent connections corrupted
sqlite's unprotected globals (caught live: distd under ab -c20 died
with free-of-unallocated inside yy_reduce). Serialized mode is
sqlite's own default and safe for every consumer; per-connection use
across threads is the supported pattern.
thread_pool_count = 0 (default) keeps handlers inline on the loop
thread — the measured fast path (BENCH-HTTPZ.md). N > 0 dispatches
each parsed request to a std.thread Pool of N workers, completing the
httpz two-pool shape: the connection freezes as CONN_HANDLING (no
reads, growth, eviction, or recycling — the worker borrows views into
its read buffer), the worker runs the handler under a per-job arena
and serializes into job-owned bytes, the completion queues under the
PoolState mutex, and the loop wakes through the new std.event wake
channel (kqueue EVFILT_USER + EV_CLEAR; the epoll twin maps to
eventfd), attaches the response, compacts the buffer, and resumes
keep-alive/pipeline handling. A full backlog sheds with 503. Stale
completions (generation mismatch after close) are dropped. Pool mode
requires the server's constructing allocator to be thread-safe
(GPA/malloc), documented on the knob.
PoolState lives behind a heap pointer (it embeds a Mutex and is shared
with workers; the Server struct itself is returned by value).
serialize_response/run_handler_job share one serialize_bytes.
examples/1633 gains the pooled section (GET, body echo, 404 across
worker threads) plus the loop-wake path exercised end to end; AOT run
five times. examples/1632 unchanged but the Event struct gains `user`.
pthread bindings with darwin opaque sizes (mutex 64B, cond 48B; glibc
divergence is a C3 per-OS item). Mutex/Cond initialize IN PLACE and
Pool lives behind Pool.create's heap pointer — POSIX sync objects are
address-sensitive, so nothing here moves after setup. Thread.spawn
takes the C2 re-entry contract entry (callconv(.c), fabricates its own
Context); Pool workers do exactly that with a per-worker malloc-backed
GPA, then run default-conv tasks inside it. submit returns false on a
full backlog (httpz thread_pool backpressure); shutdown finishes
queued work and joins every worker.
examples/1637 pins: 4 raw threads x 1000 locked increments, 100 pool
tasks summing exactly once across 4 workers, a held worker + full
backlog refusing the next submit, clean shutdown. JIT + AOT (AOT run
three times). The std.sx barrel carries thread; .ir snapshot regen is
the usual renumbering.
Both halves of the C2 contract already work in JIT and AOT; these
examples pin them. 1635: libc qsort drives an sx callconv(.c)
comparator passed by name as a typed fn-pointer param. 1636: a real
pthread enters sx through a callconv(.c) entry, fabricates its own
Context (push Context with a local GPA), and runs default-conv sx code
that allocates through it — the re-entry contract std.thread (S6)
stands on. Also unblocks the sqlite callback APIs (hooks/UDFs) left
unbound by design in P5.1.
emitProtocolDispatch now requires the user-arg count to equal the
protocol method's parameter list — exact, since protocol signatures
have no defaults, packs, or variadics — and emits the same
"expects N arguments, but M were given" diagnostic plain calls get.
Previously extra args were silently dropped (and missing args left the
thunk reading garbage). The dispatch gains the call-site span for the
diagnostic. examples/1634 pins the rejection; full sweep confirms no
existing code relied on the leniency.
The protocol declares dealloc_bytes(ptr) — the size argument I passed
at three sites was silently accepted and dropped by the compiler
(issue 0131); these calls would stop compiling the moment that
diagnostic gap is fixed.
No conjured GPA: the arena chunks come from own_alloc (captured at
Server.init), so all server memory flows from the allocator the app
constructed it with — the point of the implicit context model.
Handler and serialization allocations through the implicit context die
with the request; response bytes survive via the own_alloc copy made
inside the push scope. Without this every request leaked its render
concats into the loop's long-lived context.
read_buf_cap is now the per-request LIMIT, not a preallocation: slots
start at 16K, double when full (one-step sizing when a Content-Length
declares the body), and keep their grown capacity for slot reuse. At
the limit the refusal distinguishes oversized headers (431) from an
oversized body (413). Unblocks A1: distd accepts multi-hundred-MB
artifact uploads — preallocating that per slot was never an option.
examples/1633 adds a body past the initial capacity echoing intact.
Server.init(cfg, handler, ctx); the handler signature gains a usize
third argument delivered verbatim per dispatch — typically a pointer
to the app's own state, since the server owns the call site. A bare
(req, resp) handler had no way to reach app state without globals.
examples/1633 pins the round trip.
The httpz shape, one worker, handlers inline over the std.event Loop:
nonblocking accept, per-connection state machine (reading -> writing ->
keepalive/close) with incremental parsing (request line, headers,
Content-Length body), partial-write continuation via on-demand write
interest, pipelined-request draining, and timeouts as EVICTION —
request-delivery and keepalive-idle deadlines on the monotonic clock,
checked after I/O each tick. Keep-alive is the HTTP/1.1 default;
Connection header, HTTP/1.0, or the per-connection request_count cap
turn it off. Config mirrors httpz: port/backlog/max_conn/read_buf_cap/
timeout_request_ms/timeout_keepalive_ms/request_count.
API: Server.init(cfg, handler) + tick(max_wait_ms); run() is the
forever-tick loop. tick makes the server drivable single-threaded —
examples/1633 runs a live server and its client sockets in ONE thread,
pinning: GET with keep-alive, actual connection reuse, the request cap
answering Connection: close then EOF, POST body echo, 404 routing, and
a half-header client evicted at the request deadline while a healthy
client keeps being served. Verified under sx run AND sx build.
Connection slots and read buffers are reused across connections
(httpz's min_conn/buffer-pool spirit); response buffers are allocated
per response and freed on completion. Serialization happens while
request views are valid, the served bytes are compacted, and only then
does sending start — write_more's pipelining check must see only the
remainder. The std.sx barrel carries http; .ir snapshot regen is the
usual mechanical renumbering.
S7b adds worker counts + the handler thread pool (needs C2/S6); the
epoll backend activates with the linux target (S4/S7c).
Loop.init/close, add_read/del_read/add_write/del_write with a
per-registration udata word, and wait() normalizing backend events
into Event{fd, udata, readable, writable, eof, err, nbytes}. The epoll
twin (S4) slots in behind this surface when the linux target lands.
No timer registrations by design: request/keepalive eviction is
deadline math — deadline_in/expired/remaining_ms over std.time's
monotonic clock, with remaining_ms feeding wait's timeout. std.sx
barrel carries ; .ir snapshot regen is the usual mechanical
renumbering. examples/1632 pins idle timeout (and that it honors the
deadline), readable with fd/udata/nbytes, immediate writability on an
empty send buffer, and the eof flag on peer close; JIT + AOT.
32-byte darwin struct kevent, EVFILT_READ/WRITE/TIMER, EV_* flags, and
three thin helpers: kev_change (one registration entry), kq_apply
(immediate change, no drain), kq_wait (bounded drain, EINTR reissued,
negative timeout = forever). Off the std.sx barrel by design — the
OS-neutral facade over this and the epoll twin is std.event (S5).
examples/1631 pins zero-cost idle timeout, READ readiness with pending
byte count + udata round-trip, and EV_EOF on peer close; verified under
sx run AND sx build.
set_nonblocking (C-variadic fcntl), errno via __error (darwin; C3
selects per-OS), and accept_nb/read_nb/write_nb returning a typed
SockErr — WouldBlock / Closed / Fault — so readiness-loop callers never
parse -1/errno pairs. EINTR retries internally; accept_nb skips
ECONNABORTED. Adds connect, shutdown, socketpair, AF_UNIX, SHUT_*.
examples/1630 pins the result algebra on a socketpair and a nonblocking
TCP listener (WouldBlock on empty backlog, accept after loopback
connect); verified under sx run AND sx build. The .ir snapshot regen is
mechanical: new std decls shift @str/@tag.str numbering and grow the
type table (179 -> 185).
now_secs (CLOCK_REALTIME, epoch seconds) and mono_ms (CLOCK_MONOTONIC,
process-local milliseconds for deadlines). Clock ids are darwin's; the
per-OS selection mechanism is PLAN-HTTPZ C3. No error channel: with
module-constant clock ids and a stack timespec, clock_gettime is total.
std.sx namespace tail carries the time alias; examples/1629 pins epoch
plausibility, monotone advance, and the alias carry.
The JIT path already guards its object cache with hasTopLevelRun (the
#run interp executes during codegen; a cache hit skips codegen and
loses its effects). The build path had no such guard, so a second
'sx build --cache' of any app with a '#run configure_build()' block
linked WITHOUT the build.sx config — no link flags (m3te: undefined
SDL3 symbols), and on a binary-level hit the output path and bundling
would have been wrong too. Both cache levels and both save sites now
share the guard; #run-free programs keep full cache behavior
(verified: second build hits the binary cache in <1ms; m3te's
build/--cache/build sequence now links and bundles both times).
All units share one link namespace (per-unit isolation is PLAN-C C3.2,
deferred), so a symbol defined by two units previously died inside the
JIT dylib link or the AOT link with raw linker spew. The clang shim
gains sx_clang_object_exported_symbols (llvm::object scan: defined +
global, format-specific excluded) and compileCToObjects cross-checks
every unit object — collisions name both source files. Scan failures
are non-fatal; the linker remains the backstop. Covers JIT and native
AOT; the emcc path still relies on wasm-ld's own error.
compileCWithEmcc now probes/saves .sx-cache/c-<key>.o with the same
content key as the native path (source + declared headers + transitive
deps + defines/flags/incdirs), keyed by the emcc --version line and the
wasm triple so emsdk upgrades and wasm32/64 variants never collide with
each other or with native objects. Cache hits hand the linker the cache
path directly. objectMagicOk accepts the wasm magic. Verified: warm
wasm build of a c-unit drops 1.85s -> 0.61s (emcc -c skipped).
The key previously covered the #source bytes + the block's DECLARED
headers, so a unit whose impl is a thin wrapper over an undeclared
header (vendors/kb_text_shape: two-line impl.c, all code in
kb/kb_text_shape.h) would serve STALE cached objects after an
upstream upgrade. collectIncludeDepBytes now walks the transitive
closure of quoted #include lines (includer-dir first, then -I dirs;
angle/system includes never participate; unresolvable names skip) and
the dep contents fold into the key — no sidecar, no compare logic, a
changed header is just a different key. Verified live: appending to
kb_text_shape.h mints a new cache entry; reverting hits the old one.
kb_text_shape (v2.10, JimmyLefevre) had been LOST from the sx tree —
ffi/stb_truetype.sx referenced repo paths that no longer existed (and
nothing runs glyph_cache, so the dangling unit never fired). The
trimmed copy returns from the m3te project as a proper vendor:
curated c/kbts_api.h decls over the full upstream header, README with
provenance, and examples/1627 pinning context + font creation so the
unit compiles and runs in-suite. file_utils (in-house asset-read
helper with the Android AAssetManager hook) gets the same unit shape.
modules/ffi/stb_truetype.sx is gone: glyph_cache imports the three
vendored units (stb_truetype, kb_text_shape, file_utils) directly.
The stb headers move from the repo-root vendors/ (resolvable only
with CWD = sx repo) into library/vendors/ following the sqlite
convention — bindings module + c/ sources + provenance README — so
'#import "vendors/stb_image/stb_image.sx"' (image v2.30 + image_write
v1.16) and '#import "vendors/stb_truetype/stb_truetype.sx"' (v1.26)
work from any consumer via the stdlib search paths. modules/ffi/stb.sx
dissolves into the stb_image vendor; modules/ffi/stb_truetype.sx keeps
its non-stb text-shaping companions and re-imports the vendored unit.
examples/1625 pins a deterministic in-memory BMP decode; examples/1626
pins font init + metric invariants against the system Helvetica.
The vendored amalgamation (3.53.2, public domain) plus the curated
bindings move from the distribution repo into the sx library:
'#import "vendors/sqlite/sqlite.sx"' gives any sx program SQLite
with no system dependency and no build flags — the bindings declare
the C as a named #import c unit (pinned defines + -O2), compiled
through the object cache and shadow-proof via unit-first resolution.
examples/1624 pins the version and a typed round trip in-suite.
One module imported through several aliased chains materializes one
c_import_decl copy per chain, each carrying a differently-spelled
relative path to the same file (src/app/../repo/../db/../../vendor/x.c
vs src/db/../../vendor/x.c). Dedup now keys on lexically-normalized
sources/includes + defines + flags, so the unit compiles and links
exactly once — pointer-identity dedup linked it once per chain and
died with duplicate symbols at AOT link.
A named #import c unit declared inside an aliased module sits two
namespace levels deep in the merged tree; the one-level walk (the
extractLibraries/0130 pattern in c_import form) never collected it,
so the unit silently never compiled and its symbols resolved from
whatever process image carried the same names — surfaced by C4's
sqlite migration, where only the version pin could tell the OS copy
from the vendored one.
validateForeignRefs walks the merged tree (libraries + named c units,
nested namespaces included) and diagnoses any #foreign whose ref names
neither — a typo'd ref previously compiled and resolved silently
through whatever image carried the symbol. Decls synthesized from
#include headers carry no ref and are exempt. Flips the C0.2b pin;
zero collateral across the 608 other examples.
runJITFromObject now takes priority dylibs (the #import c unit's
linked objects first, then #library deps in declaration order) and
attaches a per-path search generator for each AHEAD of the
process-wide fallback, so a vendored symbol can never lose to a
same-named export of an image the host process happens to carry
(libz via LLVM, libsqlite3 via CoreServices). loadLibrary reports
the name dlopen succeeded on; the c-import handle records its dylib
path; temp link inputs are per-pid so concurrent runs can't clobber
each other. Flips the C0.3 shadowing pin to from_unit: true.
compileCToObjects now probes .sx-cache/c-<key>.o before invoking the
embedded clang and writes fresh objects back (per-pid temp + copy, the
main object cache's pattern). Default on for both JIT and AOT — the
temp-compile-and-delete behavior it replaces was strictly worse. A
cached entry must carry an object-file magic (Mach-O/ELF) or it falls
back to a fresh compile; no cache failure can fail a build. Cold/warm
verified via --time: the object compile disappears on the warm run.
Source bytes, declared-header CONTENT (header edits invalidate),
defines/flags/include dirs in order, LLVM version, and target
triple/sysroot all participate; section tags keep equal strings in
different roles distinct. Pure function + variance property tests;
nothing consumes it yet.
extractLibraries/extractFrameworks walked the merged root plus exactly
one namespace_decl level, so a #library reached through two or more
aliased imports never made it to the AOT link line or the JIT dlopen
list. Both walks now recurse over namespace_decl children.
Regression: examples/1617-modules-library-nested-namespace.sx binds
libpcap (not in the compiler's loaded images, so the JIT cannot mask
the miss via RTLD_DEFAULT) behind two aliased imports.
cstring is ONE pointer to a null-terminated u8 buffer, C's char*: thin
(8 bytes, no length; cstring_len walks to the terminator), crossing
#foreign boundaries verbatim in both directions, with ?cstring as the
nullable case lowering to the same bare pointer (null = absent).
Conversion discipline mirrors Odin: a string LITERAL coerces implicitly
(its bytes are terminated constants); any other string is rejected with
a diagnostic naming to_cstring (it may be an unterminated view); and
cstring never coerces to string implicitly — from_cstring(c) is the
explicit zero-copy view, pricing the strlen.
Plumbing: TypeId/TypeInfo builtin slot 18 (first_user 19), name
classifiers, size/align/name tables, LLVM ptr lowering, the ?T pointer
niche, the xx pointer ladder, the literal-gated coercion plan
(isConstString + data_ptr), and the reserved-spelling set. std gains
cstring_len/from_cstring/to_cstring (fmt.sx, re-exported); the old
cstring(size) allocator helper is renamed alloc_string everywhere;
getenv migrates to (name: cstring) -> ?cstring as the canonical user
and env() drops its manual strlen/memcpy.
Pinned: examples/1222 (FFI both directions, literal coercion,
?cstring null paths, round trip) and examples/1173 (both coercion
diagnostics); FAIL pre-feature. The alloc_string rename + getenv
signature shift the .ir snapshots — regenerated. zig build test
426/426; run_examples 604/604.
Spec: reserved spelling + cstring section + C-interop rows.
Two genuine defects behind the 0128 filing (whose original repros were
both poisoned by binding getenv, which std already declares -> *u8):
1. Re-declaring a C symbol was silent first-wins: every call through
the later declaration was typed by the older signature. Foreign
registration now dedupes — equal signatures share one FuncId,
conflicting ones are diagnosed.
2. Foreign -> string / -> ?string returns read garbage: C returns one
char*, but the LLVM signature declared the fat {ptr,i64} (len =
register garbage), and ?string was mis-declared SRET (the hidden
out-pointer landed in the callee's first arg register). cstrRetKind
now classifies such returns, declares them as plain ptr (never
sret), and the call site synthesizes {ptr, strlen} via a
branch-guarded strlen (NULL -> {null,0} / optional null), wrapping
{string, i1} for ?string.
?[:0]u8 itself resolves fine (it is ?string); the spelling works in
return, param, local, and alias positions.
Regression: examples/1221 (plain + optional non-null + NULL paths) and
examples/1172 (conflict diagnostic); both FAIL pre-fix. The extern
dedupe collapses duplicate libc decls, so affected .ir snapshots were
regenerated. zig build test 426/426; run_examples 602/602;
distribution suite 21/21.
The unary .not arm emitted bool_not (LLVM bitwise Not) for every
operand. Correct on i1; on an error binding — an error-set value, u32
tag at the LLVM level — a bitwise not of a nonzero tag stays nonzero,
so 'if !e' held even on a SET error and its branch read the
uninitialized success value (real segfault in the distribution repo's
sqlite tests). Plain integers had the same hole ('!7' was '~7').
Now: bool keeps bool_not; integers and error-set operands lower as the
truthiness complement (cmp_eq against a typed zero); anything else is
diagnosed instead of silently bit-flipped.
Regression: examples/1057 (set error: !e must not hold; success: !e
holds with a real value; integer truthiness) + examples/1171 (!"text"
diagnosed); both FAIL pre-fix. zig build test 426/426;
tests/run_examples.sh 600/600.
[:0]u8 aliases string (fat) and params already ABI-thin to char*, but
a foreign -> [:0]u8 return silently resolves to plain u8, and ?[:0]u8
never resolves at all (LLVM emission panic) even though ?string works.
Design contract recorded: ?[:0]u8 lowers to a nullable char* at the
boundary, length synthesized on the sx side; until then such returns
must be diagnosed, not mis-typed.
lowerEnumLiteral resolved the variant against the raw destination type,
so any non-enum destination fell into resolveVariantValue's silent
return-0 tail with the enum_init stamped as the wrong type:
- ?E destinations produced variant 0 mis-typed as the optional
(observed as variant 0 OR null, layout-dependent);
- builtin destinations (i64) silently became 0;
- unknown variants of real enums silently became variant 0;
- a destination-less literal panicked LLVM emission (unresolved
type reached codegen).
Now: optional destinations unwrap to the child enum (the coercion
layer's .optional_wrap handles E -> ?E), and the remaining shapes are
diagnosed — unknown variant (with the variant list, via the new
emitBadEnumVariant twin of emitBadVariant), non-enum destination, and
no destination (cascade-guarded: silent when the destination's type
already failed to resolve and was reported).
Regression tests: examples/0183 (return/assign/reassign into ?Enum,
non-zero variants, null path) + examples/1169/1170 (each diagnostic);
all three FAIL on pre-fix master. zig build test 426/426;
tests/run_examples.sh 598/598.
Surface rename of the signed integer family: s1..s64 become i1..i64
(u1..u64, usize, isize unchanged). 'string' keeps the s-prefix arm in
name classification; width parsing moves to the i-prefix arm next to
isize.
Internal TypeId tags follow the surface (.s8/.s16/.s32/.s64 ->
.i8/.i16/.i32/.i64), as do mono-key mangle fragments (ptr_i64,
tu_i64_bool) and all display/diagnostic formatting (i{d}).
Migrated in the same sweep: stdlib + examples + issue repros + FFI C
companions (shared symbol names like ffi_id_i64), expected
stdout/stderr/ir snapshots, specs.md, readme.md, CLAUDE.md/AGENTS.md,
implementation_plan.md, docs/, issue writeups. Vendored stb_image and
historical flow state left untouched.
zig build test: 426/426; examples suite: 595/595.
The plan producer's namespace-fn arms returned the declared return type
without checking type_params, so a qualified generic call's result
carried the unbound T stub: print boxed it as 'T{}', and a non-s64
binding failed LLVM verification (pack monomorphized for the stub,
call returning double). Both fn_ast_map-backed arms now classify
generic callees as generic_fn and infer the return through
inferGenericReturnType, mirroring the bare-identifier path.
extractTypeParam's slice arm only extracted from slice-typed args, so
first(a) with a : [3]s64 at first :: (xs: []$T) -> T left T unbound
and the mono body reached LLVM emission carrying the .unresolved
sentinel (panic). The arm now also extracts from array args via the
array's element type — mirroring the array→slice promotion concrete
slice params already perform; the existing arg coercion handles the
rest.
lowerGenericCall additionally diagnoses any still-uninferrable TYPE
param at the call site instead of monomorphizing unbound — the
deliberate string-at-[]$T gap used to hit the same sentinel panic and
now errors with a source-located message. Comptime value params
($N: u32) and ..$Ts packs bind through their own dispatch and stay
exempt.
Regressions: examples/0212-generics-array-arg-slice-param.sx (scalar /
u8 / struct elements + the slice spelling) and
examples/1168-diagnostics-generic-param-uninferrable.sx (string arg
diagnostic) — both failed pre-fix.
Two lowering sites materialized a local array as a whole LLVM value;
the legalizer scalarizes each such op into one SelectionDAG node per
element, and at ~64K elements the DAG combiner segfaults
(DAGCombiner::visitMERGE_VALUES → ReplaceAllUsesWith).
- lowerVarDecl: an array-typed `---` initializer emits NO store — the
slot stays uninitialized instead of receiving a whole-array undef
store. The tuple zero-init carve-out stays; non-array `---` keeps
the undef store. The interp is unchanged either way (slots start
.undef).
- lowerIndexExpr: element reads on an array with addressable storage
GEP the storage and load one element — the general-expression
sibling of 0110's lowerFor fix — without value-lowering the object
(a dead whole-array load would still reach the DAG). Storage-less
arrays keep the index_get fallback.
Sibling shape filed as 0125: any_to_string's per-array-type arms still
pass the array by value, so a 64K+ array type + any {} print crashes.
Regression: examples/0055-basic-large-stack-array.sx (sx build
segfaulted pre-fix). 22 .ir snapshots re-pinned: removed undef stores
and ig.tmp spills, in-place gep+load (instruction-shape-only churn,
reviewed).
