Phase 5 of PLAN-IO-UNIFY retired the bespoke fiber-task API (Task/go/wait/cancel)
and re-homed race onto *Future. Prepend SUPERSEDED banners to the three historical
planning docs (PLAN-FIBERS, CHECKPOINT-FIBERS, PLAN-RACE) pointing readers to
PLAN-IO-UNIFY for the current design, noting that the Scheduler engine primitives
+ the race type-machinery they document remain accurate. Bodies kept as the
historical record. Completes the plan's 'update roadmap/checkpoints' item.
Verified by parallel agents: a completeness pass (no retired API in compilable
code; sched.sx surface fully retired; migrated examples meaningful; readme on the
unified API) and an adversarial memory-safety pass over the heap reclamation (no
UAF/double-free; await/worker orderings, cancel timings, 8-way double-free+deinit,
blocking impl all reclaim cleanly with exact values).
The canonical sx block-body lambda is `(params) { stmts }` (and
`(params) -> Ret { stmts }`); the arrow form `=>` is for EXPRESSION bodies
(`(params) => expr`). The arrow-block hybrid `(params) => { .. }` was being
used in 33 files — convert all of them by dropping the `=>`. The two forms are
exactly equivalent (verified: identical IR and identical runtime values — the
block tail is the value with or without a `-> Ret`), so this is a pure source
cleanup: no `.ir` churn, and the only snapshot change is 0923's diagnostic
COLUMN (a negative narrowing test whose error span shifted by the removed `=> `).
Arrow EXPRESSION bodies (`=> expr`, `=> .{..}`, `=> [..]`) and `=>` inside
comments/strings were left untouched. Migrated across examples/concurrency,
examples/{closures,ffi-objc,generics,optionals,types}, issues/, and the stdlib
(io.sx, sched.sx). Suite 855/0.
Closes the documented per-spawn closure-env leak and most of the async leak,
using only the existing closure.env / closure.fn_ptr field accessors — no compiler
change. Also names the fat-pointer ABI in core.sx (ClosureRaw / SliceRaw) so the
underlying {fn_ptr, env} / {ptr, len} layout is discoverable in one place.
- Fiber body env: Scheduler.reap_fiber frees f.body.env via f.dctx.allocator (the
spawn-time allocator snapshotted in dctx) at all three reap sites (run/poll/
deinit). 1820's 'live after deinit' 3 -> 0.
- Async box + closure envs: sx_run_boxed_closure frees the ThunkBox, the
completion-closure env, and the worker's env (new ThunkBox.worker_env) the
instant the worker completes.
- Async Future: two-flag ownership — Future.worker_done (set at the end of the
completion closure) + consumed (set at the end of await); fut_release frees the
heap Future (via the captured Future.alloc) when BOTH are set, so the LAST of
{worker, await} reclaims it. await now CONSUMES the future (single-use; touching
it afterward is a use-after-free — documented). Residual for an AWAITED future
is 0 (lock: examples/concurrency/1827); a never-awaited future (fire-and-forget /
race loser) keeps only its Future struct — the structured-concurrency remainder.
Self-reviewed across orderings (await-after/before-complete, cancel-then-await,
cancel-while-parked, double-free via await+deinit, race residual, blocking impl,
cross-allocator reap) — all deterministic, no UAF/double-free. Suite 855/0;
byte-identical on aarch64-macOS + aarch64-linux; .ir churn is the core.sx +
Future/ThunkBox field additions.
Converge the Io unification (PLAN-IO-UNIFY Phase 5). The bespoke fiber-task layer
in sched.sx — Task / TaskState / TaskErr / go / wait / cancel(Task), plus
Scheduler.task_allocs and its deinit bookkeeping (~130 lines) — is removed. There
is now ONE async stack: context.io.async / await / cancel / race / sleep over the
Io protocol, with the Scheduler as the fiber Io's engine + driver (spawn /
yield_now / suspend_self / wake / run / block_on_fd remain as the raw primitives;
race stays in sched.sx because it needs meta.sx's make_enum/make_variant).
Migrated the four go/wait users to context.io:
- 1813 — interleave + cancel (sequence 1 2 3 42 100 -99)
- 1817 — m1 end-to-end (completion in deadline order, sum 123)
- 1819 — double-AWAIT loud-abort via the Future one-awaiter guard
- 1820 — deinit: dropped the go/task_allocs tasks; now exercises timers/io_waiters/
kq cleanup (freed=2, live=3 = the documented per-spawn closure-env residual)
Updated readme.md (the user-facing async section documents context.io.async /
await / race / sleep) and the stale sched.go/sched.Task comments in io.sx.
Suite 854/0; no .ir churn (Task removal touched no snapshotted IR); migrated
examples byte-identical on aarch64-macOS + aarch64-linux. PLAN-IO-UNIFY Phases 0-5
all complete — the two parallel async stacks are now one, behind context.io.
Re-home the proven first-wins race from sched.race(*Task) onto *Future handles
+ the Io protocol; the old Task-based race is REPLACED (ufcs overload-by-receiver
is rejected, and only 1821 used it).
- Protocol: add Io.current_park() -> ParkToken — the running fiber as a token,
captured WITHOUT parking — so race can register the SAME coordinator across N
futures' park slots, then park once via suspend_raw; any completion readies it.
Scheduler returns {self.current} (bails outside a fiber); CBlockingIo returns
{null} (race never parks there — futures are born .ready).
- race :: ufcs (io: Io, futures: $T) -> RaceResult(T), kept in sched.sx (it needs
meta.sx's make_enum/make_variant; pulling that into the io.sx prelude part-file
would cycle). Winner scan -> register/park/deregister -> make_variant the winner
-> Phase-3 cancel each still-.pending loser (no join). RaceResult reused
unchanged (*Future(R) projects field 0 'value' -> R).
- TRUE-cancel: parked losers stop at their next suspend (timers evicted by cancel's
wake), so race returns at WINNER-time, not slowest-loser-time.
- Adversarial review fixes: (1) an all-failing/all-cancelling racer set no longer
deadlock-aborts the scheduler — race bails loudly ('all futures settled without
a winner') when nothing is .ready and nothing is still .pending; (2) only
.pending losers are cancelled, so a loser that already .failed keeps its real
outcome label instead of being stomped to .canceled.
Re-point 1821 to context.io.async + context.io.race (winner a=111, losers
.canceled, completion log only 'task 1 @ 10ms', final clock 10ms — was 30 under
the old cooperative join). New 1826 locks the failing-loser case. Byte-identical
on aarch64-macOS + aarch64-linux. Suite 853/0; .ir churn is the current_park
vtable method.
A cancelled async worker now abandons its body at its next suspend instead
of running to completion.
- Cancel-flag back-ref (D4): SpawnOpts.cancel_flag (core.sx) + Fiber.cancel_flag
(sched.sx), set from opts.cancel_flag in Scheduler.spawn_raw; async passes
xx @f.canceled (the Future.canceled Atomic(bool) erased to *void).
- Delivery: Scheduler.suspend_raw consults fiber_canceled(self.current) PRE-park
(raise without parking — no deadlock if cancel landed before the worker ran)
and POST-resume (cancel landed while parked), raising error.Canceled.
cancel(f) flips the sticky flag, marks .canceled, and wakes the worker.
- async worker is failable Closure() -> ($R, !); the completion closure
f.value = worker() catch {…} marks .canceled/.failed and wakes the awaiter,
so post-suspend side effects never run. New failable io.sleep(ms) is the
cancellation point.
- Compiler: a -> ! fn whose only error source is try-ing a protocol method
(io.suspend_raw) was wrongly flagged 'declared ! but never errors';
collectErrorSites now marks a try of a non-identifier callee as a dynamic
(opaque) error source, suppressing the warning.
- Two UAFs found by adversarial review and fixed: (1) cancel-before-park
orphaned io.sleep's armed timer — suspend_raw's pre-park raise now evicts the
current fiber's timer/waiter first; (2) cancel(f) could wake a reaped worker —
now only wakes when was_pending.
Migrated 1805/1806/1824 to failable workers. Lock: example 1825 (seq: 1 -99,
post-suspend line never runs); byte-identical on aarch64-macOS + aarch64-linux.
.ir churn is the SpawnOpts layout change (type-table string renumbering).
Calling a closure or function-pointer value stored in a struct data field
(`box.run(args)`) typed the call as 'unresolved': value returns marshalled
as garbage, failable fields could not be try/catch-ed. Lowering already
dispatched these (call_closure / call_indirect); only CallResolver.plan
lacked a field-access arm. Add a closure/fn-pointer field arm to plan
(before the instance-method check, mirroring lowering's precedence — a
closure-typed field shadows a same-named method) and extend the lowering
closure-field arm to also handle bare .function fields via call_indirect.
Lock: examples/closures/0315-closures-struct-field-call.sx.
collectCaptures did not descend into catch/try/onfail/raise/multi_assign/
push/comptime/insert/spread/asm nodes, so a free variable referenced only
inside them (e.g. a failable worker called as `worker() catch {…}` in a
nested lambda) was never captured into the env struct — inside the lambda
it resolved against an empty scope and typed as 'unresolved'. Add the
missing traversal arms. The push_stmt arm also closes the noted
'free-var analysis does not descend into a nested push Context {…}' gap.
Unblocks the PLAN-IO-UNIFY Phase 3 async completion closure shape.
Lock: examples/closures/0314-closures-capture-failable-call.sx.
The trailing-`!`-after-the-value-type spelling (`-> T !`, `-> Tuple(A,B) !`) was a
redundant second way to write a failable return that the parser folded into the
same AST as the parenthesized `(T, !)` / `(A, B, !)` result list. Remove it so
there is ONE canonical spelling: the error channel always rides as the last slot
of the parenthesized list.
- parser: `parseFnReturnType` no longer folds a trailing `!` after a value type —
it rejects it with a located diagnostic ("a failable return is written `(T, !)`
… not `T !`"). This one chokepoint covers fn declarations, lambdas, fn-pointer
types `(A) -> R`, and closure types `Closure(A) -> R`. The error-ONLY `-> !` /
`-> !ErrSet` form is unaffected (parsed by parseTypeExpr as an error_type_expr).
- migrated every usage to canonical form across library/ + examples/ + issues/ +
tests/: `-> T !E` → `-> (T, !E)`; the value-carrying `-> Tuple(A, B) !` (which
FLATTENED to a multi-value failable) → `-> (A, B, !)`, preserving behavior. A
genuine single-tuple-value failable stays `-> (Tuple(A,B), !)`.
- parser unit tests: the "bare form folds" tests become "bare form is rejected";
canonical-form parse tests retained.
- docs: specs.md §12 + scattered refs and readme.md updated to the `(T, !)` form.
Behavior-preserving (the bare form was sugar for the same AST). Adversarial review
confirmed: rejection complete across all positions, every canonical form works on
both success/error paths, error-only `-> !` intact, no crashes. Full suite green
(unit tests + 850 corpus examples).
Type-checking gaps (segfault/corruption → compile errors):
- 0197: reject a store into an annotated slot whose value has no modeled
coercion AND a different byte width (a 16-byte string into a 4-byte i32
overran the slot and segfaulted). New checkAssignable / noneReinterpretIsUnsafe
(coerce.zig, width via the LLVM-accurate typeSizeBytes) wired into every store
site: var/const-decl, single + multi assignment (identifier/field/index/
element/deref), named-return defaults. Same-width reinterpretations (*T→[*]T,
i64→isize, fn-ref) and explicit xx/cast stay allowed; cascades suppressed via
externalErrorsExist. Examples 1205, 1206.
- 0198: an implicit `Any → T` unbox is now a compile error (it blindly
reinterpreted the boxed payload — silent garbage for a wrong scalar, a segfault
for an aggregate). xx and compiler-generated match/pack unboxes are unaffected.
Example 1207.
- 0199: `Any == <concrete>` (one operand Any) aborted the LLVM verifier — the
comparison arm now fires when either operand is Any, boxing the concrete side
first. Example 0654.
Multi-return deferrals (PLAN-MULTIRET #6 + named-order + D3 + generic):
- Reorder named return elements by name instead of requiring slot order; error on
unknown/duplicate/missing (value-only AND full-failable-tuple forms). Examples
0210, 0214.
- Reject a bare-paren (A, B) multi-return signature in generic-arg position
(return-position-only). Example 0215.
- Multi-return closure types / lambda literals work via the reused tuple
machinery (destructure, single-bind+field, lambda arg). Example 0216.
- Generic multi-return: positional works (0217); 0200: the named-slot
implicit-return form now works for generic free fns + struct methods —
monomorphizeFunction now calls bindNamedReturnSlots. Example 0218.
readme.md documents the annotated-store coercion rule; CHECKPOINT-MULTIRET.md
updated. Full corpus green (850/0).
A function may return multiple values via a bare-paren return signature:
`-> (A, B)` / `-> (x: A, y: B)` / `-> (A, B, !)` (error always the last slot),
and `-> ()` is `void`. This is DISTINCT from a `Tuple(…)` value — return-position
only (a dedicated `ReturnTypeExpr` AST node resolving to a reused `.tuple`
TypeId); a parameter / field / variable annotation `x: (A, B)` is rejected. A
single-value `-> (T, !)` stays a plain failable (= `-> T !`).
Returns use the bare comma form `return a, b` / `return x = a, y = b` (no `.( … )`
literal). Consume by destructuring (`a, b := f()`) or single-bind + field access
(`c := f(); c.sum`); a failable bound value holds only the value slots (the error
stays on the `!` channel).
Named return slots are in-scope assignable locals; with no explicit `return` the
implicit return is synthesized from them. Path-sensitive definite-assignment
enforces the must-set rule, and a slot may carry a default that exempts it.
Validation rejects arity mismatches, out-of-slot-order named elements, a
slot/parameter name collision, a comma list from a single-value function, and a
multi-return signature used as a value type.
Examples 0202-0213; readme + specs updated. issues/0197 files a pre-existing
annotated-assignment type-check gap (`x: i32 = "hi"` segfaults) surfaced by the
adversarial review.
- await: add the one-awaiter-per-future guard `sched.Task.wait` has — a second
concurrent `await` on the same pending future would overwrite the single
`park` handle and orphan the first awaiter (silent deadlock). Now aborts
loudly. (Fan-in over SEPARATE futures — `race` — registers one awaiter each,
so it stays fine.)
- Document the Future/ThunkBox ALLOCATOR-LIFETIME contract: both are allocated
from the `context.allocator` in force at `async`, which must outlive the
future (the long-lived-container rule). Calling `async` inside a transient
arena torn down before `run()` is a use-after-free; the common case (program
GPA) is safe. A deeper own-allocator capture is deferred to convergence.
- Document that `cancel` does NOT stop an already-spawned worker (model (a) —
the worker still runs; the sticky `canceled` atomic is the source of truth).
True work-cancellation is Phase 3.
- Drop the dead `f.task = null` (immediately overwritten by spawn_raw).
The new `io_abort` extern shifts the prelude type table — 40 `.ir` snapshots
regenerated (behavior-preserving; no `.exit`/`.stdout`/`.stderr` changed).
Suite 829/0.
`context.io.async(worker)` / `await` now run over the `Io` PROTOCOL, so the
same code interleaves under the fiber scheduler or runs inline under the
blocking `CBlockingIo` — one async stack, reached purely through `context.io`.
- Protocol: `suspend_raw(park: *ParkToken)` (was by-value). A suspending impl
records the parked execution context into `park.handle` before parking, so a
cross-context `ready(park)` knows whom to resume; `Scheduler.suspend_raw`
writes `self.current`, `CBlockingIo` ignores it.
- io.sx async layer rewritten colorblind: `async` submits the worker through
`io.spawn_raw` (inline under blocking, a fiber under the scheduler) and returns
a HEAP `*Future($R)` the worker fills later; `await` suspends via `suspend_raw`
until ready, then returns/raises. The generic worker is bridged to spawn_raw's
raw `(*void)->void` entry via a monomorphic `ThunkBox` (a heap-boxed nullary
completion closure) — all genericity lives in the closure env. Workers are
nullary (inputs captured at the call site) because a variadic pack can't cross
the fiber boundary. `CBlockingIo.spawn_raw` now runs the worker inline.
- Migrated 1805/1806 to the nullary `*Future` form; retrofit 1822/1823 to the
`push .{ … }` partial-context literal (inherits allocator/data).
- The async machinery adds a few prelude types, shifting the type-name table —
40 `.ir` snapshots regenerated (no behavior change; only `.exit`/`.stdout`/
`.stderr` would signal that, and none changed).
Locked by examples/concurrency/1824 — two async tasks under the fiber Io, the
completion log proving deferral (1 2 then 10 20 then 123). Suite 829/0,
byte-identical aarch64-macOS host + aarch64-linux container.
The race (9099735e), Phase 0 (2f2d7f1d), Phase 1 (5c30bfe0) and tuple-store
(6a976287) commits added each example's .sx but their `git add NNNN-*` glob
missed the sibling expected/ snapshots (a different directory). The suite
passed locally because snapshots are read from disk, but the commits were
incomplete — a fresh checkout would have no goldens. Add them now.
Addresses the review of 2f2d7f1d + 5c30bfe0:
- arm_timer: add the null-`current` guard its siblings (sleep/suspend_self/
block_on_fd) all have. Armed from the bare scheduler context it stored a
`fiber = null` Timer that segfaulted `wake` on fire; now it aborts loudly.
- spawn_raw: bail loudly on a null `entry` (a null fn-ptr jumped to 0x0 once
the fiber ran), and document the `arg` lifetime contract (must outlive the
fiber's first run, not just the spawn_raw call).
- poll: abort if io-waiters are pending (a `poll`-only driver can't progress
fd-bound fibers — previously returned -1, indistinguishable from quiescent),
and document `deadline_ms` as reserved/intentionally-unread for the
virtual-time single-step (not a silently-dropped arg).
- fib_dispatch: replace the now-stale comment (which still claimed fibers run
under `__sx_default_context` and "do not inherit a caller-scoped allocator")
with the Phase 0 reality + the dctx lifetime contract: every capability in
the spawn-time context (allocator/io/data) must outlive the fiber.
Behavior-preserving (full suite 828/0); the new aborts only fire on misuse.
`impl Io for Scheduler` folds the M:1 fiber scheduler behind the same `Io`
protocol (core.sx) the blocking `CBlockingIo` implements, so the async layer
can run colorblind over whichever impl is installed via
`push Context { io = xx scheduler }`. The six methods are thin adapters over
the existing fiber primitives:
- `spawn_raw(entry, arg, opts)` — spawn a fiber that calls the erased
`(*void)->void` worker thunk `entry(arg)` (fn-ptr round-trip through
`*void`); returns the `*Fiber` handle. The fiber inherits this context
(Phase 0), so the worker's own `context.io` is this scheduler.
- `suspend_raw(park) -> !` — park the running fiber; the `!` is the
cancellation channel a suspending impl raises on (wired in Phase 3).
- `ready(park)` — `wake` the fiber recorded in the token (guarded on
`.suspended`).
- `poll(deadline_ms)` — one step of the run loop (drain ready + fire the
earliest virtual-time timer); fd-readiness stays on `run`.
- `now_ms` — the deterministic virtual clock.
- `arm_timer(deadline_ms, park)` — arm a virtual-time timer that re-readies
the current fiber.
Locked by examples/concurrency/1823 — two workers spawned + suspended +
resumed entirely through `context.io`, deterministic deadline order
(byte-identical aarch64-macOS host + aarch64-linux container). Full suite
green (828/0).
A fiber body previously ran under the static `__sx_default_context`: the
`abi(.c)` `fib_dispatch` frame has no implicit context param, so a
`push Context { … }` around `spawn` was invisible inside the fiber. That
makes it impossible to fold a fiber scheduler behind `context.io` — a
worker's `context.io.*` would resolve to the blocking default, not the
scheduler that spawned it.
`Scheduler.spawn` now snapshots the live `context` into `Fiber.dctx`, and
`fib_dispatch` re-pushes it (`push self.dctx { self.body() }`) around the
body. So a capability installed before `spawn` (allocator, io, data) is
visible to the worker, and a worker spawned under `push Context { io = … }`
sees that `io` as `context.io`.
Behavior-preserving for fibers spawned under the default context (the
snapshot just re-pushes that same default — full suite green). Context is
parameter-threaded per fiber stack, so interleaved fibers with different
contexts don't leak across the `swap_context` (verified: two fibers with
distinct `context.data` each keep their own across a `yield_now`).
Locks: examples/concurrency/1822-concurrency-fiber-context-inherit.sx
(byte-identical aarch64-macOS host + aarch64-linux container).
`s.race((a: ta, b: tb, …))` takes a named tuple of already-spawned
`*Task(..)` handles, suspends the calling fiber until the FIRST task is
ready, and returns a comptime-synthesized tagged-union (`RaceResult`)
mirroring the tuple's labels — variant NAME = the tuple label, payload =
that task's result type. After picking the winner it CANCELS and JOINS
every loser, so no loser fiber outlives the call (structured concurrency).
- `RaceResult($T) -> Type` projects each `*Task(R)` element to `R` via
`field_type(pointee(field_type(T, i)), 0)` and mints the union with
`make_enum` (the 0649 composition shape).
- `race` Phase 1 registers the caller as waiter on all pending tasks and
parks; on wake it DEREGISTERS from every task (a later loser completion
must never wake it again) and re-scans, lowest-index-first. Phase 2
builds the winner variant with `make_variant`. Phase 3 cancels + joins
each loser one at a time — only the joined loser carries a waiter, so no
other completion can wake the caller mid-join.
- Join correctness rides a new `Task.finished` flag, set at the very end of
the `go` worker body (after the work ran OR was skipped on an early
cancel) and checked before parking, so a worker that finishes between the
cancel and the park can't be lost. Cancellation is cooperative (M:1, no
preemption): a loser parked mid-`sleep` runs to its natural end, its value
discarded — `race` returns only once every loser has `finished`.
The tuple must be NAMED; a positional `._0`/`._1` form is future work.
Locked by examples/concurrency/1821 — three tasks (i64/bool/f64) sleeping
10/20/30ms, shortest wins, losers cancelled + joined; byte-identical on
aarch64-macOS and aarch64-linux (deterministic virtual time).
Completes comptime-cursor tuple indexing (started by the read path in
fee86adf) and unblocks the `race` runtime synthesis. Five enablers:
1. Named-tuple-literal type inference preserves element NAMES. A
`.(a = x, b = y)` passed DIRECTLY as a `$T` argument inferred to a
tuple with `.names = null`, so `field_name(T, i)` reflected "" and a
`make_enum` over those labels collided on the empty name. The typer
now mirrors `lowerTupleLiteral`'s name capture.
2. `inferExprType` resolves a comptime-constant tuple index to the i-th
field's CONCRETE type (the inference sibling of the fee86adf read
path), so `tup[i].field` / methods / comparisons on it resolve.
3. Tuple-element L-VALUES by comptime index — `tup[i] = v`,
`tup[i].f = v`, `@tup[i]` — lower to a typed `structGep` of field i
across all four paths (`lowerAssignment`, the multi-assign store,
`lowerExprAsPtr`, and address-of-index). Previously each emitted an
`index_gep` with a `ptrTo(.unresolved)` element type (a tuple has no
uniform element) that panicked at LLVM emit. An out-of-range comptime
index now diagnoses loudly on every path instead of falling through to
that panic.
4. A user generic `($X..) -> Type` call is recognized as type-shaped
(`isTypeReturningCallNode`), so it can bind a `$E: Type` parameter —
e.g. `make_variant(RaceResult(T), i, …)`. The static
`isTypeShapedAstNode` only knew the type-returning builtins
(field_type/pointee/type_of).
Locked by examples/comptime/0652 (read, fee86adf) and 0653 (store +
address-of + element-pointer field store).
`tup[i]` where `i` folds to a compile-time integer (an `inline for`
cursor or a literal) now reads the i-th tuple field with its CONCRETE
type instead of failing with "cannot index a value of type '(…)'".
A tuple's elements are heterogeneous, so there is no runtime
element-indexing op — a comptime index lowers exactly like the `.N`
field-access path (a `structGet` of the i-th field). A genuinely
runtime index into a tuple value still falls through to the existing
"cannot index a value of type" error (no single element type). A
comptime index out of range gets a dedicated loud diagnostic.
This is the read side `race` needs: pull the i-th `*Task(T_i)` handle
out of a named-tuple param keeping its real type so field/method access
on it resolves.
Locked by examples/comptime/0652-comptime-tuple-cursor-index.sx.
The metatype system can MINT tagged-union types (`make_enum`) and READ them
(`field_count`/`field_name`/`field_type`/`field_value`); `make_variant` is the
missing WRITE side — build a value of a minted tagged-union `E` whose active
variant is `idx`, carrying `payload`. Needed when the variant is selected at
RUNTIME but the union was synthesized at comptime so its labels can't be spelled
as a literal `.label(payload)` — e.g. the `race` result: an `inline for 0..N (i)`
arm constructs the i-th variant of a synthesized result with the winner's value.
Pure sx in modules/std/meta.sx (NOT a compiler builtin): a minted tagged-union is
laid out `{ i64 tag @0, payload @ size_of(i64) }` (the compiler fixes `tag_type`
to i64 for minted enums), so make_variant zeroes the value then writes the tag and
payload at those offsets — the same offsets the compiler's own enum_init/payload
ops use. The header comment documents the minted-only / i64-tag layout coupling.
Adversarially reviewed (SHIP): the offset-8 payload assumption is exactly the
compiler's enum layout with no alignment hazard (the payload is an alignment-1
`[N x i8]` array immediately after the i64 header); complex payloads (multi-field
struct, string fat pointer, 40-byte struct) round-trip. Locked by
examples/comptime/0650-comptime-make-variant.sx (heterogeneous + complex payloads,
runtime-index selection via the comptime `case` form). Suite green (822/0).
A `return` inside an `inline if` / comptime `case` branch — itself inside an
`inline for`, under a runtime `if` — made the compiler wrongly reject the
function as "body produces no value" and DROP its trailing statements (e.g. a
trailing `return -1`).
Root cause: the inline-if branch lowering sets the global `block_terminated` flag
when its taken arm returns (control_flow.zig / stmt.zig `lowerInlineBranch`),
unlike a bare `return` STATEMENT which deliberately never sets it (precisely to
avoid leaking past an `if cond { return }` merge — see the comment at
stmt.zig:37-42). The enclosing runtime-`if`'s merge never reset the flag, so it
leaked past the merge and `lowerBlock` skipped the following statements as dead.
Fix: after the runtime-`if` switches to its merge block, set
`block_terminated = then_diverged and has_else and else_diverged` — the `if`
leaves the block terminated ONLY when both arms diverged with an `else` covering
the cond-false edge (otherwise the merge is reachable and the flag must be
false). Adds `else_diverged` tracking alongside the existing `then_diverged`.
Adversarially reviewed (SHIP): the reachability predicate is correct across all
arm/else/divergence cases; the both-arms-diverge case still sets true (preserving
prior behavior); value-ternary and inline-for-unroll paths are unaffected.
Regression: examples/comptime/0651-comptime-inline-if-return.sx (nested inline-if
and comptime `case`, both with per-arm returns, asserting the trailing return is
emitted). Suite green (822/0).
A comptime-type-call's `Type` result (`field_type(T, i)`, `pointee(P)`) could only
be used in a type-arg slot — not as a `Type`-typed struct-field value, a generic
`$P: Type` argument, or a nested type-call arg — when the index was an `inline for`
loop variable. It routed through value / generic-fn lowering ("cannot infer generic
type parameter" / "unknown #builtin field_type") instead of the type-call fold. This
is what blocked the variable-arity `race` result synthesis: a `($T) -> Type` builder
looping `field_type(pointee(field_type(T, i)), 0)` to mint a tagged-union.
Three coordinated changes route these through the SAME type-call fold (which folds
the index, including a loop var), so type-arg and value positions never disagree:
- `isTypeShapedAstNode` (type_bridge.zig): a `.call` to a type-returning builtin
(`field_type`/`pointee`/`type_of`, via new `isTypeReturningBuiltinName`) is
type-shaped, so generic-arg inference (buildTypeBindings Strategy 1) resolves it
via `resolveTypeArg` rather than failing value inference.
- `tryLowerReflectionCall` (call.zig): value-position `field_type`/`pointee` fold
to `constType(resolveTypeCallWithBindings(c))` — the value twin of the existing
`type_of` fold (every failure path already diagnoses before `.unresolved`).
- `field_name` (call.zig): folds to a const STRING via `memberName` when the type
resolves and the index is a compile-time constant (matching the runtime
`field_name_get` array exactly — same `memberName`, same "" for nameless
members); a dynamic index still emits the `field_name_get` instruction.
Adversarially reviewed (SHIP): no over-broadening (only type-demanding slots consult
isTypeShapedAstNode; only `$T: Type` slots are affected), no silent defaults (every
fold failure is preceded by a diagnostic; "" is the runtime-matching value for a
nameless member). Locked by examples/comptime/0649-comptime-typecall-composition.sx
(reflect a named tuple of `*Box(..)` handles → mint a tagged-union with the tuple's
labels, projecting `*Box(A)` -> `A`). Suite green (821/0). Unblocks PLAN-RACE step 2.
The int-returning type-query builtins now fold to a compile-time constant in
const-required positions (`inline for` bound, array dimension), like a plain
`K :: 3` const — previously they evaluated only as runtime values, so
`[field_count(S)]T` and `inline for 0..field_count(S)` were rejected as "not a
compile-time integer". This is what lets a `($T) -> Type` builder loop
`inline for 0..field_count(T)` to assemble a member list from a type's fields
(the `race` result synthesis).
The shared comptime-int folder `evalConstIntExpr` (program_index.zig) gained a
`.call => ctx.evalConstCallInt(node)` arm. The body-lowering ctx (`Lowering`)
implements it — resolve the type arg via `resolveTypeArg`, return
`memberCount orelse 0` / `typeSizeBytes` / `typeAlignBytes`, matching the runtime
value path in lower/call.zig exactly. `SourceConstCtx` delegates to its wrapped
Lowering; the stateless ctxs (`ModuleConstCtx`, `StatelessInner`, test `DimCtx`)
stub null (they cannot resolve a type-expr arg). A non-type-query call / wrong
arg count / unresolved type arg folds to null (not a comptime integer).
Adversarially reviewed (SHIP): the fold matches the value path across every type
kind, ctx coverage is complete, recursion is AST-depth bounded, no speculative
spurious diagnostics, `orelse 0` is field_count's definitional value for
non-aggregates (not a silent default). Locked by
examples/comptime/0648-comptime-typequery-const-fold.sx. Suite green (820/0).
Project a pointer type to its target: `pointee(*X)` -> `X`. The one reflection
primitive missing for the `race` result synthesis (`*Task(A)` -> `A` via
`field_type(pointee(*Task(A)), 0)`) — reflection could read aggregate fields but
was blind to a pointer's target type.
Mirrors the `field_type` builtin: declared `#builtin` in std/core.sx, resolved as
a lower-time type-call fold in resolveTypeCallWithBindings (src/ir/lower/generic.zig)
so it composes in any type-arg slot. `.pointer` -> pointee, `.many_pointer` ->
element; a non-pointer arg is a loud diagnostic + `.unresolved` sentinel (no silent
fallback). Adversarially reviewed (SHIP). Locked by
examples/comptime/0647-comptime-pointee-reflection.sx. Suite green (819/0).
PLAN-RACE step 1 of 6.
Plan the `race` async deliverable (roadmap A1): a structured first-wins over
the M:1 `Task` layer, returning a comptime-synthesized tagged-union mirroring
the input named tuple's labels. Identifies the one net-new compiler primitive
needed — `pointee($P: Type) -> Type` (project `*Task(A)` → `A`); everything else
(tuple reflection, union synthesis, the suspending runtime) is already in place.
Issue 0196: a named-tuple type ALIAS (`NT :: Tuple(a: i64, b: bool)`) loses its
structure — field access and reflection both fail on the alias, while the inline
and `$T`-type-parameter forms work. Not on the race critical path (race reflects
a tuple type parameter), filed for tracking.
`field_count` / `field_name` were broken on every non-struct/enum aggregate:
`field_count(Tuple(i64, bool))` silently returned 0 (a missing `.tuple` arm in
the count switches), and `field_name(tuple/array/vector, i)` SEGFAULTED — the
LLVM backend built a zero-length `[0 x string]` name array for those kinds while
sizing the runtime GEP at the (often non-zero) member count, so the indexed load
ran past the array.
Root cause was three+ parallel switches that each had to know how to count an
aggregate's members, and disagreed: `field_count` lowering and `memberCount` had
struct/union/tagged_union/enum/array/vector but no `.tuple`; the backend's
`field_name_get` build + GEP sizing had neither `.tuple` nor `.array`/`.vector`.
Fix:
- add the `.tuple` arm to `field_count` lowering (src/ir/lower/call.zig) and
`TypeTable.memberCount` (src/ir/types.zig; this also backs the COMPILER-API
`type_field_count` VM reader).
- unify the LLVM backend onto the single source of truth: both
`getOrBuildFieldNameArray` (reflection.zig) and `emitFieldNameGet`'s GEP sizing
(ops.zig) now derive from `memberCount` / `memberName`, so the name-array
length and the GEP array type can never diverge again — for any kind. A member
with no name (positional-tuple / array / vector element) reflects as "" (one
slot per member, always in-bounds); named-tuple elements recover their labels.
The array/vector clone was surfaced by adversarial review of the tuple-only fix.
Regression: examples/comptime/0646-comptime-field-reflect-tuple-array.sx exercises
field_count/field_name/field_type over struct, enum, positional + named tuple,
array, and vector. Full suite green (818/0). Unblocks the `race` synthesis, which
must reflect a named tuple's labels + element types.
Port library/modules/std/sched.sx to run on aarch64-linux alongside
aarch64-macOS, validated byte-identical on both via Apple `container`.
Per-OS bits are comptime-branched:
- MAP_AP (mmap MAP_ANON flag): linux 0x22 / macOS 0x1002.
- fd-readiness backend: epoll on linux, kqueue on darwin (epoll import
scoped to the linux branch). block_on_fd, the run-loop Mode-2 drain,
and cancel_io_waiter_for each branch; the epoll paths EPOLL_CTL_DEL on
fire and on early-wake (EPOLLONESHOT only disables a registration;
kqueue EV_ONESHOT auto-removes it).
- first-entry trampoline: a per-OS hand-written global-asm symbol becomes
a naked sx fn fib_tramp (mov x0,x19; br x20) + register-indirect
dispatch (spawn presets regs[1] == x20 == &fib_dispatch), dropping the
per-OS .global symbol entirely.
Fixes issue 0193 Bug A: the trampoline redesign bus-errored on the
go/wait/sleep capstone (1817) because fib_dispatch was not pinned to the
C ABI. Without an explicit ABI, fib_dispatch uses sx's internal calling
convention (x0 = implicit context, first arg self shifted to x1) while
the trampoline hands self over in x0 (C-ABI); on first entry the body
runs (x1 happens to alias self) but the closure then loads
regs[1] == &fib_dispatch as its first capture and re-invokes fib_dispatch
forever -> stack overflow -> bus error. Annotating fib_dispatch
`abi(.c)` pins it to the C-ABI (self in x0), matching the trampoline.
`abi(.c)` rather than `export` because the fn is reached only by address
through the trampoline, never by an external name -- so it needs the
convention, not a public symbol (it stays a local symbol). Root cause
found via lldb on an AOT build; confirmed against the compiler source.
Bug B (a top-level asm block wrapped in inline-if is dropped during the
comptime-conditional flatten) is carved out to issue 0194 (OPEN) -- no
live trigger remains, since the naked-fn trampoline sidesteps it.
1811/1814/1816/1817 run byte-identical on the aarch64-macOS host and in
an aarch64-linux container; full suite green (817/0). Documents the fiber
runtime in readme.md.
Drop experimental/Jai/Zig framing and the Acknowledgments section, trim
the verbose edge-case paragraphs (numeric limits, float narrowing,
reserved names, module visibility) to punchy summaries, and remove the
from-source build section. Describe sx as a programming language.
Port of std/sched.sx (the M:1 fiber runtime) to aarch64-linux. The epoll
bindings + std.event.Loop epoll backend are already committed and runtime-
validated (cc137002); this records the SCHEDULER port, which is WIP:
- WORKS, validated in an Apple `container` Linux VM: 1811 (round-robin) and 1816
(block_on_fd over the epoll fd path) run identically to macOS kqueue.
- Bug A: a register-indirect trampoline (naked fn + `br x20`, to avoid a per-OS
hand-written global-asm symbol) bus-errors on the 1817 go/wait/sleep capstone
on both platforms, though 1811/1816 work — unresolved.
- Bug B: wrapping the original global `asm` trampoline in an `inline if`/`case`
drops it (nm: fib_tramp U) in sched.sx's context, though every minimal repro
emits fine — a flatten/lowering interaction in src/imports.zig.
The WIP sched.sx port is preserved both in `git stash` and as
issues/0193-linux-fiber-port.patch. Two resolution paths (either suffices)
documented in the issue. sched.sx itself is left at HEAD (macOS green).
The std.event.Loop epoll backend is now runtime-validated, not just
lower-verified: a static aarch64-linux build of the 1632-equivalent Loop test
(plus the eventfd wake path) runs 6/6 green inside an Apple `container` Linux VM
(kernel 6.18 aarch64) — add_read, idle-timeout, readable+fd+udata, the MOD-mask
add_write path, the eventfd wake channel, and EPOLLRDHUP/HUP eof all behave
identically to kqueue (lone difference: nbytes is 0 on epoll). Update the
event.sx VALIDATION note (with the re-run recipe) and the fibers checkpoint;
the epoll deliverable is complete.
A `:=`-bound closure with no explicit `-> T` and a BLOCK body inferred its
return type via inferExprType(lam.body), which yields the last statement's
type. A block whose value comes only from early `return`s ends in a return
statement (void/noreturn), so the closure was built with a void return while
the body returned i64 — the call site then fed `i64 undef` and LLVM
verification failed. (A block whose tail referenced a block-local hit the
sibling failure: inferExprType returned .unresolved → an LLVM panic.)
Infer the return type exactly as a named fn does (resolveReturnType in
lower.zig): an arrow body `(params) => expr` uses the expression type; a block
body `(params) { stmts }` takes the first explicit `return <val>` type via
findReturnValueType, else void (the block tail is a discarded statement unless
an explicit `-> R` makes it the value). Regression test:
examples/closures/0313-closure-inferred-return-early.sx.
compiler.sx needs only `List` (string is a builtin), so import the std/list.sx
part-file instead of std.sx. Its standalone transitive footprint drops from
~16k to ~50 lines of IR. Enabled by core.sx now self-declaring its libc, so
list.sx → core.sx resolves without the std assembly.
Regenerates 40 .ir snapshots: compiler.sx sits in the std import graph
(std → cli → build → compiler), so narrowing its import shifts the
registration order in every std program, renumbering LLVM symbol suffixes
(@foo.N → @foo.N+1) and adding a redundant `declare void @out` (LLVM dedups
it). Verified the diffs are purely that — no .exit/.stdout/.stderr changed, no
instruction/type/constant changed — and the full suite is green (817/0).
core.sx owns the libc escape hatches (libc_write / libc_malloc / libc_free /
memcpy / memset, all `extern libc "..."`) but never declared the `libc`
#library constant — it free-rode on `#library` being program-global, satisfied
by other std modules (socket/fs/cli) in a full std build. So core.sx — and
list.sx, which imports it for Allocator/List — could not be imported without
assembling the whole std prelude (`extern library 'libc' is not declared`).
Declare `libc :: #library "c"` in core.sx itself. `#library` constants are
program-global and dedup, so this is harmless alongside the other declarers,
and it makes core.sx / list.sx self-contained — importable standalone. No
snapshot drift (a #library decl adds no type/global), full suite green 817/0.
Move OS / ARCH / POINTER_SIZE + the OperatingSystem / Architecture enums out of
build.sx into a new std/target.sx that imports NOTHING, so low-level code can
name the target enum types without dragging the build/std barrel (build.sx
transitively pulls std + compiler + bundle, ~16k lines of IR).
build.sx flat-imports target.sx so the decls stay registered in the standard
import graph (build.sx is reachable from std.sx — dropping it would shift every
std program's type table). This is not a re-export: flat import only splices
into build.sx's own scope. Consumers are unaffected — the compiler resolves
OS / ARCH / POINTER_SIZE by name (comptime constants), so `inline if OS`/value
reads need no import; a module that names the enum type imports target.sx.
No behavior change (full suite green, 817/0); the enum types stay in the same
import graph, so no .ir snapshot drift.
Add library/modules/std/net/epoll.sx — raw epoll bindings, the linux twin of
std/net/kqueue.sx — and branch std.event.Loop on `inline if OS` so the
OS-neutral readiness Loop runs on linux (epoll) as well as darwin (kqueue);
callers never see the backend.
epoll_event has no packed-struct primitive in sx, so it is modelled as an
arch-branched struct of u32 fields — { events, data_lo, data_hi } → 12 bytes on
x86_64 (matching __attribute__((packed))), { events, pad, data_lo, data_hi } →
16 bytes on aarch64 — every field 4-aligned, so the layout is byte-exact for the
kernel ABI with no packed attribute and no unaligned access. The fd is stashed
in data_lo (epoll echoes one data word, not the fd separately).
epoll.sx is self-contained (libc only, no build.sx): the `inline if ARCH`
selecting the struct is resolved by the compiler's flatten pre-pass, so the
module's IR stays small. The epoll backend is imported INSIDE event.sx's
`inline if OS == .linux` branch (not top level): event.sx rides the std.sx
barrel, so a top-level import would register epoll's types into every std
program's type table on darwin and drift every .ir snapshot.
The epoll Loop keeps a small per-fd registration table (combined EPOLLIN/OUT
mask via EPOLL_CTL_ADD/MOD/DEL), maps the fd back to the caller's udata, arms
EPOLLRDHUP so a peer half-close surfaces as Event.eof (matching kqueue EV_EOF),
and uses an eventfd as the cross-thread wake channel (kqueue's EVFILT_USER).
Validation: the kqueue path runs end-to-end on the macOS host (1632 unchanged);
the epoll bindings + ABI layout are corpus-locked ir-only by
examples/event/1633 (x86_64-linux, both arches probe-verified). The epoll Loop
is verified to lower clean for both linux arches and self-reviewed, but is not
corpus-snapshotted (a Loop example drags the std barrel → ~18k-line brittle IR);
runtime behavior validates on a linux runner.
A namespaced import's const (`m :: #import "lib.sx"; … m.CAP`) only ever
resolved as a runtime value — the const folders in program_index.zig had no
namespace-member arm, so a qualified const was rejected as an array dimension /
Vector lane / generic value-param and could not seed another const, while the
flat-import form worked everywhere.
Add a `lookupQualifiedConst` (+ float / float-typed twins) ctx hook: resolve
the alias via `namespaceAliasVerdictFrom` to its target module, then fold the
member from that module's per-source const cache (`foldQualifiedConstInt` in
lower/comptime.zig), pinned to the target source so nested const RHSs fold
there. Wire it into evalConstIntExpr / evalConstFloatExpr / isFloatValuedExpr —
both the expression-position field_access arm (`[m.CAP]T`) and the
type-argument dotted-name arm (`Vector(m.LANES, …)`, generic value-params).
Implemented on the source-aware ctxs (Lowering / SourceConstCtx); the
namespace-blind ModuleConstCtx / StatelessInner return null, so a qualified-const
dim reached only via the stateless type-alias path stays a clean unresolved-dim
diagnostic, never a fabricated length. Resolves correctly for array dims,
arithmetic, integral-float dims, Vector lanes, generic value-params, inline-for
bounds, and struct fields.
Regression: examples/modules/0842-modules-qualified-import-const-comptime.sx.
A failable function that returned by IMPLICIT success (no explicit
`return`) left its error-tag slot uninitialized, so a caller's `catch` /
`or` (or `main`) read a garbage tag and reported a phantom unhandled
error — and for value-carrying failables the success value was dropped.
The "no error" sentinel was only written on the explicit-`return;` path.
Unified all function-body-return lowering so the failable-success slot
is always written:
- void `-> !` fall-through: `ensureTerminator` (control_flow.zig) now
emits `ret constInt(0)` for a pure-failable end-of-body.
- value-failable trailing-expression success: `lowerValueBody`
(stmt.zig) routes through `lowerFailableSuccessReturn`.
- generic + pack-fn instances: `monomorphizeFunction` (generic.zig) and
`monomorphizePackFn` (pack.zig) now DELEGATE their body-return to
`lowerValueBody` instead of hand-rolling a `coerce`+`ret` that drifted
(covers generic/pack value-failables).
Also fixes the missing-value diagnostic guard added here: it now counts
`.err`-level diagnostics (new `DiagnosticList.errorCount`) rather than the
total list length, so a warning/note emitted while lowering the body
(e.g. an ObjC selector arity warning) can no longer suppress a genuine
"body produces no value" error — which previously shipped an
uninitialized return at exit 0.
Regressions: examples/errors/1061 (void fall-through), 1062 (value-failable
trailing expr), 1063 (generic value-failable trailing expr).
Two type-resolution paths silently resolved a non-type AST node in type
position to a zero-field `{}` struct that reached codegen with no
diagnostic:
- a dotted `type_expr` / field-access (`g.a`, `g` a runtime value) whose
prefix is not a namespace alias
- an `error_type_expr` (`!Name`) whose `Name` is not a declared error set
Now both reject loudly:
- `resolveTypeWithBindings` (lower.zig): "expected a type, found a value
'<name>' in type position" + `.unresolved`
- `checkTypeNodeForUnknown` (semantic_diagnostics.zig): validates a named
`!E` against the declared error-set names — "unknown error set
'<name>'" / "expected an error set after '!', found type '<name>'".
A bare `!` (void channel) and a declared `!E` in return position stay
valid; namespace-qualified types (`pkg.Type`) are unaffected.
Regression: examples/diagnostics/1195-diagnostics-non-type-in-type-position.
Rewrite specs.md tuple/failable/pack/UFCS/grammar sections to the new
syntax, update readme.md, and refresh stale tuple references in example
header comments. Also fixes two pre-existing doc inaccuracies surfaced in
review: drop the value-discarding `;` in the tuple-return examples, and
correct the §13 function-type grammar production (optional param list +
optional trailing `!` channel). Optional semantics unchanged.
current/CHECKPOINT-LANG.md logs the cutover.
