1 Commits

Author SHA1 Message Date
agra
22f4719e83 fix: aarch64-linux port of the M:1 fiber runtime (sched.sx)
Port library/modules/std/sched.sx to run on aarch64-linux alongside
aarch64-macOS, validated byte-identical on both via Apple `container`.

Per-OS bits are comptime-branched:
- MAP_AP (mmap MAP_ANON flag): linux 0x22 / macOS 0x1002.
- fd-readiness backend: epoll on linux, kqueue on darwin (epoll import
  scoped to the linux branch). block_on_fd, the run-loop Mode-2 drain,
  and cancel_io_waiter_for each branch; the epoll paths EPOLL_CTL_DEL on
  fire and on early-wake (EPOLLONESHOT only disables a registration;
  kqueue EV_ONESHOT auto-removes it).
- first-entry trampoline: a per-OS hand-written global-asm symbol becomes
  a naked sx fn fib_tramp (mov x0,x19; br x20) + register-indirect
  dispatch (spawn presets regs[1] == x20 == &fib_dispatch), dropping the
  per-OS .global symbol entirely.

Fixes issue 0193 Bug A: the trampoline redesign bus-errored on the
go/wait/sleep capstone (1817) until `export "fib_dispatch"` was restored.
Without the export, fib_dispatch reverts to sx's internal ABI (x0 =
implicit context, first arg self shifted to x1) while the trampoline
hands self over in x0 (C-ABI); on first entry the body runs (x1 happens
to alias self) but the closure then loads regs[1] == &fib_dispatch as its
first capture and re-invokes fib_dispatch forever -> stack overflow ->
bus error. The export pins fib_dispatch to the C-ABI (self in x0),
matching the trampoline. Root cause found via lldb on an AOT build;
confirmed against the compiler source.

Bug B (a top-level asm block wrapped in inline-if is dropped during the
comptime-conditional flatten) is carved out to issue 0194 (OPEN) -- no
live trigger remains, since the naked-fn trampoline sidesteps it.

1811/1814/1816/1817 run byte-identical on the aarch64-macOS host and in
an aarch64-linux container; full suite green (817/0). Documents the fiber
runtime in readme.md.
2026-06-26 11:32:01 +03:00
329 changed files with 174428 additions and 189805 deletions

394
current/CHECKPOINT-ASM.md Normal file
View File

@@ -0,0 +1,394 @@
# sx Inline Assembly — Checkpoint (ASM stream)
Companion to `current/PLAN-ASM.md`; design in
[design/inline-asm-design.md](../design/inline-asm-design.md). Update after every
commit, one step at a time per the cadence rule (no commit may both add a test
and make it pass).
## Last completed step
**G (indirect-memory `=*m` place outputs)** — the LAST substantive asm feature.
Unlike a write-through `=` output (which returns a value then stored), an
indirect output passes the place ADDRESS to the asm and the asm writes through
it — no return slot. `emitInlineAsm` (`src/backend/llvm/ops.zig`): indirect
outputs are excluded from the LLVM return type; their pointer is an opaque `ptr`
call arg placed **first** (arg-consuming constraint order = output-section
indirect pointers → inputs → read-write tied seeds); each gets an
`elementtype(T)` call-site attribute (required in the opaque-pointer era) via
`LLVMCreateTypeAttribute`/`LLVMAddCallSiteAttribute`; the store-back loop skips
them. New `asmIsIndirect(e, op)` helper. Lowering (`lowerAsmExpr`) stops
rejecting `*` (constraint kept verbatim, `=*m` reaches the constraint string
as-is). `asmOperandIndex` unchanged — indirect outputs still count as operands,
so `%[name]``${N}` holds. Verified by **running** on aarch64: store-through-
pointer (`str x9, %[out]` → 42, IR `"=*m,~{x9}"(ptr elementtype(i64) …)`) and a
mixed case (indirect + value output + input → `"=*m,=r,r"`, indirect ptr arg
first, `${0}/${1}/${2}` correct). Two commits per cadence: (1)
`examples/1652-platform-asm-indirect-mem.sx` locked the rejection; (2) implemented
+ flipped 1652 to a runnable aarch64-pinned example (`{ "target": "macos" }`,
ir-only elsewhere). `zig build test` green (661 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1652-*`.
Prior: **G (read-write `+` place outputs)** — a `+r` / `+{reg}` `-> @place` output is now
implemented. LLVM has no `+` constraint, so a
read-write place lowers to: an output **`=`** constraint (return slot, stored back
through the place after the call; the leading `+` rewritten to `=` in
`appendAsmConstraints`), **plus** a **tied input** (the decimal index of that
output) appended **after** the regular inputs, seeded with the place's loaded
value passed as a call arg. Tied inputs come **last** so existing operand indices
(`%[name]``${N}`) are undisturbed — `asmOperandIndex` unchanged. Lowering
(`lowerAsmExpr`) no longer rejects `+` (indirect `*` still rejected loudly).
`emitInlineAsm` (`src/backend/llvm/ops.zig`): grows arg/param arrays by the rw
count (`n_args = n_inputs + n_rw`), loads each seed (`asm.rw.seed`), emits the
tied constraint, and the existing store-back path writes the modified output back.
New `asmIsReadWrite(e, op)` helper. Verified by **running**: increment-in-place
(41→42, IR `"=r,0"`) and a mixed case (rw place + regular input + value output) →
textbook `"=r,=r,r,0"` with correct `${N}` indices and args `(input, seed)`. Two
commits per cadence: (1) `examples/1650-platform-asm-rw-place.sx` locked the
rejection; (2) implemented + flipped 1650 to a runnable aarch64-pinned example
(`{ "target": "macos" }`, ir-only elsewhere). `zig build test` green (658 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`examples/1650-*`.
Prior: **2**`-> @place` write-through outputs. An asm result can be **stored through
a place** (local / struct field) instead of returned; the place output does NOT
join the result tuple. Parser: `-> @place` parses the `@place` as an ordinary
address-of expression → an `out_place` operand (`src/parser.zig`). Lowering
(`lowerAsmExpr`): out_place operand = the lowered `@place` address, `out_ty` =
the pointee; read-write (`+`) and indirect-memory (`*`) constraints rejected
loudly (not yet implemented). Added `out_ty: TypeId` to the IR `AsmOperand`
(`src/ir/inst.zig`) so emit builds the **combined** return struct (ALL outputs).
`emitInlineAsm` rewrite (`src/backend/llvm/ops.zig`): the LLVM return type is now
built from every output's `out_ty`; after the call, out_place slots are
`store`d through their address and out_value slots rebuild the sx result — with a
**fast path** (no place outputs → the asm's struct return IS the result, so
pure-value asm IR is unchanged). Verified: write-to-local (`get42`→42), struct
field (`@p.b`), mixed value+place (`v=10 b=20`), `+` rejected. Locked with
`examples/1649-platform-asm-place-output.sx` (mixed, runs on aarch64). `zig build
test` green (657 corpus, 446 unit). Files: `src/parser.zig`, `src/ir/inst.zig`,
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1649-*`.
Prior: **F** — global (module-scope) asm. A top-level `asm { "tmpl", };` block (template
only) lowers to LLVM `module asm`, and a lib-less `extern` calls into the symbols
it defines. New `asm_global` AST node (`src/ast.zig`) + `parseAsmGlobal`
(`src/parser.zig`, dispatched from `parseTopLevel` on `kw_asm`) — rejects
`volatile` and any operands/clobbers. The node forced (and got) arms in the same
three `Node.Data` switches as `asm_expr` (`sema.zig` ×2, `semantic_diagnostics.zig`).
`Module` gains a `global_asm: ArrayList([]const u8)` (`src/ir/module.zig`);
`lowerMainAndComptime` captures each template (the dead `lowerDecls` is NOT the
top-level pass — `lowerRoot` Pass 2 uses `lowerMainAndComptime`); `emit_llvm.zig`'s
`emit()` appends each via `LLVMAppendModuleInlineAsm` (source order). Verified
end-to-end: an aarch64 `_my_add` global routine called via `extern` returns 42.
Locked with `examples/1648-platform-asm-global.sx`
(`.build { "aot": true, "target": "macos" }` → AOT build+run on aarch64, ir-only
elsewhere). `zig build test` green (656 corpus, 446 unit). **(Correction, later:
module asm ALSO runs under the JIT — `sx run` compiles to an in-memory object,
the integrated assembler assembles the `module asm` into it, ORC relocates and
runs it, so the symbol is resolvable at JIT main execution. The original "AOT
only" note was wrong; see 1653 for the JIT sibling. The genuine boundary is a
COMPILE-TIME `#run` call into a module-asm symbol, which fails loud via host
dlsym-miss — see 1654.)** Files: `src/ast.zig`, `src/parser.zig`, `src/sema.zig`,
`src/ir/semantic_diagnostics.zig`, `src/ir/module.zig`, `src/ir/lower/decl.zig`,
`src/ir/emit_llvm.zig`, `examples/1648-*`.
Prior: **E** — multi-output tuples. **Inline asm now returns tuples.** Replaced the
N>1 bail with a shared `asmResultType` helper (`src/ir/lower/expr.zig`, mixed
into `Lowering`) that derives the result type from the `out_value` operands
(0→void, 1→T, N→named tuple, named via the §II.5 effective-name rule). The key
realization: `toLLVMType(tuple)` already produces a literal struct `{T1,…,Tn}`
exactly LLVM's multi-output asm return — so **emit needed NO change**; building
the op with a tuple result type makes the asm call return the struct, which IS
sx's tuple value (destructured by the normal `tuple_get` path). `inferType`'s
`.asm_expr` arm now also delegates to `asmResultType` (single owner), so
`return asm`, `x := asm`, and `q, r := asm` all agree on the type. Verified
end-to-end on aarch64: `split(0x1234)``(lo=52, hi=18)`, a udiv/msub divmod→
`(3, 2)`. IR is textbook: `call { i64, i64 } asm "divq ${4}",
"={rax},={rdx},{rax},{rdx},r,~{cc}"(…)` → extractvalue → tuple. Converted 1640 to
the x86_64 multi-output IR lock (ir-only) + added `1647-platform-asm-aarch64-multi`
(runs on aarch64). `zig build test` green (655 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `src/ir/expr_typer.zig`,
`examples/164{0,7}-*`.
Prior: **C.1 + D** — inline asm CODEGEN (lowering builds the op + LLVM emit). **Inline
assembly now runs end-to-end.** `lowerAsmExpr` (`src/ir/lower/expr.zig`) stops
bailing: it resolves each operand's effective name (§II.5 auto-naming), interns
template/constraints/clobbers, lowers input `Ref`s, derives the result `TypeId`
(0→void, 1→T), and builds the `inline_asm` op. Added a `%[name]`-references-a-
real-operand check (the last deferred validation). Multi-output (N>1) still bails
loudly ("Phase E"). `emitInlineAsm` (`src/backend/llvm/ops.zig`, port of Zig's
`airAssembly`): assembles the LLVM constraint string (outputs→inputs→`~{clobber}`,
`,``|`), rewrites the template (`%[name]``${N}`, `%%``%`, `$``$$`, `%=`
`${:uid}`), then `LLVMGetInlineAsm` + `LLVMBuildCall2` (AT&T). Dispatch wired
(`emit_llvm.zig`, replacing the C.0 `@panic`). **`llvm_shim.c`**: added
`LLVMInitializeNativeAsmParser()` — the JIT must assemble inline asm at run time.
Verified end-to-end: aarch64 `add`/`mov` run on the host (exit 42), `nop volatile`
runs (1642 now exit 0), IR is textbook (`call i64 asm "add ${0},${1},${2}",
"=r,r,r"(…)`). Locked with `examples/1645-platform-asm-aarch64-add.sx` (runs on
aarch64, ir-only elsewhere via `.build` + `.ir`). Also added the `inferType`
`.asm_expr` arm (`src/ir/expr_typer.zig`, 0→void / 1→T) — without it a bare
`x := asm {…-> T}` binding inferred `.unresolved` and silently produced 0;
regression-locked with `examples/1646-platform-asm-value-binding.sx`. Updated
1640 (now Phase-E bail) + 1642 (now runs). `zig build test` green (654 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`src/ir/emit_llvm.zig`, `src/ir/expr_typer.zig`, `llvm_shim.c`,
`examples/164{0,2,5,6}-*`.
Prior: **C.0** — IR op `inline_asm` (lock; no behavior change). Added `inline_asm:
InlineAsm` to the IR `Op` union + the `InlineAsm` struct (`template: StringId`,
`operands: []const AsmOperand` {role/name/constraint/operand}, `clobbers:
[]const StringId`, `has_side_effects`) in `src/ir/inst.zig` — all strings
interned, operands in source order, result on `Inst.ty`. The new variant forced
(and got) arms in two exhaustive `Op` switches: `src/ir/interp.zig` (loud
`bailDetail` — inline asm is never comptime-evaluable) and `src/ir/print.zig`
(IR dump). `src/ir/emit_llvm.zig` gets a `@panic` **tripwire** — emit lands in
Phase D, and until then `lowerAsmExpr` still bails so no `inline_asm` op is ever
created (reaching emit would be a lowering-switched-over-too-early bug). Unit
test `inline_asm op shape` in `src/ir/inst.test.zig`. `zig build test` green
(652 corpus, 446 unit). Files: `src/ir/inst.zig`, `src/ir/interp.zig`,
`src/ir/print.zig`, `src/ir/emit_llvm.zig`, `src/ir/inst.test.zig`.
Prior: **B.1** — operand-name validation (design §II.5 auto-naming rule). Extended
`lowerAsmExpr` with a `pinnedRegister(constraint)` helper (`"={eax}"``eax`,
`"+{rax}"``rax`, `"=r"`→null) and two checks: (1) **reject the echo form**
`[eax] "={eax}"` — a label identical to its own pinned register is redundant
(the operand is already auto-named after the register); (2) **reject duplicate
operand names** (ambiguous `%[name]` / result field). Locked with
`examples/1643-platform-asm-echo-name.sx` + `1644-platform-asm-duplicate-name.sx`.
`zig build test` green (652 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`.
Prior: **B.0** — asm shape validation (compile-path diagnostics). Restructured the
`.asm_expr` lowering arm into `lowerAsmExpr` (`src/ir/lower/expr.zig`, mixed into
`Lowering` in `src/ir/lower.zig`): it validates BEFORE the not-yet-implemented
codegen bail, so the user sees the real problem first. Two checklist items now
enforced with named diagnostics: (1) **template must be a compile-time-known
string** (`"..."` / `#string`); (2) **no value outputs ⇒ must be `volatile`**
(mirrors Zig — a result-less asm could be deleted). Valid shapes still bail with
the "codegen not yet implemented" message. Result-type derivation + auto-naming
stay deferred to a later step (observable only once Phase C produces a real IR
op). Locked with `examples/1641-platform-asm-missing-volatile.sx` (volatile
error) + `1642-platform-asm-nop-volatile.sx` (volatile no-output accepted →
codegen bail). `zig build test` green (650 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `examples/164{1,2}-*`.
Prior: **A.1** — parse `asm { … }` + loud lowering bail (folded A.1+A.2 into one honest
lock commit, since the loud bail IS current correct behavior — cadence option
(a)). Added `AsmExpr`/`AsmOperand` to `src/ast.zig` + the `asm_expr` `Node.Data`
arm; `parseAsmExpr` in `src/parser.zig` (`parsePrimary` `.kw_asm` dispatch) —
parses the template, flat operand list (`[name]? "constraint" -> Type` value
output / `= expr` input), and `clobbers(.…)`; `volatile`/`clobbers` recognized
contextually via `isContextualWord`. The new `asm_expr` tag forced (and got)
arms in three exhaustive `Node.Data` switches: `src/sema.zig` `analyzeNode` +
`findNodeAtOffset`, `src/ir/semantic_diagnostics.zig` `checkBindingNames` (all
recurse into template + operand payloads). Lowering bails LOUD + named in
`src/ir/lower/expr.zig` ("inline assembly codegen is not yet implemented…") via
an explicit `.asm_expr` arm (not the generic `unknown_expr` else) returning
`emitPlaceholder`. `-> @place` write-through is rejected with a clear "Phase 2"
parse error. Locked with `examples/1640-platform-asm-parse.sx` (multi-output
`divmod`, named operands, register pins, clobbers — parses then bails; called
from `main`). `zig build test` green (648 corpus, 0 failed; 445 unit). Files:
`src/ast.zig`, `src/parser.zig`, `src/sema.zig`, `src/ir/semantic_diagnostics.zig`,
`src/ir/lower/expr.zig`, `examples/1640-*`.
Prior: **A.0**`kw_asm` keyword (first compiler code). Added the `kw_asm` `Token.Tag`
variant + `.{ "asm", .kw_asm }` keyword-map entry in `src/token.zig`; `volatile` /
`clobbers` deliberately stay OUT of the global table (contextual). New exhaustive
`Tag` switch in `src/lsp/server.zig` `classifyToken` flagged the missing arm (the
intended coverage tripwire) — added `.kw_asm` to the keyword group. Lock test in
new `src/lexer.test.zig` (`asm``kw_asm`, `volatile`/`clobbers``identifier`),
wired into the `src/root.zig` barrel as `lexer_tests`. `zig build test` green (648
corpus, 0 failed; 445 unit, 0 failed — +1). Files: `src/token.zig`,
`src/lexer.test.zig`, `src/root.zig`, `src/lsp/server.zig`.
Prior: **0.2** — CLAUDE.md docs for `<name>.build`; **Phase 0 COMPLETE**.
**0.1** — corpus runner **ir-only branch** for cross-target examples. Replaced
0.0's loud placeholder bail: when `cfg.target` doesn't match the host (`ir_only`),
`sweepRoot` skips run/build/exec and verifies via `sx ir --target` only —
asserting `.exit` (ir cmd) + `.ir` (normalized stdout) + `.stderr`, never
`.stdout` (write skipped in update mode, assertion skipped in verify mode). An
`.ir` snapshot is **required** in ir-only mode — its absence is a loud failure
("needs an .ir snapshot for ir-only mode"). Locked with
`examples/1639-platform-target-cross.sx` (asm-free `main :: () -> i64 { return 0;
}`), `.build` `{ "target": "x86_64-linux" }`, + checked-in `.ir`. Verified both
guards fire: corrupting the `.ir` → IR mismatch; deleting it → the require-failure.
`zig build test` green (647 corpus, 0 failed; 444 unit). Files:
`src/corpus_run.test.zig`, `examples/1639-*`.
## Current state
**Inline assembly works end-to-end: 0, 1, and N value outputs (tuples).** Full
pipeline: lex (A.0) → parse (A.1) → validate (B.0/B.1 + `%[name]` check) → IR op
(C.0) → lower-builds-op + LLVM emit + JIT asm-parser init (C.1/D) → multi-output
tuples (E). Register-class + register-pinned operands, inputs, **symbol operands
(`"s"` → direct `bl`/`call` to a function/global by mangled name)**, clobbers,
`#string` multi-instruction templates, `%[name]`/`%%` rewriting, and the §II.5
auto-naming rule all work and execute on the host JIT. Global `asm { … }` (Phase F) works via
lib-less `extern` under BOTH the JIT (`sx run` → 1653) and AOT (1648) — `sx run`
compiles to an object, so the integrated assembler bakes the `module asm` symbol
in and ORC resolves it. All three `-> @place` output forms now work and execute
on aarch64: **write-through** `=` (Phase 2), **read-write** `+` (tied input), and
**indirect-memory** `=*m` (pointer arg + `elementtype`, asm writes through it).
**Inline assembly is now feature-complete — no substantive features remain.** The
x86_64 syscall-write ir-only example is DONE (1651). Global asm runs under both
JIT (1653) and AOT (1648). `readme.md` now has an "Inline Assembly" section.
Known orthogonal bug: **issue 0137**`sx run` on a program with no `main`
segfaults (`src/target.zig:256-273`, unguarded JIT entry lookup). Pre-existing,
asm-independent; does NOT block the ASM stream (every example has a `main`).
Phase EF feasibility already confirmed against the live tree
(`LLVMGetInlineAsm` / `LLVMBuildCall2` / `LLVMAppendModuleInlineAsm` in LLVM@19
`Core.h`; ERR-stream `extractvalue`→tuple in `emit_llvm.zig:726-927`; lib-less
`extern`, 60 sites; `--target` a global CLI flag).
## Next step
**Inline assembly is feature-complete.** All substantive features are done:
0/1/N value outputs (tuples), register-class + pinned operands, inputs, clobbers,
`#string` templates, `%[name]`/`%%`/`$`/`%=` rewriting, §II.5 auto-naming, global
`asm { … }` (AOT), and all three `-> @place` output forms — write-through (`=`),
read-write (`+`), and indirect-memory (`=*m`). The x86_64 syscall-write ir-only
example (1651) and the output-to-`const` rejection (issue 0138) are also done.
Global asm runs under BOTH the JIT (`sx run` → object → ORC; 1653) and AOT (1648)
— the earlier "AOT only / `sx run` mishandles module-asm" note was stale and has
been corrected. The one genuine boundary is a COMPILE-TIME `#run` into a
module-asm symbol: the interpreter resolves externs via host dlsym, the symbol
isn't linked yet, so it already fails loud (`comptime extern call: symbol not
found via dlsym`) — pinned by 1654.
Remaining work, all **polish** (optional):
- None substantive. Possible niceties: tighten the `#run`-into-module-asm error
text to name module-asm specifically; broaden clobber validation to a checked
per-arch enum (design doc Phase 4).
Orthogonal: **issue 0137** (no-`main` JIT segfault).
Done since last: output-to-`const` rejection (issue 0138), x86_64 syscall-write
ir-only example (1651).
Orthogonal: **issue 0137** (no-`main` segfault).
## Log
- (init) Plan + design doc written; ASM stream opened.
- (0.0) Corpus runner target-gating: `<name>.build` JSON config (replaces `.aot`
marker), `--target` threading, `hostMatchesTarget` execute-gate, loud
cross-target placeholder bail. Migrated 1226/1227 `.aot``.build`; locked with
1638 fixture + unit tests. `zig build test` green.
- (0.1) ir-only branch: cross-target examples verify via `sx ir --target` only
(exit+ir+stderr, no stdout; `.ir` required). Locked with 1639 fixture; verified
corrupt-.ir → mismatch and missing-.ir → loud failure. `zig build test` green.
- (0.2) docs: CLAUDE.md documents `<name>.build` JSON sidecar (aot + target +
ir-only gating), replacing stale `.aot` marker prose. **Phase 0 COMPLETE.**
- (A.0) `kw_asm` keyword in token.zig (+ map entry); LSP `classifyToken` switch
coverage; lock test in new `lexer.test.zig` (wired via root.zig). `volatile` /
`clobbers` stay contextual identifiers. `zig build test` green (445 unit, +1).
- (A.1) parse `asm { … }``AsmExpr` + loud lowering bail; `asm_expr` arms in 3
exhaustive `Node.Data` switches; `-> @place` rejected (Phase 2). Adopted operand
auto-naming rule (design §II.5). Locked with 1640 fixture. Filed orthogonal
issue 0137 (no-`main` JIT segfault). `zig build test` green (648 corpus, 445 unit).
- (B.0) asm shape validation in `lowerAsmExpr`: comptime-string template +
no-output⇒volatile, with named diagnostics before the codegen bail. Locked with
1641 (volatile error) + 1642 (volatile accepted). `zig build test` green (650
corpus, 445 unit).
- (B.1) operand-name validation: `pinnedRegister` helper + reject echo form
(`[eax] "={eax}"`) and duplicate names. Locked with 1643 + 1644. `zig build
test` green (652 corpus, 445 unit).
- (C.0) IR op `inline_asm: InlineAsm` + interp `bailDetail` + print arm + emit
`@panic` tripwire (Phase D). No behavior change (lowering still bails). Unit
test `inline_asm op shape`. `zig build test` green (652 corpus, 446 unit).
- (C.1+D) CODEGEN — `lowerAsmExpr` builds the op (effective names, interned
strings, input Refs, 0/1 result type) + `%[name]` validation; `emitInlineAsm`
(constraint string + template rewrite + `LLVMGetInlineAsm`/`BuildCall2`, AT&T);
`inferType` arm; `LLVMInitializeNativeAsmParser` for the JIT. **Inline asm runs
end-to-end.** N>1 bails (Phase E). Locked with 1645 (aarch64 add, runs) + 1646
(`:=` binding); updated 1640/1642. `zig build test` green (654 corpus, 446 unit).
- (E) multi-output tuples — `asmResultType` helper (0→void/1→T/N→named tuple),
shared by lowering + `inferType`. `toLLVMType(tuple)` == LLVM multi-output
struct, so emit unchanged; the asm struct return IS the sx tuple. Runs on
aarch64 (1647: `split``(lo,hi)`); 1640 → x86 multi-output IR lock (ir-only).
`zig build test` green (655 corpus, 446 unit).
- (F) global asm — `asm_global` AST node + `parseAsmGlobal` (top-level, rejects
volatile/operands); `Module.global_asm` captured in `lowerMainAndComptime`;
`emit()` appends via `LLVMAppendModuleInlineAsm`; call-into via lib-less
`extern`. AOT-verified (1648, `_my_add`→42). `zig build test` green (656 corpus).
- (docs) readme.md "Inline Assembly" section (b8800a2).
- (2) `-> @place` write-through — `out_place` operand; `out_ty` on the IR
AsmOperand; `emitInlineAsm` builds the combined output struct + splits
(out_place → store-through, out_value → result), fast-path when no places.
`+`/`*` rejected. Locked with 1649 (mixed, runs). `zig build test` green (657
corpus, 446 unit).
- (G) read-write `+` place outputs — `+` lowers to an output `=` + a tied input
(output-index constraint) seeded with the place's loaded value, tied inputs
appended last (operand indices undisturbed). `appendAsmConstraints` rewrites
`+``=`; `emitInlineAsm` grows args by the rw count + loads seeds;
`asmIsReadWrite` helper. Lowering stops rejecting `+` (`*` still rejected). Two
commits (cadence): 1650 locked the rejection, then flipped to a runnable
aarch64 example (`"=r,0"` IR). `zig build test` green (658 corpus, 446 unit).
- (0138) output-to-`const` rejection — fixed the underlying general bug: scalar
`@const` (address-of a folded `::` constant) reinterpreted the value as a
pointer (`inttoptr`). `src/ir/lower/expr.zig` `.address_of` now diagnoses a
scalar const (local + module) instead of falling through; array/struct consts
keep storage. asm `-> @const` gets the clean diagnostic for free (same path).
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`. Issue 0138
RESOLVED. `zig build test` green (659 corpus, 446 unit).
- (x86 syscall) x86_64 Linux `write(2)` via raw `syscall` — locks the constraint
string `={rax},{rax},{rdi},{rsi},{rdx},~{rcx},~{r11},~{memory}` (register-pinned
inputs + pinned value output + pointer input + clobbers). ir-only on aarch64
(`.ir` asserted), runs on x86_64-linux (hand-authored `"ok\n"` stdout).
`examples/1651-platform-asm-x86-syscall-write.sx`. Pure additive lock, no
compiler change. `zig build test` green (660 corpus, 446 unit).
- (G indirect) indirect-memory `=*m` place outputs — the place address is passed
as an opaque `ptr` arg (with an `elementtype(T)` call-site attr), placed before
inputs; asm writes through it; no return slot; store-back skips it.
`asmIsIndirect` helper; lowering stops rejecting `*`. Verified by running on
aarch64 (store-through → 42; mixed indirect+value+input → `"=*m,=r,r"`). Two
commits (cadence): 1652 locked the rejection, then flipped to a runnable aarch64
example. **Inline asm now feature-complete.** `zig build test` green (661 corpus,
446 unit).
- (jit) explored "asm in JIT": found it ALREADY works — `sx run` emits an
in-memory object (integrated assembler bakes in both in-function inline asm and
`module asm`), then ORC relocates+runs it. The stale "AOT only / `sx run`
mishandles module-asm" checkpoint prose was corrected. Locked global-asm-under-
JIT with `examples/1653-platform-asm-global-jit.sx` (`{ "target": "macos" }`, no
aot, → 42). `zig build test` green (662 corpus, 446 unit).
- (comptime guard) pinned the one genuine module-asm boundary:
`examples/1654-platform-asm-global-comptime-call.sx``#run` into a module-asm
symbol fails loud (`comptime extern call: symbol not found via dlsym`) because
the interpreter resolves externs via host dlsym before link. Arch-independent
(no `.build`). `zig build test` green (663 corpus, 446 unit).
- (round trip) `examples/1655-platform-asm-callback-into-sx.sx` — global-asm
trampoline that `bl _cb` back into an `export`ed sx function (sx→asm→sx, → 42).
Documented that `export` (external linkage + C symbol + C ABI) is what makes
the callback resolvable; `callconv(.c)` alone leaves it `internal` (DCE'd).
`zig build test` green (664 corpus, 446 unit).
- (symbol ops) symbol operands (`"s"`) — feed a function/global symbol; the
template emits its platform-mangled name so `bl %[fn]` is a DIRECT branch (one
fewer indirection than register-indirect `blr`, portable — no hardcoded `_`).
Emit passes the operand with its own llvm type (LLVMTypeOf), no coercion
(`asmIsSymbol` helper); lowering lowers the function RHS to `ptr @fn`. Decided
AGAINST mirroring Zig (which has no symbol operand — 483 std asm sites, none
call a function) because the direct `bl` matters. Two commits (cadence): 1656
locked the rejection (replacing an LLVM-verifier crash), then implemented +
flipped to a runnable aarch64 example (objdump-confirmed direct `bl <_cb>`).
`zig build test` green (665 corpus, 446 unit).
- (x86 cross-arch) ir-only x86_64 siblings so each emit path is locked on BOTH
arches: 1657 read-write (`"incq ${0}","=r,0"`), 1658 indirect (`"movq $$42,
${0}","=*m"`(ptr elementtype)), 1659 symbol (`"call ${2:P}"`, direct call). x86
templates validated by cross-emitting an object (integrated assembler accepts;
objdump confirms 1659's direct `call` reloc). Pure additive locks. `zig build
test` green (668 corpus, 446 unit).
- (symbol portability) made `%[fn]` portable across arches — `renderAsmTemplate`
auto-injects LLVM's `:c` modifier (`${N}``${N:c}`) for symbol (`"s"`) operands
lacking an explicit modifier (`asmNamedIsSymbol` helper). Without it x86 renders
`$cb` (a bad `call` target needing a hand-written `:P`); aarch64 unaffected.
Verified `:c``:P` for x86-64 calls (both → `R_X86_64_PLT32`). Explicit
`%[fn:X]` still wins (escape hatch). 1659 dropped its `:P` → same plain `%[fn]`
as aarch64 1656; both IRs regen to `${N:c}`. `zig build test` green (668 corpus,
446 unit).
## Known issues
- **0138** — RESOLVED. `@const` (address-of a `::` comptime constant) yielded a
wild pointer (`inttoptr (i64 <value> to ptr)`). Fixed by diagnosing scalar
`@const` in `src/ir/lower/expr.zig` `.address_of` (no storage; array/struct
consts unaffected). Delivered the ASM "output-to-`const` rejection" for free.
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`.
- **0137** — `sx run` on a program with no `main` segfaults (unguarded JIT entry
lookup, `src/target.zig:256-273`). Pre-existing, asm-independent. Filed
`issues/0137-jit-run-no-main-segfault.md`. Does not block A.1.

View File

@@ -0,0 +1,132 @@
# CHECKPOINT-ATOMICS — Stream A (atomics lowering)
Companion to [PLAN-ATOMICS.md](PLAN-ATOMICS.md). Update after every step (one step at a
time, per the cadence rule). New corpus category: `17xx`.
## Last completed step
**A.2 (CAS) — DONE** (A.2a lock + A.2b green). `compare_exchange`/`_weak` → LLVM `cmpxchg`
(result **`?T`, null = SUCCESS**; failure carries the actual value for retry). New IR op
`atomic_cmpxchg` + `AtomicCmpxchg{ptr, cmp, new, val_ty, success_ordering, failure_ordering,
weak}`. `emitAtomicCmpxchg`: `LLVMBuildAtomicCmpXchg` (success/failure orderings, singleThread=0)
`{T, i1}` pair; `LLVMSetWeak` for weak; `?T` result = `{ extractvalue 0 (actual),
xor(extractvalue 1, true) }` (has_value = NOT success). comptime_vm arm does real single-thread
CAS (read/compare/store-on-equal, build `?T`; weak == strong at comptime). Recognizer
(`atomic_cmpxchg`/`_weak`, 6 args) — CAS restricted to INTEGER T; BOTH orderings via
`atomicOrderingFromNode`; dual-ordering validation (failure may not be release/acq_rel nor
stronger than success, `atomicOrderingRank`). Methods `compare_exchange`/`_weak` on `Atomic($T)`
with comptime `$success`/`$failure: Ordering`. `examples/1702` green (CAS ok→20 / fail actual=20 /
weak retry loop 100→105); `examples/1186` locks a rejected ordering pair; unit test `emit: atomic
cmpxchg (strong + weak)` asserts `cmpxchg` + `cmpxchg weak`. Suite green (718/0).
### A.1 (RMW) — DONE (A.1a lock + A.1b green)
`fetch_add/sub/and/or/xor` + `fetch_min/max` → LLVM `atomicrmw` (returns OLD value). New IR op
`atomic_rmw` + `RmwKind` (no `nand`); `LLVMBuildAtomicRMW` with binop from kind, signed/unsigned
`Min/Max` from `val_ty`. RMW restricted to INTEGER T (float fadd / pointer RMW out of scope,
rejected loudly); all five orderings valid for RMW. comptime_vm does real single-thread RMW.
`examples/1701` green; unit test locks `atomicrmw add` + signed `min` vs unsigned `umin`.
## Next step
**A.3 — fence** (`atomic_fence($o: Ordering)` → LLVM `fence`), per PLAN-ATOMICS. No value
result; ordering must be acquire/release/acq_rel/seq_cst (relaxed is meaningless for a fence —
reject loudly). New IR op `atomic_fence` + dispatch/print/comptime_vm (no-op single-thread) +
`LLVMBuildFence`. Lock-then-green cadence as before.
### Earlier — A.0c (guard hardening)
Adversarial review of A.0 found two CRITICAL silent-wrong defects (raw LLVM verifier errors
via the public intrinsics) + a latent align fallback; all fixed: scalar-kind allowlist +
per-op ordering validity (call.zig), `val_ty` align bail (ops.zig). Locked by examples
1130/1131. Suite green (713/0).
### Earlier — A.0b (green)
Real atomic load/store emission: `LLVMBuildLoad2`/`LLVMBuildStore`
+ `LLVMSetOrdering` + mandatory `LLVMSetAlignment`, ordering via an explicit
sx-tag→`LLVMAtomicOrdering` switch (`llvmOrdering`). `examples/1700` green (7/42/43); IR
shows `load atomic i64, ptr … seq_cst, align 8` + `store atomic …`. Added unit test
`emit: atomic load/store (seq_cst, aligned)` in `emit_llvm.test.zig` (asserts `load
atomic`/`store atomic`/`seq_cst`/`align 8`). No fragile full-module `.ir` snapshot for 1700
(it uses `print`); the unit test is the emission-shape gate. Suite green (710 + units).
### Earlier — A.0a (lock commit)
Full atomic load/store plumbing with LLVM emission deliberately bailing loudly;
`examples/1700` locked to the bail diagnostic.
- `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (`init`/`load`/
`store`, **seq_cst-only** — see capability gap below), `atomic_load`/`atomic_store`
`#builtin` decls. **Opt-in import**, NOT in the universal `std.sx` facade (mirrors
`trace`) — putting `Ordering` in the prelude grew every program's type table 378→380 and
churned 37 `.ir` snapshots; reverted.
- IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` (all 5) + structs (inst.zig);
print arms (print.zig); comptime_vm arms reuse load/store (single-thread correct);
recognizer `tryLowerAtomicIntrinsic` (call.zig) — const-ordering-literal guard +
scalar-size guard, both loud; emit dispatch arms (emit_llvm.zig) → `emitAtomicLoad`/
`emitAtomicStore` (ops.zig) currently BAIL via `comptime_failed`.
## A.0.5 — full ordering surface (DONE)
`Atomic($T).load($o: Ordering)` / `store(v, $o)` — ordering is a COMPTIME value param,
explicit (Rust-style, no default; design §4.6). `a.load(.acquire)` emits `load atomic …
acquire`; `a.store(v, .release)` emits `store atomic … release`; `a.load(.release)` is a
compile error (per-op validity guard fires through the method path). Recognizer
`atomicOrderingFromNode` now resolves a comptime-bound ordering identifier via
`comptimeIntNamed` (+ `atomicOrderingFromTag`, with the sx-Ordering ↔ IR-AtomicOrdering
declaration-order invariant documented). 1700 migrated to explicit orderings (output
unchanged 7/42/43). Suite green (715/0).
**Unblocked by three comptime-value-param commits (workers):** enum (3c4305f), tagged_union
(d7a6857), generic-struct methods (d95ba0a). NOTE: default VALUES for comptime params on
generic-struct methods are NOT bound (orthogonal gap — free-fn defaults work); atomics
sidesteps it cleanly by requiring explicit ordering (matches the design). Candidate
follow-up, not an atomics blocker.
## Known issues / capability gaps
- **RESOLVED:** comptime-constant ordering propagation — landed via comptime value params
(3c4305f / d7a6857 / d95ba0a); A.0.5 migrated the methods, no seq_cst-only legacy.
- **Orthogonal gap (not an atomics blocker):** default VALUES for comptime params don't bind
on generic-struct methods (free-fn defaults DO work). Atomics requires explicit ordering
(design-aligned), so it's unaffected. Candidate future fix.
- **Cosmetic:** an invalid ordering passed through a method (`a.load(.release)`) reports the
diagnostic at the lib forward site (`atomic.sx`), not the user's call. Loud + correct, but
the span could be improved by threading the call-site span. Polish.
- **Latent (observed, not yet filed):** calling an *unrecognized* bodiless `#builtin`
silently returns 0 / no-ops with exit 0 (that's how 1700 behaved before recognition
landed) — a silent-fallback footgun in the generic builtin-call path, independent of
atomics. Flag to user; candidate `issues/` entry.
## Decisions (Stream A specifics; surface locked in design §4.6)
- `Atomic($T)` = pure-sx transparent 1-field struct (NO new IR type); ops = `#builtin`
intrinsics emitted as new IR ops. Minimal compiler surface.
- Ordering is compile-time-only (const enum literal), baked into the op as a Zig enum;
non-literal = loud diagnostic. sx tag → LLVM ordering via explicit switch (LLVM enum is
non-contiguous: 2/4/5/6/7).
- Atomic load/store REQUIRE explicit alignment (`LLVMSetAlignment`) — verifier mandate.
- Comptime VM treats atomics as ordinary load/store (single-thread ⇒ correct), not a bail.
- **Snapshot scope corrected:** `.ir` (LLVM IR) is arch-invariant for atomics → ONE host
`.ir` per op, not arch-gated x86/aarch64 pairs (they'd be byte-identical). Asm-level arch
divergence + weak-memory semantics are OUT of corpus scope (stress harness, Stream C).
## Log
- **carve** — wrote PLAN-ATOMICS.md + CHECKPOINT-ATOMICS.md; grounded the intrinsic path,
switch sites, LLVM-C API (no `LLVMBuildAtomicLoad`; use `LLVMBuildLoad2`+`SetOrdering`+
`SetAlignment`), and corrected the arch-`.ir` misconception (`sx ir` emits arch-invariant
LLVM IR). Stream ready; A.0a is the first implementation step.
- **A.0a** — landed lib (atomic.sx, opt-in import) + IR ops (atomic_load/atomic_store +
AtomicOrdering) + recognizer + print/vm arms + emit BAIL; locked `examples/1700` to the
bail diagnostic. Reverted a universal-facade wiring that churned 37 `.ir` snapshots
(Ordering would bloat every program's type table). Suite green (710/0).
- **A.0b** — real atomic load/store emission (LLVMBuildLoad2/Store + SetOrdering +
SetAlignment; explicit sx→LLVM ordering switch). 1700 green (7/42/43, `load atomic …
seq_cst, align 8`). Unit test added. Suite green (710 + units).
- **A.0c** — guard hardening from the adversarial review: scalar-kind allowlist + per-op
ordering validity (call.zig), val_ty align bail (ops.zig), + diagnostic examples
1130/1131. Suite green (713/0). (comptime enum value params landed via worker 3c4305f.)
- **A.0.5** — full ordering surface: `Atomic($T).load/store($o: Ordering)` comptime ordering
(explicit). Recognizer resolves comptime-bound ordering via `comptimeIntNamed`. 1700
migrated to explicit orderings (acquire/release/relaxed/seq_cst). Unblocked by
comptime-value-param workers (3c4305f/d7a6857/d95ba0a). Suite green (715/0).
- **A.1** — RMW: atomic_rmw op + RmwKind + recognizer (rmwKindFromName, integer-only) +
7 fetch_* methods/intrinsics. A.1a bail-lock → A.1b real LLVMBuildAtomicRMW (signed|unsigned
min/max). comptime_vm real RMW. 1701 + unit test. Suite green (716/0).
- **A.2** — CAS: atomic_cmpxchg op + recognizer (dual-ordering validation) + emit (?T from
{actual,!success}) + comptime VM. compare_exchange/_weak methods. examples 1702 + 1186.
Review agent died; self-verified comptime↔runtime agreement, sub-8, ordering edges.
(Commits dca396e/79895be; A.2 has_value fix folded into A.3a.)
- **A.3** — swap (atomicrmw xchg) + fence (new atomic_fence op). A.3a bail-lock → A.3b real.
examples 1703/1704/1187 + unit test. Stream A feature-complete. Suite green (721/0).

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,861 @@
# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)
Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
every commit, one step at a time per the cadence rule.
## Last completed step
**Phase 9 COMPLETE — total `foreign` purge; 9.4 GATE PASSES.** **THE ENTIRE
FFI-LINKAGE STREAM (Parts A + B, Phases 09) IS DONE.** Final commits: 9.0 token
delete (`dfae690`), 9.3 src/docs/example/library comment purge (`811a280`, `e99383f`,
`dc51c4b`, + the capital-Foreign sweep), 9.3 example filename renames + dedup
(`b52d424`), 9.3/9.4 issues/*.md purge + GATE (`b9cfe25`).
- **9.0:** deleted the `hash_foreign` token entirely (token/lexer/parser/lsp + the lex
test); `#foreign` now → a generic "expected ';'" parse error (accepted UX cost);
deleted the obsolete 1176 rejection test.
- **9.1/9.2:** all internal identifiers renamed (linkage→`extern`/`is_extern`,
runtime-class→`Runtime*`/`runtime_*` per Decision 5, `foreign_path``runtime_path`
across the build-hook boundary); `foreign_expr` node eliminated.
- **9.3:** purged every `foreign` COMMENT (src caps + lowercase, examples, docs incl.
the obsolete inline-asm Deviation 6, editors/vscode grammar) + renamed all 10
`*-foreign*` example files (+ companions/expected/refs) to extern/runtime names
(dedup'd 1218↔1229, removed orphan 1620 dir) + rewrote 20 issues/*.md writeups +
renamed issues/0043.
- **9.4 GATE:** `grep -rniIE 'foreign' src/ library/ examples/ issues/ docs/ editors/
specs.md readme.md CLAUDE.md` → **0**, excluding only the legitimate keeps:
`SQLITE_CONSTRAINT_FOREIGNKEY` (SQLite API const) + vendored `library/vendors/sqlite/
c/*` (upstream third-party C). No `foreign`-named files in the tree (node_modules +
.sx-tmp are gitignored third-party/scratch). Suite green (644 corpus / 443 unit, 0
failed).
### Prior: Phase 9.3 — text/comment purge (src + docs + example comments) (commits
`e99383f` docs, `dc51c4b` src, + examples purge STAGED pending a classifier outage —
commit message ready; `git commit` the staged `examples/` changes when Bash is back).
`foreign` is now purged from: **all `src/` comments** (reworded to extern/runtime-class;
fixed 2 user-facing diagnostics — the type-annotation parse error no longer lists
`#foreign`, and the Android no-`#jni_main` help shows `#jni_class(…) extern`), **specs/
readme/CLAUDE** ("Foreign Function Interface"→"C Interop", etc.), and **all example .sx
comments** (1219 stdout labels Foreign→Extern, snapshot regenerated). Suite green
(646/444) throughout; snapshot-neutral except the intentional 1219 regen.
**What still contains `foreign` (the analyzed keep-list + the not-yet-done):**
- **KEEP (gate-exempt):** `src/` `hash_foreign` token + lexer entry + `lex hash_foreign`
test (`#foreignx`) + the 4 parser rejection messages ("`#foreign` has been removed…");
`1176-diagnostics-foreign-removed.sx` (its `#foreign` decl + comments ARE the rejection
test); `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored `library/vendors/sqlite/c/*`.
- **NOT YET DONE:** example FILENAMES (`*-foreign*.sx` + the `0729`/`1205`/`1218`/`1219`/
`1306`/`1318`/`1216`/`1217` families) and their `#import`/`#include`/`#source` path refs
+ `expected/` files — needs a git-mv rename step; and **`issues/*.md`** (~20 writeups).
### Prior: Phase 9.1 + 9.2 — internal IDENTIFIER purge COMPLETE (commits 9.1a `b838f63`,
9.1b `b78e7dd`, 9.1c `cd14794`, 9.1d `7ffdc7d`, 9.2a `3354446`, 9.2b `5c8af6e`,
9.2b-fix `a15a868`, 9.2c `d27be42`, 9.2d `8cca3b9`). **Every `foreign` IDENTIFIER in
`src/` is renamed** — the only `foreign` left in `src/` is COMMENTS + the kept token
(`hash_foreign` + its `#foreignx` lexer-boundary test) + the rejection-message string.
Suite green (646/444) at every commit.
- **9.1d** eliminated the `foreign_expr` AST node: migrated `c_import.zig` auto-synth
to the extern shape, deleted the node + `ForeignExpr` + all readers.
- **9.2a/b/c/d** ran the runtime-class family rename (Decision 5 → `Runtime*`):
types `ForeignClassDecl`→`RuntimeClassDecl` etc.; fns `parseForeignClassDecl`→
`parseRuntimeClassDecl`, `lowerForeignMethodCall`→`lowerRuntimeMethodCall`, …; state
`foreign_class_map`→`runtime_class_map`, `foreign_class_decl` variant→
`runtime_class_decl`; the extern-ref validators → `Extern` (linkage, `checkExternRefs`);
the reference flag → **`is_extern`** (per user: reuse existing terminology, not a new
`is_reference`); and `foreign_path`→`runtime_path` COUPLED across the hook boundary
(build.sx accessor `jni_main_runtime_path_at` + the registered hook string +
bundle.sx + specs.md), with 37 `.ir` snapshots regenerated for the renamed
`@BuildOptions.jni_main_runtime_path_at` declare stub (symbol-name change only).
- **9.1a/b/c** (linkage): 5 collision-free renames (callExtern, …); "foreign symbol"
diagnostic + panic → "extern symbol" (1172 regen); deleted dead VarDecl legacy fields.
### Prior: Phase 9.1 (partial) — internal linkage-identifier purge (commits `b838f63` 9.1a,
`b78e7dd` 9.1b, `cd14794` 9.1c). **PHASE 9 STARTED.** Decision 6 = PURGE EVERYTHING,
scoped (user, 2026-06-15): purge `foreign` from **all `.sx` files + all documentation +
all our Zig (`src/`)**, analyzing each grep hit — **legitimate keeps stay**
(SQLITE_CONSTRAINT_FOREIGNKEY + other SQLite API constant names, vendored
`library/vendors/sqlite/c/*`, `1176-diagnostics-foreign-removed.sx` [the rejection test
MUST contain `#foreign`], the parser rejection-message string + `hash_foreign` token
[kept so `#foreign` keeps its friendly deprecation error]).
- **9.1a** (`b838f63`): 5 collision-free linkage renames — `callForeign`→`callExtern`,
`marshalForeignArg`→`marshalExternArg`, `dedupeForeignSymbol`→`dedupeExternSymbol`,
`foreign_name_map`→`extern_name_map`, `is_foreign_c_api`→`is_extern_c_api`.
- **9.1b** (`b78e7dd`): the "foreign symbol already bound" diagnostic (decl.zig) +
resolveFuncByName panic (call.zig) → "extern symbol". Intentional 1172 regen.
- **9.1c** (`cd14794`): **deleted** the dead `VarDecl.is_foreign`/`foreign_lib`/
`foreign_name` fields (the global `#foreign` path rejects → write-dead; 3 coalescing
readers in decl.zig simplified to `vd.extern_name`/`vd.is_extern`).
All snapshot-neutral except the intentional 1172 regen; suite green (646/444).
**COLLISION ANALYSIS (done — drives the rest of 9.1/9.2):**
- `is_foreign` lives on FnDecl?(no — flipped to `extern_export`), **VarDecl (deleted in
9.1c)**, and **ForeignClassDecl (ast.zig:903 — STILL LIVE**, distinguishes runtime-class
reference vs define; renamed in 9.2, not 9.1).
- `is_extern`/`extern_lib`/`extern_name` already exist (VarDecl + IR insts) — so the
old `foreign_*` linkage names could NOT be blind-renamed onto them; 9.1c deleted the
dead VarDecl trio instead of renaming.
- `foreign_expr` (25) is **still BUILT by `c_import.zig` auto-synthesis** (`#import c
{#include}` synthesizes fn bodies as `foreign_expr`). To eliminate it: migrate that
synth path to build the extern shape (empty-block body + `extern_export = .extern_`),
exactly the Phase 5.0 fn-body flip but for auto-synth — THEN delete the `foreign_expr`
node + all readers. This is the last 9.1 item.
### Prior: Phase 8 — CUTOVER: parser hard-rejects `#foreign` (`feat!` commit `3811311`,
preceded by the 8.0 xfail `8180faf` + 3 pre-cutover `refactor`s `2cce6a3`/`720556b`/
`d132aab`). **PHASE 8 COMPLETE.** The prefix `#foreign` linkage directive is removed:
all four parse sites (const-with-type 316, data global 425, fn body 2065, runtime-class
prefix via caller 260) reject it with the migration message *"`#foreign` has been
removed; use the postfix `extern` (import) / `export` (define) linkage keyword
instead"*; added a span-aware `failAt` for the runtime-class case (the lookahead
consumes the token before the reject decision). New example **1176**
(`diagnostics-foreign-removed`) pins it. **Pre-cutover migrations** (all green,
behavior-preserving): the 7 identity `ffi-foreign-*` test DECLS (`2cce6a3`), the two
keyword-neutral diagnostic tests 1172 + 1228 with intentional snapshot regens
(`720556b`), and the 4 multi-file example companions Phase 7 missed (0729/a+b, 1617/c,
1623/mod — `d132aab`). **Deleted** obsolete tests 1174 (`#foreign`+postfix conflict, now
unreachable) + 1620 (`#foreign nosuchunit`, superseded by extern twin 1231), the GATE
A→B unit test + `lowerSrcToIr` helper (nothing left to compare), and converted the
in-source `parse void function with foreign body` parser test to postfix `extern`.
specs.md + readme.md document `extern`/`export` as the sole C-linkage surface. Suite
green (646 corpus / 444 unit, 0 failed).
### Prior: Phase 7.4 — migrate straggler examples `#foreign`→`extern` (`refactor` commit
`1a8991a`). **PHASE 7 MIGRATABLE WORK COMPLETE (7.17.4 done).** Migrated 16 fn/global
examples across categories (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/
1622/1628/1635/1636). Marker'd ones corpus-validated; the 3 unmarked uikit importers
(1607/1608/1616) verified byte-identical via `sx ir` probes. Empty snapshot diff; suite
green (647/444).
**Phase 7 net result:** every example that uses `#foreign` *incidentally* (FFI plumbing,
output-preserving) is now on `extern`/`export`. The **24 files still holding `#foreign`
are exactly the intended keep-list**, all deferred to the Phase 8 cutover:
- **`foreign`-asserting diagnostics** (migrating changes a snapshot): 1172, 1174, 1219
(stdout label), 1228 (equivalence test), 1620.
- **Identity `ffi-foreign-*` feature tests** (real decls; rename/dedup at cutover):
1205-global, 1205-global-helper, 1207, 1218, 1219, 1306, 1318.
- **Comment-only / provenance prose** (decls=0; `#foreign` only in comments): 0716, 0729,
1216, 1223, 1224, 1225, 1229, 1230, 1231, 1332, 1348, 1349, 1426, + issues/0030.
**Lesson (7.3):** the robust class-prefix transform is the GENERAL form
`s/#foreign\s+#(\w+)\((\"[^\"]*\")\)\s*\{/#$1($2) extern {/` — 1417 also used
`#jni_interface`/`#objc_protocol`/`#swift_class`/`#swift_struct`/`#swift_protocol`, and a
`#(objc|jni)_class`-only regex left `extern` in *prefix* position → parse error. All such
directives accept the postfix modifier (probed). Bare defined `#objc_class`/`#jni_class`
examples (no `#foreign`) were left untouched — not a purge target (define→export is an
optional consistency pass, deferrable).
### Prior: Phase 7.1 — migrate incidental 12xx ffi examples (`refactor` commit `731fb8d`).
Migrated 12 plain-C examples (1200/1206/1209-1215/1220/1221/1222); established the
keep-list policy above. Phase 7.2 (`a68f7c2`): 18 13xx obj-c examples (prefix→postfix
classes). Phase 7.3 (`2888f6f`): 13 14xx jni examples incl. 1417 multi-runtime.
### Prior: Phase 6.5 — migrate `gpu/` `#foreign`→`extern`; `library/` now `#foreign`-free
(`refactor` commit `32a7628`). **PHASE 6 COMPLETE.** Final batch: gpu/gles3.sx
(eglGetProcAddress + 1 comment) + gpu/metal.sx (MTLCreateSystemDefaultDevice), bare
fn markers → `extern`. Verified byte-identical `sx ir` on importers 1610 (gles3) +
1606 (metal). **Zero `#foreign` remains anywhere under `library/`** — verified by
`grep -rln '#foreign' library/` → no matches. Suite green (647 corpus / 444 unit, 0
failed).
### Prior: Phase 6.4 — migrate `ffi/` `#foreign`→`extern` (`refactor` commit `666a2e2`).
objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + objc.sx's 2 import runtime
classes (prefix→postfix `extern`). objc + objc_block validated by the 50 marked 13xx
corpus examples (incl. import classes 1300/1301 + defined classes 1339/1349);
raylib/ffi-sdl3/wasm verified by byte-identical `sx ir` probes pre/post. Empty snapshot
diff; suite green.
### Prior: Phase 6.3 — migrate `std/` `#foreign`→`extern` (`refactor` commit `59f90d2`).
Pure source rename across 11 std modules (~60 sites):
cli/core/fmt/fs/log/net.kqueue/process/socket/thread/time/trace. All fn-decl markers
— bare `#foreign;`, `#foreign libc;`/`#foreign tlib;` (LIB ref), and
`#foreign libc "csym";` (LIB+rename) → the same `extern …` tail (`extern` carries the
identical `[LIB] ["csym"]` axis); plus 2 stale comment mentions (fmt/fs). No class
forms in std. These modules ARE host-corpus-exercised → empty snapshot diff is direct
validation. Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 batches:
6.4 ffi (~50, has runtime classes), 6.5 remainder.
### Prior: Phase 6.2 — migrate `platform/` `#foreign`→`extern`/`export` (`refactor` commit
`2cd5d7b`). Pure source rename across uikit/android/android_jni/sdl3 (~64 sites):
30 fn `#foreign;`→`… extern;`; 34 import runtime classes
`#foreign #objc_class/#jni_class("X") {`→`#…_class("X") extern {` (prefix→postfix);
4 defined `Sx*` obj-c classes `#objc_class("X") {`→`… export {`. Behavior-preserving;
empty snapshot diff. **Verification (these modules are largely uncompiled by the
host corpus** — bundle examples import `bundle.sx`, not the runtime modules; android.sx
only compiles under `OS==.android`): byte-identical `sx ir` on uikit importers 1610 +
1606 (which DO compile uikit incl. the 4 defined `Sx*` classes on host) and an sdl3
direct-import probe; android.sx verified by an identical 4-error dedup set (host
pthread clashes — the keyword-neutral "foreign symbol already bound" dedup message is
unchanged, and the probe parsed all migrated `extern` jni classes + EGL fns cleanly
before hitting them). Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6
batches: 6.3 std (~60), 6.4 ffi (~50), 6.5 remainder.
### Prior: Phase 6.1 — migrate `vendors/sqlite` `#foreign`→`extern` (`refactor` commit
`410a52e`). **PART B PHASE 6 STARTED.** Pure source rename: all 97
`sqlite3_* … #foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ the one
stale header-comment reference, line 9). The `extern_lib` axis references the `sqlib`
`#import c` unit identically to `#foreign sqlib`, so IR/output is byte-identical —
empty snapshot diff (only `sqlite.sx` changed), and example 1624
(`vendor-sqlite-module`) stdout byte-unchanged. Suite green (647 corpus / 444 unit,
0 failed).
### Prior: Phase 5.1 — annotate A→B gate post-flip + add fn-rename case (`test` commit
`93e7b6f`). **PHASE 5 COMPLETE → PART B Phase 5 done.** The A→B gate
(`lower.test.zig`) already asserted `#foreign` ≡ `extern`/`export` byte-identical IR
for fn / global / Obj-C class; post-Phase-5.0 the fn-decl + data-global paths build
the SAME extern-named AST, so cases 1/2 are now STRUCTURALLY identical (guaranteed by
construction, not coincidence). Annotated the gate header to record this and keep it
as a regression tripwire (catches a future reader re-diverging the spellings, or a
revert of the flip); case 3 (runtime class) stays behaviorally — not structurally —
equal via the single `is_foreign_eff` field. Added a fn-rename case (case 2b,
`extern_name` axis: `c_abs` → `"abs"`) to broaden coverage beyond bare import
(verified IR-identical via `sx ir` probe before adding). Test-only, no snapshot churn.
Suite green (647 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 — fn-decl `#foreign` body-marker FLIP (`refactor` commit `6b94bb6`).
**PHASE 5.0 PARSER ROUTING COMPLETE.** The fn-body `#foreign [LIB] ["csym"]` marker
now builds the SAME extern AST postfix `extern` produces (`extern_export = .extern_`
+ `extern_lib`/`extern_name` + empty-block body) instead of a `foreign_expr` body.
Behavior-preserving — all four prereqs (visibility, variadic, plain-free, lib-ref)
ensure every downstream reader coalesces `is_foreign` with `extern_export`, so IR +
runtime are byte-identical (full corpus + A→B gate green). Decision 7 churn realised:
example 1620's lib-ref error flips "#foreign library" → "extern library" (the only
snapshot moved; hand-edited, not regen). Parser unit test updated to assert the extern
shape. Spot-checked 1219/1218/0729 (foreign rename / cvariadic / same-name) end-to-end.
**All four `#foreign` parser paths now resolved:** global (`e5ddfbe`) + fn-body
(`6b94bb6`) flipped onto extern; const-with-type is dead (deferred); runtime-class is
already coalesced (`is_foreign_eff`). `c_import.zig` auto-synthesis STILL emits
`foreign_expr` bodies (Phase 6+), so both shapes coexist — every reader stays dual.
Suite green (647 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 prereqs 3 & 4 — plain-free classification + extern lib-ref validation
(plain-free: xfail `2706521` → fix `3c94c14`; lib-ref: xfail `38c3240` → fix
`ad6aed3`). Two MORE extern/#foreign divergences found while de-risking the fn-path
flip, both now closed. **FOUR prereqs total done — the fn-decl flip fully de-risked.**
- **Prereq 3 (plain-free):** `isPlainFreeFn`/`isPlainFreeFnDecl` (resolver.zig:178,
generic.zig:815) excluded a `#foreign` body but classified an empty-block `extern`
fn as a plain free fn — so existing extern fns were wrongly counted in the bare-call
ambiguity verdict (example: two same-name `extern libc "abs"` authors errored
ambiguous, while the `#foreign` twin 0729 compiles). Both predicates now also
exclude `extern_export == .extern_`; `export` (real body) stays plain-free. Example
**1230**.
- **Prereq 4 (lib-ref validation):** `checkForeignRefs` (c_import.zig) validated only
`foreign_expr.library_ref`, so a bogus `extern nosuchunit "abs"` compiled silently
while `#foreign nosuchunit` errors (1620). Now reads the lib ref from EITHER spelling
and names the surface keyword in the diagnostic (so 1620 stays byte-unchanged).
Example **1231**.
- Two OTHER classifying sites probed and found BENIGN for extern (no flip prereq):
namespace/qualified dispatch (`registerQualifiedFn` decl.zig:2208, namespace gate
call.zig:729) — a namespaced `extern` fn resolves identically to its `#foreign` twin
(probe: `cm.c_abs(-9)` → 9 both ways; the registered qualified alias resolves to the
same extern symbol).
### Prior: Phase 5.0 prereq — extern C-variadic tail (xfail `9a2c78d` → fix `0fdc821`) — the SECOND deferred fn-path prerequisite. **BOTH original fn-path prereqs done.** The C-variadic `...` handling was keyed on the `#foreign` (`foreign_expr`)
body shape at two sites — the `is_variadic` drop in `declareFunction`
(`decl.zig:2097`) and the call-site early-out in `packVariadicCallArgs`
(`pack.zig:302`). A variadic `extern` therefore kept its trailing slice param and
slice-packed the extras → garbage at the C ABI (probe: `sum_ints(3,10,20,30)` →
53316585, not 60). Both gates now also fire for `extern_export == .extern_`, so a
variadic `extern` drops the `..args: []T`, sets `is_variadic`, and passes extras
through the C `...` slot with default argument promotion — byte-identical to its
`#foreign` twin. New example **1229** (`1229-ffi-extern-cvariadic`, JIT `#source`,
int-sum + double-avg). Suite green (645 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 prereq — visibility-gate equivalence (xfail `717c35d` → fix `7d8ba1a`) — the first of the two deferred fn-path prerequisites.
The non-transitive C-import visibility gate (`isVisible(.c_import_bare)`,
`decl.zig:2249`) used to recognise only the legacy `#foreign` body shape; a bare
`extern` fn (empty-block body + `extern_export == .extern_`) escaped the gate via
the `body != foreign_expr → return true` arm and was caught only by the general
`isNameVisible` gate — yielding the generic "not visible" wording instead of the
C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings
route to `visibleOverEdges`, and a library-bound `extern LIB` (like `#foreign LIB`)
stays unconditionally visible — so a future fn-decl `#foreign`→`extern` migration
is byte-identical at this gate. New cross-module example **1228**
(`examples/1228-ffi-extern-c-non-transitive`, main → b → c) pins the equivalence:
referencing c's lib-less `#foreign` AND `extern` twins transitively both produce
the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed).
**Empirical finding** (probe, not yet acted on): the bare-extern twin was NEVER a
silent visibility hole — the general `isNameVisible` gate already denied it; only
the *diagnostic wording* diverged. The fix aligns the wording + gate ownership.
### Prior: Phase 5.0 (global path) (`refactor` lock, commit `e5ddfbe`) — **PART B STARTED.**
First of the four `#foreign` parser paths migrated onto the extern AST: the
data-global form `name : T #foreign [lib] ["csym"];` now builds the same
extern-named `VarDecl` (`is_extern`/`extern_lib`/`extern_name`) that postfix
`extern` already produces, instead of `is_foreign`/`foreign_lib`/`foreign_name`.
Behavior-preserving — lowering coalesces both forms identically
(`decl.zig:1119,1127,1141`), so zero snapshot churn. The fn-decl, const-with-type,
and runtime-class `#foreign` paths still build the legacy AST.
### Prior: Phase 4 (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces:
(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) —
asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample
fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start
migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side
fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix
`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse
error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic
— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean
error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three
`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays
documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail).
NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a
harmless dup (both `.c`), and any non-`.c` callconv already errors on its own.
### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/
`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional
`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{`
body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime
class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx
class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same
`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/
lowering change was needed** — the new surface reuses the existing reference-vs-define path.
Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via
JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`),
**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443
unit, 0 fail).
### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
unit, 0 fail).
**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).
## Current state
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class`
(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the
**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A
COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` +
~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223
(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn,
AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426
(jni extern class), 1174/1175 (interplay diagnostics).
## Next step
**NONE — the FFI-linkage stream is COMPLETE.** `extern`/`export` fully replace
`#foreign`; the keyword is rejected; zero `foreign` remains in the gated tree (Parts
A + B, Phases 09 all done; the 9.4 gate passes). This stream can be archived.
Follow-ups (both DONE 2026-06-15, post-stream polish):
- ✅ Added `extern`/`export` to the editors/vscode tmLanguage keyword list as a
`storage.modifier.sx` pattern (`editors/vscode/syntaxes/sx.tmLanguage.json`).
- ✅ Dropped the vestigial `RuntimeClassPrefix.is_extern` field +
`parseRuntimeClassDecl`'s `is_extern` param (always-false dead path; the postfix
`extern`/`export` keyword is the sole reference-vs-define decider). Suite green
(644 corpus / 442 unit, 0 failed).
--- (historical: the finish-Phase-9 plan, now done) ---
**PART B — finish Phase 9: example FILENAME renames + `issues/*.md` + 9.0/9.4.**
(All `src/` identifiers + AST node + all comments/docs/example-comments are DONE.)
0. **FIRST: commit the staged `examples/` comment purge** (a classifier outage blocked
the commit; changes are `git add`ed). Message: "refactor(ffi-linkage): Phase
9.3-examples — purge 'foreign' from example .sx comments".
1. **Example filename rename** (git-mv step, snapshot-careful): rename the `*-foreign*`
example files to extern/runtime names and update every `#import`/`#include`/`#source`
ref + the `expected/<name>.*` companions. Families: `0729-modules-flat-same-name-foreign`
(+ `/a.sx`,`/b.sx` dir), `1205-ffi-foreign-global`(+`-helper`), `1207-ffi-foreign-global-from-helper`,
`1216-ffi-08-foreign-in-method`(+`.h`/`.c`), `1217-ffi-09-foreign-result-chain`(+`.h`/`.c`),
`1218-ffi-foreign-cvariadic`(+`.c`), `1219-ffi-foreign`, `1306-ffi-objc-foreign-class-chained-dispatch`,
`1318-ffi-objc-property-foreign`. ⚠ A renamed file with an `.ir`/`.stderr` snapshot that
echoes its own path will need that snapshot regenerated (intentional). Pick new names
that drop "foreign" (e.g. `…-extern-global`, `…-extern-in-method`, `…-runtime-class-chained-dispatch`).
NOTE: keep `1176-diagnostics-foreign-removed.sx` name (it's the rejection test — fine to keep "foreign").
2. **issues/*.md** (~20) — rewrite writeup prose `#foreign`/`foreign`→`extern`/`runtime-class`.
2b. **`docs/*.md`** — ALSO in the gate scope (was missed; the gate areas are now
`src/ library/ examples/ issues/ docs/ specs.md readme.md CLAUDE.md`). `docs/debugger.md`
referenced the renamed `callForeign` (fixed → `callExtern`, UNCOMMITTED with the staged
batch); sweep all of `docs/` for stale renamed-identifier refs + `foreign` prose.
3. **9.0 surface decision — RATIFIED (user, 2026-06-15): DELETE the `hash_foreign` token.**
The user explicitly flagged token.zig:121 + lsp/server.zig:1693 "this also needs to
go" — total purge, accept `#foreign`→generic error (no friendly migration hint). This
is the LAST src change; it is load-bearing → needs a build + test + 1176 regen (do it
when mutating Bash is back). Steps:
- token.zig: remove `hash_foreign` enum (121).
- lexer.zig: remove the `.{ "#foreign", Tag.hash_foreign }` map entry (91), drop
`#foreign` from the directive-list comment (72), DELETE the `lex hash_foreign` test
(626-631, incl. `#foreignx`).
- parser.zig: remove the 4 `self.current.tag == .hash_foreign` rejection sites (268
caller / 327 / 419 / 2024) + their messages, AND the 2 lookahead refs (`hasFnBody…`
~3658 + ~3676). ⚠ Decide what `#foreign` lexes to with no keyword entry (likely an
error/unknown-directive token) and confirm the parser surfaces a sane error.
- lsp/server.zig: remove the `.hash_foreign,` arm (1693).
- **1176-diagnostics-foreign-removed**: its expected stderr is the now-deleted
"`#foreign` has been removed…" message → it WILL change. Regen 1176's snapshot to
whatever the generic post-deletion error is (intentional), OR delete 1176 entirely
(its purpose — a friendly rejection — no longer exists). Recommend: keep 1176 as a
"`#foreign` is no longer a directive" regression, regen its snapshot. NOTE: after
this, 1176 may still contain `#foreign` in its SOURCE (the rejected token) — that's
the only legitimately-remaining `foreign` in `.sx`, OR rename/rework it to avoid even
that if the gate must be absolute.
4. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` → 0 (no keep-list
left except possibly 1176's source token + `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored C).
--- (historical: the prose-purge plan, now mostly done) ---
**PART B — finish Phase 9: the COMMENT / DOC / issues text purge** (all `src/`
identifiers + the AST node are already done; remaining is prose). Lower-risk than the
renames (text only, mostly snapshot-neutral) but needs per-instance reading — NOT a
blind sed. Footprint: `src/` ~205 (all comments now), `examples/*.sx` ~100 comments,
`issues/*.md` ~20 files, docs (specs/readme/CLAUDE).
Order:
1. **src/ comments** (~200) — reword `foreign`→`extern`/`runtime-class` to match the
renamed identifiers. KEEP: the rejection-message string, the `hash_foreign` token +
its `#foreignx`/`lex hash_foreign` test, and any comment that legitimately explains
the cutover (it must name `#foreign` to say it's removed). The ast.zig FnDecl comment
still says "mirroring `#foreign LIB "csym"` (foreign_lib/foreign_name)" — reword.
2. **examples/*.sx comments** — the deferred provenance comments (full list in the prior
Next-step revision / git). ⚠ Many CONTRAST `#foreign` vs `extern` — reword to stay
coherent. ⚠ `1219-ffi-foreign.sx` prints `"foreign-rename: {}"`/`"=== 15. Foreign ==="`
to STDOUT — changing those regens its snapshot (intentional). `1176`/`1216` legitimately
discuss `#foreign` removal — keep minimal `#foreign` mentions where the test IS about it.
3. **issues/*.md** (~20) — rewrite writeup prose to `extern`/`export`/`runtime-class`.
4. **docs** — specs.md (rename "Foreign Function Interface" heading → "C Interop"; the
`#import c` "foreign declarations" prose; the comptime "foreign function calls" line;
§3344 "foreign code can't observe the error channel"), readme.md (211-212 the `#import
c` exemption prose), CLAUDE.md (host_ffi `#foreign("c")` ref → `extern`; "foreign calls").
5. **9.0 surface decision** (recommend KEEP `hash_foreign` token + rejection for a good
deprecation — then it + the message + 1176 + `#foreignx` are permanent gate-exempt keeps).
6. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` minus the keep-list → 0.
KEEP-LIST (gate-exempt): `SQLITE_CONSTRAINT_FOREIGNKEY` + SQLite API names, vendored
`library/vendors/sqlite/c/*`, the `hash_foreign` token + `#foreignx` test + rejection
message string, `1176-diagnostics-foreign-removed.sx` (rejection test must contain it).
**Gate (scoped per user 2026-06-15):** `grep -rniIE 'foreign'` → 0 across `.sx` files,
all docs, and our `src/` Zig — EXCLUDING the legitimate keeps listed in Last completed
step (SQLite API names, vendored C, the rejection test/message + `hash_foreign` token).
Remaining, in suggested (dependency-safe) order:
1. **9.1d — eliminate `foreign_expr`** (last linkage item): migrate `c_import.zig`
auto-synthesis to build the extern shape instead of a `foreign_expr` body (the Phase
5.0 fn-body flip applied to auto-synth), then delete the `foreign_expr` AST node +
`ForeignExpr` + all readers (25). Snapshot-neutral; verify full corpus (the `#import c`
examples 1215/1216/1217 + sqlite 1624 exercise it).
2. **9.2 — runtime-class family rename → `Runtime*` (Decision 5).** The BIG one, do as
small per-identifier commits with `zig build` after each (snapshot-neutral). Targets
(counts): `ForeignClassDecl`(65)→`RuntimeClassDecl` · `foreign_path`(62)→`runtime_path`
· `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `ForeignMethodDecl`(31)
· `foreign_class_decl`(30) · `foreign_expr`-gone-by-now · `ForeignClassMember`(20) ·
`ForeignFieldDecl`(15) · `ForeignClassDecl.is_foreign`(the live one)→e.g. `is_reference`
· `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` ·
`findForeign*InChain` · `resolveForeign*` · `register*ForeignClass*` · `*ForeignRefs` ·
`ForeignRuntime` · `current_foreign_class`/`_method`. ⚠ COUPLED .sx↔.zig hook names:
`jni_main_foreign_path_at`/`jni_main_foreign_paths`/`hookJniMainForeignPathAt`/
`foreignPathToJavaName`/`splitForeignPath` span build.sx + bundle.sx + compiler_hooks.zig
+ specs.md (2975/3049) — rename all four sites together.
3. **9.x-src-comments** — the ~200 bare-`foreign` comments in `src/` (rename last, since
many reference identifiers that 9.1d/9.2 rename; do AFTER those so the comment text
matches the new names).
4. **9.3-examples comments** — the deferred `.sx` provenance comments (0716, 0729, 1205/
1207/1216/1218/1219/1220, 1223-1231, 1306/1308/1315/1318/1320/1321/1331/1332/1348/1349,
1412/1414/1417/1418/1419/1426, 0117/0415, 1140/1141/1125, issues/0030.sx). ⚠ Many
CONTRAST `#foreign` vs `extern` ("no `#foreign`, no `#library`") or reference renamed
internals — rewrite each to stay coherent (NOT blind sed). ALSO: `1219-ffi-foreign.sx`
prints `"foreign-rename: {}"` to STDOUT — changing it regens the snapshot (intentional).
5. **9.3-issues** — `issues/*.md` writeups (~20 files) → rewrite `#foreign`/`foreign` to
`extern`/`export`/`runtime-class` per the renames.
6. **9.3-docs** — specs.md (12: rename "Foreign Function Interface" heading → "C Interop";
the `#import c` "foreign declarations" prose; the jni_main_foreign_path_at refs with #2),
readme.md (2), CLAUDE.md (2: host_ffi `#foreign("c")` ref + "foreign calls").
7. **9.0 surface decision** — keep `hash_foreign` token + rejection (recommended: good
deprecation) vs delete it. If kept, the token + the rejection-message string + 1176 are
permanent legitimate keeps; the gate excludes them.
8. **9.4 gate** — `grep -rniIE 'foreign'` over the gated set minus the keep-list → 0.
- **6.2 verification note (carry forward):** the `platform/` runtime modules
(uikit/android/android_jni) are NOT compiled by any marker'd host corpus test — verify
future platform-adjacent migrations via direct `sx ir` on importers (1610/1606 compile
uikit on host) or import probes, not the corpus alone.
- **Phases 67** (`refactor` batches, empty snapshot diff per batch): migrate the
stdlib + examples from `#foreign` spelling to `extern`. Because the AST is already
unified, this is a pure SOURCE rename (`#foreign LIB "sym";` → `… extern LIB "sym";`
for fns; the global/const forms similarly), and IR/output must be byte-identical per
batch. NOTE: `c_import.zig` auto-synthesis (`#import c {#include}`) still BUILDS
`foreign_expr` bodies internally — that's a compiler-internal path, migrated separately
(likely Phase 8/9 area), not a source-spelling change.
- **Then Phase 8** (cutover: hard-reject the `#foreign` keyword) and **Phase 9** (purge
all `foreign` identifiers — needs Decision 5 [done, `Runtime*Class*`] + Decision 6
[open, historical carve-out]).
**Watch items carried forward:**
- `c_import.zig:262` auto-synthesis still emits `foreign_expr` — both shapes coexist
until that path is migrated; keep every `body.data == .foreign_expr` reader dual
(checked exhaustively this stream).
- const-with-type `#foreign` parser path (`parser.zig:316`) is still on `foreign_expr`
but DEAD (registers no const); migrate or delete it at the Phase 8 cutover.
- The `decl.zig:2055` "foreign symbol … already bound" dedupe message is keyword-neutral
and fires for both forms — no churn, but reword to "extern" at cutover for consistency. Route the fn-decl `#foreign` path so a
`#foreign` fn builds the SAME extern AST that postfix `extern` already produces,
instead of a `foreign_expr` body. This is the highest-value path (the bulk of
`#foreign` usage). Key sub-questions to resolve before/while routing:
- The `foreign_expr` node carries `library_ref` + `c_name`; the `extern` fn carries
`extern_export = .extern_` + `extern_lib` + `extern_name` on the FnDecl with an
empty-block body. Migration = the parser's fn-body `#foreign` arm
(`parser.zig:~2062`) builds the extern shape (set `extern_export`, map
`library_ref→extern_lib`, `c_name→extern_name`) rather than a `foreign_expr`.
- Lowering ALREADY coalesces the two at every fn site checked this stream
(`decl.zig` 2088/2124/2132/2156/2324/2531 read `is_foreign OR extern_export`),
and the two prereq gates (visibility `decl.zig:2249`, variadic `decl.zig:2097` +
`pack.zig:302`) now do too — so the migration should be behavior-preserving with
ZERO snapshot churn. VERIFY with the A→B gate test (`lower.test.zig`) + a full
`zig build test` after routing; any churn means a site still reads `foreign_expr`
structurally and must be coalesced first.
- ⚠ This ALSO migrates the **const-with-type** path implicitly IF it shares the same
`foreign_expr`→extern reshape (it builds `const_decl{value=foreign_expr}`). Decide:
reshape the const path's value node alongside, or leave the dead const path on
`foreign_expr` until Phase 8 cutover. The const path is dead (see findings below),
so leaving it is acceptable; but the parser arm is shared-ish — check whether the
fn-body arm change touches it.
- Cadence: because the migration is behavior-preserving (no churn), it's a single
`refactor`/lock commit (like the 5.0 global-path commit `e5ddfbe`), NOT an
xfail→fix pair.
**Investigation findings (this session — reorder the remaining paths):**
- **const-with-type** (`parser.zig:316`, `name :: type_expr #foreign`) is a
**DEAD path**: it builds `const_decl{value = foreign_expr}`, but
`registerTypedModuleConst` (`decl.zig:848-851`) bails on a `foreign_expr` value
(`else => return`), so it registers no const and emits no symbol — a probe
(`g_abs :: FP #foreign "abs";`) returns `unresolved 'g_abs'` at the use site, and
the form is used NOWHERE in `library`/`examples`/`issues`. Its migration target is
ambiguous because the `foreign_expr` value node is SHARED with the fn-decl path,
which isn't migrated yet. **Decision (user, 2026-06-14): defer it — migrate it
alongside the fn-decl path once `foreign_expr`'s extern shape is decided.** The
checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the
"confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it).
- **runtime-class prefix** (`parser.zig:~1351`, `#foreign #objc_class/#jni_class`) is
**ALREADY coalesced**: both prefix `#foreign` and postfix `extern` feed the single
`is_foreign_eff`→`is_foreign` field on `foreign_class_decl` (`parser.zig:1421-1432`),
so there is NO Phase 5.0 AST change for it — only the Phase 9.2 `Runtime*Class*`
rename remains. Drop it from the Phase 5.0 path list.
So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then
the fn-decl `#foreign` body-marker migration. const-with-type + runtime-class need
no standalone Phase 5.0 commit.
Then Phase 5.1 (`lock`): unit test that `#foreign` and `extern` produce identical IR (the
A→B gate already covers fn/global/class — extend or reuse `lowerSrcToIr`). Then Phases 67
migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject
`#foreign`), Phase 9 total `foreign` purge.
**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target
(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep
`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames;
the plan says confirm before Phase 9, but worth raising with the user before sinking Part B
effort. **Also pick up the two Deferred items below at the start of Part B** (the
visibility-gate equivalence in particular needs a cross-module example).
**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).
**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md`
document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) ~~visibility-gate
equivalence~~ — **DONE** (`717c35d`/`7d8ba1a`): the `c_import_bare` gate now polices a
lib-less `extern` fn identically to its lib-less `#foreign` twin (same C-specific
diagnostic); a library-bound `extern LIB` stays unconditionally visible. Locked by the
cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the
general `isNameVisible` gate already denied it; only the diagnostic wording diverged.)
## Open decisions
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B:
- **Decision 5 RATIFIED** (user, 2026-06-14): runtime-class rename target = `Runtime*Class*`
(object-model axis, not linkage). Drives the Phase 9.2 identifier renames.
- **Decision 6 RATIFIED** (user, 2026-06-15): **PURGE EVERYTHING** — the Phase 9.4 gate is
absolute, including `issues/*.md` writeups (NOT the recommended keep-provenance default).
Every `#foreign`/`foreign` reference in the gated tree (`src/ library/ examples/ issues/
specs.md readme.md CLAUDE.md`) is rewritten to `extern`/`export`; provenance lives in git
history + `(Regression issue NNNN)` notes, not the keyword spelling.
- **Decision 7 RATIFIED** (user, 2026-06-15): **accept the churn** — `#foreign`-spelled
decls produce `extern`-worded diagnostics; example 1620 regenerated (only snapshot moved).
Aligns with Part B's extern-only end state; the interim oddity is cosmetic and removed at
the Phase 8 cutover. Landed in the fn-body flip `6b94bb6`. (Original framing below.)
— interim diagnostic wording for `#foreign`-spelled decls (gated the fn-body flip). Once the flip lands, a `#foreign`-spelled fn builds the extern AST, so any
diagnostic that reads the unified AST can no longer tell the user wrote `#foreign` vs
`extern`. Concretely, example 1620's lib-ref error flips "#foreign library…" →
"extern library…". Options: **(A, recommended)** accept the narrow churn — regen 1620 as
intentional; it aligns with Part B's `extern`-only end state and the interim oddity
(`#foreign` source → "extern" message) is cosmetic and short-lived (Phase 8 cutover
removes `#foreign`). **(B)** retain a one-bit surface marker on `FnDecl` (`wrote_foreign`)
so interim diagnostics stay keyword-accurate (zero churn, small extra plumbing, marker
deleted at cutover). Affects only diagnostic wording — IR/behavior identical either way.
## Log
- (9.0 + 9.3 + 9.4) **PHASE 9 COMPLETE — STREAM DONE; 9.4 GATE PASSES.** Deleted the
hash_foreign token (9.0, `dfae690`); purged all `foreign` comments incl. capital-F
(src/examples/docs/editors); renamed 10 `*-foreign*` example files + dedup'd 1218
(`b52d424`); rewrote 20 issues/*.md + renamed 0043 (`b9cfe25`). Gate: zero `foreign`
in the gated tree except `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored sqlite c/. Suite
green (644/443). User flagged several leftover areas mid-purge (docs/, editors/,
capital-Foreign comments, the token) — all addressed.
- (9.3 src capital-Foreign) Fixed the case-sensitivity gap — my earlier src verify grep
was case-sensitive, missing ~21 capital `Foreign`/`FOREIGN` comments (Foreign-class→
Runtime-class, Foreign path→Runtime path, Foreign decls→Extern decls, FOREIGN function→
extern function, etc.) across calls/inst/ffi_objc/jni_descriptor/emit_llvm/c_import/
lower.* /ops.zig. All reworded via Edit (comments only — no build impact). UNCOMMITTED
(mutating Bash blocked by a classifier outage). After this, src `foreign` = ONLY the
`hash_foreign` token machinery + 4 rejection messages (the 9.0-delete targets).
- (9.0 RATIFIED) User: DELETE the hash_foreign token (total purge). Pending build+regen.
- (9.3 text purge) Purged `foreign` from all `src/` comments (`dc51c4b`), specs/readme/
CLAUDE (`e99383f`), and all example .sx comments (STAGED, commit pending a classifier
outage). Fixed 2 user-facing diagnostics (type-annotation error, Android jni_main help).
1219 stdout labels Foreign→Extern (regen). Suite green (646/444). Remaining: example
FILENAMES + issues/*.md + the 9.0 token decision + 9.4 gate.
- (9.2a-d) **RUNTIME-CLASS IDENTIFIER PURGE COMPLETE** (Decision 5 → `Runtime*`).
9.2a types (`3354446`), 9.2b fns+state+`is_extern` flag (`5c8af6e`, fixed `a15a868`
per user: reuse `is_extern` not new `is_reference`), 9.2c extern-ref validators →
`Extern` (`d27be42`), 9.2d `foreign_path`→`runtime_path` coupled across the build-hook
boundary + 37 `.ir` regens (`8cca3b9`). `src/` now has ZERO `foreign` identifiers
(only comments + the kept token/message remain). Suite green throughout.
- (9.1d) Eliminated the `foreign_expr` AST node — migrated `c_import.zig` auto-synth to
the extern shape, deleted the node + all readers. `refactor` `7ffdc7d`.
- (9.1c) Deleted dead `VarDecl.is_foreign`/`foreign_lib`/`foreign_name` (global `#foreign`
rejects → write-dead); 3 decl.zig readers simplified to `vd.extern_name`/`vd.is_extern`.
Snapshot-neutral; suite green (646/444). `refactor` `cd14794`.
- (9.1b) "foreign symbol already bound" diagnostic + resolveFuncByName panic → "extern
symbol"; intentional 1172 regen. Suite green. `refactor` `b78e7dd`.
- (9.1a) **PHASE 9 STARTED.** 5 collision-free linkage renames (callForeign→callExtern,
marshalForeignArg, dedupeForeignSymbol, foreign_name_map→extern_name_map,
is_foreign_c_api). Snapshot-neutral; suite green. `refactor` `b838f63`. Decision 6
scoped by user: purge `.sx` + docs + our `src/` Zig, keep legitimate hits (SQLite API
names, vendored C, the rejection test/message + hash_foreign token).
- (8.1 cutover) **PHASE 8 COMPLETE.** Parser hard-rejects `#foreign` at all 4 sites
(const/global/fn-body via `self.fail`; runtime-class via `self.failAt` at the caller,
new helper); greens xfail 1176. Deleted obsolete 1174 + 1620, the GATE A→B test +
`lowerSrcToIr` helper; converted the in-source parser test to postfix `extern`;
`extern_export` → `const`. specs.md + readme.md drop `#foreign`. Suite green
(646/444). `feat!` `3811311`.
- (8.0 xfail) Added `1176-diagnostics-foreign-removed.sx` pinning the desired rejection.
RED (still accepted). `test`/xfail `8180faf`.
- (8 pre-cutover) Migrated the 4 multi-file example companions Phase 7 missed
(0729/a+b, 1617/c, 1623/mod). `refactor` `d132aab`.
- (8 pre-cutover) Migrated keyword-neutral diagnostics 1172 (decl→extern, message stays
internal "foreign symbol") + 1228 (→ two foreign-free extern symbols c_abs_one/_two),
intentional snapshot regens reviewed. `refactor` `720556b`.
- (8 pre-cutover) Migrated the 7 identity `ffi-foreign-*` test decls to extern/export
(decls only; comments left for Phase 9.3). `refactor` `2cce6a3`.
- (7.4 stragglers) **PHASE 7 MIGRATABLE WORK COMPLETE.** Migrated 16 fn/global examples
(0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/1622/1628/1635/1636) `#foreign`→
`extern`; 1607/1608/1616 (unmarked) verified by `sx ir` probes. 24-file keep-list remains
by design (deferred to Phase 8). Suite green (647/444). `refactor` `1a8991a`.
- (7.3 14xx) Migrated 13 jni examples (1410-1419/1423/1424/1425). 1417 (all-runtimes) hit
a parse-error trap: a `#(objc|jni)_class`-only regex left `extern` in PREFIX position on
`#jni_interface`/`#objc_protocol`/`#swift_*` lines → fixed with the GENERAL
`#foreign #(\w+)("X") {`→`#$1("X") extern {` rewrite (all such directives accept the
postfix modifier, probed). Kept 1426 (comment-only). Suite green. `refactor` `2888f6f`.
- (7.2 13xx) Migrated 18 obj-c examples (1308/1311-1321/1341-1347): prefix→postfix import
classes + fn markers. Kept identity 1306/1318, comment-only 1332/1348/1349. No 13xx
snapshot asserts on foreign. Suite green. `refactor` `a68f7c2`.
- (7.1 12xx) **PHASE 7 STARTED.** Migrated 12 incidental plain-C examples
(1200/1206/1209-1215/1220/1221/1222) `#foreign`→`extern`; output byte-identical,
empty snapshot diff, corpus-validated. Established the keep-list policy (see Last
completed step): kept 1172/1174/1620/1228 + ffi-foreign-* (1205/1207/1216/1218/1219)
+ comment-only 1223/1229/1230/1231 for Phase 8. Suite green (647/444). `refactor`
`731fb8d`.
- (6.5 gpu) **PHASE 6 COMPLETE.** Migrated `gpu/gles3.sx` + `gpu/metal.sx` (3 sites);
`library/` now `#foreign`-free (`grep -rln '#foreign' library/` → 0). Verified
byte-identical `sx ir` on importers 1610/1606. Suite green (647/444). `refactor`
`32a7628`.
- (6.4 ffi) Migrated `ffi/` objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers +
objc.sx's 2 import classes (prefix→postfix `extern`). objc/objc_block validated by 50
marked 13xx examples; raylib/ffi-sdl3/wasm by `sx ir` probes pre/post. Empty snapshot
diff; suite green (647/444). `refactor` `666a2e2`.
- (6.3 std) Migrated 11 `std/` modules (~60 sites): cli/core/fmt/fs/log/net.kqueue/
process/socket/thread/time/trace. All fn-decl markers (bare / `libc`|`tlib` LIB ref /
`libc "csym"` rename) → `extern …` + 2 comment mentions; no class forms. Host-corpus-
exercised → empty snapshot diff validates. Suite green (647/444). `refactor` `59f90d2`.
- (6.2 platform) Migrated `platform/` (uikit/android/android_jni/sdl3, ~64 sites):
30 fn `#foreign;`→`extern;`, 34 import classes prefix `#foreign #objc/jni_class`→
postfix `… extern {`, 4 defined `Sx*` objc classes → `… export {`. Behavior-
preserving, empty snapshot diff. Verified byte-identical `sx ir` on uikit importers
1610/1606 + sdl3 probe; android via identical 4-error dedup set (host-only module).
Suite green (647/444). `refactor` `2cd5d7b`. NOTE: these runtime modules aren't in
the marker'd host corpus — verified out-of-band.
- (6.1 sqlite) **PHASE 6 STARTED.** Migrated `vendors/sqlite/sqlite.sx`: 97
`#foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ line-9 comment).
`extern_lib` references the `sqlib` `#import c` unit like `#foreign sqlib`; IR
byte-identical, empty snapshot diff, example 1624 stdout unchanged. Suite green
(647/444). `refactor` `410a52e`.
- (5.1 gate annotate) **PHASE 5 COMPLETE.** Annotated the A→B gate header
(`lower.test.zig`) to record that post-Phase-5.0 the fn/global `#foreign` paths
build the same extern-named AST → cases 1/2 are structurally (not coincidentally)
identical; the gate stays as a regression tripwire. Added fn-rename case 2b
(`c_abs` → `"abs"`, `extern_name` axis), IR-identical per a `sx ir` probe.
Test-only, no snapshot churn. Suite green (647/444). `test` `93e7b6f`.
- (5.0 fn-body flip) **PHASE 5.0 PARSER ROUTING COMPLETE.** Flipped the fn-body
`#foreign` parser arm (`parser.zig:~2062`) onto the extern AST (empty-block body +
`extern_export = .extern_` + extern_lib/extern_name); `extern_export` made `var` so
the body arm can route onto it. Updated the parser unit test to assert the extern
shape. Behavior-preserving via the four prereqs; only example 1620's lib-ref message
churned ("#foreign library"→"extern library", Decision 7, hand-edited). Suite green
(647 corpus / 444 unit). `refactor` `6b94bb6`.
- (5.0 prereq plain-free xfail) Added `1230-ffi-extern-same-name-authors` (two flat
authors of `absval` via `extern libc "abs"`; the `extern` twin of `#foreign` 0729).
RED — extern authors wrongly counted as ambiguous (646/1 fail). `test`/xfail `2706521`.
- (5.0 prereq plain-free fix) `isPlainFreeFn`/`isPlainFreeFnDecl` now also exclude
`extern_export == .extern_` (external C symbol, no sx body; name-keyed first-wins like
`#foreign`); `export` stays plain-free. 1230 green (`absval = 7`). Suite green (646/444).
`fix`/green `3c94c14`.
- (5.0 prereq lib-ref xfail) Added `1231-ffi-extern-undeclared-lib` (`extern nosuchunit
"abs"` — bogus lib ref). RED — compiles silently (extern lib ref unvalidated).
`test`/xfail `38c3240`.
- (5.0 prereq lib-ref fix) `checkForeignRefs` (c_import.zig) now reads the lib ref from
either spelling (foreign_expr.library_ref OR extern_lib) and names the surface keyword,
so 1620 (#foreign) is byte-unchanged and 1231 (extern) gets "extern library … not
declared". 1231 green. Suite green (647/444). `fix`/green `ad6aed3`. **ALL FOUR fn-path
prereqs DONE → fn-body flip de-risked; awaiting Decision 7 (interim wording).**
- (5.0 prereq variadic xfail) Added `1229-ffi-extern-cvariadic` (JIT `#source`,
int-sum + double-avg, `extern` C-variadic). Expected snapshot pins the DESIRED
correct output. RED (variadic `extern` slice-packs extras → garbage:
`sum_ints(3,10,20,30)` → 53316585; doubles → 0.0). `test`/xfail `9a2c78d`.
- (5.0 prereq variadic fix) Extended the two C-variadic gates — the `is_variadic`
drop in `declareFunction` (`decl.zig:2097`) and the early-out in
`packVariadicCallArgs` (`pack.zig:302`) — to fire for `extern_export == .extern_`
as well as a `foreign_expr` body. 1229 green (`60` / `2.000000`). Suite green
(645 corpus / 444 unit, 0 failed). `fix`/green `0fdc821`. **BOTH fn-path prereqs
DONE → fn-decl `#foreign` body-marker migration unblocked.**
- (5.0 prereq vis xfail) Added cross-module example `1228-ffi-extern-c-non-transitive`
(main → b → c). Main references c's lib-less `#foreign` + `extern` twins
transitively; expected snapshot pins the DESIRED equivalent C-specific
diagnostic for both. RED (extern twin gets the generic "not visible" wording —
443/444). `test`/xfail commit `717c35d`; the fix greens it.
- (5.0 prereq vis fix) Extended `isVisible(.c_import_bare)` (`decl.zig:2249`) to
switch on the body: a `foreign_expr` body OR an `extern_export == .extern_` decl
with no lib both route to `visibleOverEdges`; a library-bound decl stays
unconditionally visible. 1228 green — both twins emit "C function not visible".
Suite green (644 corpus / 444 unit, 0 failed). `fix`/green commit `7d8ba1a`.
**Deferred prereq (b) CLOSED.** Investigation this session also found
const-with-type is a DEAD parser path (defer per user) and the runtime-class
prefix is already coalesced (no Phase 5.0 change) — see Next step.
- (5.0 global) **PART B STARTED.** Routed the `#foreign` data-global parser path
(`parser.zig:425`) onto the extern-named `VarDecl` (`is_extern`/`extern_lib`/
`extern_name`) — the same AST postfix `extern` builds. Behavior-preserving
(lowering coalesces both at `decl.zig:1119,1127,1141`); zero snapshot churn. Suite
green (444/444 unit, 643 corpus). `refactor` lock, commit `e5ddfbe`. Remaining
Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic
prereqs), runtime-class prefix (1305).
- (init) Plan written; FFI-linkage stream opened.
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
classification + `lex linkage keywords` test. Suite green; no identifier collisions
in the corpus. `lock` commit.
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
`VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
parser unit tests. Suite green (443/633). `lock` commit.
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
(decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
Suite green (443/633). `lock` commit.
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
`lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
`declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
recovered a formatter-clobbered `parser.zig` — see Known issues.)
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
`extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
extern_name → declare under C name, map sx→C) and extends the dedupe guard to
extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
`green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
`is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
(636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
(JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
(`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
milestone (see Next step). Spike reverted; no commit.
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
`corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
(637/443). `green` commit.
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
`triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
undefined). `xfail`; 2.2b greens it.
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
the body promotes into the C-named function (`define @triple_c`). sx-side call sites
resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.
- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl`
(optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed
`is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses
the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443).
`green` commit. **PHASE 3 COMPLETE.**
- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`,
parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class,
`counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per
the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens`
newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr
0-byte per repo convention; runner normalizes both.)
- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`:
`#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class.
Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the
equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three).
Suite green (641/444). `test` commit.
- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign`
+ postfix `export` on an aggregate previously surfaced a confusing internal
"emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`:
`parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green.
- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the
`#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit.
- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on
one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`:
`parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite
green (643/444). **PHASE 4 COMPLETE → PART A DONE.**
- (golden-fix) **`-Dupdate-goldens` churn RESOLVED.** Root cause was NOT a code bug:
`writeGolden` always writes `content + "\n"` (empty → canonical 1-byte `\n`, used by 484
of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte
*outliers* (verify trims trailing `\n` so both forms passed, but regen always rewrote them
to 1-byte). Conformed all 5 to the 1-byte form → `-Dupdate-goldens` is now idempotent, no
more churn. (Separately: a flaky `0712-sha256-streaming` >10s timeout appears only under
concurrent `zig build` load — not a real failure; re-run serially.)
## Known issues
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
working-tree `src/parser.zig` between commits — it reformatted one-liners AND
silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
committed version). **After any Edit-tool change to a file the IDE may have open,
rebuild + run the affected example before trusting the edit.**

View File

@@ -0,0 +1,970 @@
# CHECKPOINT-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step at a time,
per the cadence rule). New corpus category: `18xx` concurrency.
## Last completed step
**B1.6 — aarch64-LINUX port of the M:1 fiber runtime (sched.sx).** `library/modules/std/sched.sx`
now runs end-to-end on aarch64-linux as well as aarch64-macOS, validated **byte-identical** on both
via Apple `container` (static ELF, no emulation). The per-OS bits are comptime-branched:
- `MAP_AP` (mmap MAP_ANON flag) — `inline if OS == { case .linux: 0x22 case .macos: 0x1002 }`,
exhaustive on the supported OSes (no default → a new target fails loud on use).
- The fd-readiness backend — kqueue on darwin, **epoll on linux**. The `epoll` import is scoped to
the linux branch (`inline if OS == .linux { ep :: #import "modules/std/net/epoll.sx" }`) so darwin
never pulls epoll types into the concurrency examples (the std-barrel-drift rule). `block_on_fd`, the
run-loop Mode-2 drain, and `cancel_io_waiter_for` each branch kqueue/epoll; epoll additionally
`EPOLL_CTL_DEL`s on fire + on early-wake (EPOLLONESHOT only DISABLES, kqueue EV_ONESHOT auto-removes).
- The first-entry trampoline was redesigned from a per-OS hand-written global-asm symbol to a **naked
sx fn** `fib_tramp` (`mov x0, x19; br x20`) + register-indirect dispatch (spawn presets
`regs[1] == x20 == &fib_dispatch`), so no per-OS `.global _fib_tramp`/`fib_tramp` symbol literal is
needed. This sidesteps a compiler bug (wrapped top-level `asm` dropped — now **issue 0194**, OPEN).
**Bug fixed en route (issue 0193 Bug A):** the tramp redesign initially bus-errored on the 1817
go/wait/sleep capstone (both OSes) because the WIP had dropped `export "fib_dispatch"`. Without the
export `fib_dispatch` uses sx's internal ABI (x0 = implicit `context`, `self` shifted to x1), but the
trampoline hands `self` in x0 (C-ABI) → on first entry the body runs (x1 happens to alias `self`) but
the closure then loads `regs[1] == &fib_dispatch` as its first capture and recurses forever → stack
overflow. **Fix: restore `export "fib_dispatch"`** (pins it to C-ABI, `self` in x0). Root cause found
via lldb on an AOT macOS build; confirmed by an adversarial source review (`src/ir/lower/decl.zig`).
The 1817 capstone in the suite guards the fix. Suite GREEN **817/0**; 1811/1814/1816/1817 byte-identical
macOS host ↔ aarch64-linux container.
### Earlier — B1 follow-up — `Scheduler.deinit` (close the bounded leaks). Post-B1 non-blocking cleanup: a
terminal `deinit` on `library/modules/std/sched.sx`'s `Scheduler` releases the resources B1 documented
as leaked. Frees, in order: (1) any fibers still enqueued ready (leak-safety net for `spawn`/`go`
without `run()``munmap` stack + free struct; a suspended off-queue fiber is unreachable, but a clean
`run()` aborts on orphans so none survive it); (2) every heap `*Task` from `go` — newly tracked via a
`task_allocs: List(*void)` field appended in `go` (the scheduler otherwise has no handle on its generic
`Task($R)`s); (3) the three `List` backings (`task_allocs`/`timers`/`io_waiters`, all grown through
`own_allocator`); (4) the lazily-opened kqueue fd (`close`, reset to `-1`). NOT freed (unchanged
language limitation): the per-`spawn`/`go` closure env (sx exposes no env-free). Idempotent (rests on
`List.deinit` nulling `items` + the `kq`/`ready_head` resets); TERMINAL contract — no scheduler-owned
handle (`*Task`, `*Fiber`, the scheduler) is usable after `deinit`.
- Added a canonical `close :: (i32) -> i32 extern libc` (matches the dedupe-canonical signature 1816
already uses) + the `task_allocs` field.
- Locked by `examples/concurrency/1820-concurrency-fiber-scheduler-deinit.sx` (aarch64-macOS `.build
{"target":"macos"}`, runs end-to-end): one run touches every freed resource — a SLEEPER (`timers`), a
pipe READER `block_on_fd` + WRITER (kqueue fd + `io_waiters`), two `go` tasks (`Task`s + `task_allocs`)
— then `deinit`. Verified by a tracking `GPA`: `freed by deinit: 5`, `live after deinit: 5` (the
RESIDUAL = the 5 documented closure envs, not a bug), `kq open after run: true` → `kq after deinit:
-1` (the genuinely-open kqueue fd is closed), `read: 3 [97 98 99]` (the fd path actually ran). Counts
captured into locals BEFORE printing (`print` allocates format temporaries through the same GPA).
- **Adversarially reviewed (worker):** no real memory-safety bug in the supported (deinit-after-`run`)
path — reap-loop reads `f.next` before freeing `f`, the three freed List backings + Tasks + kq are all
disjoint + scheduler-owned, no over-free, idempotent. The one CRITICAL it raised was a DOC contradiction
(step-(1) defensive reap vs step-(2) "post-run only"), reconciled by spelling out the terminal contract.
Its 0154-over-store concern (`.{}`→`List` writes in `init` could clobber `kq`) was PROBED and cleared:
`kq == -1` immediately after `init`, all fields clean. Suite GREEN **759/0**.
### Earlier — B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE
A single capstone exercises the whole
colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async
`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)`
`swap_context` on guarded `mmap` stacks (B1.0B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`:
a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in
SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in
DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual
clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**.
**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the
`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler,
suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via
kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx`
(18001817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1,
zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a;
0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`),
0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`;
routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env
/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve.
### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS)
`library/modules/std/sched.sx` now lets a
fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready.
Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes),
`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT`
constants) rather than re-deriving the FFI — sched.sx imports it as `kqb`. Added to `Scheduler`:
- `kq: i32` (LAZY — `-1` in `init`, opened by the first `block_on_fd`, so a pure-compute /
virtual-timer scheduler never opens a kqueue fd; leaks one fd at exit once opened, same class as the
documented spawn-env / go-Task leaks — no deinit yet);
- `io_waiters: List(IoWaiter)` (`IoWaiter :: struct { fd: i32; fiber: *Fiber; }`, grown through
`own_allocator` per the long-lived-container rule);
- `block_on_fd(self, fd, want_read)` — lazily opens `kq`, arms a one-shot `EVFILT_READ` registration,
records an `IoWaiter{fd, current}`, then `suspend_self()`. Guards a null `current` (loud abort, like
`sleep`); `want_read=false` (write-readiness) is not wired yet → loud abort rather than silently
arming a read filter.
- Run-loop: after the ready queue drains, **Mode 1 (virtual time)** fires the earliest pending timer
(takes precedence — a program uses `sleep` OR fds, documented non-unification limitation); **Mode 2
(real fd)** — if `io_waiters` is non-empty, BLOCK on `kq_wait(kq, evbuf, MAXEV=16, -1)` (null
timeout), then for each fired event match `ev.ident` back to its waiter, evict it, and `wake` the
fiber; **else** break. Orphan-deadlock check unchanged in spirit but now correct: an fd waiter is NOT
an orphan (while `io_waiters.len > 0` the loop blocks on kqueue rather than reaching the check), and
a genuine no-timer/no-fd suspend still aborts loudly (verified with a probe: exit 134).
- `wake` now also evicts a pending fd-waiter (`cancel_io_waiter_for`, mirror of `cancel_timer_for`) —
same UAF reasoning: a fiber woken by another path must not leave a stale `IoWaiter` pointing at a
reaped `*Fiber`. The kqueue registration is `EV_ONESHOT` so we never `EV_DELETE` (a never-fired
one-shot lingers harmlessly; the drain ignores an unmatched ident; closing the fd auto-removes it).
- DE-RISK probe (run first, no scheduler): confirmed `size_of(Kevent) == 32`, the pipe roundtrip
(`kq_wait` returned 1 with `out.ident == read_fd`, `out.filter == -1` (EVFILT_READ), `out.data == 1`
byte readable) — the struct layout reads back the fd correctly.
- Locked by `examples/concurrency/1816-concurrency-fiber-io-pipe.sx`: a `pipe`; a reader fiber spawned
FIRST blocks on the empty read end, then a writer fiber writes `a b c` → the run loop blocks on
kqueue, wakes the reader, which reads the 3 bytes. Output `log: wrote read 3 [97 98 99]` /
`n_suspended: 0` (the "wrote" before "read" ordering proves the reader actually blocked then woke via
kqueue readiness). `.build` `{ "target": "macos" }` (matches host arch → runs end-to-end; ir-only on
a mismatch, like 1814/1815 — no `.ir` snapshot needed since it runs here). The example declares its
own `read`/`write`/`close` externs with the CANONICAL signatures std already binds
(`(i32,[*]u8,usize)->isize` / `(i32)->i32`) — a divergent re-binding is rejected by the extern dedupe.
- **Adversarial review (worker) of the run-loop change — found 2 CRITICALs:**
- **(1) two fibers on the SAME fd → lost wakeup + permanent hang.** macOS `EV_ADD` for an existing
`(ident, filter)` REPLACES the registration (doesn't stack), so two waiters share one registration:
the fd fires once, one wakes, the other is stranded in `io_waiters` and the next `kq_wait(-1)` blocks
forever. FIXED: `block_on_fd` now enforces one-waiter-per-fd with a loud abort (the model already
assumed it). Verified: dup-fd → `sched: block_on_fd: fd N already has a waiter`, not a hang.
- **(2) an fd-waiter on a never-ready fd hangs instead of the timer path's loud abort.** Re-examined:
this is CORRECT event-loop semantics — blocking on I/O until ready (possibly forever, like a server
idling on a socket) is the point; the scheduler cannot know an fd will never become ready, so it must
keep waiting. NOT a scheduler deadlock. Fixed the MISLEADING comment that implied the orphan check
covers fd-waiters: it does not, by design (it covers only pure `suspend_self` parks). No code change —
the "hang" is a caller-side logic issue (waiting on input that never arrives), not a bug to abort on.
- Review CLEARED: the IoWaiter UAF parity (early-wake evicts the waiter; a lingering one-shot that later
fires hits no match → clean no-op), ident width/sign, `kq_wait` EINTR/error handling, timer-vs-io
precedence (timer wins; no hang). All probed safe.
- Suite GREEN **754/0** (incl. the dup-fd guard, no new example needed — the abort is host-fragile to
pin like 1809's guard-firing). Next: **B1.5** (end-to-end M:1 validation under the deterministic timers
/ fd readiness); a linux epoll twin of `block_on_fd` (mirror via `std/net/epoll`, the OS-neutral facade
is `std.event`) is future work.
### Earlier — B1.4b — deterministic VIRTUAL-TIME timer scheduling (the KEYSTONE) — landed + adversarially
reviewed (caught a CRITICAL UAF, fixed).
`library/modules/std/sched.sx` gained a virtual clock +
sleep timers so fibers schedule in reproducible simulated time (no real clock): `clock_ms` (advances
ONLY as timers fire), a `timers: List(Timer)` (insertion-order, linear min-scan, FIFO tiebreak),
`now_ms()`, `sleep(ms)` (arm `{clock_ms+ms, current}` + `suspend_self`), and a timer-driven `run`
(drain ready → fire earliest timer → advance clock → wake → repeat; orphan-deadlock check preserved
for a genuine no-timer suspend). Locked by `1814` (5 fibers sleep 30/10/20/15/15 → wake order
B@10, D@15, E@15 (FIFO), C@20, A@30 — deadline order, not spawn order; `now_ms()` reads each virtual
deadline; final clock 30). §8.1.3 calibration note in the header: the deterministic wake ORDER
equals what real `sleep`s produce, reproducing blocking semantics' observable ordering without real
time. The deterministic-sim `Io` is realized at the scheduler level (`sleep`/`now_ms`/timer-`run`),
not as an erased `Io`-protocol impl (same erasure reason as FiberIo).
- **Adversarial review (worker) of the run-loop change: found a CRITICAL use-after-free** — a fiber
that armed a `sleep` timer but was woken EARLY by another path (a manual/`Task` `wake`) ran to
completion + was reaped (stack `munmap`'d, `Fiber` freed) while its `Timer` still held a dangling
`*Fiber`; a later fire would `wake` freed memory (silent-corruption: "passes" only because the
freed slot coincidentally read `state != .suspended`). FIXED: `wake` now evicts the woken fiber's
pending timer (`cancel_timer_for`) — every re-ready path funnels through `wake` (the timer-fire in
`run` already removed the fired timer, so it's a harmless re-scan there), so no stale timer can
outlive its fiber. Regression `1815-concurrency-fiber-timer-early-wake.sx` (early wake → `clock: 0`,
the stale 100ms timer evicted, not fired). Review CLEARED: `n_suspended` accounting,
orphan-deadlock false-positives, timer-list integrity (re-arm during fire), clock monotonicity,
termination — all traced/probed safe.
- Suite GREEN (count below). Next: **B1.4c** (event-loop `Io` — real fd readiness, kqueue/epoll).
### Earlier — B1.4a — a truly-SUSPENDING fiber-task async layer (`go`/`wait`/`cancel`)
landed + adversarially reviewed; cleared two more compiler blockers en route. `library/modules/std/sched.sx`
now carries `Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` (a `ufcs` layer over
the M:1 scheduler). `s.go(work)` runs the nullary thunk `work` as a REAL fiber; `t.wait()` SUSPENDS
the caller until it completes (vs io.sx's blocking `context.io.async`, which runs inline). Locked by
`examples/concurrency/1813-concurrency-fiber-async-suspend.sx`: two tasks interleave (A yields
mid-body so B runs first → `1 2 3`), awaited values `42`/`100`, and a canceled task's `wait` raises
`.Canceled` → `or -99` → `sequence: 1 2 3 42 100 -99`.
- **Design: a NULLARY thunk, not `async(worker, ..args)`.** A comptime variadic pack can't cross a
deferred (fiber) boundary — `..args` captured into a closure re-expands from the spawner's
now-gone locals (issue 0156 Part 2). So `go` takes `work: Closure() -> $R`; the user captures
inputs in the lambda at the call site (the `go func(){…}()` idiom). **Self-contained in sched.sx**
(NOT io.sx): io.sx importing sched.sx duplicates the `_fib_tramp` global asm when a program also
imports sched.sx directly (global asm emits per import-path) — so the Io-protocol
`spawn_raw`/`suspend_raw`/`ready` hooks stay reserved for the future M:N model; M:1 uses
`go`/`wait` directly. Heap `*Task` (must outlive `go`'s frame; leak documented). `TaskErr` is
LOCAL (the `!` failable detection doesn't see through io.sx's `IoErr` re-export alias).
- **Two compiler blockers hit + FIXED (user-authorized in-session):**
- **issue 0156 Part 1** — a single-type generic `$R` (parsed as `comptime_pack_ref`) used as a
type-arg (`Box($R)`, `size_of(Box($R))`) inside a pack-fn body hit a missing arm in
`resolveTypeWithBindings` → `.unresolved` → LLVM panic. Fix: mirror `resolveTypeArg`'s
`comptime_pack_ref` arm (look up `type_bindings`, else a loud diagnostic). Regression
`examples/generics/0216-generics-typearg-in-pack-fn-body.sx`. (Part 2 — deferred `..` spread
crashes — reframed OPEN/non-blocking, `issues/0156`.)
- **issue 0157** — a user generic `ufcs` method whose name collides with a stdlib re-export
(`cancel` on `*Task` vs io.sx's `cancel` on `*Future`) resolved via last-wins `fn_ast_map` with
NO receiver filtering → wrong overload → `$R` unbound → LLVM panic. Fix
(`src/ir/lower/call.zig` `selectUfcsGenericByReceiver`): every generic-ufcs dispatch enumerates
ALL module authors (`module_decls`), keeps receiver-binding ones, picks the most
receiver-SPECIFIC (concrete > bare `$T`), dedups re-exports, and flags a genuine 2-specific tie
as a deterministic "ambiguous — qualify" diagnostic (never a silent order-dependent pick).
Regression `examples/generics/0217-generics-ufcs-method-name-collides-stdlib.sx`.
- **Adversarial review (worker) of the 0157 fix + Task layer.** Caught the determinism CRITICAL
(fixed: always-run selection + specificity + ambiguity), `wait`-outside-a-fiber null-deref (fixed:
loud guard in `suspend_self`/`yield_now`), and cancel-doesn't-skip-work (fixed: worker skips
`work()` if already canceled). Lost-wakeup / cancel-after-complete / reap traced safe. Also
simplified `1812` (`**Fiber` shared handle → a `Sh.parked` field; output identical).
- Suite GREEN 751/0 (749 + 1813 + 0217). Next: **B1.4b** (deterministic-sim `Io`).
### Earlier — B1.5a — the M:1 cooperative fiber scheduler CORE — landed + adversarially reviewed
The hand-bootstrapped ping-pong (1807-1810) is now a reusable scheduler API in pure sx:
`library/modules/std/sched.sx` — a generic `Fiber` (`body: Closure() -> void`) + `Scheduler`
with `init`/`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over the proven `swap_context` on
guarded `mmap` stacks. The ONE generic dispatch (`fib_dispatch`, reached from the `_fib_tramp`
trampoline) runs ANY stored closure body on a fresh stack — replacing the fixed `bl _fib_body`.
Reaping `munmap`s the stack + frees the heap `Fiber` on completion; an intrusive FIFO gives
round-robin order.
- **Foundational design de-risked by probe before building:** a fiber can store + call a
`Closure() -> void` on its fresh stack via the generic dispatch; outputs flow OUT through
pointers captured in the closure (capture-by-value does NOT write back — pushed onto the user).
- **Hit + FIXED a blocker compiler bug — issue 0154** (user-authorized in-session fix). `null` /
`---` assigned to a struct field picked up a leaked enclosing `target_type` (the function's
RETURN type, set for the whole body at decl.zig:2691) and built a WHOLE-STRUCT-typed null →
an oversized `zeroinitializer` store through the field's GEP that overran the field's slot and
clobbered the saved x29/x30, so the fn `ret`'d to 0x0. This was EXACTLY the `Scheduler.init()`
by-value-return shape (`sched_ctx: [13]u64` before `current: *Fiber`). Fix: added
`.null_literal, .undef_literal` to the `needs_target` switch in `lowerAssignment`
(`src/ir/lower/stmt.zig`) so the field's type is used. Repro → regression test
`examples/types/0193-types-sret-array-before-pointer.sx`; `issues/0154-*.md` RESOLVED.
- **Adversarial review (worker): asm/bootstrap/lifetime SOUND** (the headline closure-env-lifetime
fear was disproven — envs are heap-promoted, survive the spawn frame). Found **1 CRITICAL** +
robustness gaps, ALL hardened: (CRITICAL) `wake` re-enqueued an already-queued fiber →
FIFO corruption/segfault → now GUARDED on `.suspended` (spurious/double/stale wake = safe
no-op); orphan-suspend leak/deadlock → `n_suspended` accounting + a loud `run()`-drain
diagnostic+abort; `mmap` `MAP_FAILED` (=-1, not null) / `mprotect` / Fiber-OOM → loud bails
(per §8.1.1 the guard is mandatory); the per-fiber closure-env leak (sx exposes no env-free) →
documented as a KNOWN LIMITATION (bounded by spawn count; invisible under the default GPA).
- **Locked two `18xx` examples** (aarch64-macos `.build`-pinned, ir-only on a mismatch):
`1811-concurrency-fiber-scheduler.sx` (3 fibers round-robin via `yield_now` → ordering contract
`sequence: 0 1 2 0 1 2 0 1 2`, all `.done`) + `1812-concurrency-fiber-suspend-wake.sx` (park via
`suspend_self`, resumed by another fiber's `wake`, + the spurious-wake no-op — the CRITICAL-fix
regression → `log: 10 20 21 11` / `suspended-left: 0`).
- **Filed issue 0155 (NON-blocking, NOT fixed)** — found incidentally in the review: indexing a
scalar pointer (`pc[0]`, `pc: *i64`) panics codegen (`.unresolved` reaching LLVM emission). The
scheduler uses array-field indexing + `.*`, never this, so it's filed for its own session.
- Suite GREEN **748/0** (746 base + 1811 + 1812 + 0193 regression). Next: **B1.4a** (FiberIo —
wire `Io.spawn_raw`/`suspend_raw`/`ready` onto the scheduler so `async`/`await` truly suspend).
### Earlier — B1.3b-1 — the x86_64 / Win64 `swap_context` sibling — VALIDATED on real hardware
The
context switch is now proven on a SECOND architecture + ABI. A Win64 `swap_context` saves the
COMPLETE Win64 callee-saved set — 8 GP (rbx, rbp, rdi, rsi, r12-r15) + rsp **and xmm6-xmm15**
(10 XMM, 128-bit via `movups` — Win64 has callee-saved XMM, unlike SysV/aarch64) — plus a Win64
`scribble_verify` (32-byte shadow + 16-align at each `call`, COFF symbols, rsp-carried return
addr). Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only here): the 2-fiber mutual scribble printed **`0 0 P`** when built `--target
x86_64-windows-gnu --self-contained` and **run on a Windows 7 x64 VM (UTM)** — every GP + XMM
callee-saved survived. **Adversarially reviewed before the VM run** (worker emitted the real `.s`
and verified every `call` alignment, the 264-byte frame offsets, the rsp/return-addr round-trip,
swap ordering, and COFF naming against the Win64 ABI — no critical/minor bugs). The build→VM→run
loop was set up this session (cross-build needs `--self-contained`; output via the Win32
`WriteFile` boundary, the 1660 pattern). Suite green. Note: this is the GOOD-swap-only mutual
scribble (self-validating by construction; the in-process negative control was dropped to avoid an
sx fn-ptr-convention rabbit hole — the detection of this exact logic was negative-controlled on
aarch64 in 1808). The SysV/Linux x86_64 sibling (different reg set: no callee-saved XMM, args
rdi/rsi) remains for a Linux x86_64 host.
### Earlier — B1.3b-2 — mmap guard-page stacks (commit `dd532ab`)
Fiber stacks are `mmap`'d with a `PROT_NONE` GUARD PAGE at the low end (§8.1.1: a
fixed stack without a guard silently corrupts neighbors on overflow). `mmap` the `[guard |
usable]` region, `mprotect` the low 16KB page `PROT_NONE`; SP descends into the guard and faults
loudly at the boundary instead of corrupting a neighbor. Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
- **Guard FIRING validated** (manually, not corpus-pinned — a deliberate overflow crash is
host-fragile): a fiber recursing past its 128KB stack faults with `Bus error` at the guard page
(`region+GUARD`); the sx crash handler turns it into exit 134. Documented in the example header.
- **x86_64 sibling:** was deferred here (couldn't run x86_64 on this arm64 host), then DONE as
Win64 once a Windows 7 x64 VM became available — see B1.3b-1 above (`examples/1810`, `0 0 P`).
### Earlier — B1.3a-2 — the context-switch STRESS GATE (design §10.7) — DONE + adversarially reviewed
The explicit every-callee-saved-register scribble that B1.3a-1 owed. `swap_context` now saves the
COMPLETE AAPCS64 callee-saved set — integer x19-x28 + fp/lr + sp AND FP **d8-d15** (per §6.1.2
only the low 64 bits of v8-v15 are callee-saved, so `d8-d15` is exactly sufficient; x18 is Apple's
reserved platform reg, untouched). A naked `scribble_verify(self_ctx, peer, base)` loads a unique
sentinel into all 18 callee-saved regs, yields, and on resume counts the ones that didn't survive
(honoring its own caller ABI via a 176-byte frame that saves+restores the caller's callee-saved;
base reloaded from the frame post-swap; the original lr round-trips through the swap). The gate is
a **2-fiber MUTUAL scribble** (A and B scribble DISTINCT sentinels into the same physical regs, so
each survives only if `swap_context` saved+restored it — a lone fiber yielding to an idle peer
would NOT exercise preservation). Locked by `examples/1808-concurrency-fiber-switch-stress.sx`
(aarch64-pinned): `A mismatches: 0` / `B mismatches: 0`.
- **Validity proven by NEGATIVE controls:** dropping the d8-d15 save/restore → 8/8 mismatches
(exactly the FP regs); dropping x27/x28 → 2/2. The gate genuinely catches a broken switch.
- **Adversarial review (worker, per the plan): no CRITICAL bugs.** Verified the callee-saved set
is complete + correct, all frame offsets/16-alignment, the lr/sp dance, and swap read-ordering
against AAPCS64. Applied its one recommendation: `boot` now zeroes the FP ctx slots [13..20] so a
first switch-to loads 0 (not garbage) into d8-d15. Residual gaps it flagged (all spec-correct
for a call-boundary swap, documented in the example header): NZCV/FPSR not swapped; **FPCR**
(rounding mode — thread-global, bleeds across fibers if changed) and **TPIDR_EL0/TLS** (errno,
allocator thread-caches — shared by same-thread fibers) not swapped; fp=0 bootstrap blocks
unwind/signal walking past a fiber trampoline. These bite at the N×M:1 / signals stages, not the
single-thread switch.
- Suite green **734/0**, master clean. WIP probes: `.sx-tmp/scribble2.sx` (+ `_broken`/`_gp`).
### Earlier — B1.3a-1 — the foundational stackful context switch (commit `b234b7d`)
Pure sx over `abi(.naked)`: naked `swap_context` (GP-only 13-slot save) + by-hand fiber bootstrap
(SP = `alloc_bytes` stack top, LR = global-asm trampoline, x19 = `*Fiber`). Locked by
`examples/1807-concurrency-fiber-context-switch.sx`: 2-fiber ping-pong (`rounds: 6` / `canary
fails: 0`) + 64-frame deep recursion (`frames verified: 64` / `depth fails: 0`). Indirect
register/stack survival; 1808 supersedes its switch with the complete GP+FP save area + the
explicit gate.
### Earlier — B1.2 COMPLETE — the async surface works end-to-end
All three surface blockers (0151, 0152, 0153) FIXED + committed; async examples landed + green.
- **0151 fixed** (`362674f`): generic `$T` infers through generic-struct / pointer / UFCS-pack
params. Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` load/store byte-promoted to `i8` in the codegen
emitters. Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` now pins return-type resolution to the
fn's DEFINING module (mirroring `monomorphizeFunction`), so a re-exported value-failable's
`!E` resolves to the real `.error_set` TypeId — the failable channel survives the re-export
alias. Regression `1058-errors-reexport-value-failable-channel.sx`.
- **Async examples landed:** `examples/1805-concurrency-io-blocking-async.sx`
(`context.io.async((a,b)->i64 => a+b, 40, 2).await() or {…}` → `sum: 42` / `double: 42` /
`clock ok`) + `examples/1806-concurrency-io-cancel.sx` (`f.cancel()` → `await` raises
`.Canceled` → `or` default; `ok: 7` / `canceled: -99`). Both green, snapshots captured.
### Earlier — the three B1.2 surface fixes (committed)
Generic `$T` inference, `Atomic(bool)` byte-promotion, and re-export failable-channel pin —
details below.
- **0151 fix (committed):** four gaps closed on the inference + UFCS-dispatch path —
(1) `extractTypeParam`/`matchTypeParam(Static)` got a `parameterized_type_expr` arm
(recover the arg instance's recorded per-param bindings via `struct_instance_bindings` +
the template's ordered `type_params`, recurse positionally; this also fixes `*Box($T)` —
it recurses into its `Box($T)` pointee); (2) the `pointer_type_expr` arm now falls through
to match the pointee against a non-pointer arg (auto-address-of: a `*Box($T)` param accepts
a by-value `Box($T)`, e.g. a UFCS receiver `b.m()`); (3) `ExprTyper.inferType` got a
`.lambda` arm building the closure type from the lambda's annotations (the UFCS binder types
args from the raw AST before they're lowered, so it can now bind `Closure(..) -> $R` from
the worker's declared return type); (4) a pack UFCS target routes through the SAME
`lowerPackFnCall` the direct call uses, with the receiver spliced in as `args[0]`.
- Regression tests: `examples/0214-generics-ufcs-closure-return-pack.sx` (direct + UFCS
closure-return pack) + `examples/0215-generics-infer-through-pointer.sx` (by-value /
pointer / multi-param / nested / UFCS-auto-ref struct-head inference). Issue 0151 marked
RESOLVED; repro moved into the suite.
### Earlier — B1.2 (Io capability) — LANDED + adversarially reviewed
Commits `a1b14f0` (lock) + `45d869d` (Io capability) + `3eeb965` (issue 0151 lock).
- **LANDED + review-confirmed correct** (commit 45d869d): `Io :: protocol #inline`
(spawn_raw/suspend_raw/ready/poll/now_ms/arm_timer) + `io` field on `Context`
(`{allocator; data; io}`, io LAST); BOTH `__sx_default_context` materializers
(protocol.zig + comptime_vm.zig) build an identical CBlockingIo→Io vtable (review verified
byte-for-byte agreement; `context.io.now_ms()` dispatches at runtime AND comptime); the
`push Context.{…}` omitted-field-**inherits-ambient** fix (review: correct, right fix, no
bad blast radius); `library/modules/std/io.sx` (`Future($R)`, `CBlockingIo`,
`async`/`await`/`cancel`); the `!`-protocol-impl-lint suppression; 37 `.ir` regens
(review: pure layout/type-table, no error text, zero .exit/.stdout/.stderr change).
- **BLOCKED — async surface non-functional:** `await`/`cancel` take `*Future($R)` and are
**uncallable in EVERY form** (not just UFCS) — sx can't infer a generic `$T` from a
pointer-wrapped arg (`*Future($R)`). `async(...)` (create) works via explicit call and
produces a correct `.ready` Future, but you can't `await` it. Root bug = **issue 0151
(WIDENED)**: infer `$T` from `*T`-wrapped params + closure-return-via-pack + UFCS dispatch.
Minimal repro: `unbox :: (b: *Box($T)) -> $T` fails to infer `T`.
- **No async example in the corpus** (1805 was removed because it needs the blocked surface)
→ the green suite does NOT cover async. Restore `1805` (async/await) + add `1806` (cancel)
once 0151 is fixed.
### Earlier — B1.1 (per-fiber `context` root) — DONE. Zero compiler change (confirmed by probe).
The fiber-spawn context convention works end-to-end with ordinary language features:
- `snap := context` captures the spawner's `Context` as a value;
- the snapshot is stored in a struct (the stand-in `Fiber`);
- a trampoline running under a *different* ambient context installs the fiber's stored root
with `push f.root { … }`, and the body reads the snapshot — not the trampoline's ambient
context — because `context` is an implicit slot-0 `*Context` param (call-carried, rides the
callee's own stack) and `push` allocates on the caller frame (no global, no TLS).
- Locked by `examples/1804-concurrency-context-snapshot.sx`: prints `fiber root: 42` (the
installed snapshot wins over ambient 99) + `ambient after: 99` (the `push` scope restores
the ambient context on exit). No fiber runtime yet (that's B1.3) — this proves the plumbing
it will build on. No `.build` pin (pure sx, host-independent).
- **Probe result:** the design doc's "lower as swappable indirection, never raw TLS" guarded
a non-problem — context was already param-carried, never TLS. No path re-reads
`__sx_default_context` mid-stack, so there is **no compiler obligation** here.
- `zig build && zig build test` green: **726 ran, 0 failed**.
### Earlier — B1.0 (`abi(.naked)` codegen) — complete
Replaced the emit bail with real LLVM `naked` emission:
- `emit_llvm` declaration pass: for `func.is_naked`, add the LLVM `naked` + `noinline` +
`nounwind` attributes and **skip** the `frame-pointer=all` attribute (incompatible with a
frameless function). Pass 2 now emits the `.naked` body normally — `naked` makes the
backend emit it verbatim (the inline asm + its own `ret`) with no prologue/epilogue.
- IR shape (verified): `; Function Attrs: naked noinline nounwind` / `define internal i64
@answer() #0 { entry: call void asm sideeffect "…ret…", ""() unreachable }` /
`attributes #0 = { naked noinline nounwind }`. The caller invokes it as an ordinary
`() -> i64` call (`.naked` is `call_conv == .default`).
- `examples/1800-concurrency-naked-asm.sx` — now GREEN, aarch64-pinned (`.build {"target":
"macos"}`): runs end-to-end → **exit 42** on this host, ir-only on a mismatch; `.ir`
snapshot captured.
- `examples/1801-concurrency-naked-generic.sx` (renamed from `-bail`) — the generic `.naked`
now emits a correct naked `answer__i64` (exit 42), proving generic.zig produces a naked
body, not a framed one. aarch64-pinned.
- `examples/1802-concurrency-naked-asm-x86.sx` — x86_64 cross sibling (`.build {"target":
"x86_64-linux"}`, ir-only here): `.ir` locks `naked` + `movl $42, %eax` / `ret`.
- Unit test `emit: abi(.naked) function gets the naked attribute (no frame-pointer)` in
`emit_llvm.test.zig` (asserts `naked` present, `frame-pointer` absent).
- **B1.0c (review-hardening):** a param-bearing `.naked` fn emitted invalid LLVM (loud
verifier error "cannot use argument of naked function") because the param-alloca loop
wasn't gated. Fixed forward (this *enables* the B1.3 context-switch use case rather than
rejecting it): gated the param-alloca loop on `fd.abi != .naked` in decl.zig (both paths) +
generic.zig; a naked fn's args stay in registers (read by asm), declared-but-unused in
LLVM. Locked by `examples/1803-concurrency-naked-asm-param.sx` (`add(a,b)` → x0+x1 → 42).
- `zig build && zig build test` green: **725 ran, 0 failed** + unit tests.
### Earlier — B1.0a (lock + review hardening)
Plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites + generic.zig +
pack.zig); `funcWantsImplicitCtx` skips `.naked` (no synthetic ctx, like `.c`); all
body-lowering paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx
return); `emit_llvm` Pass 2 bailed loudly (since flipped to real emission). Adversarial
review caught the generic/pack `is_naked` gap (a generic `.naked` silently shipped a framed
body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positive
(unparseable — `isLambda` breaks on the `abi` keyword).
## Current state
**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime:
the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task
async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b:
`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`,
`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded
`mmap` stacks (B1.0B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw
suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). A terminal
`deinit` (B1 follow-up) closes the previously-documented leaks: heap `Task`s (tracked via `task_allocs`),
the `timers`/`io_waiters`/`task_allocs` List backings, and the kqueue fd; the per-`spawn`/`go` closure
env remains unfreeable (language limitation). Locked by `18xx` 18001820 (naked-asm, context-snapshot,
blocking async, the switch + §10.7 stress gate + guarded stacks + Win64 sibling, scheduler round-robin,
suspend/wake, async go/wait/cancel, sim-timer ordering, timer early-wake eviction, kqueue pipe I/O, the
**1817 end-to-end capstone**, sleep-negative/double-wait guards, and **1820 scheduler-deinit**). Suite
GREEN **817/0**, committed. **B1.6: now also runs on aarch64-linux** (epoll fd-backend + comptime-branched
`MAP_AP` + naked-fn trampoline) — validated byte-identical to macOS in an Apple `container`.
Future work (none blocking B1): routing the suspending async through
the erased `context.io` (forces sched.sx into every std consumer — deferred to the M:N model, where
the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/
`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task /
closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream
B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started.
### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists
`library/modules/std/sched.sx` drives N fibers
(generic `Closure() -> void` bodies) cooperatively over the proven `swap_context`, on guarded
`mmap` stacks: `spawn` / `yield_now` (round-robin) / `suspend_self` + `wake` (off-queue park/resume)
/ `run` (drives to drain, reaps on `.done`). Adversarially reviewed + hardened (wake guarded, loud
mmap/mprotect/OOM/deadlock bails, env-leak documented). Locked by `1811` (round-robin ordering
contract) + `1812` (suspend/wake park-resume + spurious-wake guard). Suite GREEN **748/0**.
The remaining B1.4 work wires this scheduler under the `Io` capability: **B1.4a (FiberIo)** makes
`context.io` route `spawn_raw`/`suspend_raw`/`ready` onto the `Scheduler` so `async`/`await` truly
SUSPEND (today's `CBlockingIo` runs the worker to completion inline); **B1.4b** the deterministic-sim
`Io` (virtual clock + timer queue, calibrated against blocking — the KEYSTONE test harness);
**B1.4c** the event-loop `Io` (kqueue/epoll). Then **B1.5** is the end-to-end M:1 validation under
the deterministic `Io`.
### Earlier — B1.2 COMPLETE
The full async surface (Io capability on Context + `async`/`await`/`cancel` +
blocking `CBlockingIo`) works end-to-end. Master GREEN (732/0), installed `sx` clean. All four
B1.2 surface bugs resolved or deferred:
- **0151 fixed** (`362674f`): generic `$T` through generic-struct / pointer / UFCS-pack params.
Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` byte-promoted to `i8` in the load/store emitters.
Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` pins return-type resolution to the fn's
defining module, so a re-exported value-failable keeps its `!` channel. Regression `1058`.
- Issue **0150** (`void` struct field → SIGTRAP) DEFERRED — only `Future(void)` / `timeout`,
which are B1.4.
The async examples are landed + green: `1805` (`async`/`await` + `now_ms` → `sum: 42` /
`double: 42` / `clock ok`) + `1806` (`cancel` → `await` raises `.Canceled` → `or` default).
The `18xx` concurrency category now covers naked-asm (1800-1803), context-snapshot (1804), and
the async surface (1805-1806).
### B1.2 Io capability — what is LANDED + verified (commit 45d869d)
- `Io :: protocol #inline { spawn_raw; suspend_raw -> !; ready; poll; now_ms; arm_timer; }`
in `core.sx` next to `Allocator`, with `SpawnOpts{ pin: PinTarget }` + `ParkToken{ handle }`.
Six methods, each justified by a downstream consumer (B1.3-B1.5).
- `Context :: struct { allocator; data; io: Io; }` — `io` appended LAST so `allocator` stays
index 0 (the `call.zig:1229` hardcode) and `data` keeps index 1 (minimal VM-fallback churn).
- Both `__sx_default_context` materializers updated in lockstep + verified: `protocol.zig`
`emitDefaultContextGlobal` (extended `ctx_fields` 2→3, built the `CBlockingIo→Io` inline
7-word vtable `{null-ctx, fn0..fn5}` via `getOrCreateThunks("Io","CBlockingIo")`) and
`comptime_vm.zig` `materializeDefaultContext` fallback (wrote the 6 thunk func-refs at
`io_base = addr + 4*ps`, offset `+ (i+1)*ps`). The global path auto-followed the 3-field
Context type. **`context.io.now_ms()` printed `clock ok` live — the capability threads + the
vtable dispatches correctly.**
- Stateless `CBlockingIo :: struct {}` + `impl Io for CBlockingIo` (mirror of `CAllocator`):
blocking semantics — `spawn_raw`/`ready`/`poll`/`arm_timer` no-op/0, `now_ms` → `time.mono_ms()`.
- **push-inherit-omitted fix** (`stmt.zig` `lowerPush`): a `push Context.{...}` now SEEDS the
new slot from the ambient context (load+store), then overwrites ONLY the literal's named
fields — so omitted fields (now incl. `io`) are INHERITED, never zero-inited to a null
vtable. Eliminates the omitted-field footgun globally (zero per-site churn across the 17
partial-literal sites). This is the correct capability-bag semantics; it compiled clean.
- **`!`-protocol-method warning fix** (`error_analysis.zig` + a new `Lowering.impl_method_names`
set populated in `protocols.zig` `registerImplBlock`): a protocol impl method may be declared
`!` by contract (e.g. `Io.suspend_raw`) yet never raise; the "declared `!` but never errors —
drop the `!`" hint is a false positive for impl methods, now suppressed for them.
Status of the blockers that originally stopped B1.2:
- **issue 0151 — FIXED this session** (generic `$T` through generic-struct / pointer /
UFCS-pack params). `async`/`await`/`cancel` are callable. See "Last completed step".
- **issue 0152 — NEW, the current blocker** (`Atomic(bool)` → sub-byte i1 atomic; LLVM reject).
Blocks the async examples via `Future.canceled: Atomic(bool)`. Filed; codegen-level fix.
- **issue 0150** — `void` struct field SIGTRAP; only `Future(void)`/`timeout` (B1.4). DEFERRED.
Per the IMPASSABLE STOP rule: 0151 fix shipped (suite green 728/0), 0152 filed, STOPPED.
Resume B1.2's async examples once 0152 lands.
### Earlier — B1.0 + B1.1 complete
Stream A (atomics) is feature-complete (✅). Stream B1: **B1.0 + B1.1 complete.** The two
compiler-floor preconditions for the fiber runtime are in place: (1) `abi(.naked)` emits a
real LLVM `naked` function end-to-end (decl, generic, pack paths) — the context-switch
substrate; (2) per-fiber `context` root needs **no compiler change** — the spawn convention
(snapshot `context`, store, `push` it from the trampoline) is pure library sx. No
fibers/Io/scheduler code yet. Grounded floor facts:
- `context` is an implicit slot-0 `*Context` param + `push Context` is a stack `alloca` ⇒
**fiber-local for free** (confirmed by the B1.1 probe — never TLS, never re-read from the
`__sx_default_context` global mid-stack). A spawn passes the snapshot as the fiber-entry
fn's slot-0 ctx via `push f.root { entry(args) }`. Locked by `1804-...-context-snapshot`.
- Inline asm works end-to-end (lower→emit→JIT, aarch64 + x86_64) — the `.naked` body reuses it.
- **`.naked` with PARAMS works** (B1.0c, the B1.3 substrate): the param-alloca loop is gated
on `fd.abi != .naked` in decl.zig (both paths) + generic.zig — a naked fn's args stay in
ABI registers (read by the asm body), declared-but-unused in LLVM (verifier-legal).
Example `1803-concurrency-naked-asm-param.sx` (`add(a,b)` reads x0/x1). **Unsupported (loud,
not silent):** a `.naked` *variadic-pack* fn (pack.zig's param loop is intertwined with
comptime-param/`#insert` handling, and a naked fn can't read a runtime-sized pack from
registers anyway) → loud LLVM-verifier error for that nonsensical construct. Acceptable
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
## Next step
**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature-
complete and committed (18001820 green, 759/0), now WITH a `Scheduler.deinit` closing the bounded
leaks. Pick up **Stream B2** (channels / structured cancel / async stdlib) as a fresh carve
(PLAN-CHANNELS.md), OR one of the remaining non-blocking follow-ups: the linux `epoll` twin of
`block_on_fd`, `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the
erased `context.io` for the M:N model. (`Scheduler.deinit` — DONE, see Last completed step.) None of
these block B1. The closure-env leak survives `deinit` (no language affordance to free a closure env);
revisit if/when sx grows closure-env ownership.
**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS
kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via
`std/net/epoll` and the OS-neutral facade is `std.event` — keep the two as separate run modes inside
`run`, branching on the platform, exactly as the timer-vs-fd modes are kept separate now. Documented
non-unification: virtual-time timers and real kqueue timeouts are NOT merged — `run` fires a pending
timer before ever blocking on kqueue (a program uses `sleep` OR fds); a true "fd-or-real-timeout" wants
a kqueue `EVFILT_TIMER`, future work.
> **▶ LINUX EPOLL — in progress (2026-06-26), via `std.event.Loop` (the OS-neutral facade).**
> Chosen over the sched.sx `block_on_fd` twin because the facade is the named home for epoll, is pure
> sx + libc (zero compiler change), is consumed by http.sx, and has a runnable darwin sibling. Landed:
> (A) **`library/modules/std/net/epoll.sx`** — raw bindings, the linux twin of `std/net/kqueue.sx`.
> `epoll_event` is modelled as an **arch-branched struct** (`{events, data_lo, data_hi}` u32 fields →
> 12 B x86_64 packed / 16 B aarch64), so layout is byte-exact with NO packed attribute, NO unaligned
> access, NO scalar-pointer indexing (issue 0155) — the struct-per-arch approach the user flagged as
> better than raw byte poking. Self-contained (libc only — NO build.sx import; the top-level `inline if
> ARCH` resolves via the compiler's flatten pre-pass, keeping the IR small). Locked by
> `examples/event/1633-event-epoll-bindings-linux.sx` (ir-only x86_64-linux, durable 244-line .ir;
> aarch64 16 B layout also probe-verified). (B) **`std.event.Loop` branched on `inline if OS`** into two
> top-level OS-selected structs (sx has no conditional struct fields): the kqueue Loop unchanged
> (darwin, runs — 1632 green), a new epoll Loop (linux) with the per-fd registration table (combined
> EPOLLIN/OUT mask via ADD/MOD/DEL), eventfd wake channel, and EPOLLRDHUP→eof. **RUNTIME-VALIDATED on
> real Linux:** a static `aarch64-linux` build of the 1632-equivalent Loop test (+ the eventfd wake path)
> ran **6/6 green inside an Apple `container` Linux VM** (kernel 6.18 aarch64) — add_read, idle-timeout,
> readable+fd+udata, the MOD-mask add_write path, the eventfd wake channel, and EPOLLRDHUP/HUP eof all
> behave identically to kqueue (lone documented difference: `nbytes` is 0 on epoll). Also lowers clean for
> both linux arches; the ABI is corpus-locked by 1633. NOT corpus-snapshotted (the corpus runner is
> host-based, not container-aware; a Loop example drags the std barrel → ~18k-line brittle IR).
> **The epoll deliverable is COMPLETE.** Re-validation recipe in the event.sx VALIDATION note. Optional
> follow-on: route sched.sx `block_on_fd` through `std.event` (still needs the linux sched.sx port — mmap
> consts, tramp symbol, errno, x86_64 SysV switch).
> **✅ issue 0192 FIXED (2026-06-26) — epoll work UNBLOCKED.** A qualified-import-member const
> (`m.EV_SIZE`) now folds as a compile-time constant in every position the bare/flat form does
> (array dim, arithmetic, Vector lane, generic value-param, inline-for) — so the clean
> `[MAXEV * ep.EV_SIZE]u8` event buffer the bindings want will work. Fix: a `lookupQualifiedConst`
> ctx hook resolving the namespace alias → target module's per-source const, wired into the int/float
> const folders (`src/ir/program_index.zig` + `src/ir/lower/comptime.zig`). Regression:
> `examples/modules/0842-modules-qualified-import-const-comptime.sx`. The hint stands for the rebuild:
> **a struct-per-arch `EpollEvent` (arch-branched u32 fields, 12 B x86_64 / 16 B aarch64) beat raw
> byte access** — idiomatic field reads, no issue-0155 scalar-pointer indexing, no unaligned u64.
> Resume: rebuild `std/net/epoll.sx`, branch `std.event.Loop` on `inline if OS`, lock with a darwin run
> + ir-only linux example.
> **⛔ (HISTORICAL) BLOCKED on issue 0192 (filed 2026-06-26).** Started the epoll work: chose the `std.event.Loop`
> backend (pure sx + libc externs, zero compiler change — per "do this in sx as much as possible") as
> the first deliverable, since event.sx already names epoll as its linux backend and it's runnable
> (darwin via kqueue) + ir-only-verifiable (linux). De-risked four landmines by probe — arch-dependent
> layout const via module-scope `inline if ARCH` (folds + validates in linux IR), slice-based byte access
> (sidesteps issue 0155), no unaligned u64 (store the 32-bit fd in epoll `data`), and comptime-dead linux
> externs don't break the darwin corpus (just an unreferenced `declare`). Then hit a compiler bug while
> sizing the event buffer: a **qualified-import-member const is not a compile-time constant** —
> `[m.CAP]u8` / `A :: m.CAP` fail (a *flat*-imported const works). Root cause located:
> `evalConstIntExpr` (`src/ir/program_index.zig:325`) has no namespace-member-const arm. Per the STOP
> rule the half-built `std/net/epoll.sx` (which used a struct-based layout to route around the bug) was
> **removed**, not landed — the unblock session rebuilds it cleanly with the fix in hand. Repro +
> investigation prompt: `issues/0192-qualified-import-const-not-comptime.{md,sx}`.
Design note carried forward: an event-loop `Io` needs a current-`Scheduler` handle. `sched.*` methods
thread it via `self`/the `Task`; if B1.4c wants the capability-threaded `context.io` form it'll need
an ambient current-scheduler accessor in sched.sx (still deferred — the `sched.*`-method form
suffices). The `Io` protocol's `poll`/`arm_timer` map onto this when/if that wiring is built.
**Side thread (optional, low priority): the SysV/Linux x86_64 sibling.** A THIRD switch variant
for `x86_64-linux`: SysV callee-saved = rbx, rbp, r12-r15 + rsp (6 GP + sp; **no** callee-saved
XMM, unlike Win64) — a 7-slot ctx, args rdi/rsi/rdx, the rsp-carried return addr. Needs a Linux
x86_64 host (or a working cross-run) to RUN + the mutual-scribble gate. Not blocking — the switch
is already validated on two arch/ABI pairs.
**Deferred (do NOT block on these):** issue **0150** (`void` struct field SIGTRAP) — only
`Future(void)`/`timeout` (B1.4). The **`::` callable-parameter feature** (named-fn async workers
`async(read_a, conn)`) — WIP at `.sx-tmp/wip-callable-params/patch.diff` (parser done, inference
incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
`Context` layout settled: `{ allocator; data; io; }` (allocator index 0 fixed by
`call.zig:1229`, io last). Io protocol + materializers + push-inherit are LANDED + reviewed.
## Known issues / capability gaps
- **issue 0157 (OPEN, BLOCKING B1.4a)** — a user-defined generic ufcs method whose NAME collides
with a stdlib re-export (`cancel`, re-exported by `std.sx` from `io.sx` as `ufcs (f: *Future($R))`),
called via UFCS on a different generic struct (`*Task($R)`), leaves `$R` unresolved → `.unresolved`
reaches LLVM emission → panic (`src/backend/llvm/types.zig:196`). Renaming → works; the non-UFCS
call form already diagnoses `cannot infer generic type parameter 'R'`, so the UFCS path skips that
diagnostic. Surfaced by `cancel :: ufcs (t: *Task($R))` in `std/sched.sx`. Minimal repro (no
fibers/closures): `issues/0157-ufcs-generic-method-name-collides-stdlib-unresolved.{md,sx}`.
- **✅ issue 0154 — FIXED** (`null`/`---` to a struct field over-stored a whole-struct null when
the function's return type leaked as `target_type`, corrupting the frame → `ret` to 0x0;
surfaced building `Scheduler.init()`'s by-value return). Fix: `.null_literal`/`.undef_literal`
added to `needs_target` in `lowerAssignment` (`src/ir/lower/stmt.zig`). Regression:
`examples/types/0193`.
- **issue 0155 (OPEN, NON-blocking)** — indexing a scalar pointer (`pc[0]`, `pc: *i64`) panics
codegen (`.unresolved` reaching LLVM emission, `src/backend/llvm/types.zig:196`). Found in the
B1.5a review; the scheduler doesn't use it (array-field index + `.*` only). Filed for its own
session: `issues/0155-scalar-pointer-index-llvm-panic.{md,sx}`.
- **✅ issue 0158 — FIXED** — a plain `union` struct-literal (`b : Overlay = .{ f = 3.14 }`) fell
through the generic struct-literal path (`getStructFields` empty for a union → malformed
`structInit`, overlapping zero-fill clobbered the member → silent `0.0`). Fix: `lowerStructLiteral`
detects a plain-union target → new `lowerUnionLiteral` (`src/ir/lower/stmt.zig`) writes each named
member into a union-sized slot via the assignment-path lvalue resolver, then loads it back.
Single-arm only (one direct member, or same-arm promoted members); overlapping/different-arm/
positional literals are diagnosed. specs.md updated. Regressions: `examples/types/0194` +
`examples/diagnostics/1191`.
- **✅ issue 0157 — FIXED** (B1.4a) — a user generic `ufcs` method whose name collides with a
stdlib re-export resolved via last-wins `fn_ast_map` with no receiver filtering → wrong overload →
`$R` unbound → LLVM panic. Fix: `selectUfcsGenericByReceiver` (`src/ir/lower/call.zig`) — most
receiver-specific binding author across ALL module authors, deterministic, ambiguity-diagnosing.
Regression: `examples/generics/0217`.
- **✅ issue 0156 Part 1 — FIXED** (B1.4a) — single-type generic `$R` as a type-arg in a pack-fn
body (`Box($R)`/`size_of(Box($R))`) → `.unresolved` → panic. Fix: `comptime_pack_ref` arm in
`resolveTypeWithBindings`. Regression: `examples/generics/0216`.
- **Part 2 (OPEN, NON-blocking)** — a deferred `..` spread (a comptime pack captured into a
closure, or a tuple `..t` spread) crashes instead of working/diagnosing. The fiber async layer
avoids it by design (nullary thunks), so it's filed for its own session: `issues/0156`.
- **Heap leaks in the fiber runtime (documented limitations, NOT bugs):** `spawn`'s closure env +
`go`'s heap `Task` are never freed (sx exposes no closure-env free; Task ownership is deferred).
Bounded by spawn/go count, invisible under the default GPA. Revisit for a long-running
arena-backed scheduler.
- **✅ issue 0153 — FIXED** (re-exported generic value-failable `($R, !E)` kept its `!` channel:
`inferGenericReturnType` now pins return-type resolution to the fn's defining module).
Regression: `examples/1058`. Was the LAST B1.2 surface blocker.
- **✅ issue 0152 — FIXED** (`Atomic(bool)` sub-byte i1 atomic → byte-promoted to i8 in the
load/store emitters). Regression: `examples/1705`. Unblocked `Future.canceled`.
- **✅ issue 0151 — FIXED** (generic `$T` through generic-struct / pointer / UFCS-pack params).
Regression: `examples/0214` + `0215`. Was the original B1.2 surface blocker.
- **issue 0150** (deferred) — a `void` struct field crashes the compiler (unsized-type SIGTRAP
in LLVM `getTypeSizeInBits`). Blocks `Future(void)``timeout` (B1.4). Repro: `issues/0150-...`.
- (Note: **issue 0149**, filed by another session against an earlier dirty binary, was a
manifestation of the pre-fix 0151 — now moot.)
- **Orthogonal (not a B1 blocker):** default VALUES for comptime params don't bind on
generic-struct methods (free-fn defaults DO work) — inherited from Stream A. Only matters
if a B2 lib type wants a defaulted comptime param; atomics/fibers require explicit, so
unaffected.
- **Issue 0144 (open, independent):** calling an unrecognized bodiless `#builtin` silently
returns 0 / exit 0 — a silent-fallback footgun in the generic builtin-call path. Filed;
leave for its own fix session unless prioritized. Not a B1 blocker.
- **Deferred design gap (documented):** the B1.4 event-loop `Io` does not yet cooperate with
a platform UI run loop (CFRunLoop/NSRunLoop/ALooper); pinning gives thread-affinity, not
run-loop integration — a §6 app-target concern, out of B1 scope.
## Decisions (Stream B1 specifics; surface locked in design §4 / §4.6)
- **The async runtime is sx LIBRARY code.** The compiler provides only: the general
primitives (inline asm ✅, `abi(.naked)` naked [B1.0], atomics ✅) + fiber-safe codegen
(`context` already fiber-local — B1.1). Schedulers, fibers, channels, futures, `Io`
vtables, `mmap` stacks are all sx.
- **`abi(.naked)` is the real spelling of the design's `callconv(.naked)`** — postfix slot,
`name :: (sig) -> Ret abi(.naked) { asm { … }; }`. B1.0 = carry it into IR + emit LLVM
`naked` + skip prologue/ctx (mirror the existing `.c` skip), NOT extend the enum (it's
already there, just inert).
- **`.naked``.c`:** a `.c` epilogue would restore SP from the wrong stack across a context
switch (SP-in ≠ SP-out by design). `.naked` = no prologue/epilogue/frame; the asm emits its
own `ret`. This is why the switch must be `.naked`.
- **Naming:** sx-facing name is **`naked`** (keyword `abi(.naked)`, field `is_naked`, the
diagnostic), matching LLVM's `naked` attribute and the industry term (Zig/Rust/GCC/Clang).
The ABI variant was renamed `.pure → .naked` (user direction): "pure" universally means
*side-effect-free*, the opposite of a register-clobbering context switch.
- **B1.0 snapshot scope:** a `.naked` body is raw per-arch asm; LLVM's `naked` attr text is
arch-invariant. **B1.0a** = one host example locked to the emit bail (host-independent —
fires before instruction selection; no `.build` pin). **B1.0b** = pin aarch64 + add an
x86_64 cross sibling (`.build` target-gated, ir-only on mismatch), like the asm corpus
split. The `.ir` proves the `naked` attr + asm emitted, NOT register-save correctness
(that's B1.3's stress harness).
- **B1.1 — per-fiber context is library-only (CONFIRMED by probe):** push frames are
stack-`alloca`'d and the implicit ctx rides slot 0, so the spawn convention — snapshot
`context`, store it, `push f.root { entry(args) }` from the trampoline — installs the
fiber's root with no compiler change. Verified: the body reads the snapshot over a different
ambient context, and `push` restores ambient on exit (`1804-...-context-snapshot`). The
design doc's "never raw TLS" guarded a non-problem (context was never TLS).
- **Test keystones (design §10):** the **B1.3 switch-stress harness** gates the
context-switch (the one piece the deterministic `Io` can't test — §8.1.1, §10.7); the
**B1.4 deterministic-sim `Io`** (calibrated against blocking `Io` — §8.1.3) gates all
scheduling tests. Both must exist + be calibrated before the async tests they gate are
trusted. `18xx` asserts program-emitted ordering contracts, not raw interleaving.
## Log
- **B1.6 — aarch64-linux port of sched.sx.** Comptime-branched the per-OS bits: `MAP_AP` (linux
`0x22` / macOS `0x1002`), the fd-readiness backend (epoll on linux, kqueue on darwin — epoll import
scoped to the linux branch; `block_on_fd` / run-loop Mode-2 / `cancel_io_waiter_for` each branch,
epoll `EPOLL_CTL_DEL`s on fire + early-wake), and the first-entry trampoline (per-OS global-asm
symbol → naked sx fn `fib_tramp` + register-indirect `br x20` to `&fib_dispatch` preset in
`regs[1]`). **Fixed issue 0193 Bug A:** the tramp redesign bus-errored on 1817 (both OSes) until
`export "fib_dispatch"` was restored — without it the fn uses sx's internal ABI (x0 = implicit
`context`, `self` → x1) while the trampoline supplies `self` in x0, so the closure loads
`regs[1] == &fib_dispatch` as its first capture and recurses forever → stack-overflow bus error.
Root cause found via lldb (AOT macOS build) + an adversarial source review. **Bug B** (wrapped
top-level `asm` dropped) carved to **issue 0194** (OPEN; no live trigger — the naked-fn tramp
sidesteps it). Validated byte-identical on aarch64-macOS host AND aarch64-linux Apple `container`
for 1811/1814/1816/1817; full suite GREEN **817/0**.
- **B1 follow-up — `Scheduler.deinit`.** Closes the bounded leaks B1 documented. Added a `task_allocs:
List(*void)` field (appended in `go` so the scheduler can reach its generic `Task($R)`s) + a canonical
`close` extern, then a terminal idempotent `deinit`: reap leftover ready fibers (`munmap` + free) →
free tracked Tasks → `List.deinit` the 3 backings → `close` the lazy kqueue fd (reset `-1`). Closure
envs stay unfreeable (documented). Probe-observed the accounting under a tracking GPA (deinit drives
live allocs 7→3 in a spawn+sleep+2×go run; residual = envs). Locked by
`1820-concurrency-fiber-scheduler-deinit.sx` (one run hits timers + kqueue fd + Tasks; `freed by
deinit: 5`, `live after deinit: 5` (env residual), `kq open after run: true`→`kq after deinit: -1`,
`read: 3 [97 98 99]`), `.build {"target":"macos"}`. Adversarial review: no real UAF/over-free in the
supported deinit-after-`run` path; reconciled a doc contradiction (terminal-contract wording); 0154
over-store concern probed + cleared (`kq == -1` right after `init`). Suite GREEN **759/0**.
- **B1.4c — real fd-readiness blocking via kqueue (macOS).** De-risked first with a no-scheduler probe
(confirmed `size_of(Kevent)==32` and the pipe→kevent roundtrip: `kq_wait` returned 1, `out.ident ==
read_fd`, `out.filter == -1`, `out.data == 1` — the struct layout reads the fd back correctly). Then
added to `library/modules/std/sched.sx` (importing the existing verified `std/net/kqueue.sx` as `kqb`
rather than re-deriving the FFI): a lazy `kq: i32` (-1 until first use), `io_waiters: List(IoWaiter)`,
`block_on_fd(fd, want_read)` (arm one-shot `EVFILT_READ`, record waiter, `suspend_self`), a run-loop
Mode 2 (block on `kq_wait(kq, evbuf, MAXEV=16, -1)` when only fd waiters remain, wake the fiber whose
fd fired), and `wake` now also evicts a stale fd-waiter (`cancel_io_waiter_for`, the same UAF guard as
`cancel_timer_for`). Timers keep precedence over fds (documented non-unification). Orphan-deadlock
check still fires for a genuine no-timer/no-fd suspend (probed: exit 134). Locked by
`1816-concurrency-fiber-io-pipe.sx` (reader blocks on empty pipe → writer writes `a b c` → kqueue
wakes reader → reads 3 bytes; `log: wrote read 3 [97 98 99]`, `n_suspended: 0`), `.build`
`{ "target": "macos" }`, runs end-to-end on host. The example's `read`/`write`/`close` externs use the
canonical signatures std already binds (extern-dedupe rejects a divergent re-binding). Suite GREEN
**754/0**. Next: B1.5 (end-to-end M:1 validation); linux epoll twin deferred.
- **carve** — wrote PLAN-FIBERS.md + CHECKPOINT-FIBERS.md. Grounded the B1 compiler floor:
`ABI.naked` inert (type_resolver.zig:237), IR `Function` has no naked flag (inst.zig:605),
attribute API pattern (emit_llvm.zig:1339 nounwind), `.c` ctx-skip precedent
(decl.zig:515), `push Context` stack-alloca + slot-0 implicit ctx (stmt.zig:1263,
lower.zig:259), `__sx_default_context` root (decl.zig:2667/2815), inline-asm corpus
(1645/1651). Corrected the design's `callconv(.naked)` → real `abi(.naked)` spelling and
the B1.0 snapshot story. B1.1 grounded as likely library-only. Baseline green (721/0).
- **B1.0a** — plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites);
`funcWantsImplicitCtx` skips `.naked` (no implicit ctx, like `.c`); both body-lowering
paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx return);
`emit_llvm` Pass 2 bails loudly on `func.is_naked`. `examples/1800-concurrency-naked-asm.sx`
locked to the bail (exit 1 + diagnostic). Suite green (722/0). (ABI variant later renamed
`.pure → .naked` — see the Naming decision above — so all `is_*`/`abi(.*)`/example names
here read `naked`.)
- **B1.0a review-hardening** — adversarial review found generic/pack Function-creation paths
left `is_naked` false (silent framed body for a generic `.naked` instance — returned 42 but
corrupted the stack). Fixed generic.zig + pack.zig (set `is_naked` + asm-only `unreachable`
cap); locked by `examples/1801-concurrency-naked-generic-bail.sx`. The review's `.naked`-
lambda CRITICAL was a false positive (unparseable — `isLambda` breaks on `abi`). Suite
green (723/0).
- **B1.0b** — real `naked` emission: emit_llvm declaration pass adds LLVM `naked`/`noinline`/
`nounwind` + skips `frame-pointer` for `func.is_naked`; Pass 2 emits the body verbatim (no
prologue). `1800` green aarch64-pinned (exit 42 + `.ir`); renamed `1801` → `-generic`
(generic `.naked` emits a naked body, exit 42); added x86_64 sibling `1802` (ir-only, `.ir`
locks `naked` + `movl $42, %eax`). Unit test asserts `naked` present + `frame-pointer`
absent. Suite green (724/0).
- **B1.0c** — review-hardening: param-bearing `.naked` emitted invalid LLVM (loud verifier
error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both paths + generic.zig)
— naked args stay in registers, read by the asm body (the B1.3 context-switch shape).
Locked by `examples/1803-concurrency-naked-asm-param.sx`. Pack `.naked` left unsupported
(loud, nonsensical). **B1.0 complete.** Suite green (725/0).
- **rename** — ABI variant `.pure → .naked` (keyword, `Function.is_naked`, diagnostics,
examples 1800-1803 `*-pure-* → *-naked-*`, docs). "pure" universally means side-effect-free
— wrong for a register-clobbering switch; "naked" matches LLVM/Zig/Rust/GCC/Clang. Pure
cosmetics, no semantic change. Suite green (725/0).
- **B1.1** — per-fiber `context` root: **zero compiler change** (probe-confirmed). The spawn
convention (snapshot `context` → store in a struct → `push f.root { entry() }` from the
trampoline) installs the fiber's root via the implicit slot-0 `*Context` param; the body
reads the snapshot, not the trampoline's ambient ctx, and the `push` scope restores ambient
on exit. Locked by `examples/1804-concurrency-context-snapshot.sx` (prints `fiber root: 42`
/ `ambient after: 99`). Suite green (726/0). **Next: B1.2 (Io interface + context.io).**
- **B1.2 (BLOCKED)** — built the full `Io` capability (protocol on `Context`, stateless
`CBlockingIo` blocking default, both `__sx_default_context` materializers, push-inherit-omitted
fix, `!`-impl-method warning fix) and VERIFIED the core works live (`context.io.now_ms()` →
`clock ok`). Two independent compiler bugs blocked the `async`/`await`/`timeout` layer:
**0150** (`void` struct field → unsized SIGTRAP, blocks `Future(void)`) and **0151** (type-var
from a fn-ptr param's return type not bound in the body, blocks `async`'s `Future(R)`). Both
filed with standalone repros + investigation prompts. Per the STOP rule: reverted ALL B1.2
working changes (master green again, 726/0; the dirty binary had broken the photo project —
see the now-moot 0149), saved WIP to `.sx-tmp/b12-wip/`, STOPPED. Resume after 0150 + 0151.
- **0151 FIXED** — generic inference now binds `$T` through a generic-struct param head, a
pointer (`*Box($T)`, incl. UFCS auto-ref), and a closure-return-via-pack on the UFCS path.
Four gaps closed: `parameterized_type_expr` arm in `extractTypeParam`/`matchTypeParam(Static)`
(recovers the arg instance's recorded per-param bindings, recurses positionally); pointer arm
falls through to match a value arg (auto-address-of); `ExprTyper.inferType` `.lambda` arm
(closure type from annotations — UFCS types args from raw AST pre-lowering); pack UFCS target
routes through `lowerPackFnCall` with the receiver spliced in as `args[0]`. Issue 0151 marked
RESOLVED; repro → `examples/0214-generics-ufcs-closure-return-pack.sx`; widened cases →
`examples/0215-generics-infer-through-pointer.sx`. Suite green 728/0. The now-callable async
surface immediately exposed a SEPARATE codegen bug — **issue 0152** (`Atomic(bool)` → sub-byte
i1 atomic, LLVM reject; `Future.canceled` hits it). Filed with standalone repro + fix prompt.
Per the STOP rule: shipped the 0151 fix, filed 0152, STOPPED. Resume the async examples
(1805/1806) after 0152.
- **0152 FIXED** — the atomic load/store emitters (`src/backend/llvm/ops.zig`) byte-promote a
sub-byte (`bool`→`i1`) access to its `i8` storage type and `trunc`/`zext` the value at the
boundary (new `atomicByteType` helper). rmw/cmpxchg left as-is (a `bool` rmw/CAS is rejected
at the sx level — integer-only — so a sub-byte element never reaches them; comments record
this). Regression `examples/1705-atomics-bool-byte-promoted.sx` (load/store round-trip). Issue
0152 marked RESOLVED. Suite green 729/0. With `Atomic(bool)` working, the async surface
exposed the TRUE remaining blocker — **issue 0153**: a re-exported generic value-failable
`($R, !E)` loses its `!` channel at the call site (the earlier "secondary `or` PHI" symptom
was this, NOT an `Atomic` cascade — confirmed it persists after 0152). Narrowed to the
generic+re-export co-requirement (non-generic re-export OK; direct generic import OK; only the
combination drops `!`). Root cause: the monomorphized return-type's error-set, reached via the
re-export alias, resolves to a non-`.error_set` TypeId, so `errorChannelOf`
(`lower/error.zig:148`) misses the channel. Filed `issues/0153-...` with a minimal co-located
2-file repro + a single-file stdlib-`await` repro + investigation prompt. Per the STOP rule:
shipped the 0152 fix, filed 0153, STOPPED. Resume the async examples after 0153.
- **0153 FIXED → B1.2 COMPLETE** — `inferGenericReturnType` (`src/ir/generics.zig`) resolved the
return-type AST in the CALL-SITE module, so a re-exported error set (`LE :: lib.LE`) resolved
to a non-`.error_set` alias and the planned call-result was a plain tuple (channel lost). Fix:
pin the source to `fd.body.source_file` around the return-type resolution, exactly as
`monomorphizeFunction` does — the `!E` now resolves to the real `.error_set`. One-function
change; full suite green (732/0), no regression. Issue 0153 RESOLVED; repro →
`examples/1058-errors-reexport-value-failable-channel.sx` (+ companion `lib.sx`). With the
channel preserved, landed the async examples: **`1805`** (`async`/`await` + `now_ms` → `sum:
42` / `double: 42` / `clock ok`) + **`1806`** (`cancel` → `await` raises `.Canceled` → `or`
default; `ok: 7` / `canceled: -99`). **B1.2 (Io capability + M:1 async surface) is COMPLETE.**
Next: B1.3 (fiber runtime) on the `.naked` context-switch substrate.
- **B1.3a-1 — context switch works.** Implemented the stackful switch in pure sx over
`abi(.naked)`: `swap_context(from, to)` (save callee-saved x19-x28 + fp/lr + sp into `*from`,
load from `*to`, `ret` onto `to`'s stack) + by-hand fiber bootstrap (SP = top of an
`alloc_bytes` stack, LR = a `.global _fib_tramp` global-asm trampoline that does `mov x0, x19;
bl _fib_body`, x19 = `*Fiber`). Proven via a probe (main↔fiber), then locked by
`examples/1807-concurrency-fiber-context-switch.sx` (aarch64-pinned): a 2-fiber ping-pong
(`rounds: 6`, `canary fails: 0` — a per-fiber stack canary survives every switch) + a 64-frame
deep recursive chain suspended at the bottom and resumed (`frames verified: 64` / `depth fails:
0`). The `bl _fib_body` reaches the sx body via `export "fib_body"` (the 1655 asm→sx pattern);
runs under JIT, ir-only on a non-arm host (`.ir` captured — `swap_context` shows `naked noinline
nounwind`). Suite green 733/0. **Honest scope:** indirect register/stack survival only; the
EXPLICIT every-callee-saved + FP scribble (§10.7) is B1.3a-2, still owed. Next: B1.3a-2.
- **B1.3a-2 — the §10.7 stress gate, adversarially reviewed.** Extended `swap_context` to the
COMPLETE AAPCS64 callee-saved set (added FP d8-d15 → 21-slot ctx) and wrote a naked
`scribble_verify` that loads a unique sentinel into all 18 callee-saved regs, yields, and counts
non-survivors on resume (176-byte frame saves/restores the caller's callee-saved + base; lr
round-trips the swap). The gate is a 2-fiber MUTUAL scribble (each clobbers the other's regs, so
survival ⇒ the switch saved+restored them). Locked by
`examples/1808-concurrency-fiber-switch-stress.sx` (`A/B mismatches: 0`). Validity proven by
negative controls (drop d8-d15 → 8/8; drop x27/x28 → 2/2). **Spawned an adversarial-review
worker (per the plan + user request): NO critical bugs** — callee-saved set complete (x18 rightly
excluded; d8-d15 suffices per §6.1.2), offsets/alignment/lr-sp dance all verified. Applied its
one rec: `boot` zeroes FP ctx slots so first-entry loads 0, not garbage. Honest residual gaps
(spec-correct for a call-boundary swap; in the example header): FPCR/FPSR/NZCV + TPIDR/TLS not
swapped, fp=0 blocks unwind — relevant at N×M:1 / signals, not here. Suite green 734/0.
Next: B1.3b (x86_64 sibling + mmap guard-page stacks).
- **B1.3b — mmap guard-page stacks (x86_64 sibling deferred).** Fiber stacks now `mmap` a
`[guard | usable]` region and `mprotect` the low 16KB page `PROT_NONE`, so a stack overflow
faults at the guard boundary instead of silently corrupting a neighbor (§8.1.1). Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
Guard FIRING validated manually (overflow → `Bus error` at `region+GUARD`, exit 134 via the sx
crash handler) — not corpus-pinned because a deliberate-overflow crash is host-fragile (and a
mere "child faulted" fork test wouldn't prove the BOUNDARY catch). The x86_64 `swap_context`
sibling was DEFERRED: `--target x86_64-macos` mislinks on this arm64 host and `x86_64-linux`
can't run here, so it could only ship un-run/un-negative-controlled — which §10.7 forbids for the
highest-risk asm. SysV target notes (rbx/rbp/r12-r15/rsp, no callee-saved XMM, rsp-carried return
addr) recorded in Next step. Suite green **735/0**. Next: x86_64 sibling (needs an x86_64 host)
OR B1.4 (`Io` impls / scheduler) on the proven aarch64 substrate.
- **B1.3b-1 — x86_64 / Win64 switch sibling VALIDATED on real hardware.** The user provided a
Windows 7 x64 VM (UTM), so the x86_64 switch became RUNNABLE (as Win64). Validated the
cross-build→VM→run loop (`--target x86_64-windows-gnu --self-contained` → PE32+; output via the
Win32 `WriteFile` boundary, the 1660 pattern). Wrote a Win64 `swap_context` (8 GP rbx/rbp/rdi/
rsi/r12-r15 + rsp + **xmm6-xmm15** via `movups` — Win64 has callee-saved XMM) + a Win64
`scribble_verify` (264-byte frame, 32-byte shadow + 16-align at each `call`, COFF symbols,
rsp-carried return addr) driving the 2-fiber mutual scribble. **Adversarially reviewed (worker
emitted the real `.s`, verified every alignment/offset/round-trip against the Win64 ABI — no
critical/minor bugs), THEN run on the VM → `0 0 P`** (all 8 GP + 10 XMM callee-saved survived).
Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only on this host; the VM run is the runtime-correctness provenance). Good-swap-only (the
in-process negative control was dropped to avoid an sx fn-ptr-convention rabbit hole; the
detection of this exact logic was negative-controlled on aarch64 in 1808). Suite green **736/0**.
The B1.3 context switch is now proven on TWO arch/ABI pairs. Next: **B1.4** (Io impls / M:1
scheduler) on the proven substrate. (Side thread: the SysV/Linux x86_64 sibling, when a Linux
x86_64 host is available.)
- **B1.5a — M:1 scheduler CORE + a fixed blocker bug.** Built `library/modules/std/sched.sx`: a
generic `Fiber`/`Scheduler` over `swap_context` on guarded `mmap` stacks. `spawn` heap-allocs a
fiber, bootstraps its ctx, enqueues it; the ONE generic dispatch (`fib_dispatch` via `_fib_tramp`)
runs ANY stored `Closure() -> void` on a fresh stack (replacing the fixed `bl _fib_body`);
`yield_now` round-robins, `suspend_self`/`wake` park/resume off-queue, `run` drives to drain +
reaps `.done` fibers (`munmap` + free). **De-risked first by probe** (closure-on-fiber + output
via captured pointer). **Hit blocker bug 0154** (user-authorized fix): `null`/`---` to a struct
field over-stored a whole-struct null when the fn return type leaked as `target_type`, corrupting
the frame (`ret` 0x0) — exactly the `Scheduler.init()` by-value-return shape. Fixed in `stmt.zig`
(`needs_target` += `null`/`undef` literals); regression `examples/types/0193`; `0154` RESOLVED.
**Adversarial review:** asm/bootstrap/lifetime sound (env-lifetime fear disproven — heap-promoted);
1 CRITICAL (`wake` re-enqueue → FIFO segfault) + robustness gaps ALL hardened (wake guarded on
`.suspended`, `n_suspended` deadlock diagnostic+abort, loud mmap/mprotect/OOM bails, env-leak
documented). Locked `1811` (round-robin `0 1 2 ×3`) + `1812` (suspend/wake + spurious-wake guard,
`log: 10 20 21 11`). Filed NON-blocking `0155` (scalar-pointer index panics codegen — review
incidental, unused by sched). Suite GREEN **748/0**. Next: **B1.4a** (FiberIo).
- **B1.4a (truly-suspending fiber-task async, nullary-thunk design) — BLOCKED on issue 0157.**
Implemented the async layer SELF-CONTAINED in `library/modules/std/sched.sx` (kept its lone
`#import "modules/std.sx"` to avoid the duplicate-`_fib_tramp` trap): `TaskState`, a LOCAL
`TaskErr :: error { Canceled }` (the re-exported `IoErr` alias is NOT seen through by the
`raise`/failable-type check — verified), `Task($R)`, and `go`/`wait`/`cancel` ufcs. Design is
the validated nullary-thunk (`.sx-tmp/pnullary.sx` → `log: 1 2 3 42 100`): `work` is a
`Closure() -> $R`, user captures inputs at the call site, NO `..args` crosses the fiber boundary
(deliberately sidesteps 0156). `go`+`wait` run correctly; both wake-orderings traced. Wrote the
example `examples/concurrency/1813-concurrency-fiber-async-suspend.sx` (+ `{ "target": "macos" }`
`.build`) but its `cancel` ufcs surfaced a NEW compiler bug — issue **0157**: a user generic
ufcs whose name collides with a stdlib re-export (`cancel` from io.sx) is mis-resolved on UFCS
call over a different generic struct, leaving `$R` unresolved → LLVM panic. Bisected to a minimal
no-fiber repro (name is the sole trigger; non-UFCS form diagnoses correctly). Example NOT seeded
into the corpus (no `.exit` marker) — do NOT regen its goldens until 0157 lands. Per the STOP
rule: filed `issues/0157-*.{md,sx}`, marked state BLOCKED, paused.
- **B1.4a COMPLETE (this session) — suspending fiber-task async + two compiler fixes.** Built the
`Task($R)` + `go`/`wait`/`cancel` layer in `sched.sx` (nullary-thunk design; self-contained to
avoid the `_fib_tramp` duplicate-symbol trap). Locked `1813` (`sequence: 1 2 3 42 100 -99`).
FIXED the two blockers the worker had filed: **0156 Part 1** (`comptime_pack_ref` arm in
`resolveTypeWithBindings`; regression `0216`) and **0157** (receiver-driven UFCS overload
selection `selectUfcsGenericByReceiver`; regression `0217`). Adversarial review of the 0157 fix +
Task layer found a determinism CRITICAL (always-run selection + specificity + ambiguity
diagnostic), a `wait`-outside-fiber null-deref (loud guard), and cancel-not-skipping-work (skip
if pre-canceled) — all fixed. Simplified `1812` (`**Fiber` → `Sh.parked`). 0156 Part 2 reframed
OPEN/non-blocking. Suite GREEN **751/0**. Next: B1.4b (deterministic-sim `Io`, the KEYSTONE).
- **B1.4b COMPLETE (this session) — deterministic virtual-time timers + a CRITICAL UAF fix.** Added
`clock_ms`/`timers`/`now_ms`/`sleep` + a timer-driven `run` to `sched.sx` (worker-built): fibers
sleep in reproducible simulated time, waking in deadline order (FIFO tiebreak). Locked `1814`
(5 fibers, wake order B@10/D@15/E@15/C@20/A@30). Adversarial review of the run-loop change found a
CRITICAL use-after-free — a fiber woken EARLY (manual/Task `wake`) before its `sleep` timer fired
was reaped while its `Timer` kept a dangling `*Fiber`; a later fire dereferenced freed memory
(silent "pass" only by luck). Fixed: `wake` evicts the fiber's pending timer (`cancel_timer_for`);
regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended
accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination.
Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll).
- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a
lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx`
(worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd
lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a
never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment
corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0.
- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack
(`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order
(task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async
runtime is feature-complete end-to-end (18001817), all adversarially reviewed. Suite GREEN
**755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in
B1.2). Next carve: Stream B2 (channels / cancel / async stdlib).
- **Post-review hardening (this session) — 6 findings from an adversarial review of the B1 commits.**
Fixed: **P1-a** the UFCS generic PLANNER (`calls.zig`) used the last-wins `fn_ast_map` winner while
lowering reselected by receiver → plan/lowering could disagree and MISBOX the result; now both share
`selectUfcsGenericByReceiver`. **P1-b** the selection scanned `module_decls` globally, flagging a
transitively-hidden same-named overload as a FALSE ambiguity; now two-tier — directly-visible authors
first (ambiguity only among those), global fallback for receiver-reachable namespaced methods (e.g.
`Task.cancel`) that defers to `fd0` on a hidden tie. **P2-b** boolean specificity tied `*$T` with
`*Box($T)`; now peels pointer layers so the structurally-narrower receiver wins. **P1-c** a second
concurrent `Task.wait` overwrote the single waiter slot → silent deadlock; now one-awaiter-per-task
loud abort. **P2-c** `sleep(negative)` rewound the virtual clock; now rejected loudly. (**P2-a**
non-generic-winner-hides-generic did not reproduce — the non-generic arm already falls through.)
Regressions: `examples/generics/0218` (receiver specificity + plan/lowering agreement),
`examples/concurrency/1818` (negative-sleep abort), `1819` (double-wait abort). Suite GREEN **758/0**.

View File

@@ -1,62 +0,0 @@
# CHECKPOINT-HTTPZ — Stream HTTPZ (production HTTP-server readiness)
Tracker for the HTTP-server production-readiness stream. Plan:
[PLAN-HTTPZ.md](PLAN-HTTPZ.md). Update after every step.
## Last completed step
**Stream established (planning only).** Audited the existing HTTP/socket/thread/event
stack against the user's production-readiness checklist and wrote
[PLAN-HTTPZ.md](PLAN-HTTPZ.md) + this checkpoint. **No code changed.** Prior HTTP work
(socket `S2`, thread `S6`, http `S7a`, pool `S7b`) shipped without a tracked plan; this
brings the stream under checkpoint discipline.
## Current state
- **Done & Linux-validated (do NOT rebuild):** `event.sx` (epoll+kqueue, 6/6 green on real
aarch64 Linux in Apple `container`), `net/epoll.sx`, `net/kqueue.sx`, `sched.sx` M:1
runtime, `json.sx`.
- **BROKEN on Linux (Phase C3 keystone):**
- `socket.sx` — Darwin-only `SockAddr` (`sin_len`), `O_NONBLOCK=4`, macOS errno values,
`__error` binding. Corrupts addresses + breaks WouldBlock detection on Linux.
- `thread.sx``MutexBuf=64B` (Darwin) vs glibc 40B → 24-byte heap overflow on
`pthread_mutex_init`. Pool unsafe on Linux.
- **Works, unhardened — `http.sx`:** single-worker loop + inline/pool handlers, keep-alive,
delivery timeouts, conn/request caps, 400/413/431/503. Gaps: parser limits, `Server.close()`
leaks (`conns`/`PoolState`/`done`), no graceful stop, no handler-exec timeout, zero
observability, no streaming.
- **Absent entirely:** CI (no Linux CI), fuzz, sanitizers/leak-check (`tests/stress-http.sh`
broken — references deleted `32-http-server.sx`), releases/tags, SECURITY.md, deploy docs,
routing/form helpers. **TLS:** none yet — to be added natively via mbedTLS FFI (Phase T).
Full grounded audit (file:line) lives in PLAN-HTTPZ.md "Audit of record".
## Next step
**Phase C3a — `socket.sx` per-OS selection.** Branch `SockAddr`, `O_NONBLOCK`, errno
constants, and `errno_slot` on `OS`/`ARCH` (mirror the `inline if OS ==` pattern in
`event.sx`/`sched.sx`). Lock a Linux-vs-Darwin layout/const assertion red (cadence rule),
then flip green; validate under the Apple `container` Linux VM. No silent fallback defaults.
> **NOT STARTED** — user requested plan-only this session. Execution begins next session.
## Known issues / capability gaps
- `socket.sx` / `thread.sx` Linux-broken (above) — blocks all Linux P0 acceptance.
- No CI of any kind in the repo → "tested on Linux" cannot be claimed until C4.
- Corpus runner (10s/example timeout, no net sandbox) cannot host stress/fuzz/load — those
go in separate CI-wired scripts (PLAN decision).
- `http.sx` `Server.close()` leaks on shutdown (H2).
- No handler-execution timeout in either handler mode (H5).
## Decisions (HTTPZ specifics — full list in PLAN-HTTPZ.md)
- Native TLS via an mbedTLS FFI binding (Phase T) — supersedes the original proxy-only
posture; proxy deployment stays supported/documented. No pure-sx TLS stack.
- `Transfer-Encoding: chunked` rejected (501) in H1, implemented in S1/S2.
- Stress/fuzz/load harnesses live outside the corpus, wired into CI.
- C3 branching bails loudly on unhandled OS/arch arms — no Darwin-default fallback.
## Log
- **2026-06-26** — Stream established. Parallel audit of `http.sx`, `socket.sx`,
`thread.sx`, `event.sx`, `net/epoll.sx`, `net/kqueue.sx`, `sched.sx`, the test/CI/bench
infra, and the docs/release/security posture against the production-readiness checklist.
Wrote PLAN-HTTPZ.md (phases C/H/S/D mapping checklist P0/P1/P2) + this checkpoint.
No code changes. Next: Phase C3a.
- **2026-06-26** — Added **Phase T (native TLS via mbedTLS FFI)** to PLAN-HTTPZ.md, slotted
after Phase H; flipped the proxy-only decision to native-TLS-plus-proxy; updated D1.
Backend chosen: mbedTLS (static-link-friendly, clean non-blocking API). Still plan-only.

View File

@@ -0,0 +1,276 @@
# CHECKPOINT-METATYPE — comptime type metaprogramming (`declare` / `define`)
Companion to [PLAN-METATYPE.md](PLAN-METATYPE.md). Update after every step (one
step at a time, per the cadence rule).
## Last completed step
**`type_info` / `define` widened to TUPLE types — reflect/construct triad
complete.** `TypeInfo` gained a `` `tuple(TupleInfo) `` variant (`TupleInfo{
elements: []Type }`, positional/unnamed). `reflectTypeInfo` builds `.tuple`
(tag 2) as bare `type_tag` elements; `defineTuple` decodes `[]Type` and completes
the declare slot as a structural `.tuple` via `replaceKeyedInfo` (tuples are
structural, so the declared name is vestigial, but the slot is completed in place
so `define` returns the handle like enum/struct). `call.zig`'s `type_info` guard
admits `.tuple`. `examples/0623` (programmatic `Pair` + source-tuple round-trip).
Suite green (684). All three TypeInfo shapes now reflect + construct + round-trip
(`0619` enum, `0622` struct, `0623` tuple).
## Earlier — struct widening
**`type_info` / `define` widened to STRUCT types.** `TypeInfo` gained a
`` `struct(StructInfo) `` variant (`StructField{ name, type }`); the metatype
system now reflects AND constructs structs, not only enums.
- `meta.sx`: `StructField` / `StructInfo` / `` `struct `` TypeInfo variant.
- `interp.zig`: `reflectTypeInfo` builds `.struct` (tag 1) for a source
`@"struct"`; `define` dispatches on the TypeInfo tag (`defineType` →
`defineEnum` (0) / `defineStruct` (1)). `defineStruct` mirrors `defineEnum`
(duplicate-field-name check included) but completes the declare slot AS a
struct via `replaceKeyedInfo` — a KIND change re-keys the intern map, whereas
`updatePreservingKey` (the enum path) asserts the key is unchanged.
- `lower/call.zig`: the lower-time `type_info` guard now admits `@"struct"`.
- `examples/0622`: programmatic `Vec2` via `.struct(.{ fields = … })` + a
source-struct round-trip `define(declare("RowCopy"), type_info(Row))`. Enum
path (`0619`) unchanged. Suite green (683). Tuple is the last shape (Next step).
## Earlier — make_enum
**`make_enum(name, variants: []EnumVariant) -> Type`** — the general enum
constructor over `declare`/`define`, minting a nominal enum from a variant list
passed as a VALUE. Pure sx in `meta.sx`. `examples/0620` assembles the list in a
local then mints, exercising `define`'s value-arg SLICE decode.
## Prior step
**`type_info($T)` reflection — enum round-trip.** Reflect a type INTO a `TypeInfo`
value (the inverse of `define`'s decode), so `define(declare(n), type_info(T))`
mints a byte-identical copy with NO literal variant list.
- `inst.zig`: new `BuiltinId.type_info` (comptime-only, alongside `declare`/`define`).
- `lower/call.zig:tryLowerReflectionCall`: the old "not yet implemented" bail is
gone. Resolve `$T` at lower time, reject a non-`enum`/non-`tagged_union` arg
loudly (good span: `"type_info: 'X' is not an enum …"`), else emit
`callBuiltin(.type_info, [const_type], TypeInfo)`.
- `interp.zig:reflectTypeInfo`: builds the exact nested-aggregate Value
`defineEnum` decodes — variant `{name, payload}`, slice `{data, len}`, EnumInfo
`{variants}`, TypeInfo `{tag0, EnumInfo}`. A `tagged_union` reflects each
`field.ty` (tagless variants already carry `void`); a payloadless `` `enum ``
reflects `void` per variant. Round-trips both source enums AND constructed
(declare/define) enums.
- emit unchanged — `type_info` is always comptime-evaluated; the existing
comptime-only `else` arm in `emitCallBuiltin` (shared with declare/define)
never fires.
- Scope: **enum-only** (the symmetric inverse of `define`'s current capability).
Struct/tuple `TypeInfo` widening is a separate later step.
`examples/0619` locks it (source enum `circle:f64 / rect:i64 / empty` reflected →
reconstructed → constructs + matches). Full suite green (676 examples + units).
## Earlier step
**Self-reference — recursive enums via `declare("Name")` + `*Name`.** The
`declare`/`define` floor now supports self-referential types.
- `declare(name) -> Type` mints an empty (undefined) nominal slot NAMED `name`;
`define(handle, info) -> Type` decodes the `TypeInfo` value (variant names +
payload Type-tags), fills the slot byte-identical to a source enum, and returns
the handle (one-shot form chains: `T :: define(declare("T"), info)`). Interp
executes both against a `mint` TypeTable handle; `defineEnum` +
`decodeVariantElements` in `interp.zig`.
- **Self-reference:** `evalComptimeType`'s `preregisterForwardTypes` scans the
comptime expression (and a called ctor fn's body) for `declare("Name")` calls
and, before the body lowers, registers each as an empty forward nominal type AND
binds it as a type alias. The alias is essential — a `Name :: ctor()` decl makes
`Name` a const_decl author, so a `*Name` self-reference resolves through the
forward-ALIAS path (`type_aliases_by_source`), which a bare `findByName`
registration alone does NOT satisfy (it returns a pending empty-struct stub). The
interp's `declare` returns that same slot; `define` fills it.
- A `::` binding or type-fn body calling a `Type`-returning fn is
**comptime-evaluated** (`evalComptimeType`) — no constructor-name knowledge.
`decl.zig` trigger = `fnReturnsTypeValue`; type-fn trigger = `returnExprMintsType`.
- Nominal identity rides the type-fn instantiation cache (`renameNominalType`).
- The type NAME is on `declare(name)` (compile-time string), not `EnumInfo`.
Examples green: `0614` (one-shot), `0615` (type-fn identity), `0617` (channel
results), **`0618` (recursive `*List`: construct, match through pointer, recursive
traversal)**; `field_type` reflection `0616`. Full suite green (674 examples).
## Current state
- `modules/std/meta.sx`: `EnumVariant` / `EnumInfo{ name, variants }` / `TypeInfo`
data types; `declare` / `define` / `type_info` / `field_type` `#builtin`s;
`RecvResult($T)` / `TryResult($T)` + the general `make_enum(name, variants)` sx
constructors over `define(declare(), …)`.
- Compiler primitives only: `declare`/`define` (construction), `field_type`
(reflection). No constructor-name knowledge anywhere in the compiler — every
named constructor is sx. `declare(name)` carries the type name (compile-time
string) for forward-type registration.
- `type_info($T)` reflects an `enum`/`tagged_union`/`struct`/`tuple` INTO a
`TypeInfo` value (`call.zig` emits `callBuiltin(.type_info)`;
`interp.zig:reflectTypeInfo` builds the Value). `define` decodes `.enum` →
tagged_union, `.struct` → struct, `.tuple` → tuple (the last via
`replaceKeyedInfo`). `examples/0619` (enum) / `0622` (struct) / `0623` (tuple)
round-trip. All three TypeInfo shapes ship.
## Decision (kept)
**Meta lives in `modules/std/meta.sx`, not the prelude.** Declaring its data types
in the always-loaded prelude interns them into every module's type table and
shifts every `.ir` snapshot. On-demand import keeps the prelude clean.
## Next step
The reflect/construct triad is COMPLETE — `` `enum `` (`0619`), `` `struct ``
(`0622`), `` `tuple `` (`0623`) all reflect AND construct + round-trip. Remaining
METATYPE work is ONE deferred enhancement, a clean diagnostic rather than a crash
— filed as **issue 0141** (repro `issues/0141-*.sx` + full two-layer writeup +
investigation prompt):
- **Comptime `List` growth** — `List(T).append` at comptime bails ("struct_get:
base has no fields"). Doesn't block anything: array-literal locals already build
variant lists (`examples/0620`/`0624`). Probe `.sx-tmp/probe_makeenum.sx` /
`probe_li64.sx`. **Investigated — it's TWO layers** (both reproduce with plain
`List(i64)`, not metatype-specific; List works via `#run` because that evaluates
at EMIT time, after everything is lowered, while a metatype `::` const evaluates
at `scanDecls` time):
1. **Null comptime allocator.** `interp.zig:defaultContextValue` builds the
comptime `context.allocator` by looking up `__thunk_CAllocator_Allocator_alloc_bytes`
by name in the module's functions — but at `scanDecls` time those protocol
thunks aren't lowered yet, so `alloc_fn`/`dealloc_fn` are `.null_val` and any
comptime allocation fails. FIX (tried, works for this layer): call
`self.getOrCreateThunks("Allocator", "CAllocator")` (guarded by the same
Context/Allocator/CAllocator-registered check `emitDefaultContextGlobal` uses)
before the interp runs in `comptime.zig:runComptimeTypeFunc`.
`createProtocolThunk` saves/restores builder state, so calling it mid-lowering
is safe. After this, `alloc_fn=func_ref` — but layer 2 still bails.
2. **`struct_get` through a `*T` slot_ptr chain.** A `*List` struct receiver
(`vs.append(…)` → `append(self: *List, …)`) lands in the interp as a slot
whose contents are a slot_ptr to the actual value — `self.field` does
`struct_get` on `base=slot_ptr field_index=1` and bails. The auto-deref in
`interp.zig:.struct_get` does a single `loadSlot`; a chain-resolve loop did
NOT fix it (the final loaded value is a field-pointer aggregate that
`resolveFieldLoad` turns back into a slot_ptr — List's comptime representation
uses field-pointers + slot_ptrs the struct_get path doesn't fully resolve).
This is the deep part: comptime pointer/struct/slot resolution for `*T`
receivers, its own focused effort. Both speculative fixes were REVERTED (no
end-to-end testable win without layer 2).
The metatype surface (declare/define/type_info/field_type + make_enum) is
feature-complete for the locked design; generic type-fn body locals now work too.
- ~~**Validation + loud diagnostics**~~ — COMPLETE. duplicate variant names
(`examples/1180`); `declare()` never `define()`d (`examples/1181`, was a
`verifySizes` panic); by-value self-reference for both source (`1178`) and
CONSTRUCTED (`1182`) types via `checkInfiniteSize`. **use-before-define needs no
new check** — it's subsumed by the existing guards: a by-value cycle →
`checkInfiniteSize` ("infinitely sized"); an unfinished slot → declare-never-
defined; a bad/non-Type payload → a 0140 clean bail; a forward reference resolves
correctly via in-place slot mutation (`updatePreservingKey`); a `*Name` pointer
needs no layout. Probes `.sx-tmp/probe_ubd{1..4}.sx` confirmed: no remaining
crash or silent-corruption, only clean diagnostics / correct results.
### make_enum follow-ups (deferred capability gaps — NOT crashes; clean diagnostics)
`make_enum` itself is DONE (see Last completed step). Remaining adjacent
capabilities would let the variant list be built more freely; both error cleanly
(post-0140) rather than crash, so they're enhancements, not blockers:
- ~~Comptime slice over a non-string aggregate~~ — DONE. `arr[lo..hi]` over a
`[]EnumVariant` array now yields a real slice value at comptime (was: bailed,
string-only). Fix threaded `base_ty` onto the `Subslice` op so the interp tells
an array from a `{ptr,len}` slice, folded open-ended `hi` to a fixed array's
static length at lower time (no runtime/.ir change), and added
`interp.zig:subsliceElements`. `examples/0621` locks it.
- **Comptime `List` growth** (issue 0141). `List(T).append` at comptime bails
("struct_get: base has no fields"). Investigated — two layers (null comptime
allocator at scanDecls + `struct_get` through a `*T` slot_ptr chain); see the
detailed writeup under "Next step" and `issues/0141-*.md`. Layer 1 has a known
fix; layer 2 is deep. Probe `.sx-tmp/probe_makeenum.sx`.
- ~~Generic type-fn body locals~~ — DONE. A generic `($T) -> Type` now
comptime-evaluates its FULL body (prelude statements + return), so a local
before the return resolves. `createComptimeFunctionWithPrelude` +
`evalComptimeTypeBody`; no-prelude bodies stay on the old path. `examples/0624`.
## Known issues
- issue 0141 (OPEN, deferred enhancement — not a blocker) — `List(T).append` at
comptime bails in a type-construction `::` (two layers: null comptime allocator
+ `*T` slot_ptr `struct_get`). Workaround: array-literal locals
(`examples/0620`/`0624`). Full writeup + investigation prompt in
`issues/0141-*.md`.
- issue 0140 — comptime type-construction bail panicked instead of diagnosing —
RESOLVED. `evalComptimeType` now clears `last_bail_detail` before the interp
call and, on the `catch`, emits a build-gating `.err` at the construction span
("comptime type construction failed: {detail}") before returning the
`.unresolved` poison — so the reason is shown and no unresolved type reaches
emission unannounced. `examples/1179` locks it.
- issue 0139 — by-value self-reference segfault — RESOLVED (`checkInfiniteSize`
Pass 1g emits a loud "infinitely sized" diagnostic + breaks the cycle;
`examples/1178` locks it).
## Log
- **Generic type-fn body locals.** A generic `($T) -> Type` comptime-evaluated
only its return EXPRESSION, so a local before the return was unresolved. Now a
body with a prelude (statements before the return) has its FULL body evaluated:
`createComptimeFunctionWithPrelude` lowers the pre-return statements into the
comptime function's scope, then the return expr. No-prelude bodies (RecvResult
etc.) stay on the old path → zero regression. `examples/0624`. Suite green (685).
- **Tuple widening done — reflect/construct triad complete.** `TypeInfo` gained
`` `tuple(TupleInfo) `` (positional `[]Type`); `reflectTypeInfo` reflects a
`.tuple` (bare type_tags, tag 2), `defineType` dispatches tag 2 → `defineTuple`
(completes the slot as a structural tuple via `replaceKeyedInfo`), and the
lower-time `type_info` guard admits `.tuple`. `examples/0623`. Suite green (684).
enum/struct/tuple all reflect + construct + round-trip.
- **Struct widening done.** `TypeInfo` gained `` `struct(StructInfo) ``; `define`
dispatches on the tag (`defineType` → `defineEnum`/`defineStruct`), `reflectTypeInfo`
reflects a `@"struct"`, and the lower-time `type_info` guard admits structs.
`defineStruct` uses `replaceKeyedInfo` (kind change: tagged_union declare slot →
struct). `examples/0622` (programmatic build + source round-trip). Suite green
(683). Tuple is the last remaining shape.
- **Validation story COMPLETE.** use-before-define needs no new check — subsumed
by `checkInfiniteSize` (by-value cycles), declare-never-defined (unfinished
slots), 0140 bails (bad payloads), and in-place slot mutation (forward refs);
`*Name` pointer use needs no layout. Probed `.sx-tmp/probe_ubd{1..4}.sx`: all
clean diagnostics / correct results, no crash. `examples/1182` locks the
by-value self-ref rejection for CONSTRUCTED enums (companion to source `1178`).
- **declare()-never-defined validation.** A bare `declare("X")` with no `define`
left a zero-field nominal slot that panicked at codegen (`verifySizes`).
`evalComptimeType` now detects a zero-variant `tagged_union` result and emits a
clean diagnostic naming the type. Self-reference (declared slot completed by
`define`) is unaffected. `examples/1181` locks it. Suite green (681).
- **Duplicate variant-name validation.** Two same-named variants in a constructed
enum used to silently succeed (ambiguous construction/match). `defineEnum` now
bails naming the duplicate; `evalComptimeType` renders it (post-0140).
`examples/1180` locks it. Suite green (680).
- **Comptime subslice over non-string aggregates.** `arr[lo..hi]` at comptime
used to bail (interp `.subslice` was string-only) and the open-ended `hi` came
from a `.length` op that misread a 2-elem array as a `{ptr,len}` fat pointer.
Fix (interp-only; runtime already correct via `LLVMTypeOf`): thread `base_ty`
onto the `Subslice` op, fold open-ended `hi` to a fixed array's static length at
lower time (no IR/.ir change), add `subsliceElements`. `examples/0621` mints an
enum from `dirs[0..2]`. Suite green (679).
- **`make_enum` done.** General enum constructor `make_enum(name, variants:
[]EnumVariant) -> Type` in `meta.sx` (pure sx over declare/define). A non-generic
builder assembles the variant list in a local, then mints from it —
`examples/0620` exercises `define`'s value-arg SLICE decode. No compiler change.
Suite green (678). Deferred free-form gaps (subslice/List at comptime,
generic-type-fn locals) noted under Next step — all clean diagnostics now, not
crashes (post-0140), so enhancements rather than blockers.
- **issue 0140 fixed.** A comptime type-construction bail (`declare`/`define`/
reflection) used to panic at LLVM emission ("unresolved type reached LLVM
emission") or hide behind a cascade — `evalComptimeType` swallowed the interp's
`last_bail_detail`. Now it clears the detail before the call and renders a
build-gating `.err` at the construction span on the `catch`. `examples/1179`
locks the empty-variants case. Suite green (677). Unblocks make_enum (its
computed-slice decode failures now surface cleanly).
- **`type_info($T)` reflection done (enum round-trip).** New `BuiltinId.type_info`;
`lower/call.zig` resolves `$T`, rejects non-enum loudly, emits the builtin;
`interp.zig:reflectTypeInfo` constructs the exact nested-aggregate Value
`defineEnum` decodes (variant `{name,payload}` / slice `{data,len}` / EnumInfo /
TypeInfo `.enum`). `tagged_union` reflects `field.ty`; payloadless `` `enum ``
reflects `void`. Round-trips source AND constructed enums. Enum-only;
struct/tuple widening deferred. `examples/0619` locks it. Suite green (676).
- **By-value self-reference rejected (issue 0139, F5 partial).** New
`checkInfiniteSize` pass (Pass 1g) detects by-VALUE containment cycles (source +
comptime types, direct + mutual), emits a loud "infinitely sized" diagnostic,
and breaks the cycle (was a `typeSizeBytes` stack-overflow segfault). `*Self`
(pointer) stays valid. `examples/1178` locks the message. Suite green (675).
- **Self-reference done.** `declare(name)` + `preregisterForwardTypes` (forward
type + alias before body lowers) → `*Name` resolves; recursive `*List` enum
constructs, matches through the pointer, and traverses recursively. `0618` locks
it. `declare` gained its `name` arg; `EnumInfo.name` dropped. Suite green (674).
- **declare/define floor established.** The comptime type-construction surface is
two primitives (`declare`/`define`); all named constructors are sx. A `::` binding
or type-fn body that calls a `Type`-returning fn is comptime-evaluated (the
builtins mint the type) — no syntactic constructor recognition in the compiler.
Examples 0614 (one-shot) / 0615 (type-fn identity) / 0617 (channel results) on the
floor; `field_type` reflection (0616) unchanged.
- **Stream carved (earlier).** Selected as the first async-first foundation: gates
channel result types (`RecvResult($T)`) and `race`'s synthesized union, fully
validated, self-contained, testable in isolation (`06xx` comptime).

167
current/PLAN-ASM.md Normal file
View File

@@ -0,0 +1,167 @@
# sx Inline Assembly — Implementation Plan (ASM stream)
**Design source of truth:** [design/inline-asm-design.md](../design/inline-asm-design.md).
This plan turns that doc's §II.7 stage-map + §II.8 phasing into ordered,
commit-sized, testable steps. Read the design doc first — this file is the
*how/when*, not the *what/why*.
**Surface (decided):**
`asm volatile { "template", "=r" -> T, "r" = expr, clobbers(.cc, .memory) }`
— brace block; `->` output / `=` input; `clobbers(.…)` dot-name list; N `-> Type`
outputs return a tuple; templates are pure AT&T (via LLVM).
**Feasibility (confirmed):** sx links LLVM@19; `src/llvm_api.zig` `@cImport`s
`llvm-c/Core.h`, so `llvm_api.c.*` already exposes `LLVMGetInlineAsm` (9-arg),
`LLVMInlineAsmDialectATT`, `LLVMBuildCall2`, `LLVMAppendModuleInlineAsm`. No shim.
**Relationship to other streams:**
- Phases AE (the inline-asm *expression*) are independent of EXTERN-EXPORT.
- Phase F (global asm) consumes `extern`/`export` to import/expose asm symbols —
do it **after** `PLAN-EXTERN-EXPORT.md` Phase 2.
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass. Each feature step is either a
behavior-locking PASSING test, or an xfail test the *next* commit turns green.
Arch-pinned tests live in `examples/16xx-platform-asm-*` and declare their target
via the `expected/<name>.target` sidecar marker (Phase 0). Never regenerate
snapshots while red.
## Phase 0 — corpus target-gating (test-infra prerequisite; no compiler code)
**Why first.** The flagship v1 examples are `x86_64` (syscall-write, divmod,
cpuid) but the dev host is `aarch64`-Darwin, and the corpus runner
([src/corpus_run.test.zig](../src/corpus_run.test.zig)) currently (a) never threads
a per-example `--target` and (b) has no host-arch gate — its only skip is "marker
has no `.sx`". So D.0's `…-syscall-write` markers asserting exit/stdout describe
output the harness *cannot* produce on this host, which would violate the cadence
rule (the "next commit turns it green" can never happen). Phase 0 closes that gap.
It touches **only the runner + two fixtures** — zero compiler code, zero risk to
AE, and unblocks every arch-pinned asm example.
**Marker taxonomy (the cleanup).** The runner currently spreads per-example
*directives* across standalone boolean/value sidecars (`.aot` now, `.target`
proposed, more later). Replace that sprawl with **one optional config file,
`expected/<name>.build`**, holding all build/run directives; the output snapshots
(`.exit` / `.stdout` / `.stderr` / `.ir`) stay separate — they are
machine-regenerated data, not config. `.exit` remains the **test-discovery key**
(every test has one; `.build` is optional).
**`.build` format** — JSON, parsed with `std.json`:
```json
{ "aot": true, "target": "x86_64-linux" }
```
Parse via `std.json.parseFromSlice(BuildConfig, …)` into
`struct { aot: bool = false, target: ?[]const u8 = null }`. Field defaults cover
omitted keys; `std.json`'s default `ignore_unknown_fields = false` makes an
**unknown key a loud `error.UnknownField`** (surfaced as a runner failure, never a
silent ignore — CLAUDE.md no-silent-default rule). Extensible: future `"cpu"`,
`"link"`, `"cwd"` are just new optional struct fields, no new sidecar file and no
custom parser.
**What the directives do:**
1. **`target = <triple|shorthand>`** threads `--target <value>` into every `sx`
invocation for that example (`run` / `build` / `ir``--target` is a global
flag, confirmed [main.zig:39](../src/main.zig#L39)), AND **host-match selects
the mode.** The runner parses the leading `arch` + `os` tokens of the resolved
triple and compares them to `@import("builtin").target` (normalizing
`arm64``aarch64`):
- **match** → *execute* exactly as today (`sx run`, or `aot` build+exec) with
the target threaded, plus the `.ir` diff if an `.ir` snapshot exists. ⇒ an
x86_64 example gives **real end-to-end coverage on an x86_64 CI runner**.
- **mismatch** → **ir-only**: run *only* `sx ir <file> --target <t>`; assert
`.exit` (the ir command's exit), `.ir` (normalized stdout), and `.stderr`
(diagnostics, normally empty). Do **not** run/build/exec; do **not** assert
`.stdout`. An `.ir` snapshot is **required** in ir-only mode — its absence is
a loud runner failure ("arch-pinned <name>: ir-only mode requires an .ir
snapshot"), never a silent pass. Robust even if `sx ir` treats `--target` as
a partial no-op: the `inline_asm` op carries the template + constraint string
verbatim, so the IR snapshot still locks the exact thing §II.11 flags as
silently-miscompiling (the constraint assembler + template rewrite).
2. **`aot`** is the existing JIT-vs-build+exec switch, just relocated from the
standalone `.aot` marker into `.build`.
**Negative compile-error examples need NO `.build`.** `…-missing-volatile`
(no-output-without-`volatile`) is a Sema diagnostic raised before codegen/JIT, so
plain `sx run` reports it identically on any host — it stays a normal example with
no config file.
**update-goldens interaction:** in ir-only mode, `-Dupdate-goldens` writes `.exit`
(ir exit) + `.ir` (+ `.stderr` if non-empty) and skips `.stdout`. Execute mode
(incl. `aot`) is unchanged. `.build` is hand-authored — update-goldens never
writes it.
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | Add `BuildConfig` + `std.json` parse of `expected/<name>.build` (unknown-key ⇒ `error.UnknownField`); **migrate** the 2 existing `.aot` markers → `.build` (content `{ "aot": true }`) and delete them; thread `target`'s `--target` into the spawned argv; add `hostMatchesTarget(value) bool` (arch+os token parse, `arm64``aarch64`) gating the **execute** path. Lock with `examples/16xx-platform-target-host.sx` (trivial `main`) + a `.build` `{ "target": "<host arch triple>" }` (still runs+passes) and unit `test`s for the JSON parse + `hostMatchesTarget`. | `src/corpus_run.test.zig`, `examples/expected/1226-*.{aot→build}`, `…/1227-*`, + fixture |
| 0.1 | lock | Implement the **mismatch ⇒ ir-only** branch (skip run/build/exec; assert `.exit`+`.ir`+`.stderr` from `sx ir --target`; require `.ir`). Lock with `examples/16xx-platform-target-cross.sx` (asm-free `() -> i64 { return 0; }`) + `.build` `{ "target": "x86_64-linux" }` + a checked-in `.ir` snapshot — exercises ir-only on the arm64 host. | `src/corpus_run.test.zig` + fixture |
| 0.2 | docs | Update CLAUDE.md §"Test layout"/§"Testing" to document `.build` (format + `aot`/`target` keys) replacing the standalone `.aot` marker prose (lines ~435, ~492). | `CLAUDE.md` |
Both 0.0 and 0.1 are **lock** commits: the runner change and the fixture that
exercises it land together and pass the moment they land (the mechanism works
immediately — nothing is left red), which is the cadence rule's "lock in current
behavior" flavor, not a feature red→green. No asm lowering is gated on either.
**Phase 0 verification:** `zig build test` green; deliberately corrupt the
cross-target `.ir` fixture and confirm the runner reports an IR mismatch (proves
ir-only actually asserts, isn't a no-op); delete it and confirm the
"requires an .ir snapshot" failure fires.
**Estimated runner delta:** ~7090 lines (sidecar read + `--target` argv threading
+ `hostMatchesTarget` + the ir-only branch + update-mode tweak). Within the
"no step > ~500 new lines" rule; well under the read budget.
## Phase A — keyword + AST + parser (parses; no codegen)
| Step | Commit | What | Files |
|---|---|---|---|
| A.0 | lock | add `kw_asm` keyword + map entry; unit lex test `asm → kw_asm` | `src/token.zig`, `src/lexer.zig` + `.test.zig` |
| A.1 | xfail | parse `asm { … }``AsmExpr`/`AsmOperand` in `parsePrimary`; pin an AST/`sx ir` parse snapshot; lowering still `bailDetail("inline asm codegen unimplemented")` | `src/ast.zig` (:85 union arm, :721 structs), `src/parser.zig` (parsePrimary), `src/ir/interp.zig` |
| A.2 | green | parse-shape snapshot lands green; the unimplemented bail is loud + named | — |
## Phase B — sema / typing
| Step | Commit | What | Files |
|---|---|---|---|
| B.0 | xfail | result-type rule (0→`void` / 1→`T` / N→named-or-positional tuple) + checklist (no-output⇒`volatile`, layout, comptime-string template) — pin error messages | `src/ir/expr_typer.zig` |
| B.1 | green | typing + diagnostics implemented; `.unresolved` sentinel on failure (no silent default) | `src/ir/expr_typer.zig`, `src/ir/semantic_diagnostics.zig` |
## Phase C — IR op + lowering
| Step | Commit | What | Files |
|---|---|---|---|
| C.0 | lock | add `inline_asm: InlineAsm` to `Op` + `AsmOperand` (role/name/constraint/operand) + interp `bailDetail` arm; unit tests for the IR shape | `src/ir/inst.zig` (:80), `src/ir/interp.zig` |
| C.1 | xfail→green | `lowerAsmExpr` in `lowerExpr` dispatch — interns template/constraints/clobber-names, lowers input `Ref`s, sets result `TypeId` | `src/ir/lower/expr.zig` |
## Phase D — LLVM emit (single value-output; the core)
| Step | Commit | What | Files |
|---|---|---|---|
| D.0 | xfail | `examples/16xx-platform-asm-syscall-write.sx` + `…-register-read.sx` + `…-no-output-volatile.sx` + `…-missing-volatile.sx` (expected compile error) — all red | examples + `expected/` markers |
| D.1 | green | `emitInlineAsm`: **port `FuncGen.airAssembly`** — constraint-string assembler (outputs `=`/`+`, inputs, `clobbers(.name)``~{name}`), `%[name]``${N}` / `%%` / `%=` template rewriter, `LLVMGetInlineAsm`+`LLVMBuildCall2`, `sideeffect=volatile`, AT&T dialect | `src/ir/emit_llvm.zig` (emitInst dispatch + handler) |
| D.2 | green | lock the template-rewrite + constraint string via an `expected/*.ir` snapshot on `…-template-subst.sx` | examples |
**Phase D verification:** `zig build test`; the syscall example runs on
`x86_64-linux`; IR snapshot matches the design doc's worked `sys_write` lowering.
## Phase E — multi-return tuples + `clobbers(.…)`
| Step | Commit | What | Files |
|---|---|---|---|
| E.0 | xfail | `…-asm-multi-return.sx` (`divmod``(quot,rem)`, `cpuid`→4-tuple) red | examples |
| E.1 | green | N `out_value` → LLVM struct return + `extractvalue i` → sx tuple (named when operands named); `clobbers(.name)` dot-name lowering finalized | `src/ir/emit_llvm.zig`, `src/ir/lower/expr.zig` |
## Phase F — global asm (needs EXTERN-EXPORT Phase 2)
| Step | Commit | What | Files |
|---|---|---|---|
| F.0 | xfail | top-level `asm { … }` decl parsed (reject operands/`volatile`); `…-asm-global.sx` (defines a symbol, imported via `extern`) red | `src/parser.zig`, `src/ast.zig` |
| F.1 | green | lower `asm_global``c.LLVMAppendModuleInlineAsm`; comptime-call guard (dlsym-miss is loud); blocks concatenate in source order | `src/ir/lower/decl.zig`, `src/ir/emit_llvm.zig`, `src/ir/interp.zig` |
## Phase G — later (own steps when scheduled)
`-> @place` write-through + read-write (`"+r" -> @place`) + indirect-memory
(`"=*m"`) outputs · `%=` unique-id · output-to-const rejection · Intel-dialect
opt-in · naked functions (`callconv(.naked)`, coordinate with EXTERN-EXPORT).
## Open decisions (design doc §II.10)
Dialect (AT&T-only v1, recommended) · `volatile` contextual-keyword (recommended)
· brace separator comma (recommended) · `clobbers(.name)` dot-name sugar now →
checked per-arch `Clobber` enum later (Phase 4 of the design doc).
## End-to-end verification (per phase)
`zig build && zig build test`; for arch-pinned examples confirm they run on a
matching host or assert on `sx ir`/`.s` snapshots. After intentional output
changes only: `zig build test -Dupdate-goldens`, then review the diff.

225
current/PLAN-ATOMICS.md Normal file
View File

@@ -0,0 +1,225 @@
# PLAN-ATOMICS — Stream A (atomics lowering)
> **STATUS: ✅ COMPLETE (feature-complete).** All phases A.0 → A.3 landed + green.
> Surface shipped: `Atomic($T)` `load`/`store`/`fetch_add`/`sub`/`and`/`or`/`xor`/`min`/`max`/
> `swap`/`compare_exchange`/`compare_exchange_weak` (all comptime `$o: Ordering`) + free
> `fence(.ordering)`. IR ops `atomic_load`/`store`/`rmw`/`cmpxchg`/`fence`. Both LLVM emit
> AND the comptime VM implemented (verified to agree). Enabled by net-new comptime value
> params (enum/tagged_union/generic-struct methods — 3 commits). Corpus `17xx` (1700-1704) +
> `11xx` diagnostics (1130/1131/1186/1187). Commits: 22af404, 64c7db5, 8144a88, acf3183,
> 718f27e, 0531164, 68ed732, dca396e, 79895be, fca4304, b65544a (+ comptime-param 3c4305f,
> d7a6857, d95ba0a). **Unblocks Stream B2 channels + Stream C parallel schedulers.**
>
> Deferred (documented, NOT legacy — intentional scope): RMW/CAS/swap are integer-only
> (float fadd / pointer atomics out of scope); fence/orderings explicit (no defaults — the
> comptime-default-on-generic-method gap is orthogonal). Asm-level arch divergence +
> weak-memory *semantics* remain OUT of corpus scope (Stream-C stress harness).
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream A + the design-of-record
[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §3 (N1)
+ §4.6 (locked surface). Progress in [CHECKPOINT-ATOMICS.md](CHECKPOINT-ATOMICS.md).
**Goal:** net-new LLVM atomic codegen. Surface = a pure-sx `Atomic($T)` generic struct +
an `Ordering` enum (ordinary sx), with the actual atomic operations recognized as
`#builtin` intrinsics at lower-time and emitted as new IR ops. This is **100% net-new**
no atomics scaffolding exists (the only `lower.zig` "ordering" is *comparison* ordering
`< <= >=`, unrelated to memory ordering — do not mistake it for groundwork).
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `17xx` atomics.
---
## Design (grounded against the tree)
### Representation — minimal compiler surface
- **`Ordering`** is an ordinary sx enum, zero compiler coupling:
```sx
Ordering :: enum { relaxed; acquire; release; acq_rel; seq_cst; } // tags 0..4
```
- **`Atomic($T)`** is an ordinary sx **generic struct** (mirrors `List :: struct ($T: Type)`
at [list.sx:5](../library/modules/std/list.sx#L5)), a transparent 1-field wrapper —
atomicity is a property of the *operation*, not the storage, so `Atomic(i64)` has the
exact layout/size/align of `i64`. NO new IR *type*, NO type-system coupling:
```sx
Atomic :: struct ($T: Type) {
value: T;
init :: (v: T) -> Atomic(T) { return .{ value = v }; }
load :: (self: *Atomic(T), o: Ordering) -> T { return atomic_load(T, @self.value, o); }
store :: (self: *Atomic(T), v: T, o: Ordering) { atomic_store(T, @self.value, v, o); }
}
```
- The **operations** are `#builtin` intrinsic free functions, recognized by name at
lower-time (the established pattern — `size_of`/`type_info` in
[`tryLowerReflectionCall`](../src/ir/lower/call.zig#L1672), recognized BEFORE arg lowering):
```sx
atomic_load :: ($T: Type, ptr: *T, o: Ordering) -> T #builtin;
atomic_store :: ($T: Type, ptr: *T, v: T, o: Ordering) #builtin;
```
Explicit `$T` first arg follows the `size_of($T)` / `field_name($T, idx)` mixed
type+value precedent (lowest-risk; the reflection path already resolves type args).
### Ordering is compile-time-only by construction — and that forces a capability gap
LLVM atomic ordering is an **instruction attribute**, not a runtime operand, so the
ordering MUST be known at emit time. The lower-time handler reads the ordering arg's
variant name statically (it must be a **constant enum literal** `.seq_cst`) and bakes it
into the IR op as a Zig enum field (`AtomicOrdering`). A non-literal ordering is a **loud
diagnostic**, never a silent default (REJECTED-PATTERNS).
**Discovered gap (grounded):** a generic `Atomic(T)` method `load(self, o: Ordering)` would
forward `o` — a *runtime parameter* — to the intrinsic, where it is NOT a literal. And
**comptime enum value params don't exist** (`$o: Ordering` → `o` is "unresolved" in the
body; `resolveValueParamArg` folds integer constraints only). A runtime dispatch hack
(`if o == { case .acquire: atomic_load(…, .acquire) … }`) also fails: `load` with a
`release`/`acq_rel` ordering is *invalid LLVM*, so the arms can't be uniform. Therefore the
**full ordering surface is blocked on a net-new capability** (comptime-constant ordering
propagation — either comptime enum value params, or compiler-recognized `Atomic` method
calls). That capability is its **own step (A.0.5)**, sequenced before ordering-bearing ops.
### sx tag → LLVM ordering is EXPLICIT (non-contiguous!)
LLVM's `LLVMAtomicOrdering` is **not** 0..4: `Monotonic=2, Acquire=4, Release=5,
AcquireRelease=6, SequentiallyConsistent=7` ([Core.h:338-354]). The sx `Ordering` tags
(relaxed=0…seq_cst=4) map via an explicit `switch`, never an identity cast:
`relaxed→Monotonic, acquire→Acquire, release→Release, acq_rel→AcquireRelease,
seq_cst→SequentiallyConsistent`.
### LLVM-C API (verified present in `llvm-c/Core.h`, no new extern decls needed)
- Atomic load = `LLVMBuildLoad2` + `LLVMSetOrdering(v, ord)` + `LLVMSetAlignment(v, size)`
(**alignment is mandatory** on atomic load/store — LLVM verifier rejects atomics without
it). There is **no** `LLVMBuildAtomicLoad`/`Store` (the Explore agent was wrong).
- Atomic store = `LLVMBuildStore` + `LLVMSetOrdering` + `LLVMSetAlignment`.
- (Later) `LLVMBuildAtomicRMW(B, op, ptr, val, ord, singleThread)`,
`LLVMBuildAtomicCmpXchg(B, ptr, cmp, new, succOrd, failOrd, singleThread)`,
`LLVMBuildFence(B, ord, singleThread, name)`, `LLVMSetWeak`.
- `singleThread = 0` (multi-thread / cross-thread ordering). Atomic-eligible `T` =
integer / pointer / float of size 1·2·4·8(·16). **Reject non-scalar / bad-size `T`
loudly** (diagnostic), do not silently emit.
### Comptime VM treats atomics as ordinary load/store
Comptime is single-threaded, so seq_cst is trivially satisfied — the
[`comptime_vm`](../src/ir/comptime_vm.zig#L659) arms for `atomic_load`/`atomic_store`
reuse the ordinary `load`/`store` paths (correct, NOT a bail). `sx run` JITs via LLVM so
runtime atomics execute the real ops; the VM arm only matters for `#run`/const-init.
### Files the new IR op variants force (exhaustive switches)
`atomic_load` / `atomic_store` variants must be handled in every `Op` switch or the Zig
build fails (this is the desired tripwire):
- [inst.zig:159](../src/ir/inst.zig#L159) — add `atomic_load: AtomicLoad`, `atomic_store: AtomicStore` + the structs (mirror `Store` at [inst.zig:286](../src/ir/inst.zig#L286)).
- [lower/call.zig:1672](../src/ir/lower/call.zig#L1672) — recognize the intrinsics, emit the ops (new `tryLowerAtomicIntrinsic`, called alongside `tryLowerReflectionCall` at [call.zig:80](../src/ir/lower/call.zig#L80)).
- [print.zig:231](../src/ir/print.zig#L231) — print arms (sx-IR / `ir-dump`).
- [emit_llvm.zig:1566](../src/ir/emit_llvm.zig#L1566) — dispatch arms → ops.zig.
- [backend/llvm/ops.zig:325](../src/backend/llvm/ops.zig#L325) — `emitAtomicLoad`/`emitAtomicStore` (mirror `emitLoad`/`emitStore`).
- [comptime_vm.zig:659](../src/ir/comptime_vm.zig#L659) — arms reusing load/store.
- Any other `.op` switch the Zig compiler flags (module.zig / program_index.zig) — let the build tell you.
### Test snapshots — the arch-`.ir` requirement is a MISCONCEPTION for atomics
`sx ir` = [`emitIR`](../src/main.zig#L210), which emits **LLVM IR** (respects `--target`);
`sx ir-dump` is the sx-IR printer. At the **LLVM-IR level**, `load atomic i64, ptr %x
seq_cst, align 8` is **arch-invariant** — identical text for x86_64 and aarch64. The
x86-`lock`/MOV vs aarch64-`ldar`/`stlr` divergence happens only at *instruction selection*
(`sx asm`), which the corpus does **not** snapshot. So:
- **A single host `.ir` snapshot** proves the achievable gate (the `load atomic <ordering>`
keyword + correct ordering + alignment emitted). PLAN-POST §A / design §10.3's
"arch-gated x86_64 + aarch64 `.ir`" would capture **byte-identical** files — drop it.
- Optionally add ONE cross-arch ir-only example (`.build {"target":"x86_64-linux"}` on an
aarch64 host) purely as a **cross-target-emission-doesn't-crash** smoke — note in its
header that the IR body is identical to host.
- **State loudly (out of snapshot scope, parallel to the ordering-semantics caveat):**
asm-level arch lowering AND weak-memory ordering *semantics* are NOT proven by `.ir`;
those need the Stream-C stress harness, not the corpus.
---
## Phases
### A.0 — `Atomic($T)` + `Ordering` + **`seq_cst`-only** `load`/`store` ← START HERE
**Scope (descoped per the discovered gap above):** ship the net-new atomic load/store
codegen with a **`seq_cst` literal baked in the method bodies** — `load(self) -> T` /
`store(self, v)` (NO ordering param yet). The intrinsic still carries the full
`AtomicOrdering` field (always `.seq_cst` here); the recognizer + emit handle all five
orderings already, so A.0.5 only has to plumb the *constant* through. Explicit orderings
(`a.load(.acquire)`) land in A.0.5. seq_cst-only is correct (conservative-strongest), not a
silent fallback.
Two-commit cadence (lock-to-bail → green):
- **A.0a (lock)** — land the lib + IR plumbing with emit deliberately bailing:
1. New `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (value +
`init`/`load`/`store`), `atomic_load`/`atomic_store` `#builtin` decls. **Opt-in import
(`#import "modules/std/atomic.sx"`), NOT carried by the universal `std.sx` facade** —
mirrors `trace`. Rationale (grounded): adding the concrete `Ordering` enum to the
universal prelude registers it into EVERY program's global type table, growing
`@__sx_type_is_unsigned` (378→380) and shifting all string-global numbering → churned
37 unrelated `.ir` snapshots + bloats every binary. Atomics is a deliberate concurrency
capability, so consumers import it explicitly.
2. Add IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` + the two op structs
(inst.zig); print arms; comptime_vm arms (reuse load/store); lower recognition
(`tryLowerAtomicIntrinsic`) incl. the const-ordering-literal guard + non-scalar-`T`
reject.
3. emit_llvm/ops.zig arms **bail loudly** for now: `emitAtomicLoad`/`Store` call the
emitter's bail-with-diagnostic path ("atomic load/store LLVM emission not yet
implemented") so the Zig build is exhaustive but the example is red-by-diagnostic.
4. Add `examples/1700-atomics-load-store.sx` (construct `Atomic(i64).init`, `store`,
`load`, `print`). Seed marker; capture snapshot = the emit-bail diagnostic (nonzero
exit). `zig build && zig build test` green (matches the locked bail snapshot). Commit.
- **A.0b (green)** — replace the emit bail with real emission:
`LLVMBuildLoad2`+`LLVMSetOrdering`+`LLVMSetAlignment` / `LLVMBuildStore`+`LLVMSetOrdering`
+`LLVMSetAlignment`, ordering via the explicit sx-tag→LLVM `switch`. Regen `1700` to
success output + capture its host `.ir` (asserts `load atomic`/`store atomic` + ordering).
Add a unit test in `emit_llvm.test.zig` (correct op + ordering + alignment emission).
Review the diff (no stray error text). Commit.
### A.0.5 — comptime-constant ordering propagation (the capability gap)
Enable `a.load(.acquire)` etc. — i.e. an `Ordering` that reaches the intrinsic as a
compile-time constant through a method. Two candidate designs (pick at pickup):
- **(a) comptime enum value params** — make `$o: Ordering` resolve in the body to its
variant tag (extend `comptime_value_bindings`/the typer beyond integers). General,
reusable; larger typer change.
- **(b) compiler-recognized `Atomic` methods** — special-case `Atomic(T).load/store/…`
calls (read the literal ordering arg at the method call site), bounded coupling to the
std `Atomic` type (cf. how `Vector` is special-cased). Smaller; less general.
Also enforce per-op ordering validity (load: relaxed/acquire/seq_cst; store:
relaxed/release/seq_cst; CAS's dual orderings) as **compile errors**, which is exactly what
the constant-ordering path buys. Retrofit the ordering param onto `load`/`store` here.
### A.1 — RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`)
One IR op `atomic_rmw` carrying an `RmwKind` (maps to `LLVMAtomicRMWBinOp*`). Signed vs
unsigned min/max picks `Max/Min` vs `UMax/UMin` from `T`'s signedness. Same lock→green
cadence; `17xx` examples.
### A.2 — `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**)
`atomic_cmpxchg` op (ptr, cmp, new, success_ord, failure_ord, weak). LLVM `cmpxchg`
returns `{T, i1}`; lower to `?T` where **null = success** (extract the i1, invert).
**Validate the two orderings in the compiler** (design §4.6): failure ordering may not be
`release`/`acq_rel` nor stronger than success — loud diagnostic. `_weak` sets `LLVMSetWeak`.
### A.3 — `swap` + `fence(.ordering)`
`swap` = `atomic_rmw` with `Xchg` kind (folds into A.1's op). `fence` = a new `atomic_fence`
op (ordering only) → `LLVMBuildFence`. `17xx` examples.
---
## Gates (per the corrected snapshot story)
- **unit** `emit_llvm.test.zig`: each op emits the right LLVM builder + ordering + alignment.
- **corpus** `17xx` single-thread deterministic (`sx run`, JIT executes real atomics).
- **host `.ir`** snapshot per op proves the keyword/ordering/alignment lowered.
- **OUT of snapshot scope, stated loudly:** asm-level arch divergence (`sx asm`) and
weak-memory ordering *semantics* — Stream-C stress harness territory, not the corpus.
## Kickoff prompt (A.0a — paste into a fresh session)
> Implement Stream A step A.0a (atomics lock commit) per `current/PLAN-ATOMICS.md`. Verify
> `zig build && zig build test` is green first. Then: (1) create
> `library/modules/std/atomic.sx` with the `Ordering` enum, `Atomic($T)` struct, and
> `atomic_load`/`atomic_store` `#builtin` decls; wire into `library/modules/std.sx`'s tail.
> (2) Add the `atomic_load`/`atomic_store` IR ops + `AtomicOrdering` + op structs in
> `src/ir/inst.zig`; handle them in every exhaustive `Op` switch the Zig build flags
> (print.zig, comptime_vm.zig reuse load/store, emit_llvm dispatch). (3) Add
> `tryLowerAtomicIntrinsic` in `src/ir/lower/call.zig` (recognize the two builtins, bake the
> const ordering literal into the op, loud-reject non-literal ordering AND non-scalar/bad-size
> `T`). (4) Make `emitAtomicLoad`/`emitAtomicStore` in `src/backend/llvm/ops.zig` BAIL loudly
> ("not yet implemented") this commit. (5) Add `examples/1700-atomics-load-store.sx`, seed the
> marker, capture the bail diagnostic as the locked snapshot, confirm `zig build test` green,
> commit. STOP — A.0b (real emission) is the next step. Do NOT implement emission in the same
> commit that adds the example.

741
current/PLAN-COMPILER-VM.md Normal file
View File

@@ -0,0 +1,741 @@
# PLAN — Comptime Bytecode VM + comptime memory (then re-home the compiler-API on it)
> **Direction change (2026-06-17).** The comptime compiler-API stream pivots off the
> **byte-weld**. The weld (sx structs whose layout is validated to mirror the
> compiler's Zig types) + the **serialization / marshaling** bridge at the call
> boundary is the wrong direction — it bolts a parallel layout regime and hand-built
> byte-copies onto a comptime value model that fundamentally isn't bytes. We strip it
> and build the right foundation: a **bytecode VM over byte-addressable
> memory**, where comptime values ARE native bytes (like runtime). On that base the
> compiler-API needs no weld, no validation, no marshaling — the compiler's own types
> are read/built directly as memory and its functions take/return real pointers.
>
> Supersedes the build order in `design/comptime-compiler-api.md` (kept for history).
> This is the active plan for the stream. Branch: `reify`.
## Why
`src/ir/interp.zig` is a tree-walking interpreter over the SSA IR that represents
every value as a tagged `Value` union (`int`, `float`, `aggregate: []const Value`,
`type_tag`, `heap_ptr`, …). Two consequences:
1. **Slow.** Per-value boxing in a tagged union; per-op `switch` over `Inst`; an
aggregate is a heap `[]const Value`, walked element-by-element.
2. **Not native memory.** A struct value is `[]const Value` (tagged unions), NOT the
struct's bytes. So a comptime `@ptrCast(*StructInfo)` reads the `Value` union's
memory, not a `StructInfo` — which forced the whole weld+marshal detour.
Make comptime values **native bytes in byte-addressable memory** and both problems dissolve:
structs/arrays/slices are their bytes at natural layout (no weld), the compiler's own
records are directly addressable (no marshal), and a bytecode loop over comptime memory is
fast.
## End state
- Comptime execution = a **bytecode VM** over a **byte-addressable memory** (real
host-allocated bytes; layout is **target-aware** via the type table's sizes). Values
are bytes at addresses plus a scalar register file. No tagged `Value` union.
- The comptime compiler-API: the compiler **exposes its real types + functions** to
comptime sx. sx reads/builds them as native memory and calls compiler functions by
pointer. No `abi(.zig)` weld, no `validateStructLayout`, no `register_struct`
field-by-field marshaling — gone.
- `declare`/`define`/`type_info` and `#compiler`/`BuildOptions` ride this one
mechanism; the bespoke interp arms are deleted.
- **ONE evaluator at the end — non-negotiable.** The legacy tagged-`Value` interpreter
(`interp.zig`) is **DELETED**. We do NOT ship both permanently. "Dual-path"
(a compiler-API fn with both a legacy `compiler_lib` handler AND a VM-native impl) and
the emit-time legacy fallback are **transitional only** — scaffolding while the VM
reaches parity at BOTH comptime sites (emit time AND lowering time). The flag
`-Dcomptime-flat` is the swap mechanism; once the VM runs everywhere with parity, the
flag, the fallback, and `interp.zig` all go. Any "VM-only at emit, legacy at lowering"
split is a waypoint, never the destination.
## Principles (hold at every step)
- **Green at every step.** `zig build && zig build test` pass after each sub-step. The
existing tagged-`Value` interpreter stays the live evaluator until the VM reaches
corpus parity; swap behind a build flag, then delete the old path.
- **Target-aware, not host-baked.** Flat-memory layout uses the type table's target
sizes (`pointer_size`, `typeSizeBytes`/offsets), NEVER host `@sizeOf`. This is what
keeps cross-compilation correct (the JIT-comptime alternative could not).
- **Sandboxed.** Flat-memory accesses are bounds-checked; step/call-depth budgets
remain; an OOB / bad access traps to a build-gating diagnostic with a source span —
never a compiler-process crash.
- **No silent fallbacks** (per CLAUDE.md): an unhandled op / shape bails loudly with a
named reason, never a zero/default that looks like success.
## Phases
### Phase 0 — Strip the weld / serialize / marshal machinery
Delete the wrong-direction code so the VM builds on a clean base. Pure removal +
corpus rebaseline; suite green.
- `src/ir/compiler_lib.zig`: the reflection (`weldStruct` / `bound_types` /
`FieldLayout` / `BoundType`), the layout validation (`validateStructLayout` /
`LayoutMismatch` / `SxField`). Decide the fate of the `bound_fns` host-call registry
(`intern`/`text_of` handlers) — it is likely subsumed by the VM's compiler-call path
in Phase 3, but `intern`/`text_of` may survive as the first such calls.
- `src/ir/lower/nominal.zig`: `validateWeldedStruct` + `weldedFieldOrderStr` + the
`sd.abi == .zig` validation call in `registerStructDecl`.
- `src/ir/interp.zig`: the `compiler_welded` dispatch branch.
- `src/backend/llvm/ops.zig`: the `emitCall` comptime-only gate keyed on
`compiler_welded` (re-derive the comptime-only guard from a non-weld signal if still
needed).
- Corpus: retire / convert the weld examples + diagnostics — `0625`, `0627` (welded
struct), `1183`, `1186` (weld-layout diagnostics), `1184`/`1185` (welded-fn). Keep
`0626` (`intern`/`text_of` round-trip) only if it survives the new call path.
- **Keep (re-evaluate in Phase 3), independent of the weld semantics:** the
`#library "compiler"` decl, the `abi(.x)` annotation + `extern <lib>` syntax, and the
`callconv → abi` unification. These are surface syntax that may still serve the
compiler-API; only the *weld semantics* are stripped here.
**Verification:** `zig build test` green with the weld machinery gone; the surviving
syntax still parses (parser unit tests).
### Phase 1 — Flat-memory value model (still IR-walking, no bytecode yet)
Introduce comptime memory and move comptime values onto it, **decoupled from bytecode** so
the value-model change is isolated. Each sub-step ports one op group and keeps the
corpus green; the OLD tagged path stays behind a build flag (`-Dcomptime-flat`) until
all groups land, then the shim is deleted.
1. **Machine + scalars.** A comptime memory region (host `[]u8`) with a stack (frames) +
bump-allocated heap, and a scalar register file. Port `int`/`float`/`bool`/`undef`
and arithmetic/compare/branch. Aggregates still go through a compat shim to the old
representation.
2. **Aggregates.** Structs/arrays/tuples laid out in comptime memory at **target** layout;
port `struct_init` / `struct_get` / `array` / `index_gep` to read/write bytes at
computed offsets.
3. **Slices / strings.** `{ptr, len}` fat pointers in comptime memory.
4. **Optionals / enums / tagged unions.** Tag + payload bytes.
5. **Pointers.** `alloca` / `store` / `load` / GEP unified onto comptime addresses; retire
`slot_ptr` / `heap_ptr` / `byte_ptr` in favor of comptime addresses.
6. **Closures.** Fn id + captured env materialized in comptime memory.
7. **Extern / host calls.** A struct arg is already bytes → pass its address; this
removes most of `marshalExternArg`.
8. **Reflection / minting.** `declare` / `define` / `type_info` read comptime
values; type-table mutation copies escaping data into compiler-owned memory at the
boundary (lifetime), as today.
**Verification:** with `-Dcomptime-flat` the full corpus (currently 692) is byte-for-
byte identical to the tagged path; then make the VM the default and delete the shim.
### Phase 2 — Bytecode
Compile a comptime function's IR → a compact bytecode and execute the bytecode instead
of walking `Inst`. Pure encoding/speed; semantics identical to Phase 1. Land at least a
minimal register-bytecode loop (the stream's stated goal is a *bytecode* VM); a
fragment cache is optional follow-up.
**Verification:** corpus identical to Phase 1; comptime throughput measurably improved
on a heavy-comptime micro-benchmark.
### Phase 1.final — host wiring (the remaining integration)
The wiring ENTRY POINT exists: `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a
comptime function entirely on the VM and returns a legacy `Value`, or `null` to fall
back. Unit-tested (pure `6*7` → 42; unsupported → null). Remaining to actually route the
host through it:
1. **Panic→error hardening (prerequisite).** `Machine.readWord`/`writeWord`/`bytes`
currently `assert` (debug panic) on null/OOB. For arbitrary host functions to be
safe, make them return `error.OutOfBounds` so a malformed run BAILS (→ null → legacy)
instead of crashing the compiler. Ripples through `readField`/`writeField`/slice
helpers (add `try`).
2. **Implicit context.** Host comptime functions may have `has_implicit_ctx` (param 0 =
`*Context`); the legacy `run` materializes a default ctx. The VM `run` does not — so
either materialize it too, or only route `tryEval` at funcs without implicit ctx.
3. **Wire one site** behind a flag/env (`SX_COMPTIME_FLAT`, → `-Dcomptime-flat` later):
the const-init fold in `emit_llvm.zig` `emitGlobals` (`result = tryEval(...) orelse
interp.call(...)`). Default off → corpus unaffected.
4. **Parity + coverage.** Run the corpus with the flag ON; results must be byte-identical
to legacy. Measure how many comptime evals the VM already handles; the bail `detail`s
name what to port next (tagged-union payload / any / closures / builtins).
5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge)
until the VM is the default and the legacy path is deleted.
**Status (2026-06-17): steps 14 DONE; step 5 = the next session.**
- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return
`error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of
asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through
`readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and
every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a
null-deref function → `tryEval` returns `null` (legacy fallback), not a panic.
- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a
conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with
`has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the
right size/align allocated in comptime memory, its address passed as arg 0. The common
const body never reads the ctx; a body that USES the allocator loads a fn from it and
`call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed:
handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed)
empirically confirms no divergence. (A body that read+branched on a null allocator fn
could in principle diverge; none does — parity is the guard.)
- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat)
tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT`
(a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`).
Default OFF → corpus unaffected (688 green).
- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed);
manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace
facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`,
printing HANDLED / fallback+reason per init).
- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.**
Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity
stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx
materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer
optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the
destination = none/empty; unit-test regression); curated libc MEMORY builtins on comptime
memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped,
`free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed, target-aware,
result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32
storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single —
`readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had
written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added).
- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage
**27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF.
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context (not a zeroed one): the implicit-ctx param is an
opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying
the CAllocator thunk func-refs) into comptime memory via a new recursive `layoutConst`.
With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays
reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect`
(decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body
that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` →
`context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → the VM's native comptime `malloc`. Unlocked `0606` (string global via
the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func`
(memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/
`struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by
`alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to
`.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s
rejection diagnostic byte-identical). Unit tests added (global_get, func_ref +
call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk
bottoms out at libc `malloc`, and the VM can't use a host pointer with comptime
load/store, so comptime `malloc` must allocate from comptime memory. The default context
lets the allocator PROTOCOL run; native `malloc` is its final step.
- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage
**31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF.
- **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`.
- **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set)
→ word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)`
shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 |
span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push`
/ `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions —
linked into the compiler — so the return-trace buffer the host reads is populated
identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now.
- **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):**
`readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a
stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently
hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded;
gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests
pass via 1037/1038 in the corpus).
- **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):**
`intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm
global call (`1654`, never comptime-evaluable). Every other measured corpus const-init
is handled on the VM.
At this point the comptime VM handles essentially the entire real comptime corpus
(scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls +
recursion, the implicit context + allocator protocol, globals, failables + return
traces). Phase 2 (bytecode) and Phase 3 (compiler-API on comptime memory) are the forward
work; flipping the VM to default + deleting the legacy path awaits those.
- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second
comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run <expr>;`) now
routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields
`.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed
(`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy
re-run double-push them — both sites now `sx_trace_clear()` right before the legacy
fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All
comptime evaluation now routes through the VM-with-fallback (uniform).
- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM
gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init`
reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test
-Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity
gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var;
default-built does not). This is the prerequisite to eventually making the VM default +
deleting the legacy path (which still awaits Phase 2/3 + broader confidence).
- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.**
`invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is
the safety boundary) via `Vm.callCompilerFn` — natively on comptime memory, NO legacy
`Interpreter`: `intern(s: string) -> StringId` reads the string bytes from comptime memory and
`internString`s into the (const-cast) table (pool-only, never touches type layout, so the
VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back
into comptime memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback
is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688**
both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next
compiler functions (`find_type`, `register_struct`, the reflection readers) are added the
same way — comptime pointer in, handle/pointer out, no marshaling.
**Phase 3 progress (2026-06-18):**
- **(P3.1) First read-only reflection readers — `find_type` + `type_field_count` (DONE).**
Two more `compiler`-library fns bound the same way as the `intern`/`text_of` seed
(added to `compiler_lib.bound_fns` AND `Vm.callCompilerFn`, native on comptime memory, no
marshaling). A **type handle is a plain `u32` `TypeId`** (exactly like `StringId`), so
both calls keep the seed's clean scalar shape — handle in, scalar out:
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`) and
`type_field_count(t: TypeId) -> i64` (a new `TypeTable.memberCount` query — struct/union/
tagged-union fields, enum variants, array/vector length — that BOTH the legacy handler
and the VM call, so the two paths can't drift). Example `0628` chains
`intern → find_type → type_field_count` and a not-found lookup, both folded at `#run`,
both VM-HANDLED natively (no fallback). Parity **689/689** (gate ON and OFF); VM unit test
added.
- **Decision (resolves the plan's `find_type → ?Type` sketch):** `find_type` returns a
NON-optional `TypeId`, using the codebase's dedicated `unresolved` (0) sentinel for
not-found — NOT an `?Type`. Rationale: a `Type` value resolves to `.any`
(`type_resolver.zig`), which the comptime VM does not represent; and an optional
return can't cross the legacy↔VM eval boundary (`regToValue` bridges only
word/string/struct/tuple). `unresolved` is the project-blessed unmistakable "no type"
marker (see CLAUDE.md REJECTED PATTERNS — a dedicated sentinel is the required shape),
so the caller checks the handle against 0. This keeps the reader a clean scalar mirror
of `intern`/`text_of` and defers `.any`/optional plumbing to when it's actually needed.
- **(P3.2) Field-level reflection readers — `type_nominal_name` + `type_field_name` +
`type_field_type` (DONE).** Three more readers on the same `TypeId`-handle shape (each
backed by a new `TypeTable` query that BOTH the legacy handler and the VM call, so no
drift): `type_nominal_name(t: TypeId) -> StringId` (`nominalName` — a named type's own
name; loud-bail for unnamed types), `type_field_name(t: TypeId, idx: i64) -> StringId`
(`memberName` — struct/union/tagged-union field, enum variant, named-tuple element), and
`type_field_type(t: TypeId, idx: i64) -> TypeId` (`memberType` — struct/tuple/array/vector
member type). All loud-bail on out-of-range idx / no-member (no silent default). These are
the first MULTI-ARG compiler fns (the VM's `callCompilerFn` now reads arg 1 = idx); added
`Vm.argHandle`/`argTypeId` helpers (range-checked u32/TypeId arg reads). Naming uses the
`type_*` family so nothing collides with the std metatype builtins (`field_name`/`type_name`
exist in `core.sx`). Example `0629` reflects `Pair { lo: Point; hi: Point }` — reads each
field name and the nominal name of a field's type, all folded at `#run`, all VM-HANDLED
natively. Parity **690/690** (gate ON and OFF); VM unit test added.
- **(P3.2b) Kind + enum-value readers — `type_kind` + `type_field_value` (DONE).** The last
two read-only readers the metatype's `type_info(T)` needs, completing the READ side: a
comptime sx fn can now fully reflect a struct/enum/tagged-union/tuple into data with no
`#builtin`. `type_kind(t: TypeId) -> i64` (`TypeTable.kindCode` — a stable, compiler-owned
discriminant: 0 other · 1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array ·
7 vector · 8 error_set; TOTAL — never bails, an unnamed/non-aggregate type reads `other`)
and `type_field_value(t: TypeId, idx: i64) -> i64` (`TypeTable.memberValue` — an enum
variant's explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for
a non-enum / out-of-range idx). Example `0630` reflects `Color`/`WindowFlags`(flags)/`Point`.
Parity **691/691** (gate ON and OFF); VM unit test added.
- **READ side now complete:** `find_type` + `type_kind` + `type_field_count` +
`type_field_name` + `type_field_type` + `type_nominal_name` + `type_field_value` cover
everything `reflectTypeInfo` reads.
- **(P3.3) WRITE side — `declare_type` + `pointer_to` + ONE kind-branching `register_type` (DONE).**
The mutating side is a SINGLE `register_type(handle, kind, members)` that branches on `kind`
IN THE COMPILER (subsuming `define`'s `defineStruct`/`defineEnum`/`defineTuple`), plus
`declare_type(name) -> Type` (forward handle) and `pointer_to(t) -> Type` (build `*T`
references). They take/return real `Type` values (matching meta.sx's declare/define).
- **Timing decision (per the user):** mint LAZILY at LOWERING time (single pass, NOT a
pre-emit phase, NOT two-pass) — the existing `runComptimeTypeFunc` path. So the write
side is **legacy-only** (`compiler_lib` handlers); the VM isn't wired at lowering time, so
no VM mirror is needed (the read-side readers stay dual-path for emit-time reflection). A
non-generic `-> Type` builder is now flagged `is_comptime` (`decl.zig`) so its dead body
permits the welded calls (the comptime-only gate).
- **Graph support:** forward `declare_type` handles + `pointer_to` express a
mutually-recursive A↔B graph (`*A`, `*B`, B-by-value) before bodies are filled.
`register_type` is **idempotent** — re-filling a nominal slot (same module reached via two
import edges) re-mints identically instead of erroring (`nominalIdent` reads identity from
any nominal kind). `kind` codes match `type_kind`: 1 struct · 2 enum (actual `.@"enum"`) ·
3 tagged_union · 4 tuple.
- **Two bugs fixed en route** (issue 0142): (a) a fully payloadless comptime-minted enum
was minted as an all-void `tagged_union` → `verifySizes` panic; now mints a real
`.@"enum"` (both `register_type` kind 2 AND the metatype `defineEnum`). (b) bare
`EnumType.variant` qualified construction of a payloadless variant wasn't supported (failed
for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
- Examples: `0631` (graph + actual-enum + reflection), `0632` (make_enum all-void),
`0633`/`0634`/`0635` (namespaced / bare / multi-edge import of a minted type), `0187`
(qualified variant construction). Parity 697/697 (gate ON and OFF); unit tests added.
- **Next (P3.4):** re-express `declare`/`define`/`type_info` as sx over the read+write
compiler-API and DELETE the bespoke interp arms — needs the VM hardened against malformed
lowering-time IR first (the metatype runs at lowering time), so either harden + wire the VM
there, or migrate the metatype onto the legacy compiler-API calls first. Decide when reached.
Phase 2 (bytecode) is the orthogonal speed work.
### Phase 3 — Compiler-API on comptime memory (resume the stream — no weld)
With native-byte comptime values, re-home the compiler-API:
- **Expose the compiler's real types.** Register the actual `types.zig` records
(`StructInfo`, `EnumInfo`, `Field`, …) into the comptime type table under sx-visible
names, with their **real (host) layout** — the type IS the compiler's, so there is
nothing to validate or keep in sync. (This is the projection that *replaces* the
weld's reflection — owned by the compiler, not declared in sx.)
- **Expose the compiler's functions.** `register_struct`, `find_type`, `intern`,
`text_of`, and the reflection readers operate on comptime pointers / handles
directly (no marshaling — the bytes already ARE the record).
- **Re-express** `declare` / `define` / `type_info` as sx over these; delete the
bespoke interp arms (`defineStruct` / `defineEnum` / `defineTuple` / `reflectTypeInfo`);
migrate `examples/0622` (struct), `0619`/`0620`/`0623` (enum/tuple).
- **Migrate `BuildOptions`** off `#compiler` onto this mechanism; **delete `#compiler`**.
**Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over
the exposed compiler-API; full corpus green.
### Phase 4 — Retire the legacy interp (the ONE-evaluator end state)
The metatype CONSTRUCTION + REFLECTION surface is VM-native (steps 7/8 — `0614``0624`,
`0632` all HANDLED). This phase moves EVERYTHING ELSE off `interp.zig` and deletes it.
**What the legacy interp is still used for (audited 2026-06-18) — five roles:**
| Role | Wired to VM? | Site |
|------|--------------|------|
| **A. Comptime folds** (type-fn / `::` const-init / `#run`) | ✅ VM + legacy fallback | `comptime.zig:530`, `emit_llvm.zig:871`/`971` |
| **B. `#insert` string eval** | ❌ legacy-only (VM wiring reverted — 0737 malformed-IR crash) | `comptime.zig:634` |
| **C. Post-link bundler** (`platform.bundle` — Info.plist/codesign/process/fs) | ❌ legacy-only | `core.zig:invokeByFuncId` ← `main.zig:769` |
| **D. `#compiler` hooks** (`compiler_call` — BuildOptions/bundling) | ❌ legacy-only; `Value`-based ABI | `compiler_hooks.zig`, `interp.zig:1130` |
| **E. Bail diagnostics** (`Interpreter.last_bail_*` statics) | n/a | `main.zig:464` |
Shared substrate everything traffics in: the **`Value`** tagged union (the
`regToValue`/`valueToReg` bridge + the hooks + `core.zig`) and the **host-FFI bridge**
(`host_ffi.zig` + `interp.callExtern` — dlsym + cdecl trampolines for real libc).
**DECISION (2026-06-18, user): UNIFY.** The VM gains a host-FFI escape + real-pointer
translation and runs BOTH sandboxed comptime folds AND the unsandboxed post-link bundler.
`interp.zig` is fully deleted — true ONE evaluator, two modes (sandboxed / host-effects).
**Remaining comptime-fold gaps** (full corpus fallback inventory — 15 examples; 1179/1180
are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan artifact):
`box_any`/`unbox_any` (6), `out`/print (2), `global_addr` (1), trace frames (1),
`compiler_call` (2 — role D).
**Sub-phases (dependency order; each its own session, both gates 697/0 after each):**
- **4A — finish comptime ops (small, parity-guarded).** Drive the fold fallback list to
empty except `compiler_call`:
- **4A.1** `box_any`/`unbox_any`. Word case = alloc 16B `{tag@0, value@8}`, tag =
`source_type.index()` (matches legacy comptime; note runtime `anyTag` normalizes
arbitrary-width ints), value via `writeField(source_type)` (so f32 etc. round-trip);
unbox = `readField(addr+8, target)`. Aggregate-Any payload needs the runtime
pointer-in-value-slot shape (`coerceToI64` alloca+ptrtoint) — implement or bail loudly.
- **4A.2** `out`/print → add a VM output buffer; flush through the same path as
`core.flushInterpOutput`.
- **4A.3** `global_addr` (address-of a global in comptime memory).
- **4A.4** trace frames (`sx_trace_*` / `interp_print_frames`).
- **4B — VM-native diagnostics (role E). MUST land before deleting legacy.** Today a VM
bail silently falls back; with legacy gone the VM bail IS the user-facing build-gating
diagnostic. Surface the VM's `detail`/span/file into what `main.zig` renders; turn
1179/1180-style bails into proper diagnostics. No diagnostic may regress.
- **4C — `#insert` on the VM (role B).** Re-wire `evalComptimeString` through `tryEval`;
the lowering-time-IR hardening that forced the 0737 revert is already in place. Verify
the `#insert` corpus parity.
- **4D — host FFI on the VM (role D substrate). DONE.** Solved by a better allocator, not a
pin/tag scheme: the comptime memory is now an **arena** of stable host allocations and `Addr`
IS a real host pointer (`4D.0`, `625ba0f`), so a comptime pointer and an FFI-returned host
pointer are the same value — no translation, no realloc hazard. `Vm.callHostExtern`
(`4D.1`, `e7a8708`) dispatches ANY extern via `host_ffi` dlsym + trampolines (args/returns pass
untouched); `4D.2` (`6a7f690`) adds slice/string args (→ NUL-term `char*`) + float guards.
Examples 0636/0637. **(Superseded sub-note:** the earlier "pin the buffer / comptime↔host translate"
hazard is moot — the arena never moves an allocation.)
- **`#compiler` / `compiler_call` — DELETED, replaced by the `abi(.compiler)` ABI (decision 2026-06-18,
REVISED from the earlier `abi(.zig) extern compiler` shape).** A function is *compiler-domain* — it runs in
the comptime evaluator (VM/interp), NEVER in the shipped binary — because its **ABI says so**: `abi(.compiler)`.
No `extern <lib>`, no fake `#library "compiler"`. One annotation covers BOTH roles: (a) the **compiler-API
surface** (`intern`/`find_type`/`build_options`/`set_post_link_callback`/… — bodiless decls whose Zig/VM
handler is the impl, on `compiler_lib`'s export list, dispatched by `Vm.callCompilerFn`); (b) **user
compiler-domain functions** like post-link callbacks (`bundle_main` — BODIED `abi(.compiler)`, lowered for VM
eval but emit-skipped). The `#compiler` struct attribute + the `compiler_call` IR op + the `Value`-based hook
`Registry` (`compiler_hooks.zig`) all **go away**. **Why this is cleaner than the welded-fn approach:** the
former runtime-call enforcement blocker (a `build_options()` call inside an LLVM-emitted callback body) is
MOOT — a compiler-domain function is never emitted, so its compiler-API calls never reach `emitCall`.
**Staged build (each its own step, both gates green):**
- **S1+S2 — DONE (2026-06-18):** introduced `abi(.compiler)`, REMOVED the `.zig` ABI + `abi(.zig) extern
compiler` + `#library "compiler"` (clean cutover, no legacy); migrated all compiler-API examples. The
binding now keys off `fd.abi == .compiler` (`decl.zig` `weldedCompilerFn`); a bodiless `abi(.compiler)`
decl lowers extern-like (declared-not-defined) with no implicit ctx. **700/0 both gates.**
- **S3 — DONE (2026-06-18):** emit_llvm skips BODIED `abi(.compiler)` function bodies. Added an
`is_compiler_domain` flag to the IR `Function`; a bodied `abi(.compiler)` function LOWERS its body (for VM
eval) + is flagged `is_comptime` but is NOT emitted (Pass 2 skip; declared external-linkage so the empty
decl verifies). KEY fix: a call to a comptime-only callee (compiler-API `compiler_welded` OR
`is_compiler_domain`) inside a dead comptime body now emits `undef` instead of a real `call` (`ops.zig`
`emitCall`) — the old `compiler_call` did this; without it an AOT link leaves an undefined `_double`/`_intern`
reference (this also fixed a pre-existing untested AOT breakage of the bodiless compiler-API examples).
`fnIsBodilessCompiler` distinguishes the API surface (declare-only) from a compiler-domain callback (lowered,
emit-skipped). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` const,
absent from the binary, JIT+AOT). **701/0 both gates.**
- **S4 — callback-param propagation: OPTIONAL / DEFERRED (ergonomics only).** Verified 2026-06-18: an
`abi(.compiler)` function is TYPE-compatible with a plain `() -> R` param (the ABI marks the *function* —
`is_compiler_domain` — not its *type*, which stays `() -> R` CC-default). So a callback that needs to be
compiler-domain just declares itself `abi(.compiler)` (S3) and passes to a plain param fine; auto-propagation
from an `abi(.compiler)` PARAM type is a nicety, not a prerequisite for S5. Skipped for now.
- **S5a — DONE (2026-06-18):** the corpus-covered slice. `build_options` + `set_post_link_callback` →
free `abi(.compiler)` functions (VM `callCompilerFn` arms + legacy `compiler_lib` handlers); **`BuildConfig`
threaded into the VM** via a `tryEval` param (the same one `main.zig` forwards — shared with 4E). `build.sx`
extracts `set_post_link_callback` from the `struct #compiler` as a free `ufcs` fn; `bundle_main` + the
platform registrars (`configure`) are `abi(.compiler)`. 37 examples' `.ir` snapshots regen'd (benign:
declaration renumber + `@str` suffix shift — every example imports build.sx via the prelude). Strict
`compiler_call` bails 6→2; 0602/0603/1604/1611 HANDLED. **701/0 both gates.**
- **S5b/S5c (port the ~37 hooks) — SUPERSEDED 2026-06-18 by the sx-driven build pipeline (below).**
Porting each `BuildOptions` accessor to an `abi(.compiler)` function that delegates to a `compiler_hooks`
hook just re-encodes sx-level logic (string setters/getters, `is_macos` triple-matching, list appends) as
compiler hooks. The hooks need NOTHING from the compiler except the `BuildConfig` state. So instead of 37
hooks, **drive the whole build pipeline from sx** (the logical end of "bundling lives in sx"). S5a stays as
a green intermediate; the sx-build-pipeline replaces `build_options`/`set_post_link_callback`/the whole
`#compiler` surface wholesale.
### Phase 5 — sx-driven build pipeline (replaces the BuildOptions hooks; decision 2026-06-18, user)
**The build pipeline becomes an sx program.** `BuildConfig` is plain sx data (an ordinary struct, sx-owned
end-to-end — no `#compiler`, no hooks, no shared Zig state, no weld/offset access). The compiler shrinks to
a few `abi(.compiler)` PRIMITIVES that take **explicit args** (so nothing is shared by memory), and an sx
`build()` driver orchestrates configure → emit → link → bundle. **Chosen boundary: Option B** — the compiler
keeps the proven Zig linker as a primitive; sx owns config + orchestration + bundle. (Option A — sx shells
`cc`/`ld` itself — is a later refinement once the per-target link-line logic is ported to sx.)
**File split (user decision 2026-06-19):** the low-level compiler-API PRIMITIVES live in
`library/modules/compiler.sx` (the comptime `compiler` library — renamed from the interim `std/build.sx`); the
default `build` IMPLEMENTATION (`default_build` + the `on_build` slot + the sx `BuildConfig`) lives in
`library/modules/build.sx` alongside the existing `BuildOptions` DSL. So `compiler.sx` = primitives, `build.sx` =
orchestration/default impl. **Build-callback fallibility was DROPPED (user 2026-06-19):** the primitives + the
build callback are NOT `-> !` — a failed action (e.g. `link`) BAILS on the VM (hard build error). So the shapes
below shed their `-> !`.
Shape (build-callback fallibility dropped 2026-06-19):
```sx
// library/modules/compiler.sx (the comptime `compiler` library — PRIMITIVES)
emit_object :: () -> string abi(.compiler); // emitted .o path (query)
link :: (objects: List(string), output: string, libraries: List(string),
frameworks: List(string), flags: List(string), target: string) abi(.compiler); // void; bails on failure
c_object_paths :: () -> List(string) abi(.compiler); // metadata queries
link_libraries :: () -> List(string) abi(.compiler);
// library/modules/build.sx (the build DSL — DEFAULT IMPLEMENTATION + slot)
BuildConfig :: struct { output: string; target: string; flags: List(string);
frameworks: List(string); bundle_path: string; bundle_id: string; ... }
default_build :: (config: BuildConfig) abi(.compiler) { // the default pipeline (void)
obj := emit_object(); objs := c_object_paths(); objs.append(obj);
link(objs, config.output, link_libraries(), config.frameworks, config.flags, config.target);
if config.bundle_path.len > 0 { bundle_app(config); } } // bundle_app = today's sx bundler
on_build : (BuildConfig) abi(.compiler) = default_build; // the override slot
// user overrides: build :: (config: BuildConfig) abi(.compiler) { ... } #run on_build = build;
```
The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfig` → read `on_build` → invoke
`on_build(config)` on the VM; a `raise` fails the build. Plain `sx run` fires none of it.
**Steps (each its own green step; depends on 4E first):**
- **P5.1 — 4E prereq — DONE (2026-06-19).** `core.invokeByFuncId` routes the post-link callback through the
**VM** (`comptime_vm.tryEval`), NO fallback (a side-effecting callback can't double-execute): a bail is a hard
build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). `BuildConfig` +
`import_sources` threaded in; `flushInterpOutput` deleted (VM `out` writes direct via host-FFI). Smoke test
`examples/1661-platform-post-link-vm-list` (AOT): a post-link callback GROWS a `List` (0141 — works on the VM,
bails on legacy with `struct_get`), so the build succeeds (exit 0) only via the VM. Non-empty callback `args`
rejected loudly (the `on_build(config)` arg-marshaling entry is P5.3). **702/0 both gates.**
- **P5.2 — primitives.** Split: the read-only **metadata queries are DONE (2026-06-19)** — `c_object_paths() ->
List(string)` + `link_libraries() -> List(string)` as `abi(.compiler)` fns (stdlib `library/modules/compiler.sx`),
serviced by `comptime_vm.callCompilerFn` over `BuildConfig` fields `main.zig` forwards; new VM `makeStringList`
builds the `List(string)` in comptime memory from the call's result type (`ins.ty` now threaded through
`invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both
gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits
the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all
three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER
DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard
build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks`
vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM
readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test
`1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output —
the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0.
- **P5.3 — `on_build` registrar — DONE (2026-06-19).** `on_build(cb)` registers the build callback
(`cb: (opt: BuildOptions) -> bool abi(.compiler)`); the compiler force-lowers + auto-invokes the well-known
`default_pipeline` when no override. (Implemented as a registrar, not an assignable slot — the opaque
`BuildOptions` handle is one word, so arg-passing needs no struct marshaling.)
- **P5.4 core — DONE (2026-06-19).** `default_pipeline` in `build.sx` drives the whole build; NO Zig
auto-emit/auto-link; `emit_object`/`link` are sx-called actions via the `BuildHooks` vtable;
`set_post_link_callback` deleted (all callers on `on_build`). Build-path auto-imports `modules/build.sx`.
703/0 both gates.
### THE FINAL DIRECTION (user, 2026-06-19): FULL MIGRATION — NO LEGACY LEFT.
**Decision: DROP gate-OFF entirely.** The VM becomes the SOLE comptime evaluator; `-Dcomptime-flat` is made
permanent then removed; `interp.zig` (the legacy tagged-`Value` `Interpreter`) is DELETED. There is no
dual-path, no legacy `compiler_lib` handler, no `regToValue`/`valueToReg` bridge, no VM→legacy fallback. We
migrate the BuildOptions surface DIRECTLY to VM-native `abi(.compiler)` arms (no legacy handler — there is no
legacy to handle). **All bundling + code signing for EVERY target lives in the sx `default_pipeline`.**
- **P5.5 — DONE (2026-06-19).** The 35 `BuildOptions :: struct #compiler` methods migrated to VM-native
`abi(.compiler)`: `BuildOptions :: struct { }` (opaque null-sentinel handle) + 35 free
`ufcs (self: BuildOptions, …) abi(.compiler)` decls in `build.sx`, serviced by a new
`comptime_vm.callBuildOptionFn` arm off `callCompilerFn` — **NO legacy `compiler_lib` handler** (names
registered in `bound_fns` with a single bailing stub only so `weldedCompilerFn` accepts them). Setters dupe the
arg string into the PERSISTENT `Vm.gpa` (the Compilation allocator — threaded into both `tryEval` and
`runBuildCallback` — NOT the per-eval VM arena) and write/append to the threaded `BuildConfig`; string getters
return the field (or `""`); bool getters compute from the triple (`predIsMacOS`/…); count/index getters read the
`BuildConfig` slices. **Dispatch routing (Option B):** a `#run`/const-init entry that directly calls a
compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with NO legacy fallback regardless of the
`-Dcomptime-flat` gate → gate-OFF stays green without a legacy BuildOptions handler. 5 `platform/bundle.sx`
getter-calling helpers marked `abi(.compiler)` (comptime-only bundler code). 37 `.ir` regenerated (string-pool
churn; behavior-identical, verified `.ir`-only). **703/0 BOTH gates.** BuildOptions `compiler_call` bails GONE
(1609/1614/1615 strict-clean); 1616 now bails on `shr` — a SEPARATE unported bitwise/shift VM gap
(`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`), to port FIRST in P5.6 (1616 is unpinned + can't JIT-run on
macOS regardless). Also swept the outdated "flat memory" terminology → "comptime/byte-addressable" (the VM is
arena-backed, `Addr` = real host pointer; flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
- **P5.6 — ALL bundling + code signing in `default_pipeline` (every target).** `default_pipeline` (or a
`bundle()` it calls, in `platform/bundle.sx`) performs, after `link`, the full per-target bundle when
`bundle_path()` is set — branching on `is_macos`/`is_ios_device`/`is_ios_simulator`/`is_android`:
- **macOS `.app`** — `Contents/{MacOS,Resources,Frameworks}`, `Info.plist`, embed `-framework` dylibs +
`install_name_tool` fixups, `codesign` (ad-hoc or with `codesign_identity`).
- **iOS device `.app`** — device slice, embedded `.mobileprovision` (`provisioning_profile`), entitlements,
`codesign` with the real identity; **iOS simulator `.app`** — sim slice, no provisioning, ad-hoc sign.
- **Android `.apk`** — `AndroidManifest.xml` (or `manifest_path` override), asset tree (`add_asset_dir`),
`#jni_main` Java → `javac` → `d8` → `classes.dex`, `aapt2` package, `zipalign`, `apksigner` with the
debug/`keystore_path` keystore.
All of it runs on the VM via the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI (the existing
`platform/bundle.sx` logic, now reading the VM-native accessors instead of `#compiler` hooks). The compiler
keeps ONLY the linker as a primitive (Option B). Remove the `--bundle`/`post_link_module` Zig shim — bundling
is `default_pipeline`'s job; CLI flags feed `BuildConfig` and `default_pipeline` branches on it.
- **P5.7 — DELETE all legacy.** Remove the `#compiler` attribute (parse + lower), the `compiler_call` IR op
(`inst.zig` + every switch arm + the `interp.zig:1130` dispatch), `compiler_hooks.zig`
(`HookFn`/`Registry`/all hooks). Make `-Dcomptime-flat` permanent (VM always) and **delete `interp.zig`**
(`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`/`last_bail_*`); drop the
`regToValue`/`valueToReg` bridge and the VM→legacy fallback in `emit_llvm` (`#run`/const-init) and
`comptime.zig` (type-fn / `#insert`) — a VM bail is now ALWAYS a build-gating diagnostic (4B wiring), never a
fallback. `core.invokeByFuncId` is already VM-only. Re-express `define`/`make_enum` as sx over the
compiler-API. Land the 0141 repro as a corpus test. Reconcile 1654 (asm-global at comptime) to the VM wording.
- **P5.8 — real-project validation (integration).** Build `~/projects/m3te` and `~/projects/distribution` with
the new pipeline end-to-end (their real bundle/codesign/target configs) — these are the acceptance test that
`default_pipeline` covers all targets. Fix gaps surfaced there. Add dedicated bundle smoke tests (min `.app` +
`.apk`) to the corpus (the bundler still has no `zig build test` coverage — the stream's top risk).
**End state:** ONE evaluator (the VM); ZERO legacy; the entire build — emit, link, and all bundling + code
signing for macOS/iOS-device/iOS-sim/Android — is sx in `default_pipeline`, overridable via `#run on_build(...)`.
The compiler is: parse → IR → codegen → invoke `on_build`/`default_pipeline` on the VM (which calls back into
the linker primitive). `m3te` + `distribution` build clean.
**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ;
FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F.
**Top risks:** (1) the bundler has no corpus guard (4E needs dedicated tests); (2) deleting
`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API (`abi(.compiler)`) touches the
whole build/bundle path — stage it behind real bundle builds; (3) S3's emit-skip relies on DCE dropping the
unreferenced compiler-domain declaration — verify no stray runtime reference keeps it alive (link error).
## Open questions (resolve as reached, record decisions here)
- **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed
records are host-laid-out; user comptime types are target-laid-out. The comptime
model must carry both regimes (a per-type ABI tag on layout queries). Confirm the
boundary where a comptime pointer to a compiler record is handed to host Zig code
uses host layout.
- **Exposing compiler types to sx.** Mechanism for projecting `types.zig` records into
the comptime type table with real offsets (the non-weld replacement) — a registry the
compiler owns, keyed by sx-visible name → real Zig type's layout + a host-call ABI.
- **Bytecode shape.** IR-derived vs a fresh ISA; register vs stack; fragment caching.
- **Pointer escape / lifetime.** Flat-memory pointers stored into the persistent type
table must be copied into compiler-owned memory at the boundary (as today).
- **Old-path retirement.** Keep the tagged interpreter until Phase 1 parity, then
delete — confirm no non-comptime consumer depends on `Value`.
## File map (current → touched)
| Area | File | Phase |
|------|------|-------|
| Comptime evaluator | `src/ir/interp.zig` | 0 (strip weld dispatch), 12 (rebuild) |
| Weld registry | `src/ir/compiler_lib.zig` | 0 (strip), 3 (replace with type/fn exposure) |
| Weld validation | `src/ir/lower/nominal.zig` | 0 (strip `validateWeldedStruct`) |
| Comptime-only gate | `src/backend/llvm/ops.zig` | 0 (re-derive without weld signal) |
| Host-FFI marshalling | `src/ir/host_ffi.zig` | 1 (struct-by-pointer trims it) |
| Metatype arms | `src/ir/interp.zig` (`defineStruct`/…/`reflectTypeInfo`) | 3 (delete, re-express in sx) |
| `#compiler` / BuildOptions | `library/modules/build.sx`, `src/ir/compiler_hooks.zig` | 3 (migrate, delete `#compiler`) |
| Surface syntax | `src/parser.zig`, `src/ast.zig` (`abi`/`extern`/`#library`) | kept; revisited Phase 3 |
## Status
- **Phase 0 — DONE (2026-06-17).** The struct-weld machinery is stripped:
`compiler_lib.zig` lost the type registry (`weldStruct`/`bound_types`/`BoundType`/
`FieldLayout`/`findType`/`SxField`/`LayoutMismatch`/`validateStructLayout`);
`nominal.zig` lost `validateWeldedStruct`/`weldedFieldOrderStr` + the
`sd.abi == .zig` call; the struct-weld unit tests + examples `0625`/`0627`/`1183`/
`1186` are removed. **Decision (recorded):** the `intern`/`text_of` function
host-call bridge is KEPT — it is a clean scalar dispatch (string→handle), not
weld/serialize/marshal, and is the seed Phase 3 grows the compiler-call path from.
So the `compiler_welded` dispatch (`interp.callExtern` is unchanged at HEAD — the
pre-branch in `call()`), `weldedCompilerFn` (decl.zig), the `emitCall` comptime-only
gate (ops.zig), and examples `0626`/`1184`/`1185` stay. The `#library`/`abi`/`extern`
SYNTAX stays. `zig build test` green (688 corpus, 0 failed; unit tests pass).
- **Phase 1 — in progress.**
- **Sub-step 1 — DONE.** `src/ir/comptime_vm.zig`: the comptime `Machine`
(linear byte memory + bump/stack allocator with `mark`/`reset` reclamation +
scalar `readWord`/`writeWord` (1/2/4/8, little-endian) + `bytes` views; addr 0
reserved as `null_addr`) and `Frame` (register file indexed by Ref + stack
reclamation on `deinit`). A register `Reg` is a raw u64 — immediate scalar OR
`Addr`. Standalone + unit-tested (`comptime_vm.test.zig`, in the barrel); does
NOT touch the live interpreter, so the corpus stays green (688). No op execution
yet.
- **Sub-step 2 — DONE.** The executor (`Vm` in `comptime_vm.zig`): walks the SAME
IR `Inst` over comptime frames, mirroring the legacy interp's scalar semantics
(i64 wrapping/signed + f64 register words, keyed off the result/operand `TypeId`).
Ported: constants (`const_int`/`float`/`bool`/`null`/`undef`), arithmetic
(`add`/`sub`/`mul`/`div`/`mod`/`neg`), comparison (`cmp_*`), logical
(`bool_and`/`or`/`not`), conversions (`widen`/`narrow`/`bitcast` passthrough,
`int_to_float`/`float_to_int`), terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param` (branch args passed as Refs — the same frame persists, SSA-safe).
Any other op bails loudly (`error.Unsupported` + `detail = @tagName(op)`).
Unit-tested on hand-built IR (`Fb` builder): integer add, f64 arithmetic, cond_br
branch selection, a block-param loop summing i..1, div-by-zero + unsupported-op
bails. Corpus untouched (688 green) — the executor is exercised by unit tests only,
not yet wired to real comptime eval.
- **Sub-step 3 — DONE.** Memory + structs on comptime memory. `Vm` gained an optional
`table: *const TypeTable` (target-aware layout). Ported `alloca`/`load`/`store`
(over comptime addresses, `Store.val_ty` drives width) and `struct_init`/`struct_get`/
`struct_gep` (structs laid out at the table's natural offsets). The value model: a
`Kind.word` (scalar/pointer ≤8B) sits in a register; a `Kind.aggregate` (struct)
lives in comptime memory and its "value" IS its address (read returns the address,
write memcpys), so nested structs compose and `struct_gep` is just base+offset (no
field-pointer dance). `kindOf` bails loudly on the not-yet-ported types
(slice/string/any/optional/enum/array/tuple/…). The Addr-based value model survives
allocator realloc (offsets are stable; slices are only materialized transiently).
Unit-tested: struct_init+get round-trip, alloca+gep+store+load, nested-struct
aggregate copy + nested read. Corpus untouched (688 green).
- **Sub-step 4a — DONE.** Tuples + arrays. `kindOf` widened (`tuple`/`array` →
aggregate). Ported `tuple_init`/`tuple_get` (positional, `tupleFieldOffset`),
`index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over array/pointer/
many_pointer bases; slice/string bases bail), and `length` on an array value
(static `ArrayInfo.length`). Unit-tested: mixed tuple round-trip, `[3]i64`
gep/store + index_get sum (42), array `length` (3). 688 corpus green.
- **Sub-step 4b — DONE.** Slices + strings as `{ptr@0 (pointer_size), len@8 (i64)}`
fat pointers (`kindOf`: string/slice → aggregate). Ported `const_string` (materializes
text+NUL in comptime memory + a fat pointer), `length`/`data_ptr` (read len/ptr fields),
`array_to_slice`, `subslice`, indexing *through* a slice/string (`elemAddr` loads
`.ptr` first), and `str_eq`/`str_ne` (len+memcmp). Helpers `makeSlice`/`sliceLen`/
`sliceData`. Unit-tested: string length + str_eq/ne, array→slice + slice index +
slice length (23), array subslice (43). 688 corpus green.
- **Sub-step 4c — DONE (optionals + payloadless enums).** `kindOf`: `enum` → word;
`?T` → word if pointer-child (null==0) else `{T@0, i1@sizeof(T)}` aggregate. Ported
`optional_wrap`/`unwrap`/`has_value`/`coalesce` (with `optChildIsPtr`/`optHas`
helpers; `const_null` → `null_addr` reads as none), `enum_init` (payloadless: tag is
the value), `enum_tag` (payloadless/word). Unit-tested: non-pointer `?i64`
wrap/unwrap/coalesce (91), pointer `?*i64` null==0 (99), payloadless enum tag (11).
688 corpus green.
- **Sub-step 4d — partial (`addr_of`/`deref` DONE).** `addr_of` passes through (an
aggregate value already IS its address; a pointer is already an address — mirrors
the legacy); `deref` = `readField` through the pointer (`ins.ty` is the pointee).
Unit-tested (deref a `*i64` → 77; addr_of a struct value + field read → 80).
**Deferred to the wiring phase (intentionally, not ported blind):** tagged-union
payload (`enum_init` w/ payload, `enum_payload` — the legacy stores *untyped* Values
and `field_index` indexes payload sub-fields, not variants, so a byte model's
payload type is ambiguous without a real call site), `any` boxing, closures, and the
bitwise ops. These have subtleties best resolved against actual corpus cases — the
VM's loud `error.Unsupported` + `detail` will name exactly what each real eval needs.
- **Sub-step 1.5 — direct `call` DONE.** `Vm` gained `module: *const Module`
(resolves a callee `FuncId`) + a `depth`/`max_depth` recursion guard. `call`
marshals arg Refs → Reg words and recursively `run`s the callee; aggregate args/
results pass as their `Addr` over the SHARED comptime memory (no copy). **Stack-lifetime
change:** `Frame` no longer reclaims the machine on exit (a returned aggregate's
Addr would dangle) — a comptime eval's allocations live to `Vm.deinit`;
`Machine.mark`/`reset` stay for explicit use. Extern/builtin callees (no blocks)
bail loudly (1.5b). Unit-tested: direct call (`add(20,22)+100` → 142) and recursion
(`sum(0..n)` → 15/55). 688 corpus green.
- **Sub-step 1.5b — `Reg`↔`Value` boundary bridge DONE.** The builtin/`compiler_call`/
extern handlers are all coupled to the legacy `Interpreter` (e.g. `compiler_lib`
handlers take `*Interpreter`), so the VM can't call them directly — the wiring uses
WHOLE-FUNCTION fallback instead (VM runs pure functions; a bail re-runs the whole
eval in the legacy). That needs the boundary bridge: `valueToReg` (host `Value` arg →
VM `Reg`, materializing aggregates into comptime memory) + `regToValue` (VM result →
`Value`, deep-copied out). Covers scalars + strings + structs (other aggregate shapes
bail loudly; added as wiring surfaces them). Transitional — deleted once the VM owns
comptime end-to-end. Unit-tested with round-trips. 688 corpus green.
- **Then the wiring step** (below) — now unblocked.
### Decision (2026-06-17): pivot from blind op-porting to CALLS + hybrid wiring
The common leaf ops are ported (scalars, control flow, structs, tuples, arrays, slices,
strings, optionals, payloadless enums, deref/addr_of) and unit-tested. Continuing to
port the rarer ops (tagged-union payload, any, closures) in isolation risks subtle
bugs and has low signal. The higher-value path:
1. **Calls (sub-step 1.5)** — `call` (direct), then `call_builtin`/`compiler_call`. The
shared comptime memory makes aggregate args/results pass naturally (they're Addrs). The
one design point: **aggregate-return lifetime** — a callee's stack-reclaim would
dangle a returned struct Addr, so for comptime (bounded) the VM should stop
reclaiming per-frame and let the whole eval's allocations live until `Vm.deinit`
(keep `Machine.mark/reset` for explicit use; drop it from `Frame.deinit`).
2. **Hybrid wiring** — `-Dcomptime-flat` routes a comptime eval through the VM, falling
back to the legacy interp on `error.Unsupported`. This makes the VM run the REAL
corpus, proving parity incrementally and surfacing exactly which ops each real eval
needs — far better signal than more isolated unit tests.

View File

@@ -0,0 +1,207 @@
# sx `extern` / `export` + `#foreign` retirement — Plan (FFI-linkage stream)
**One stream, two parts.** **Part A** adds `extern`/`export` (the linkage surface);
**Part B** migrates every `#foreign` onto it and purges `foreign` from the tree.
They are *one* plan: Part B can't start until Part A is a behavior-equivalent
superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant.
**Design rationale:** [design/inline-asm-design.md](../design/inline-asm-design.md) §II.2
(Deviation 6) + §II.10 #4 + the syntax evaluation.
**Decided syntax**
```sx
name :: (sig) -> Ret [callconv(.x)] [extern | export] [LIB] ["csym"] [;|{…}]; // functions
Name :: #objc_class("X") [extern | export] { … }; // aggregates (mirrors `struct #compiler`)
g : Type extern [LIB] ["csym"]; // extern global
```
- `extern` = import (no body, external linkage, C ABI, no sx ctx) — `#foreign`'s role.
- `export` = define **and** expose (body + external linkage + C ABI + no ctx) — **new**.
- `extern`/`export` imply `callconv(.c)`; write `callconv` only to override.
- Optional `LIB` (a `#library` alias) + `"csym"` rename mirror `#foreign LIB "csym"`,
so `extern` is a true `#foreign` **superset** (Gate A→B): carried on
`extern_lib`/`extern_name`. The `#library` declaration + build-flag linking
mechanism stays a separate axis — `extern` *references* a lib, it doesn't fold
in `#library` itself. (Revises the original "library fully separate" decision 4.)
> **END-STATE INVARIANT (hard requirement).** After this stream, `foreign` appears
> **nowhere** in the live tree — not the `#foreign` surface, and **not** internal
> identifiers. The extern AST is **not** named `foreign_expr`. Enforced by the
> Phase 9.4 grep gate. Scope today: 643 `foreign` lines / ~57 identifiers in `src/`
> + 28 in live docs — most of it the objc/jni **runtime-class** machinery.
**Naming constraint (so we can actually reach the invariant):** introduce
`extern`-named representations only — do **not** reuse or extend
`ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Carry extern/export on a new
`FnDecl.extern_export` modifier with a `;`/`{…}` body (so there is **no** `*_expr`
node for it) + `FnDecl.extern_lib`/`extern_name`; add `VarDecl.is_extern`/
`extern_lib`/`extern_name`. The IR is already extern-named (`Function.is_extern`,
`Builder.declareExtern`).
**Key finding (scopes Part A):** the IR + LLVM emit **already support everything**
`Function.linkage` (`.external/.internal/.private`), `is_extern`, `call_conv`, and a
raw un-mangled symbol name are all emitted by `declareFunction`
(`emit_llvm.zig:1225-1300`). Part A is a **parser + lowering** job, no codegen change.
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior-lock).
`zig build && zig build test` after every step. Never regenerate snapshots while red.
---
# PART A — add `extern` / `export` (alongside `#foreign`)
## Phase 0 — tokens + parser plumbing
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | add `kw_extern`, `kw_export` (Tag enum + `StaticStringMap`, beside `kw_callconv` at `token.zig:45,282`); unit lex test | `src/token.zig` |
| 0.1 | lock | `parseOptionalExternExport()` (mirror `parseOptionalCallConv`, `parser.zig:3669`) + `ast.ExternExportModifier` enum + `FnDecl.extern_export` + `VarDecl.is_extern`/`extern_name` fields; **not yet consumed**; unit AST test | `src/parser.zig`, `src/ast.zig` |
## Phase 1 — `extern` (import; equivalent to lib-less `#foreign`)
| Step | Commit | What | Files |
|---|---|---|---|
| 1.0 | xfail | accept postfix `extern` after the callconv slot (`parser.zig:1950`); `examples/12xx-ffi-extern-fn.sx` extern-binds a libc symbol — red (lowering not wired) | `src/parser.zig` |
| 1.1 | green | lowering: `extern``is_extern`, `.external`, `callconv(.c)`, no ctx — route through `declareExtern` like a lib-less `#foreign` (anchors `decl.zig:1123,387,2110,2113`). Example green | `src/ir/lower/decl.zig` |
| 1.2 | green | optional `extern "csym"` rename + extern-global form `g : T extern;` (`parser.zig:425` path) | `src/parser.zig`, `src/ir/lower/decl.zig` |
## Phase 2 — `export` (define + expose; the NEW capability)
Fills the four export-gap conditions (all in `src/ir/lower/decl.zig`):
| Gap | Anchor | Fix |
|---|---|---|
| (i) linkage forced `.internal` | `:2382`, `:2514` | also `.external` when `extern_export == .export` |
| (ii) C ABI not promoted | `:2110` | also `.c` when `== .export` |
| (iii) no symbol-name override | `emit_llvm.zig:1226` raw name | parse optional `export "csym"`; map in the name map |
| (iv) ctx param not suppressed | `:387` `funcWantsImplicitCtx` | also suppress when `== .export` |
| Step | Commit | What | Files |
|---|---|---|---|
| 2.0 | xfail | multi-file test: an `export fn` called from a companion `.c` caller (same `XXXX-` prefix) — red (still internal) | `examples/12xx-ffi-export-fn.{sx,c}` + `expected/` |
| 2.1 | green | gaps (i),(ii),(iv): `export` ⇒ external + C-ABI + no-ctx on a **defined** fn (uses `beginFunction`, not `declareExtern`) | `src/ir/lower/decl.zig` |
| 2.2 | green | gap (iii): `export "csym"` symbol-name override | `src/parser.zig`, `src/ir/lower/decl.zig` |
## Phase 3 — aggregates (objc / jni runtime classes)
| Step | Commit | What | Files |
|---|---|---|---|
| 3.0 | xfail | `#objc_class("X") extern { … }` (import) + `… export { … }` (define) parse alongside legacy `#foreign #objc_class` | `src/parser.zig` (`tryParseForeignClassPrefix` :1305, `parseForeignClassDecl` :1369) |
| 3.1 | green | map postfix `extern`→reference, `export`→define+register; per-runtime tests (objc, jni) | `src/parser.zig`, `src/ir/lower/decl.zig`, `src/ir/lower/objc_class.zig` |
## Phase 4 — interplay, diagnostics, docs
`extern`+`callconv` stacking/redundancy; reject `extern`+`export` together;
`specs.md` documents `extern`/`export` (the three axes); `#foreign` still documented
until Part B cutover.
> **GATE A→B.** `extern`/`export` are a behavior-equivalent **superset** of
> `#foreign`. Lock with a unit test asserting `#foreign` and `extern` lower to
> identical IR for a sample fn / global / class. Do not start Part B before this.
---
# PART B — migrate `#foreign` → `extern`/`export`, then purge `foreign`
**Inventory (drives the batches):** `#foreign` = 466 uses. ~391 sx-code (308 fns
[207 lib / 196 rename], 75 classes [39 objc / 31 jni], 8 globals) + ~145 example
snapshots. 6 libs (`sqlib`98 `libc`61 `objc`22 `tlib`12 `raylib`7 `clib/pcaplib`3).
Hotspots: `vendors/sqlite`(98), `platform/{android,uikit,android_jni,sdl3}`,
`std/{socket,thread,fs,time}`, `ffi/{objc,raylib}`.
## Phase 5 — `#foreign` becomes an alias for `extern`
| Step | Commit | What | Files |
|---|---|---|---|
| 5.0 | lock | route the `#foreign` parser paths (`parser.zig:316,425,1305,1970`) to build the *same extern-named* AST as `extern`/`export`. Suite green, snapshots unchanged | `src/parser.zig` |
| 5.1 | lock | unit test: `#foreign` and `extern` produce identical IR (fn/global/class) | `src/ir/lower/decl.test.zig` |
## Phase 6 — migrate stdlib (behavior-preserving; snapshot diff must be EMPTY)
One commit per batch; rewrite `#foreign``extern` (fns/globals),
`#foreign #objc_class``#objc_class … extern`, defined classes → `… export`.
| Step | Batch | ~sites |
|---|---|---|
| 6.1 | `library/vendors/sqlite/` | 98 |
| 6.2 | `library/modules/platform/` (uikit/android/android_jni/sdl3) | ~95 |
| 6.3 | `library/modules/std/` (socket/thread/fs/time/process/…) | ~60 |
| 6.4 | `library/modules/ffi/` (objc/raylib/objc_block/…) | ~50 |
| 6.5 | remaining `library/` + `vendors/` | remainder |
## Phase 7 — migrate examples + issues (empty snapshot diff; review every diff)
| Step | Batch |
|---|---|
| 7.1 | `examples/12xx-ffi-*` (plain C) |
| 7.2 | `examples/13xx-ffi-objc-*` |
| 7.3 | `examples/14xx-ffi-jni-*` |
| 7.4 | `issues/*` repros + stragglers |
A non-empty diff ⇒ the alias wasn't behavior-equivalent — stop, fix Phase 5.
## Phase 8 — cutover
| Step | Commit | What |
|---|---|---|
| 8.0 | xfail | `examples/11xx-diagnostics-foreign-removed.sx` expects a "`#foreign` removed; use `extern`/`export`" diagnostic — still accepted (red) |
| 8.1 | green | parser hard-rejects `#foreign` (mirrors the variadic `name: ..T` cutover); `specs.md` drops `#foreign`, documents `extern`/`export` |
## Phase 9 — total `foreign` purge (the invariant)
`foreign` must not appear anywhere in the live tree, surface *or* internal. Each step
a mechanical, behavior-preserving rename commit (snapshots unchanged), small
per-file/subsystem commits — not one sweep.
| Step | What | Identifiers (count → new) |
|---|---|---|
| 9.0 | delete the surface | `hash_foreign`(11) + lexer entry + the 4 parse paths + the alias |
| 9.1 | rename **linkage**`extern*` | `foreign_expr`(25) **eliminated** (folds into modifier) · `is_foreign`(39)→`is_extern` · `foreign_lib`/`foreign_name``extern_*` · `foreign_name_map``extern_name_map` · `callForeign`(8)→`callExtern` · `marshalForeignArg``marshalExternArg` · `is_foreign_c_api`(5)→`is_extern_c_api` · `dedupeForeignSymbol``dedupeExternSymbol` |
| 9.2 | rename **runtime-class** machinery → `runtime*` (decision 5) | `ForeignClassDecl`(65) · `ForeignMethodDecl`(31) · `ForeignClassMember`(20) · `ForeignFieldDecl`(15) · `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `foreign_path`(62) · `ForeignRuntime` · `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` · `findForeign{Method,Property}InChain` · `resolveForeign*` · `register*ForeignClass*` · `foreignClass*Type` · `*ForeignRefs` |
| 9.3 | purge **live docs** (28 lines) | `specs.md`/`readme.md`/`CLAUDE.md`: drop `#foreign`, document `extern`/`export`; fix file-roles + FFI/bundling notes |
| 9.4 | **acceptance gate** | `grep -rniE 'foreign' src/ library/ examples/ specs.md readme.md CLAUDE.md`**0** |
---
## Open decisions
*Part A (ratified — recommendations stand):* 1. bare keywords (not `#extern`).
2. aggregate position postfix (`#objc_class(…) extern`, like `struct #compiler`).
3. `extern ⇒ callconv(.c)`. 4. **REVISED** (user, 2026-06-14): `extern` carries an
optional `LIB`+`"csym"` axis (`extern_lib`/`extern_name`), mirroring `#foreign LIB
"csym"`, so it's a true `#foreign` superset (Gate A→B). The `#library` declaration +
build-flag linking mechanism stays separate — `extern` references a lib, doesn't
fold in `#library`. (Was: "library fully separate / not on `extern`".)
*Part B:* 5. runtime-class rename target — **RATIFIED `Runtime*Class*`** (user, 2026-06-14;
it's the object-model axis, not linkage). 6. historical carve-out — **STILL OPEN** (user did
not confirm at the Part A milestone): keep `issues/*.md` (+ design-doc prose) as provenance &
gate only the live tree (recommended) vs purge everything. Confirm 6 before Phase 9.
## Relationship to ASM
`PLAN-ASM.md` Phase F (global asm) consumes `extern` (import the asm symbol) and
`export` (let asm call back into sx) — do it after **Part A Phase 2**.
---
## Kickoff prompt (paste into a fresh session to start Part A)
> Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint
> `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided
> syntax, Naming constraint, Key finding) and Part A; rationale is in
> `design/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4.
>
> **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix
> keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays
> untouched). Do NOT start Phase 2 (`export`) or Part B (migration).
>
> **Cadence (IMPASSIBLE):** no commit may both add a test and make it pass — lock
> behavior with a passing test, or land an xfail the next commit turns green.
> `zig build && zig build test` after every step.
>
> **Naming constraint (hard):** introduce only `extern`-named AST — do NOT reuse or
> extend `ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Use a new
> `FnDecl.extern_export` modifier (body `;` or `{…}`) and `VarDecl.is_extern`/
> `extern_name`. IR is already extern-named (`Function.is_extern`, `declareExtern`).
>
> Steps (commit after each; update the checkpoint each time):
> - 0.0 lock: `kw_extern`/`kw_export` tokens + map entries beside `kw_callconv`
> (`src/token.zig:45,282`) + unit lex test.
> - 0.1 lock: `parseOptionalExternExport()` (mirror `parseOptionalCallConv`,
> `parser.zig:3669`) + `ast.ExternExportModifier` + `FnDecl.extern_export` +
> `VarDecl.is_extern`/`extern_name` (parsed, unconsumed) + unit AST test.
> - 1.0 xfail: accept postfix `extern` after the callconv slot (`parser.zig:1950`);
> add `examples/12xx-ffi-extern-fn.sx` that extern-binds a libc symbol (red).
> - 1.1 green: in `src/ir/lower/decl.zig`, lower `extern` like a lib-less `#foreign`
> import — `is_extern`, `.external`, `callconv(.c)`, no ctx, via `declareExtern`
> (anchors :1123, :387, :2110, :2113). Example goes green.
> - 1.2 green: optional `extern "csym"` rename + extern-global `g : T extern;`
> (`parser.zig:425`).
>
> Stop at end of Phase 1. Verify: suite green; the `extern` libc binding runs;
> `#foreign` still works with no snapshot diffs. If you hit an unrelated compiler
> bug, follow the CLAUDE.md IMPASSIBLE RULE (file an issue, stop).

276
current/PLAN-FIBERS.md Normal file
View File

@@ -0,0 +1,276 @@
# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
> **STATUS: ✅ COMPLETE.** The pure-sx M:1 async runtime is feature-complete end-to-end
> (`library/modules/std/sched.sx`, examples 18001817, suite 755/0): `abi(.naked)` context switch
> (aarch64 + x86_64/Win64), M:1 scheduler, suspending `go`/`wait`/`cancel`, deterministic virtual-time
> timers (`sleep`/`now_ms`), and real fd readiness via kqueue (`block_on_fd`). Five compiler bugs fixed
> en route (0151/0152/0153/0154/0156-P1/0157). Deferred non-blocking follow-ups: linux `epoll` twin,
> `Scheduler.deinit`, `Future(void)`/`timeout` (0150), `context.io`-routed async (M:N). Next carve:
> Stream B2 (channels / cancel / async stdlib). Historical step-status below.
>
> B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
> (`Io` interface + `async`/`await`/`cancel` over blocking `CBlockingIo`) ✅ · B1.3 (fiber
> runtime: naked `swap_context` + §10.7 stress gate + guarded `mmap` stacks, proven on aarch64
> AND x86_64/Win64) ✅ · **B1.5a (M:1 scheduler CORE — `std/sched.sx`: `spawn`/`yield_now`/
> `suspend_self`/`wake`/`run`) ✅** (fixed blocker 0154) · **B1.4a (suspending fiber-task async —
> `sched.go`/`wait`/`cancel` over `Task($R)`, nullary-thunk) ✅** (adversarially reviewed; fixed
> blockers 0156-Part1 + 0157 en route; locked `1813`).
> **B1.4b (deterministic virtual-time timers — sched.sleep/now_ms/timer-run) ✅** (reviewed; fixed a CRITICAL timer-vs-early-wake UAF; locked 1814/1815).
> **B1.4c (event-loop — real fd readiness via kqueue: `block_on_fd` + run-loop Mode 2) ✅** (reviewed; fixed a CRITICAL same-fd lost-wakeup hang; locked 1816). macOS only — linux epoll twin deferred.
> **B1.5 (end-to-end M:1 capstone — `go`/`wait`+`sleep`+scheduler, deterministic ordering) ✅** (locked 1817). **STREAM B1 COMPLETE.** Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
> deterministic timers live as `sched.*` methods (M:1), NOT routed through the erased `context.io` (avoids forcing sched.sx into every std consumer + the `_fib_tramp` dup-symbol
> trap); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready` stay reserved for M:N. Deferred:
> issue 0150 (`Future(void)`/`timeout`); 0156-Part2 (deferred `..` spread); the `::` callable-param
> feature.
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream B (§B1) + the
design-of-record [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md)
§4 (async), §7 steps 49, §8.1 (risks), §10 (testing). Progress in
[CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). Stream B2 (channels/cancel/stdlib) is a
separate carve ([PLAN-CHANNELS.md], when reached) and depends on this + atomics (✅).
**Goal:** the colorblind, stackful, **pure-sx** async runtime — fibers behind an `Io`
interface, an M:1 scheduler, blocking + deterministic-sim + event-loop `Io` impls. The
**compiler floor is small and net-new**: make `abi(.naked)` actually emit an LLVM `naked`
function (B1.0), and confirm/close the per-fiber `context` root (B1.1). **Everything
else — the context-switch asm, fiber bootstrap, `mmap` stacks, the scheduler, futures,
the `Io` vtables — is ordinary sx library code** (design §4, §4.4). The irreducible FFI
floor: the per-arch asm context-switch (in `.sx`), syscall `extern`s, and `mmap`.
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `18xx` concurrency. On an **unrelated** compiler bug → file
`issues/NNNN`, mark this checkpoint BLOCKED, STOP (CLAUDE.md). The in-session
worker-fix override (delegate a blocker to a worker) applies only with explicit user
authorization.
---
## Design (grounded against the tree)
### B1.0 — `abi(.naked)` codegen (the one genuinely net-new compiler piece in B1)
The design doc spells this `callconv(.naked)`; the **real sx surface is `abi(.naked)`**
written in the postfix slot, `name :: (sig) -> Ret abi(.naked) { asm { … }; }` (cf.
`build_options :: () -> BuildOptions abi(.compiler);` in [build.sx:28](../library/modules/build.sx#L28)).
The sx-facing name is **`naked`** throughout (keyword, field `is_naked`, diagnostics) —
matching LLVM's `naked` attribute (the lowering mechanism) and the industry term
(Zig/Rust/GCC/Clang). The ABI variant was renamed `.pure → .naked`: "pure" universally
means *side-effect-free*, the opposite of a register-clobbering context switch.
**Grounding (verified — do not re-derive):**
- The `ABI` enum **already carries `.naked`**`ABI = enum { default, c, compiler, naked }`
([ast.zig:142](../src/ast.zig#L142)), documented "naked function (inline asm
body), no calling-convention prologue/epilogue." So B1.0 is **NOT** "extend the enum."
- `.naked` is **inert today**: [type_resolver.zig:237](../src/ir/type_resolver.zig#L237)
maps `.compiler, .naked → .default` CC, and `emit_llvm` emits **no LLVM `naked`
attribute**. So the net-new work is exactly: **carry `abi == .naked` into the IR
`Function`, emit LLVM's `naked` attr, and skip the implicit-`Context` / prologue
lowering** so the body is just the asm block + its own `ret`.
- The IR `Function` struct ([inst.zig:605](../src/ir/inst.zig#L605)) carries `call_conv`
(default/c) + `is_compiler_domain`, but **no naked flag** — add one (`is_naked: bool`).
- Attribute API is in-tree: `nounwind` is set at
[emit_llvm.zig:1339](../src/ir/emit_llvm.zig#L1339) via
`LLVMGetEnumAttributeKindForName("nounwind", 8)``LLVMCreateEnumAttribute(ctx, id, 0)`
`LLVMAddAttributeAtIndex(func, func_idx_attr /* -1 */, attr)`. The LLVM `naked` attr
is the same shape: `LLVMGetEnumAttributeKindForName("naked", 5)`.
- The `.c` ABI **already skips the implicit ctx** at lowering — `lam.abi == .c` /
`fd.abi == .c` gates (closure.zig:171, [decl.zig:515](../src/ir/lower/decl.zig#L515)).
`.naked` must skip it **too** (a `.naked` fn gets no synthetic `__sx_ctx`, no stack frame,
no prologue — args arrive in ABI registers and are read directly from asm). The
implicit-return machinery (`lowerValueBody`) must also be bypassed: a `.naked` body has no
sx return (the asm rets itself), so lower its statements and cap the block with
`unreachable`.
- **Inline asm already works end-to-end** (lower→emit→JIT): aarch64
([examples/1645](../examples/1645-platform-asm-aarch64-add.sx)), x86_64
([examples/1651](../examples/1651-platform-asm-x86-syscall-write.sx)), global asm, JIT
([1653](../examples/1653-platform-asm-global-jit.sx)). `emitInlineAsm` /
`LLVMGetInlineAsm` at [ops.zig:915](../src/backend/llvm/ops.zig#L915). The `.naked` body
is a single asm block reusing this path.
**`.naked``.c` (design §4.6 context-switch note):** a `.c` epilogue restores SP from the
frame; a context switch deliberately makes SP-in ≠ SP-out, so the `.c` epilogue would
restore from the *wrong* stack. `.naked` = no prologue/epilogue/frame — the asm emits its
own `ret`. This is *why* the switch must be `.naked`, not `.c`.
**Snapshot story (per the atomics precedent):** a `.naked` fn's *body is raw per-arch asm*
(it can't be portable — that's the point), while LLVM's `naked` attribute text is
arch-invariant. **B1.0a** (lock) needs only **one host example** locked to the emit bail —
the bail fires at the function level *before* any asm/instruction selection, so it is
host-independent (no `.build` target pin). **B1.0b** (green) adds emission, pins that
example aarch64 (`.build {"target": "aarch64-macos"}`, end-to-end on a matching host,
ir-only on a mismatch), and adds an x86_64 cross sibling — mirroring the existing asm
corpus split (1645 aarch64 / 1651 x86). The ir-only `.ir` (only producible once emission
lands in B1.0b) asserts the `naked` attribute + the asm body. State loudly: **the `.ir`
proves the `naked` keyword + asm emitted, NOT that any hand-written register save/restore
is correct** — that is the B1.3 switch-stress harness's job, never the corpus's.
### B1.1 — per-fiber `context` root (grounding says this is SMALL, likely library-only)
**Grounding (verified — closes the design doc's open sizing question):**
- `context` is an **implicit `*Context` parameter** (`__sx_ctx`, slot 0), threaded through
every default-conv sx call ([lower.zig:259](../src/ir/lower.zig#L259)) — **not raw TLS**.
Inside a function `current_ctx_ref = Ref.fromIndex(0)` (the param) → it **rides the fiber
stack frame for free**.
- `push Context.{…}` allocates the new `Context` with a **stack `alloca`** and rebinds
`current_ctx_ref` to that slot ([stmt.zig:1263](../src/ir/lower/stmt.zig#L1263)) — "No
global, no walk." So **push frames are fiber-local for free**.
- The **only shared root** is the `__sx_default_context` **global**, bound at
entry-points / `abi(.c)` fns *before any user code runs*
([decl.zig:2667](../src/ir/lower/decl.zig#L2667), :2815).
⇒ The design doc's "lower as swappable indirection, never raw TLS" guards a **non-problem**
(confirmed). The **real, now-sized** B1.1 work is purely a **library convention**: a
freshly-`spawn`ed fiber must take its root `Context` from the **spawner's snapshot** (passed
as the fiber-entry fn's `__sx_ctx` slot-0 arg by the spawn trampoline), **not** the
`__sx_default_context` global. That is sx-side (the trampoline already controls slot 0) —
**expected to be ZERO compiler change.** B1.1's first action is a probe confirming this; if
a fiber genuinely re-reads the global root mid-stack (it should not — entry binds once),
*then* and only then is there a compiler obligation. **Ground the probe before sizing any
compiler work.** Prerequisite of B1.3 (a fiber needs a valid root before it switches).
### B1.2B1.5 — pure sx over the primitives (design §4)
- **B1.2 (A1):** `Io` interface + `context.io` + `Future` + `cancel()` — a protocol/vtable
threaded exactly like `Allocator` (which already lives at `Context` field 0; see
`allocViaContext` [call.zig:1214](../src/ir/lower/call.zig#L1214)). `Io` becomes another
`Context` field. No compiler change — protocols + context already carry it.
- **B1.3 (A2):** the fiber runtime — naked context-switch asm (per-arch), bootstrap, `mmap`
stacks **with mandatory guard pages**. All sx. **Highest corruption risk in the stream**
(§8.1.1) and **untestable by the deterministic `Io`** (which tests *scheduling*, not the
*switch*). Its **first deliverable, before the scheduler AND the deterministic `Io`**: a
standalone **2-fiber ping-pong switch-stress harness** (§10.7) — scribble every
callee-saved register + a stack canary before each suspend, deep/recursive chains, verify
all survive post-resume. This harness — not B1.4 — is A2's correctness gate.
- **B1.4 (A3):** `Io` impls in order **blocking → deterministic-sim (KEYSTONE) → event-loop**
(kqueue/epoll/io_uring). Build the deterministic `Io` right after blocking; **calibrate it
against blocking `Io`** before trusting it to gate everything async (§8.1.3, §10.7) — a
deterministic-but-wrong scheduler snapshots garbage. (Open, deferred: the event loop does
**not** yet cooperate with a platform UI run loop — CFRunLoop/ALooper; that's a §6
app-target gap, out of B1.)
- **B1.5 (A5·M:1):** the single-thread scheduler — validates the whole colorblind stack
end-to-end. `18xx` corpus runs under the deterministic `Io`, asserting a **program-emitted
ordering contract** (sequence markers), not raw interleaving, so scheduler-policy tweaks
don't churn every snapshot.
### Files the compiler floor touches (B1.0 only; B1.1B1.5 are library + tests)
B1.0 (`.naked`) forces these plumbing sites:
- [ast.zig:142](../src/ast.zig#L142) — `ABI.naked` (exists; reference only).
- [inst.zig:605](../src/ir/inst.zig#L605) — add `is_naked: bool = false` to `Function`.
- [decl.zig](../src/ir/lower/decl.zig) — set `is_naked` from `fd.abi == .naked`; gate the
implicit-ctx off for `.naked` in `funcWantsImplicitCtx` (mirror the `.c` skip at
decl.zig:515) and bypass `lowerValueBody` for `.naked` bodies (lower statements + cap with
`unreachable`, in both body-lowering paths) — a `.naked` fn binds no ctx and has no sx
return.
- [type_resolver.zig:237](../src/ir/type_resolver.zig#L237) — leave CC `.default` (a `.naked`
fn-pointer type has no CC of its own; nakedness is a decl-level emit attribute).
- [emit_llvm.zig:402](../src/ir/emit_llvm.zig#L402) Pass 2 — **B1.0a:** bail loudly when
`func.is_naked` (build-gating). **B1.0b:** instead emit LLVM's `naked` attr (shape per
`nounwind` at emit_llvm.zig:1339) + the asm-only body (no prologue).
- Any `.op`/`Function`-field switch the Zig build flags — let the build tell you.
---
## Phases (xfail→green steps)
### B1.0 — `abi(.naked)` codegen — ✅ COMPLETE
- **B1.0a (lock) — ✅ DONE.** Carried `abi == .naked` into IR `Function.is_naked`; threaded
through `decl.zig` (`funcWantsImplicitCtx` skips `.naked` like `.c`; all body-lowering paths
bypass `lowerValueBody` for `.naked`, lowering the asm body + capping with `unreachable`) +
generic.zig + pack.zig; `emit_llvm` Pass 2 bailed loudly on `func.is_naked`. Locked by
`examples/1800-concurrency-naked-asm.sx` + the generic regression (review-found gap).
- **B1.0b (green) — ✅ DONE.** `emit_llvm` declaration pass adds LLVM `naked` + `noinline` +
`nounwind` for `func.is_naked` and skips `frame-pointer=all` (incompatible with a frameless
function); Pass 2 emits the body normally (`naked` ⇒ verbatim asm + own `ret`, no
prologue). `1800` pinned aarch64 → exit 42 + `.ir`; `1801-concurrency-naked-generic.sx`
(renamed from `-bail`) proves the generic path emits a naked body (exit 42);
`1802-concurrency-naked-asm-x86.sx` x86_64 cross sibling (ir-only here, `.ir` locks `naked`
+ `movl $42, %eax`). Unit test `emit: abi(.naked) function gets the naked attribute` asserts
`naked` present + `frame-pointer` absent. Suite green (724/0).
- **B1.0c (review-hardening) — ✅ DONE.** A param-bearing `.naked` fn emitted invalid LLVM
(loud verifier error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both
paths + generic.zig) so a naked fn's args stay in registers (read by the asm body) — this
*enables* B1.3's `swap_context(from, to)`. Locked by `1803-concurrency-naked-asm-param.sx`.
Pack `.naked` (variadic + naked, nonsensical) left unsupported → loud verifier error.
### B1.1 — per-fiber `context` root — ✅ COMPLETE (zero compiler change)
Probe confirmed the spawn convention works with ordinary language features: snapshot
`context` (`snap := context`), store it in a struct, and `push f.root { entry(args) }` from a
trampoline running under a different ambient context — the body reads the snapshot (via the
implicit slot-0 `*Context` param), not the ambient ctx, and `push` restores ambient on exit.
No path re-reads `__sx_default_context` mid-stack ⇒ **no compiler obligation**; this is a pure
library convention. Locked by `examples/1804-concurrency-context-snapshot.sx` (`fiber root:
42` / `ambient after: 99`). The design doc's "never raw TLS" guarded a non-problem.
### B1.2 — A1: `Io` interface + `context.io` + `Future` + `cancel()` API
Library-only. `Io` as a protocol added to `Context` (mirror `Allocator`). `Future`/`cancel`
API surface. xfail→green via an `18xx` example exercising the blocking `Io` default (real
suspend lands in B1.3). No compiler change expected; if a protocol-in-context gap appears,
file it.
### B1.3 — A2: fiber runtime (naked switch + bootstrap + guarded `mmap` stacks) — ✅ COMPLETE
- **B1.3a (switch-stress harness FIRST) — ✅** the §10.7 register/canary-survival gate (1807/1808),
validity proven by negative controls, adversarially reviewed.
- **B1.3b — ✅** fiber bootstrap + guarded `mmap` stacks (1809); the x86_64 sibling landed as Win64
on a real VM (1810, `0 0 P`). Switch proven on TWO arch/ABI pairs.
### B1.5a — M:1 scheduler CORE (`std/sched.sx`) — ✅ COMPLETE
The reusable scheduler wrapping `swap_context`: generic `Fiber`/`Scheduler`,
`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over guarded `mmap` stacks, one generic
`fib_dispatch` running any stored closure body. Adversarially reviewed + hardened; fixed blocker
bug 0154 (struct-field `null`/`---` over-store) en route. Locked by `1811` (round-robin) + `1812`
(suspend/wake). Built BEFORE the deterministic `Io` because FiberIo (B1.4a) needs it as substrate.
### B1.4a — suspending fiber-task async (`sched.go`/`wait`/`cancel`) — ✅ COMPLETE
`Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` in `sched.sx` (nullary-thunk;
self-contained). `go` spawns `work` as a fiber, `wait` parks the caller until it completes. Locked
by `1813`. Two compiler blockers fixed (0156-Part1, 0157) + adversarially reviewed/hardened.
### B1.4b/c — A3: `Io` impls (deterministic-sim KEYSTONE → event-loop)
Blocking exists (io.sx `CBlockingIo`). Next the deterministic-sim `Io`, **calibrated against
blocking** before any `18xx` test trusts it; then the event loop. The deterministic `Io` is the
test harness for *all* of B1.5 + Stream B2.
### B1.5 — A5: M:1 scheduler — ✅ COMPLETE
End-to-end validation of the colorblind stack. The `18xx` corpus asserts program-emitted ordering
contracts under the scheduler + deterministic timers; the capstone `1817` composes `go`/`wait` +
`sleep`/`now_ms` + the scheduler (three tasks complete in deadline order, deterministic sum). Stream
B1 is feature-complete.
---
## Gates
- **B1.0:** unit `emit_llvm.test.zig` (the `naked` attr present on a `.naked` fn); two
arch-gated examples (aarch64 + x86_64) run end-to-end on a matching host, ir-only on a
mismatch (assert `naked` + asm in `.ir`). **OUT of corpus scope, stated loudly:** the
*correctness* of any hand-written register save/restore — that's the B1.3 stress harness.
- **B1.1:** an `18xx` example locking context-carried-by-slot-0 behavior + a checkpoint note
on the spawn-trampoline convention.
- **B1.3:** the **switch-stress harness is A2's gate** (register/canary survival — §10.7),
NOT a run/snapshot test; plus arch-gated run tests.
- **B1.4:** deterministic `Io` **calibrated** against blocking `Io` (§8.1.3) before trusting
it; `18xx` under the deterministic `Io`.
- **B1.5:** `18xx` ordering-contract snapshots under the deterministic `Io`.
## Kickoff prompt (B1.0b — paste into a fresh session)
> Implement Stream B1 step **B1.0b** (`abi(.naked)` real emission) per
> `current/PLAN-FIBERS.md`. Verify `zig build && zig build test` is green first (B1.0a is
> already landed: `Function.is_naked` plumbed, `decl.zig` skips ctx + bypasses implicit-return
> for `.naked`, `emit_llvm` Pass 2 bails loudly, `examples/1800-concurrency-naked-asm.sx`
> locked to the bail). Then: (1) in `src/ir/emit_llvm.zig` Pass 2 (~line 402), REPLACE the
> `func.is_naked` bail with real emission — set LLVM's `naked` attribute on the function
> (`LLVMGetEnumAttributeKindForName("naked", 5)` → `LLVMCreateEnumAttribute(ctx, id, 0)` →
> `LLVMAddAttributeAtIndex(llvm_func, -1, attr)`; shape per the `nounwind` set at
> emit_llvm.zig:1339) and emit the `.naked` body as its asm block only, no prologue/epilogue
> (the body already lowers to the inline-asm op + an `unreachable` terminator). (2) Pin
> `examples/1800-concurrency-naked-asm.sx` aarch64 with a `.build` sidecar
> `{"target":"aarch64-macos"}`; on this aarch64 host it runs end-to-end (exit 42), capture
> `.ir` + regen (`-Dname=examples/1800-concurrency-naked-asm.sx -Dupdate-goldens`), review the
> diff (assert the `.ir` shows the `naked` attr + `mov x0, #42` / `ret`, NO stray error
> text). (3) Add `examples/1802-concurrency-naked-asm-x86.sx` (x86_64 body, `.build
> {"target":"x86_64-linux"}`, ir-only on this host — requires its `.ir`, now producible).
> (4) Add a unit test in `src/ir/emit_llvm.test.zig` asserting the `naked` attribute is
> present on an `abi(.naked)` function. Confirm `zig build test` green, commit. NOTE: the
> `.ir` proves the keyword + asm emitted, NOT register-save correctness (that's the B1.3
> switch-stress harness). If you hit an UNRELATED compiler bug, file `issues/NNNN`, mark
> `CHECKPOINT-FIBERS.md` BLOCKED, and STOP.

View File

@@ -1,227 +0,0 @@
# PLAN-HTTPZ — Stream HTTPZ (production HTTP-server readiness)
> **STATUS: 🟡 PLANNED — not started.** This stream is being (re)established as a
> *tracked* stream. The HTTP/socket/thread work to date shipped ad-hoc under phase
> tags in source comments (`S2` socket nonblocking, `S6` pthreads, `S7a` http server,
> `S7b` thread-pool handlers, `C3` per-OS selection) but **never had a PLAN/CHECKPOINT
> file** — the comments in [socket.sx:3](../library/modules/std/socket.sx#L3) /
> [thread.sx:23](../library/modules/std/thread.sx#L23) reference a "PLAN-HTTPZ" that did
> not exist until now. Progress tracked in [CHECKPOINT-HTTPZ.md](CHECKPOINT-HTTPZ.md).
**Goal:** the low-level guarantees needed to run a long-lived HTTP service on Linux in
production — *not* a web framework. Survive malformed clients, slow clients, overload,
restarts, memory pressure, and a normal Linux deployment without every app author
rediscovering the same failure modes. Driven by the user's production-readiness checklist
(P0 blockers, P1 hardening, P2 ergonomics), mapped below to concrete sx work.
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/<cat>/<file>.sx -Dupdate-goldens` + review the
diff. HTTP corpus lives in `examples/http/` (`16xx`/`http` category) + `examples/event/`.
Stress/fuzz/load harnesses live OUTSIDE the corpus (the corpus runner has a 10s/example
timeout and no network sandbox — see [corpus_run.test.zig](../src/corpus_run.test.zig)).
---
## Audit of record (grounded against the tree, 2026-06-26)
What already exists, so the next session does not redo discovery. **Two layers of P0 #1
are already done to a high standard; one piece is a literal blocker.**
### ✅ Solid / Linux-validated — do NOT rebuild
- **[event.sx](../library/modules/std/event.sx)** — `Loop` fully branches `OS == .linux`
(epoll, lines ~85251) vs kqueue (~251323). Ran **6/6 green on real aarch64 Linux** in
an Apple `container` VM (kernel 6.18); ABI corpus-locked by `examples/event/1633`.
- **[net/epoll.sx](../library/modules/std/net/epoll.sx)** — arch-aware `EpollEvent` layout
(12B packed x86_64 / 16B aligned aarch64 via the u32-split trick), correct flags, EINTR
retry, `__errno_location`.
- **[net/kqueue.sx](../library/modules/std/net/kqueue.sx)** — macOS-only, correct.
- **[sched.sx](../library/modules/std/sched.sx)** — M:1 fiber runtime; epoll/kqueue
fd-readiness fully branched incl. `EPOLL_CTL_DEL` after `EPOLLONESHOT`. Linux-tested.
- **[json.sx](../library/modules/std/json.sx)** — streaming writer, zero-copy views,
explicit allocators, stable key order. (Integers only — no floats.)
### ❌ BROKEN on Linux — the keystone blocker (Phase C3)
- **[socket.sx](../library/modules/std/socket.sx)** — Darwin-only, no `OS` branching:
- `SockAddr` ([:32](../library/modules/std/socket.sx#L32)) carries Darwin's `sin_len:u8`
at offset 0; Linux `sockaddr_in` has no such field → family/port written to wrong
offsets, addresses corrupted.
- `O_NONBLOCK = 4` — Linux is `2048`; `set_nonblocking` sets the wrong bit.
- errno constants are macOS values (`EAGAIN=35`→Linux 11, `EINPROGRESS=36`→115,
`ECONNRESET=54`→104, …) → WouldBlock/reset detection silently breaks.
- `errno_slot` binds `__error` ([:52](../library/modules/std/socket.sx#L52)) — Linux is
`__errno_location`.
- **[thread.sx](../library/modules/std/thread.sx)** — Darwin pthread struct sizes:
- `MutexBuf = 64B` ([:44](../library/modules/std/thread.sx#L44)) is Darwin's
`pthread_mutex_t`; glibc is **40B**`pthread_mutex_init` overflows the buffer by
24B. **Heap corruption on first mutex init under the thread pool.** (`CondBuf = 48B`
happens to match glibc — fragile coincidence.)
### ⚠️ Works, unhardened — [http.sx](../library/modules/std/http.sx)
Single-worker event loop; inline (`thread_pool_count = 0`) + pooled handlers; keep-alive
+ pipelining; delivery timeouts (`timeout_request_ms`/`timeout_keepalive_ms`); conn cap
(`max_conn`) + per-conn request cap (`request_count`); emits 400/413/431/503. Connection
state machine: `CONN_FREE/READING/WRITING/KEEPALIVE/HANDLING` with `gen` counter.
**Gaps (this stream's HTTP work):**
- **Parser:** `Content-Length` only (no `Transfer-Encoding`); no per-header-line size
limit; no header-count limit; no request-line/version syntax validation; no duplicate
`Content-Length` rejection; no `Content-Length` overflow guard.
- **Memory:** `Server.close()` ([:317](../library/modules/std/http.sx#L317)) frees neither
the `conns` array, the `PoolState` struct, nor `ps.done` → shutdown leaks.
- **Shutdown:** `run()` is an infinite loop; no `Server.stop()`; `close()` is abrupt (no
drain).
- **Timeouts:** delivery-only. **No handler-execution timeout** — a hung handler blocks
the loop (inline) or pins a pool worker forever; no cancellation.
- **Observability:** **none.** All accept/read/write/loop faults close the connection
silently — no log hook, no counters/metrics.
- **Response:** whole response built in one allocation; no streaming, no body-size
backpressure; alloc-failure path unhandled.
### ❌ Absent entirely
- **CI:** no `.github/workflows`, no Linux CI. Local `zig build test` on macOS only.
- **Fuzz / sanitizers / leak-check:** none. [tests/stress-http.sh](../tests/stress-http.sh)
is broken (references deleted `examples/32-http-server.sx`).
- **Releases:** no git tags, no CHANGELOG, no stability tiers.
- **Security:** no SECURITY.md, no disclosure process, no posture statement.
- **Deploy docs:** cross-compile/static-link documented in [readme.md](../readme.md); no
systemd/Docker/reverse-proxy/health-check/graceful-shutdown examples.
- **TLS:** none yet — to be added natively via an mbedTLS FFI binding (Phase T). Proxy
deployment stays documented as an option (D1).
- **Routing / query / form helpers:** manual `if req.path == …` dispatch only.
---
## Phases (dependency-ordered; checklist item in parens)
C3 is the keystone — until socket.sx + thread.sx are correct on Linux, **nothing in P0 is
honestly testable on Linux**, so Phase C precedes all else regardless of how the rest is
sliced.
### Phase C — Linux foundation (P0 #1) — unblocks everything
- **C3a — `socket.sx` per-OS.** Branch `SockAddr` (drop `sin_len` on Linux), `O_NONBLOCK`,
the errno constants, and `errno_slot` (`__error` vs `__errno_location`) on `OS`/`ARCH`,
mirroring the `inline if OS ==` pattern already proven in `event.sx`/`sched.sx`. No
silent fallback defaults (CLAUDE.md rule). Lock a Linux-vs-Darwin layout/const test red,
then green.
- **C3b — `thread.sx` per-OS.** Correct `MutexBuf`/`CondBuf` sizes per glibc (40/48) vs
Darwin (64/48), branched. Memory-safety fix, not cosmetics. Validate under the Apple
`container` Linux VM that the pool no longer corrupts the heap.
- **C4 — Linux CI.** A workflow building + running `zig build test` (incl. the HTTP corpus)
on Linux. The Apple-`container` path is proven for local validation; CI needs a real
Linux runner (GH Actions `ubuntu` and/or self-hosted aarch64). First CI of any kind for
the repo.
- **C5 — Linux socket I/O corpus.** Examples covering accept/read/write/close/error on
Linux (today only the macOS-friendly `1633` covers the happy path). Threaded-handler
example included.
- *Acceptance:* basic server compiles + runs on Linux; HTTP suite passes on Linux; accept/
read/write/close/error paths covered; threaded mode correct.
### Phase H — HTTP hardening (P0 #26)
- **H1 — Parser hardening (#2).** Max header-line size, max header count, strict
request-line + version validation, CRLF strictness, `Content-Length` overflow guard +
duplicate/conflicting rejection, **`Transfer-Encoding: chunked` → 501** (full impl in
S1), slowloris coverage (delivery-timeout already mitigates). Outcomes: 400 / 413 / 431 /
501 / safe-close. Unit + fuzz-seed corpus.
- **H2 — Memory lifecycle (#3).** Fix `Server.close()` to free `conns`, `PoolState`, and
`ps.done`. Document allocator ownership (long-lived containers must capture their owner —
CLAUDE.md rule; the read buffers are intentionally per-conn). Leak gate: start/stop loop
with GPA counters asserting zero + repaired stress script for RSS-over-churn.
- **H3 — Graceful shutdown (#4).** `Server.stop()` — stop accepting, drain in-flight within
a timeout, close idle keep-alives, return from `run()` cleanly. Tests: start/stop/restart
in one process; no FD leak; no mem leak.
- **H4 — Explicit errors + observability hooks (#5, #9).** Route accept/read/write/loop
faults through a pluggable log/error hook instead of silent close; add counters (active /
accepted / closed conns, requests served, parser errors, timeouts, rejected, 4xx/5xx,
pool queue depth, optional request duration). Hook-based — no forced logging format.
(#5 and #9 interlock; land together.)
- **H5 — Handler timeout + cancellation (#6).** Per-request deadline enforced in BOTH
inline and pool modes; bound time in `CONN_HANDLING`; timed-out → 504 or safe close. A
never-returning handler must not permanently consume capacity.
### Phase T — Native TLS via mbedTLS (#15) — revises the proxy-only posture
Native in-process HTTPS by binding a vetted C library (mbedTLS) over FFI — **not** a
pure-sx TLS stack (out of scope: security-critical, multi-year). Slotted after H because
TLS folds into the same connection state machine + `read_more`/`write_more` paths, which
must be stable first. Backend: **mbedTLS** (small pure-C, clean `WANT_READ`/`WANT_WRITE`
non-blocking API, static-links cleanly into `--self-contained` musl ELF; Apache-2.0).
- **T1 — mbedTLS FFI binding.** New `library/modules/ffi/mbedtls.sx` (or `std/tls.sx`):
`extern "c"` decls for `mbedtls_ssl_{init,setup,handshake,read,write,close_notify}`,
`mbedtls_ssl_config`, `mbedtls_x509_crt`, `mbedtls_pk_context`, `mbedtls_ctr_drbg` +
`mbedtls_entropy`, `mbedtls_ssl_set_bio`, and the `WANT_READ`/`WANT_WRITE` error
constants. Loud failure on any setup error (no silent default — CLAUDE.md rule).
- **T2 — Transport abstraction in `http.sx`.** Introduce a transport seam so `read_more`/
`write_more` go through plaintext (today's `socket.*_nb`) OR TLS, instead of calling the
socket directly. mbedTLS BIO callbacks bridge to the non-blocking fd: map socket
`WouldBlock``MBEDTLS_ERR_SSL_WANT_READ/WANT_WRITE`.
- **T3 — Handshake state + event-loop integration.** New `CONN_TLS_HANDSHAKE` state before
`CONN_READING`; drive `mbedtls_ssl_handshake` incrementally, mapping `WANT_READ`
`loop.add_read`, `WANT_WRITE``loop.add_write`; handshake deadline (reuse
`timeout_request_ms`); graceful `close_notify` on shutdown (ties into H3).
- **T4 — TLS config surface.** `Config` gains `tls_enabled`, cert/key/chain paths, min
version (default TLS 1.2+, prefer 1.3), optional ALPN, SNI (single default cert first;
multi-cert later). Cert/key load failure is a loud `HttpErr`, never a silent fallthrough.
- **T5 — Tests + static-link + Linux validation.** TLS corpus example: in-process mbedTLS
*client* handshakes against the server over loopback with a self-signed cert fixture
(under `examples/http/16xx-…/`); cover bad-cert, handshake-failure, and mid-handshake
client-abort paths. Verify a `--self-contained` static build links mbedTLS; run on macOS
+ aarch64 Linux (Apple `container`). Document the per-target mbedTLS static-archive
requirement for self-contained builds (vendor vs system).
### Phase S — Streaming, stress, stability (P1)
- **S1 — Streaming responses + chunked out (#10).** Explicit `Content-Length`; stream large
bodies without buffering the whole response; write-backpressure-aware send; header-set /
status / content-type helpers. (Builds on H1's chunked scaffolding.)
- **S2 — Request-body streaming (#11).** Incremental body reader, configurable max, early
reject, mid-body-disconnect handling, backpressure-aware reads. Enables real inbound
chunked bodies.
- **S3 — Fuzz harness (#7).** libFuzzer/AFL targets: request-line, header, `Content-Length`,
keep-alive + pipeline state machine, partial reads, malformed bodies, random close
timing. Runs manually + in CI. Crash/panic/hang = bug.
- **S4 — Load/stress suite (#8).** Repair + expand the stress scripts: many short-lived,
many keep-alive, slow clients, large bodies at the limit, pool saturation, FD exhaustion
→ 503/backpressure (not crash), RSS-over-time. Document expected overload behavior.
- **S5 — Concurrency model docs (#12).** Write up the allocator/thread-safety rules already
asserted in [thread.sx:11](../library/modules/std/thread.sx#L11) + the http.sx header:
handler execution model, per-request lifetime, what may be retained after a handler
returns, misuse cases.
- **S6 — API stability + security posture (#13, #14).** Tag a milestone; define the stable
std subset (`http`/`socket`/`event`/`thread`/`mem`); SECURITY.md + disclosure process +
the "reverse-proxy-only, not for direct internet exposure" posture statement + known
limitations.
### Phase D — Deploy & ergonomics (P2)
- **D1 — Reverse-proxy + deployment docs (#15, #20).** With native TLS shipping in Phase T,
proxy deployment is now *an option, not the only option* — document both. Cover proxy TLS
termination, forwarded headers, client-IP, size limits, timeouts, keep-alive, recommended
proxy settings; AND native-TLS direct-exposure guidance (cert rotation, cipher/version
policy). Plus systemd unit, Docker, health-check endpoint, graceful-shutdown, logging
examples; release-binary build + static/dynamic linking notes (incl. the mbedTLS
static-archive note from T5; cross-compile already in readme.md).
- **D2 — Routing + query/form helpers (#16, #17).** Thin layer over manual dispatch: method
+ path routing, path params, query parsing, 404/405, per-route limits/timeouts; form
(urlencoded/multipart) + JSON request/response helpers over the existing json.sx.
- **D3 — Honest benchmarks (#18).** Revive [bench/run.sh](../bench/run.sh): plain-text,
JSON, keep-alive, concurrency, pool, slow-client vs a baseline server; record hardware/
OS/flags/command; measure latency, throughput, memory, error rate.
- **D4 — Compiler-confidence framing (#19).** Largely already true (corpus + `issues/`
regressions, subprocess-isolated runner). Add the "supported vs experimental" labelling
for language + std features; ensure production-critical features have corpus coverage.
---
## Decisions Log (HTTPZ specifics)
- **Native TLS via an mbedTLS FFI binding (Phase T)** — supersedes the original
reverse-proxy-only posture (2026-06-26). The server gains in-process HTTPS; reverse-proxy
deployment stays supported and documented (D1) as an option. **No pure-sx TLS stack**
TLS is security-critical and is delegated to the vetted C library. mbedTLS chosen over
OpenSSL/LibreSSL for its small pure-C footprint, clean non-blocking `WANT_READ`/
`WANT_WRITE` API, and clean static-linking into `--self-contained` musl builds
(Apache-2.0).
- **`Transfer-Encoding: chunked`: reject (501) in H1, implement in S1/S2.** Pragmatic P0
minimum is explicit rejection; full chunked support is gated on the streaming work.
- **Stress/fuzz/load live outside the corpus.** The corpus runner has a 10s/example
timeout and no network sandbox; long-running adversarial harnesses are separate scripts
wired into CI, not `examples/`.
- **No silent fallback defaults in any C3 branching** (CLAUDE.md REJECTED PATTERNS): a
failed/unhandled OS or arch arm bails loudly, never picks a "reasonable-looking" Darwin
default.

View File

@@ -1,265 +0,0 @@
# PLAN-IO-UNIFY — fold the fiber scheduler behind `context.io`, re-home `race`
## Why
Today there are **two parallel async stacks**:
| stack | behind `context.io`? | real suspension? | cancellation channel |
|---|---|---|---|
| io.sx `async`/`await`/`cancel`/`Future` | yes (`impl Io for CBlockingIo`) | **no** — runs the worker inline to completion | `suspend_raw -> !` / `IoErr.Canceled` (designed, unused) |
| sched.sx `go`/`wait`/`cancel`/`race` (just landed) | **no** | yes (`swap_context` fibers) | none — `suspend_self -> void` |
`context.io` is structurally Zig's `std.Io` (an `Io` protocol carried *implicitly* in `Context` — better
ergonomics than Zig's explicit `io:` param), and the roadmap (§A5, §4.6) already says the fiber
scheduler should be **one of its `Io` vtables** and that `race` is **`context.io.race(..)` over Futures**.
The just-landed `race` on `sched.Scheduler` over `*Task` is the proven LOGIC at the wrong LAYER.
**Goal:** make the fiber `Scheduler` an `impl Io`, lift `async`/`await`/`cancel`/`race` onto the `Io`
protocol so they run colorblind under either impl, and let cancellation fall out of the existing
`suspend_raw -> !` contract (the "true cancellation, model A" the user picked — already the interface's
design). One async stack, behind `context.io`.
## The fiber → `Io` mapping (the crux)
`Io :: protocol { spawn_raw, suspend_raw -> !, ready, poll, now_ms, arm_timer }` (core.sx). Map each onto
the existing fiber primitives in sched.sx (`spawn`/`suspend_self`/`wake`/`sleep`/`block_on_fd`/`run`):
| `Io` method | fiber realization |
|---|---|
| `spawn_raw(entry, arg, opts) -> *void` | `spawn` a fiber whose body invokes `entry(arg)` (raw C-ABI thunk, not a closure — see Bridge below). Returns the `*Fiber` as the opaque handle. |
| `suspend_raw(park) -> !` | `suspend_self()`, then on resume CHECK the current task's cancel flag and `raise IoErr.Canceled` if set. `park.handle` = the `*Fiber` to re-ready. **This is the cancellation delivery point.** |
| `ready(park)` | `wake(park.handle as *Fiber)` (already guarded on `.suspended`). |
| `arm_timer(deadline_ms, park) -> *void` | arm a `Timer{deadline, fiber=park.handle}` (today's `sleep` minus the self-suspend); return the timer handle so a cancel can evict it. |
| `poll(deadline_ms) -> i64` | ONE iteration of the `run` loop: drain ready, then fire the earliest timer / block on fds up to `deadline_ms`. Returns the next pending deadline (or sentinel when idle). |
| `now_ms() -> i64` | the virtual `clock_ms` (deterministic), NOT a wall clock — keeps 1817/1821-style tests reproducible. |
`Scheduler.run()` stays as the explicit DRIVER (the top-level loop that calls `poll` to quiescence),
installed via `push Context { io = xx scheduler } { … s.run(); }` — exactly the existing sched examples,
just with the scheduler now reachable as `context.io`.
## Status (2026-06-28)
- **Follow-up — heap leak reclamation (fiber-env + async). DONE.** Closed the
documented per-spawn closure-env leak and most of the async leak, using only the
existing `closure.env`/`.fn_ptr` field accessors (now also named by
`ClosureRaw`/`SliceRaw` ABI-view structs in core.sx) — NO compiler change.
- **Fiber body env:** `Scheduler.reap_fiber` frees `f.body.env` via
`f.dctx.allocator` (the spawn-time allocator snapshotted in `dctx`) at all 3
reap sites. 1820's `live after deinit` 3 → **0**.
- **Async box + closure envs:** `sx_run_boxed_closure` frees the `ThunkBox`, the
completion-closure env, and the worker's env (new `ThunkBox.worker_env`) the
instant the worker completes.
- **Async Future:** two-flag ownership — `Future.worker_done` (set at the end of
the completion closure) + `consumed` (set at the end of `await`); `fut_release`
frees the heap `Future` (via the stored `Future.alloc`) when BOTH are set, so
the LAST of {worker, await} reclaims it. `await` now CONSUMES the future
(single-use; documented). Residual for an AWAITED future: **0** (lock:
`examples/concurrency/1827-...`). A NEVER-awaited future (fire-and-forget /
`race` loser) keeps only its `Future` struct (consumed never set) — the
structured-concurrency remainder, deferred.
- Self-reviewed across orderings (await-after/before-complete, cancel-then-await,
cancel-while-parked, double-free via await+deinit, race residual, blocking
impl, cross-allocator reap) — all deterministic, no UAF/double-free. Suite
855/0; byte-identical on aarch64-macOS + aarch64-linux; `.ir` churn (core.sx +
Future/ThunkBox field additions) regenerated, only 1820 stdout changed
otherwise.
- **Phase 5 — CONVERGE: retire the bespoke fiber async API. DONE. Io unification
COMPLETE.** The bespoke `Task` layer (`Task`/`TaskState`/`TaskErr`/`go`/`wait`/
`cancel(Task)` + `Scheduler.task_allocs` and its deinit handling, ~130 lines)
is removed from sched.sx. There is now ONE async stack: `context.io.async`/
`await`/`cancel`/`race`/`sleep` over the `Io` protocol, with the `Scheduler` as
the fiber Io's engine + driver (`spawn`/`yield_now`/`suspend_self`/`wake`/`run`/
`block_on_fd` stay as the raw primitives). Migrated the four `go`/`wait` users to
`context.io`: 1813 (interleave + cancel), 1817 (m1 end-to-end sum=123), 1819
(double-AWAIT loud-abort via the Future one-awaiter guard), 1820 (deinit — the
`go`/`task_allocs` tasks dropped; it now exercises timers/io_waiters/kq cleanup,
`freed=2`/`live=3`). `race` stays in sched.sx (needs meta.sx). Updated readme.md
(the user-facing async section now documents `context.io.async`/`await`/`race`/
`sleep`) and the stale `sched.go`/`sched.Task` comments in io.sx. Suite 854/0; no
`.ir` churn (the Task removal touched no snapshotted IR); migrated examples
byte-identical on aarch64-macOS + aarch64-linux. **PLAN-IO-UNIFY Phases 05 all
complete — the two parallel async stacks are now one, behind `context.io`.**
- **Phase 4 — `race` over Futures via `context.io.race`. DONE.** Re-homed the
proven first-wins race from `sched.race(*Task)` onto `*Future` handles + the
`Io` protocol; the old Task-based `race` is REPLACED (ufcs overload-by-receiver
is rejected — "duplicate top-level decl" — and only 1821 used it).
- **Protocol affordance:** added `Io.current_park() -> ParkToken` (the running
fiber as a token, captured WITHOUT parking) so race can register the SAME
coordinator across N futures' `park` slots, then park once via `suspend_raw`;
any completion `ready`s it. Scheduler returns `{self.current}` (bails outside
a fiber); CBlockingIo returns `{null}` (race never parks there — futures born
`.ready`). The await comment already anticipated this fan-in.
- **race** (`ufcs (io: Io, futures: $T) -> RaceResult(T)`, in sched.sx — it
needs meta.sx's `make_enum`/`make_variant`, and pulling that into the io.sx
prelude part-file would cycle): winner scan → register+park → deregister →
`make_variant` the winner → Phase-3 `cancel` each loser (NO join). `RaceResult`
reused unchanged (`*Future(R)` projects field 0 `value` → R).
- **Winner-time return:** with true cancellation the parked losers stop at their
next suspend (their timers evicted by cancel's wake), so race returns at the
winner's virtual time, not the slowest loser's. 1821 re-pointed to
`context.io.async` + `context.io.race`: `winner a=111`, losers `.canceled`,
completion log ONLY `task 1 @ 10ms`, final clock `10ms` (was 30 under the old
cooperative join). Byte-identical on aarch64-macOS + aarch64-linux. Suite
853/0; `.ir` churn (current_park vtable method) regenerated, only 1821 stdout
changed otherwise.
- **Phase 3 — TRUE cancellation via `suspend_raw -> !`. DONE.** A cancelled async
worker now abandons its body at its next suspend instead of running to
completion. Pieces:
- **Cancel-flag back-ref (D4 — back-ref pointer, chosen):** `SpawnOpts.cancel_flag:
*void` (core.sx) + `Fiber.cancel_flag: *void` (sched.sx), set from
`opts.cancel_flag` in `Scheduler.spawn_raw`. `async` passes `xx @f.canceled`
(the `Future.canceled` `Atomic(bool)` erased to `*void`).
- **Delivery:** `Scheduler.suspend_raw` checks `fiber_canceled(self.current)` (a
`*Atomic(bool)` load) PRE-park (raise without parking — no deadlock if cancel
landed before the worker ran) and POST-resume (cancel landed while parked),
raising `error.Canceled` (a bare `-> !`; set inferred). `cancel(f)` flips the
sticky flag, marks `.canceled`, and `ready(.{handle=f.task})`s the worker.
- **Worker is failable** `Closure() -> ($R, !)`: the `async` completion closure
`f.value = worker() catch { … }` (the captured-failable-closure-call the
Phase-3-prereq fix enabled) marks `.canceled`/`.failed` and wakes the awaiter;
the worker's post-suspend side effects never run. New failable `io.sleep(ms)`
(arm_timer + `try suspend_raw`) is the cancellation point.
- **Compiler gap fixed:** a `-> !` fn whose only error source is `try`-ing a
protocol method (`io.suspend_raw`) was wrongly flagged "declared `!` but never
errors". `collectErrorSites` (error_analysis.zig) now sets a `dyn` flag for a
`try` of a non-identifier callee (opaque error channel), suppressing the
warning.
- **Two UAFs found by adversarial review and FIXED:** (1) cancel-before-park
orphaned `io.sleep`'s armed timer → `suspend_raw`'s pre-park raise now evicts
the current fiber's timer/waiter first. (2) `cancel(f)` woke a possibly-reaped
worker → now only wakes when `was_pending` (`.pending` before the store).
- Migrated 1805/1806/1824 to failable workers. Lock:
`examples/concurrency/1825-concurrency-fiber-cancel-suspend.sx` (`seq: 1 -99`
— post-suspend line never runs). **Validated byte-identical on aarch64-macOS
host AND aarch64-linux container** (1824 + 1825). Suite 853/0. Expected `.ir`
churn (SpawnOpts layout) regenerated; no non-`.ir` snapshot changed.
- **Phase 3 PREREQUISITE — captured-failable-closure call typing. DONE.** The
async completion closure (`b.run = () => { f.value = worker() catch {…} }`)
captures a failable `worker` and consumes its error channel; the free-variable
capture analysis (`collectCaptures` in `src/ir/lower/closure.zig`) did not
descend into the error-handling / context / asm / multi-assign nodes, so
`worker` was never captured — inside the lambda it resolved against an empty
scope and typed as `.unresolved` (`catch`/`try` then rejected it). Fixed: added
`try_expr`, `catch_expr`, `onfail_stmt`, `raise_stmt`, `multi_assign`,
`push_stmt`, `comptime_expr`, `insert_expr`, `spread_expr`, `asm_expr` arms to
`collectCaptures`. Adversarially reviewed (captures resolve, locals correctly
excluded, no false-positive captures, 851/0). Lock: example
`examples/closures/0314-closures-capture-failable-call.sx` (catch + try over a
captured failable closure; pure language feature, host-only). The `push_stmt`
arm also fixes the previously-noted "free-var analysis doesn't descend into a
nested `push Context {…}`" gap. **Phase 3 is now unblocked.**
- Two PRE-EXISTING, orthogonal bugs surfaced during review (neither blocked
Phase 3): (1) calling a closure stored in a **struct data field** typed as
`unresolved` (value → garbage; failable → can't `catch`) — **RESOLVED**
(`issues/0201`): `CallResolver.plan` gained a closure/fn-pointer field arm and
the lowering closure-field arm now also handles bare `.function` fields;
regression `examples/closures/0315-closures-struct-field-call.sx`. (2) asm
write-through place through a deref (`asm { … "+r" -> @(p.*) }`) fails LLVM
verification — repros with NO closure (independent of capture analysis);
possibly an unsupported deref-place form rather than a confirmed bug, not
filed.
## Status (2026-06-27)
- **Phase 0 — fibers inherit the spawn-time context. DONE** (`2f2d7f1d`). Discovered during Phase 1: a
fiber body ran under `__sx_default_context` (the `abi(.c)` `fib_dispatch` dropped the implicit
context), so a scheduler installed as `context.io` was invisible inside a worker. Fixed:
`Scheduler.spawn` snapshots `context` → `Fiber.dctx`; `fib_dispatch` re-pushes it. Behavior-preserving
(suite 828/0), no cross-fiber leak (context is parameter-threaded per stack). Lock: example 1822.
- **Phase 1 — `impl Io for Scheduler`. DONE** (`5c30bfe0`, hardened `da7dd1f1`). Six methods over the
fiber primitives; `spawn_raw` bridges the erased `(*void)->void` worker thunk via an fn-ptr round-trip.
Lock: example 1823 (spawn→arm→suspend→ready→resume entirely through `context.io`, deterministic).
Adversarial review fixed: `arm_timer`/`spawn_raw` null guards, `poll` fd-pending abort + `deadline_ms`
doc, stale `fib_dispatch` comment.
- **Resolved design decisions:** D1 = direct `impl Io for Scheduler` (chosen). D2 = `now_ms` returns the
virtual `clock_ms` (deterministic) — a real-clock variant is later. D4 = deferred to Phase 3.
- **Phase 2 — `async`/`await` colorblind over the fiber Io. DONE** (`967aed67`, hardened `ada8d162`).
`async` heap-allocs a `*Future`, boxes a completion closure in a monomorphic `ThunkBox`, and submits
via `io.spawn_raw` (inline under `CBlockingIo`, a fiber under the scheduler); `await` parks via
`suspend_raw` until ready. Protocol changed to `suspend_raw(park: *ParkToken)` (write-back of the
awaiter). Workers are nullary (call-site capture). Migrated 1805/1806; adopted `push .{ … }`. Lock:
example 1824 (deferral visible: `1 2 10 20 123`). Review fixed: one-awaiter `await` guard; documented
the Future allocator-lifetime contract + that `cancel` doesn't stop an already-spawned worker (Phase 3).
- **Resolved D2 (ParkToken):** `suspend_raw(*ParkToken)` write-back (chosen over a registry). **ready()
liveness (CONCERN 6):** safe for single async/await (awaiter is suspended, not reaped, when readied);
`race` fan-in must still deregister (Phase 4).
- **Carried to convergence:** `async` should capture the scheduler's long-lived allocator (like
`sched.go`'s `own_allocator`) instead of the call-site `context.allocator` — needs a protocol
affordance; documented as a contract for now.
- **Open for later phases:**
- **ParkToken↔fiber binding.** `ready(park)` needs `park.handle` = the awaiter `*Fiber`. The scheduler
knows `self.current` at suspend; the cleanest is `suspend_raw(park: *ParkToken)` writing
`park.handle = self.current` before parking (a small protocol change: the materializer installs
thunks by name/order, signature-agnostic — verified low-risk). Decide vs a token→fiber registry.
- **`ready()` liveness (review CONCERN 6).** Casting a stale/reaped `*Fiber` handle and `wake`-ing it is
a latent UAF once real `await` runs — `wake`'s `.suspended` value-check on freed bytes is luck, not
safety. Phase 2 must guarantee single-ready / deregistration (mirror the bespoke-race deregister).
- **Out-of-scope compiler bug found by review (not filed yet):** closure free-var analysis does not
descend into a nested `push Context {…}` block inside a closure body — a var used only there reports
`unresolved`. Phase 0 sidesteps it (capture is at the `Fiber` level, not via closure), so it does NOT
block the unification; worth an `issues/` entry in a separate session.
## Phases (each: implement → lock with an example → `zig build test` green → both platforms)
1. **`impl Io for Scheduler` (the vehicle).** Implement the six methods over the fiber primitives. Add
a `Fiber.canceled`/task back-ref so `suspend_raw` can raise on resume. Keep `CBlockingIo` intact.
Lock: install the fiber Io into `context.io`, run a root fiber that `suspend_raw`s and is `ready()`'d —
asserts real park/resume through the protocol (not inline). **Bridge** (the one fiddly bit): `async`'s
generic `Closure(..$args) -> $R` worker → `spawn_raw`'s raw `entry/arg`. Box the worker thunk on the
heap; `entry` is a C-ABI `(env: *void) -> void` invoke-thunk (mirrors `fib_dispatch`), `arg` is the env.
2. **`async`/`await` over the fiber Io (real interleaving).** Under a suspending Io, `async` calls
`spawn_raw` and returns a PENDING `Future($R)` (no longer born `.ready`); the spawned body fills
`f.value`/`f.state` and `ready(f.park)`s the awaiter. `await(f)` checks `.ready` else `suspend_raw(f.park)`
then returns/raises — the suspending sibling of today's immediate `await`. `CBlockingIo` keeps the
run-inline path (degenerate, still correct). Lock: two `context.io.async` tasks interleave under the
fiber Io (the io.sx layer, replacing the bespoke `sched.go`).
3. **True cancellation via `suspend_raw -> !`.** `cancel(f)` flips `f.canceled` AND `ready(f.park)`s /
wakes the worker fiber so its NEXT `suspend_raw` raises `IoErr.Canceled`. The worker's suspends
(`await`, a future `io.sleep`) propagate via `try`/`!`; the worker body unwinds, the future ends
`.canceled`, its post-cancel side-effects DON'T run. This is the model-A "true cancellation" — now
delivered through the protocol, not bespoke. Lock: a cancelled task's work stops at its next suspend
(assert via a shared log: the post-suspend line never prints).
4. **`race` over Futures — `context.io.race((a: fa, b: fb))`.** Re-home the proven race logic (winner
scan, deregister-all-on-wake, structured cancel+join of losers) from `sched.race(*Task tuple)` onto
`*Future` handles + the `Io` protocol. The type-level machinery ports UNCHANGED — `RaceResult($T)`,
`make_variant`, the tuple reflection (GAP 1/2, all landed) — only the runtime swaps `*Task`→`*Future`
and `suspend_self`→`suspend_raw`/`ready`. Cancellation of losers now uses Phase 3 (their next suspend
raises), so `race` returns at WINNER-time, not slowest-loser-time. Lock: re-point 1821 at
`context.io.race`; assert winner value + losers' work stopped (not merely flagged).
5. **Converge — retire the bespoke fiber async API.** Fold `sched.go`/`wait`/`cancel`/`race` into the
io.sx layer; `Scheduler` stays as the fiber Io's engine + driver. Migrate 18111821 to the
`context.io` API. One async stack, all behind the protocol. Update the roadmap/checkpoints.
## Open decisions (need a call before/within the phase noted)
- **D1 (Phase 1) — `impl Io for Scheduler` vs a `FiberIo` wrapper.** Direct impl makes `context.io` BE the
scheduler (`xx scheduler` as the Io value, stateful receiver — mirrors the allocator `xx local` rule).
A wrapper adds a level but decouples the public Io vtable from the scheduler internals. *Lean: direct
impl* (simplest, matches the allocator convention).
- **D2 (Phase 1) — virtual vs real clock under the fiber Io.** Tests need the deterministic virtual clock
(`clock_ms`); a real deployment wants `time.mono_ms`. Thread it as a Scheduler mode, or two Io impls
(`FiberIo` virtual-clock for tests, real-clock for prod). *Lean: a `clock: enum { virtual; real }` field
so one impl serves both; tests pin `.virtual`.*
- **D3 (Phase 2) — `Future(void)` (issue 0150 SIGTRAP).** A `void`-result task can't build `Future(void)`
today. Defer (race/async target non-void), or fix the `void` struct-field path. *Lean: defer, gate with
a diagnostic.*
- **D4 (Phase 3) — where the cancel flag lives.** The `Future` already has `canceled: Atomic(bool)`; the
fiber needs to reach it from `suspend_raw`. Give `Fiber` a `*Atomic(bool)` back-ref to its future's flag
(set at `spawn_raw`), so `suspend_raw` consults it with no per-suspend lookup. *Lean: back-ref pointer.*
## Validation (every phase)
- `zig build && zig build test` green (full corpus).
- New/changed `18xx` examples byte-identical on aarch64-macOS host AND aarch64-linux container
(deterministic virtual clock).
- Adversarial review of each phase (worker + read-only reviewer), per the session workflow.
## What this supersedes
- `sched.sx`'s bespoke `go`/`wait`/`cancel`/`race` (Phase 5 retires them; the proven logic moves onto the
protocol). The just-landed `race` (commit `9099735e`) is the reference logic for Phase 4, not the final
home.
- PLAN-RACE.md's "race on `sched.Scheduler`" framing — this plan moves it onto `context.io` per the
roadmap's §A5 / §4.6 design-of-record.

142
current/PLAN-METATYPE.md Normal file
View File

@@ -0,0 +1,142 @@
# PLAN-METATYPE — comptime type metaprogramming (`declare` / `define` + reflection)
## Goal
Comptime type metaprogramming with the smallest possible compiler surface:
- **`declare(name) -> Type`** — mint a NEW empty (undefined) nominal type NAMED
`name`, returned as a first-class `Type` handle. The compiler registers the
forward type at compile time, so the body can reference it (`*Name`).
- **`define(handle, info) -> Type`** — fill a declared handle's body from a
`TypeInfo` *value*, and return the handle (so the one-shot form chains).
- **`type_info($T) -> TypeInfo`** — reflect a type INTO data (the inverse of
`define`'s decode). *Done for enums* (`interp.zig:reflectTypeInfo`,
`examples/0619`); struct/tuple widening pending.
- **`field_type($T, i) -> Type`** — the i-th field / variant-payload / element
type of `$T`. *Done.*
These four `#builtin`s in `library/modules/std/meta.sx` are the **entire**
compiler surface. Every higher-level constructor is **plain sx built over
`declare`/`define`** — the compiler knows none of them by name:
```sx
// one-shot (non-recursive): declare + define chained, define returns the handle
T :: define(declare("T"), .enum(.{ variants = .[ … ] }));
// recursive: a ctor fn names the forward type via declare, references it as *Name
List :: make_list();
make_list :: () -> Type {
h := declare("List");
return define(h, .enum(.{ variants = .[
EnumVariant.{ name = "cons", payload = *List }, // self-reference
EnumVariant.{ name = "nil", payload = void } ] }));
}
// type-fns are ordinary sx (channel result types, etc.)
RecvResult :: ($T: Type) -> Type {
return define(declare("RecvResult"), .enum(.{ variants = .[
EnumVariant.{ name = "value", payload = T },
EnumVariant.{ name = "closed", payload = void } ] }));
}
```
This gates channel result types (`RecvResult($T)`) and `race`'s synthesized
tagged-union (design [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §7 step 3), and replaces a would-be `enum($T)` language feature.
## How it works (the locked design)
1. **Two comptime interp builtins.** `declare` mints an empty `tagged_union` slot
in the type table; `define` decodes the `TypeInfo` value (variant-name strings +
payload `Type`-tags) and completes the slot byte-identical to a source enum's
`buildEnumInfo` output, so it flows through enum codegen unmodified. The interp
mutates the type table via a `mint` handle the host sets (`setMintTable`).
2. **No syntactic constructor recognition.** A `::` binding or type-fn body that
calls a `Type`-returning fn is **comptime-evaluated** (`evalComptimeType`): the
expression runs through the interpreter, the `declare`/`define` builtins mint the
type, and the result `type_tag` is bound. `decl.zig` triggers on a non-generic
`-> Type` fn call; `instantiateTypeFunction` triggers on a type-fn body that
returns a `define(…)` call (or a bodied `-> Type` helper) — see
`generic.zig:returnExprMintsType`.
3. **Name on `declare`.** `declare("Name")` carries the name as a compile-time
string so `preregisterForwardTypes` (in `evalComptimeType`) can register the
forward type — and bind it as a type alias — BEFORE the body lowers. That's
what makes a `*Name` self-reference resolve (a `Name :: ctor()` decl makes
`Name` a const_decl author, so `*Name` resolves through the forward-ALIAS path;
the alias binding, not just the table registration, is what satisfies it). The
interp's `declare` returns the same slot by name; `define` fills it in place.
4. **Nominal identity** rides the existing type-fn mangled-name instantiation cache:
`RecvResult(i64)` at two sites memoizes to ONE `TypeId` (the body runs once;
`renameNominalType` re-keys the minted type to the mangled name).
5. **Comptime-only, JIT-free.** `declare`/`define` are interp ops; reaching them at
runtime / emit is a hard error.
6. **Undefined-until-defined.** `declare()` mints an undefined slot; *using* it
(construct / match / size) before its `define` is a loud diagnostic. A *pointer*
to an undefined slot (`*Self`) is fine — that's what self-reference needs.
## Key code anchors
- Builtins: `BuiltinId.declare` / `.define` (`src/ir/inst.zig`); lowering to
`callBuiltin` (`src/ir/lower/call.zig:tryLowerReflectionCall`); interp exec +
`defineEnum` + `decodeVariantElements` (`src/ir/interp.zig`); `mint` field +
`setMintTable`.
- Comptime evaluation: `evalComptimeType` / `renameNominalType`
(`src/ir/lower/comptime.zig`); decl trigger `fnReturnsTypeValue`
(`src/ir/lower/decl.zig`); type-fn trigger `returnExprMintsType` +
`instantiateTypeFunction` (`src/ir/lower/generic.zig`).
- Reflection: `field_type``fieldTypeOf` (`src/ir/lower/generic.zig`).
- Surface: `library/modules/std/meta.sx` (on-demand import — NOT the prelude, to
avoid shifting every `.ir` snapshot).
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior
lock). `zig build && zig build test` after every step. Never regenerate snapshots
while red. Examples: `06xx` (comptime), `11xx` (diagnostics).
## Status
- [x] `declare` / `define` comptime builtins + the `mint` plumbing.
- [x] Comptime evaluation of a `Type`-returning `::` RHS and type-fn body
(the only triggers; no constructor-name knowledge in the compiler).
- [x] Name-in-`TypeInfo`; nominal identity via the instantiation cache.
- [x] `field_type` reflection (`examples/0616`).
- [x] Examples green on the floor: `0614` (one-shot), `0615` (type-fn identity),
`0617` (channel result types).
- [x] **Self-reference** — recursive enums via `declare("Name")` + `*Name` in a
constructor fn (`preregisterForwardTypes` registers the forward type + alias
before the body lowers). `examples/0618` (recursive `*List`: construct, match
through the pointer, recursive traversal). Mutual recursion / by-value-self-ref
rejection fall out of the same mechanism (F5 adds the loud by-value check).
- [x] **`make_enum(name, variants: []EnumVariant)`** — the general enum constructor
over a COMPUTED (value, non-literal) variant list. Pure sx in `meta.sx`;
exercises `define` decoding a value-arg slice. `examples/0620` (array-literal
local) / `0624` (generic builder).
- [x] **Comptime slice over a non-string aggregate**`arr[lo..hi]` over an array
yields a real slice value at comptime (`base_ty` threaded onto `Subslice`;
open-ended `hi` folded to the array's static length; `subsliceElements`).
`examples/0621`.
- [x] **`type_info($T) -> TypeInfo`** — reflect `enum`/`tagged_union`/`struct`/`tuple`
INTO a value (inverse of `define`'s decode); `define` decodes all three back
(`defineEnum`/`defineStruct`/`defineTuple`, dispatched on the TypeInfo tag).
Round-trips: `examples/0619` (enum) / `0622` (struct) / `0623` (tuple). The
reflect/construct triad is complete.
- [x] **Generic type-fn body locals** — a generic `($T) -> Type` comptime-evaluates
its FULL body (prelude statements + return), so a local before the return
resolves (`createComptimeFunctionWithPrelude` / `evalComptimeTypeBody`).
`examples/0624`.
- [x] **Validation + loud diagnostics** — by-value self-reference (`checkInfiniteSize`,
source `1178` + constructed `1182`; issue 0139), duplicate variant/field names
(`1180`), `declare()` never `define()`d (`1181`, was a `verifySizes` panic),
and the 0140 bail-surfacing (`1179`). use-before-define is subsumed by these
(no new check needed). Validation story COMPLETE.
- [ ] **Comptime `List` growth** (issue 0141, DEFERRED) — `List(T).append` at
comptime bails (two layers: null comptime allocator at scanDecls + `*T`
slot_ptr `struct_get`). Non-blocking; array-literal locals cover the use case.
## Risks / watch
- **Self-ref timing** — `define` for the two-statement form must complete before any
code uses the type's layout; a use-before-define must be a loud diagnostic, not a
silent empty enum.
- Keep `declare`/`define` **comptime-only**: reaching them at runtime is a hard error
(emit should bail loudly if one ever leaks into codegen).

View File

@@ -17,7 +17,7 @@ E :: error { Neg }
const_one :: () -> i64 { return 1; return 99; }
// dead `return x;` after an unconditional raise (the failable closure shape)
always_raise :: (x: i64) -> (i64, !E) { raise error.Neg; return x; }
always_raise :: (x: i64) -> i64 !E { raise error.Neg; return x; }
// guard: a conditional return must still fall through to the trailing return
clamp :: (x: i64) -> i64 { if x > 10 { return 10; } return x; }

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -10,7 +10,7 @@ Ctx :: struct {
}
main :: () -> i32 {
c : Ctx = .{ on = (f: Fmt) {
c : Ctx = .{ on = (f: Fmt) => {
n : i64 = xx f;
print("cl f = {}\n", n);
}};

View File

@@ -31,7 +31,7 @@ ticks : i32 = 0;
main :: () -> i32 {
h : Holder = .{};
h.set(() { ticks += 1; });
h.set(() => { ticks += 1; });
h.call_direct();
h.call_hoisted();

View File

@@ -11,7 +11,7 @@
#import "modules/std.sx";
main :: () {
pick := (p: ?i64) -> i64 {
pick := (p: ?i64) -> i64 => {
if p == null { return -1; }
return p; // narrowed inside the lambda body
};

View File

@@ -1,44 +0,0 @@
// A captured FAILABLE closure stays failable when CALLED inside a nested
// closure body. The free-variable capture analysis must descend into the
// error-handling expressions (`catch`, `try`) that the nested closure uses to
// consume the captured worker's error channel — otherwise the worker is never
// captured into the env, resolves against an empty scope inside the lambda, and
// the call types as `unresolved` (so `catch`/`try` reject it).
//
// Regression (PLAN-IO-UNIFY Phase 3 blocker): the async completion closure
// `() { f.value = worker() catch {…} }` captures a `Closure() -> ($R, !)`
// worker and consumes its error channel — exactly this shape.
#import "modules/std.sx";
Box :: struct { run: Closure() -> void; }
// `catch` path: the nested closure absorbs the worker's error.
run_catch :: (worker: Closure() -> (i64, !)) {
b : Box = ---;
b.run = () {
v := worker() catch {
print("caught\n");
return;
};
print("ok {}\n", v);
};
b.run();
}
// `try` path: the nested closure is itself failable and propagates.
mk_trier :: (worker: Closure() -> (i64, !)) -> Closure() -> (i64, !) {
return () -> (i64, !) {
v := try worker();
v + 100
};
}
main :: () -> i64 {
run_catch(() -> (i64, !) { 7 }); // ok 7
run_catch(() -> (i64, !) { raise error.Bad; }); // caught
t := mk_trier(() -> (i64, !) { 5 });
r := t() catch { return 1; };
print("try {}\n", r); // try 105
return 0;
}

View File

@@ -1,41 +0,0 @@
// Calling a closure / function-pointer value stored in a STRUCT FIELD
// (`box.run(args)`) resolves the call's return type correctly — value returns
// marshal properly, and a failable field (`Closure(..) -> (T, !)`) is `try`/
// `catch`-able. The call-type resolver mirrors the lowering dispatch: a
// closure/fn-ptr field is called directly (and shadows a same-named method).
//
// Regression (issue 0201): the field-access call path typed such calls as
// `unresolved` — value returns came out as garbage, failable returns rejected
// `catch`/`try` ("operand has type 'unresolved'").
#import "modules/std.sx";
CB :: struct {
add: Closure(i64, i64) -> i64; // closure field, with args
fp: (i64) -> i64; // bare function-pointer field
work: Closure(i64) -> (i64, !); // failable closure field
}
triple :: (x: i64) -> i64 { return x * 3; }
// Field call through a `*CB` receiver inside a method, consuming the failable
// field's error channel.
run_work :: (self: *CB, n: i64) -> i64 {
v := self.work(n) catch { return -1; };
return v;
}
main :: () -> i64 {
b : CB = ---;
b.add = (x: i64, y: i64) => x + y;
b.fp = triple;
b.work = (n: i64) -> (i64, !) {
if n < 0 { raise error.Negative; }
n * 10
};
print("{}\n", b.add(3, 4)); // 7
print("{}\n", b.fp(5)); // 15
print("{}\n", run_work(@b, 6)); // 60
print("{}\n", run_work(@b, -1)); // -1 (error path)
return 0;
}

File diff suppressed because one or more lines are too long

View File

@@ -1,3 +0,0 @@
ok 7
caught
try 105

View File

@@ -1,47 +0,0 @@
// Comptime field reflection (`field_count` / `field_name` / `field_type`) over
// ALL aggregate kinds — struct, enum, tuple (positional + named), array, vector.
//
// Regression (issue 0195): `field_count` / `field_name` were broken on tuples
// and arrays/vectors. `field_count` silently returned 0 (a missing `.tuple` arm
// in the count switches), and `field_name` SEGFAULTED — the LLVM backend built a
// zero-length name array for those kinds while sizing the GEP at the (sometimes
// non-zero) count, so `field_name(T, i)` indexed past a `[0 x string]` global.
// Fixed by driving BOTH the name-array build and the GEP sizing from the one
// source of truth (`memberCount`/`memberName`), so they can never diverge again.
// A member with no name (positional-tuple / array / vector element) reflects as
// the empty string "" — one slot per member, always in-bounds.
#import "modules/std.sx";
S :: struct { a: i64; b: bool; }
E :: enum { X; Y; Z; }
main :: () -> i32 {
// struct: named fields
print("struct: fc={} fn=({},{}) ft0={}\n",
field_count(S), field_name(S, 0), field_name(S, 1), type_name(field_type(S, 0)));
// enum: variant names
print("enum: fc={} fn=({},{},{})\n",
field_count(E), field_name(E, 0), field_name(E, 1), field_name(E, 2));
// positional tuple: element types, no names → ""
print("postuple: fc={} ft=({},{}) fn0=[{}]\n",
field_count(Tuple(i64, bool)),
type_name(field_type(Tuple(i64, bool), 0)), type_name(field_type(Tuple(i64, bool), 1)),
field_name(Tuple(i64, bool), 0));
// named tuple: element labels recovered
print("namtuple: fc={} fn=({},{})\n",
field_count(Tuple(a: i64, b: bool)),
field_name(Tuple(a: i64, b: bool), 0), field_name(Tuple(a: i64, b: bool), 1));
// array: length-many elements, type known, no names → ""
print("array: fc={} ft0={} fn0=[{}]\n",
field_count([4]i64), type_name(field_type([4]i64, 0)), field_name([4]i64, 0));
// vector: same shape as array
print("vector: fc={} ft0={} fn0=[{}]\n",
field_count(Vector(4, f32)), type_name(field_type(Vector(4, f32), 0)), field_name(Vector(4, f32), 0));
return 0;
}

View File

@@ -1,26 +0,0 @@
// `pointee($P: Type) -> Type` — comptime reflection for a pointer's target type
// (`pointee(*X)` -> `X`). Folds at lower time like `field_type`, so it composes
// inside other type-arg slots — e.g. project a generic handle `*Box(A)` to its
// payload type `A` via `field_type(pointee(*Box(A)), 0)`. (Foundation for the
// `race` result synthesis, which projects `*Task(A)` -> `A`.)
#import "modules/std.sx";
Box :: struct ($R: Type) { value: R; tag: i64; }
// Project a pointer-to-Box to the Box's payload type: *Box(A) -> A.
Payload :: ($P: Type) -> Type { return field_type(pointee(P), 0); }
main :: () -> i32 {
// plain pointer
print("pointee(*i64) = {}\n", type_name(pointee(*i64)));
print("pointee(*bool) = {}\n", type_name(pointee(*bool)));
// pointer to a generic struct → the struct type
print("pointee(*Box(f64)) field0 = {}\n", type_name(field_type(pointee(*Box(f64)), 0)));
// composed projection, used as a real type
v : Payload(*Box(i64)) = 42;
print("Payload(*Box(i64)) value = {}\n", v);
return 0;
}

View File

@@ -1,39 +0,0 @@
// Int-returning type-query builtins (`field_count` / `size_of` / `align_of`)
// fold as comptime CONSTANTS — usable as an `inline for` bound and an array
// dimension, exactly like a plain `K :: 3` const. Previously
// they evaluated only as runtime values, so `[field_count(S)]T` and
// `inline for 0..field_count(S)` were rejected as "not a compile-time integer".
// (This is what lets a `($T) -> Type` builder loop `inline for 0..field_count(T)`
// to assemble a variant list from a type's members — the `race` result synthesis.)
#import "modules/std.sx";
S :: struct { a: i64; b: bool; c: f64; }
E :: enum { X; Y; Z; W; }
main :: () -> i32 {
// field_count as an inline-for bound
s := 0;
inline for 0..field_count(S) (i) { s = s + i; } // 0+1+2 = 3
print("field_count(S) loop sum = {}\n", s);
// field_count as an array dimension; fill it in a folded loop
xs : [field_count(S)]i64 = ---;
inline for 0..field_count(S) (i) { xs[i] = i * 10; }
print("array[field_count(S)] len = {} xs[2] = {}\n", xs.len, xs[2]);
// field_count of an enum (4 variants) driving a loop
e := 0;
inline for 0..field_count(E) (i) { e = e + 1; }
print("field_count(E) = {}\n", e);
// size_of / align_of fold too
bytes : [size_of(i64)]u8 = ---;
print("size_of(i64) array len = {}\n", bytes.len);
print("align_of(f64) = {}\n", align_of(f64));
// composed const expression as a dim
ys : [field_count(S) + 1]i64 = ---;
print("[field_count(S) + 1] len = {}\n", ys.len);
return 0;
}

View File

@@ -1,48 +0,0 @@
// Comptime type-call COMPOSITION: a `($T) -> Type` builder reflects a named
// tuple, projects each element type through `pointee` + `field_type`, and mints a
// tagged-union whose variant labels mirror the tuple's labels — the shape the
// `race` result synthesis needs (`(a: *Task(A), b: *Task(B))` -> `{ a: A; b: B }`).
//
// Exercises three things that previously failed when the index was an `inline for`
// loop var: a type-call RESULT used as (1) a `Type`-typed struct field value
// (`payload = field_type(...)`), (2) a nested type-call arg
// (`field_type(pointee(field_type(T, i)), 0)`), and a `field_name(T, i)` folded to
// a comptime string for a minted variant NAME. All resolve through the same
// type-call fold as a literal index would.
#import "modules/std.sx";
#import "modules/std/meta.sx";
// Stand-in for a task handle: a pointer to a generic box carrying the result.
Box :: struct ($R: Type) { value: R; }
// Mint a tagged-union mirroring a named tuple of `*Box(..)` handles:
// variant name = tuple label, payload = the box's value type (`*Box(A)` -> `A`).
ResultOf :: ($T: Type) -> Type {
vs : [field_count(T)]EnumVariant = ---;
inline for 0..field_count(T) (i) {
vs[i] = EnumVariant.{
name = field_name(T, i), // folded to a const string
payload = field_type(pointee(field_type(T, i)), 0), // *Box(A) -> Box(A) -> A
};
}
return make_enum("ResultOf", vs[0..field_count(T)]);
}
R :: ResultOf(Tuple(a: *Box(i64), b: *Box(bool), c: *Box(f64)));
use :: (r: R) {
if r == {
case .a: (v) { print("a (i64) = {}\n", v); }
case .b: (v) { print("b (bool) = {}\n", v); }
case .c: (v) { print("c (f64) = {}\n", v); }
}
}
main :: () -> i32 {
use(.a(42));
use(.b(true));
use(.c(2.5));
print("R: variants={} names=({},{},{})\n",
field_count(R), field_name(R, 0), field_name(R, 1), field_name(R, 2));
return 0;
}

View File

@@ -1,63 +0,0 @@
// `make_variant($E, idx, payload)` (modules/std/meta.sx) — the WRITE side of the
// metatype reflection triad: construct a value of a MINTED tagged-union by
// VARIANT INDEX, when the variant is chosen at runtime and the union was
// synthesized at comptime (so its labels can't be a literal `.label(…)`). This
// is the shape the `race` result uses: an `inline for 0..N (i)` arm builds the
// i-th variant of a synthesized result carrying the winner's value.
//
// Covers heterogeneous + COMPLEX payloads (multi-field struct, string fat
// pointer, a larger struct, scalar) — make_variant zeroes the value then writes
// the i64 tag @0 and the payload @ size_of(i64), so payloads of any size/shape
// round-trip. Built with the natural early-return-per-arm pattern (a `return`
// inside an `inline if` inside an `inline for`).
#import "modules/std.sx";
#import "modules/std/meta.sx";
Vec3 :: struct { x: f64; y: f64; z: f64; } // 24 bytes
Big :: struct { a: i64; b: i64; c: i64; d: i64; tag: bool; } // 40 bytes (largest payload)
// A synthesized 4-variant tagged-union with complex, differently-sized payloads.
R :: make_enum("R", .[
EnumVariant.{ name = "v", payload = Vec3 },
EnumVariant.{ name = "s", payload = string },
EnumVariant.{ name = "big", payload = Big },
EnumVariant.{ name = "n", payload = i64 },
]);
// Build the variant chosen by a RUNTIME index, in the matching unrolled arm —
// each arm's payload type differs. The `return` inside the `inline if` inside the
// `inline for` is the pattern make_variant exists to enable.
pick :: (idx: i64, vv: Vec3, sv: string, bv: Big, nv: i64) -> R {
inline for 0..field_count(R) (i) {
if idx == i {
// Comptime match on the loop cursor `i` selects the arm whose payload
// type matches variant `i` — cleaner than nested `inline if`/`else`.
inline if i == {
case 0: { return make_variant(R, i, vv); }
case 1: { return make_variant(R, i, sv); }
case 2: { return make_variant(R, i, bv); }
else: { return make_variant(R, i, nv); }
}
}
}
return make_variant(R, 3, -1); // unreachable for a valid idx
}
show :: (r: R) {
if r == {
case .v: (p) { print("v = {} {} {}\n", p.x, p.y, p.z); }
case .s: (p) { print("s = {}\n", p); }
case .big: (p) { print("big = {} {} {} {} {}\n", p.a, p.b, p.c, p.d, p.tag); }
case .n: (p) { print("n = {}\n", p); }
}
}
main :: () -> i32 {
v :: Vec3.{ x = 1.5, y = 2.5, z = 3.5 };
b :: Big.{ a = 10, b = 20, c = 30, d = 40, tag = true };
show(pick(0, v, "hello", b, 99));
show(pick(1, v, "hello", b, 99));
show(pick(2, v, "hello", b, 99));
show(pick(3, v, "hello", b, 99));
return 0;
}

View File

@@ -1,39 +0,0 @@
// Regression: a `return` inside an `inline if` (a comptime-folded branch),
// itself inside an `inline for`, must NOT make the compiler drop the function's
// trailing statements. The `inline if`/`case` branch sets a "block terminated"
// flag when its taken arm returns; that flag used to leak past the enclosing
// runtime `if`'s merge block, so the trailing `return -1` was skipped and the
// function was wrongly rejected as "produces no value". Now the runtime-`if`
// merge resets the flag to the merge's actual reachability.
#import "modules/std.sx";
// nested inline-if/else with returns, inside an inline-for, under a runtime if:
classify :: (idx: i64) -> i64 {
inline for 0..3 (i) {
if idx == i {
inline if i == 0 { return 100; }
else { inline if i == 1 { return 200; } else { return 300; } }
}
}
return -1; // trailing statement — must still be emitted (idx out of range)
}
// the comptime `case` match form, also with per-arm returns:
tag :: (idx: i64) -> i64 {
inline for 0..3 (i) {
if idx == i {
inline if i == {
case 0: { return 10; }
case 1: { return 20; }
else: { return 30; }
}
}
}
return -1;
}
main :: () -> i32 {
print("classify: {} {} {} {}\n", classify(0), classify(1), classify(2), classify(9));
print("tag: {} {} {} {}\n", tag(0), tag(1), tag(2), tag(9));
return 0;
}

View File

@@ -1,40 +0,0 @@
// Comptime-cursor indexing of a named-tuple VALUE: `tup[i]` where `i` is an
// `inline for` cursor (or a literal) reads the i-th tuple field with its
// CONCRETE type — not a type-erased `Any`. This is the read side `race` needs:
// pull the i-th `*Task(T_i)` handle out of a named-tuple param keeping its real
// type, so `field`/method access on it resolves. A tuple's elements are
// heterogeneous, so there is no runtime element-indexing op — a comptime index
// lowers exactly like the `.N` field-access path (a `structGet`). A runtime
// index into a tuple value remains an error (no single element type).
//
// (GAP 1 of PLAN-RACE.)
#import "modules/std.sx";
Box :: struct ($R: Type) { value: R; }
// Read each handle with its concrete type via the `inline for` cursor.
show_all :: (tup: $T) {
inline for 0..field_count(T) (i) {
h := tup[i]; // concrete `*Box(T_i)`, not `Any`
print("[{}] = {}\n", i, h.value); // field access resolves
}
}
// Literal-index form, positional tuple.
first_two :: (tup: $T) -> i64 {
a := tup[0];
b := tup[1];
return a.value + b.value;
}
main :: () -> i32 {
ba : Box(i64) = .{ value = 7 };
bb : Box(bool) = .{ value = true };
bc : Box(f64) = .{ value = 2.5 };
show_all(.(a = @ba, b = @bb, c = @bc)); // named tuple of *Box(..)
p0 : Box(i64) = .{ value = 10 };
p1 : Box(i64) = .{ value = 32 };
print("sum = {}\n", first_two(.(@p0, @p1))); // positional, literal index
return 0;
}

View File

@@ -1,35 +0,0 @@
// Comptime-cursor tuple-element L-VALUES: writing a named-tuple element by a
// comptime-constant index — the store/address-of siblings of 0652's read path.
// A tuple is heterogeneous, so each element L-value is a typed `structGep` of the
// i-th field (not a uniform `index_gep`): `tup[i] = v` (direct store), a field
// store through an element pointer (`tup[i].f = v`), and `@tup[i]` (address-of).
// These are what the `race` runtime needs to register a waiter on the i-th task
// handle (`tasks[i].waiter = …`). An out-of-range comptime index is a loud
// compile error on every one of these paths (no silent `ptrTo(unresolved)` panic).
#import "modules/std.sx";
Box :: struct ($R: Type) { value: R; }
main :: () -> i32 {
// Direct element store by literal index.
t := .(a = 1, b = 2, c = 3);
t[0] = 100;
t[2] = 300;
print("t = ({}, {}, {})\n", t.a, t.b, t.c);
// Address-of an element, write through the pointer.
p := @t[1];
p.* = 200;
print("t.b via @t[1] = {}\n", t.b);
// Field store THROUGH an element pointer — `tup[i].field = v` — the exact
// L-value shape `race` uses to register a waiter (`tasks[i].waiter = …`): the
// i-th element is a `*Box`, and `.value` writes through it to the pointee.
ba : Box(i64) = .{ value = 0 };
bb : Box(bool) = .{ value = false };
handles := .(x = @ba, y = @bb);
handles[0].value = 7;
handles[1].value = true;
print("ba.value = {}, bb.value = {}\n", ba.value, bb.value);
return 0;
}

View File

@@ -1,27 +0,0 @@
// Comparing an `Any` against a concrete value (a MIXED `Any == <concrete>`, in
// either operand order) compares the boxed value words — the same value-identity
// the both-`Any` comparison uses. Boxing the concrete side first keeps the
// operands shape-compatible.
//
// Regression (issue 0199): a mixed `Any == <concrete>` fell through to a plain
// `icmp` on a 16-byte `{tag, value}` aggregate vs a scalar, aborting the LLVM
// verifier ("Both operands to ICmp are not of the same type"). The both-`Any`
// form already worked; this extends it to one-sided `Any` comparisons.
#import "modules/std.sx";
main :: () -> i64 {
x : Any = 5;
print("{}\n", x == 5); // true
print("{}\n", x == 6); // false
print("{}\n", x != 6); // true
print("{}\n", 5 == x); // true (concrete on the left)
b : Any = true;
print("{}\n", b == true); // true
print("{}\n", b == false); // false
y : Any = 5;
print("{}\n", x == y); // true (both Any — unchanged)
return 0;
}

View File

@@ -1,6 +0,0 @@
struct: fc=2 fn=(a,b) ft0=i64
enum: fc=3 fn=(X,Y,Z)
postuple: fc=2 ft=(i64,bool) fn0=[]
namtuple: fc=2 fn=(a,b)
array: fc=4 ft0=i64 fn0=[]
vector: fc=4 ft0=f32 fn0=[]

View File

@@ -1,4 +0,0 @@
pointee(*i64) = i64
pointee(*bool) = bool
pointee(*Box(f64)) field0 = f64
Payload(*Box(i64)) value = 42

View File

@@ -1,6 +0,0 @@
field_count(S) loop sum = 3
array[field_count(S)] len = 3 xs[2] = 20
field_count(E) = 4
size_of(i64) array len = 8
align_of(f64) = 8
[field_count(S) + 1] len = 4

View File

@@ -1,4 +0,0 @@
a (i64) = 42
b (bool) = true
c (f64) = 2.500000
R: variants=3 names=(a,b,c)

View File

@@ -1,4 +0,0 @@
v = 1.500000 2.500000 3.500000
s = hello
big = 10 20 30 40 true
n = 99

View File

@@ -1,2 +0,0 @@
classify: 100 200 300 -1
tag: 10 20 30 -1

View File

@@ -1,4 +0,0 @@
[0] = 7
[1] = true
[2] = 2.500000
sum = 42

View File

@@ -1,3 +0,0 @@
t = (100, 2, 300)
t.b via @t[1] = 200
ba.value = 7, bb.value = true

View File

@@ -1,7 +0,0 @@
true
false
true
true
true
false
true

View File

@@ -1,31 +1,28 @@
// B1.2 / B2 — the async ergonomic layer over the `Io` capability, blocking
// default. `context.io.async(worker)` submits a NULLARY `worker: Closure() -> $R`
// and returns a `*Future($R)` handle; under the blocking `CBlockingIo` the worker
// runs to completion inline, so the Future is born `.ready`. `f.await()` yields
// the result (a value-failable `($R, !IoErr)`, handled with `or`).
// `context.io.now_ms()` reads the clock through the same capability.
// B1.2 — the async ergonomic layer over the `Io` capability, blocking
// default. `context.io.async(worker, ..args)` runs the worker to completion
// inline and returns a `.ready` Future($R); `f.await()` yields the result
// (a value-failable `($R, !IoErr)`, handled with `or`). `context.io.now_ms()`
// reads the monotonic clock through the same capability.
//
// Worker form: a nullary failable lambda capturing any inputs at the CALL SITE
// (`() -> (i64, !) => compute(a, b)`) — the colorblind shape that also works when
// the worker is deferred onto a fiber (a captured variadic pack can't cross the
// fiber boundary).
// Worker form: a lambda whose params are annotated at the call site
// (`(a: i64, b: i64) -> i64 => …`); `..args` forwards the call-site
// arguments to it.
#import "modules/std.sx";
main :: () {
// Inputs captured at the call site. The worker is FAILABLE
// (`Closure() -> ($R, !)`) — the unified Phase 3 shape; a body that never
// raises is a degenerate failable that always succeeds.
s := context.io.async(() -> (i64, !) => 40 + 2);
// Homogeneous args.
s := context.io.async((a: i64, b: i64) -> i64 => a + b, 40, 2);
print("sum: {}\n", s.await() or { -1 });
d := context.io.async(() -> (i64, !) => 21 * 2);
// Single arg.
d := context.io.async((x: i64) -> i64 => x * 2, 21);
print("double: {}\n", d.await() or { -1 });
// A worker that closes over a local.
base := 42;
n := context.io.async(() -> (i64, !) => base);
// Nullary worker — the variadic `async` binds an empty pack, so no separate
// `async_void` entry is needed.
n := context.io.async(() -> i64 => 42);
print("nullary: {}\n", n.await() or { -1 });
// The Io capability also carries a clock.
// The Io capability also carries a monotonic clock.
if context.io.now_ms() >= 0 { print("clock ok\n"); }
}

View File

@@ -6,13 +6,12 @@
#import "modules/std.sx";
main :: () {
// Not canceled → await yields the value. The worker is FAILABLE
// (`Closure() -> ($R, !)`) — the unified Phase 3 shape.
ok := context.io.async(() -> (i64, !) => 7);
// Not canceled → await yields the value.
ok := context.io.async((n: i64) -> i64 => n, 7);
print("ok: {}\n", ok.await() or { -1 });
// Canceled → await raises .Canceled → the `or` default is taken.
c := context.io.async(() -> (i64, !) => 7);
c := context.io.async((n: i64) -> i64 => n, 7);
c.cancel();
print("canceled: {}\n", c.await() or { -99 });
}

View File

@@ -46,7 +46,7 @@ main :: () -> i64 {
// Three DIFFERENT fiber bodies (distinct captured ids), interleaving via
// yield_now. Each appends its id once per round for 3 rounds.
spawn_worker :: (ps: *sched.Scheduler, psh: *Shared, my_id: i64) {
ps.spawn(() {
ps.spawn(() => {
r := 0;
while r < 3 {
append(psh, my_id);

View File

@@ -36,7 +36,7 @@ main :: () -> i64 {
// Fiber A: record 10, park, then (after wake) record 11. Store A's handle in
// the shared state so B can wake it.
mk_a :: (ps: *sched.Scheduler, psh: *Sh) {
psh.parked = ps.spawn(() {
psh.parked = ps.spawn(() => {
rec(psh, 10);
ps.suspend_self();
rec(psh, 11);
@@ -45,7 +45,7 @@ main :: () -> i64 {
// Fiber B: record 20, wake A (legit) + a spurious second wake (safe no-op),
// record 21.
mk_b :: (ps: *sched.Scheduler, psh: *Sh) {
ps.spawn(() {
ps.spawn(() => {
rec(psh, 20);
ps.wake(psh.parked); // legitimate: A is parked
ps.wake(psh.parked); // spurious: A is now .ready/queued — must no-op

View File

@@ -1,24 +1,23 @@
// Stream B2 — the SUSPENDING `context.io.async` layer over the M:1 fiber
// scheduler (PLAN-IO-UNIFY: the unified async stack — the bespoke `go`/`wait` was
// retired in Phase 5). In contrast with 1805's `context.io.async` UNDER THE
// BLOCKING `Io` (which runs each worker INLINE to completion — no interleaving),
// here the scheduler is installed as `context.io`, so `context.io.async(work)`
// runs `work` as a REAL fiber and `await()` SUSPENDS the caller until it finishes
// — a worker that yields mid-body lets a sibling run first (cooperative
// interleaving).
// Stream B1 (fibers) B1.4a — a truly-SUSPENDING fiber-task async layer
// (`go` / `wait` / `cancel`) over the M:1 scheduler, in pure sx. In contrast
// with 1805's `context.io.async` (which runs each worker INLINE to completion
// before returning a `.ready` future — no interleaving), here `s.go(work)` runs
// `work` as a REAL fiber and `t.wait()` SUSPENDS the caller until that fiber
// finishes, so a task that yields mid-body lets a sibling task run before the
// first completes — genuine cooperative interleaving.
//
// `work` is a NULLARY worker: any inputs are captured in the lambda at the call
// `work` is a NULLARY thunk: any inputs are captured in the lambda at the call
// site (no `..args` pack crosses the fiber boundary — that would hit issue 0156
// Part 2). Outputs flow OUT through pointers captured in the worker (the shared
// Part 2). Outputs flow OUT through pointers captured in the thunk (the shared
// `Log` struct), since closure capture-by-value does not write back.
//
// What this proves:
// - REAL suspend + interleave: worker A records 1, YIELDS; worker B then records
// 2 and completes; A resumes, records 3, completes → interleave order 1 2 3.
// - awaited VALUES: A returns 42, B returns 100 (recorded after both awaits).
// - REAL suspend + interleave: task A records 1, YIELDS; task B then records 2
// and completes; A resumes, records 3, completes → interleave order 1 2 3.
// - awaited VALUES: A returns 42, B returns 100 (recorded after both waits).
// → sequence: 1 2 3 42 100.
// - cancel rides the `!` channel (model (a), like 1806): a canceled worker's
// `await()` raises `.Canceled`, taken by the `or` default → -99.
// - cancel rides the `!` channel (model (a), like 1806): a canceled task's
// `wait()` raises `.Canceled`, taken by the `or` default → -99.
//
// `wait` must run inside a fiber (it parks `self.current`), so the "main task"
// is itself a `s.spawn(...)` fiber that drives the two `go` tasks.
@@ -39,39 +38,36 @@ main :: () -> i64 {
ps := @s;
pl := @lg;
// The coordinator fiber: drives two async workers, awaits both, then exercises
// cancel. It runs as a fiber so `await` has a `self.current` to park. The
// scheduler is installed as `context.io`, so the unified async layer reaches it.
push .{ io = xx s } {
ps.spawn(() {
// Worker A yields mid-body so B interleaves before A completes.
a := context.io.async(() -> (i64, !) {
// The "main task" fiber: drives two real tasks, waits both, then exercises
// cancel. It runs as a fiber so `wait` has a `self.current` to park.
s.spawn(() => {
// Task A yields mid-body so B interleaves before A completes.
a := ps.go(() -> i64 => {
rec(pl, 1);
ps.yield_now(); // suspend A; B (already spawned) runs to completion
rec(pl, 3);
42
});
// Worker B runs straight through (no yield).
b := context.io.async(() -> (i64, !) {
// Task B runs straight through (no yield).
b := ps.go(() -> i64 => {
rec(pl, 2);
100
});
// Await both — suspends the coordinator fiber until each completes.
va := a.await() or { -1 };
vb := b.await() or { -1 };
// Wait both — suspends the main-task fiber until each completes.
va := a.wait() or { -1 };
vb := b.wait() or { -1 };
rec(pl, va);
rec(pl, vb);
// Cancel case: cancel before the worker runs; `await` raises .Canceled
// off the sticky flag, the `or` default (-99) is taken.
c := context.io.async(() -> (i64, !) => 7);
// Cancel case: cancel before the worker runs; `wait` raises .Canceled,
// the `or` default (-99) is taken.
c := ps.go(() -> i64 => 7);
c.cancel();
rec(pl, c.await() or { -99 });
rec(pl, c.wait() or { -99 });
});
ps.run();
}
s.run();
// Interleaving + value contract: 1 2 3 42 100, then the cancel default -99.
print("sequence:");

View File

@@ -50,12 +50,12 @@ main :: () -> i64 {
pl := @lg;
// Spawn order A, B, C, D, E — but the WAKE order is set by deadline.
ps.spawn(() { ps.sleep(30); rec(pl, 1, ps.now_ms()); }); // A: latest
ps.spawn(() { ps.sleep(10); rec(pl, 2, ps.now_ms()); }); // B: earliest
ps.spawn(() { ps.sleep(20); rec(pl, 3, ps.now_ms()); }); // C: middle
ps.spawn(() => { ps.sleep(30); rec(pl, 1, ps.now_ms()); }); // A: latest
ps.spawn(() => { ps.sleep(10); rec(pl, 2, ps.now_ms()); }); // B: earliest
ps.spawn(() => { ps.sleep(20); rec(pl, 3, ps.now_ms()); }); // C: middle
// Same-deadline FIFO pair: D before E, both at t=15 → wake D then E.
ps.spawn(() { ps.sleep(15); rec(pl, 4, ps.now_ms()); }); // D
ps.spawn(() { ps.sleep(15); rec(pl, 5, ps.now_ms()); }); // E
ps.spawn(() => { ps.sleep(15); rec(pl, 4, ps.now_ms()); }); // D
ps.spawn(() => { ps.sleep(15); rec(pl, 5, ps.now_ms()); }); // E
s.run();

View File

@@ -29,11 +29,11 @@ main :: () -> i64 {
// Sleeper: arm sleep(100), park; when woken (early), record 1 and finish.
mk_sleeper :: (ps: *sched.Scheduler, pst: *S) {
pst.sleeper = ps.spawn(() { ps.sleep(100); rec(pst, 1); });
pst.sleeper = ps.spawn(() => { ps.sleep(100); rec(pst, 1); });
}
// Waker: record 2, then wake the sleeper BEFORE its 100ms timer fires.
mk_waker :: (ps: *sched.Scheduler, pst: *S) {
ps.spawn(() { rec(pst, 2); ps.wake(pst.sleeper); });
ps.spawn(() => { rec(pst, 2); ps.wake(pst.sleeper); });
}
mk_sleeper(ps, pst);
mk_waker(ps, pst);

View File

@@ -56,7 +56,7 @@ main :: () -> i64 {
// Reader: block on the (empty) pipe until it is readable, then read 3 bytes.
mk_reader :: (ps: *sched.Scheduler, pst: *S, rfd: i32) {
ps.spawn(() {
ps.spawn(() => {
ps.block_on_fd(rfd, true); // parks until read_fd is readable
n := read(rfd, xx @pst.bytes[0], xx 3);
pst.read_n = xx n;
@@ -65,7 +65,7 @@ main :: () -> i64 {
}
// Writer: write 3 bytes ('a','b','c') to the write end.
mk_writer :: (ps: *sched.Scheduler, pst: *S, wfd: i32) {
ps.spawn(() {
ps.spawn(() => {
buf : [3]u8 = ---;
buf[0] = xx 97; buf[1] = xx 98; buf[2] = xx 99; // 'a' 'b' 'c'
write(wfd, xx @buf[0], xx 3);

View File

@@ -36,26 +36,22 @@ main :: () -> i64 {
s := sched.Scheduler.init();
ps := @s; pl := @lg;
// The coordinator runs as a fiber so `await` has a `current` to park. The
// scheduler is installed as `context.io`, so the unified async layer
// (`context.io.async`/`await`/`sleep`/`now_ms`) reaches it inside the workers.
push .{ io = xx s } {
ps.spawn(() {
// Launch three async workers; each sleeps, logs its completion, returns.
a := context.io.async(() -> (i64, !) { try context.io.sleep(30); rec(pl, 1, context.io.now_ms()); 100 });
b := context.io.async(() -> (i64, !) { try context.io.sleep(10); rec(pl, 2, context.io.now_ms()); 20 });
c := context.io.async(() -> (i64, !) { try context.io.sleep(20); rec(pl, 3, context.io.now_ms()); 3 });
// The coordinator runs as a fiber so `wait` has a `current` to park.
s.spawn(() => {
// Launch three async tasks; each sleeps, logs its completion, returns.
a := ps.go(() -> i64 => { ps.sleep(30); rec(pl, 1, ps.now_ms()); 100 });
b := ps.go(() -> i64 => { ps.sleep(10); rec(pl, 2, ps.now_ms()); 20 });
c := ps.go(() -> i64 => { ps.sleep(20); rec(pl, 3, ps.now_ms()); 3 });
// Await in SPAWN order; results come back correct regardless.
va := a.await() or { -1 };
vb := b.await() or { -1 };
vc := c.await() or { -1 };
va := a.wait() or { -1 };
vb := b.wait() or { -1 };
vc := c.wait() or { -1 };
sum := va + vb + vc;
rec(pl, 9, sum); // sentinel row: id=9 carries the sum in `at`
});
ps.run();
}
s.run();
print("completion order (id @ virtual-ms):\n");
i := 0;

View File

@@ -8,7 +8,7 @@
sched :: #import "modules/std/sched.sx";
main :: () -> i64 {
s := sched.Scheduler.init(); ps := @s;
ps.spawn(() { ps.sleep(10); ps.sleep(-5); }); // -5 → loud abort
ps.spawn(() => { ps.sleep(10); ps.sleep(-5); }); // -5 → loud abort
s.run();
print("unreachable\n");
return 0;

View File

@@ -1,22 +1,18 @@
// A `Future` allows ONE awaiter — a second concurrent `await` on the same pending
// future would overwrite the single `park` slot, and completion would wake only
// A `Task` allows ONE awaiter — a second concurrent `wait` on the same pending
// task would overwrite the single `waiter` slot, and completion would wake only
// the second, stranding the first forever. Regression (B1.4a review, P1-c): the
// guard aborts loudly instead of silently deadlocking. Now over the unified
// `context.io` async layer (PLAN-IO-UNIFY Phase 5 — the bespoke `Task`/`wait` is
// retired).
// guard aborts loudly instead of silently deadlocking.
//
// aborts (exit 134) after the diagnostic — aarch64-macOS-pinned.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
S :: struct { t: *Future(i64); }
S :: struct { t: *sched.Task(i64); }
main :: () -> i64 {
st : S = ---; st.t = null;
s := sched.Scheduler.init(); ps := @s; pst := @st;
mkprod :: (ps: *sched.Scheduler, pst: *S) { pst.t = context.io.async(() -> (i64, !) { ps.yield_now(); 42 }); }
mkw :: (ps: *sched.Scheduler, pst: *S) { ps.spawn(() { x := pst.t.await() or { -1 }; print("got {}\n", x); }); }
push .{ io = xx s } {
mkprod :: (ps: *sched.Scheduler, pst: *S) { pst.t = ps.go(() -> i64 => { ps.yield_now(); 42 }); }
mkw :: (ps: *sched.Scheduler, pst: *S) { ps.spawn(() => { x := pst.t.wait() or { -1 }; print("got {}\n", x); }); }
mkprod(ps, pst); mkw(ps, pst); mkw(ps, pst); // second waiter → loud abort
s.run();
}
return 0;
}

View File

@@ -1,25 +1,25 @@
// Stream B1 (fibers) — `Scheduler.deinit` releases the scheduler's owned heap
// + fd resources, closing the documented bounded leaks (kq fd / List backings).
// Verified by a tracking `GPA`: deinit drives the live allocation count DOWN,
// and resets the kqueue fd to -1.
// + fd resources, closing the documented bounded leaks (kq fd / heap Tasks /
// List backings). Verified by a tracking `GPA`: deinit drives the live
// allocation count DOWN, and resets the kqueue fd to -1.
//
// Scenario (one run that touches every freed resource):
// - a SLEEPER fiber `sleep(5)`s → exercises the `timers` List
// - a READER fiber `block_on_fd`s a pipe → exercises the kqueue fd + the
// `io_waiters` List
// - a WRITER fiber writes 3 bytes → makes the pipe readable
// After `run()` drains all of it, `deinit()` frees: the `timers` / `io_waiters`
// List backings, and CLOSES the kqueue fd (resetting `kq` to -1). The Fibers
// were already reaped during `run()`. (The unified `context.io.async` layer's
// Futures are NOT scheduler-tracked — they leak with the closure-env residual
// below; the bespoke `go`/`Task`/`task_allocs` path was retired in Phase 5.)
// - two `go` tasks compute 42 / 7 → exercise the heap `Task`s +
// the `task_allocs` List
// After `run()` drains all of it, `deinit()` frees: the 2 heap Tasks, the
// `timers` / `io_waiters` / `task_allocs` List backings, and CLOSES the kqueue
// fd (resetting `kq` to -1). The Fibers were already reaped during `run()`.
//
// WHAT IT PROVES (the contract; numbers below are the snapshot):
// - `freed by deinit: N` — live allocations reclaimed by `deinit` (> 0).
// - `live after deinit: 0` — NO residual. Each spawned fiber's body-closure heap
// env is reclaimed at reap (`reap_fiber` frees `body.env` via the spawn-time
// allocator snapshotted in `dctx`), and `deinit` frees the List backings + kq
// fd — so the live count returns to zero.
// - `live after deinit` — the RESIDUAL. This is NOT zero and NOT a bug: it is
// exactly the documented closure-env leak — one heap env per `spawn`/`go`
// that sx cannot free (the runtime has no name for the env pointer). deinit
// reclaims everything it CAN; the env residual is a language limitation.
// - `kq open after run: 1` then `kq after deinit: -1` — the lazily-opened
// kqueue fd was genuinely open after the fd round and is closed by deinit.
// - `read: 3 [97 98 99]` — the fd path actually ran (reader blocked, woke via
@@ -70,11 +70,11 @@ main :: () -> i64 {
ps := @s; pst := @st;
// SLEEPER — arms a virtual-time timer, then parks.
ps.spawn(() { ps.sleep(5); });
ps.spawn(() => { ps.sleep(5); });
// READER — blocks on the empty pipe until kqueue reports it readable.
mk_reader :: (ps: *sched.Scheduler, pst: *S, rfd: i32) {
ps.spawn(() {
ps.spawn(() => {
ps.block_on_fd(rfd, true);
n := read(rfd, xx @pst.bytes[0], xx 3);
pst.read_n = xx n;
@@ -83,7 +83,7 @@ main :: () -> i64 {
}
// WRITER — writes 'a' 'b' 'c', making the pipe readable.
mk_writer :: (ps: *sched.Scheduler, wfd: i32) {
ps.spawn(() {
ps.spawn(() => {
buf : [3]u8 = ---;
buf[0] = xx 97; buf[1] = xx 98; buf[2] = xx 99;
write(wfd, xx @buf[0], xx 3);
@@ -92,6 +92,10 @@ main :: () -> i64 {
mk_reader(ps, pst, read_fd);
mk_writer(ps, write_fd);
// Two async tasks — heap Tasks tracked for deinit to free.
ps.go(() -> i64 => 42);
ps.go(() -> i64 => 7);
ps.run();
after_run = gpa.alloc_count;

View File

@@ -1,63 +0,0 @@
// Stream B2 — structured first-wins `race` over `context.io` (PLAN-IO-UNIFY
// Phase 4). `context.io.race(.(a = fa, b = fb, c = fc))` takes a named tuple of
// already-spawned `*Future(..)` handles (from `context.io.async`), SUSPENDS the
// calling fiber until the FIRST is `.ready`, and returns a comptime-SYNTHESIZED
// tagged-union (`RaceResult`) mirroring the tuple's labels — variant NAME = the
// tuple label, payload = that future's result type. Here the three workers return
// DIFFERENT types (i64 / bool / f64), so the minted union is
// `enum { a: i64; b: bool; c: f64 }` and the winner is matched by label.
//
// TRUE cancellation (Phase 3): the workers sleep 10/20/30 ms (deterministic
// virtual clock), so `a` wins at t=10. The losers `b`/`c` are parked mid-`sleep`
// when cancelled; their next `suspend_raw` raises `Canceled` and unwinds the body,
// so their POST-SLEEP `rec(...)` NEVER runs and `race` returns at WINNER-time. The
// completion log therefore shows ONLY `a @ 10ms`, and the final virtual clock is
// 10 — NOT 30 (the old cooperative-join behaviour that let losers run to their
// natural end). The losers end `.canceled` with their work stopped.
//
// aarch64-pinned (the scheduler's per-arch asm + per-OS mmap/event constants):
// runs end-to-end on a matching host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
Log :: struct { id: [8]i64; at: [8]i64; n: i64; }
rec :: (l: *Log, id: i64, at: i64) { l.id[l.n] = id; l.at[l.n] = at; l.n = l.n + 1; }
main :: () -> i64 {
lg : Log = ---; lg.n = 0;
s := sched.Scheduler.init();
ps := @s; pl := @lg;
// The coordinator runs as a fiber so `race` has a `current` to park.
push .{ io = xx s } {
ps.spawn(() {
// Three async workers, DIFFERENT result types and sleep durations.
a := context.io.async(() -> (i64, !) { try context.io.sleep(10); rec(pl, 1, context.io.now_ms()); 111 });
b := context.io.async(() -> (bool, !) { try context.io.sleep(20); rec(pl, 2, context.io.now_ms()); true });
c := context.io.async(() -> (f64, !) { try context.io.sleep(30); rec(pl, 3, context.io.now_ms()); 2.5 });
// Race them. `a` (sleep 10) wins; `b` and `c` are cancelled — their
// post-sleep work never runs (true cancellation).
winner := context.io.race(.(a = a, b = b, c = c));
if winner == {
case .a: (v) { print("winner: a (i64) = {}\n", v); }
case .b: (v) { print("winner: b (bool) = {}\n", v); }
case .c: (v) { print("winner: c (f64) = {}\n", v); }
}
// The losers were cancelled; their work was stopped at the suspend.
print("loser b: canceled={}\n", b.state == .canceled);
print("loser c: canceled={}\n", c.state == .canceled);
});
ps.run();
}
print("completion order (id @ virtual-ms):\n");
i := 0;
while i < lg.n {
print(" task {} @ {}ms\n", lg.id[i], lg.at[i]);
i = i + 1;
}
print("final virtual clock: {}ms\n", s.now_ms());
return 0;
}

View File

@@ -1,35 +0,0 @@
// Stream B2/A1 — a fiber INHERITS the dynamic `context` in force when it was
// spawned. Previously a fiber body ran under the static `__sx_default_context`
// (the `abi(.c)` `fib_dispatch` dropped the implicit context), so a
// `push Context { … }` around `spawn` was invisible inside the fiber. Now
// `Scheduler.spawn` snapshots `context` into the fiber and `fib_dispatch`
// re-pushes it around the body — so a capability installed before `spawn`
// (here a marker in `context.data`) is visible to the worker.
//
// This is the foundation for folding a fiber scheduler behind `context.io`: a
// worker's `context.io.*` must resolve to the scheduler that spawned it, not the
// blocking default. Behavior-preserving for fibers spawned under the default
// context (the snapshot just re-pushes that same default).
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
Marker :: struct { id: i64; }
main :: () -> i64 {
mk := Marker.{ id = 7 };
s := sched.Scheduler.init();
ps := @s;
print("outside: marker id = {}\n", mk.id);
push .{ data = xx @mk } {
ps.spawn(() {
m : *Marker = xx context.data; // inherited from the spawn-time context
print("inside fiber: context.data marker id = {}\n", m.id);
});
ps.run();
}
print("done\n");
return 0;
}

View File

@@ -1,42 +0,0 @@
// Stream B2/A1 — the M:1 fiber scheduler installed AS an `Io` capability vtable,
// driven entirely through `context.io`. `impl Io for Scheduler` (sched.sx) folds
// the scheduler behind the same `Io` protocol the blocking `CBlockingIo`
// implements, so the worker below reaches real suspension/timers through the
// PROTOCOL (`context.io.spawn_raw`/`arm_timer`/`suspend_raw`/`now_ms`) rather
// than bespoke scheduler methods — the foundation for a colorblind
// `async`/`await`/`race` that runs over whichever `Io` is installed.
//
// Two workers are spawned via `context.io.spawn_raw`; each arms a virtual-time
// timer and `suspend_raw`s until it fires. They resume in DEADLINE order (10 then
// 20), deterministic on the virtual clock — proving the protocol round-trips
// spawn → arm → suspend → ready → resume against the fiber engine. Phase 0 (fibers
// inherit the spawn-time context) is what lets the worker's own `context.io`
// resolve back to this scheduler.
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
// Worker entry: an sx (*void)->void fn, erased to *void by spawn_raw.
sleeper :: (arg: *void) {
n : *i64 = xx arg;
tok : ParkToken = .{ handle = null };
context.io.arm_timer(context.io.now_ms() + n.*, tok);
context.io.suspend_raw(@tok) catch {};
print("worker(sleep {}) resumed at now_ms = {}\n", n.*, context.io.now_ms());
}
main :: () -> i64 {
s := sched.Scheduler.init();
ps := @s;
d1 : i64 = 20;
d2 : i64 = 10;
push .{ io = xx s } {
context.io.spawn_raw(xx sleeper, xx @d1, .{});
context.io.spawn_raw(xx sleeper, xx @d2, .{});
ps.run();
}
print("final clock: {}ms\n", ps.now_ms());
return 0;
}

View File

@@ -1,42 +0,0 @@
// Stream B2 — `async`/`await` (the io.sx ergonomic layer) running COLORBLIND over
// the fiber `Io` scheduler. The SAME `context.io.async(worker)` that runs inline
// under the blocking `CBlockingIo` (1805) here spawns the worker as a real fiber
// and returns a PENDING `*Future`; `await` suspends the calling fiber until the
// worker completes. No bespoke `go`/`wait` — this is the unified async stack
// (io.sx async over the `Io` protocol), reaching the fiber scheduler purely
// through `context.io`.
//
// The completion log makes the deferral visible: the coordinator records 1,2
// BEFORE either worker runs (async only SPAWNS them), then `await` parks it while
// the workers run (10,20), then it resumes and sums (123). Deterministic.
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
Log :: struct { seq: [8]i64; n: i64; }
rec :: (l: *Log, v: i64) { l.seq[l.n] = v; l.n = l.n + 1; }
main :: () -> i64 {
lg : Log = ---; lg.n = 0;
s := sched.Scheduler.init();
ps := @s; pl := @lg;
push .{ io = xx s } {
ps.spawn(() {
rec(pl, 1); // coordinator starts
a := context.io.async(() -> (i64, !) { rec(pl, 10); 100 }); // worker A — deferred
b := context.io.async(() -> (i64, !) { rec(pl, 20); 23 }); // worker B — deferred
rec(pl, 2); // both spawned, neither has run
va := a.await() or { -1 }; // park; A runs, wakes us
vb := b.await() or { -1 };
rec(pl, va + vb); // 123
});
ps.run();
}
print("sequence:");
i := 0;
while i < lg.n { print(" {}", lg.seq[i]); i = i + 1; }
print("\n");
return 0;
}

View File

@@ -1,49 +0,0 @@
// Stream B2 — TRUE cancellation (PLAN-IO-UNIFY Phase 3). A `cancel` delivered to
// a worker that is PARKED at a suspend point makes the worker ABANDON its body:
// the worker's next `suspend_raw` raises `IoErr.Canceled`, which unwinds out
// through `try context.io.sleep(..)` and the failable worker, so every line AFTER
// the suspend never runs. This is "true cancellation, model (a)" — cancel rides
// the `!` channel and stops in-flight work at the next suspend, not merely flags
// a result.
//
// Flow (deterministic, virtual clock): the worker records 1 and parks in
// `sleep`; the coordinator (a fiber, so it can `yield`) lets the worker reach its
// park, then `cancel`s it. The worker's parked `suspend_raw` is woken and raises
// `Canceled` → the post-sleep `rec(pl, 2)` and the `42` return NEVER execute. The
// coordinator's `await` raises `Canceled` (sticky flag) → `or` default -99.
// Sequence: `1 -99` — the absence of `2` is the proof that the post-suspend work
// was truly cancelled.
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux, byte-identical under the deterministic virtual clock).
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
Log :: struct { seq: [8]i64; n: i64; }
rec :: (l: *Log, v: i64) { l.seq[l.n] = v; l.n = l.n + 1; }
main :: () -> i64 {
lg : Log = .{ n = 0 };
s := sched.Scheduler.init();
ps := @s; pl := @lg;
push .{ io = xx s } {
ps.spawn(() {
w := context.io.async(() -> (i64, !) {
rec(pl, 1); // worker started
try context.io.sleep(10); // park; cancel delivers Canceled HERE
rec(pl, 2); // POST-SUSPEND — must NEVER run
42
});
ps.yield_now(); // let the worker run & park in sleep
w.cancel(); // cancel while parked → wakes + raises
r := w.await() or { -99 }; // await raises Canceled → -99
rec(pl, r);
});
ps.run();
}
print("seq:");
i := 0;
while i < lg.n { print(" {}", lg.seq[i]); i = i + 1; }
print("\n");
return 0;
}

View File

@@ -1,35 +0,0 @@
// Stream B2 — `context.io.race` tolerates a FAILING racer (PLAN-IO-UNIFY Phase 4).
// A `race` is first-SUCCESS-wins: a racer that ends `.failed` is simply not a
// winner candidate; as long as ANOTHER racer succeeds, `race` returns that winner.
// Here `a` raises at t=5 and `b` succeeds (42) at t=10, so `b` wins. The failed
// racer keeps its real outcome label (`.failed`) — `race` only cancels still-
// in-flight (`.pending`) losers, so it never stomps `a`'s `.failed` to `.canceled`.
//
// (Regression: an all-FAILING racer set instead bails loudly — "race — all
// futures settled without a winner" — rather than dead-locking the scheduler.)
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
main :: () -> i64 {
s := sched.Scheduler.init();
ps := @s;
push .{ io = xx s } {
ps.spawn(() {
a := context.io.async(() -> (i64, !) { try context.io.sleep(5); raise error.Boom; });
b := context.io.async(() -> (i64, !) { try context.io.sleep(10); 42 });
winner := context.io.race(.(a = a, b = b));
if winner == {
case .a: (v) { print("winner: a = {}\n", v); }
case .b: (v) { print("winner: b = {}\n", v); }
}
// The failing loser keeps its real outcome — not stomped to .canceled.
print("a: failed={} canceled={}\n", a.state == .failed, a.state == .canceled);
});
ps.run();
}
print("final clock: {}ms\n", s.now_ms());
return 0;
}

View File

@@ -1,46 +0,0 @@
// The unified `context.io.async` layer reclaims its per-task heap once a future is
// AWAITED (PLAN-IO-UNIFY follow-up — closing the documented leaks). Each `async`
// allocates: the `Future`, the `ThunkBox`, the completion-closure env, the worker's
// env, and the spawn_raw fiber-body env. With ownership wired through, ALL of it is
// freed: the box + envs by `sx_run_boxed_closure` the instant the worker completes,
// the fiber-body env at fiber reap, and the `Future` by the last of {worker,
// `await`} (the two-flag handshake). Verified by a tracking `GPA`: after running +
// awaiting three workers and `deinit`, the live-allocation count returns to the
// pre-spawn baseline — zero residual.
//
// (A future that is never awaited — fire-and-forget, or a `race` loser — keeps only
// its `Future` struct, since nothing consumes it; that remainder needs a
// structured-concurrency scope and is out of scope here.)
//
// aarch64-pinned (the scheduler's per-arch asm): runs end-to-end on a matching
// host (macOS + linux), ir-only on a mismatch.
#import "modules/std.sx";
sched :: #import "modules/std/sched.sx";
main :: () -> i64 {
sum : i64 = 0; psum := @sum;
base : i64 = 0; pbase := @base;
after : i64 = 0; pafter := @after;
gpa := mem.GPA.init();
push Context.{ allocator = xx gpa, data = null } {
s := sched.Scheduler.init();
ps := @s;
pbase.* = gpa.alloc_count; // baseline: scheduler is live, no tasks yet
push .{ io = xx s, allocator = xx gpa, data = null } {
ps.spawn(() {
a := context.io.async(() -> (i64, !) { try context.io.sleep(10); 100 });
b := context.io.async(() -> (i64, !) { try context.io.sleep(20); 20 });
c := context.io.async(() -> (i64, !) { try context.io.sleep(30); 3 });
psum.* = (a.await() or 0) + (b.await() or 0) + (c.await() or 0);
});
ps.run();
}
s.deinit();
pafter.* = gpa.alloc_count; // after run + await-all + deinit
}
print("sum: {}\n", sum);
print("residual above baseline: {}\n", after - base); // 0 — every async heap reclaimed
return 0;
}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -1 +1 @@
io: await — future already has an awaiter (one awaiter per future in the M:1 model)
sched: wait()task already has a waiter (one awaiter per task in the M:1 model)

View File

@@ -1,5 +1,5 @@
read: 3 [97 98 99]
freed by deinit: 2
live after deinit: 0
freed by deinit: 5
live after deinit: 5
kq open after run: true
kq after deinit: -1

View File

@@ -1 +0,0 @@
{ "target": "macos" }

View File

@@ -1,6 +0,0 @@
winner: a (i64) = 111
loser b: canceled=true
loser c: canceled=true
completion order (id @ virtual-ms):
task 1 @ 10ms
final virtual clock: 10ms

View File

@@ -1 +0,0 @@
{ "target": "macos" }

View File

@@ -1,3 +0,0 @@
outside: marker id = 7
inside fiber: context.data marker id = 7
done

View File

@@ -1 +0,0 @@
{ "target": "macos" }

Some files were not shown because too many files have changed in this diff Show More