lab/sx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(current): prune completed + superseded stream plans/checkpoints

Browse Source 
Remove fully-landed or superseded stream docs from current/:
- superseded: FIBERS, RACE (folded into PLAN-IO-UNIFY)
- complete: ASM, ATOMICS, METATYPE, MULTIRET, EXTERN-EXPORT, COMPILER-API/VM
This commit is contained in:
agra
2026-06-28 17:25:35 +03:00
parent 3328d3fe52
commit 155b99c3c2
15 changed files with 0 additions and 6526 deletions
Download Patch File Download Diff File Expand all files Collapse all files

394
current/CHECKPOINT-ASM.md
View File

@@ -1,394 +0,0 @@
# sx Inline Assembly — Checkpoint (ASM stream)
Companion to `current/PLAN-ASM.md`; design in
[design/inline-asm-design.md](../design/inline-asm-design.md). Update after every
commit, one step at a time per the cadence rule (no commit may both add a test
and make it pass).
## Last completed step
**G (indirect-memory `=*m` place outputs)** — the LAST substantive asm feature.
Unlike a write-through `=` output (which returns a value then stored), an
indirect output passes the place ADDRESS to the asm and the asm writes through
it — no return slot. `emitInlineAsm` (`src/backend/llvm/ops.zig`): indirect
outputs are excluded from the LLVM return type; their pointer is an opaque `ptr`
call arg placed **first** (arg-consuming constraint order = output-section
indirect pointers → inputs → read-write tied seeds); each gets an
`elementtype(T)` call-site attribute (required in the opaque-pointer era) via
`LLVMCreateTypeAttribute`/`LLVMAddCallSiteAttribute`; the store-back loop skips
them. New `asmIsIndirect(e, op)` helper. Lowering (`lowerAsmExpr`) stops
rejecting `*` (constraint kept verbatim, `=*m` reaches the constraint string
as-is). `asmOperandIndex` unchanged — indirect outputs still count as operands,
so `%[name]`→`${N}` holds. Verified by **running** on aarch64: store-through-
pointer (`str x9, %[out]` → 42, IR `"=*m,~{x9}"(ptr elementtype(i64) …)`) and a
mixed case (indirect + value output + input → `"=*m,=r,r"`, indirect ptr arg
first, `${0}/${1}/${2}` correct). Two commits per cadence: (1)
`examples/1652-platform-asm-indirect-mem.sx` locked the rejection; (2) implemented
+ flipped 1652 to a runnable aarch64-pinned example (`{ "target": "macos" }`,
ir-only elsewhere). `zig build test` green (661 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1652-*`.
Prior: **G (read-write `+` place outputs)** — a `+r` / `+{reg}` `-> @place` output is now
implemented. LLVM has no `+` constraint, so a
read-write place lowers to: an output **`=`** constraint (return slot, stored back
through the place after the call; the leading `+` rewritten to `=` in
`appendAsmConstraints`), **plus** a **tied input** (the decimal index of that
output) appended **after** the regular inputs, seeded with the place's loaded
value passed as a call arg. Tied inputs come **last** so existing operand indices
(`%[name]`→`${N}`) are undisturbed — `asmOperandIndex` unchanged. Lowering
(`lowerAsmExpr`) no longer rejects `+` (indirect `*` still rejected loudly).
`emitInlineAsm` (`src/backend/llvm/ops.zig`): grows arg/param arrays by the rw
count (`n_args = n_inputs + n_rw`), loads each seed (`asm.rw.seed`), emits the
tied constraint, and the existing store-back path writes the modified output back.
New `asmIsReadWrite(e, op)` helper. Verified by **running**: increment-in-place
(41→42, IR `"=r,0"`) and a mixed case (rw place + regular input + value output) →
textbook `"=r,=r,r,0"` with correct `${N}` indices and args `(input, seed)`. Two
commits per cadence: (1) `examples/1650-platform-asm-rw-place.sx` locked the
rejection; (2) implemented + flipped 1650 to a runnable aarch64-pinned example
(`{ "target": "macos" }`, ir-only elsewhere). `zig build test` green (658 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`examples/1650-*`.
Prior: **2** — `-> @place` write-through outputs. An asm result can be **stored through
a place** (local / struct field) instead of returned; the place output does NOT
join the result tuple. Parser: `-> @place` parses the `@place` as an ordinary
address-of expression → an `out_place` operand (`src/parser.zig`). Lowering
(`lowerAsmExpr`): out_place operand = the lowered `@place` address, `out_ty` =
the pointee; read-write (`+`) and indirect-memory (`*`) constraints rejected
loudly (not yet implemented). Added `out_ty: TypeId` to the IR `AsmOperand`
(`src/ir/inst.zig`) so emit builds the **combined** return struct (ALL outputs).
`emitInlineAsm` rewrite (`src/backend/llvm/ops.zig`): the LLVM return type is now
built from every output's `out_ty`; after the call, out_place slots are
`store`d through their address and out_value slots rebuild the sx result — with a
**fast path** (no place outputs → the asm's struct return IS the result, so
pure-value asm IR is unchanged). Verified: write-to-local (`get42`→42), struct
field (`@p.b`), mixed value+place (`v=10 b=20`), `+` rejected. Locked with
`examples/1649-platform-asm-place-output.sx` (mixed, runs on aarch64). `zig build
test` green (657 corpus, 446 unit). Files: `src/parser.zig`, `src/ir/inst.zig`,
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1649-*`.
Prior: **F** — global (module-scope) asm. A top-level `asm { "tmpl", };` block (template
only) lowers to LLVM `module asm`, and a lib-less `extern` calls into the symbols
it defines. New `asm_global` AST node (`src/ast.zig`) + `parseAsmGlobal`
(`src/parser.zig`, dispatched from `parseTopLevel` on `kw_asm`) — rejects
`volatile` and any operands/clobbers. The node forced (and got) arms in the same
three `Node.Data` switches as `asm_expr` (`sema.zig` ×2, `semantic_diagnostics.zig`).
`Module` gains a `global_asm: ArrayList([]const u8)` (`src/ir/module.zig`);
`lowerMainAndComptime` captures each template (the dead `lowerDecls` is NOT the
top-level pass — `lowerRoot` Pass 2 uses `lowerMainAndComptime`); `emit_llvm.zig`'s
`emit()` appends each via `LLVMAppendModuleInlineAsm` (source order). Verified
end-to-end: an aarch64 `_my_add` global routine called via `extern` returns 42.
Locked with `examples/1648-platform-asm-global.sx`
(`.build { "aot": true, "target": "macos" }` → AOT build+run on aarch64, ir-only
elsewhere). `zig build test` green (656 corpus, 446 unit). **(Correction, later:
module asm ALSO runs under the JIT — `sx run` compiles to an in-memory object,
the integrated assembler assembles the `module asm` into it, ORC relocates and
runs it, so the symbol is resolvable at JIT main execution. The original "AOT
only" note was wrong; see 1653 for the JIT sibling. The genuine boundary is a
COMPILE-TIME `#run` call into a module-asm symbol, which fails loud via host
dlsym-miss — see 1654.)** Files: `src/ast.zig`, `src/parser.zig`, `src/sema.zig`,
`src/ir/semantic_diagnostics.zig`, `src/ir/module.zig`, `src/ir/lower/decl.zig`,
`src/ir/emit_llvm.zig`, `examples/1648-*`.
Prior: **E** — multi-output tuples. **Inline asm now returns tuples.** Replaced the
N>1 bail with a shared `asmResultType` helper (`src/ir/lower/expr.zig`, mixed
into `Lowering`) that derives the result type from the `out_value` operands
(0→void, 1→T, N→named tuple, named via the §II.5 effective-name rule). The key
realization: `toLLVMType(tuple)` already produces a literal struct `{T1,…,Tn}` —
exactly LLVM's multi-output asm return — so **emit needed NO change**; building
the op with a tuple result type makes the asm call return the struct, which IS
sx's tuple value (destructured by the normal `tuple_get` path). `inferType`'s
`.asm_expr` arm now also delegates to `asmResultType` (single owner), so
`return asm`, `x := asm`, and `q, r := asm` all agree on the type. Verified
end-to-end on aarch64: `split(0x1234)`→`(lo=52, hi=18)`, a udiv/msub divmod→
`(3, 2)`. IR is textbook: `call { i64, i64 } asm "divq ${4}",
"={rax},={rdx},{rax},{rdx},r,~{cc}"(…)` → extractvalue → tuple. Converted 1640 to
the x86_64 multi-output IR lock (ir-only) + added `1647-platform-asm-aarch64-multi`
(runs on aarch64). `zig build test` green (655 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `src/ir/expr_typer.zig`,
`examples/164{0,7}-*`.
Prior: **C.1 + D** — inline asm CODEGEN (lowering builds the op + LLVM emit). **Inline
assembly now runs end-to-end.** `lowerAsmExpr` (`src/ir/lower/expr.zig`) stops
bailing: it resolves each operand's effective name (§II.5 auto-naming), interns
template/constraints/clobbers, lowers input `Ref`s, derives the result `TypeId`
(0→void, 1→T), and builds the `inline_asm` op. Added a `%[name]`-references-a-
real-operand check (the last deferred validation). Multi-output (N>1) still bails
loudly ("Phase E"). `emitInlineAsm` (`src/backend/llvm/ops.zig`, port of Zig's
`airAssembly`): assembles the LLVM constraint string (outputs→inputs→`~{clobber}`,
`,`→`|`), rewrites the template (`%[name]`→`${N}`, `%%`→`%`, `$`→`$$`, `%=`→
`${:uid}`), then `LLVMGetInlineAsm` + `LLVMBuildCall2` (AT&T). Dispatch wired
(`emit_llvm.zig`, replacing the C.0 `@panic`). **`llvm_shim.c`**: added
`LLVMInitializeNativeAsmParser()` — the JIT must assemble inline asm at run time.
Verified end-to-end: aarch64 `add`/`mov` run on the host (exit 42), `nop volatile`
runs (1642 now exit 0), IR is textbook (`call i64 asm "add ${0},${1},${2}",
"=r,r,r"(…)`). Locked with `examples/1645-platform-asm-aarch64-add.sx` (runs on
aarch64, ir-only elsewhere via `.build` + `.ir`). Also added the `inferType`
`.asm_expr` arm (`src/ir/expr_typer.zig`, 0→void / 1→T) — without it a bare
`x := asm {…-> T}` binding inferred `.unresolved` and silently produced 0;
regression-locked with `examples/1646-platform-asm-value-binding.sx`. Updated
1640 (now Phase-E bail) + 1642 (now runs). `zig build test` green (654 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`src/ir/emit_llvm.zig`, `src/ir/expr_typer.zig`, `llvm_shim.c`,
`examples/164{0,2,5,6}-*`.
Prior: **C.0** — IR op `inline_asm` (lock; no behavior change). Added `inline_asm:
InlineAsm` to the IR `Op` union + the `InlineAsm` struct (`template: StringId`,
`operands: []const AsmOperand` {role/name/constraint/operand}, `clobbers:
[]const StringId`, `has_side_effects`) in `src/ir/inst.zig` — all strings
interned, operands in source order, result on `Inst.ty`. The new variant forced
(and got) arms in two exhaustive `Op` switches: `src/ir/interp.zig` (loud
`bailDetail` — inline asm is never comptime-evaluable) and `src/ir/print.zig`
(IR dump). `src/ir/emit_llvm.zig` gets a `@panic` **tripwire** — emit lands in
Phase D, and until then `lowerAsmExpr` still bails so no `inline_asm` op is ever
created (reaching emit would be a lowering-switched-over-too-early bug). Unit
test `inline_asm op shape` in `src/ir/inst.test.zig`. `zig build test` green
(652 corpus, 446 unit). Files: `src/ir/inst.zig`, `src/ir/interp.zig`,
`src/ir/print.zig`, `src/ir/emit_llvm.zig`, `src/ir/inst.test.zig`.
Prior: **B.1** — operand-name validation (design §II.5 auto-naming rule). Extended
`lowerAsmExpr` with a `pinnedRegister(constraint)` helper (`"={eax}"`→`eax`,
`"+{rax}"`→`rax`, `"=r"`→null) and two checks: (1) **reject the echo form**
`[eax] "={eax}"` — a label identical to its own pinned register is redundant
(the operand is already auto-named after the register); (2) **reject duplicate
operand names** (ambiguous `%[name]` / result field). Locked with
`examples/1643-platform-asm-echo-name.sx` + `1644-platform-asm-duplicate-name.sx`.
`zig build test` green (652 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`.
Prior: **B.0** — asm shape validation (compile-path diagnostics). Restructured the
`.asm_expr` lowering arm into `lowerAsmExpr` (`src/ir/lower/expr.zig`, mixed into
`Lowering` in `src/ir/lower.zig`): it validates BEFORE the not-yet-implemented
codegen bail, so the user sees the real problem first. Two checklist items now
enforced with named diagnostics: (1) **template must be a compile-time-known
string** (`"..."` / `#string`); (2) **no value outputs ⇒ must be `volatile`**
(mirrors Zig — a result-less asm could be deleted). Valid shapes still bail with
the "codegen not yet implemented" message. Result-type derivation + auto-naming
stay deferred to a later step (observable only once Phase C produces a real IR
op). Locked with `examples/1641-platform-asm-missing-volatile.sx` (volatile
error) + `1642-platform-asm-nop-volatile.sx` (volatile no-output accepted →
codegen bail). `zig build test` green (650 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `examples/164{1,2}-*`.
Prior: **A.1** — parse `asm { … }` + loud lowering bail (folded A.1+A.2 into one honest
lock commit, since the loud bail IS current correct behavior — cadence option
(a)). Added `AsmExpr`/`AsmOperand` to `src/ast.zig` + the `asm_expr` `Node.Data`
arm; `parseAsmExpr` in `src/parser.zig` (`parsePrimary` `.kw_asm` dispatch) —
parses the template, flat operand list (`[name]? "constraint" -> Type` value
output / `= expr` input), and `clobbers(.…)`; `volatile`/`clobbers` recognized
contextually via `isContextualWord`. The new `asm_expr` tag forced (and got)
arms in three exhaustive `Node.Data` switches: `src/sema.zig` `analyzeNode` +
`findNodeAtOffset`, `src/ir/semantic_diagnostics.zig` `checkBindingNames` (all
recurse into template + operand payloads). Lowering bails LOUD + named in
`src/ir/lower/expr.zig` ("inline assembly codegen is not yet implemented…") via
an explicit `.asm_expr` arm (not the generic `unknown_expr` else) returning
`emitPlaceholder`. `-> @place` write-through is rejected with a clear "Phase 2"
parse error. Locked with `examples/1640-platform-asm-parse.sx` (multi-output
`divmod`, named operands, register pins, clobbers — parses then bails; called
from `main`). `zig build test` green (648 corpus, 0 failed; 445 unit). Files:
`src/ast.zig`, `src/parser.zig`, `src/sema.zig`, `src/ir/semantic_diagnostics.zig`,
`src/ir/lower/expr.zig`, `examples/1640-*`.
Prior: **A.0** — `kw_asm` keyword (first compiler code). Added the `kw_asm` `Token.Tag`
variant + `.{ "asm", .kw_asm }` keyword-map entry in `src/token.zig`; `volatile` /
`clobbers` deliberately stay OUT of the global table (contextual). New exhaustive
`Tag` switch in `src/lsp/server.zig` `classifyToken` flagged the missing arm (the
intended coverage tripwire) — added `.kw_asm` to the keyword group. Lock test in
new `src/lexer.test.zig` (`asm`→`kw_asm`, `volatile`/`clobbers`→`identifier`),
wired into the `src/root.zig` barrel as `lexer_tests`. `zig build test` green (648
corpus, 0 failed; 445 unit, 0 failed — +1). Files: `src/token.zig`,
`src/lexer.test.zig`, `src/root.zig`, `src/lsp/server.zig`.
Prior: **0.2** — CLAUDE.md docs for `<name>.build`; **Phase 0 COMPLETE**.
**0.1** — corpus runner **ir-only branch** for cross-target examples. Replaced
0.0's loud placeholder bail: when `cfg.target` doesn't match the host (`ir_only`),
`sweepRoot` skips run/build/exec and verifies via `sx ir --target` only —
asserting `.exit` (ir cmd) + `.ir` (normalized stdout) + `.stderr`, never
`.stdout` (write skipped in update mode, assertion skipped in verify mode). An
`.ir` snapshot is **required** in ir-only mode — its absence is a loud failure
("needs an .ir snapshot for ir-only mode"). Locked with
`examples/1639-platform-target-cross.sx` (asm-free `main :: () -> i64 { return 0;
}`), `.build` `{ "target": "x86_64-linux" }`, + checked-in `.ir`. Verified both
guards fire: corrupting the `.ir` → IR mismatch; deleting it → the require-failure.
`zig build test` green (647 corpus, 0 failed; 444 unit). Files:
`src/corpus_run.test.zig`, `examples/1639-*`.
## Current state
**Inline assembly works end-to-end: 0, 1, and N value outputs (tuples).** Full
pipeline: lex (A.0) → parse (A.1) → validate (B.0/B.1 + `%[name]` check) → IR op
(C.0) → lower-builds-op + LLVM emit + JIT asm-parser init (C.1/D) → multi-output
tuples (E). Register-class + register-pinned operands, inputs, **symbol operands
(`"s"` → direct `bl`/`call` to a function/global by mangled name)**, clobbers,
`#string` multi-instruction templates, `%[name]`/`%%` rewriting, and the §II.5
auto-naming rule all work and execute on the host JIT. Global `asm { … }` (Phase F) works via
lib-less `extern` under BOTH the JIT (`sx run` → 1653) and AOT (1648) — `sx run`
compiles to an object, so the integrated assembler bakes the `module asm` symbol
in and ORC resolves it. All three `-> @place` output forms now work and execute
on aarch64: **write-through** `=` (Phase 2), **read-write** `+` (tied input), and
**indirect-memory** `=*m` (pointer arg + `elementtype`, asm writes through it).
**Inline assembly is now feature-complete — no substantive features remain.** The
x86_64 syscall-write ir-only example is DONE (1651). Global asm runs under both
JIT (1653) and AOT (1648). `readme.md` now has an "Inline Assembly" section.
Known orthogonal bug: **issue 0137** — `sx run` on a program with no `main`
segfaults (`src/target.zig:256-273`, unguarded JIT entry lookup). Pre-existing,
asm-independent; does NOT block the ASM stream (every example has a `main`).
Phase E–F feasibility already confirmed against the live tree
(`LLVMGetInlineAsm` / `LLVMBuildCall2` / `LLVMAppendModuleInlineAsm` in LLVM@19
`Core.h`; ERR-stream `extractvalue`→tuple in `emit_llvm.zig:726-927`; lib-less
`extern`, 60 sites; `--target` a global CLI flag).
## Next step
**Inline assembly is feature-complete.** All substantive features are done:
0/1/N value outputs (tuples), register-class + pinned operands, inputs, clobbers,
`#string` templates, `%[name]`/`%%`/`$`/`%=` rewriting, §II.5 auto-naming, global
`asm { … }` (AOT), and all three `-> @place` output forms — write-through (`=`),
read-write (`+`), and indirect-memory (`=*m`). The x86_64 syscall-write ir-only
example (1651) and the output-to-`const` rejection (issue 0138) are also done.
Global asm runs under BOTH the JIT (`sx run` → object → ORC; 1653) and AOT (1648)
— the earlier "AOT only / `sx run` mishandles module-asm" note was stale and has
been corrected. The one genuine boundary is a COMPILE-TIME `#run` into a
module-asm symbol: the interpreter resolves externs via host dlsym, the symbol
isn't linked yet, so it already fails loud (`comptime extern call: symbol not
found via dlsym`) — pinned by 1654.
Remaining work, all **polish** (optional):
- None substantive. Possible niceties: tighten the `#run`-into-module-asm error
text to name module-asm specifically; broaden clobber validation to a checked
per-arch enum (design doc Phase 4).
Orthogonal: **issue 0137** (no-`main` JIT segfault).
Done since last: output-to-`const` rejection (issue 0138), x86_64 syscall-write
ir-only example (1651).
Orthogonal: **issue 0137** (no-`main` segfault).
## Log
- (init) Plan + design doc written; ASM stream opened.
- (0.0) Corpus runner target-gating: `<name>.build` JSON config (replaces `.aot`
marker), `--target` threading, `hostMatchesTarget` execute-gate, loud
cross-target placeholder bail. Migrated 1226/1227 `.aot`→`.build`; locked with
1638 fixture + unit tests. `zig build test` green.
- (0.1) ir-only branch: cross-target examples verify via `sx ir --target` only
(exit+ir+stderr, no stdout; `.ir` required). Locked with 1639 fixture; verified
corrupt-.ir → mismatch and missing-.ir → loud failure. `zig build test` green.
- (0.2) docs: CLAUDE.md documents `<name>.build` JSON sidecar (aot + target +
ir-only gating), replacing stale `.aot` marker prose. **Phase 0 COMPLETE.**
- (A.0) `kw_asm` keyword in token.zig (+ map entry); LSP `classifyToken` switch
coverage; lock test in new `lexer.test.zig` (wired via root.zig). `volatile` /
`clobbers` stay contextual identifiers. `zig build test` green (445 unit, +1).
- (A.1) parse `asm { … }` → `AsmExpr` + loud lowering bail; `asm_expr` arms in 3
exhaustive `Node.Data` switches; `-> @place` rejected (Phase 2). Adopted operand
auto-naming rule (design §II.5). Locked with 1640 fixture. Filed orthogonal
issue 0137 (no-`main` JIT segfault). `zig build test` green (648 corpus, 445 unit).
- (B.0) asm shape validation in `lowerAsmExpr`: comptime-string template +
no-output⇒volatile, with named diagnostics before the codegen bail. Locked with
1641 (volatile error) + 1642 (volatile accepted). `zig build test` green (650
corpus, 445 unit).
- (B.1) operand-name validation: `pinnedRegister` helper + reject echo form
(`[eax] "={eax}"`) and duplicate names. Locked with 1643 + 1644. `zig build
test` green (652 corpus, 445 unit).
- (C.0) IR op `inline_asm: InlineAsm` + interp `bailDetail` + print arm + emit
`@panic` tripwire (Phase D). No behavior change (lowering still bails). Unit
test `inline_asm op shape`. `zig build test` green (652 corpus, 446 unit).
- (C.1+D) CODEGEN — `lowerAsmExpr` builds the op (effective names, interned
strings, input Refs, 0/1 result type) + `%[name]` validation; `emitInlineAsm`
(constraint string + template rewrite + `LLVMGetInlineAsm`/`BuildCall2`, AT&T);
`inferType` arm; `LLVMInitializeNativeAsmParser` for the JIT. **Inline asm runs
end-to-end.** N>1 bails (Phase E). Locked with 1645 (aarch64 add, runs) + 1646
(`:=` binding); updated 1640/1642. `zig build test` green (654 corpus, 446 unit).
- (E) multi-output tuples — `asmResultType` helper (0→void/1→T/N→named tuple),
shared by lowering + `inferType`. `toLLVMType(tuple)` == LLVM multi-output
struct, so emit unchanged; the asm struct return IS the sx tuple. Runs on
aarch64 (1647: `split`→`(lo,hi)`); 1640 → x86 multi-output IR lock (ir-only).
`zig build test` green (655 corpus, 446 unit).
- (F) global asm — `asm_global` AST node + `parseAsmGlobal` (top-level, rejects
volatile/operands); `Module.global_asm` captured in `lowerMainAndComptime`;
`emit()` appends via `LLVMAppendModuleInlineAsm`; call-into via lib-less
`extern`. AOT-verified (1648, `_my_add`→42). `zig build test` green (656 corpus).
- (docs) readme.md "Inline Assembly" section (b8800a2).
- (2) `-> @place` write-through — `out_place` operand; `out_ty` on the IR
AsmOperand; `emitInlineAsm` builds the combined output struct + splits
(out_place → store-through, out_value → result), fast-path when no places.
`+`/`*` rejected. Locked with 1649 (mixed, runs). `zig build test` green (657
corpus, 446 unit).
- (G) read-write `+` place outputs — `+` lowers to an output `=` + a tied input
(output-index constraint) seeded with the place's loaded value, tied inputs
appended last (operand indices undisturbed). `appendAsmConstraints` rewrites
`+`→`=`; `emitInlineAsm` grows args by the rw count + loads seeds;
`asmIsReadWrite` helper. Lowering stops rejecting `+` (`*` still rejected). Two
commits (cadence): 1650 locked the rejection, then flipped to a runnable
aarch64 example (`"=r,0"` IR). `zig build test` green (658 corpus, 446 unit).
- (0138) output-to-`const` rejection — fixed the underlying general bug: scalar
`@const` (address-of a folded `::` constant) reinterpreted the value as a
pointer (`inttoptr`). `src/ir/lower/expr.zig` `.address_of` now diagnoses a
scalar const (local + module) instead of falling through; array/struct consts
keep storage. asm `-> @const` gets the clean diagnostic for free (same path).
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`. Issue 0138
RESOLVED. `zig build test` green (659 corpus, 446 unit).
- (x86 syscall) x86_64 Linux `write(2)` via raw `syscall` — locks the constraint
string `={rax},{rax},{rdi},{rsi},{rdx},~{rcx},~{r11},~{memory}` (register-pinned
inputs + pinned value output + pointer input + clobbers). ir-only on aarch64
(`.ir` asserted), runs on x86_64-linux (hand-authored `"ok\n"` stdout).
`examples/1651-platform-asm-x86-syscall-write.sx`. Pure additive lock, no
compiler change. `zig build test` green (660 corpus, 446 unit).
- (G indirect) indirect-memory `=*m` place outputs — the place address is passed
as an opaque `ptr` arg (with an `elementtype(T)` call-site attr), placed before
inputs; asm writes through it; no return slot; store-back skips it.
`asmIsIndirect` helper; lowering stops rejecting `*`. Verified by running on
aarch64 (store-through → 42; mixed indirect+value+input → `"=*m,=r,r"`). Two
commits (cadence): 1652 locked the rejection, then flipped to a runnable aarch64
example. **Inline asm now feature-complete.** `zig build test` green (661 corpus,
446 unit).
- (jit) explored "asm in JIT": found it ALREADY works — `sx run` emits an
in-memory object (integrated assembler bakes in both in-function inline asm and
`module asm`), then ORC relocates+runs it. The stale "AOT only / `sx run`
mishandles module-asm" checkpoint prose was corrected. Locked global-asm-under-
JIT with `examples/1653-platform-asm-global-jit.sx` (`{ "target": "macos" }`, no
aot, → 42). `zig build test` green (662 corpus, 446 unit).
- (comptime guard) pinned the one genuine module-asm boundary:
`examples/1654-platform-asm-global-comptime-call.sx` — `#run` into a module-asm
symbol fails loud (`comptime extern call: symbol not found via dlsym`) because
the interpreter resolves externs via host dlsym before link. Arch-independent
(no `.build`). `zig build test` green (663 corpus, 446 unit).
- (round trip) `examples/1655-platform-asm-callback-into-sx.sx` — global-asm
trampoline that `bl _cb` back into an `export`ed sx function (sx→asm→sx, → 42).
Documented that `export` (external linkage + C symbol + C ABI) is what makes
the callback resolvable; `callconv(.c)` alone leaves it `internal` (DCE'd).
`zig build test` green (664 corpus, 446 unit).
- (symbol ops) symbol operands (`"s"`) — feed a function/global symbol; the
template emits its platform-mangled name so `bl %[fn]` is a DIRECT branch (one
fewer indirection than register-indirect `blr`, portable — no hardcoded `_`).
Emit passes the operand with its own llvm type (LLVMTypeOf), no coercion
(`asmIsSymbol` helper); lowering lowers the function RHS to `ptr @fn`. Decided
AGAINST mirroring Zig (which has no symbol operand — 483 std asm sites, none
call a function) because the direct `bl` matters. Two commits (cadence): 1656
locked the rejection (replacing an LLVM-verifier crash), then implemented +
flipped to a runnable aarch64 example (objdump-confirmed direct `bl <_cb>`).
`zig build test` green (665 corpus, 446 unit).
- (x86 cross-arch) ir-only x86_64 siblings so each emit path is locked on BOTH
arches: 1657 read-write (`"incq ${0}","=r,0"`), 1658 indirect (`"movq $$42,
${0}","=*m"`(ptr elementtype)), 1659 symbol (`"call ${2:P}"`, direct call). x86
templates validated by cross-emitting an object (integrated assembler accepts;
objdump confirms 1659's direct `call` reloc). Pure additive locks. `zig build
test` green (668 corpus, 446 unit).
- (symbol portability) made `%[fn]` portable across arches — `renderAsmTemplate`
auto-injects LLVM's `:c` modifier (`${N}`→`${N:c}`) for symbol (`"s"`) operands
lacking an explicit modifier (`asmNamedIsSymbol` helper). Without it x86 renders
`$cb` (a bad `call` target needing a hand-written `:P`); aarch64 unaffected.
Verified `:c` ≡ `:P` for x86-64 calls (both → `R_X86_64_PLT32`). Explicit
`%[fn:X]` still wins (escape hatch). 1659 dropped its `:P` → same plain `%[fn]`
as aarch64 1656; both IRs regen to `${N:c}`. `zig build test` green (668 corpus,
446 unit).
## Known issues
- **0138** — RESOLVED. `@const` (address-of a `::` comptime constant) yielded a
wild pointer (`inttoptr (i64 <value> to ptr)`). Fixed by diagnosing scalar
`@const` in `src/ir/lower/expr.zig` `.address_of` (no storage; array/struct
consts unaffected). Delivered the ASM "output-to-`const` rejection" for free.
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`.
- **0137** — `sx run` on a program with no `main` segfaults (unguarded JIT entry
lookup, `src/target.zig:256-273`). Pre-existing, asm-independent. Filed
`issues/0137-jit-run-no-main-segfault.md`. Does not block A.1.

132
current/CHECKPOINT-ATOMICS.md
View File

@@ -1,132 +0,0 @@
# CHECKPOINT-ATOMICS — Stream A (atomics lowering)
Companion to [PLAN-ATOMICS.md](PLAN-ATOMICS.md). Update after every step (one step at a
time, per the cadence rule). New corpus category: `17xx`.
## Last completed step
**A.2 (CAS) — DONE** (A.2a lock + A.2b green). `compare_exchange`/`_weak` → LLVM `cmpxchg`
(result **`?T`, null = SUCCESS**; failure carries the actual value for retry). New IR op
`atomic_cmpxchg` + `AtomicCmpxchg{ptr, cmp, new, val_ty, success_ordering, failure_ordering,
weak}`. `emitAtomicCmpxchg`: `LLVMBuildAtomicCmpXchg` (success/failure orderings, singleThread=0)
→ `{T, i1}` pair; `LLVMSetWeak` for weak; `?T` result = `{ extractvalue 0 (actual),
xor(extractvalue 1, true) }` (has_value = NOT success). comptime_vm arm does real single-thread
CAS (read/compare/store-on-equal, build `?T`; weak == strong at comptime). Recognizer
(`atomic_cmpxchg`/`_weak`, 6 args) — CAS restricted to INTEGER T; BOTH orderings via
`atomicOrderingFromNode`; dual-ordering validation (failure may not be release/acq_rel nor
stronger than success, `atomicOrderingRank`). Methods `compare_exchange`/`_weak` on `Atomic($T)`
with comptime `$success`/`$failure: Ordering`. `examples/1702` green (CAS ok→20 / fail actual=20 /
weak retry loop 100→105); `examples/1186` locks a rejected ordering pair; unit test `emit: atomic
cmpxchg (strong + weak)` asserts `cmpxchg` + `cmpxchg weak`. Suite green (718/0).
### A.1 (RMW) — DONE (A.1a lock + A.1b green)
`fetch_add/sub/and/or/xor` + `fetch_min/max` → LLVM `atomicrmw` (returns OLD value). New IR op
`atomic_rmw` + `RmwKind` (no `nand`); `LLVMBuildAtomicRMW` with binop from kind, signed/unsigned
`Min/Max` from `val_ty`. RMW restricted to INTEGER T (float fadd / pointer RMW out of scope,
rejected loudly); all five orderings valid for RMW. comptime_vm does real single-thread RMW.
`examples/1701` green; unit test locks `atomicrmw add` + signed `min` vs unsigned `umin`.
## Next step
**A.3 — fence** (`atomic_fence($o: Ordering)` → LLVM `fence`), per PLAN-ATOMICS. No value
result; ordering must be acquire/release/acq_rel/seq_cst (relaxed is meaningless for a fence —
reject loudly). New IR op `atomic_fence` + dispatch/print/comptime_vm (no-op single-thread) +
`LLVMBuildFence`. Lock-then-green cadence as before.
### Earlier — A.0c (guard hardening)
Adversarial review of A.0 found two CRITICAL silent-wrong defects (raw LLVM verifier errors
via the public intrinsics) + a latent align fallback; all fixed: scalar-kind allowlist +
per-op ordering validity (call.zig), `val_ty` align bail (ops.zig). Locked by examples
1130/1131. Suite green (713/0).
### Earlier — A.0b (green)
Real atomic load/store emission: `LLVMBuildLoad2`/`LLVMBuildStore`
+ `LLVMSetOrdering` + mandatory `LLVMSetAlignment`, ordering via an explicit
sx-tag→`LLVMAtomicOrdering` switch (`llvmOrdering`). `examples/1700` green (7/42/43); IR
shows `load atomic i64, ptr … seq_cst, align 8` + `store atomic …`. Added unit test
`emit: atomic load/store (seq_cst, aligned)` in `emit_llvm.test.zig` (asserts `load
atomic`/`store atomic`/`seq_cst`/`align 8`). No fragile full-module `.ir` snapshot for 1700
(it uses `print`); the unit test is the emission-shape gate. Suite green (710 + units).
### Earlier — A.0a (lock commit)
Full atomic load/store plumbing with LLVM emission deliberately bailing loudly;
`examples/1700` locked to the bail diagnostic.
- `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (`init`/`load`/
`store`, **seq_cst-only** — see capability gap below), `atomic_load`/`atomic_store`
`#builtin` decls. **Opt-in import**, NOT in the universal `std.sx` facade (mirrors
`trace`) — putting `Ordering` in the prelude grew every program's type table 378→380 and
churned 37 `.ir` snapshots; reverted.
- IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` (all 5) + structs (inst.zig);
print arms (print.zig); comptime_vm arms reuse load/store (single-thread correct);
recognizer `tryLowerAtomicIntrinsic` (call.zig) — const-ordering-literal guard +
scalar-size guard, both loud; emit dispatch arms (emit_llvm.zig) → `emitAtomicLoad`/
`emitAtomicStore` (ops.zig) currently BAIL via `comptime_failed`.
## A.0.5 — full ordering surface (DONE)
`Atomic($T).load($o: Ordering)` / `store(v, $o)` — ordering is a COMPTIME value param,
explicit (Rust-style, no default; design §4.6). `a.load(.acquire)` emits `load atomic …
acquire`; `a.store(v, .release)` emits `store atomic … release`; `a.load(.release)` is a
compile error (per-op validity guard fires through the method path). Recognizer
`atomicOrderingFromNode` now resolves a comptime-bound ordering identifier via
`comptimeIntNamed` (+ `atomicOrderingFromTag`, with the sx-Ordering ↔ IR-AtomicOrdering
declaration-order invariant documented). 1700 migrated to explicit orderings (output
unchanged 7/42/43). Suite green (715/0).
**Unblocked by three comptime-value-param commits (workers):** enum (3c4305f), tagged_union
(d7a6857), generic-struct methods (d95ba0a). NOTE: default VALUES for comptime params on
generic-struct methods are NOT bound (orthogonal gap — free-fn defaults work); atomics
sidesteps it cleanly by requiring explicit ordering (matches the design). Candidate
follow-up, not an atomics blocker.
## Known issues / capability gaps
- **RESOLVED:** comptime-constant ordering propagation — landed via comptime value params
(3c4305f / d7a6857 / d95ba0a); A.0.5 migrated the methods, no seq_cst-only legacy.
- **Orthogonal gap (not an atomics blocker):** default VALUES for comptime params don't bind
on generic-struct methods (free-fn defaults DO work). Atomics requires explicit ordering
(design-aligned), so it's unaffected. Candidate future fix.
- **Cosmetic:** an invalid ordering passed through a method (`a.load(.release)`) reports the
diagnostic at the lib forward site (`atomic.sx`), not the user's call. Loud + correct, but
the span could be improved by threading the call-site span. Polish.
- **Latent (observed, not yet filed):** calling an *unrecognized* bodiless `#builtin`
silently returns 0 / no-ops with exit 0 (that's how 1700 behaved before recognition
landed) — a silent-fallback footgun in the generic builtin-call path, independent of
atomics. Flag to user; candidate `issues/` entry.
## Decisions (Stream A specifics; surface locked in design §4.6)
- `Atomic($T)` = pure-sx transparent 1-field struct (NO new IR type); ops = `#builtin`
intrinsics emitted as new IR ops. Minimal compiler surface.
- Ordering is compile-time-only (const enum literal), baked into the op as a Zig enum;
non-literal = loud diagnostic. sx tag → LLVM ordering via explicit switch (LLVM enum is
non-contiguous: 2/4/5/6/7).
- Atomic load/store REQUIRE explicit alignment (`LLVMSetAlignment`) — verifier mandate.
- Comptime VM treats atomics as ordinary load/store (single-thread ⇒ correct), not a bail.
- **Snapshot scope corrected:** `.ir` (LLVM IR) is arch-invariant for atomics → ONE host
`.ir` per op, not arch-gated x86/aarch64 pairs (they'd be byte-identical). Asm-level arch
divergence + weak-memory semantics are OUT of corpus scope (stress harness, Stream C).
## Log
- **carve** — wrote PLAN-ATOMICS.md + CHECKPOINT-ATOMICS.md; grounded the intrinsic path,
switch sites, LLVM-C API (no `LLVMBuildAtomicLoad`; use `LLVMBuildLoad2`+`SetOrdering`+
`SetAlignment`), and corrected the arch-`.ir` misconception (`sx ir` emits arch-invariant
LLVM IR). Stream ready; A.0a is the first implementation step.
- **A.0a** — landed lib (atomic.sx, opt-in import) + IR ops (atomic_load/atomic_store +
AtomicOrdering) + recognizer + print/vm arms + emit BAIL; locked `examples/1700` to the
bail diagnostic. Reverted a universal-facade wiring that churned 37 `.ir` snapshots
(Ordering would bloat every program's type table). Suite green (710/0).
- **A.0b** — real atomic load/store emission (LLVMBuildLoad2/Store + SetOrdering +
SetAlignment; explicit sx→LLVM ordering switch). 1700 green (7/42/43, `load atomic …
seq_cst, align 8`). Unit test added. Suite green (710 + units).
- **A.0c** — guard hardening from the adversarial review: scalar-kind allowlist + per-op
ordering validity (call.zig), val_ty align bail (ops.zig), + diagnostic examples
1130/1131. Suite green (713/0). (comptime enum value params landed via worker 3c4305f.)
- **A.0.5** — full ordering surface: `Atomic($T).load/store($o: Ordering)` comptime ordering
(explicit). Recognizer resolves comptime-bound ordering via `comptimeIntNamed`. 1700
migrated to explicit orderings (acquire/release/relaxed/seq_cst). Unblocked by
comptime-value-param workers (3c4305f/d7a6857/d95ba0a). Suite green (715/0).
- **A.1** — RMW: atomic_rmw op + RmwKind + recognizer (rmwKindFromName, integer-only) +
7 fetch_* methods/intrinsics. A.1a bail-lock → A.1b real LLVMBuildAtomicRMW (signed|unsigned
min/max). comptime_vm real RMW. 1701 + unit test. Suite green (716/0).
- **A.2** — CAS: atomic_cmpxchg op + recognizer (dual-ordering validation) + emit (?T from
{actual,!success}) + comptime VM. compare_exchange/_weak methods. examples 1702 + 1186.
Review agent died; self-verified comptime↔runtime agreement, sub-8, ordering edges.
(Commits dca396e/79895be; A.2 has_value fix folded into A.3a.)
- **A.3** — swap (atomicrmw xchg) + fence (new atomic_fence op). A.3a bail-lock → A.3b real.
examples 1703/1704/1187 + unit test. Stream A feature-complete. Suite green (721/0).

1663
current/CHECKPOINT-COMPILER-API.md
View File

File diff suppressed because it is too large Load Diff

861
current/CHECKPOINT-EXTERN-EXPORT.md
View File

@@ -1,861 +0,0 @@
# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)
Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
every commit, one step at a time per the cadence rule.
## Last completed step
**Phase 9 COMPLETE — total `foreign` purge; 9.4 GATE PASSES.** **THE ENTIRE
FFI-LINKAGE STREAM (Parts A + B, Phases 0–9) IS DONE.** Final commits: 9.0 token
delete (`dfae690`), 9.3 src/docs/example/library comment purge (`811a280`, `e99383f`,
`dc51c4b`, + the capital-Foreign sweep), 9.3 example filename renames + dedup
(`b52d424`), 9.3/9.4 issues/*.md purge + GATE (`b9cfe25`).
- **9.0:** deleted the `hash_foreign` token entirely (token/lexer/parser/lsp + the lex
test); `#foreign` now → a generic "expected ';'" parse error (accepted UX cost);
deleted the obsolete 1176 rejection test.
- **9.1/9.2:** all internal identifiers renamed (linkage→`extern`/`is_extern`,
runtime-class→`Runtime*`/`runtime_*` per Decision 5, `foreign_path`→`runtime_path`
across the build-hook boundary); `foreign_expr` node eliminated.
- **9.3:** purged every `foreign` COMMENT (src caps + lowercase, examples, docs incl.
the obsolete inline-asm Deviation 6, editors/vscode grammar) + renamed all 10
`*-foreign*` example files (+ companions/expected/refs) to extern/runtime names
(dedup'd 1218↔1229, removed orphan 1620 dir) + rewrote 20 issues/*.md writeups +
renamed issues/0043.
- **9.4 GATE:** `grep -rniIE 'foreign' src/ library/ examples/ issues/ docs/ editors/
specs.md readme.md CLAUDE.md` → **0**, excluding only the legitimate keeps:
`SQLITE_CONSTRAINT_FOREIGNKEY` (SQLite API const) + vendored `library/vendors/sqlite/
c/*` (upstream third-party C). No `foreign`-named files in the tree (node_modules +
.sx-tmp are gitignored third-party/scratch). Suite green (644 corpus / 443 unit, 0
failed).
### Prior: Phase 9.3 — text/comment purge (src + docs + example comments) (commits
`e99383f` docs, `dc51c4b` src, + examples purge STAGED pending a classifier outage —
commit message ready; `git commit` the staged `examples/` changes when Bash is back).
`foreign` is now purged from: **all `src/` comments** (reworded to extern/runtime-class;
fixed 2 user-facing diagnostics — the type-annotation parse error no longer lists
`#foreign`, and the Android no-`#jni_main` help shows `#jni_class(…) extern`), **specs/
readme/CLAUDE** ("Foreign Function Interface"→"C Interop", etc.), and **all example .sx
comments** (1219 stdout labels Foreign→Extern, snapshot regenerated). Suite green
(646/444) throughout; snapshot-neutral except the intentional 1219 regen.
**What still contains `foreign` (the analyzed keep-list + the not-yet-done):**
- **KEEP (gate-exempt):** `src/` `hash_foreign` token + lexer entry + `lex hash_foreign`
test (`#foreignx`) + the 4 parser rejection messages ("`#foreign` has been removed…");
`1176-diagnostics-foreign-removed.sx` (its `#foreign` decl + comments ARE the rejection
test); `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored `library/vendors/sqlite/c/*`.
- **NOT YET DONE:** example FILENAMES (`*-foreign*.sx` + the `0729`/`1205`/`1218`/`1219`/
`1306`/`1318`/`1216`/`1217` families) and their `#import`/`#include`/`#source` path refs
+ `expected/` files — needs a git-mv rename step; and **`issues/*.md`** (~20 writeups).
### Prior: Phase 9.1 + 9.2 — internal IDENTIFIER purge COMPLETE (commits 9.1a `b838f63`,
9.1b `b78e7dd`, 9.1c `cd14794`, 9.1d `7ffdc7d`, 9.2a `3354446`, 9.2b `5c8af6e`,
9.2b-fix `a15a868`, 9.2c `d27be42`, 9.2d `8cca3b9`). **Every `foreign` IDENTIFIER in
`src/` is renamed** — the only `foreign` left in `src/` is COMMENTS + the kept token
(`hash_foreign` + its `#foreignx` lexer-boundary test) + the rejection-message string.
Suite green (646/444) at every commit.
- **9.1d** eliminated the `foreign_expr` AST node: migrated `c_import.zig` auto-synth
to the extern shape, deleted the node + `ForeignExpr` + all readers.
- **9.2a/b/c/d** ran the runtime-class family rename (Decision 5 → `Runtime*`):
types `ForeignClassDecl`→`RuntimeClassDecl` etc.; fns `parseForeignClassDecl`→
`parseRuntimeClassDecl`, `lowerForeignMethodCall`→`lowerRuntimeMethodCall`, …; state
`foreign_class_map`→`runtime_class_map`, `foreign_class_decl` variant→
`runtime_class_decl`; the extern-ref validators → `Extern` (linkage, `checkExternRefs`);
the reference flag → **`is_extern`** (per user: reuse existing terminology, not a new
`is_reference`); and `foreign_path`→`runtime_path` COUPLED across the hook boundary
(build.sx accessor `jni_main_runtime_path_at` + the registered hook string +
bundle.sx + specs.md), with 37 `.ir` snapshots regenerated for the renamed
`@BuildOptions.jni_main_runtime_path_at` declare stub (symbol-name change only).
- **9.1a/b/c** (linkage): 5 collision-free renames (callExtern, …); "foreign symbol"
diagnostic + panic → "extern symbol" (1172 regen); deleted dead VarDecl legacy fields.
### Prior: Phase 9.1 (partial) — internal linkage-identifier purge (commits `b838f63` 9.1a,
`b78e7dd` 9.1b, `cd14794` 9.1c). **PHASE 9 STARTED.** Decision 6 = PURGE EVERYTHING,
scoped (user, 2026-06-15): purge `foreign` from **all `.sx` files + all documentation +
all our Zig (`src/`)**, analyzing each grep hit — **legitimate keeps stay**
(SQLITE_CONSTRAINT_FOREIGNKEY + other SQLite API constant names, vendored
`library/vendors/sqlite/c/*`, `1176-diagnostics-foreign-removed.sx` [the rejection test
MUST contain `#foreign`], the parser rejection-message string + `hash_foreign` token
[kept so `#foreign` keeps its friendly deprecation error]).
- **9.1a** (`b838f63`): 5 collision-free linkage renames — `callForeign`→`callExtern`,
`marshalForeignArg`→`marshalExternArg`, `dedupeForeignSymbol`→`dedupeExternSymbol`,
`foreign_name_map`→`extern_name_map`, `is_foreign_c_api`→`is_extern_c_api`.
- **9.1b** (`b78e7dd`): the "foreign symbol already bound" diagnostic (decl.zig) +
resolveFuncByName panic (call.zig) → "extern symbol". Intentional 1172 regen.
- **9.1c** (`cd14794`): **deleted** the dead `VarDecl.is_foreign`/`foreign_lib`/
`foreign_name` fields (the global `#foreign` path rejects → write-dead; 3 coalescing
readers in decl.zig simplified to `vd.extern_name`/`vd.is_extern`).
All snapshot-neutral except the intentional 1172 regen; suite green (646/444).
**COLLISION ANALYSIS (done — drives the rest of 9.1/9.2):**
- `is_foreign` lives on FnDecl?(no — flipped to `extern_export`), **VarDecl (deleted in
9.1c)**, and **ForeignClassDecl (ast.zig:903 — STILL LIVE**, distinguishes runtime-class
reference vs define; renamed in 9.2, not 9.1).
- `is_extern`/`extern_lib`/`extern_name` already exist (VarDecl + IR insts) — so the
old `foreign_*` linkage names could NOT be blind-renamed onto them; 9.1c deleted the
dead VarDecl trio instead of renaming.
- `foreign_expr` (25) is **still BUILT by `c_import.zig` auto-synthesis** (`#import c
{#include}` synthesizes fn bodies as `foreign_expr`). To eliminate it: migrate that
synth path to build the extern shape (empty-block body + `extern_export = .extern_`),
exactly the Phase 5.0 fn-body flip but for auto-synth — THEN delete the `foreign_expr`
node + all readers. This is the last 9.1 item.
### Prior: Phase 8 — CUTOVER: parser hard-rejects `#foreign` (`feat!` commit `3811311`,
preceded by the 8.0 xfail `8180faf` + 3 pre-cutover `refactor`s `2cce6a3`/`720556b`/
`d132aab`). **PHASE 8 COMPLETE.** The prefix `#foreign` linkage directive is removed:
all four parse sites (const-with-type 316, data global 425, fn body 2065, runtime-class
prefix via caller 260) reject it with the migration message *"`#foreign` has been
removed; use the postfix `extern` (import) / `export` (define) linkage keyword
instead"*; added a span-aware `failAt` for the runtime-class case (the lookahead
consumes the token before the reject decision). New example **1176**
(`diagnostics-foreign-removed`) pins it. **Pre-cutover migrations** (all green,
behavior-preserving): the 7 identity `ffi-foreign-*` test DECLS (`2cce6a3`), the two
keyword-neutral diagnostic tests 1172 + 1228 with intentional snapshot regens
(`720556b`), and the 4 multi-file example companions Phase 7 missed (0729/a+b, 1617/c,
1623/mod — `d132aab`). **Deleted** obsolete tests 1174 (`#foreign`+postfix conflict, now
unreachable) + 1620 (`#foreign nosuchunit`, superseded by extern twin 1231), the GATE
A→B unit test + `lowerSrcToIr` helper (nothing left to compare), and converted the
in-source `parse void function with foreign body` parser test to postfix `extern`.
specs.md + readme.md document `extern`/`export` as the sole C-linkage surface. Suite
green (646 corpus / 444 unit, 0 failed).
### Prior: Phase 7.4 — migrate straggler examples `#foreign`→`extern` (`refactor` commit
`1a8991a`). **PHASE 7 MIGRATABLE WORK COMPLETE (7.1–7.4 done).** Migrated 16 fn/global
examples across categories (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/
1622/1628/1635/1636). Marker'd ones corpus-validated; the 3 unmarked uikit importers
(1607/1608/1616) verified byte-identical via `sx ir` probes. Empty snapshot diff; suite
green (647/444).
**Phase 7 net result:** every example that uses `#foreign` *incidentally* (FFI plumbing,
output-preserving) is now on `extern`/`export`. The **24 files still holding `#foreign`
are exactly the intended keep-list**, all deferred to the Phase 8 cutover:
- **`foreign`-asserting diagnostics** (migrating changes a snapshot): 1172, 1174, 1219
(stdout label), 1228 (equivalence test), 1620.
- **Identity `ffi-foreign-*` feature tests** (real decls; rename/dedup at cutover):
1205-global, 1205-global-helper, 1207, 1218, 1219, 1306, 1318.
- **Comment-only / provenance prose** (decls=0; `#foreign` only in comments): 0716, 0729,
1216, 1223, 1224, 1225, 1229, 1230, 1231, 1332, 1348, 1349, 1426, + issues/0030.
**Lesson (7.3):** the robust class-prefix transform is the GENERAL form
`s/#foreign\s+#(\w+)\((\"[^\"]*\")\)\s*\{/#$1($2) extern {/` — 1417 also used
`#jni_interface`/`#objc_protocol`/`#swift_class`/`#swift_struct`/`#swift_protocol`, and a
`#(objc|jni)_class`-only regex left `extern` in *prefix* position → parse error. All such
directives accept the postfix modifier (probed). Bare defined `#objc_class`/`#jni_class`
examples (no `#foreign`) were left untouched — not a purge target (define→export is an
optional consistency pass, deferrable).
### Prior: Phase 7.1 — migrate incidental 12xx ffi examples (`refactor` commit `731fb8d`).
Migrated 12 plain-C examples (1200/1206/1209-1215/1220/1221/1222); established the
keep-list policy above. Phase 7.2 (`a68f7c2`): 18 13xx obj-c examples (prefix→postfix
classes). Phase 7.3 (`2888f6f`): 13 14xx jni examples incl. 1417 multi-runtime.
### Prior: Phase 6.5 — migrate `gpu/` `#foreign`→`extern`; `library/` now `#foreign`-free
(`refactor` commit `32a7628`). **PHASE 6 COMPLETE.** Final batch: gpu/gles3.sx
(eglGetProcAddress + 1 comment) + gpu/metal.sx (MTLCreateSystemDefaultDevice), bare
fn markers → `extern`. Verified byte-identical `sx ir` on importers 1610 (gles3) +
1606 (metal). **Zero `#foreign` remains anywhere under `library/`** — verified by
`grep -rln '#foreign' library/` → no matches. Suite green (647 corpus / 444 unit, 0
failed).
### Prior: Phase 6.4 — migrate `ffi/` `#foreign`→`extern` (`refactor` commit `666a2e2`).
objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + objc.sx's 2 import runtime
classes (prefix→postfix `extern`). objc + objc_block validated by the 50 marked 13xx
corpus examples (incl. import classes 1300/1301 + defined classes 1339/1349);
raylib/ffi-sdl3/wasm verified by byte-identical `sx ir` probes pre/post. Empty snapshot
diff; suite green.
### Prior: Phase 6.3 — migrate `std/` `#foreign`→`extern` (`refactor` commit `59f90d2`).
Pure source rename across 11 std modules (~60 sites):
cli/core/fmt/fs/log/net.kqueue/process/socket/thread/time/trace. All fn-decl markers
— bare `#foreign;`, `#foreign libc;`/`#foreign tlib;` (LIB ref), and
`#foreign libc "csym";` (LIB+rename) → the same `extern …` tail (`extern` carries the
identical `[LIB] ["csym"]` axis); plus 2 stale comment mentions (fmt/fs). No class
forms in std. These modules ARE host-corpus-exercised → empty snapshot diff is direct
validation. Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 batches:
6.4 ffi (~50, has runtime classes), 6.5 remainder.
### Prior: Phase 6.2 — migrate `platform/` `#foreign`→`extern`/`export` (`refactor` commit
`2cd5d7b`). Pure source rename across uikit/android/android_jni/sdl3 (~64 sites):
30 fn `… #foreign;`→`… extern;`; 34 import runtime classes
`#foreign #objc_class/#jni_class("X") {`→`#…_class("X") extern {` (prefix→postfix);
4 defined `Sx*` obj-c classes `#objc_class("X") {`→`… export {`. Behavior-preserving;
empty snapshot diff. **Verification (these modules are largely uncompiled by the
host corpus** — bundle examples import `bundle.sx`, not the runtime modules; android.sx
only compiles under `OS==.android`): byte-identical `sx ir` on uikit importers 1610 +
1606 (which DO compile uikit incl. the 4 defined `Sx*` classes on host) and an sdl3
direct-import probe; android.sx verified by an identical 4-error dedup set (host
pthread clashes — the keyword-neutral "foreign symbol already bound" dedup message is
unchanged, and the probe parsed all migrated `extern` jni classes + EGL fns cleanly
before hitting them). Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6
batches: 6.3 std (~60), 6.4 ffi (~50), 6.5 remainder.
### Prior: Phase 6.1 — migrate `vendors/sqlite` `#foreign`→`extern` (`refactor` commit
`410a52e`). **PART B PHASE 6 STARTED.** Pure source rename: all 97
`sqlite3_* … #foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ the one
stale header-comment reference, line 9). The `extern_lib` axis references the `sqlib`
`#import c` unit identically to `#foreign sqlib`, so IR/output is byte-identical —
empty snapshot diff (only `sqlite.sx` changed), and example 1624
(`vendor-sqlite-module`) stdout byte-unchanged. Suite green (647 corpus / 444 unit,
0 failed).
### Prior: Phase 5.1 — annotate A→B gate post-flip + add fn-rename case (`test` commit
`93e7b6f`). **PHASE 5 COMPLETE → PART B Phase 5 done.** The A→B gate
(`lower.test.zig`) already asserted `#foreign` ≡ `extern`/`export` byte-identical IR
for fn / global / Obj-C class; post-Phase-5.0 the fn-decl + data-global paths build
the SAME extern-named AST, so cases 1/2 are now STRUCTURALLY identical (guaranteed by
construction, not coincidence). Annotated the gate header to record this and keep it
as a regression tripwire (catches a future reader re-diverging the spellings, or a
revert of the flip); case 3 (runtime class) stays behaviorally — not structurally —
equal via the single `is_foreign_eff` field. Added a fn-rename case (case 2b,
`extern_name` axis: `c_abs` → `"abs"`) to broaden coverage beyond bare import
(verified IR-identical via `sx ir` probe before adding). Test-only, no snapshot churn.
Suite green (647 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 — fn-decl `#foreign` body-marker FLIP (`refactor` commit `6b94bb6`).
**PHASE 5.0 PARSER ROUTING COMPLETE.** The fn-body `#foreign [LIB] ["csym"]` marker
now builds the SAME extern AST postfix `extern` produces (`extern_export = .extern_`
+ `extern_lib`/`extern_name` + empty-block body) instead of a `foreign_expr` body.
Behavior-preserving — all four prereqs (visibility, variadic, plain-free, lib-ref)
ensure every downstream reader coalesces `is_foreign` with `extern_export`, so IR +
runtime are byte-identical (full corpus + A→B gate green). Decision 7 churn realised:
example 1620's lib-ref error flips "#foreign library" → "extern library" (the only
snapshot moved; hand-edited, not regen). Parser unit test updated to assert the extern
shape. Spot-checked 1219/1218/0729 (foreign rename / cvariadic / same-name) end-to-end.
**All four `#foreign` parser paths now resolved:** global (`e5ddfbe`) + fn-body
(`6b94bb6`) flipped onto extern; const-with-type is dead (deferred); runtime-class is
already coalesced (`is_foreign_eff`). `c_import.zig` auto-synthesis STILL emits
`foreign_expr` bodies (Phase 6+), so both shapes coexist — every reader stays dual.
Suite green (647 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 prereqs 3 & 4 — plain-free classification + extern lib-ref validation
(plain-free: xfail `2706521` → fix `3c94c14`; lib-ref: xfail `38c3240` → fix
`ad6aed3`). Two MORE extern/#foreign divergences found while de-risking the fn-path
flip, both now closed. **FOUR prereqs total done — the fn-decl flip fully de-risked.**
- **Prereq 3 (plain-free):** `isPlainFreeFn`/`isPlainFreeFnDecl` (resolver.zig:178,
generic.zig:815) excluded a `#foreign` body but classified an empty-block `extern`
fn as a plain free fn — so existing extern fns were wrongly counted in the bare-call
ambiguity verdict (example: two same-name `extern libc "abs"` authors errored
ambiguous, while the `#foreign` twin 0729 compiles). Both predicates now also
exclude `extern_export == .extern_`; `export` (real body) stays plain-free. Example
**1230**.
- **Prereq 4 (lib-ref validation):** `checkForeignRefs` (c_import.zig) validated only
`foreign_expr.library_ref`, so a bogus `extern nosuchunit "abs"` compiled silently
while `#foreign nosuchunit` errors (1620). Now reads the lib ref from EITHER spelling
and names the surface keyword in the diagnostic (so 1620 stays byte-unchanged).
Example **1231**.
- Two OTHER classifying sites probed and found BENIGN for extern (no flip prereq):
namespace/qualified dispatch (`registerQualifiedFn` decl.zig:2208, namespace gate
call.zig:729) — a namespaced `extern` fn resolves identically to its `#foreign` twin
(probe: `cm.c_abs(-9)` → 9 both ways; the registered qualified alias resolves to the
same extern symbol).
### Prior: Phase 5.0 prereq — extern C-variadic tail (xfail `9a2c78d` → fix `0fdc821`) — the SECOND deferred fn-path prerequisite. **BOTH original fn-path prereqs done.** The C-variadic `...` handling was keyed on the `#foreign` (`foreign_expr`)
body shape at two sites — the `is_variadic` drop in `declareFunction`
(`decl.zig:2097`) and the call-site early-out in `packVariadicCallArgs`
(`pack.zig:302`). A variadic `extern` therefore kept its trailing slice param and
slice-packed the extras → garbage at the C ABI (probe: `sum_ints(3,10,20,30)` →
53316585, not 60). Both gates now also fire for `extern_export == .extern_`, so a
variadic `extern` drops the `..args: []T`, sets `is_variadic`, and passes extras
through the C `...` slot with default argument promotion — byte-identical to its
`#foreign` twin. New example **1229** (`1229-ffi-extern-cvariadic`, JIT `#source`,
int-sum + double-avg). Suite green (645 corpus / 444 unit, 0 failed).
### Prior: Phase 5.0 prereq — visibility-gate equivalence (xfail `717c35d` → fix `7d8ba1a`) — the first of the two deferred fn-path prerequisites.
The non-transitive C-import visibility gate (`isVisible(.c_import_bare)`,
`decl.zig:2249`) used to recognise only the legacy `#foreign` body shape; a bare
`extern` fn (empty-block body + `extern_export == .extern_`) escaped the gate via
the `body != foreign_expr → return true` arm and was caught only by the general
`isNameVisible` gate — yielding the generic "not visible" wording instead of the
C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings
route to `visibleOverEdges`, and a library-bound `extern LIB` (like `#foreign LIB`)
stays unconditionally visible — so a future fn-decl `#foreign`→`extern` migration
is byte-identical at this gate. New cross-module example **1228**
(`examples/1228-ffi-extern-c-non-transitive`, main → b → c) pins the equivalence:
referencing c's lib-less `#foreign` AND `extern` twins transitively both produce
the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed).
**Empirical finding** (probe, not yet acted on): the bare-extern twin was NEVER a
silent visibility hole — the general `isNameVisible` gate already denied it; only
the *diagnostic wording* diverged. The fix aligns the wording + gate ownership.
### Prior: Phase 5.0 (global path) (`refactor` lock, commit `e5ddfbe`) — **PART B STARTED.**
First of the four `#foreign` parser paths migrated onto the extern AST: the
data-global form `name : T #foreign [lib] ["csym"];` now builds the same
extern-named `VarDecl` (`is_extern`/`extern_lib`/`extern_name`) that postfix
`extern` already produces, instead of `is_foreign`/`foreign_lib`/`foreign_name`.
Behavior-preserving — lowering coalesces both forms identically
(`decl.zig:1119,1127,1141`), so zero snapshot churn. The fn-decl, const-with-type,
and runtime-class `#foreign` paths still build the legacy AST.
### Prior: Phase 4 (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces:
(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) —
asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample
fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start
migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side
fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix
`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse
error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic
— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean
error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three
`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays
documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail).
NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a
harmless dup (both `.c`), and any non-`.c` callconv already errors on its own.
### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/
`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional
`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{`
body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime
class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx
class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same
`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/
lowering change was needed** — the new surface reuses the existing reference-vs-define path.
Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via
JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`),
**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443
unit, 0 fail).
### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
unit, 0 fail).
**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).
## Current state
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class`
(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the
**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A
COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` +
~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223
(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn,
AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426
(jni extern class), 1174/1175 (interplay diagnostics).
## Next step
**NONE — the FFI-linkage stream is COMPLETE.** `extern`/`export` fully replace
`#foreign`; the keyword is rejected; zero `foreign` remains in the gated tree (Parts
A + B, Phases 0–9 all done; the 9.4 gate passes). This stream can be archived.
Follow-ups (both DONE 2026-06-15, post-stream polish):
- ✅ Added `extern`/`export` to the editors/vscode tmLanguage keyword list as a
`storage.modifier.sx` pattern (`editors/vscode/syntaxes/sx.tmLanguage.json`).
- ✅ Dropped the vestigial `RuntimeClassPrefix.is_extern` field +
`parseRuntimeClassDecl`'s `is_extern` param (always-false dead path; the postfix
`extern`/`export` keyword is the sole reference-vs-define decider). Suite green
(644 corpus / 442 unit, 0 failed).
--- (historical: the finish-Phase-9 plan, now done) ---
**PART B — finish Phase 9: example FILENAME renames + `issues/*.md` + 9.0/9.4.**
(All `src/` identifiers + AST node + all comments/docs/example-comments are DONE.)
0. **FIRST: commit the staged `examples/` comment purge** (a classifier outage blocked
the commit; changes are `git add`ed). Message: "refactor(ffi-linkage): Phase
9.3-examples — purge 'foreign' from example .sx comments".
1. **Example filename rename** (git-mv step, snapshot-careful): rename the `*-foreign*`
example files to extern/runtime names and update every `#import`/`#include`/`#source`
ref + the `expected/<name>.*` companions. Families: `0729-modules-flat-same-name-foreign`
(+ `/a.sx`,`/b.sx` dir), `1205-ffi-foreign-global`(+`-helper`), `1207-ffi-foreign-global-from-helper`,
`1216-ffi-08-foreign-in-method`(+`.h`/`.c`), `1217-ffi-09-foreign-result-chain`(+`.h`/`.c`),
`1218-ffi-foreign-cvariadic`(+`.c`), `1219-ffi-foreign`, `1306-ffi-objc-foreign-class-chained-dispatch`,
`1318-ffi-objc-property-foreign`. ⚠ A renamed file with an `.ir`/`.stderr` snapshot that
echoes its own path will need that snapshot regenerated (intentional). Pick new names
that drop "foreign" (e.g. `…-extern-global`, `…-extern-in-method`, `…-runtime-class-chained-dispatch`).
NOTE: keep `1176-diagnostics-foreign-removed.sx` name (it's the rejection test — fine to keep "foreign").
2. **issues/*.md** (~20) — rewrite writeup prose `#foreign`/`foreign`→`extern`/`runtime-class`.
2b. **`docs/*.md`** — ALSO in the gate scope (was missed; the gate areas are now
`src/ library/ examples/ issues/ docs/ specs.md readme.md CLAUDE.md`). `docs/debugger.md`
referenced the renamed `callForeign` (fixed → `callExtern`, UNCOMMITTED with the staged
batch); sweep all of `docs/` for stale renamed-identifier refs + `foreign` prose.
3. **9.0 surface decision — RATIFIED (user, 2026-06-15): DELETE the `hash_foreign` token.**
The user explicitly flagged token.zig:121 + lsp/server.zig:1693 "this also needs to
go" — total purge, accept `#foreign`→generic error (no friendly migration hint). This
is the LAST src change; it is load-bearing → needs a build + test + 1176 regen (do it
when mutating Bash is back). Steps:
- token.zig: remove `hash_foreign` enum (121).
- lexer.zig: remove the `.{ "#foreign", Tag.hash_foreign }` map entry (91), drop
`#foreign` from the directive-list comment (72), DELETE the `lex hash_foreign` test
(626-631, incl. `#foreignx`).
- parser.zig: remove the 4 `self.current.tag == .hash_foreign` rejection sites (268
caller / 327 / 419 / 2024) + their messages, AND the 2 lookahead refs (`hasFnBody…`
~3658 + ~3676). ⚠ Decide what `#foreign` lexes to with no keyword entry (likely an
error/unknown-directive token) and confirm the parser surfaces a sane error.
- lsp/server.zig: remove the `.hash_foreign,` arm (1693).
- **1176-diagnostics-foreign-removed**: its expected stderr is the now-deleted
"`#foreign` has been removed…" message → it WILL change. Regen 1176's snapshot to
whatever the generic post-deletion error is (intentional), OR delete 1176 entirely
(its purpose — a friendly rejection — no longer exists). Recommend: keep 1176 as a
"`#foreign` is no longer a directive" regression, regen its snapshot. NOTE: after
this, 1176 may still contain `#foreign` in its SOURCE (the rejected token) — that's
the only legitimately-remaining `foreign` in `.sx`, OR rename/rework it to avoid even
that if the gate must be absolute.
4. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` → 0 (no keep-list
left except possibly 1176's source token + `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored C).
--- (historical: the prose-purge plan, now mostly done) ---
**PART B — finish Phase 9: the COMMENT / DOC / issues text purge** (all `src/`
identifiers + the AST node are already done; remaining is prose). Lower-risk than the
renames (text only, mostly snapshot-neutral) but needs per-instance reading — NOT a
blind sed. Footprint: `src/` ~205 (all comments now), `examples/*.sx` ~100 comments,
`issues/*.md` ~20 files, docs (specs/readme/CLAUDE).
Order:
1. **src/ comments** (~200) — reword `foreign`→`extern`/`runtime-class` to match the
renamed identifiers. KEEP: the rejection-message string, the `hash_foreign` token +
its `#foreignx`/`lex hash_foreign` test, and any comment that legitimately explains
the cutover (it must name `#foreign` to say it's removed). The ast.zig FnDecl comment
still says "mirroring `#foreign LIB "csym"` (foreign_lib/foreign_name)" — reword.
2. **examples/*.sx comments** — the deferred provenance comments (full list in the prior
Next-step revision / git). ⚠ Many CONTRAST `#foreign` vs `extern` — reword to stay
coherent. ⚠ `1219-ffi-foreign.sx` prints `"foreign-rename: {}"`/`"=== 15. Foreign ==="`
to STDOUT — changing those regens its snapshot (intentional). `1176`/`1216` legitimately
discuss `#foreign` removal — keep minimal `#foreign` mentions where the test IS about it.
3. **issues/*.md** (~20) — rewrite writeup prose to `extern`/`export`/`runtime-class`.
4. **docs** — specs.md (rename "Foreign Function Interface" heading → "C Interop"; the
`#import c` "foreign declarations" prose; the comptime "foreign function calls" line;
§3344 "foreign code can't observe the error channel"), readme.md (211-212 the `#import
c` exemption prose), CLAUDE.md (host_ffi `#foreign("c")` ref → `extern`; "foreign calls").
5. **9.0 surface decision** (recommend KEEP `hash_foreign` token + rejection for a good
deprecation — then it + the message + 1176 + `#foreignx` are permanent gate-exempt keeps).
6. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` minus the keep-list → 0.
KEEP-LIST (gate-exempt): `SQLITE_CONSTRAINT_FOREIGNKEY` + SQLite API names, vendored
`library/vendors/sqlite/c/*`, the `hash_foreign` token + `#foreignx` test + rejection
message string, `1176-diagnostics-foreign-removed.sx` (rejection test must contain it).
**Gate (scoped per user 2026-06-15):** `grep -rniIE 'foreign'` → 0 across `.sx` files,
all docs, and our `src/` Zig — EXCLUDING the legitimate keeps listed in Last completed
step (SQLite API names, vendored C, the rejection test/message + `hash_foreign` token).
Remaining, in suggested (dependency-safe) order:
1. **9.1d — eliminate `foreign_expr`** (last linkage item): migrate `c_import.zig`
auto-synthesis to build the extern shape instead of a `foreign_expr` body (the Phase
5.0 fn-body flip applied to auto-synth), then delete the `foreign_expr` AST node +
`ForeignExpr` + all readers (25). Snapshot-neutral; verify full corpus (the `#import c`
examples 1215/1216/1217 + sqlite 1624 exercise it).
2. **9.2 — runtime-class family rename → `Runtime*` (Decision 5).** The BIG one, do as
small per-identifier commits with `zig build` after each (snapshot-neutral). Targets
(counts): `ForeignClassDecl`(65)→`RuntimeClassDecl` · `foreign_path`(62)→`runtime_path`
· `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `ForeignMethodDecl`(31)
· `foreign_class_decl`(30) · `foreign_expr`-gone-by-now · `ForeignClassMember`(20) ·
`ForeignFieldDecl`(15) · `ForeignClassDecl.is_foreign`(the live one)→e.g. `is_reference`
· `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` ·
`findForeign*InChain` · `resolveForeign*` · `register*ForeignClass*` · `*ForeignRefs` ·
`ForeignRuntime` · `current_foreign_class`/`_method`. ⚠ COUPLED .sx↔.zig hook names:
`jni_main_foreign_path_at`/`jni_main_foreign_paths`/`hookJniMainForeignPathAt`/
`foreignPathToJavaName`/`splitForeignPath` span build.sx + bundle.sx + compiler_hooks.zig
+ specs.md (2975/3049) — rename all four sites together.
3. **9.x-src-comments** — the ~200 bare-`foreign` comments in `src/` (rename last, since
many reference identifiers that 9.1d/9.2 rename; do AFTER those so the comment text
matches the new names).
4. **9.3-examples comments** — the deferred `.sx` provenance comments (0716, 0729, 1205/
1207/1216/1218/1219/1220, 1223-1231, 1306/1308/1315/1318/1320/1321/1331/1332/1348/1349,
1412/1414/1417/1418/1419/1426, 0117/0415, 1140/1141/1125, issues/0030.sx). ⚠ Many
CONTRAST `#foreign` vs `extern` ("no `#foreign`, no `#library`") or reference renamed
internals — rewrite each to stay coherent (NOT blind sed). ALSO: `1219-ffi-foreign.sx`
prints `"foreign-rename: {}"` to STDOUT — changing it regens the snapshot (intentional).
5. **9.3-issues** — `issues/*.md` writeups (~20 files) → rewrite `#foreign`/`foreign` to
`extern`/`export`/`runtime-class` per the renames.
6. **9.3-docs** — specs.md (12: rename "Foreign Function Interface" heading → "C Interop";
the `#import c` "foreign declarations" prose; the jni_main_foreign_path_at refs with #2),
readme.md (2), CLAUDE.md (2: host_ffi `#foreign("c")` ref + "foreign calls").
7. **9.0 surface decision** — keep `hash_foreign` token + rejection (recommended: good
deprecation) vs delete it. If kept, the token + the rejection-message string + 1176 are
permanent legitimate keeps; the gate excludes them.
8. **9.4 gate** — `grep -rniIE 'foreign'` over the gated set minus the keep-list → 0.
- **6.2 verification note (carry forward):** the `platform/` runtime modules
(uikit/android/android_jni) are NOT compiled by any marker'd host corpus test — verify
future platform-adjacent migrations via direct `sx ir` on importers (1610/1606 compile
uikit on host) or import probes, not the corpus alone.
- **Phases 6–7** (`refactor` batches, empty snapshot diff per batch): migrate the
stdlib + examples from `#foreign` spelling to `extern`. Because the AST is already
unified, this is a pure SOURCE rename (`… #foreign LIB "sym";` → `… extern LIB "sym";`
for fns; the global/const forms similarly), and IR/output must be byte-identical per
batch. NOTE: `c_import.zig` auto-synthesis (`#import c {#include}`) still BUILDS
`foreign_expr` bodies internally — that's a compiler-internal path, migrated separately
(likely Phase 8/9 area), not a source-spelling change.
- **Then Phase 8** (cutover: hard-reject the `#foreign` keyword) and **Phase 9** (purge
all `foreign` identifiers — needs Decision 5 [done, `Runtime*Class*`] + Decision 6
[open, historical carve-out]).
**Watch items carried forward:**
- `c_import.zig:262` auto-synthesis still emits `foreign_expr` — both shapes coexist
until that path is migrated; keep every `body.data == .foreign_expr` reader dual
(checked exhaustively this stream).
- const-with-type `#foreign` parser path (`parser.zig:316`) is still on `foreign_expr`
but DEAD (registers no const); migrate or delete it at the Phase 8 cutover.
- The `decl.zig:2055` "foreign symbol … already bound" dedupe message is keyword-neutral
and fires for both forms — no churn, but reword to "extern" at cutover for consistency. Route the fn-decl `#foreign` path so a
`#foreign` fn builds the SAME extern AST that postfix `extern` already produces,
instead of a `foreign_expr` body. This is the highest-value path (the bulk of
`#foreign` usage). Key sub-questions to resolve before/while routing:
- The `foreign_expr` node carries `library_ref` + `c_name`; the `extern` fn carries
`extern_export = .extern_` + `extern_lib` + `extern_name` on the FnDecl with an
empty-block body. Migration = the parser's fn-body `#foreign` arm
(`parser.zig:~2062`) builds the extern shape (set `extern_export`, map
`library_ref→extern_lib`, `c_name→extern_name`) rather than a `foreign_expr`.
- Lowering ALREADY coalesces the two at every fn site checked this stream
(`decl.zig` 2088/2124/2132/2156/2324/2531 read `is_foreign OR extern_export`),
and the two prereq gates (visibility `decl.zig:2249`, variadic `decl.zig:2097` +
`pack.zig:302`) now do too — so the migration should be behavior-preserving with
ZERO snapshot churn. VERIFY with the A→B gate test (`lower.test.zig`) + a full
`zig build test` after routing; any churn means a site still reads `foreign_expr`
structurally and must be coalesced first.
- ⚠ This ALSO migrates the **const-with-type** path implicitly IF it shares the same
`foreign_expr`→extern reshape (it builds `const_decl{value=foreign_expr}`). Decide:
reshape the const path's value node alongside, or leave the dead const path on
`foreign_expr` until Phase 8 cutover. The const path is dead (see findings below),
so leaving it is acceptable; but the parser arm is shared-ish — check whether the
fn-body arm change touches it.
- Cadence: because the migration is behavior-preserving (no churn), it's a single
`refactor`/lock commit (like the 5.0 global-path commit `e5ddfbe`), NOT an
xfail→fix pair.
**Investigation findings (this session — reorder the remaining paths):**
- **const-with-type** (`parser.zig:316`, `name :: type_expr #foreign …`) is a
**DEAD path**: it builds `const_decl{value = foreign_expr}`, but
`registerTypedModuleConst` (`decl.zig:848-851`) bails on a `foreign_expr` value
(`else => return`), so it registers no const and emits no symbol — a probe
(`g_abs :: FP #foreign "abs";`) returns `unresolved 'g_abs'` at the use site, and
the form is used NOWHERE in `library`/`examples`/`issues`. Its migration target is
ambiguous because the `foreign_expr` value node is SHARED with the fn-decl path,
which isn't migrated yet. **Decision (user, 2026-06-14): defer it — migrate it
alongside the fn-decl path once `foreign_expr`'s extern shape is decided.** The
checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the
"confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it).
- **runtime-class prefix** (`parser.zig:~1351`, `#foreign #objc_class/#jni_class`) is
**ALREADY coalesced**: both prefix `#foreign` and postfix `extern` feed the single
`is_foreign_eff`→`is_foreign` field on `foreign_class_decl` (`parser.zig:1421-1432`),
so there is NO Phase 5.0 AST change for it — only the Phase 9.2 `Runtime*Class*`
rename remains. Drop it from the Phase 5.0 path list.
So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then
the fn-decl `#foreign` body-marker migration. const-with-type + runtime-class need
no standalone Phase 5.0 commit.
Then Phase 5.1 (`lock`): unit test that `#foreign` and `extern` produce identical IR (the
A→B gate already covers fn/global/class — extend or reuse `lowerSrcToIr`). Then Phases 6–7
migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject
`#foreign`), Phase 9 total `foreign` purge.
**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target
(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep
`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames;
the plan says confirm before Phase 9, but worth raising with the user before sinking Part B
effort. **Also pick up the two Deferred items below at the start of Part B** (the
visibility-gate equivalence in particular needs a cross-module example).
**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).
**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md`
document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) ~~visibility-gate
equivalence~~ — **DONE** (`717c35d`/`7d8ba1a`): the `c_import_bare` gate now polices a
lib-less `extern` fn identically to its lib-less `#foreign` twin (same C-specific
diagnostic); a library-bound `extern LIB` stays unconditionally visible. Locked by the
cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the
general `isNameVisible` gate already denied it; only the diagnostic wording diverged.)
## Open decisions
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B:
- **Decision 5 RATIFIED** (user, 2026-06-14): runtime-class rename target = `Runtime*Class*`
(object-model axis, not linkage). Drives the Phase 9.2 identifier renames.
- **Decision 6 RATIFIED** (user, 2026-06-15): **PURGE EVERYTHING** — the Phase 9.4 gate is
absolute, including `issues/*.md` writeups (NOT the recommended keep-provenance default).
Every `#foreign`/`foreign` reference in the gated tree (`src/ library/ examples/ issues/
specs.md readme.md CLAUDE.md`) is rewritten to `extern`/`export`; provenance lives in git
history + `(Regression issue NNNN)` notes, not the keyword spelling.
- **Decision 7 RATIFIED** (user, 2026-06-15): **accept the churn** — `#foreign`-spelled
decls produce `extern`-worded diagnostics; example 1620 regenerated (only snapshot moved).
Aligns with Part B's extern-only end state; the interim oddity is cosmetic and removed at
the Phase 8 cutover. Landed in the fn-body flip `6b94bb6`. (Original framing below.)
— interim diagnostic wording for `#foreign`-spelled decls (gated the fn-body flip). Once the flip lands, a `#foreign`-spelled fn builds the extern AST, so any
diagnostic that reads the unified AST can no longer tell the user wrote `#foreign` vs
`extern`. Concretely, example 1620's lib-ref error flips "#foreign library…" →
"extern library…". Options: **(A, recommended)** accept the narrow churn — regen 1620 as
intentional; it aligns with Part B's `extern`-only end state and the interim oddity
(`#foreign` source → "extern" message) is cosmetic and short-lived (Phase 8 cutover
removes `#foreign`). **(B)** retain a one-bit surface marker on `FnDecl` (`wrote_foreign`)
so interim diagnostics stay keyword-accurate (zero churn, small extra plumbing, marker
deleted at cutover). Affects only diagnostic wording — IR/behavior identical either way.
## Log
- (9.0 + 9.3 + 9.4) **PHASE 9 COMPLETE — STREAM DONE; 9.4 GATE PASSES.** Deleted the
hash_foreign token (9.0, `dfae690`); purged all `foreign` comments incl. capital-F
(src/examples/docs/editors); renamed 10 `*-foreign*` example files + dedup'd 1218
(`b52d424`); rewrote 20 issues/*.md + renamed 0043 (`b9cfe25`). Gate: zero `foreign`
in the gated tree except `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored sqlite c/. Suite
green (644/443). User flagged several leftover areas mid-purge (docs/, editors/,
capital-Foreign comments, the token) — all addressed.
- (9.3 src capital-Foreign) Fixed the case-sensitivity gap — my earlier src verify grep
was case-sensitive, missing ~21 capital `Foreign`/`FOREIGN` comments (Foreign-class→
Runtime-class, Foreign path→Runtime path, Foreign decls→Extern decls, FOREIGN function→
extern function, etc.) across calls/inst/ffi_objc/jni_descriptor/emit_llvm/c_import/
lower.* /ops.zig. All reworded via Edit (comments only — no build impact). UNCOMMITTED
(mutating Bash blocked by a classifier outage). After this, src `foreign` = ONLY the
`hash_foreign` token machinery + 4 rejection messages (the 9.0-delete targets).
- (9.0 RATIFIED) User: DELETE the hash_foreign token (total purge). Pending build+regen.
- (9.3 text purge) Purged `foreign` from all `src/` comments (`dc51c4b`), specs/readme/
CLAUDE (`e99383f`), and all example .sx comments (STAGED, commit pending a classifier
outage). Fixed 2 user-facing diagnostics (type-annotation error, Android jni_main help).
1219 stdout labels Foreign→Extern (regen). Suite green (646/444). Remaining: example
FILENAMES + issues/*.md + the 9.0 token decision + 9.4 gate.
- (9.2a-d) **RUNTIME-CLASS IDENTIFIER PURGE COMPLETE** (Decision 5 → `Runtime*`).
9.2a types (`3354446`), 9.2b fns+state+`is_extern` flag (`5c8af6e`, fixed `a15a868`
per user: reuse `is_extern` not new `is_reference`), 9.2c extern-ref validators →
`Extern` (`d27be42`), 9.2d `foreign_path`→`runtime_path` coupled across the build-hook
boundary + 37 `.ir` regens (`8cca3b9`). `src/` now has ZERO `foreign` identifiers
(only comments + the kept token/message remain). Suite green throughout.
- (9.1d) Eliminated the `foreign_expr` AST node — migrated `c_import.zig` auto-synth to
the extern shape, deleted the node + all readers. `refactor` `7ffdc7d`.
- (9.1c) Deleted dead `VarDecl.is_foreign`/`foreign_lib`/`foreign_name` (global `#foreign`
rejects → write-dead); 3 decl.zig readers simplified to `vd.extern_name`/`vd.is_extern`.
Snapshot-neutral; suite green (646/444). `refactor` `cd14794`.
- (9.1b) "foreign symbol already bound" diagnostic + resolveFuncByName panic → "extern
symbol"; intentional 1172 regen. Suite green. `refactor` `b78e7dd`.
- (9.1a) **PHASE 9 STARTED.** 5 collision-free linkage renames (callForeign→callExtern,
marshalForeignArg, dedupeForeignSymbol, foreign_name_map→extern_name_map,
is_foreign_c_api). Snapshot-neutral; suite green. `refactor` `b838f63`. Decision 6
scoped by user: purge `.sx` + docs + our `src/` Zig, keep legitimate hits (SQLite API
names, vendored C, the rejection test/message + hash_foreign token).
- (8.1 cutover) **PHASE 8 COMPLETE.** Parser hard-rejects `#foreign` at all 4 sites
(const/global/fn-body via `self.fail`; runtime-class via `self.failAt` at the caller,
new helper); greens xfail 1176. Deleted obsolete 1174 + 1620, the GATE A→B test +
`lowerSrcToIr` helper; converted the in-source parser test to postfix `extern`;
`extern_export` → `const`. specs.md + readme.md drop `#foreign`. Suite green
(646/444). `feat!` `3811311`.
- (8.0 xfail) Added `1176-diagnostics-foreign-removed.sx` pinning the desired rejection.
RED (still accepted). `test`/xfail `8180faf`.
- (8 pre-cutover) Migrated the 4 multi-file example companions Phase 7 missed
(0729/a+b, 1617/c, 1623/mod). `refactor` `d132aab`.
- (8 pre-cutover) Migrated keyword-neutral diagnostics 1172 (decl→extern, message stays
internal "foreign symbol") + 1228 (→ two foreign-free extern symbols c_abs_one/_two),
intentional snapshot regens reviewed. `refactor` `720556b`.
- (8 pre-cutover) Migrated the 7 identity `ffi-foreign-*` test decls to extern/export
(decls only; comments left for Phase 9.3). `refactor` `2cce6a3`.
- (7.4 stragglers) **PHASE 7 MIGRATABLE WORK COMPLETE.** Migrated 16 fn/global examples
(0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/1622/1628/1635/1636) `#foreign`→
`extern`; 1607/1608/1616 (unmarked) verified by `sx ir` probes. 24-file keep-list remains
by design (deferred to Phase 8). Suite green (647/444). `refactor` `1a8991a`.
- (7.3 14xx) Migrated 13 jni examples (1410-1419/1423/1424/1425). 1417 (all-runtimes) hit
a parse-error trap: a `#(objc|jni)_class`-only regex left `extern` in PREFIX position on
`#jni_interface`/`#objc_protocol`/`#swift_*` lines → fixed with the GENERAL
`#foreign #(\w+)("X") {`→`#$1("X") extern {` rewrite (all such directives accept the
postfix modifier, probed). Kept 1426 (comment-only). Suite green. `refactor` `2888f6f`.
- (7.2 13xx) Migrated 18 obj-c examples (1308/1311-1321/1341-1347): prefix→postfix import
classes + fn markers. Kept identity 1306/1318, comment-only 1332/1348/1349. No 13xx
snapshot asserts on foreign. Suite green. `refactor` `a68f7c2`.
- (7.1 12xx) **PHASE 7 STARTED.** Migrated 12 incidental plain-C examples
(1200/1206/1209-1215/1220/1221/1222) `#foreign`→`extern`; output byte-identical,
empty snapshot diff, corpus-validated. Established the keep-list policy (see Last
completed step): kept 1172/1174/1620/1228 + ffi-foreign-* (1205/1207/1216/1218/1219)
+ comment-only 1223/1229/1230/1231 for Phase 8. Suite green (647/444). `refactor`
`731fb8d`.
- (6.5 gpu) **PHASE 6 COMPLETE.** Migrated `gpu/gles3.sx` + `gpu/metal.sx` (3 sites);
`library/` now `#foreign`-free (`grep -rln '#foreign' library/` → 0). Verified
byte-identical `sx ir` on importers 1610/1606. Suite green (647/444). `refactor`
`32a7628`.
- (6.4 ffi) Migrated `ffi/` objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers +
objc.sx's 2 import classes (prefix→postfix `extern`). objc/objc_block validated by 50
marked 13xx examples; raylib/ffi-sdl3/wasm by `sx ir` probes pre/post. Empty snapshot
diff; suite green (647/444). `refactor` `666a2e2`.
- (6.3 std) Migrated 11 `std/` modules (~60 sites): cli/core/fmt/fs/log/net.kqueue/
process/socket/thread/time/trace. All fn-decl markers (bare / `libc`|`tlib` LIB ref /
`libc "csym"` rename) → `extern …` + 2 comment mentions; no class forms. Host-corpus-
exercised → empty snapshot diff validates. Suite green (647/444). `refactor` `59f90d2`.
- (6.2 platform) Migrated `platform/` (uikit/android/android_jni/sdl3, ~64 sites):
30 fn `#foreign;`→`extern;`, 34 import classes prefix `#foreign #objc/jni_class`→
postfix `… extern {`, 4 defined `Sx*` objc classes → `… export {`. Behavior-
preserving, empty snapshot diff. Verified byte-identical `sx ir` on uikit importers
1610/1606 + sdl3 probe; android via identical 4-error dedup set (host-only module).
Suite green (647/444). `refactor` `2cd5d7b`. NOTE: these runtime modules aren't in
the marker'd host corpus — verified out-of-band.
- (6.1 sqlite) **PHASE 6 STARTED.** Migrated `vendors/sqlite/sqlite.sx`: 97
`… #foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ line-9 comment).
`extern_lib` references the `sqlib` `#import c` unit like `#foreign sqlib`; IR
byte-identical, empty snapshot diff, example 1624 stdout unchanged. Suite green
(647/444). `refactor` `410a52e`.
- (5.1 gate annotate) **PHASE 5 COMPLETE.** Annotated the A→B gate header
(`lower.test.zig`) to record that post-Phase-5.0 the fn/global `#foreign` paths
build the same extern-named AST → cases 1/2 are structurally (not coincidentally)
identical; the gate stays as a regression tripwire. Added fn-rename case 2b
(`c_abs` → `"abs"`, `extern_name` axis), IR-identical per a `sx ir` probe.
Test-only, no snapshot churn. Suite green (647/444). `test` `93e7b6f`.
- (5.0 fn-body flip) **PHASE 5.0 PARSER ROUTING COMPLETE.** Flipped the fn-body
`#foreign` parser arm (`parser.zig:~2062`) onto the extern AST (empty-block body +
`extern_export = .extern_` + extern_lib/extern_name); `extern_export` made `var` so
the body arm can route onto it. Updated the parser unit test to assert the extern
shape. Behavior-preserving via the four prereqs; only example 1620's lib-ref message
churned ("#foreign library"→"extern library", Decision 7, hand-edited). Suite green
(647 corpus / 444 unit). `refactor` `6b94bb6`.
- (5.0 prereq plain-free xfail) Added `1230-ffi-extern-same-name-authors` (two flat
authors of `absval` via `extern libc "abs"`; the `extern` twin of `#foreign` 0729).
RED — extern authors wrongly counted as ambiguous (646/1 fail). `test`/xfail `2706521`.
- (5.0 prereq plain-free fix) `isPlainFreeFn`/`isPlainFreeFnDecl` now also exclude
`extern_export == .extern_` (external C symbol, no sx body; name-keyed first-wins like
`#foreign`); `export` stays plain-free. 1230 green (`absval = 7`). Suite green (646/444).
`fix`/green `3c94c14`.
- (5.0 prereq lib-ref xfail) Added `1231-ffi-extern-undeclared-lib` (`extern nosuchunit
"abs"` — bogus lib ref). RED — compiles silently (extern lib ref unvalidated).
`test`/xfail `38c3240`.
- (5.0 prereq lib-ref fix) `checkForeignRefs` (c_import.zig) now reads the lib ref from
either spelling (foreign_expr.library_ref OR extern_lib) and names the surface keyword,
so 1620 (#foreign) is byte-unchanged and 1231 (extern) gets "extern library … not
declared". 1231 green. Suite green (647/444). `fix`/green `ad6aed3`. **ALL FOUR fn-path
prereqs DONE → fn-body flip de-risked; awaiting Decision 7 (interim wording).**
- (5.0 prereq variadic xfail) Added `1229-ffi-extern-cvariadic` (JIT `#source`,
int-sum + double-avg, `extern` C-variadic). Expected snapshot pins the DESIRED
correct output. RED (variadic `extern` slice-packs extras → garbage:
`sum_ints(3,10,20,30)` → 53316585; doubles → 0.0). `test`/xfail `9a2c78d`.
- (5.0 prereq variadic fix) Extended the two C-variadic gates — the `is_variadic`
drop in `declareFunction` (`decl.zig:2097`) and the early-out in
`packVariadicCallArgs` (`pack.zig:302`) — to fire for `extern_export == .extern_`
as well as a `foreign_expr` body. 1229 green (`60` / `2.000000`). Suite green
(645 corpus / 444 unit, 0 failed). `fix`/green `0fdc821`. **BOTH fn-path prereqs
DONE → fn-decl `#foreign` body-marker migration unblocked.**
- (5.0 prereq vis xfail) Added cross-module example `1228-ffi-extern-c-non-transitive`
(main → b → c). Main references c's lib-less `#foreign` + `extern` twins
transitively; expected snapshot pins the DESIRED equivalent C-specific
diagnostic for both. RED (extern twin gets the generic "not visible" wording —
443/444). `test`/xfail commit `717c35d`; the fix greens it.
- (5.0 prereq vis fix) Extended `isVisible(.c_import_bare)` (`decl.zig:2249`) to
switch on the body: a `foreign_expr` body OR an `extern_export == .extern_` decl
with no lib both route to `visibleOverEdges`; a library-bound decl stays
unconditionally visible. 1228 green — both twins emit "C function not visible".
Suite green (644 corpus / 444 unit, 0 failed). `fix`/green commit `7d8ba1a`.
**Deferred prereq (b) CLOSED.** Investigation this session also found
const-with-type is a DEAD parser path (defer per user) and the runtime-class
prefix is already coalesced (no Phase 5.0 change) — see Next step.
- (5.0 global) **PART B STARTED.** Routed the `#foreign` data-global parser path
(`parser.zig:425`) onto the extern-named `VarDecl` (`is_extern`/`extern_lib`/
`extern_name`) — the same AST postfix `extern` builds. Behavior-preserving
(lowering coalesces both at `decl.zig:1119,1127,1141`); zero snapshot churn. Suite
green (444/444 unit, 643 corpus). `refactor` lock, commit `e5ddfbe`. Remaining
Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic
prereqs), runtime-class prefix (1305).
- (init) Plan written; FFI-linkage stream opened.
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
classification + `lex linkage keywords` test. Suite green; no identifier collisions
in the corpus. `lock` commit.
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
`VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
parser unit tests. Suite green (443/633). `lock` commit.
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
(decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
Suite green (443/633). `lock` commit.
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
`lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
`declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
recovered a formatter-clobbered `parser.zig` — see Known issues.)
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
`extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
extern_name → declare under C name, map sx→C) and extends the dedupe guard to
extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
`green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
`is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
(636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
(JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
(`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
milestone (see Next step). Spike reverted; no commit.
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
`corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
(637/443). `green` commit.
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
`triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
undefined). `xfail`; 2.2b greens it.
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
the body promotes into the C-named function (`define @triple_c`). sx-side call sites
resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.
- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl`
(optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed
`is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses
the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443).
`green` commit. **PHASE 3 COMPLETE.**
- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`,
parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class,
`counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per
the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens`
newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr
0-byte per repo convention; runner normalizes both.)
- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`:
`#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class.
Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the
equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three).
Suite green (641/444). `test` commit.
- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign`
+ postfix `export` on an aggregate previously surfaced a confusing internal
"emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`:
`parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green.
- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the
`#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit.
- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on
one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`:
`parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite
green (643/444). **PHASE 4 COMPLETE → PART A DONE.**
- (golden-fix) **`-Dupdate-goldens` churn RESOLVED.** Root cause was NOT a code bug:
`writeGolden` always writes `content + "\n"` (empty → canonical 1-byte `\n`, used by 484
of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte
*outliers* (verify trims trailing `\n` so both forms passed, but regen always rewrote them
to 1-byte). Conformed all 5 to the 1-byte form → `-Dupdate-goldens` is now idempotent, no
more churn. (Separately: a flaky `0712-sha256-streaming` >10s timeout appears only under
concurrent `zig build` load — not a real failure; re-run serially.)
## Known issues
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
working-tree `src/parser.zig` between commits — it reformatted one-liners AND
silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
committed version). **After any Edit-tool change to a file the IDE may have open,
rebuild + run the affected example before trusting the edit.**

980
current/CHECKPOINT-FIBERS.md
View File

@@ -1,980 +0,0 @@
> **SUPERSEDED (2026-06-28).** The fiber-async API tracked here — `Task` / `go` / `wait`
> / `cancel` — was RETIRED and folded into the unified `context.io` async stack in
> PLAN-IO-UNIFY Phase 5. See `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for
> the current design. The `Scheduler` ENGINE primitives (swap_context, mmap stacks,
> spawn / run / yield_now / suspend_self / wake / sleep / block_on_fd, virtual-time
> timers) REMAIN and are still accurate. Below is a historical record.
# CHECKPOINT-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step at a time,
per the cadence rule). New corpus category: `18xx` concurrency.
## Last completed step
**B1.6 — aarch64-LINUX port of the M:1 fiber runtime (sched.sx).** `library/modules/std/sched.sx`
now runs end-to-end on aarch64-linux as well as aarch64-macOS, validated **byte-identical** on both
via Apple `container` (static ELF, no emulation). The per-OS bits are comptime-branched:
- `MAP_AP` (mmap MAP_ANON flag) — `inline if OS == { case .linux: 0x22 case .macos: 0x1002 }`,
exhaustive on the supported OSes (no default → a new target fails loud on use).
- The fd-readiness backend — kqueue on darwin, **epoll on linux**. The `epoll` import is scoped to
the linux branch (`inline if OS == .linux { ep :: #import "modules/std/net/epoll.sx" }`) so darwin
never pulls epoll types into the concurrency examples (the std-barrel-drift rule). `block_on_fd`, the
run-loop Mode-2 drain, and `cancel_io_waiter_for` each branch kqueue/epoll; epoll additionally
`EPOLL_CTL_DEL`s on fire + on early-wake (EPOLLONESHOT only DISABLES, kqueue EV_ONESHOT auto-removes).
- The first-entry trampoline was redesigned from a per-OS hand-written global-asm symbol to a **naked
sx fn** `fib_tramp` (`mov x0, x19; br x20`) + register-indirect dispatch (spawn presets
`regs[1] == x20 == &fib_dispatch`), so no per-OS `.global _fib_tramp`/`fib_tramp` symbol literal is
needed. This sidesteps a compiler bug (wrapped top-level `asm` dropped — now **issue 0194**, OPEN).
**Bug fixed en route (issue 0193 Bug A):** the tramp redesign initially bus-errored on the 1817
go/wait/sleep capstone (both OSes) because `fib_dispatch` was no longer pinned to the C-ABI (the
original pinned it via `export "fib_dispatch"`, which the redesign dropped). Without a C-ABI pin
`fib_dispatch` uses sx's internal ABI (x0 = implicit `context`, `self` shifted to x1), but the
trampoline hands `self` in x0 (C-ABI) → on first entry the body runs (x1 happens to alias `self`) but
the closure then loads `regs[1] == &fib_dispatch` as its first capture and recurses forever → stack
overflow. **Fix: annotate `fib_dispatch` `abi(.c)`** (pins it to C-ABI, `self` in x0, with no public
symbol — it's reached only by address through the trampoline). Root cause found
via lldb on an AOT macOS build; confirmed by an adversarial source review (`src/ir/lower/decl.zig`).
The 1817 capstone in the suite guards the fix. Suite GREEN **817/0**; 1811/1814/1816/1817 byte-identical
macOS host ↔ aarch64-linux container.
### Earlier — B1 follow-up — `Scheduler.deinit` (close the bounded leaks). Post-B1 non-blocking cleanup: a
terminal `deinit` on `library/modules/std/sched.sx`'s `Scheduler` releases the resources B1 documented
as leaked. Frees, in order: (1) any fibers still enqueued ready (leak-safety net for `spawn`/`go`
without `run()` — `munmap` stack + free struct; a suspended off-queue fiber is unreachable, but a clean
`run()` aborts on orphans so none survive it); (2) every heap `*Task` from `go` — newly tracked via a
`task_allocs: List(*void)` field appended in `go` (the scheduler otherwise has no handle on its generic
`Task($R)`s); (3) the three `List` backings (`task_allocs`/`timers`/`io_waiters`, all grown through
`own_allocator`); (4) the lazily-opened kqueue fd (`close`, reset to `-1`). NOT freed (unchanged
language limitation): the per-`spawn`/`go` closure env (sx exposes no env-free). Idempotent (rests on
`List.deinit` nulling `items` + the `kq`/`ready_head` resets); TERMINAL contract — no scheduler-owned
handle (`*Task`, `*Fiber`, the scheduler) is usable after `deinit`.
- Added a canonical `close :: (i32) -> i32 extern libc` (matches the dedupe-canonical signature 1816
already uses) + the `task_allocs` field.
- Locked by `examples/concurrency/1820-concurrency-fiber-scheduler-deinit.sx` (aarch64-macOS `.build
{"target":"macos"}`, runs end-to-end): one run touches every freed resource — a SLEEPER (`timers`), a
pipe READER `block_on_fd` + WRITER (kqueue fd + `io_waiters`), two `go` tasks (`Task`s + `task_allocs`)
— then `deinit`. Verified by a tracking `GPA`: `freed by deinit: 5`, `live after deinit: 5` (the
RESIDUAL = the 5 documented closure envs, not a bug), `kq open after run: true` → `kq after deinit:
-1` (the genuinely-open kqueue fd is closed), `read: 3 [97 98 99]` (the fd path actually ran). Counts
captured into locals BEFORE printing (`print` allocates format temporaries through the same GPA).
- **Adversarially reviewed (worker):** no real memory-safety bug in the supported (deinit-after-`run`)
path — reap-loop reads `f.next` before freeing `f`, the three freed List backings + Tasks + kq are all
disjoint + scheduler-owned, no over-free, idempotent. The one CRITICAL it raised was a DOC contradiction
(step-(1) defensive reap vs step-(2) "post-run only"), reconciled by spelling out the terminal contract.
Its 0154-over-store concern (`.{}`→`List` writes in `init` could clobber `kq`) was PROBED and cleared:
`kq == -1` immediately after `init`, all fields clean. Suite GREEN **759/0**.
### Earlier — B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE
A single capstone exercises the whole
colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async
`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)`
`swap_context` on guarded `mmap` stacks (B1.0–B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`:
a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in
SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in
DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual
clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**.
**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the
`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler,
suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via
kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx`
(1800–1817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1,
zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a;
0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`),
0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`;
routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env
/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve.
### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS)
`library/modules/std/sched.sx` now lets a
fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready.
Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes),
`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT`
constants) rather than re-deriving the FFI — sched.sx imports it as `kqb`. Added to `Scheduler`:
- `kq: i32` (LAZY — `-1` in `init`, opened by the first `block_on_fd`, so a pure-compute /
virtual-timer scheduler never opens a kqueue fd; leaks one fd at exit once opened, same class as the
documented spawn-env / go-Task leaks — no deinit yet);
- `io_waiters: List(IoWaiter)` (`IoWaiter :: struct { fd: i32; fiber: *Fiber; }`, grown through
`own_allocator` per the long-lived-container rule);
- `block_on_fd(self, fd, want_read)` — lazily opens `kq`, arms a one-shot `EVFILT_READ` registration,
records an `IoWaiter{fd, current}`, then `suspend_self()`. Guards a null `current` (loud abort, like
`sleep`); `want_read=false` (write-readiness) is not wired yet → loud abort rather than silently
arming a read filter.
- Run-loop: after the ready queue drains, **Mode 1 (virtual time)** fires the earliest pending timer
(takes precedence — a program uses `sleep` OR fds, documented non-unification limitation); **Mode 2
(real fd)** — if `io_waiters` is non-empty, BLOCK on `kq_wait(kq, evbuf, MAXEV=16, -1)` (null
timeout), then for each fired event match `ev.ident` back to its waiter, evict it, and `wake` the
fiber; **else** break. Orphan-deadlock check unchanged in spirit but now correct: an fd waiter is NOT
an orphan (while `io_waiters.len > 0` the loop blocks on kqueue rather than reaching the check), and
a genuine no-timer/no-fd suspend still aborts loudly (verified with a probe: exit 134).
- `wake` now also evicts a pending fd-waiter (`cancel_io_waiter_for`, mirror of `cancel_timer_for`) —
same UAF reasoning: a fiber woken by another path must not leave a stale `IoWaiter` pointing at a
reaped `*Fiber`. The kqueue registration is `EV_ONESHOT` so we never `EV_DELETE` (a never-fired
one-shot lingers harmlessly; the drain ignores an unmatched ident; closing the fd auto-removes it).
- DE-RISK probe (run first, no scheduler): confirmed `size_of(Kevent) == 32`, the pipe roundtrip
(`kq_wait` returned 1 with `out.ident == read_fd`, `out.filter == -1` (EVFILT_READ), `out.data == 1`
byte readable) — the struct layout reads back the fd correctly.
- Locked by `examples/concurrency/1816-concurrency-fiber-io-pipe.sx`: a `pipe`; a reader fiber spawned
FIRST blocks on the empty read end, then a writer fiber writes `a b c` → the run loop blocks on
kqueue, wakes the reader, which reads the 3 bytes. Output `log: wrote read 3 [97 98 99]` /
`n_suspended: 0` (the "wrote" before "read" ordering proves the reader actually blocked then woke via
kqueue readiness). `.build` `{ "target": "macos" }` (matches host arch → runs end-to-end; ir-only on
a mismatch, like 1814/1815 — no `.ir` snapshot needed since it runs here). The example declares its
own `read`/`write`/`close` externs with the CANONICAL signatures std already binds
(`(i32,[*]u8,usize)->isize` / `(i32)->i32`) — a divergent re-binding is rejected by the extern dedupe.
- **Adversarial review (worker) of the run-loop change — found 2 CRITICALs:**
- **(1) two fibers on the SAME fd → lost wakeup + permanent hang.** macOS `EV_ADD` for an existing
`(ident, filter)` REPLACES the registration (doesn't stack), so two waiters share one registration:
the fd fires once, one wakes, the other is stranded in `io_waiters` and the next `kq_wait(-1)` blocks
forever. FIXED: `block_on_fd` now enforces one-waiter-per-fd with a loud abort (the model already
assumed it). Verified: dup-fd → `sched: block_on_fd: fd N already has a waiter`, not a hang.
- **(2) an fd-waiter on a never-ready fd hangs instead of the timer path's loud abort.** Re-examined:
this is CORRECT event-loop semantics — blocking on I/O until ready (possibly forever, like a server
idling on a socket) is the point; the scheduler cannot know an fd will never become ready, so it must
keep waiting. NOT a scheduler deadlock. Fixed the MISLEADING comment that implied the orphan check
covers fd-waiters: it does not, by design (it covers only pure `suspend_self` parks). No code change —
the "hang" is a caller-side logic issue (waiting on input that never arrives), not a bug to abort on.
- Review CLEARED: the IoWaiter UAF parity (early-wake evicts the waiter; a lingering one-shot that later
fires hits no match → clean no-op), ident width/sign, `kq_wait` EINTR/error handling, timer-vs-io
precedence (timer wins; no hang). All probed safe.
- Suite GREEN **754/0** (incl. the dup-fd guard, no new example needed — the abort is host-fragile to
pin like 1809's guard-firing). Next: **B1.5** (end-to-end M:1 validation under the deterministic timers
/ fd readiness); a linux epoll twin of `block_on_fd` (mirror via `std/net/epoll`, the OS-neutral facade
is `std.event`) is future work.
### Earlier — B1.4b — deterministic VIRTUAL-TIME timer scheduling (the KEYSTONE) — landed + adversarially
reviewed (caught a CRITICAL UAF, fixed).
`library/modules/std/sched.sx` gained a virtual clock +
sleep timers so fibers schedule in reproducible simulated time (no real clock): `clock_ms` (advances
ONLY as timers fire), a `timers: List(Timer)` (insertion-order, linear min-scan, FIFO tiebreak),
`now_ms()`, `sleep(ms)` (arm `{clock_ms+ms, current}` + `suspend_self`), and a timer-driven `run`
(drain ready → fire earliest timer → advance clock → wake → repeat; orphan-deadlock check preserved
for a genuine no-timer suspend). Locked by `1814` (5 fibers sleep 30/10/20/15/15 → wake order
B@10, D@15, E@15 (FIFO), C@20, A@30 — deadline order, not spawn order; `now_ms()` reads each virtual
deadline; final clock 30). §8.1.3 calibration note in the header: the deterministic wake ORDER
equals what real `sleep`s produce, reproducing blocking semantics' observable ordering without real
time. The deterministic-sim `Io` is realized at the scheduler level (`sleep`/`now_ms`/timer-`run`),
not as an erased `Io`-protocol impl (same erasure reason as FiberIo).
- **Adversarial review (worker) of the run-loop change: found a CRITICAL use-after-free** — a fiber
that armed a `sleep` timer but was woken EARLY by another path (a manual/`Task` `wake`) ran to
completion + was reaped (stack `munmap`'d, `Fiber` freed) while its `Timer` still held a dangling
`*Fiber`; a later fire would `wake` freed memory (silent-corruption: "passes" only because the
freed slot coincidentally read `state != .suspended`). FIXED: `wake` now evicts the woken fiber's
pending timer (`cancel_timer_for`) — every re-ready path funnels through `wake` (the timer-fire in
`run` already removed the fired timer, so it's a harmless re-scan there), so no stale timer can
outlive its fiber. Regression `1815-concurrency-fiber-timer-early-wake.sx` (early wake → `clock: 0`,
the stale 100ms timer evicted, not fired). Review CLEARED: `n_suspended` accounting,
orphan-deadlock false-positives, timer-list integrity (re-arm during fire), clock monotonicity,
termination — all traced/probed safe.
- Suite GREEN (count below). Next: **B1.4c** (event-loop `Io` — real fd readiness, kqueue/epoll).
### Earlier — B1.4a — a truly-SUSPENDING fiber-task async layer (`go`/`wait`/`cancel`)
landed + adversarially reviewed; cleared two more compiler blockers en route. `library/modules/std/sched.sx`
now carries `Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` (a `ufcs` layer over
the M:1 scheduler). `s.go(work)` runs the nullary thunk `work` as a REAL fiber; `t.wait()` SUSPENDS
the caller until it completes (vs io.sx's blocking `context.io.async`, which runs inline). Locked by
`examples/concurrency/1813-concurrency-fiber-async-suspend.sx`: two tasks interleave (A yields
mid-body so B runs first → `1 2 3`), awaited values `42`/`100`, and a canceled task's `wait` raises
`.Canceled` → `or -99` → `sequence: 1 2 3 42 100 -99`.
- **Design: a NULLARY thunk, not `async(worker, ..args)`.** A comptime variadic pack can't cross a
deferred (fiber) boundary — `..args` captured into a closure re-expands from the spawner's
now-gone locals (issue 0156 Part 2). So `go` takes `work: Closure() -> $R`; the user captures
inputs in the lambda at the call site (the `go func(){…}()` idiom). **Self-contained in sched.sx**
(NOT io.sx): io.sx importing sched.sx duplicates the `_fib_tramp` global asm when a program also
imports sched.sx directly (global asm emits per import-path) — so the Io-protocol
`spawn_raw`/`suspend_raw`/`ready` hooks stay reserved for the future M:N model; M:1 uses
`go`/`wait` directly. Heap `*Task` (must outlive `go`'s frame; leak documented). `TaskErr` is
LOCAL (the `!` failable detection doesn't see through io.sx's `IoErr` re-export alias).
- **Two compiler blockers hit + FIXED (user-authorized in-session):**
- **issue 0156 Part 1** — a single-type generic `$R` (parsed as `comptime_pack_ref`) used as a
type-arg (`Box($R)`, `size_of(Box($R))`) inside a pack-fn body hit a missing arm in
`resolveTypeWithBindings` → `.unresolved` → LLVM panic. Fix: mirror `resolveTypeArg`'s
`comptime_pack_ref` arm (look up `type_bindings`, else a loud diagnostic). Regression
`examples/generics/0216-generics-typearg-in-pack-fn-body.sx`. (Part 2 — deferred `..` spread
crashes — reframed OPEN/non-blocking, `issues/0156`.)
- **issue 0157** — a user generic `ufcs` method whose name collides with a stdlib re-export
(`cancel` on `*Task` vs io.sx's `cancel` on `*Future`) resolved via last-wins `fn_ast_map` with
NO receiver filtering → wrong overload → `$R` unbound → LLVM panic. Fix
(`src/ir/lower/call.zig` `selectUfcsGenericByReceiver`): every generic-ufcs dispatch enumerates
ALL module authors (`module_decls`), keeps receiver-binding ones, picks the most
receiver-SPECIFIC (concrete > bare `$T`), dedups re-exports, and flags a genuine 2-specific tie
as a deterministic "ambiguous — qualify" diagnostic (never a silent order-dependent pick).
Regression `examples/generics/0217-generics-ufcs-method-name-collides-stdlib.sx`.
- **Adversarial review (worker) of the 0157 fix + Task layer.** Caught the determinism CRITICAL
(fixed: always-run selection + specificity + ambiguity), `wait`-outside-a-fiber null-deref (fixed:
loud guard in `suspend_self`/`yield_now`), and cancel-doesn't-skip-work (fixed: worker skips
`work()` if already canceled). Lost-wakeup / cancel-after-complete / reap traced safe. Also
simplified `1812` (`**Fiber` shared handle → a `Sh.parked` field; output identical).
- Suite GREEN 751/0 (749 + 1813 + 0217). Next: **B1.4b** (deterministic-sim `Io`).
### Earlier — B1.5a — the M:1 cooperative fiber scheduler CORE — landed + adversarially reviewed
The hand-bootstrapped ping-pong (1807-1810) is now a reusable scheduler API in pure sx:
`library/modules/std/sched.sx` — a generic `Fiber` (`body: Closure() -> void`) + `Scheduler`
with `init`/`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over the proven `swap_context` on
guarded `mmap` stacks. The ONE generic dispatch (`fib_dispatch`, reached from the `_fib_tramp`
trampoline) runs ANY stored closure body on a fresh stack — replacing the fixed `bl _fib_body`.
Reaping `munmap`s the stack + frees the heap `Fiber` on completion; an intrusive FIFO gives
round-robin order.
- **Foundational design de-risked by probe before building:** a fiber can store + call a
`Closure() -> void` on its fresh stack via the generic dispatch; outputs flow OUT through
pointers captured in the closure (capture-by-value does NOT write back — pushed onto the user).
- **Hit + FIXED a blocker compiler bug — issue 0154** (user-authorized in-session fix). `null` /
`---` assigned to a struct field picked up a leaked enclosing `target_type` (the function's
RETURN type, set for the whole body at decl.zig:2691) and built a WHOLE-STRUCT-typed null →
an oversized `zeroinitializer` store through the field's GEP that overran the field's slot and
clobbered the saved x29/x30, so the fn `ret`'d to 0x0. This was EXACTLY the `Scheduler.init()`
by-value-return shape (`sched_ctx: [13]u64` before `current: *Fiber`). Fix: added
`.null_literal, .undef_literal` to the `needs_target` switch in `lowerAssignment`
(`src/ir/lower/stmt.zig`) so the field's type is used. Repro → regression test
`examples/types/0193-types-sret-array-before-pointer.sx`; `issues/0154-*.md` RESOLVED.
- **Adversarial review (worker): asm/bootstrap/lifetime SOUND** (the headline closure-env-lifetime
fear was disproven — envs are heap-promoted, survive the spawn frame). Found **1 CRITICAL** +
robustness gaps, ALL hardened: (CRITICAL) `wake` re-enqueued an already-queued fiber →
FIFO corruption/segfault → now GUARDED on `.suspended` (spurious/double/stale wake = safe
no-op); orphan-suspend leak/deadlock → `n_suspended` accounting + a loud `run()`-drain
diagnostic+abort; `mmap` `MAP_FAILED` (=-1, not null) / `mprotect` / Fiber-OOM → loud bails
(per §8.1.1 the guard is mandatory); the per-fiber closure-env leak (sx exposes no env-free) →
documented as a KNOWN LIMITATION (bounded by spawn count; invisible under the default GPA).
- **Locked two `18xx` examples** (aarch64-macos `.build`-pinned, ir-only on a mismatch):
`1811-concurrency-fiber-scheduler.sx` (3 fibers round-robin via `yield_now` → ordering contract
`sequence: 0 1 2 0 1 2 0 1 2`, all `.done`) + `1812-concurrency-fiber-suspend-wake.sx` (park via
`suspend_self`, resumed by another fiber's `wake`, + the spurious-wake no-op — the CRITICAL-fix
regression → `log: 10 20 21 11` / `suspended-left: 0`).
- **Filed issue 0155 (NON-blocking, NOT fixed)** — found incidentally in the review: indexing a
scalar pointer (`pc[0]`, `pc: *i64`) panics codegen (`.unresolved` reaching LLVM emission). The
scheduler uses array-field indexing + `.*`, never this, so it's filed for its own session.
- Suite GREEN **748/0** (746 base + 1811 + 1812 + 0193 regression). Next: **B1.4a** (FiberIo —
wire `Io.spawn_raw`/`suspend_raw`/`ready` onto the scheduler so `async`/`await` truly suspend).
### Earlier — B1.3b-1 — the x86_64 / Win64 `swap_context` sibling — VALIDATED on real hardware
The
context switch is now proven on a SECOND architecture + ABI. A Win64 `swap_context` saves the
COMPLETE Win64 callee-saved set — 8 GP (rbx, rbp, rdi, rsi, r12-r15) + rsp **and xmm6-xmm15**
(10 XMM, 128-bit via `movups` — Win64 has callee-saved XMM, unlike SysV/aarch64) — plus a Win64
`scribble_verify` (32-byte shadow + 16-align at each `call`, COFF symbols, rsp-carried return
addr). Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only here): the 2-fiber mutual scribble printed **`0 0 P`** when built `--target
x86_64-windows-gnu --self-contained` and **run on a Windows 7 x64 VM (UTM)** — every GP + XMM
callee-saved survived. **Adversarially reviewed before the VM run** (worker emitted the real `.s`
and verified every `call` alignment, the 264-byte frame offsets, the rsp/return-addr round-trip,
swap ordering, and COFF naming against the Win64 ABI — no critical/minor bugs). The build→VM→run
loop was set up this session (cross-build needs `--self-contained`; output via the Win32
`WriteFile` boundary, the 1660 pattern). Suite green. Note: this is the GOOD-swap-only mutual
scribble (self-validating by construction; the in-process negative control was dropped to avoid an
sx fn-ptr-convention rabbit hole — the detection of this exact logic was negative-controlled on
aarch64 in 1808). The SysV/Linux x86_64 sibling (different reg set: no callee-saved XMM, args
rdi/rsi) remains for a Linux x86_64 host.
### Earlier — B1.3b-2 — mmap guard-page stacks (commit `dd532ab`)
Fiber stacks are `mmap`'d with a `PROT_NONE` GUARD PAGE at the low end (§8.1.1: a
fixed stack without a guard silently corrupts neighbors on overflow). `mmap` the `[guard |
usable]` region, `mprotect` the low 16KB page `PROT_NONE`; SP descends into the guard and faults
loudly at the boundary instead of corrupting a neighbor. Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
- **Guard FIRING validated** (manually, not corpus-pinned — a deliberate overflow crash is
host-fragile): a fiber recursing past its 128KB stack faults with `Bus error` at the guard page
(`region+GUARD`); the sx crash handler turns it into exit 134. Documented in the example header.
- **x86_64 sibling:** was deferred here (couldn't run x86_64 on this arm64 host), then DONE as
Win64 once a Windows 7 x64 VM became available — see B1.3b-1 above (`examples/1810`, `0 0 P`).
### Earlier — B1.3a-2 — the context-switch STRESS GATE (design §10.7) — DONE + adversarially reviewed
The explicit every-callee-saved-register scribble that B1.3a-1 owed. `swap_context` now saves the
COMPLETE AAPCS64 callee-saved set — integer x19-x28 + fp/lr + sp AND FP **d8-d15** (per §6.1.2
only the low 64 bits of v8-v15 are callee-saved, so `d8-d15` is exactly sufficient; x18 is Apple's
reserved platform reg, untouched). A naked `scribble_verify(self_ctx, peer, base)` loads a unique
sentinel into all 18 callee-saved regs, yields, and on resume counts the ones that didn't survive
(honoring its own caller ABI via a 176-byte frame that saves+restores the caller's callee-saved;
base reloaded from the frame post-swap; the original lr round-trips through the swap). The gate is
a **2-fiber MUTUAL scribble** (A and B scribble DISTINCT sentinels into the same physical regs, so
each survives only if `swap_context` saved+restored it — a lone fiber yielding to an idle peer
would NOT exercise preservation). Locked by `examples/1808-concurrency-fiber-switch-stress.sx`
(aarch64-pinned): `A mismatches: 0` / `B mismatches: 0`.
- **Validity proven by NEGATIVE controls:** dropping the d8-d15 save/restore → 8/8 mismatches
(exactly the FP regs); dropping x27/x28 → 2/2. The gate genuinely catches a broken switch.
- **Adversarial review (worker, per the plan): no CRITICAL bugs.** Verified the callee-saved set
is complete + correct, all frame offsets/16-alignment, the lr/sp dance, and swap read-ordering
against AAPCS64. Applied its one recommendation: `boot` now zeroes the FP ctx slots [13..20] so a
first switch-to loads 0 (not garbage) into d8-d15. Residual gaps it flagged (all spec-correct
for a call-boundary swap, documented in the example header): NZCV/FPSR not swapped; **FPCR**
(rounding mode — thread-global, bleeds across fibers if changed) and **TPIDR_EL0/TLS** (errno,
allocator thread-caches — shared by same-thread fibers) not swapped; fp=0 bootstrap blocks
unwind/signal walking past a fiber trampoline. These bite at the N×M:1 / signals stages, not the
single-thread switch.
- Suite green **734/0**, master clean. WIP probes: `.sx-tmp/scribble2.sx` (+ `_broken`/`_gp`).
### Earlier — B1.3a-1 — the foundational stackful context switch (commit `b234b7d`)
Pure sx over `abi(.naked)`: naked `swap_context` (GP-only 13-slot save) + by-hand fiber bootstrap
(SP = `alloc_bytes` stack top, LR = global-asm trampoline, x19 = `*Fiber`). Locked by
`examples/1807-concurrency-fiber-context-switch.sx`: 2-fiber ping-pong (`rounds: 6` / `canary
fails: 0`) + 64-frame deep recursion (`frames verified: 64` / `depth fails: 0`). Indirect
register/stack survival; 1808 supersedes its switch with the complete GP+FP save area + the
explicit gate.
### Earlier — B1.2 COMPLETE — the async surface works end-to-end
All three surface blockers (0151, 0152, 0153) FIXED + committed; async examples landed + green.
- **0151 fixed** (`362674f`): generic `$T` infers through generic-struct / pointer / UFCS-pack
params. Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` load/store byte-promoted to `i8` in the codegen
emitters. Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` now pins return-type resolution to the
fn's DEFINING module (mirroring `monomorphizeFunction`), so a re-exported value-failable's
`!E` resolves to the real `.error_set` TypeId — the failable channel survives the re-export
alias. Regression `1058-errors-reexport-value-failable-channel.sx`.
- **Async examples landed:** `examples/1805-concurrency-io-blocking-async.sx`
(`context.io.async((a,b)->i64 => a+b, 40, 2).await() or {…}` → `sum: 42` / `double: 42` /
`clock ok`) + `examples/1806-concurrency-io-cancel.sx` (`f.cancel()` → `await` raises
`.Canceled` → `or` default; `ok: 7` / `canceled: -99`). Both green, snapshots captured.
### Earlier — the three B1.2 surface fixes (committed)
Generic `$T` inference, `Atomic(bool)` byte-promotion, and re-export failable-channel pin —
details below.
- **0151 fix (committed):** four gaps closed on the inference + UFCS-dispatch path —
(1) `extractTypeParam`/`matchTypeParam(Static)` got a `parameterized_type_expr` arm
(recover the arg instance's recorded per-param bindings via `struct_instance_bindings` +
the template's ordered `type_params`, recurse positionally; this also fixes `*Box($T)` —
it recurses into its `Box($T)` pointee); (2) the `pointer_type_expr` arm now falls through
to match the pointee against a non-pointer arg (auto-address-of: a `*Box($T)` param accepts
a by-value `Box($T)`, e.g. a UFCS receiver `b.m()`); (3) `ExprTyper.inferType` got a
`.lambda` arm building the closure type from the lambda's annotations (the UFCS binder types
args from the raw AST before they're lowered, so it can now bind `Closure(..) -> $R` from
the worker's declared return type); (4) a pack UFCS target routes through the SAME
`lowerPackFnCall` the direct call uses, with the receiver spliced in as `args[0]`.
- Regression tests: `examples/0214-generics-ufcs-closure-return-pack.sx` (direct + UFCS
closure-return pack) + `examples/0215-generics-infer-through-pointer.sx` (by-value /
pointer / multi-param / nested / UFCS-auto-ref struct-head inference). Issue 0151 marked
RESOLVED; repro moved into the suite.
### Earlier — B1.2 (Io capability) — LANDED + adversarially reviewed
Commits `a1b14f0` (lock) + `45d869d` (Io capability) + `3eeb965` (issue 0151 lock).
- **LANDED + review-confirmed correct** (commit 45d869d): `Io :: protocol #inline`
(spawn_raw/suspend_raw/ready/poll/now_ms/arm_timer) + `io` field on `Context`
(`{allocator; data; io}`, io LAST); BOTH `__sx_default_context` materializers
(protocol.zig + comptime_vm.zig) build an identical CBlockingIo→Io vtable (review verified
byte-for-byte agreement; `context.io.now_ms()` dispatches at runtime AND comptime); the
`push Context.{…}` omitted-field-**inherits-ambient** fix (review: correct, right fix, no
bad blast radius); `library/modules/std/io.sx` (`Future($R)`, `CBlockingIo`,
`async`/`await`/`cancel`); the `!`-protocol-impl-lint suppression; 37 `.ir` regens
(review: pure layout/type-table, no error text, zero .exit/.stdout/.stderr change).
- **BLOCKED — async surface non-functional:** `await`/`cancel` take `*Future($R)` and are
**uncallable in EVERY form** (not just UFCS) — sx can't infer a generic `$T` from a
pointer-wrapped arg (`*Future($R)`). `async(...)` (create) works via explicit call and
produces a correct `.ready` Future, but you can't `await` it. Root bug = **issue 0151
(WIDENED)**: infer `$T` from `*T`-wrapped params + closure-return-via-pack + UFCS dispatch.
Minimal repro: `unbox :: (b: *Box($T)) -> $T` fails to infer `T`.
- **No async example in the corpus** (1805 was removed because it needs the blocked surface)
→ the green suite does NOT cover async. Restore `1805` (async/await) + add `1806` (cancel)
once 0151 is fixed.
### Earlier — B1.1 (per-fiber `context` root) — DONE. Zero compiler change (confirmed by probe).
The fiber-spawn context convention works end-to-end with ordinary language features:
- `snap := context` captures the spawner's `Context` as a value;
- the snapshot is stored in a struct (the stand-in `Fiber`);
- a trampoline running under a *different* ambient context installs the fiber's stored root
with `push f.root { … }`, and the body reads the snapshot — not the trampoline's ambient
context — because `context` is an implicit slot-0 `*Context` param (call-carried, rides the
callee's own stack) and `push` allocates on the caller frame (no global, no TLS).
- Locked by `examples/1804-concurrency-context-snapshot.sx`: prints `fiber root: 42` (the
installed snapshot wins over ambient 99) + `ambient after: 99` (the `push` scope restores
the ambient context on exit). No fiber runtime yet (that's B1.3) — this proves the plumbing
it will build on. No `.build` pin (pure sx, host-independent).
- **Probe result:** the design doc's "lower as swappable indirection, never raw TLS" guarded
a non-problem — context was already param-carried, never TLS. No path re-reads
`__sx_default_context` mid-stack, so there is **no compiler obligation** here.
- `zig build && zig build test` green: **726 ran, 0 failed**.
### Earlier — B1.0 (`abi(.naked)` codegen) — complete
Replaced the emit bail with real LLVM `naked` emission:
- `emit_llvm` declaration pass: for `func.is_naked`, add the LLVM `naked` + `noinline` +
`nounwind` attributes and **skip** the `frame-pointer=all` attribute (incompatible with a
frameless function). Pass 2 now emits the `.naked` body normally — `naked` makes the
backend emit it verbatim (the inline asm + its own `ret`) with no prologue/epilogue.
- IR shape (verified): `; Function Attrs: naked noinline nounwind` / `define internal i64
@answer() #0 { entry: call void asm sideeffect "…ret…", ""() unreachable }` /
`attributes #0 = { naked noinline nounwind }`. The caller invokes it as an ordinary
`() -> i64` call (`.naked` is `call_conv == .default`).
- `examples/1800-concurrency-naked-asm.sx` — now GREEN, aarch64-pinned (`.build {"target":
"macos"}`): runs end-to-end → **exit 42** on this host, ir-only on a mismatch; `.ir`
snapshot captured.
- `examples/1801-concurrency-naked-generic.sx` (renamed from `-bail`) — the generic `.naked`
now emits a correct naked `answer__i64` (exit 42), proving generic.zig produces a naked
body, not a framed one. aarch64-pinned.
- `examples/1802-concurrency-naked-asm-x86.sx` — x86_64 cross sibling (`.build {"target":
"x86_64-linux"}`, ir-only here): `.ir` locks `naked` + `movl $42, %eax` / `ret`.
- Unit test `emit: abi(.naked) function gets the naked attribute (no frame-pointer)` in
`emit_llvm.test.zig` (asserts `naked` present, `frame-pointer` absent).
- **B1.0c (review-hardening):** a param-bearing `.naked` fn emitted invalid LLVM (loud
verifier error "cannot use argument of naked function") because the param-alloca loop
wasn't gated. Fixed forward (this *enables* the B1.3 context-switch use case rather than
rejecting it): gated the param-alloca loop on `fd.abi != .naked` in decl.zig (both paths) +
generic.zig; a naked fn's args stay in registers (read by asm), declared-but-unused in
LLVM. Locked by `examples/1803-concurrency-naked-asm-param.sx` (`add(a,b)` → x0+x1 → 42).
- `zig build && zig build test` green: **725 ran, 0 failed** + unit tests.
### Earlier — B1.0a (lock + review hardening)
Plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites + generic.zig +
pack.zig); `funcWantsImplicitCtx` skips `.naked` (no synthetic ctx, like `.c`); all
body-lowering paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx
return); `emit_llvm` Pass 2 bailed loudly (since flipped to real emission). Adversarial
review caught the generic/pack `is_naked` gap (a generic `.naked` silently shipped a framed
body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positive
(unparseable — `isLambda` breaks on the `abi` keyword).
## Current state
**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime:
the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task
async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b:
`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`,
`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded
`mmap` stacks (B1.0–B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw
suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). A terminal
`deinit` (B1 follow-up) closes the previously-documented leaks: heap `Task`s (tracked via `task_allocs`),
the `timers`/`io_waiters`/`task_allocs` List backings, and the kqueue fd; the per-`spawn`/`go` closure
env remains unfreeable (language limitation). Locked by `18xx` 1800–1820 (naked-asm, context-snapshot,
blocking async, the switch + §10.7 stress gate + guarded stacks + Win64 sibling, scheduler round-robin,
suspend/wake, async go/wait/cancel, sim-timer ordering, timer early-wake eviction, kqueue pipe I/O, the
**1817 end-to-end capstone**, sleep-negative/double-wait guards, and **1820 scheduler-deinit**). Suite
GREEN **817/0**, committed. **B1.6: now also runs on aarch64-linux** (epoll fd-backend + comptime-branched
`MAP_AP` + naked-fn trampoline) — validated byte-identical to macOS in an Apple `container`.
Future work (none blocking B1): routing the suspending async through
the erased `context.io` (forces sched.sx into every std consumer — deferred to the M:N model, where
the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/
`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task /
closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream
B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started.
### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists
`library/modules/std/sched.sx` drives N fibers
(generic `Closure() -> void` bodies) cooperatively over the proven `swap_context`, on guarded
`mmap` stacks: `spawn` / `yield_now` (round-robin) / `suspend_self` + `wake` (off-queue park/resume)
/ `run` (drives to drain, reaps on `.done`). Adversarially reviewed + hardened (wake guarded, loud
mmap/mprotect/OOM/deadlock bails, env-leak documented). Locked by `1811` (round-robin ordering
contract) + `1812` (suspend/wake park-resume + spurious-wake guard). Suite GREEN **748/0**.
The remaining B1.4 work wires this scheduler under the `Io` capability: **B1.4a (FiberIo)** makes
`context.io` route `spawn_raw`/`suspend_raw`/`ready` onto the `Scheduler` so `async`/`await` truly
SUSPEND (today's `CBlockingIo` runs the worker to completion inline); **B1.4b** the deterministic-sim
`Io` (virtual clock + timer queue, calibrated against blocking — the KEYSTONE test harness);
**B1.4c** the event-loop `Io` (kqueue/epoll). Then **B1.5** is the end-to-end M:1 validation under
the deterministic `Io`.
### Earlier — B1.2 COMPLETE
The full async surface (Io capability on Context + `async`/`await`/`cancel` +
blocking `CBlockingIo`) works end-to-end. Master GREEN (732/0), installed `sx` clean. All four
B1.2 surface bugs resolved or deferred:
- **0151 fixed** (`362674f`): generic `$T` through generic-struct / pointer / UFCS-pack params.
Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` byte-promoted to `i8` in the load/store emitters.
Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` pins return-type resolution to the fn's
defining module, so a re-exported value-failable keeps its `!` channel. Regression `1058`.
- Issue **0150** (`void` struct field → SIGTRAP) DEFERRED — only `Future(void)` / `timeout`,
which are B1.4.
The async examples are landed + green: `1805` (`async`/`await` + `now_ms` → `sum: 42` /
`double: 42` / `clock ok`) + `1806` (`cancel` → `await` raises `.Canceled` → `or` default).
The `18xx` concurrency category now covers naked-asm (1800-1803), context-snapshot (1804), and
the async surface (1805-1806).
### B1.2 Io capability — what is LANDED + verified (commit 45d869d)
- `Io :: protocol #inline { spawn_raw; suspend_raw -> !; ready; poll; now_ms; arm_timer; }`
in `core.sx` next to `Allocator`, with `SpawnOpts{ pin: PinTarget }` + `ParkToken{ handle }`.
Six methods, each justified by a downstream consumer (B1.3-B1.5).
- `Context :: struct { allocator; data; io: Io; }` — `io` appended LAST so `allocator` stays
index 0 (the `call.zig:1229` hardcode) and `data` keeps index 1 (minimal VM-fallback churn).
- Both `__sx_default_context` materializers updated in lockstep + verified: `protocol.zig`
`emitDefaultContextGlobal` (extended `ctx_fields` 2→3, built the `CBlockingIo→Io` inline
7-word vtable `{null-ctx, fn0..fn5}` via `getOrCreateThunks("Io","CBlockingIo")`) and
`comptime_vm.zig` `materializeDefaultContext` fallback (wrote the 6 thunk func-refs at
`io_base = addr + 4*ps`, offset `+ (i+1)*ps`). The global path auto-followed the 3-field
Context type. **`context.io.now_ms()` printed `clock ok` live — the capability threads + the
vtable dispatches correctly.**
- Stateless `CBlockingIo :: struct {}` + `impl Io for CBlockingIo` (mirror of `CAllocator`):
blocking semantics — `spawn_raw`/`ready`/`poll`/`arm_timer` no-op/0, `now_ms` → `time.mono_ms()`.
- **push-inherit-omitted fix** (`stmt.zig` `lowerPush`): a `push Context.{...}` now SEEDS the
new slot from the ambient context (load+store), then overwrites ONLY the literal's named
fields — so omitted fields (now incl. `io`) are INHERITED, never zero-inited to a null
vtable. Eliminates the omitted-field footgun globally (zero per-site churn across the 17
partial-literal sites). This is the correct capability-bag semantics; it compiled clean.
- **`!`-protocol-method warning fix** (`error_analysis.zig` + a new `Lowering.impl_method_names`
set populated in `protocols.zig` `registerImplBlock`): a protocol impl method may be declared
`!` by contract (e.g. `Io.suspend_raw`) yet never raise; the "declared `!` but never errors —
drop the `!`" hint is a false positive for impl methods, now suppressed for them.
Status of the blockers that originally stopped B1.2:
- **issue 0151 — FIXED this session** (generic `$T` through generic-struct / pointer /
UFCS-pack params). `async`/`await`/`cancel` are callable. See "Last completed step".
- **issue 0152 — NEW, the current blocker** (`Atomic(bool)` → sub-byte i1 atomic; LLVM reject).
Blocks the async examples via `Future.canceled: Atomic(bool)`. Filed; codegen-level fix.
- **issue 0150** — `void` struct field SIGTRAP; only `Future(void)`/`timeout` (B1.4). DEFERRED.
Per the IMPASSABLE STOP rule: 0151 fix shipped (suite green 728/0), 0152 filed, STOPPED.
Resume B1.2's async examples once 0152 lands.
### Earlier — B1.0 + B1.1 complete
Stream A (atomics) is feature-complete (✅). Stream B1: **B1.0 + B1.1 complete.** The two
compiler-floor preconditions for the fiber runtime are in place: (1) `abi(.naked)` emits a
real LLVM `naked` function end-to-end (decl, generic, pack paths) — the context-switch
substrate; (2) per-fiber `context` root needs **no compiler change** — the spawn convention
(snapshot `context`, store, `push` it from the trampoline) is pure library sx. No
fibers/Io/scheduler code yet. Grounded floor facts:
- `context` is an implicit slot-0 `*Context` param + `push Context` is a stack `alloca` ⇒
**fiber-local for free** (confirmed by the B1.1 probe — never TLS, never re-read from the
`__sx_default_context` global mid-stack). A spawn passes the snapshot as the fiber-entry
fn's slot-0 ctx via `push f.root { entry(args) }`. Locked by `1804-...-context-snapshot`.
- Inline asm works end-to-end (lower→emit→JIT, aarch64 + x86_64) — the `.naked` body reuses it.
- **`.naked` with PARAMS works** (B1.0c, the B1.3 substrate): the param-alloca loop is gated
on `fd.abi != .naked` in decl.zig (both paths) + generic.zig — a naked fn's args stay in
ABI registers (read by the asm body), declared-but-unused in LLVM (verifier-legal).
Example `1803-concurrency-naked-asm-param.sx` (`add(a,b)` reads x0/x1). **Unsupported (loud,
not silent):** a `.naked` *variadic-pack* fn (pack.zig's param loop is intertwined with
comptime-param/`#insert` handling, and a naked fn can't read a runtime-sized pack from
registers anyway) → loud LLVM-verifier error for that nonsensical construct. Acceptable
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
## Next step
**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature-
complete and committed (1800–1820 green, 759/0), now WITH a `Scheduler.deinit` closing the bounded
leaks. Pick up **Stream B2** (channels / structured cancel / async stdlib) as a fresh carve
(PLAN-CHANNELS.md), OR one of the remaining non-blocking follow-ups: the linux `epoll` twin of
`block_on_fd`, `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the
erased `context.io` for the M:N model. (`Scheduler.deinit` — DONE, see Last completed step.) None of
these block B1. The closure-env leak survives `deinit` (no language affordance to free a closure env);
revisit if/when sx grows closure-env ownership.
**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS
kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via
`std/net/epoll` and the OS-neutral facade is `std.event` — keep the two as separate run modes inside
`run`, branching on the platform, exactly as the timer-vs-fd modes are kept separate now. Documented
non-unification: virtual-time timers and real kqueue timeouts are NOT merged — `run` fires a pending
timer before ever blocking on kqueue (a program uses `sleep` OR fds); a true "fd-or-real-timeout" wants
a kqueue `EVFILT_TIMER`, future work.
> **▶ LINUX EPOLL — in progress (2026-06-26), via `std.event.Loop` (the OS-neutral facade).**
> Chosen over the sched.sx `block_on_fd` twin because the facade is the named home for epoll, is pure
> sx + libc (zero compiler change), is consumed by http.sx, and has a runnable darwin sibling. Landed:
> (A) **`library/modules/std/net/epoll.sx`** — raw bindings, the linux twin of `std/net/kqueue.sx`.
> `epoll_event` is modelled as an **arch-branched struct** (`{events, data_lo, data_hi}` u32 fields →
> 12 B x86_64 packed / 16 B aarch64), so layout is byte-exact with NO packed attribute, NO unaligned
> access, NO scalar-pointer indexing (issue 0155) — the struct-per-arch approach the user flagged as
> better than raw byte poking. Self-contained (libc only — NO build.sx import; the top-level `inline if
> ARCH` resolves via the compiler's flatten pre-pass, keeping the IR small). Locked by
> `examples/event/1633-event-epoll-bindings-linux.sx` (ir-only x86_64-linux, durable 244-line .ir;
> aarch64 16 B layout also probe-verified). (B) **`std.event.Loop` branched on `inline if OS`** into two
> top-level OS-selected structs (sx has no conditional struct fields): the kqueue Loop unchanged
> (darwin, runs — 1632 green), a new epoll Loop (linux) with the per-fd registration table (combined
> EPOLLIN/OUT mask via ADD/MOD/DEL), eventfd wake channel, and EPOLLRDHUP→eof. **RUNTIME-VALIDATED on
> real Linux:** a static `aarch64-linux` build of the 1632-equivalent Loop test (+ the eventfd wake path)
> ran **6/6 green inside an Apple `container` Linux VM** (kernel 6.18 aarch64) — add_read, idle-timeout,
> readable+fd+udata, the MOD-mask add_write path, the eventfd wake channel, and EPOLLRDHUP/HUP eof all
> behave identically to kqueue (lone documented difference: `nbytes` is 0 on epoll). Also lowers clean for
> both linux arches; the ABI is corpus-locked by 1633. NOT corpus-snapshotted (the corpus runner is
> host-based, not container-aware; a Loop example drags the std barrel → ~18k-line brittle IR).
> **The epoll deliverable is COMPLETE.** Re-validation recipe in the event.sx VALIDATION note. Optional
> follow-on: route sched.sx `block_on_fd` through `std.event` (still needs the linux sched.sx port — mmap
> consts, tramp symbol, errno, x86_64 SysV switch).
> **✅ issue 0192 FIXED (2026-06-26) — epoll work UNBLOCKED.** A qualified-import-member const
> (`m.EV_SIZE`) now folds as a compile-time constant in every position the bare/flat form does
> (array dim, arithmetic, Vector lane, generic value-param, inline-for) — so the clean
> `[MAXEV * ep.EV_SIZE]u8` event buffer the bindings want will work. Fix: a `lookupQualifiedConst`
> ctx hook resolving the namespace alias → target module's per-source const, wired into the int/float
> const folders (`src/ir/program_index.zig` + `src/ir/lower/comptime.zig`). Regression:
> `examples/modules/0842-modules-qualified-import-const-comptime.sx`. The hint stands for the rebuild:
> **a struct-per-arch `EpollEvent` (arch-branched u32 fields, 12 B x86_64 / 16 B aarch64) beat raw
> byte access** — idiomatic field reads, no issue-0155 scalar-pointer indexing, no unaligned u64.
> Resume: rebuild `std/net/epoll.sx`, branch `std.event.Loop` on `inline if OS`, lock with a darwin run
> + ir-only linux example.
> **⛔ (HISTORICAL) BLOCKED on issue 0192 (filed 2026-06-26).** Started the epoll work: chose the `std.event.Loop`
> backend (pure sx + libc externs, zero compiler change — per "do this in sx as much as possible") as
> the first deliverable, since event.sx already names epoll as its linux backend and it's runnable
> (darwin via kqueue) + ir-only-verifiable (linux). De-risked four landmines by probe — arch-dependent
> layout const via module-scope `inline if ARCH` (folds + validates in linux IR), slice-based byte access
> (sidesteps issue 0155), no unaligned u64 (store the 32-bit fd in epoll `data`), and comptime-dead linux
> externs don't break the darwin corpus (just an unreferenced `declare`). Then hit a compiler bug while
> sizing the event buffer: a **qualified-import-member const is not a compile-time constant** —
> `[m.CAP]u8` / `A :: m.CAP` fail (a *flat*-imported const works). Root cause located:
> `evalConstIntExpr` (`src/ir/program_index.zig:325`) has no namespace-member-const arm. Per the STOP
> rule the half-built `std/net/epoll.sx` (which used a struct-based layout to route around the bug) was
> **removed**, not landed — the unblock session rebuilds it cleanly with the fix in hand. Repro +
> investigation prompt: `issues/0192-qualified-import-const-not-comptime.{md,sx}`.
Design note carried forward: an event-loop `Io` needs a current-`Scheduler` handle. `sched.*` methods
thread it via `self`/the `Task`; if B1.4c wants the capability-threaded `context.io` form it'll need
an ambient current-scheduler accessor in sched.sx (still deferred — the `sched.*`-method form
suffices). The `Io` protocol's `poll`/`arm_timer` map onto this when/if that wiring is built.
**Side thread (optional, low priority): the SysV/Linux x86_64 sibling.** A THIRD switch variant
for `x86_64-linux`: SysV callee-saved = rbx, rbp, r12-r15 + rsp (6 GP + sp; **no** callee-saved
XMM, unlike Win64) — a 7-slot ctx, args rdi/rsi/rdx, the rsp-carried return addr. Needs a Linux
x86_64 host (or a working cross-run) to RUN + the mutual-scribble gate. Not blocking — the switch
is already validated on two arch/ABI pairs.
**Deferred (do NOT block on these):** issue **0150** (`void` struct field SIGTRAP) — only
`Future(void)`/`timeout` (B1.4). The **`::` callable-parameter feature** (named-fn async workers
`async(read_a, conn)`) — WIP at `.sx-tmp/wip-callable-params/patch.diff` (parser done, inference
incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
`Context` layout settled: `{ allocator; data; io; }` (allocator index 0 fixed by
`call.zig:1229`, io last). Io protocol + materializers + push-inherit are LANDED + reviewed.
## Known issues / capability gaps
- **issue 0157 (OPEN, BLOCKING B1.4a)** — a user-defined generic ufcs method whose NAME collides
with a stdlib re-export (`cancel`, re-exported by `std.sx` from `io.sx` as `ufcs (f: *Future($R))`),
called via UFCS on a different generic struct (`*Task($R)`), leaves `$R` unresolved → `.unresolved`
reaches LLVM emission → panic (`src/backend/llvm/types.zig:196`). Renaming → works; the non-UFCS
call form already diagnoses `cannot infer generic type parameter 'R'`, so the UFCS path skips that
diagnostic. Surfaced by `cancel :: ufcs (t: *Task($R))` in `std/sched.sx`. Minimal repro (no
fibers/closures): `issues/0157-ufcs-generic-method-name-collides-stdlib-unresolved.{md,sx}`.
- **✅ issue 0154 — FIXED** (`null`/`---` to a struct field over-stored a whole-struct null when
the function's return type leaked as `target_type`, corrupting the frame → `ret` to 0x0;
surfaced building `Scheduler.init()`'s by-value return). Fix: `.null_literal`/`.undef_literal`
added to `needs_target` in `lowerAssignment` (`src/ir/lower/stmt.zig`). Regression:
`examples/types/0193`.
- **issue 0155 (OPEN, NON-blocking)** — indexing a scalar pointer (`pc[0]`, `pc: *i64`) panics
codegen (`.unresolved` reaching LLVM emission, `src/backend/llvm/types.zig:196`). Found in the
B1.5a review; the scheduler doesn't use it (array-field index + `.*` only). Filed for its own
session: `issues/0155-scalar-pointer-index-llvm-panic.{md,sx}`.
- **✅ issue 0158 — FIXED** — a plain `union` struct-literal (`b : Overlay = .{ f = 3.14 }`) fell
through the generic struct-literal path (`getStructFields` empty for a union → malformed
`structInit`, overlapping zero-fill clobbered the member → silent `0.0`). Fix: `lowerStructLiteral`
detects a plain-union target → new `lowerUnionLiteral` (`src/ir/lower/stmt.zig`) writes each named
member into a union-sized slot via the assignment-path lvalue resolver, then loads it back.
Single-arm only (one direct member, or same-arm promoted members); overlapping/different-arm/
positional literals are diagnosed. specs.md updated. Regressions: `examples/types/0194` +
`examples/diagnostics/1191`.
- **✅ issue 0157 — FIXED** (B1.4a) — a user generic `ufcs` method whose name collides with a
stdlib re-export resolved via last-wins `fn_ast_map` with no receiver filtering → wrong overload →
`$R` unbound → LLVM panic. Fix: `selectUfcsGenericByReceiver` (`src/ir/lower/call.zig`) — most
receiver-specific binding author across ALL module authors, deterministic, ambiguity-diagnosing.
Regression: `examples/generics/0217`.
- **✅ issue 0156 Part 1 — FIXED** (B1.4a) — single-type generic `$R` as a type-arg in a pack-fn
body (`Box($R)`/`size_of(Box($R))`) → `.unresolved` → panic. Fix: `comptime_pack_ref` arm in
`resolveTypeWithBindings`. Regression: `examples/generics/0216`.
- **Part 2 (OPEN, NON-blocking)** — a deferred `..` spread (a comptime pack captured into a
closure, or a tuple `..t` spread) crashes instead of working/diagnosing. The fiber async layer
avoids it by design (nullary thunks), so it's filed for its own session: `issues/0156`.
- **Heap leaks in the fiber runtime (documented limitations, NOT bugs):** `spawn`'s closure env +
`go`'s heap `Task` are never freed (sx exposes no closure-env free; Task ownership is deferred).
Bounded by spawn/go count, invisible under the default GPA. Revisit for a long-running
arena-backed scheduler.
- **✅ issue 0153 — FIXED** (re-exported generic value-failable `($R, !E)` kept its `!` channel:
`inferGenericReturnType` now pins return-type resolution to the fn's defining module).
Regression: `examples/1058`. Was the LAST B1.2 surface blocker.
- **✅ issue 0152 — FIXED** (`Atomic(bool)` sub-byte i1 atomic → byte-promoted to i8 in the
load/store emitters). Regression: `examples/1705`. Unblocked `Future.canceled`.
- **✅ issue 0151 — FIXED** (generic `$T` through generic-struct / pointer / UFCS-pack params).
Regression: `examples/0214` + `0215`. Was the original B1.2 surface blocker.
- **issue 0150** (deferred) — a `void` struct field crashes the compiler (unsized-type SIGTRAP
in LLVM `getTypeSizeInBits`). Blocks `Future(void)` → `timeout` (B1.4). Repro: `issues/0150-...`.
- (Note: **issue 0149**, filed by another session against an earlier dirty binary, was a
manifestation of the pre-fix 0151 — now moot.)
- **Orthogonal (not a B1 blocker):** default VALUES for comptime params don't bind on
generic-struct methods (free-fn defaults DO work) — inherited from Stream A. Only matters
if a B2 lib type wants a defaulted comptime param; atomics/fibers require explicit, so
unaffected.
- **Issue 0144 (open, independent):** calling an unrecognized bodiless `#builtin` silently
returns 0 / exit 0 — a silent-fallback footgun in the generic builtin-call path. Filed;
leave for its own fix session unless prioritized. Not a B1 blocker.
- **Deferred design gap (documented):** the B1.4 event-loop `Io` does not yet cooperate with
a platform UI run loop (CFRunLoop/NSRunLoop/ALooper); pinning gives thread-affinity, not
run-loop integration — a §6 app-target concern, out of B1 scope.
## Decisions (Stream B1 specifics; surface locked in design §4 / §4.6)
- **The async runtime is sx LIBRARY code.** The compiler provides only: the general
primitives (inline asm ✅, `abi(.naked)` naked [B1.0], atomics ✅) + fiber-safe codegen
(`context` already fiber-local — B1.1). Schedulers, fibers, channels, futures, `Io`
vtables, `mmap` stacks are all sx.
- **`abi(.naked)` is the real spelling of the design's `callconv(.naked)`** — postfix slot,
`name :: (sig) -> Ret abi(.naked) { asm { … }; }`. B1.0 = carry it into IR + emit LLVM
`naked` + skip prologue/ctx (mirror the existing `.c` skip), NOT extend the enum (it's
already there, just inert).
- **`.naked` ≠ `.c`:** a `.c` epilogue would restore SP from the wrong stack across a context
switch (SP-in ≠ SP-out by design). `.naked` = no prologue/epilogue/frame; the asm emits its
own `ret`. This is why the switch must be `.naked`.
- **Naming:** sx-facing name is **`naked`** (keyword `abi(.naked)`, field `is_naked`, the
diagnostic), matching LLVM's `naked` attribute and the industry term (Zig/Rust/GCC/Clang).
The ABI variant was renamed `.pure → .naked` (user direction): "pure" universally means
*side-effect-free*, the opposite of a register-clobbering context switch.
- **B1.0 snapshot scope:** a `.naked` body is raw per-arch asm; LLVM's `naked` attr text is
arch-invariant. **B1.0a** = one host example locked to the emit bail (host-independent —
fires before instruction selection; no `.build` pin). **B1.0b** = pin aarch64 + add an
x86_64 cross sibling (`.build` target-gated, ir-only on mismatch), like the asm corpus
split. The `.ir` proves the `naked` attr + asm emitted, NOT register-save correctness
(that's B1.3's stress harness).
- **B1.1 — per-fiber context is library-only (CONFIRMED by probe):** push frames are
stack-`alloca`'d and the implicit ctx rides slot 0, so the spawn convention — snapshot
`context`, store it, `push f.root { entry(args) }` from the trampoline — installs the
fiber's root with no compiler change. Verified: the body reads the snapshot over a different
ambient context, and `push` restores ambient on exit (`1804-...-context-snapshot`). The
design doc's "never raw TLS" guarded a non-problem (context was never TLS).
- **Test keystones (design §10):** the **B1.3 switch-stress harness** gates the
context-switch (the one piece the deterministic `Io` can't test — §8.1.1, §10.7); the
**B1.4 deterministic-sim `Io`** (calibrated against blocking `Io` — §8.1.3) gates all
scheduling tests. Both must exist + be calibrated before the async tests they gate are
trusted. `18xx` asserts program-emitted ordering contracts, not raw interleaving.
## Log
- **B1.6 — aarch64-linux port of sched.sx.** Comptime-branched the per-OS bits: `MAP_AP` (linux
`0x22` / macOS `0x1002`), the fd-readiness backend (epoll on linux, kqueue on darwin — epoll import
scoped to the linux branch; `block_on_fd` / run-loop Mode-2 / `cancel_io_waiter_for` each branch,
epoll `EPOLL_CTL_DEL`s on fire + early-wake), and the first-entry trampoline (per-OS global-asm
symbol → naked sx fn `fib_tramp` + register-indirect `br x20` to `&fib_dispatch` preset in
`regs[1]`). **Fixed issue 0193 Bug A:** the tramp redesign bus-errored on 1817 (both OSes) until
`fib_dispatch` was annotated `abi(.c)` (C-ABI pin; the original pinned it via `export`, which the
redesign dropped) — without the pin the fn uses sx's internal ABI (x0 = implicit
`context`, `self` → x1) while the trampoline supplies `self` in x0, so the closure loads
`regs[1] == &fib_dispatch` as its first capture and recurses forever → stack-overflow bus error.
Root cause found via lldb (AOT macOS build) + an adversarial source review. **Bug B** (wrapped
top-level `asm` dropped) carved to **issue 0194** (OPEN; no live trigger — the naked-fn tramp
sidesteps it). Validated byte-identical on aarch64-macOS host AND aarch64-linux Apple `container`
for 1811/1814/1816/1817; full suite GREEN **817/0**.
- **B1 follow-up — `Scheduler.deinit`.** Closes the bounded leaks B1 documented. Added a `task_allocs:
List(*void)` field (appended in `go` so the scheduler can reach its generic `Task($R)`s) + a canonical
`close` extern, then a terminal idempotent `deinit`: reap leftover ready fibers (`munmap` + free) →
free tracked Tasks → `List.deinit` the 3 backings → `close` the lazy kqueue fd (reset `-1`). Closure
envs stay unfreeable (documented). Probe-observed the accounting under a tracking GPA (deinit drives
live allocs 7→3 in a spawn+sleep+2×go run; residual = envs). Locked by
`1820-concurrency-fiber-scheduler-deinit.sx` (one run hits timers + kqueue fd + Tasks; `freed by
deinit: 5`, `live after deinit: 5` (env residual), `kq open after run: true`→`kq after deinit: -1`,
`read: 3 [97 98 99]`), `.build {"target":"macos"}`. Adversarial review: no real UAF/over-free in the
supported deinit-after-`run` path; reconciled a doc contradiction (terminal-contract wording); 0154
over-store concern probed + cleared (`kq == -1` right after `init`). Suite GREEN **759/0**.
- **B1.4c — real fd-readiness blocking via kqueue (macOS).** De-risked first with a no-scheduler probe
(confirmed `size_of(Kevent)==32` and the pipe→kevent roundtrip: `kq_wait` returned 1, `out.ident ==
read_fd`, `out.filter == -1`, `out.data == 1` — the struct layout reads the fd back correctly). Then
added to `library/modules/std/sched.sx` (importing the existing verified `std/net/kqueue.sx` as `kqb`
rather than re-deriving the FFI): a lazy `kq: i32` (-1 until first use), `io_waiters: List(IoWaiter)`,
`block_on_fd(fd, want_read)` (arm one-shot `EVFILT_READ`, record waiter, `suspend_self`), a run-loop
Mode 2 (block on `kq_wait(kq, evbuf, MAXEV=16, -1)` when only fd waiters remain, wake the fiber whose
fd fired), and `wake` now also evicts a stale fd-waiter (`cancel_io_waiter_for`, the same UAF guard as
`cancel_timer_for`). Timers keep precedence over fds (documented non-unification). Orphan-deadlock
check still fires for a genuine no-timer/no-fd suspend (probed: exit 134). Locked by
`1816-concurrency-fiber-io-pipe.sx` (reader blocks on empty pipe → writer writes `a b c` → kqueue
wakes reader → reads 3 bytes; `log: wrote read 3 [97 98 99]`, `n_suspended: 0`), `.build`
`{ "target": "macos" }`, runs end-to-end on host. The example's `read`/`write`/`close` externs use the
canonical signatures std already binds (extern-dedupe rejects a divergent re-binding). Suite GREEN
**754/0**. Next: B1.5 (end-to-end M:1 validation); linux epoll twin deferred.
- **carve** — wrote PLAN-FIBERS.md + CHECKPOINT-FIBERS.md. Grounded the B1 compiler floor:
`ABI.naked` inert (type_resolver.zig:237), IR `Function` has no naked flag (inst.zig:605),
attribute API pattern (emit_llvm.zig:1339 nounwind), `.c` ctx-skip precedent
(decl.zig:515), `push Context` stack-alloca + slot-0 implicit ctx (stmt.zig:1263,
lower.zig:259), `__sx_default_context` root (decl.zig:2667/2815), inline-asm corpus
(1645/1651). Corrected the design's `callconv(.naked)` → real `abi(.naked)` spelling and
the B1.0 snapshot story. B1.1 grounded as likely library-only. Baseline green (721/0).
- **B1.0a** — plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites);
`funcWantsImplicitCtx` skips `.naked` (no implicit ctx, like `.c`); both body-lowering
paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx return);
`emit_llvm` Pass 2 bails loudly on `func.is_naked`. `examples/1800-concurrency-naked-asm.sx`
locked to the bail (exit 1 + diagnostic). Suite green (722/0). (ABI variant later renamed
`.pure → .naked` — see the Naming decision above — so all `is_*`/`abi(.*)`/example names
here read `naked`.)
- **B1.0a review-hardening** — adversarial review found generic/pack Function-creation paths
left `is_naked` false (silent framed body for a generic `.naked` instance — returned 42 but
corrupted the stack). Fixed generic.zig + pack.zig (set `is_naked` + asm-only `unreachable`
cap); locked by `examples/1801-concurrency-naked-generic-bail.sx`. The review's `.naked`-
lambda CRITICAL was a false positive (unparseable — `isLambda` breaks on `abi`). Suite
green (723/0).
- **B1.0b** — real `naked` emission: emit_llvm declaration pass adds LLVM `naked`/`noinline`/
`nounwind` + skips `frame-pointer` for `func.is_naked`; Pass 2 emits the body verbatim (no
prologue). `1800` green aarch64-pinned (exit 42 + `.ir`); renamed `1801` → `-generic`
(generic `.naked` emits a naked body, exit 42); added x86_64 sibling `1802` (ir-only, `.ir`
locks `naked` + `movl $42, %eax`). Unit test asserts `naked` present + `frame-pointer`
absent. Suite green (724/0).
- **B1.0c** — review-hardening: param-bearing `.naked` emitted invalid LLVM (loud verifier
error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both paths + generic.zig)
— naked args stay in registers, read by the asm body (the B1.3 context-switch shape).
Locked by `examples/1803-concurrency-naked-asm-param.sx`. Pack `.naked` left unsupported
(loud, nonsensical). **B1.0 complete.** Suite green (725/0).
- **rename** — ABI variant `.pure → .naked` (keyword, `Function.is_naked`, diagnostics,
examples 1800-1803 `*-pure-* → *-naked-*`, docs). "pure" universally means side-effect-free
— wrong for a register-clobbering switch; "naked" matches LLVM/Zig/Rust/GCC/Clang. Pure
cosmetics, no semantic change. Suite green (725/0).
- **B1.1** — per-fiber `context` root: **zero compiler change** (probe-confirmed). The spawn
convention (snapshot `context` → store in a struct → `push f.root { entry() }` from the
trampoline) installs the fiber's root via the implicit slot-0 `*Context` param; the body
reads the snapshot, not the trampoline's ambient ctx, and the `push` scope restores ambient
on exit. Locked by `examples/1804-concurrency-context-snapshot.sx` (prints `fiber root: 42`
/ `ambient after: 99`). Suite green (726/0). **Next: B1.2 (Io interface + context.io).**
- **B1.2 (BLOCKED)** — built the full `Io` capability (protocol on `Context`, stateless
`CBlockingIo` blocking default, both `__sx_default_context` materializers, push-inherit-omitted
fix, `!`-impl-method warning fix) and VERIFIED the core works live (`context.io.now_ms()` →
`clock ok`). Two independent compiler bugs blocked the `async`/`await`/`timeout` layer:
**0150** (`void` struct field → unsized SIGTRAP, blocks `Future(void)`) and **0151** (type-var
from a fn-ptr param's return type not bound in the body, blocks `async`'s `Future(R)`). Both
filed with standalone repros + investigation prompts. Per the STOP rule: reverted ALL B1.2
working changes (master green again, 726/0; the dirty binary had broken the photo project —
see the now-moot 0149), saved WIP to `.sx-tmp/b12-wip/`, STOPPED. Resume after 0150 + 0151.
- **0151 FIXED** — generic inference now binds `$T` through a generic-struct param head, a
pointer (`*Box($T)`, incl. UFCS auto-ref), and a closure-return-via-pack on the UFCS path.
Four gaps closed: `parameterized_type_expr` arm in `extractTypeParam`/`matchTypeParam(Static)`
(recovers the arg instance's recorded per-param bindings, recurses positionally); pointer arm
falls through to match a value arg (auto-address-of); `ExprTyper.inferType` `.lambda` arm
(closure type from annotations — UFCS types args from raw AST pre-lowering); pack UFCS target
routes through `lowerPackFnCall` with the receiver spliced in as `args[0]`. Issue 0151 marked
RESOLVED; repro → `examples/0214-generics-ufcs-closure-return-pack.sx`; widened cases →
`examples/0215-generics-infer-through-pointer.sx`. Suite green 728/0. The now-callable async
surface immediately exposed a SEPARATE codegen bug — **issue 0152** (`Atomic(bool)` → sub-byte
i1 atomic, LLVM reject; `Future.canceled` hits it). Filed with standalone repro + fix prompt.
Per the STOP rule: shipped the 0151 fix, filed 0152, STOPPED. Resume the async examples
(1805/1806) after 0152.
- **0152 FIXED** — the atomic load/store emitters (`src/backend/llvm/ops.zig`) byte-promote a
sub-byte (`bool`→`i1`) access to its `i8` storage type and `trunc`/`zext` the value at the
boundary (new `atomicByteType` helper). rmw/cmpxchg left as-is (a `bool` rmw/CAS is rejected
at the sx level — integer-only — so a sub-byte element never reaches them; comments record
this). Regression `examples/1705-atomics-bool-byte-promoted.sx` (load/store round-trip). Issue
0152 marked RESOLVED. Suite green 729/0. With `Atomic(bool)` working, the async surface
exposed the TRUE remaining blocker — **issue 0153**: a re-exported generic value-failable
`($R, !E)` loses its `!` channel at the call site (the earlier "secondary `or` PHI" symptom
was this, NOT an `Atomic` cascade — confirmed it persists after 0152). Narrowed to the
generic+re-export co-requirement (non-generic re-export OK; direct generic import OK; only the
combination drops `!`). Root cause: the monomorphized return-type's error-set, reached via the
re-export alias, resolves to a non-`.error_set` TypeId, so `errorChannelOf`
(`lower/error.zig:148`) misses the channel. Filed `issues/0153-...` with a minimal co-located
2-file repro + a single-file stdlib-`await` repro + investigation prompt. Per the STOP rule:
shipped the 0152 fix, filed 0153, STOPPED. Resume the async examples after 0153.
- **0153 FIXED → B1.2 COMPLETE** — `inferGenericReturnType` (`src/ir/generics.zig`) resolved the
return-type AST in the CALL-SITE module, so a re-exported error set (`LE :: lib.LE`) resolved
to a non-`.error_set` alias and the planned call-result was a plain tuple (channel lost). Fix:
pin the source to `fd.body.source_file` around the return-type resolution, exactly as
`monomorphizeFunction` does — the `!E` now resolves to the real `.error_set`. One-function
change; full suite green (732/0), no regression. Issue 0153 RESOLVED; repro →
`examples/1058-errors-reexport-value-failable-channel.sx` (+ companion `lib.sx`). With the
channel preserved, landed the async examples: **`1805`** (`async`/`await` + `now_ms` → `sum:
42` / `double: 42` / `clock ok`) + **`1806`** (`cancel` → `await` raises `.Canceled` → `or`
default; `ok: 7` / `canceled: -99`). **B1.2 (Io capability + M:1 async surface) is COMPLETE.**
Next: B1.3 (fiber runtime) on the `.naked` context-switch substrate.
- **B1.3a-1 — context switch works.** Implemented the stackful switch in pure sx over
`abi(.naked)`: `swap_context(from, to)` (save callee-saved x19-x28 + fp/lr + sp into `*from`,
load from `*to`, `ret` onto `to`'s stack) + by-hand fiber bootstrap (SP = top of an
`alloc_bytes` stack, LR = a `.global _fib_tramp` global-asm trampoline that does `mov x0, x19;
bl _fib_body`, x19 = `*Fiber`). Proven via a probe (main↔fiber), then locked by
`examples/1807-concurrency-fiber-context-switch.sx` (aarch64-pinned): a 2-fiber ping-pong
(`rounds: 6`, `canary fails: 0` — a per-fiber stack canary survives every switch) + a 64-frame
deep recursive chain suspended at the bottom and resumed (`frames verified: 64` / `depth fails:
0`). The `bl _fib_body` reaches the sx body via `export "fib_body"` (the 1655 asm→sx pattern);
runs under JIT, ir-only on a non-arm host (`.ir` captured — `swap_context` shows `naked noinline
nounwind`). Suite green 733/0. **Honest scope:** indirect register/stack survival only; the
EXPLICIT every-callee-saved + FP scribble (§10.7) is B1.3a-2, still owed. Next: B1.3a-2.
- **B1.3a-2 — the §10.7 stress gate, adversarially reviewed.** Extended `swap_context` to the
COMPLETE AAPCS64 callee-saved set (added FP d8-d15 → 21-slot ctx) and wrote a naked
`scribble_verify` that loads a unique sentinel into all 18 callee-saved regs, yields, and counts
non-survivors on resume (176-byte frame saves/restores the caller's callee-saved + base; lr
round-trips the swap). The gate is a 2-fiber MUTUAL scribble (each clobbers the other's regs, so
survival ⇒ the switch saved+restored them). Locked by
`examples/1808-concurrency-fiber-switch-stress.sx` (`A/B mismatches: 0`). Validity proven by
negative controls (drop d8-d15 → 8/8; drop x27/x28 → 2/2). **Spawned an adversarial-review
worker (per the plan + user request): NO critical bugs** — callee-saved set complete (x18 rightly
excluded; d8-d15 suffices per §6.1.2), offsets/alignment/lr-sp dance all verified. Applied its
one rec: `boot` zeroes FP ctx slots so first-entry loads 0, not garbage. Honest residual gaps
(spec-correct for a call-boundary swap; in the example header): FPCR/FPSR/NZCV + TPIDR/TLS not
swapped, fp=0 blocks unwind — relevant at N×M:1 / signals, not here. Suite green 734/0.
Next: B1.3b (x86_64 sibling + mmap guard-page stacks).
- **B1.3b — mmap guard-page stacks (x86_64 sibling deferred).** Fiber stacks now `mmap` a
`[guard | usable]` region and `mprotect` the low 16KB page `PROT_NONE`, so a stack overflow
faults at the guard boundary instead of silently corrupting a neighbor (§8.1.1). Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
Guard FIRING validated manually (overflow → `Bus error` at `region+GUARD`, exit 134 via the sx
crash handler) — not corpus-pinned because a deliberate-overflow crash is host-fragile (and a
mere "child faulted" fork test wouldn't prove the BOUNDARY catch). The x86_64 `swap_context`
sibling was DEFERRED: `--target x86_64-macos` mislinks on this arm64 host and `x86_64-linux`
can't run here, so it could only ship un-run/un-negative-controlled — which §10.7 forbids for the
highest-risk asm. SysV target notes (rbx/rbp/r12-r15/rsp, no callee-saved XMM, rsp-carried return
addr) recorded in Next step. Suite green **735/0**. Next: x86_64 sibling (needs an x86_64 host)
OR B1.4 (`Io` impls / scheduler) on the proven aarch64 substrate.
- **B1.3b-1 — x86_64 / Win64 switch sibling VALIDATED on real hardware.** The user provided a
Windows 7 x64 VM (UTM), so the x86_64 switch became RUNNABLE (as Win64). Validated the
cross-build→VM→run loop (`--target x86_64-windows-gnu --self-contained` → PE32+; output via the
Win32 `WriteFile` boundary, the 1660 pattern). Wrote a Win64 `swap_context` (8 GP rbx/rbp/rdi/
rsi/r12-r15 + rsp + **xmm6-xmm15** via `movups` — Win64 has callee-saved XMM) + a Win64
`scribble_verify` (264-byte frame, 32-byte shadow + 16-align at each `call`, COFF symbols,
rsp-carried return addr) driving the 2-fiber mutual scribble. **Adversarially reviewed (worker
emitted the real `.s`, verified every alignment/offset/round-trip against the Win64 ABI — no
critical/minor bugs), THEN run on the VM → `0 0 P`** (all 8 GP + 10 XMM callee-saved survived).
Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only on this host; the VM run is the runtime-correctness provenance). Good-swap-only (the
in-process negative control was dropped to avoid an sx fn-ptr-convention rabbit hole; the
detection of this exact logic was negative-controlled on aarch64 in 1808). Suite green **736/0**.
The B1.3 context switch is now proven on TWO arch/ABI pairs. Next: **B1.4** (Io impls / M:1
scheduler) on the proven substrate. (Side thread: the SysV/Linux x86_64 sibling, when a Linux
x86_64 host is available.)
- **B1.5a — M:1 scheduler CORE + a fixed blocker bug.** Built `library/modules/std/sched.sx`: a
generic `Fiber`/`Scheduler` over `swap_context` on guarded `mmap` stacks. `spawn` heap-allocs a
fiber, bootstraps its ctx, enqueues it; the ONE generic dispatch (`fib_dispatch` via `_fib_tramp`)
runs ANY stored `Closure() -> void` on a fresh stack (replacing the fixed `bl _fib_body`);
`yield_now` round-robins, `suspend_self`/`wake` park/resume off-queue, `run` drives to drain +
reaps `.done` fibers (`munmap` + free). **De-risked first by probe** (closure-on-fiber + output
via captured pointer). **Hit blocker bug 0154** (user-authorized fix): `null`/`---` to a struct
field over-stored a whole-struct null when the fn return type leaked as `target_type`, corrupting
the frame (`ret` 0x0) — exactly the `Scheduler.init()` by-value-return shape. Fixed in `stmt.zig`
(`needs_target` += `null`/`undef` literals); regression `examples/types/0193`; `0154` RESOLVED.
**Adversarial review:** asm/bootstrap/lifetime sound (env-lifetime fear disproven — heap-promoted);
1 CRITICAL (`wake` re-enqueue → FIFO segfault) + robustness gaps ALL hardened (wake guarded on
`.suspended`, `n_suspended` deadlock diagnostic+abort, loud mmap/mprotect/OOM bails, env-leak
documented). Locked `1811` (round-robin `0 1 2 ×3`) + `1812` (suspend/wake + spurious-wake guard,
`log: 10 20 21 11`). Filed NON-blocking `0155` (scalar-pointer index panics codegen — review
incidental, unused by sched). Suite GREEN **748/0**. Next: **B1.4a** (FiberIo).
- **B1.4a (truly-suspending fiber-task async, nullary-thunk design) — BLOCKED on issue 0157.**
Implemented the async layer SELF-CONTAINED in `library/modules/std/sched.sx` (kept its lone
`#import "modules/std.sx"` to avoid the duplicate-`_fib_tramp` trap): `TaskState`, a LOCAL
`TaskErr :: error { Canceled }` (the re-exported `IoErr` alias is NOT seen through by the
`raise`/failable-type check — verified), `Task($R)`, and `go`/`wait`/`cancel` ufcs. Design is
the validated nullary-thunk (`.sx-tmp/pnullary.sx` → `log: 1 2 3 42 100`): `work` is a
`Closure() -> $R`, user captures inputs at the call site, NO `..args` crosses the fiber boundary
(deliberately sidesteps 0156). `go`+`wait` run correctly; both wake-orderings traced. Wrote the
example `examples/concurrency/1813-concurrency-fiber-async-suspend.sx` (+ `{ "target": "macos" }`
`.build`) but its `cancel` ufcs surfaced a NEW compiler bug — issue **0157**: a user generic
ufcs whose name collides with a stdlib re-export (`cancel` from io.sx) is mis-resolved on UFCS
call over a different generic struct, leaving `$R` unresolved → LLVM panic. Bisected to a minimal
no-fiber repro (name is the sole trigger; non-UFCS form diagnoses correctly). Example NOT seeded
into the corpus (no `.exit` marker) — do NOT regen its goldens until 0157 lands. Per the STOP
rule: filed `issues/0157-*.{md,sx}`, marked state BLOCKED, paused.
- **B1.4a COMPLETE (this session) — suspending fiber-task async + two compiler fixes.** Built the
`Task($R)` + `go`/`wait`/`cancel` layer in `sched.sx` (nullary-thunk design; self-contained to
avoid the `_fib_tramp` duplicate-symbol trap). Locked `1813` (`sequence: 1 2 3 42 100 -99`).
FIXED the two blockers the worker had filed: **0156 Part 1** (`comptime_pack_ref` arm in
`resolveTypeWithBindings`; regression `0216`) and **0157** (receiver-driven UFCS overload
selection `selectUfcsGenericByReceiver`; regression `0217`). Adversarial review of the 0157 fix +
Task layer found a determinism CRITICAL (always-run selection + specificity + ambiguity
diagnostic), a `wait`-outside-fiber null-deref (loud guard), and cancel-not-skipping-work (skip
if pre-canceled) — all fixed. Simplified `1812` (`**Fiber` → `Sh.parked`). 0156 Part 2 reframed
OPEN/non-blocking. Suite GREEN **751/0**. Next: B1.4b (deterministic-sim `Io`, the KEYSTONE).
- **B1.4b COMPLETE (this session) — deterministic virtual-time timers + a CRITICAL UAF fix.** Added
`clock_ms`/`timers`/`now_ms`/`sleep` + a timer-driven `run` to `sched.sx` (worker-built): fibers
sleep in reproducible simulated time, waking in deadline order (FIFO tiebreak). Locked `1814`
(5 fibers, wake order B@10/D@15/E@15/C@20/A@30). Adversarial review of the run-loop change found a
CRITICAL use-after-free — a fiber woken EARLY (manual/Task `wake`) before its `sleep` timer fired
was reaped while its `Timer` kept a dangling `*Fiber`; a later fire dereferenced freed memory
(silent "pass" only by luck). Fixed: `wake` evicts the fiber's pending timer (`cancel_timer_for`);
regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended
accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination.
Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll).
- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a
lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx`
(worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd
lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a
never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment
corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0.
- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack
(`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order
(task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async
runtime is feature-complete end-to-end (1800–1817), all adversarially reviewed. Suite GREEN
**755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in
B1.2). Next carve: Stream B2 (channels / cancel / async stdlib).
- **Post-review hardening (this session) — 6 findings from an adversarial review of the B1 commits.**
Fixed: **P1-a** the UFCS generic PLANNER (`calls.zig`) used the last-wins `fn_ast_map` winner while
lowering reselected by receiver → plan/lowering could disagree and MISBOX the result; now both share
`selectUfcsGenericByReceiver`. **P1-b** the selection scanned `module_decls` globally, flagging a
transitively-hidden same-named overload as a FALSE ambiguity; now two-tier — directly-visible authors
first (ambiguity only among those), global fallback for receiver-reachable namespaced methods (e.g.
`Task.cancel`) that defers to `fd0` on a hidden tie. **P2-b** boolean specificity tied `*$T` with
`*Box($T)`; now peels pointer layers so the structurally-narrower receiver wins. **P1-c** a second
concurrent `Task.wait` overwrote the single waiter slot → silent deadlock; now one-awaiter-per-task
loud abort. **P2-c** `sleep(negative)` rewound the virtual clock; now rejected loudly. (**P2-a**
non-generic-winner-hides-generic did not reproduce — the non-generic arm already falls through.)
Regressions: `examples/generics/0218` (receiver specificity + plan/lowering agreement),
`examples/concurrency/1818` (negative-sleep abort), `1819` (double-wait abort). Suite GREEN **758/0**.

276
current/CHECKPOINT-METATYPE.md
View File

@@ -1,276 +0,0 @@
# CHECKPOINT-METATYPE — comptime type metaprogramming (`declare` / `define`)
Companion to [PLAN-METATYPE.md](PLAN-METATYPE.md). Update after every step (one
step at a time, per the cadence rule).
## Last completed step
**`type_info` / `define` widened to TUPLE types — reflect/construct triad
complete.** `TypeInfo` gained a `` `tuple(TupleInfo) `` variant (`TupleInfo{
elements: []Type }`, positional/unnamed). `reflectTypeInfo` builds `.tuple`
(tag 2) as bare `type_tag` elements; `defineTuple` decodes `[]Type` and completes
the declare slot as a structural `.tuple` via `replaceKeyedInfo` (tuples are
structural, so the declared name is vestigial, but the slot is completed in place
so `define` returns the handle like enum/struct). `call.zig`'s `type_info` guard
admits `.tuple`. `examples/0623` (programmatic `Pair` + source-tuple round-trip).
Suite green (684). All three TypeInfo shapes now reflect + construct + round-trip
(`0619` enum, `0622` struct, `0623` tuple).
## Earlier — struct widening
**`type_info` / `define` widened to STRUCT types.** `TypeInfo` gained a
`` `struct(StructInfo) `` variant (`StructField{ name, type }`); the metatype
system now reflects AND constructs structs, not only enums.
- `meta.sx`: `StructField` / `StructInfo` / `` `struct `` TypeInfo variant.
- `interp.zig`: `reflectTypeInfo` builds `.struct` (tag 1) for a source
`@"struct"`; `define` dispatches on the TypeInfo tag (`defineType` →
`defineEnum` (0) / `defineStruct` (1)). `defineStruct` mirrors `defineEnum`
(duplicate-field-name check included) but completes the declare slot AS a
struct via `replaceKeyedInfo` — a KIND change re-keys the intern map, whereas
`updatePreservingKey` (the enum path) asserts the key is unchanged.
- `lower/call.zig`: the lower-time `type_info` guard now admits `@"struct"`.
- `examples/0622`: programmatic `Vec2` via `.struct(.{ fields = … })` + a
source-struct round-trip `define(declare("RowCopy"), type_info(Row))`. Enum
path (`0619`) unchanged. Suite green (683). Tuple is the last shape (Next step).
## Earlier — make_enum
**`make_enum(name, variants: []EnumVariant) -> Type`** — the general enum
constructor over `declare`/`define`, minting a nominal enum from a variant list
passed as a VALUE. Pure sx in `meta.sx`. `examples/0620` assembles the list in a
local then mints, exercising `define`'s value-arg SLICE decode.
## Prior step
**`type_info($T)` reflection — enum round-trip.** Reflect a type INTO a `TypeInfo`
value (the inverse of `define`'s decode), so `define(declare(n), type_info(T))`
mints a byte-identical copy with NO literal variant list.
- `inst.zig`: new `BuiltinId.type_info` (comptime-only, alongside `declare`/`define`).
- `lower/call.zig:tryLowerReflectionCall`: the old "not yet implemented" bail is
gone. Resolve `$T` at lower time, reject a non-`enum`/non-`tagged_union` arg
loudly (good span: `"type_info: 'X' is not an enum …"`), else emit
`callBuiltin(.type_info, [const_type], TypeInfo)`.
- `interp.zig:reflectTypeInfo`: builds the exact nested-aggregate Value
`defineEnum` decodes — variant `{name, payload}`, slice `{data, len}`, EnumInfo
`{variants}`, TypeInfo `{tag0, EnumInfo}`. A `tagged_union` reflects each
`field.ty` (tagless variants already carry `void`); a payloadless `` `enum ``
reflects `void` per variant. Round-trips both source enums AND constructed
(declare/define) enums.
- emit unchanged — `type_info` is always comptime-evaluated; the existing
comptime-only `else` arm in `emitCallBuiltin` (shared with declare/define)
never fires.
- Scope: **enum-only** (the symmetric inverse of `define`'s current capability).
Struct/tuple `TypeInfo` widening is a separate later step.
`examples/0619` locks it (source enum `circle:f64 / rect:i64 / empty` reflected →
reconstructed → constructs + matches). Full suite green (676 examples + units).
## Earlier step
**Self-reference — recursive enums via `declare("Name")` + `*Name`.** The
`declare`/`define` floor now supports self-referential types.
- `declare(name) -> Type` mints an empty (undefined) nominal slot NAMED `name`;
`define(handle, info) -> Type` decodes the `TypeInfo` value (variant names +
payload Type-tags), fills the slot byte-identical to a source enum, and returns
the handle (one-shot form chains: `T :: define(declare("T"), info)`). Interp
executes both against a `mint` TypeTable handle; `defineEnum` +
`decodeVariantElements` in `interp.zig`.
- **Self-reference:** `evalComptimeType`'s `preregisterForwardTypes` scans the
comptime expression (and a called ctor fn's body) for `declare("Name")` calls
and, before the body lowers, registers each as an empty forward nominal type AND
binds it as a type alias. The alias is essential — a `Name :: ctor()` decl makes
`Name` a const_decl author, so a `*Name` self-reference resolves through the
forward-ALIAS path (`type_aliases_by_source`), which a bare `findByName`
registration alone does NOT satisfy (it returns a pending empty-struct stub). The
interp's `declare` returns that same slot; `define` fills it.
- A `::` binding or type-fn body calling a `Type`-returning fn is
**comptime-evaluated** (`evalComptimeType`) — no constructor-name knowledge.
`decl.zig` trigger = `fnReturnsTypeValue`; type-fn trigger = `returnExprMintsType`.
- Nominal identity rides the type-fn instantiation cache (`renameNominalType`).
- The type NAME is on `declare(name)` (compile-time string), not `EnumInfo`.
Examples green: `0614` (one-shot), `0615` (type-fn identity), `0617` (channel
results), **`0618` (recursive `*List`: construct, match through pointer, recursive
traversal)**; `field_type` reflection `0616`. Full suite green (674 examples).
## Current state
- `modules/std/meta.sx`: `EnumVariant` / `EnumInfo{ name, variants }` / `TypeInfo`
data types; `declare` / `define` / `type_info` / `field_type` `#builtin`s;
`RecvResult($T)` / `TryResult($T)` + the general `make_enum(name, variants)` sx
constructors over `define(declare(), …)`.
- Compiler primitives only: `declare`/`define` (construction), `field_type`
(reflection). No constructor-name knowledge anywhere in the compiler — every
named constructor is sx. `declare(name)` carries the type name (compile-time
string) for forward-type registration.
- `type_info($T)` reflects an `enum`/`tagged_union`/`struct`/`tuple` INTO a
`TypeInfo` value (`call.zig` emits `callBuiltin(.type_info)`;
`interp.zig:reflectTypeInfo` builds the Value). `define` decodes `.enum` →
tagged_union, `.struct` → struct, `.tuple` → tuple (the last via
`replaceKeyedInfo`). `examples/0619` (enum) / `0622` (struct) / `0623` (tuple)
round-trip. All three TypeInfo shapes ship.
## Decision (kept)
**Meta lives in `modules/std/meta.sx`, not the prelude.** Declaring its data types
in the always-loaded prelude interns them into every module's type table and
shifts every `.ir` snapshot. On-demand import keeps the prelude clean.
## Next step
The reflect/construct triad is COMPLETE — `` `enum `` (`0619`), `` `struct ``
(`0622`), `` `tuple `` (`0623`) all reflect AND construct + round-trip. Remaining
METATYPE work is ONE deferred enhancement, a clean diagnostic rather than a crash
— filed as **issue 0141** (repro `issues/0141-*.sx` + full two-layer writeup +
investigation prompt):
- **Comptime `List` growth** — `List(T).append` at comptime bails ("struct_get:
base has no fields"). Doesn't block anything: array-literal locals already build
variant lists (`examples/0620`/`0624`). Probe `.sx-tmp/probe_makeenum.sx` /
`probe_li64.sx`. **Investigated — it's TWO layers** (both reproduce with plain
`List(i64)`, not metatype-specific; List works via `#run` because that evaluates
at EMIT time, after everything is lowered, while a metatype `::` const evaluates
at `scanDecls` time):
1. **Null comptime allocator.** `interp.zig:defaultContextValue` builds the
comptime `context.allocator` by looking up `__thunk_CAllocator_Allocator_alloc_bytes`
by name in the module's functions — but at `scanDecls` time those protocol
thunks aren't lowered yet, so `alloc_fn`/`dealloc_fn` are `.null_val` and any
comptime allocation fails. FIX (tried, works for this layer): call
`self.getOrCreateThunks("Allocator", "CAllocator")` (guarded by the same
Context/Allocator/CAllocator-registered check `emitDefaultContextGlobal` uses)
before the interp runs in `comptime.zig:runComptimeTypeFunc`.
`createProtocolThunk` saves/restores builder state, so calling it mid-lowering
is safe. After this, `alloc_fn=func_ref` — but layer 2 still bails.
2. **`struct_get` through a `*T` slot_ptr chain.** A `*List` struct receiver
(`vs.append(…)` → `append(self: *List, …)`) lands in the interp as a slot
whose contents are a slot_ptr to the actual value — `self.field` does
`struct_get` on `base=slot_ptr field_index=1` and bails. The auto-deref in
`interp.zig:.struct_get` does a single `loadSlot`; a chain-resolve loop did
NOT fix it (the final loaded value is a field-pointer aggregate that
`resolveFieldLoad` turns back into a slot_ptr — List's comptime representation
uses field-pointers + slot_ptrs the struct_get path doesn't fully resolve).
This is the deep part: comptime pointer/struct/slot resolution for `*T`
receivers, its own focused effort. Both speculative fixes were REVERTED (no
end-to-end testable win without layer 2).
The metatype surface (declare/define/type_info/field_type + make_enum) is
feature-complete for the locked design; generic type-fn body locals now work too.
- ~~**Validation + loud diagnostics**~~ — COMPLETE. duplicate variant names
(`examples/1180`); `declare()` never `define()`d (`examples/1181`, was a
`verifySizes` panic); by-value self-reference for both source (`1178`) and
CONSTRUCTED (`1182`) types via `checkInfiniteSize`. **use-before-define needs no
new check** — it's subsumed by the existing guards: a by-value cycle →
`checkInfiniteSize` ("infinitely sized"); an unfinished slot → declare-never-
defined; a bad/non-Type payload → a 0140 clean bail; a forward reference resolves
correctly via in-place slot mutation (`updatePreservingKey`); a `*Name` pointer
needs no layout. Probes `.sx-tmp/probe_ubd{1..4}.sx` confirmed: no remaining
crash or silent-corruption, only clean diagnostics / correct results.
### make_enum follow-ups (deferred capability gaps — NOT crashes; clean diagnostics)
`make_enum` itself is DONE (see Last completed step). Remaining adjacent
capabilities would let the variant list be built more freely; both error cleanly
(post-0140) rather than crash, so they're enhancements, not blockers:
- ~~Comptime slice over a non-string aggregate~~ — DONE. `arr[lo..hi]` over a
`[]EnumVariant` array now yields a real slice value at comptime (was: bailed,
string-only). Fix threaded `base_ty` onto the `Subslice` op so the interp tells
an array from a `{ptr,len}` slice, folded open-ended `hi` to a fixed array's
static length at lower time (no runtime/.ir change), and added
`interp.zig:subsliceElements`. `examples/0621` locks it.
- **Comptime `List` growth** (issue 0141). `List(T).append` at comptime bails
("struct_get: base has no fields"). Investigated — two layers (null comptime
allocator at scanDecls + `struct_get` through a `*T` slot_ptr chain); see the
detailed writeup under "Next step" and `issues/0141-*.md`. Layer 1 has a known
fix; layer 2 is deep. Probe `.sx-tmp/probe_makeenum.sx`.
- ~~Generic type-fn body locals~~ — DONE. A generic `($T) -> Type` now
comptime-evaluates its FULL body (prelude statements + return), so a local
before the return resolves. `createComptimeFunctionWithPrelude` +
`evalComptimeTypeBody`; no-prelude bodies stay on the old path. `examples/0624`.
## Known issues
- issue 0141 (OPEN, deferred enhancement — not a blocker) — `List(T).append` at
comptime bails in a type-construction `::` (two layers: null comptime allocator
+ `*T` slot_ptr `struct_get`). Workaround: array-literal locals
(`examples/0620`/`0624`). Full writeup + investigation prompt in
`issues/0141-*.md`.
- issue 0140 — comptime type-construction bail panicked instead of diagnosing —
RESOLVED. `evalComptimeType` now clears `last_bail_detail` before the interp
call and, on the `catch`, emits a build-gating `.err` at the construction span
("comptime type construction failed: {detail}") before returning the
`.unresolved` poison — so the reason is shown and no unresolved type reaches
emission unannounced. `examples/1179` locks it.
- issue 0139 — by-value self-reference segfault — RESOLVED (`checkInfiniteSize`
Pass 1g emits a loud "infinitely sized" diagnostic + breaks the cycle;
`examples/1178` locks it).
## Log
- **Generic type-fn body locals.** A generic `($T) -> Type` comptime-evaluated
only its return EXPRESSION, so a local before the return was unresolved. Now a
body with a prelude (statements before the return) has its FULL body evaluated:
`createComptimeFunctionWithPrelude` lowers the pre-return statements into the
comptime function's scope, then the return expr. No-prelude bodies (RecvResult
etc.) stay on the old path → zero regression. `examples/0624`. Suite green (685).
- **Tuple widening done — reflect/construct triad complete.** `TypeInfo` gained
`` `tuple(TupleInfo) `` (positional `[]Type`); `reflectTypeInfo` reflects a
`.tuple` (bare type_tags, tag 2), `defineType` dispatches tag 2 → `defineTuple`
(completes the slot as a structural tuple via `replaceKeyedInfo`), and the
lower-time `type_info` guard admits `.tuple`. `examples/0623`. Suite green (684).
enum/struct/tuple all reflect + construct + round-trip.
- **Struct widening done.** `TypeInfo` gained `` `struct(StructInfo) ``; `define`
dispatches on the tag (`defineType` → `defineEnum`/`defineStruct`), `reflectTypeInfo`
reflects a `@"struct"`, and the lower-time `type_info` guard admits structs.
`defineStruct` uses `replaceKeyedInfo` (kind change: tagged_union declare slot →
struct). `examples/0622` (programmatic build + source round-trip). Suite green
(683). Tuple is the last remaining shape.
- **Validation story COMPLETE.** use-before-define needs no new check — subsumed
by `checkInfiniteSize` (by-value cycles), declare-never-defined (unfinished
slots), 0140 bails (bad payloads), and in-place slot mutation (forward refs);
`*Name` pointer use needs no layout. Probed `.sx-tmp/probe_ubd{1..4}.sx`: all
clean diagnostics / correct results, no crash. `examples/1182` locks the
by-value self-ref rejection for CONSTRUCTED enums (companion to source `1178`).
- **declare()-never-defined validation.** A bare `declare("X")` with no `define`
left a zero-field nominal slot that panicked at codegen (`verifySizes`).
`evalComptimeType` now detects a zero-variant `tagged_union` result and emits a
clean diagnostic naming the type. Self-reference (declared slot completed by
`define`) is unaffected. `examples/1181` locks it. Suite green (681).
- **Duplicate variant-name validation.** Two same-named variants in a constructed
enum used to silently succeed (ambiguous construction/match). `defineEnum` now
bails naming the duplicate; `evalComptimeType` renders it (post-0140).
`examples/1180` locks it. Suite green (680).
- **Comptime subslice over non-string aggregates.** `arr[lo..hi]` at comptime
used to bail (interp `.subslice` was string-only) and the open-ended `hi` came
from a `.length` op that misread a 2-elem array as a `{ptr,len}` fat pointer.
Fix (interp-only; runtime already correct via `LLVMTypeOf`): thread `base_ty`
onto the `Subslice` op, fold open-ended `hi` to a fixed array's static length at
lower time (no IR/.ir change), add `subsliceElements`. `examples/0621` mints an
enum from `dirs[0..2]`. Suite green (679).
- **`make_enum` done.** General enum constructor `make_enum(name, variants:
[]EnumVariant) -> Type` in `meta.sx` (pure sx over declare/define). A non-generic
builder assembles the variant list in a local, then mints from it —
`examples/0620` exercises `define`'s value-arg SLICE decode. No compiler change.
Suite green (678). Deferred free-form gaps (subslice/List at comptime,
generic-type-fn locals) noted under Next step — all clean diagnostics now, not
crashes (post-0140), so enhancements rather than blockers.
- **issue 0140 fixed.** A comptime type-construction bail (`declare`/`define`/
reflection) used to panic at LLVM emission ("unresolved type reached LLVM
emission") or hide behind a cascade — `evalComptimeType` swallowed the interp's
`last_bail_detail`. Now it clears the detail before the call and renders a
build-gating `.err` at the construction span on the `catch`. `examples/1179`
locks the empty-variants case. Suite green (677). Unblocks make_enum (its
computed-slice decode failures now surface cleanly).
- **`type_info($T)` reflection done (enum round-trip).** New `BuiltinId.type_info`;
`lower/call.zig` resolves `$T`, rejects non-enum loudly, emits the builtin;
`interp.zig:reflectTypeInfo` constructs the exact nested-aggregate Value
`defineEnum` decodes (variant `{name,payload}` / slice `{data,len}` / EnumInfo /
TypeInfo `.enum`). `tagged_union` reflects `field.ty`; payloadless `` `enum ``
reflects `void`. Round-trips source AND constructed enums. Enum-only;
struct/tuple widening deferred. `examples/0619` locks it. Suite green (676).
- **By-value self-reference rejected (issue 0139, F5 partial).** New
`checkInfiniteSize` pass (Pass 1g) detects by-VALUE containment cycles (source +
comptime types, direct + mutual), emits a loud "infinitely sized" diagnostic,
and breaks the cycle (was a `typeSizeBytes` stack-overflow segfault). `*Self`
(pointer) stays valid. `examples/1178` locks the message. Suite green (675).
- **Self-reference done.** `declare(name)` + `preregisterForwardTypes` (forward
type + alias before body lowers) → `*Name` resolves; recursive `*List` enum
constructs, matches through the pointer, and traverses recursively. `0618` locks
it. `declare` gained its `name` arg; `EnumInfo.name` dropped. Suite green (674).
- **declare/define floor established.** The comptime type-construction surface is
two primitives (`declare`/`define`); all named constructors are sx. A `::` binding
or type-fn body that calls a `Type`-returning fn is comptime-evaluated (the
builtins mint the type) — no syntactic constructor recognition in the compiler.
Examples 0614 (one-shot) / 0615 (type-fn identity) / 0617 (channel results) on the
floor; `field_type` reflection (0616) unchanged.
- **Stream carved (earlier).** Selected as the first async-first foundation: gates
channel result types (`RecvResult($T)`) and `race`'s synthesized union, fully
validated, self-contained, testable in isolation (`06xx` comptime).

153
current/CHECKPOINT-MULTIRET.md
View File

@@ -1,153 +0,0 @@
# CHECKPOINT-MULTIRET — bare-paren multi-value + named returns
Plan: `current/PLAN-MULTIRET.md`. Branch: `feat/multi-return`.
## Last completed step
**Phases 0–3 implemented** (final suite + snapshot capture in progress). Examples
renumbered to the free `types` block 0202–0206 (0130/0131 already had duplicate
existing owners).
- **Phase 0** — empty `()` in the type path → `void`. (0202)
- **Phase 1** — multi-return SIGNATURES `(A, B)` / `(x: A, y: B)` / `(A, B, !)`
(≥2 value slots) parse to a `tuple_type_expr` tagged `is_multi_return`; a
single-value `(T, !)` is a plain failable (= `-> T !`). Return resolver yields
the reused tuple TypeId; `resolveParamType` rejects a multi-return tuple
(return-position-only). Consumed by destructuring. (0203, 0204)
- **Phase 2** — bare comma `return v1, v2` (positional) / `return x = v, y = w`
(named): the return parser builds the same `tuple_literal` the `.(…)` form
produces. Single positional `return v` unchanged. (used throughout 0203–0205)
- **Phase 3** — NAMED-return slots are in-scope assignable LOCALS: bound as
zero-init allocas (`bindNamedReturnSlots`), the implicit return is synthesized
from them (`synthesizeNamedReturn` → reuses `lowerReturn`), and the MUST-SET
rule errors on an unset/undefaulted slot (`bodyAssignsTo`, path-insensitive
MVP). Works with the failable error channel too. (0205 positive, 0206 negative)
Earlier foundation: parser `collectGenericNames` descends tuple/optional/function
nodes; generic.zig `extractTypeParam` handle the `(value, !)` tuple.
## Current state (works, verified by probes)
- `() -> ()` ≡ void; `-> (A, B)` / `-> (x: A, y: B)` / `-> (A, B, !)` multi-return.
- `return a, b` / `return x = a, y = b` bare comma; named-return locals + implicit
return + must-set diagnostic; failable named multi-return.
- `(A, B)` in a PARAM slot → loud diagnostic.
- Representation: `TupleTypeExpr.is_multi_return` flag + `Lowering.named_return_names`
state (reuses tuple ABI; no new TypeInfo variant; multi-return-ness derivable
from the FnDecl AST).
## Post-Phase-3 changes (this session)
- **Representation refactored to a dedicated `ReturnTypeExpr` AST node** (user
preferred it over the `TupleTypeExpr.is_multi_return` flag). Resolves to a
reused `.tuple` TypeId via the shared `internTupleLike` helper. Forced
`.return_type_expr` arms onto the exhaustive `node.data` switches (sema,
semantic_diagnostics) — the coverage benefit. Param-position reject + the
named-return-locals / must-set sites now key off `.return_type_expr`.
- **Destructure-only enforcement REVERSED** (user): single-binding a multi-return
is ALLOWED. `c := f(); c.sum` works (the result is a tuple of the value slots).
For a failable multi-return, `c := f() catch …` binds only the value slots —
the error stays on the `!` channel (verified). The `callIsMultiReturn` reject
was removed. Examples 0203/0204 updated to show single-bind + field access
(output byte-identical, snapshots unchanged).
## Phase 4 — named-return DEFAULTS (done, suite pending)
`-> (sum: i32 = 0, good: bool)`: the parser parses `= <expr>` per slot into
`ReturnTypeExpr.field_defaults`; `bindNamedReturnSlots` seeds a defaulted slot
with its (lowered+coerced) default; a defaulted slot is EXEMPT from the must-set
rule. Also fixed `hasFnBodyAfterArrow` (the fn-def-vs-type-const lookahead) to
skip `=` + literal tokens in the return-type scan — otherwise a `=` made the decl
misread as a bodyless type-const ("expected ';'" at the body `{`). Lock: 0207.
## Adversarial-review fixes (this session)
An adversarial review found 8 issues; fixed the soundness + silent-wrong ones:
- **#1 (segfault on a conditionally-assigned non-scalar slot)** → must-set is now
PATH-SENSITIVE definite-assignment (`definitelyAssigns`, stmt.zig): a slot not
assigned on every non-diverging path (and undefaulted) is a COMPILE ERROR, not
a runtime garbage read. `return`/`raise` count as divergence; `if` needs both
branches; `push` bodies count; `match` needs an else arm + all arms.
- **#2 (wrong-type default → segfault)** → `bindNamedReturnSlots` type-checks a
default via the coercion classifier (`.none` ⇒ diagnostic). (NOTE: the same
silent bitcast/segfault exists for ANY annotated assignment `x: i32 = "hi"` — a
broader PRE-EXISTING type-checking gap, not multi-return-specific.)
- **#3 / #8 (return arity garbage)** + **#4 (named elements ignored)** →
`validateMultiReturn` (stmt.zig, called from `lowerReturn`): rejects a bare
value where ≥2 are required, wrong arity, a comma list from a single-value fn,
and named elements out of slot order. (Reordering-by-name is a future nicety;
for now a mismatch is a loud error, never silent-wrong.)
- **#5 (slot shadows param)** → collision diagnostic in `bindNamedReturnSlots`.
- **#7 (push/defer false must-set error)** → subsumed by the DA rewrite (push
bodies count; defer correctly does NOT, as it runs after the implicit return).
## Known limitations / next
- ~~**#6 (design gap)**: `ReturnTypeExpr` silently accepted in non-return positions~~
— **DONE** (2026-06-27): generic-type-arg position now rejected
(`rejectMultiReturnValueType` at both `instantiateGenericStruct` arg-resolution
sites, generic.zig). Param / field / variable already rejected. Type-alias
`T :: (A,B)` is value-parsed → already rejected. Closure-RETURN `(A,B)` is a
legitimate return position → see D3 below (works as a multi-return closure).
Lock: 0215 (negative generic-arg).
- ~~**Reordering named return elements by name** (vs requiring slot order)~~ —
**DONE** (2026-06-27): `reorderNamedReturn` (stmt.zig) permutes a fully-named
multi-return list to slot order by name (value-only AND full-failable-tuple
forms); errors on unknown / duplicate / missing-slot names; positional & mixed
lists pass through unchanged. `validateMultiReturn`'s old slot-order check was
removed. Adversarial review caught a silent mis-permute in the full-failable-
tuple named form (now reordered/validated, not positionally dropped). Lock:
0210 (positive reorder, incl. failable) + 0214 (negative: unknown / duplicate).
- ~~**PRE-EXISTING**: annotated-assignment type mismatch (`x: i32 = "hi"`) segfaults~~
— **RESOLVED** as issue 0197 (2026-06-27): width-mismatch guard
(`checkAssignable` / `noneReinterpretIsUnsafe`, coerce.zig) at every
annotated-slot store site; the named-return-default guard now shares it. Locked
by `examples/diagnostics/1205` + `1206`.
- ~~Multi-return CLOSURE-TYPE values / lambda literals deferred (D3).~~ —
**RESOLVED** (2026-06-27): they ALREADY WORK via the reused tuple machinery. A
`Closure() -> (A, B)` value's call result destructures (`a, b := cb()`),
single-binds + field-accesses (`c := cb(); c.0`), and a `() => { return v1, v2; }`
lambda literal satisfies a multi-return closure param — verified identical to
the function-decl surface. NO `ClosureInfo.multi_return` marker needed (the
destructure-only rule was reversed, so there's nothing extra to enforce). Lock:
0216.
- **Generic multi-return (Task 2d): DONE.** POSITIONAL works — `(a: $T, b: $U) -> (T, U)`
(inferred) and `($T: Type, …) -> (T, U)` (explicit); lock 0217. NAMED-slot
implicit-return form now works too (issue **0200 RESOLVED** —
`monomorphizeFunction` now calls `bindNamedReturnSlots`; covers free fns +
generic struct methods, defaults, failable); lock 0218.
- Docs: readme.md / specs.md not yet updated for multi-return (docs-track rule).
## Known issues
- ~~**issue 0198**: implicit `Any → T` unbox unchecked (segfault / silent garbage)~~
— **RESOLVED** (2026-06-27): implicit `Any → T` is now a compile error
(`coerceMode` `.unbox_any` arm, mode == .implicit); `xx` + match dispatch
unaffected. Locked by `examples/diagnostics/1207`.
- ~~**issue 0199**: `Any == <concrete>` aborts the LLVM verifier~~ — **RESOLVED**
(2026-06-27): the `Any`-shaped `==`/`!=` arm (expr.zig) now fires when EITHER
operand is `.any`, boxing the concrete side first. Lock 0654.
- ~~**issue 0200**: NAMED generic multi-return implicit-return "produces no value"~~
— **RESOLVED** (2026-06-27): `monomorphizeFunction` now calls
`bindNamedReturnSlots` (it previously bound params but skipped named-return
slots). Covers generic free fns + struct methods, defaults, failable. Lock 0218.
## Log
- **2026-06-27 session** (handover: issue 0197 → finish multi-return → Io Phase 3):
- **issue 0197 RESOLVED** — width-mismatch guard at every annotated-slot store
site (var/const-decl, single + multi assignment for identifier/field/index/
element/deref, named-return defaults). Examples 1205 + 1206. Adversarial review
caught & fixed: a bare-fn-ref false-positive (size-discriminator via
`typeSizeBytes`, not the wrong fn-ref typing) and an aggregate-overrun
false-negative (sx-padded `sizeOf` → LLVM-accurate `typeSizeBytes`); cascade
suppression via `externalErrorsExist` (guard tallies its own diagnostics).
- **issue 0198 RESOLVED** — implicit `Any → T` unbox is now a compile error
(reviewer-confirmed sound). Example 1207. **issue 0199 FILED** (Any==concrete
LLVM-verify abort, loud, open).
- **multi-return Task 2 DONE** (2a reorder 0210/0214; 2b reject in generic-arg
0215; 2c D3 closures already work 0216; 2d positional generic works 0217 +
named-generic gap filed as 0200). Multi-return feature surface complete.
- **REMAINING** (next session): **Task 3 Io-unification Phase 3** (the
capture-typing blocker below + true cancellation — needs fresh context + both
macOS & aarch64-linux validation per PLAN-IO-UNIFY.md). (0198/0199/0200 all
resolved this session; no open multi-return/type-check issues remain.)
- Pivoted here from the Io-unification Phase 3 (true cancellation), which is
PAUSED at its blocker: capturing a failable closure into a nested closure loses
its failability (`worker() catch` → operand type 'unresolved'; repro
`.sx-tmp/pD.sx`/`pE.sx`). That capture-typing gap is unrelated to multi-return
and waits for a later session. The Io-Phase-3 stdlib edits (core/sched/io +
example 1825) were REVERTED to keep the tree green; the multi-return-relevant
compiler changes were kept.
- Foundation landed + suite green; plan + checkpoint written.

167
current/PLAN-ASM.md
View File

@@ -1,167 +0,0 @@
# sx Inline Assembly — Implementation Plan (ASM stream)
**Design source of truth:** [design/inline-asm-design.md](../design/inline-asm-design.md).
This plan turns that doc's §II.7 stage-map + §II.8 phasing into ordered,
commit-sized, testable steps. Read the design doc first — this file is the
*how/when*, not the *what/why*.
**Surface (decided):**
`asm volatile { "template", "=r" -> T, "r" = expr, clobbers(.cc, .memory) }`
— brace block; `->` output / `=` input; `clobbers(.…)` dot-name list; N `-> Type`
outputs return a tuple; templates are pure AT&T (via LLVM).
**Feasibility (confirmed):** sx links LLVM@19; `src/llvm_api.zig` `@cImport`s
`llvm-c/Core.h`, so `llvm_api.c.*` already exposes `LLVMGetInlineAsm` (9-arg),
`LLVMInlineAsmDialectATT`, `LLVMBuildCall2`, `LLVMAppendModuleInlineAsm`. No shim.
**Relationship to other streams:**
- Phases A–E (the inline-asm *expression*) are independent of EXTERN-EXPORT.
- Phase F (global asm) consumes `extern`/`export` to import/expose asm symbols —
do it **after** `PLAN-EXTERN-EXPORT.md` Phase 2.
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass. Each feature step is either a
behavior-locking PASSING test, or an xfail test the *next* commit turns green.
Arch-pinned tests live in `examples/16xx-platform-asm-*` and declare their target
via the `expected/<name>.target` sidecar marker (Phase 0). Never regenerate
snapshots while red.
## Phase 0 — corpus target-gating (test-infra prerequisite; no compiler code)
**Why first.** The flagship v1 examples are `x86_64` (syscall-write, divmod,
cpuid) but the dev host is `aarch64`-Darwin, and the corpus runner
([src/corpus_run.test.zig](../src/corpus_run.test.zig)) currently (a) never threads
a per-example `--target` and (b) has no host-arch gate — its only skip is "marker
has no `.sx`". So D.0's `…-syscall-write` markers asserting exit/stdout describe
output the harness *cannot* produce on this host, which would violate the cadence
rule (the "next commit turns it green" can never happen). Phase 0 closes that gap.
It touches **only the runner + two fixtures** — zero compiler code, zero risk to
A–E, and unblocks every arch-pinned asm example.
**Marker taxonomy (the cleanup).** The runner currently spreads per-example
*directives* across standalone boolean/value sidecars (`.aot` now, `.target`
proposed, more later). Replace that sprawl with **one optional config file,
`expected/<name>.build`**, holding all build/run directives; the output snapshots
(`.exit` / `.stdout` / `.stderr` / `.ir`) stay separate — they are
machine-regenerated data, not config. `.exit` remains the **test-discovery key**
(every test has one; `.build` is optional).
**`.build` format** — JSON, parsed with `std.json`:
```json
{ "aot": true, "target": "x86_64-linux" }
```
Parse via `std.json.parseFromSlice(BuildConfig, …)` into
`struct { aot: bool = false, target: ?[]const u8 = null }`. Field defaults cover
omitted keys; `std.json`'s default `ignore_unknown_fields = false` makes an
**unknown key a loud `error.UnknownField`** (surfaced as a runner failure, never a
silent ignore — CLAUDE.md no-silent-default rule). Extensible: future `"cpu"`,
`"link"`, `"cwd"` are just new optional struct fields, no new sidecar file and no
custom parser.
**What the directives do:**
1. **`target = <triple|shorthand>`** threads `--target <value>` into every `sx`
invocation for that example (`run` / `build` / `ir` — `--target` is a global
flag, confirmed [main.zig:39](../src/main.zig#L39)), AND **host-match selects
the mode.** The runner parses the leading `arch` + `os` tokens of the resolved
triple and compares them to `@import("builtin").target` (normalizing
`arm64`→`aarch64`):
- **match** → *execute* exactly as today (`sx run`, or `aot` build+exec) with
the target threaded, plus the `.ir` diff if an `.ir` snapshot exists. ⇒ an
x86_64 example gives **real end-to-end coverage on an x86_64 CI runner**.
- **mismatch** → **ir-only**: run *only* `sx ir <file> --target <t>`; assert
`.exit` (the ir command's exit), `.ir` (normalized stdout), and `.stderr`
(diagnostics, normally empty). Do **not** run/build/exec; do **not** assert
`.stdout`. An `.ir` snapshot is **required** in ir-only mode — its absence is
a loud runner failure ("arch-pinned <name>: ir-only mode requires an .ir
snapshot"), never a silent pass. Robust even if `sx ir` treats `--target` as
a partial no-op: the `inline_asm` op carries the template + constraint string
verbatim, so the IR snapshot still locks the exact thing §II.11 flags as
silently-miscompiling (the constraint assembler + template rewrite).
2. **`aot`** is the existing JIT-vs-build+exec switch, just relocated from the
standalone `.aot` marker into `.build`.
**Negative compile-error examples need NO `.build`.** `…-missing-volatile`
(no-output-without-`volatile`) is a Sema diagnostic raised before codegen/JIT, so
plain `sx run` reports it identically on any host — it stays a normal example with
no config file.
**update-goldens interaction:** in ir-only mode, `-Dupdate-goldens` writes `.exit`
(ir exit) + `.ir` (+ `.stderr` if non-empty) and skips `.stdout`. Execute mode
(incl. `aot`) is unchanged. `.build` is hand-authored — update-goldens never
writes it.
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | Add `BuildConfig` + `std.json` parse of `expected/<name>.build` (unknown-key ⇒ `error.UnknownField`); **migrate** the 2 existing `.aot` markers → `.build` (content `{ "aot": true }`) and delete them; thread `target`'s `--target` into the spawned argv; add `hostMatchesTarget(value) bool` (arch+os token parse, `arm64`→`aarch64`) gating the **execute** path. Lock with `examples/16xx-platform-target-host.sx` (trivial `main`) + a `.build` `{ "target": "<host arch triple>" }` (still runs+passes) and unit `test`s for the JSON parse + `hostMatchesTarget`. | `src/corpus_run.test.zig`, `examples/expected/1226-*.{aot→build}`, `…/1227-*`, + fixture |
| 0.1 | lock | Implement the **mismatch ⇒ ir-only** branch (skip run/build/exec; assert `.exit`+`.ir`+`.stderr` from `sx ir --target`; require `.ir`). Lock with `examples/16xx-platform-target-cross.sx` (asm-free `() -> i64 { return 0; }`) + `.build` `{ "target": "x86_64-linux" }` + a checked-in `.ir` snapshot — exercises ir-only on the arm64 host. | `src/corpus_run.test.zig` + fixture |
| 0.2 | docs | Update CLAUDE.md §"Test layout"/§"Testing" to document `.build` (format + `aot`/`target` keys) replacing the standalone `.aot` marker prose (lines ~435, ~492). | `CLAUDE.md` |
Both 0.0 and 0.1 are **lock** commits: the runner change and the fixture that
exercises it land together and pass the moment they land (the mechanism works
immediately — nothing is left red), which is the cadence rule's "lock in current
behavior" flavor, not a feature red→green. No asm lowering is gated on either.
**Phase 0 verification:** `zig build test` green; deliberately corrupt the
cross-target `.ir` fixture and confirm the runner reports an IR mismatch (proves
ir-only actually asserts, isn't a no-op); delete it and confirm the
"requires an .ir snapshot" failure fires.
**Estimated runner delta:** ~70–90 lines (sidecar read + `--target` argv threading
+ `hostMatchesTarget` + the ir-only branch + update-mode tweak). Within the
"no step > ~500 new lines" rule; well under the read budget.
## Phase A — keyword + AST + parser (parses; no codegen)
| Step | Commit | What | Files |
|---|---|---|---|
| A.0 | lock | add `kw_asm` keyword + map entry; unit lex test `asm → kw_asm` | `src/token.zig`, `src/lexer.zig` + `.test.zig` |
| A.1 | xfail | parse `asm { … }` → `AsmExpr`/`AsmOperand` in `parsePrimary`; pin an AST/`sx ir` parse snapshot; lowering still `bailDetail("inline asm codegen unimplemented")` | `src/ast.zig` (:85 union arm, :721 structs), `src/parser.zig` (parsePrimary), `src/ir/interp.zig` |
| A.2 | green | parse-shape snapshot lands green; the unimplemented bail is loud + named | — |
## Phase B — sema / typing
| Step | Commit | What | Files |
|---|---|---|---|
| B.0 | xfail | result-type rule (0→`void` / 1→`T` / N→named-or-positional tuple) + checklist (no-output⇒`volatile`, layout, comptime-string template) — pin error messages | `src/ir/expr_typer.zig` |
| B.1 | green | typing + diagnostics implemented; `.unresolved` sentinel on failure (no silent default) | `src/ir/expr_typer.zig`, `src/ir/semantic_diagnostics.zig` |
## Phase C — IR op + lowering
| Step | Commit | What | Files |
|---|---|---|---|
| C.0 | lock | add `inline_asm: InlineAsm` to `Op` + `AsmOperand` (role/name/constraint/operand) + interp `bailDetail` arm; unit tests for the IR shape | `src/ir/inst.zig` (:80), `src/ir/interp.zig` |
| C.1 | xfail→green | `lowerAsmExpr` in `lowerExpr` dispatch — interns template/constraints/clobber-names, lowers input `Ref`s, sets result `TypeId` | `src/ir/lower/expr.zig` |
## Phase D — LLVM emit (single value-output; the core)
| Step | Commit | What | Files |
|---|---|---|---|
| D.0 | xfail | `examples/16xx-platform-asm-syscall-write.sx` + `…-register-read.sx` + `…-no-output-volatile.sx` + `…-missing-volatile.sx` (expected compile error) — all red | examples + `expected/` markers |
| D.1 | green | `emitInlineAsm`: **port `FuncGen.airAssembly`** — constraint-string assembler (outputs `=`/`+`, inputs, `clobbers(.name)`→`~{name}`), `%[name]`→`${N}` / `%%` / `%=` template rewriter, `LLVMGetInlineAsm`+`LLVMBuildCall2`, `sideeffect=volatile`, AT&T dialect | `src/ir/emit_llvm.zig` (emitInst dispatch + handler) |
| D.2 | green | lock the template-rewrite + constraint string via an `expected/*.ir` snapshot on `…-template-subst.sx` | examples |
**Phase D verification:** `zig build test`; the syscall example runs on
`x86_64-linux`; IR snapshot matches the design doc's worked `sys_write` lowering.
## Phase E — multi-return tuples + `clobbers(.…)`
| Step | Commit | What | Files |
|---|---|---|---|
| E.0 | xfail | `…-asm-multi-return.sx` (`divmod`→`(quot,rem)`, `cpuid`→4-tuple) red | examples |
| E.1 | green | N `out_value` → LLVM struct return + `extractvalue i` → sx tuple (named when operands named); `clobbers(.name)` dot-name lowering finalized | `src/ir/emit_llvm.zig`, `src/ir/lower/expr.zig` |
## Phase F — global asm (needs EXTERN-EXPORT Phase 2)
| Step | Commit | What | Files |
|---|---|---|---|
| F.0 | xfail | top-level `asm { … }` decl parsed (reject operands/`volatile`); `…-asm-global.sx` (defines a symbol, imported via `extern`) red | `src/parser.zig`, `src/ast.zig` |
| F.1 | green | lower `asm_global` → `c.LLVMAppendModuleInlineAsm`; comptime-call guard (dlsym-miss is loud); blocks concatenate in source order | `src/ir/lower/decl.zig`, `src/ir/emit_llvm.zig`, `src/ir/interp.zig` |
## Phase G — later (own steps when scheduled)
`-> @place` write-through + read-write (`"+r" -> @place`) + indirect-memory
(`"=*m"`) outputs · `%=` unique-id · output-to-const rejection · Intel-dialect
opt-in · naked functions (`callconv(.naked)`, coordinate with EXTERN-EXPORT).
## Open decisions (design doc §II.10)
Dialect (AT&T-only v1, recommended) · `volatile` contextual-keyword (recommended)
· brace separator comma (recommended) · `clobbers(.name)` dot-name sugar now →
checked per-arch `Clobber` enum later (Phase 4 of the design doc).
## End-to-end verification (per phase)
`zig build && zig build test`; for arch-pinned examples confirm they run on a
matching host or assert on `sx ir`/`.s` snapshots. After intentional output
changes only: `zig build test -Dupdate-goldens`, then review the diff.

225
current/PLAN-ATOMICS.md
View File

@@ -1,225 +0,0 @@
# PLAN-ATOMICS — Stream A (atomics lowering)
> **STATUS: ✅ COMPLETE (feature-complete).** All phases A.0 → A.3 landed + green.
> Surface shipped: `Atomic($T)` `load`/`store`/`fetch_add`/`sub`/`and`/`or`/`xor`/`min`/`max`/
> `swap`/`compare_exchange`/`compare_exchange_weak` (all comptime `$o: Ordering`) + free
> `fence(.ordering)`. IR ops `atomic_load`/`store`/`rmw`/`cmpxchg`/`fence`. Both LLVM emit
> AND the comptime VM implemented (verified to agree). Enabled by net-new comptime value
> params (enum/tagged_union/generic-struct methods — 3 commits). Corpus `17xx` (1700-1704) +
> `11xx` diagnostics (1130/1131/1186/1187). Commits: 22af404, 64c7db5, 8144a88, acf3183,
> 718f27e, 0531164, 68ed732, dca396e, 79895be, fca4304, b65544a (+ comptime-param 3c4305f,
> d7a6857, d95ba0a). **Unblocks Stream B2 channels + Stream C parallel schedulers.**
>
> Deferred (documented, NOT legacy — intentional scope): RMW/CAS/swap are integer-only
> (float fadd / pointer atomics out of scope); fence/orderings explicit (no defaults — the
> comptime-default-on-generic-method gap is orthogonal). Asm-level arch divergence +
> weak-memory *semantics* remain OUT of corpus scope (Stream-C stress harness).
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream A + the design-of-record
[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §3 (N1)
+ §4.6 (locked surface). Progress in [CHECKPOINT-ATOMICS.md](CHECKPOINT-ATOMICS.md).
**Goal:** net-new LLVM atomic codegen. Surface = a pure-sx `Atomic($T)` generic struct +
an `Ordering` enum (ordinary sx), with the actual atomic operations recognized as
`#builtin` intrinsics at lower-time and emitted as new IR ops. This is **100% net-new** —
no atomics scaffolding exists (the only `lower.zig` "ordering" is *comparison* ordering
`< <= >=`, unrelated to memory ordering — do not mistake it for groundwork).
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `17xx` atomics.
---
## Design (grounded against the tree)
### Representation — minimal compiler surface
- **`Ordering`** is an ordinary sx enum, zero compiler coupling:
```sx
Ordering :: enum { relaxed; acquire; release; acq_rel; seq_cst; } // tags 0..4
```
- **`Atomic($T)`** is an ordinary sx **generic struct** (mirrors `List :: struct ($T: Type)`
at [list.sx:5](../library/modules/std/list.sx#L5)), a transparent 1-field wrapper —
atomicity is a property of the *operation*, not the storage, so `Atomic(i64)` has the
exact layout/size/align of `i64`. NO new IR *type*, NO type-system coupling:
```sx
Atomic :: struct ($T: Type) {
value: T;
init :: (v: T) -> Atomic(T) { return .{ value = v }; }
load :: (self: *Atomic(T), o: Ordering) -> T { return atomic_load(T, @self.value, o); }
store :: (self: *Atomic(T), v: T, o: Ordering) { atomic_store(T, @self.value, v, o); }
}
```
- The **operations** are `#builtin` intrinsic free functions, recognized by name at
lower-time (the established pattern — `size_of`/`type_info` in
[`tryLowerReflectionCall`](../src/ir/lower/call.zig#L1672), recognized BEFORE arg lowering):
```sx
atomic_load :: ($T: Type, ptr: *T, o: Ordering) -> T #builtin;
atomic_store :: ($T: Type, ptr: *T, v: T, o: Ordering) #builtin;
```
Explicit `$T` first arg follows the `size_of($T)` / `field_name($T, idx)` mixed
type+value precedent (lowest-risk; the reflection path already resolves type args).
### Ordering is compile-time-only by construction — and that forces a capability gap
LLVM atomic ordering is an **instruction attribute**, not a runtime operand, so the
ordering MUST be known at emit time. The lower-time handler reads the ordering arg's
variant name statically (it must be a **constant enum literal** `.seq_cst`) and bakes it
into the IR op as a Zig enum field (`AtomicOrdering`). A non-literal ordering is a **loud
diagnostic**, never a silent default (REJECTED-PATTERNS).
**Discovered gap (grounded):** a generic `Atomic(T)` method `load(self, o: Ordering)` would
forward `o` — a *runtime parameter* — to the intrinsic, where it is NOT a literal. And
**comptime enum value params don't exist** (`$o: Ordering` → `o` is "unresolved" in the
body; `resolveValueParamArg` folds integer constraints only). A runtime dispatch hack
(`if o == { case .acquire: atomic_load(…, .acquire) … }`) also fails: `load` with a
`release`/`acq_rel` ordering is *invalid LLVM*, so the arms can't be uniform. Therefore the
**full ordering surface is blocked on a net-new capability** (comptime-constant ordering
propagation — either comptime enum value params, or compiler-recognized `Atomic` method
calls). That capability is its **own step (A.0.5)**, sequenced before ordering-bearing ops.
### sx tag → LLVM ordering is EXPLICIT (non-contiguous!)
LLVM's `LLVMAtomicOrdering` is **not** 0..4: `Monotonic=2, Acquire=4, Release=5,
AcquireRelease=6, SequentiallyConsistent=7` ([Core.h:338-354]). The sx `Ordering` tags
(relaxed=0…seq_cst=4) map via an explicit `switch`, never an identity cast:
`relaxed→Monotonic, acquire→Acquire, release→Release, acq_rel→AcquireRelease,
seq_cst→SequentiallyConsistent`.
### LLVM-C API (verified present in `llvm-c/Core.h`, no new extern decls needed)
- Atomic load = `LLVMBuildLoad2` + `LLVMSetOrdering(v, ord)` + `LLVMSetAlignment(v, size)`
(**alignment is mandatory** on atomic load/store — LLVM verifier rejects atomics without
it). There is **no** `LLVMBuildAtomicLoad`/`Store` (the Explore agent was wrong).
- Atomic store = `LLVMBuildStore` + `LLVMSetOrdering` + `LLVMSetAlignment`.
- (Later) `LLVMBuildAtomicRMW(B, op, ptr, val, ord, singleThread)`,
`LLVMBuildAtomicCmpXchg(B, ptr, cmp, new, succOrd, failOrd, singleThread)`,
`LLVMBuildFence(B, ord, singleThread, name)`, `LLVMSetWeak`.
- `singleThread = 0` (multi-thread / cross-thread ordering). Atomic-eligible `T` =
integer / pointer / float of size 1·2·4·8(·16). **Reject non-scalar / bad-size `T`
loudly** (diagnostic), do not silently emit.
### Comptime VM treats atomics as ordinary load/store
Comptime is single-threaded, so seq_cst is trivially satisfied — the
[`comptime_vm`](../src/ir/comptime_vm.zig#L659) arms for `atomic_load`/`atomic_store`
reuse the ordinary `load`/`store` paths (correct, NOT a bail). `sx run` JITs via LLVM so
runtime atomics execute the real ops; the VM arm only matters for `#run`/const-init.
### Files the new IR op variants force (exhaustive switches)
`atomic_load` / `atomic_store` variants must be handled in every `Op` switch or the Zig
build fails (this is the desired tripwire):
- [inst.zig:159](../src/ir/inst.zig#L159) — add `atomic_load: AtomicLoad`, `atomic_store: AtomicStore` + the structs (mirror `Store` at [inst.zig:286](../src/ir/inst.zig#L286)).
- [lower/call.zig:1672](../src/ir/lower/call.zig#L1672) — recognize the intrinsics, emit the ops (new `tryLowerAtomicIntrinsic`, called alongside `tryLowerReflectionCall` at [call.zig:80](../src/ir/lower/call.zig#L80)).
- [print.zig:231](../src/ir/print.zig#L231) — print arms (sx-IR / `ir-dump`).
- [emit_llvm.zig:1566](../src/ir/emit_llvm.zig#L1566) — dispatch arms → ops.zig.
- [backend/llvm/ops.zig:325](../src/backend/llvm/ops.zig#L325) — `emitAtomicLoad`/`emitAtomicStore` (mirror `emitLoad`/`emitStore`).
- [comptime_vm.zig:659](../src/ir/comptime_vm.zig#L659) — arms reusing load/store.
- Any other `.op` switch the Zig compiler flags (module.zig / program_index.zig) — let the build tell you.
### Test snapshots — the arch-`.ir` requirement is a MISCONCEPTION for atomics
`sx ir` = [`emitIR`](../src/main.zig#L210), which emits **LLVM IR** (respects `--target`);
`sx ir-dump` is the sx-IR printer. At the **LLVM-IR level**, `load atomic i64, ptr %x
seq_cst, align 8` is **arch-invariant** — identical text for x86_64 and aarch64. The
x86-`lock`/MOV vs aarch64-`ldar`/`stlr` divergence happens only at *instruction selection*
(`sx asm`), which the corpus does **not** snapshot. So:
- **A single host `.ir` snapshot** proves the achievable gate (the `load atomic <ordering>`
keyword + correct ordering + alignment emitted). PLAN-POST §A / design §10.3's
"arch-gated x86_64 + aarch64 `.ir`" would capture **byte-identical** files — drop it.
- Optionally add ONE cross-arch ir-only example (`.build {"target":"x86_64-linux"}` on an
aarch64 host) purely as a **cross-target-emission-doesn't-crash** smoke — note in its
header that the IR body is identical to host.
- **State loudly (out of snapshot scope, parallel to the ordering-semantics caveat):**
asm-level arch lowering AND weak-memory ordering *semantics* are NOT proven by `.ir`;
those need the Stream-C stress harness, not the corpus.
---
## Phases
### A.0 — `Atomic($T)` + `Ordering` + **`seq_cst`-only** `load`/`store` ← START HERE
**Scope (descoped per the discovered gap above):** ship the net-new atomic load/store
codegen with a **`seq_cst` literal baked in the method bodies** — `load(self) -> T` /
`store(self, v)` (NO ordering param yet). The intrinsic still carries the full
`AtomicOrdering` field (always `.seq_cst` here); the recognizer + emit handle all five
orderings already, so A.0.5 only has to plumb the *constant* through. Explicit orderings
(`a.load(.acquire)`) land in A.0.5. seq_cst-only is correct (conservative-strongest), not a
silent fallback.
Two-commit cadence (lock-to-bail → green):
- **A.0a (lock)** — land the lib + IR plumbing with emit deliberately bailing:
1. New `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (value +
`init`/`load`/`store`), `atomic_load`/`atomic_store` `#builtin` decls. **Opt-in import
(`#import "modules/std/atomic.sx"`), NOT carried by the universal `std.sx` facade** —
mirrors `trace`. Rationale (grounded): adding the concrete `Ordering` enum to the
universal prelude registers it into EVERY program's global type table, growing
`@__sx_type_is_unsigned` (378→380) and shifting all string-global numbering → churned
37 unrelated `.ir` snapshots + bloats every binary. Atomics is a deliberate concurrency
capability, so consumers import it explicitly.
2. Add IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` + the two op structs
(inst.zig); print arms; comptime_vm arms (reuse load/store); lower recognition
(`tryLowerAtomicIntrinsic`) incl. the const-ordering-literal guard + non-scalar-`T`
reject.
3. emit_llvm/ops.zig arms **bail loudly** for now: `emitAtomicLoad`/`Store` call the
emitter's bail-with-diagnostic path ("atomic load/store LLVM emission not yet
implemented") so the Zig build is exhaustive but the example is red-by-diagnostic.
4. Add `examples/1700-atomics-load-store.sx` (construct `Atomic(i64).init`, `store`,
`load`, `print`). Seed marker; capture snapshot = the emit-bail diagnostic (nonzero
exit). `zig build && zig build test` green (matches the locked bail snapshot). Commit.
- **A.0b (green)** — replace the emit bail with real emission:
`LLVMBuildLoad2`+`LLVMSetOrdering`+`LLVMSetAlignment` / `LLVMBuildStore`+`LLVMSetOrdering`
+`LLVMSetAlignment`, ordering via the explicit sx-tag→LLVM `switch`. Regen `1700` to
success output + capture its host `.ir` (asserts `load atomic`/`store atomic` + ordering).
Add a unit test in `emit_llvm.test.zig` (correct op + ordering + alignment emission).
Review the diff (no stray error text). Commit.
### A.0.5 — comptime-constant ordering propagation (the capability gap)
Enable `a.load(.acquire)` etc. — i.e. an `Ordering` that reaches the intrinsic as a
compile-time constant through a method. Two candidate designs (pick at pickup):
- **(a) comptime enum value params** — make `$o: Ordering` resolve in the body to its
variant tag (extend `comptime_value_bindings`/the typer beyond integers). General,
reusable; larger typer change.
- **(b) compiler-recognized `Atomic` methods** — special-case `Atomic(T).load/store/…`
calls (read the literal ordering arg at the method call site), bounded coupling to the
std `Atomic` type (cf. how `Vector` is special-cased). Smaller; less general.
Also enforce per-op ordering validity (load: relaxed/acquire/seq_cst; store:
relaxed/release/seq_cst; CAS's dual orderings) as **compile errors**, which is exactly what
the constant-ordering path buys. Retrofit the ordering param onto `load`/`store` here.
### A.1 — RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`)
One IR op `atomic_rmw` carrying an `RmwKind` (maps to `LLVMAtomicRMWBinOp*`). Signed vs
unsigned min/max picks `Max/Min` vs `UMax/UMin` from `T`'s signedness. Same lock→green
cadence; `17xx` examples.
### A.2 — `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**)
`atomic_cmpxchg` op (ptr, cmp, new, success_ord, failure_ord, weak). LLVM `cmpxchg`
returns `{T, i1}`; lower to `?T` where **null = success** (extract the i1, invert).
**Validate the two orderings in the compiler** (design §4.6): failure ordering may not be
`release`/`acq_rel` nor stronger than success — loud diagnostic. `_weak` sets `LLVMSetWeak`.
### A.3 — `swap` + `fence(.ordering)`
`swap` = `atomic_rmw` with `Xchg` kind (folds into A.1's op). `fence` = a new `atomic_fence`
op (ordering only) → `LLVMBuildFence`. `17xx` examples.
---
## Gates (per the corrected snapshot story)
- **unit** `emit_llvm.test.zig`: each op emits the right LLVM builder + ordering + alignment.
- **corpus** `17xx` single-thread deterministic (`sx run`, JIT executes real atomics).
- **host `.ir`** snapshot per op proves the keyword/ordering/alignment lowered.
- **OUT of snapshot scope, stated loudly:** asm-level arch divergence (`sx asm`) and
weak-memory ordering *semantics* — Stream-C stress harness territory, not the corpus.
## Kickoff prompt (A.0a — paste into a fresh session)
> Implement Stream A step A.0a (atomics lock commit) per `current/PLAN-ATOMICS.md`. Verify
> `zig build && zig build test` is green first. Then: (1) create
> `library/modules/std/atomic.sx` with the `Ordering` enum, `Atomic($T)` struct, and
> `atomic_load`/`atomic_store` `#builtin` decls; wire into `library/modules/std.sx`'s tail.
> (2) Add the `atomic_load`/`atomic_store` IR ops + `AtomicOrdering` + op structs in
> `src/ir/inst.zig`; handle them in every exhaustive `Op` switch the Zig build flags
> (print.zig, comptime_vm.zig reuse load/store, emit_llvm dispatch). (3) Add
> `tryLowerAtomicIntrinsic` in `src/ir/lower/call.zig` (recognize the two builtins, bake the
> const ordering literal into the op, loud-reject non-literal ordering AND non-scalar/bad-size
> `T`). (4) Make `emitAtomicLoad`/`emitAtomicStore` in `src/backend/llvm/ops.zig` BAIL loudly
> ("not yet implemented") this commit. (5) Add `examples/1700-atomics-load-store.sx`, seed the
> marker, capture the bail diagnostic as the locked snapshot, confirm `zig build test` green,
> commit. STOP — A.0b (real emission) is the next step. Do NOT implement emission in the same
> commit that adds the example.

741
current/PLAN-COMPILER-VM.md
View File

@@ -1,741 +0,0 @@
# PLAN — Comptime Bytecode VM + comptime memory (then re-home the compiler-API on it)
> **Direction change (2026-06-17).** The comptime compiler-API stream pivots off the
> **byte-weld**. The weld (sx structs whose layout is validated to mirror the
> compiler's Zig types) + the **serialization / marshaling** bridge at the call
> boundary is the wrong direction — it bolts a parallel layout regime and hand-built
> byte-copies onto a comptime value model that fundamentally isn't bytes. We strip it
> and build the right foundation: a **bytecode VM over byte-addressable
> memory**, where comptime values ARE native bytes (like runtime). On that base the
> compiler-API needs no weld, no validation, no marshaling — the compiler's own types
> are read/built directly as memory and its functions take/return real pointers.
>
> Supersedes the build order in `design/comptime-compiler-api.md` (kept for history).
> This is the active plan for the stream. Branch: `reify`.
## Why
`src/ir/interp.zig` is a tree-walking interpreter over the SSA IR that represents
every value as a tagged `Value` union (`int`, `float`, `aggregate: []const Value`,
`type_tag`, `heap_ptr`, …). Two consequences:
1. **Slow.** Per-value boxing in a tagged union; per-op `switch` over `Inst`; an
aggregate is a heap `[]const Value`, walked element-by-element.
2. **Not native memory.** A struct value is `[]const Value` (tagged unions), NOT the
struct's bytes. So a comptime `@ptrCast(*StructInfo)` reads the `Value` union's
memory, not a `StructInfo` — which forced the whole weld+marshal detour.
Make comptime values **native bytes in byte-addressable memory** and both problems dissolve:
structs/arrays/slices are their bytes at natural layout (no weld), the compiler's own
records are directly addressable (no marshal), and a bytecode loop over comptime memory is
fast.
## End state
- Comptime execution = a **bytecode VM** over a **byte-addressable memory** (real
host-allocated bytes; layout is **target-aware** via the type table's sizes). Values
are bytes at addresses plus a scalar register file. No tagged `Value` union.
- The comptime compiler-API: the compiler **exposes its real types + functions** to
comptime sx. sx reads/builds them as native memory and calls compiler functions by
pointer. No `abi(.zig)` weld, no `validateStructLayout`, no `register_struct`
field-by-field marshaling — gone.
- `declare`/`define`/`type_info` and `#compiler`/`BuildOptions` ride this one
mechanism; the bespoke interp arms are deleted.
- **ONE evaluator at the end — non-negotiable.** The legacy tagged-`Value` interpreter
(`interp.zig`) is **DELETED**. We do NOT ship both permanently. "Dual-path"
(a compiler-API fn with both a legacy `compiler_lib` handler AND a VM-native impl) and
the emit-time legacy fallback are **transitional only** — scaffolding while the VM
reaches parity at BOTH comptime sites (emit time AND lowering time). The flag
`-Dcomptime-flat` is the swap mechanism; once the VM runs everywhere with parity, the
flag, the fallback, and `interp.zig` all go. Any "VM-only at emit, legacy at lowering"
split is a waypoint, never the destination.
## Principles (hold at every step)
- **Green at every step.** `zig build && zig build test` pass after each sub-step. The
existing tagged-`Value` interpreter stays the live evaluator until the VM reaches
corpus parity; swap behind a build flag, then delete the old path.
- **Target-aware, not host-baked.** Flat-memory layout uses the type table's target
sizes (`pointer_size`, `typeSizeBytes`/offsets), NEVER host `@sizeOf`. This is what
keeps cross-compilation correct (the JIT-comptime alternative could not).
- **Sandboxed.** Flat-memory accesses are bounds-checked; step/call-depth budgets
remain; an OOB / bad access traps to a build-gating diagnostic with a source span —
never a compiler-process crash.
- **No silent fallbacks** (per CLAUDE.md): an unhandled op / shape bails loudly with a
named reason, never a zero/default that looks like success.
## Phases
### Phase 0 — Strip the weld / serialize / marshal machinery
Delete the wrong-direction code so the VM builds on a clean base. Pure removal +
corpus rebaseline; suite green.
- `src/ir/compiler_lib.zig`: the reflection (`weldStruct` / `bound_types` /
`FieldLayout` / `BoundType`), the layout validation (`validateStructLayout` /
`LayoutMismatch` / `SxField`). Decide the fate of the `bound_fns` host-call registry
(`intern`/`text_of` handlers) — it is likely subsumed by the VM's compiler-call path
in Phase 3, but `intern`/`text_of` may survive as the first such calls.
- `src/ir/lower/nominal.zig`: `validateWeldedStruct` + `weldedFieldOrderStr` + the
`sd.abi == .zig` validation call in `registerStructDecl`.
- `src/ir/interp.zig`: the `compiler_welded` dispatch branch.
- `src/backend/llvm/ops.zig`: the `emitCall` comptime-only gate keyed on
`compiler_welded` (re-derive the comptime-only guard from a non-weld signal if still
needed).
- Corpus: retire / convert the weld examples + diagnostics — `0625`, `0627` (welded
struct), `1183`, `1186` (weld-layout diagnostics), `1184`/`1185` (welded-fn). Keep
`0626` (`intern`/`text_of` round-trip) only if it survives the new call path.
- **Keep (re-evaluate in Phase 3), independent of the weld semantics:** the
`#library "compiler"` decl, the `abi(.x)` annotation + `extern <lib>` syntax, and the
`callconv → abi` unification. These are surface syntax that may still serve the
compiler-API; only the *weld semantics* are stripped here.
**Verification:** `zig build test` green with the weld machinery gone; the surviving
syntax still parses (parser unit tests).
### Phase 1 — Flat-memory value model (still IR-walking, no bytecode yet)
Introduce comptime memory and move comptime values onto it, **decoupled from bytecode** so
the value-model change is isolated. Each sub-step ports one op group and keeps the
corpus green; the OLD tagged path stays behind a build flag (`-Dcomptime-flat`) until
all groups land, then the shim is deleted.
1. **Machine + scalars.** A comptime memory region (host `[]u8`) with a stack (frames) +
bump-allocated heap, and a scalar register file. Port `int`/`float`/`bool`/`undef`
and arithmetic/compare/branch. Aggregates still go through a compat shim to the old
representation.
2. **Aggregates.** Structs/arrays/tuples laid out in comptime memory at **target** layout;
port `struct_init` / `struct_get` / `array` / `index_gep` to read/write bytes at
computed offsets.
3. **Slices / strings.** `{ptr, len}` fat pointers in comptime memory.
4. **Optionals / enums / tagged unions.** Tag + payload bytes.
5. **Pointers.** `alloca` / `store` / `load` / GEP unified onto comptime addresses; retire
`slot_ptr` / `heap_ptr` / `byte_ptr` in favor of comptime addresses.
6. **Closures.** Fn id + captured env materialized in comptime memory.
7. **Extern / host calls.** A struct arg is already bytes → pass its address; this
removes most of `marshalExternArg`.
8. **Reflection / minting.** `declare` / `define` / `type_info` read comptime
values; type-table mutation copies escaping data into compiler-owned memory at the
boundary (lifetime), as today.
**Verification:** with `-Dcomptime-flat` the full corpus (currently 692) is byte-for-
byte identical to the tagged path; then make the VM the default and delete the shim.
### Phase 2 — Bytecode
Compile a comptime function's IR → a compact bytecode and execute the bytecode instead
of walking `Inst`. Pure encoding/speed; semantics identical to Phase 1. Land at least a
minimal register-bytecode loop (the stream's stated goal is a *bytecode* VM); a
fragment cache is optional follow-up.
**Verification:** corpus identical to Phase 1; comptime throughput measurably improved
on a heavy-comptime micro-benchmark.
### Phase 1.final — host wiring (the remaining integration)
The wiring ENTRY POINT exists: `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a
comptime function entirely on the VM and returns a legacy `Value`, or `null` to fall
back. Unit-tested (pure `6*7` → 42; unsupported → null). Remaining to actually route the
host through it:
1. **Panic→error hardening (prerequisite).** `Machine.readWord`/`writeWord`/`bytes`
currently `assert` (debug panic) on null/OOB. For arbitrary host functions to be
safe, make them return `error.OutOfBounds` so a malformed run BAILS (→ null → legacy)
instead of crashing the compiler. Ripples through `readField`/`writeField`/slice
helpers (add `try`).
2. **Implicit context.** Host comptime functions may have `has_implicit_ctx` (param 0 =
`*Context`); the legacy `run` materializes a default ctx. The VM `run` does not — so
either materialize it too, or only route `tryEval` at funcs without implicit ctx.
3. **Wire one site** behind a flag/env (`SX_COMPTIME_FLAT`, → `-Dcomptime-flat` later):
the const-init fold in `emit_llvm.zig` `emitGlobals` (`result = tryEval(...) orelse
interp.call(...)`). Default off → corpus unaffected.
4. **Parity + coverage.** Run the corpus with the flag ON; results must be byte-identical
to legacy. Measure how many comptime evals the VM already handles; the bail `detail`s
name what to port next (tagged-union payload / any / closures / builtins).
5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge)
until the VM is the default and the legacy path is deleted.
**Status (2026-06-17): steps 1–4 DONE; step 5 = the next session.**
- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return
`error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of
asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through
`readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and
every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a
null-deref function → `tryEval` returns `null` (legacy fallback), not a panic.
- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a
conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with
`has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the
right size/align allocated in comptime memory, its address passed as arg 0. The common
const body never reads the ctx; a body that USES the allocator loads a fn from it and
`call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed:
handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed)
empirically confirms no divergence. (A body that read+branched on a null allocator fn
could in principle diverge; none does — parity is the guard.)
- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat)
tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT`
(a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`).
Default OFF → corpus unaffected (688 green).
- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed);
manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace
facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`,
printing HANDLED / fallback+reason per init).
- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.**
Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity
stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx
materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer
optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the
destination = none/empty; unit-test regression); curated libc MEMORY builtins on comptime
memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped,
`free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed, target-aware,
result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32
storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single —
`readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had
written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added).
- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage
**27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF.
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context (not a zeroed one): the implicit-ctx param is an
opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying
the CAllocator thunk func-refs) into comptime memory via a new recursive `layoutConst`.
With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays
reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect`
(decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body
that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` →
`context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → the VM's native comptime `malloc`. Unlocked `0606` (string global via
the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func`
(memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/
`struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by
`alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to
`.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s
rejection diagnostic byte-identical). Unit tests added (global_get, func_ref +
call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk
bottoms out at libc `malloc`, and the VM can't use a host pointer with comptime
load/store, so comptime `malloc` must allocate from comptime memory. The default context
lets the allocator PROTOCOL run; native `malloc` is its final step.
- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage
**31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF.
- **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`.
- **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set)
→ word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)`
shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 |
span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push`
/ `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions —
linked into the compiler — so the return-trace buffer the host reads is populated
identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now.
- **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):**
`readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a
stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently
hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded;
gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests
pass via 1037/1038 in the corpus).
- **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):**
`intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm
global call (`1654`, never comptime-evaluable). Every other measured corpus const-init
is handled on the VM.
At this point the comptime VM handles essentially the entire real comptime corpus
(scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls +
recursion, the implicit context + allocator protocol, globals, failables + return
traces). Phase 2 (bytecode) and Phase 3 (compiler-API on comptime memory) are the forward
work; flipping the VM to default + deleting the legacy path awaits those.
- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second
comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run <expr>;`) now
routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields
`.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed
(`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy
re-run double-push them — both sites now `sx_trace_clear()` right before the legacy
fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All
comptime evaluation now routes through the VM-with-fallback (uniform).
- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM
gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init`
reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test
-Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity
gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var;
default-built does not). This is the prerequisite to eventually making the VM default +
deleting the legacy path (which still awaits Phase 2/3 + broader confidence).
- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.**
`invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is
the safety boundary) via `Vm.callCompilerFn` — natively on comptime memory, NO legacy
`Interpreter`: `intern(s: string) -> StringId` reads the string bytes from comptime memory and
`internString`s into the (const-cast) table (pool-only, never touches type layout, so the
VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back
into comptime memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback
is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688**
both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next
compiler functions (`find_type`, `register_struct`, the reflection readers) are added the
same way — comptime pointer in, handle/pointer out, no marshaling.
**Phase 3 progress (2026-06-18):**
- **(P3.1) First read-only reflection readers — `find_type` + `type_field_count` (DONE).**
Two more `compiler`-library fns bound the same way as the `intern`/`text_of` seed
(added to `compiler_lib.bound_fns` AND `Vm.callCompilerFn`, native on comptime memory, no
marshaling). A **type handle is a plain `u32` `TypeId`** (exactly like `StringId`), so
both calls keep the seed's clean scalar shape — handle in, scalar out:
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`) and
`type_field_count(t: TypeId) -> i64` (a new `TypeTable.memberCount` query — struct/union/
tagged-union fields, enum variants, array/vector length — that BOTH the legacy handler
and the VM call, so the two paths can't drift). Example `0628` chains
`intern → find_type → type_field_count` and a not-found lookup, both folded at `#run`,
both VM-HANDLED natively (no fallback). Parity **689/689** (gate ON and OFF); VM unit test
added.
- **Decision (resolves the plan's `find_type → ?Type` sketch):** `find_type` returns a
NON-optional `TypeId`, using the codebase's dedicated `unresolved` (0) sentinel for
not-found — NOT an `?Type`. Rationale: a `Type` value resolves to `.any`
(`type_resolver.zig`), which the comptime VM does not represent; and an optional
return can't cross the legacy↔VM eval boundary (`regToValue` bridges only
word/string/struct/tuple). `unresolved` is the project-blessed unmistakable "no type"
marker (see CLAUDE.md REJECTED PATTERNS — a dedicated sentinel is the required shape),
so the caller checks the handle against 0. This keeps the reader a clean scalar mirror
of `intern`/`text_of` and defers `.any`/optional plumbing to when it's actually needed.
- **(P3.2) Field-level reflection readers — `type_nominal_name` + `type_field_name` +
`type_field_type` (DONE).** Three more readers on the same `TypeId`-handle shape (each
backed by a new `TypeTable` query that BOTH the legacy handler and the VM call, so no
drift): `type_nominal_name(t: TypeId) -> StringId` (`nominalName` — a named type's own
name; loud-bail for unnamed types), `type_field_name(t: TypeId, idx: i64) -> StringId`
(`memberName` — struct/union/tagged-union field, enum variant, named-tuple element), and
`type_field_type(t: TypeId, idx: i64) -> TypeId` (`memberType` — struct/tuple/array/vector
member type). All loud-bail on out-of-range idx / no-member (no silent default). These are
the first MULTI-ARG compiler fns (the VM's `callCompilerFn` now reads arg 1 = idx); added
`Vm.argHandle`/`argTypeId` helpers (range-checked u32/TypeId arg reads). Naming uses the
`type_*` family so nothing collides with the std metatype builtins (`field_name`/`type_name`
exist in `core.sx`). Example `0629` reflects `Pair { lo: Point; hi: Point }` — reads each
field name and the nominal name of a field's type, all folded at `#run`, all VM-HANDLED
natively. Parity **690/690** (gate ON and OFF); VM unit test added.
- **(P3.2b) Kind + enum-value readers — `type_kind` + `type_field_value` (DONE).** The last
two read-only readers the metatype's `type_info(T)` needs, completing the READ side: a
comptime sx fn can now fully reflect a struct/enum/tagged-union/tuple into data with no
`#builtin`. `type_kind(t: TypeId) -> i64` (`TypeTable.kindCode` — a stable, compiler-owned
discriminant: 0 other · 1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array ·
7 vector · 8 error_set; TOTAL — never bails, an unnamed/non-aggregate type reads `other`)
and `type_field_value(t: TypeId, idx: i64) -> i64` (`TypeTable.memberValue` — an enum
variant's explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for
a non-enum / out-of-range idx). Example `0630` reflects `Color`/`WindowFlags`(flags)/`Point`.
Parity **691/691** (gate ON and OFF); VM unit test added.
- **READ side now complete:** `find_type` + `type_kind` + `type_field_count` +
`type_field_name` + `type_field_type` + `type_nominal_name` + `type_field_value` cover
everything `reflectTypeInfo` reads.
- **(P3.3) WRITE side — `declare_type` + `pointer_to` + ONE kind-branching `register_type` (DONE).**
The mutating side is a SINGLE `register_type(handle, kind, members)` that branches on `kind`
IN THE COMPILER (subsuming `define`'s `defineStruct`/`defineEnum`/`defineTuple`), plus
`declare_type(name) -> Type` (forward handle) and `pointer_to(t) -> Type` (build `*T`
references). They take/return real `Type` values (matching meta.sx's declare/define).
- **Timing decision (per the user):** mint LAZILY at LOWERING time (single pass, NOT a
pre-emit phase, NOT two-pass) — the existing `runComptimeTypeFunc` path. So the write
side is **legacy-only** (`compiler_lib` handlers); the VM isn't wired at lowering time, so
no VM mirror is needed (the read-side readers stay dual-path for emit-time reflection). A
non-generic `-> Type` builder is now flagged `is_comptime` (`decl.zig`) so its dead body
permits the welded calls (the comptime-only gate).
- **Graph support:** forward `declare_type` handles + `pointer_to` express a
mutually-recursive A↔B graph (`*A`, `*B`, B-by-value) before bodies are filled.
`register_type` is **idempotent** — re-filling a nominal slot (same module reached via two
import edges) re-mints identically instead of erroring (`nominalIdent` reads identity from
any nominal kind). `kind` codes match `type_kind`: 1 struct · 2 enum (actual `.@"enum"`) ·
3 tagged_union · 4 tuple.
- **Two bugs fixed en route** (issue 0142): (a) a fully payloadless comptime-minted enum
was minted as an all-void `tagged_union` → `verifySizes` panic; now mints a real
`.@"enum"` (both `register_type` kind 2 AND the metatype `defineEnum`). (b) bare
`EnumType.variant` qualified construction of a payloadless variant wasn't supported (failed
for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
- Examples: `0631` (graph + actual-enum + reflection), `0632` (make_enum all-void),
`0633`/`0634`/`0635` (namespaced / bare / multi-edge import of a minted type), `0187`
(qualified variant construction). Parity 697/697 (gate ON and OFF); unit tests added.
- **Next (P3.4):** re-express `declare`/`define`/`type_info` as sx over the read+write
compiler-API and DELETE the bespoke interp arms — needs the VM hardened against malformed
lowering-time IR first (the metatype runs at lowering time), so either harden + wire the VM
there, or migrate the metatype onto the legacy compiler-API calls first. Decide when reached.
Phase 2 (bytecode) is the orthogonal speed work.
### Phase 3 — Compiler-API on comptime memory (resume the stream — no weld)
With native-byte comptime values, re-home the compiler-API:
- **Expose the compiler's real types.** Register the actual `types.zig` records
(`StructInfo`, `EnumInfo`, `Field`, …) into the comptime type table under sx-visible
names, with their **real (host) layout** — the type IS the compiler's, so there is
nothing to validate or keep in sync. (This is the projection that *replaces* the
weld's reflection — owned by the compiler, not declared in sx.)
- **Expose the compiler's functions.** `register_struct`, `find_type`, `intern`,
`text_of`, and the reflection readers operate on comptime pointers / handles
directly (no marshaling — the bytes already ARE the record).
- **Re-express** `declare` / `define` / `type_info` as sx over these; delete the
bespoke interp arms (`defineStruct` / `defineEnum` / `defineTuple` / `reflectTypeInfo`);
migrate `examples/0622` (struct), `0619`/`0620`/`0623` (enum/tuple).
- **Migrate `BuildOptions`** off `#compiler` onto this mechanism; **delete `#compiler`**.
**Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over
the exposed compiler-API; full corpus green.
### Phase 4 — Retire the legacy interp (the ONE-evaluator end state)
The metatype CONSTRUCTION + REFLECTION surface is VM-native (steps 7/8 — `0614`–`0624`,
`0632` all HANDLED). This phase moves EVERYTHING ELSE off `interp.zig` and deletes it.
**What the legacy interp is still used for (audited 2026-06-18) — five roles:**
| Role | Wired to VM? | Site |
|------|--------------|------|
| **A. Comptime folds** (type-fn / `::` const-init / `#run`) | ✅ VM + legacy fallback | `comptime.zig:530`, `emit_llvm.zig:871`/`971` |
| **B. `#insert` string eval** | ❌ legacy-only (VM wiring reverted — 0737 malformed-IR crash) | `comptime.zig:634` |
| **C. Post-link bundler** (`platform.bundle` — Info.plist/codesign/process/fs) | ❌ legacy-only | `core.zig:invokeByFuncId` ← `main.zig:769` |
| **D. `#compiler` hooks** (`compiler_call` — BuildOptions/bundling) | ❌ legacy-only; `Value`-based ABI | `compiler_hooks.zig`, `interp.zig:1130` |
| **E. Bail diagnostics** (`Interpreter.last_bail_*` statics) | n/a | `main.zig:464` |
Shared substrate everything traffics in: the **`Value`** tagged union (the
`regToValue`/`valueToReg` bridge + the hooks + `core.zig`) and the **host-FFI bridge**
(`host_ffi.zig` + `interp.callExtern` — dlsym + cdecl trampolines for real libc).
**DECISION (2026-06-18, user): UNIFY.** The VM gains a host-FFI escape + real-pointer
translation and runs BOTH sandboxed comptime folds AND the unsandboxed post-link bundler.
`interp.zig` is fully deleted — true ONE evaluator, two modes (sandboxed / host-effects).
**Remaining comptime-fold gaps** (full corpus fallback inventory — 15 examples; 1179/1180
are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan artifact):
`box_any`/`unbox_any` (6), `out`/print (2), `global_addr` (1), trace frames (1),
`compiler_call` (2 — role D).
**Sub-phases (dependency order; each its own session, both gates 697/0 after each):**
- **4A — finish comptime ops (small, parity-guarded).** Drive the fold fallback list to
empty except `compiler_call`:
- **4A.1** `box_any`/`unbox_any`. Word case = alloc 16B `{tag@0, value@8}`, tag =
`source_type.index()` (matches legacy comptime; note runtime `anyTag` normalizes
arbitrary-width ints), value via `writeField(source_type)` (so f32 etc. round-trip);
unbox = `readField(addr+8, target)`. Aggregate-Any payload needs the runtime
pointer-in-value-slot shape (`coerceToI64` alloca+ptrtoint) — implement or bail loudly.
- **4A.2** `out`/print → add a VM output buffer; flush through the same path as
`core.flushInterpOutput`.
- **4A.3** `global_addr` (address-of a global in comptime memory).
- **4A.4** trace frames (`sx_trace_*` / `interp_print_frames`).
- **4B — VM-native diagnostics (role E). MUST land before deleting legacy.** Today a VM
bail silently falls back; with legacy gone the VM bail IS the user-facing build-gating
diagnostic. Surface the VM's `detail`/span/file into what `main.zig` renders; turn
1179/1180-style bails into proper diagnostics. No diagnostic may regress.
- **4C — `#insert` on the VM (role B).** Re-wire `evalComptimeString` through `tryEval`;
the lowering-time-IR hardening that forced the 0737 revert is already in place. Verify
the `#insert` corpus parity.
- **4D — host FFI on the VM (role D substrate). DONE.** Solved by a better allocator, not a
pin/tag scheme: the comptime memory is now an **arena** of stable host allocations and `Addr`
IS a real host pointer (`4D.0`, `625ba0f`), so a comptime pointer and an FFI-returned host
pointer are the same value — no translation, no realloc hazard. `Vm.callHostExtern`
(`4D.1`, `e7a8708`) dispatches ANY extern via `host_ffi` dlsym + trampolines (args/returns pass
untouched); `4D.2` (`6a7f690`) adds slice/string args (→ NUL-term `char*`) + float guards.
Examples 0636/0637. **(Superseded sub-note:** the earlier "pin the buffer / comptime↔host translate"
hazard is moot — the arena never moves an allocation.)
- **`#compiler` / `compiler_call` — DELETED, replaced by the `abi(.compiler)` ABI (decision 2026-06-18,
REVISED from the earlier `abi(.zig) extern compiler` shape).** A function is *compiler-domain* — it runs in
the comptime evaluator (VM/interp), NEVER in the shipped binary — because its **ABI says so**: `abi(.compiler)`.
No `extern <lib>`, no fake `#library "compiler"`. One annotation covers BOTH roles: (a) the **compiler-API
surface** (`intern`/`find_type`/`build_options`/`set_post_link_callback`/… — bodiless decls whose Zig/VM
handler is the impl, on `compiler_lib`'s export list, dispatched by `Vm.callCompilerFn`); (b) **user
compiler-domain functions** like post-link callbacks (`bundle_main` — BODIED `abi(.compiler)`, lowered for VM
eval but emit-skipped). The `#compiler` struct attribute + the `compiler_call` IR op + the `Value`-based hook
`Registry` (`compiler_hooks.zig`) all **go away**. **Why this is cleaner than the welded-fn approach:** the
former runtime-call enforcement blocker (a `build_options()` call inside an LLVM-emitted callback body) is
MOOT — a compiler-domain function is never emitted, so its compiler-API calls never reach `emitCall`.
**Staged build (each its own step, both gates green):**
- **S1+S2 — DONE (2026-06-18):** introduced `abi(.compiler)`, REMOVED the `.zig` ABI + `abi(.zig) extern
compiler` + `#library "compiler"` (clean cutover, no legacy); migrated all compiler-API examples. The
binding now keys off `fd.abi == .compiler` (`decl.zig` `weldedCompilerFn`); a bodiless `abi(.compiler)`
decl lowers extern-like (declared-not-defined) with no implicit ctx. **700/0 both gates.**
- **S3 — DONE (2026-06-18):** emit_llvm skips BODIED `abi(.compiler)` function bodies. Added an
`is_compiler_domain` flag to the IR `Function`; a bodied `abi(.compiler)` function LOWERS its body (for VM
eval) + is flagged `is_comptime` but is NOT emitted (Pass 2 skip; declared external-linkage so the empty
decl verifies). KEY fix: a call to a comptime-only callee (compiler-API `compiler_welded` OR
`is_compiler_domain`) inside a dead comptime body now emits `undef` instead of a real `call` (`ops.zig`
`emitCall`) — the old `compiler_call` did this; without it an AOT link leaves an undefined `_double`/`_intern`
reference (this also fixed a pre-existing untested AOT breakage of the bodiless compiler-API examples).
`fnIsBodilessCompiler` distinguishes the API surface (declare-only) from a compiler-domain callback (lowered,
emit-skipped). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` const,
absent from the binary, JIT+AOT). **701/0 both gates.**
- **S4 — callback-param propagation: OPTIONAL / DEFERRED (ergonomics only).** Verified 2026-06-18: an
`abi(.compiler)` function is TYPE-compatible with a plain `() -> R` param (the ABI marks the *function* —
`is_compiler_domain` — not its *type*, which stays `() -> R` CC-default). So a callback that needs to be
compiler-domain just declares itself `abi(.compiler)` (S3) and passes to a plain param fine; auto-propagation
from an `abi(.compiler)` PARAM type is a nicety, not a prerequisite for S5. Skipped for now.
- **S5a — DONE (2026-06-18):** the corpus-covered slice. `build_options` + `set_post_link_callback` →
free `abi(.compiler)` functions (VM `callCompilerFn` arms + legacy `compiler_lib` handlers); **`BuildConfig`
threaded into the VM** via a `tryEval` param (the same one `main.zig` forwards — shared with 4E). `build.sx`
extracts `set_post_link_callback` from the `struct #compiler` as a free `ufcs` fn; `bundle_main` + the
platform registrars (`configure`) are `abi(.compiler)`. 37 examples' `.ir` snapshots regen'd (benign:
declaration renumber + `@str` suffix shift — every example imports build.sx via the prelude). Strict
`compiler_call` bails 6→2; 0602/0603/1604/1611 HANDLED. **701/0 both gates.**
- **S5b/S5c (port the ~37 hooks) — SUPERSEDED 2026-06-18 by the sx-driven build pipeline (below).**
Porting each `BuildOptions` accessor to an `abi(.compiler)` function that delegates to a `compiler_hooks`
hook just re-encodes sx-level logic (string setters/getters, `is_macos` triple-matching, list appends) as
compiler hooks. The hooks need NOTHING from the compiler except the `BuildConfig` state. So instead of 37
hooks, **drive the whole build pipeline from sx** (the logical end of "bundling lives in sx"). S5a stays as
a green intermediate; the sx-build-pipeline replaces `build_options`/`set_post_link_callback`/the whole
`#compiler` surface wholesale.
### Phase 5 — sx-driven build pipeline (replaces the BuildOptions hooks; decision 2026-06-18, user)
**The build pipeline becomes an sx program.** `BuildConfig` is plain sx data (an ordinary struct, sx-owned
end-to-end — no `#compiler`, no hooks, no shared Zig state, no weld/offset access). The compiler shrinks to
a few `abi(.compiler)` PRIMITIVES that take **explicit args** (so nothing is shared by memory), and an sx
`build()` driver orchestrates configure → emit → link → bundle. **Chosen boundary: Option B** — the compiler
keeps the proven Zig linker as a primitive; sx owns config + orchestration + bundle. (Option A — sx shells
`cc`/`ld` itself — is a later refinement once the per-target link-line logic is ported to sx.)
**File split (user decision 2026-06-19):** the low-level compiler-API PRIMITIVES live in
`library/modules/compiler.sx` (the comptime `compiler` library — renamed from the interim `std/build.sx`); the
default `build` IMPLEMENTATION (`default_build` + the `on_build` slot + the sx `BuildConfig`) lives in
`library/modules/build.sx` alongside the existing `BuildOptions` DSL. So `compiler.sx` = primitives, `build.sx` =
orchestration/default impl. **Build-callback fallibility was DROPPED (user 2026-06-19):** the primitives + the
build callback are NOT `-> !` — a failed action (e.g. `link`) BAILS on the VM (hard build error). So the shapes
below shed their `-> !`.
Shape (build-callback fallibility dropped 2026-06-19):
```sx
// library/modules/compiler.sx (the comptime `compiler` library — PRIMITIVES)
emit_object :: () -> string abi(.compiler); // emitted .o path (query)
link :: (objects: List(string), output: string, libraries: List(string),
frameworks: List(string), flags: List(string), target: string) abi(.compiler); // void; bails on failure
c_object_paths :: () -> List(string) abi(.compiler); // metadata queries
link_libraries :: () -> List(string) abi(.compiler);
// library/modules/build.sx (the build DSL — DEFAULT IMPLEMENTATION + slot)
BuildConfig :: struct { output: string; target: string; flags: List(string);
frameworks: List(string); bundle_path: string; bundle_id: string; ... }
default_build :: (config: BuildConfig) abi(.compiler) { // the default pipeline (void)
obj := emit_object(); objs := c_object_paths(); objs.append(obj);
link(objs, config.output, link_libraries(), config.frameworks, config.flags, config.target);
if config.bundle_path.len > 0 { bundle_app(config); } } // bundle_app = today's sx bundler
on_build : (BuildConfig) abi(.compiler) = default_build; // the override slot
// user overrides: build :: (config: BuildConfig) abi(.compiler) { ... } #run on_build = build;
```
The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfig` → read `on_build` → invoke
`on_build(config)` on the VM; a `raise` fails the build. Plain `sx run` fires none of it.
**Steps (each its own green step; depends on 4E first):**
- **P5.1 — 4E prereq — DONE (2026-06-19).** `core.invokeByFuncId` routes the post-link callback through the
**VM** (`comptime_vm.tryEval`), NO fallback (a side-effecting callback can't double-execute): a bail is a hard
build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). `BuildConfig` +
`import_sources` threaded in; `flushInterpOutput` deleted (VM `out` writes direct via host-FFI). Smoke test
`examples/1661-platform-post-link-vm-list` (AOT): a post-link callback GROWS a `List` (0141 — works on the VM,
bails on legacy with `struct_get`), so the build succeeds (exit 0) only via the VM. Non-empty callback `args`
rejected loudly (the `on_build(config)` arg-marshaling entry is P5.3). **702/0 both gates.**
- **P5.2 — primitives.** Split: the read-only **metadata queries are DONE (2026-06-19)** — `c_object_paths() ->
List(string)` + `link_libraries() -> List(string)` as `abi(.compiler)` fns (stdlib `library/modules/compiler.sx`),
serviced by `comptime_vm.callCompilerFn` over `BuildConfig` fields `main.zig` forwards; new VM `makeStringList`
builds the `List(string)` in comptime memory from the call's result type (`ins.ty` now threaded through
`invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both
gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits
the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all
three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER
DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard
build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks`
vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM
readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test
`1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output —
the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0.
- **P5.3 — `on_build` registrar — DONE (2026-06-19).** `on_build(cb)` registers the build callback
(`cb: (opt: BuildOptions) -> bool abi(.compiler)`); the compiler force-lowers + auto-invokes the well-known
`default_pipeline` when no override. (Implemented as a registrar, not an assignable slot — the opaque
`BuildOptions` handle is one word, so arg-passing needs no struct marshaling.)
- **P5.4 core — DONE (2026-06-19).** `default_pipeline` in `build.sx` drives the whole build; NO Zig
auto-emit/auto-link; `emit_object`/`link` are sx-called actions via the `BuildHooks` vtable;
`set_post_link_callback` deleted (all callers on `on_build`). Build-path auto-imports `modules/build.sx`.
703/0 both gates.
### THE FINAL DIRECTION (user, 2026-06-19): FULL MIGRATION — NO LEGACY LEFT.
**Decision: DROP gate-OFF entirely.** The VM becomes the SOLE comptime evaluator; `-Dcomptime-flat` is made
permanent then removed; `interp.zig` (the legacy tagged-`Value` `Interpreter`) is DELETED. There is no
dual-path, no legacy `compiler_lib` handler, no `regToValue`/`valueToReg` bridge, no VM→legacy fallback. We
migrate the BuildOptions surface DIRECTLY to VM-native `abi(.compiler)` arms (no legacy handler — there is no
legacy to handle). **All bundling + code signing for EVERY target lives in the sx `default_pipeline`.**
- **P5.5 — DONE (2026-06-19).** The 35 `BuildOptions :: struct #compiler` methods migrated to VM-native
`abi(.compiler)`: `BuildOptions :: struct { }` (opaque null-sentinel handle) + 35 free
`ufcs (self: BuildOptions, …) abi(.compiler)` decls in `build.sx`, serviced by a new
`comptime_vm.callBuildOptionFn` arm off `callCompilerFn` — **NO legacy `compiler_lib` handler** (names
registered in `bound_fns` with a single bailing stub only so `weldedCompilerFn` accepts them). Setters dupe the
arg string into the PERSISTENT `Vm.gpa` (the Compilation allocator — threaded into both `tryEval` and
`runBuildCallback` — NOT the per-eval VM arena) and write/append to the threaded `BuildConfig`; string getters
return the field (or `""`); bool getters compute from the triple (`predIsMacOS`/…); count/index getters read the
`BuildConfig` slices. **Dispatch routing (Option B):** a `#run`/const-init entry that directly calls a
compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with NO legacy fallback regardless of the
`-Dcomptime-flat` gate → gate-OFF stays green without a legacy BuildOptions handler. 5 `platform/bundle.sx`
getter-calling helpers marked `abi(.compiler)` (comptime-only bundler code). 37 `.ir` regenerated (string-pool
churn; behavior-identical, verified `.ir`-only). **703/0 BOTH gates.** BuildOptions `compiler_call` bails GONE
(1609/1614/1615 strict-clean); 1616 now bails on `shr` — a SEPARATE unported bitwise/shift VM gap
(`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`), to port FIRST in P5.6 (1616 is unpinned + can't JIT-run on
macOS regardless). Also swept the outdated "flat memory" terminology → "comptime/byte-addressable" (the VM is
arena-backed, `Addr` = real host pointer; flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
- **P5.6 — ALL bundling + code signing in `default_pipeline` (every target).** `default_pipeline` (or a
`bundle()` it calls, in `platform/bundle.sx`) performs, after `link`, the full per-target bundle when
`bundle_path()` is set — branching on `is_macos`/`is_ios_device`/`is_ios_simulator`/`is_android`:
- **macOS `.app`** — `Contents/{MacOS,Resources,Frameworks}`, `Info.plist`, embed `-framework` dylibs +
`install_name_tool` fixups, `codesign` (ad-hoc or with `codesign_identity`).
- **iOS device `.app`** — device slice, embedded `.mobileprovision` (`provisioning_profile`), entitlements,
`codesign` with the real identity; **iOS simulator `.app`** — sim slice, no provisioning, ad-hoc sign.
- **Android `.apk`** — `AndroidManifest.xml` (or `manifest_path` override), asset tree (`add_asset_dir`),
`#jni_main` Java → `javac` → `d8` → `classes.dex`, `aapt2` package, `zipalign`, `apksigner` with the
debug/`keystore_path` keystore.
All of it runs on the VM via the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI (the existing
`platform/bundle.sx` logic, now reading the VM-native accessors instead of `#compiler` hooks). The compiler
keeps ONLY the linker as a primitive (Option B). Remove the `--bundle`/`post_link_module` Zig shim — bundling
is `default_pipeline`'s job; CLI flags feed `BuildConfig` and `default_pipeline` branches on it.
- **P5.7 — DELETE all legacy.** Remove the `#compiler` attribute (parse + lower), the `compiler_call` IR op
(`inst.zig` + every switch arm + the `interp.zig:1130` dispatch), `compiler_hooks.zig`
(`HookFn`/`Registry`/all hooks). Make `-Dcomptime-flat` permanent (VM always) and **delete `interp.zig`**
(`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`/`last_bail_*`); drop the
`regToValue`/`valueToReg` bridge and the VM→legacy fallback in `emit_llvm` (`#run`/const-init) and
`comptime.zig` (type-fn / `#insert`) — a VM bail is now ALWAYS a build-gating diagnostic (4B wiring), never a
fallback. `core.invokeByFuncId` is already VM-only. Re-express `define`/`make_enum` as sx over the
compiler-API. Land the 0141 repro as a corpus test. Reconcile 1654 (asm-global at comptime) to the VM wording.
- **P5.8 — real-project validation (integration).** Build `~/projects/m3te` and `~/projects/distribution` with
the new pipeline end-to-end (their real bundle/codesign/target configs) — these are the acceptance test that
`default_pipeline` covers all targets. Fix gaps surfaced there. Add dedicated bundle smoke tests (min `.app` +
`.apk`) to the corpus (the bundler still has no `zig build test` coverage — the stream's top risk).
**End state:** ONE evaluator (the VM); ZERO legacy; the entire build — emit, link, and all bundling + code
signing for macOS/iOS-device/iOS-sim/Android — is sx in `default_pipeline`, overridable via `#run on_build(...)`.
The compiler is: parse → IR → codegen → invoke `on_build`/`default_pipeline` on the VM (which calls back into
the linker primitive). `m3te` + `distribution` build clean.
**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ;
FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F.
**Top risks:** (1) the bundler has no corpus guard (4E needs dedicated tests); (2) deleting
`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API (`abi(.compiler)`) touches the
whole build/bundle path — stage it behind real bundle builds; (3) S3's emit-skip relies on DCE dropping the
unreferenced compiler-domain declaration — verify no stray runtime reference keeps it alive (link error).
## Open questions (resolve as reached, record decisions here)
- **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed
records are host-laid-out; user comptime types are target-laid-out. The comptime
model must carry both regimes (a per-type ABI tag on layout queries). Confirm the
boundary where a comptime pointer to a compiler record is handed to host Zig code
uses host layout.
- **Exposing compiler types to sx.** Mechanism for projecting `types.zig` records into
the comptime type table with real offsets (the non-weld replacement) — a registry the
compiler owns, keyed by sx-visible name → real Zig type's layout + a host-call ABI.
- **Bytecode shape.** IR-derived vs a fresh ISA; register vs stack; fragment caching.
- **Pointer escape / lifetime.** Flat-memory pointers stored into the persistent type
table must be copied into compiler-owned memory at the boundary (as today).
- **Old-path retirement.** Keep the tagged interpreter until Phase 1 parity, then
delete — confirm no non-comptime consumer depends on `Value`.
## File map (current → touched)
| Area | File | Phase |
|------|------|-------|
| Comptime evaluator | `src/ir/interp.zig` | 0 (strip weld dispatch), 1–2 (rebuild) |
| Weld registry | `src/ir/compiler_lib.zig` | 0 (strip), 3 (replace with type/fn exposure) |
| Weld validation | `src/ir/lower/nominal.zig` | 0 (strip `validateWeldedStruct`) |
| Comptime-only gate | `src/backend/llvm/ops.zig` | 0 (re-derive without weld signal) |
| Host-FFI marshalling | `src/ir/host_ffi.zig` | 1 (struct-by-pointer trims it) |
| Metatype arms | `src/ir/interp.zig` (`defineStruct`/…/`reflectTypeInfo`) | 3 (delete, re-express in sx) |
| `#compiler` / BuildOptions | `library/modules/build.sx`, `src/ir/compiler_hooks.zig` | 3 (migrate, delete `#compiler`) |
| Surface syntax | `src/parser.zig`, `src/ast.zig` (`abi`/`extern`/`#library`) | kept; revisited Phase 3 |
## Status
- **Phase 0 — DONE (2026-06-17).** The struct-weld machinery is stripped:
`compiler_lib.zig` lost the type registry (`weldStruct`/`bound_types`/`BoundType`/
`FieldLayout`/`findType`/`SxField`/`LayoutMismatch`/`validateStructLayout`);
`nominal.zig` lost `validateWeldedStruct`/`weldedFieldOrderStr` + the
`sd.abi == .zig` call; the struct-weld unit tests + examples `0625`/`0627`/`1183`/
`1186` are removed. **Decision (recorded):** the `intern`/`text_of` function
host-call bridge is KEPT — it is a clean scalar dispatch (string→handle), not
weld/serialize/marshal, and is the seed Phase 3 grows the compiler-call path from.
So the `compiler_welded` dispatch (`interp.callExtern` is unchanged at HEAD — the
pre-branch in `call()`), `weldedCompilerFn` (decl.zig), the `emitCall` comptime-only
gate (ops.zig), and examples `0626`/`1184`/`1185` stay. The `#library`/`abi`/`extern`
SYNTAX stays. `zig build test` green (688 corpus, 0 failed; unit tests pass).
- **Phase 1 — in progress.**
- **Sub-step 1 — DONE.** `src/ir/comptime_vm.zig`: the comptime `Machine`
(linear byte memory + bump/stack allocator with `mark`/`reset` reclamation +
scalar `readWord`/`writeWord` (1/2/4/8, little-endian) + `bytes` views; addr 0
reserved as `null_addr`) and `Frame` (register file indexed by Ref + stack
reclamation on `deinit`). A register `Reg` is a raw u64 — immediate scalar OR
`Addr`. Standalone + unit-tested (`comptime_vm.test.zig`, in the barrel); does
NOT touch the live interpreter, so the corpus stays green (688). No op execution
yet.
- **Sub-step 2 — DONE.** The executor (`Vm` in `comptime_vm.zig`): walks the SAME
IR `Inst` over comptime frames, mirroring the legacy interp's scalar semantics
(i64 wrapping/signed + f64 register words, keyed off the result/operand `TypeId`).
Ported: constants (`const_int`/`float`/`bool`/`null`/`undef`), arithmetic
(`add`/`sub`/`mul`/`div`/`mod`/`neg`), comparison (`cmp_*`), logical
(`bool_and`/`or`/`not`), conversions (`widen`/`narrow`/`bitcast` passthrough,
`int_to_float`/`float_to_int`), terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param` (branch args passed as Refs — the same frame persists, SSA-safe).
Any other op bails loudly (`error.Unsupported` + `detail = @tagName(op)`).
Unit-tested on hand-built IR (`Fb` builder): integer add, f64 arithmetic, cond_br
branch selection, a block-param loop summing i..1, div-by-zero + unsupported-op
bails. Corpus untouched (688 green) — the executor is exercised by unit tests only,
not yet wired to real comptime eval.
- **Sub-step 3 — DONE.** Memory + structs on comptime memory. `Vm` gained an optional
`table: *const TypeTable` (target-aware layout). Ported `alloca`/`load`/`store`
(over comptime addresses, `Store.val_ty` drives width) and `struct_init`/`struct_get`/
`struct_gep` (structs laid out at the table's natural offsets). The value model: a
`Kind.word` (scalar/pointer ≤8B) sits in a register; a `Kind.aggregate` (struct)
lives in comptime memory and its "value" IS its address (read returns the address,
write memcpys), so nested structs compose and `struct_gep` is just base+offset (no
field-pointer dance). `kindOf` bails loudly on the not-yet-ported types
(slice/string/any/optional/enum/array/tuple/…). The Addr-based value model survives
allocator realloc (offsets are stable; slices are only materialized transiently).
Unit-tested: struct_init+get round-trip, alloca+gep+store+load, nested-struct
aggregate copy + nested read. Corpus untouched (688 green).
- **Sub-step 4a — DONE.** Tuples + arrays. `kindOf` widened (`tuple`/`array` →
aggregate). Ported `tuple_init`/`tuple_get` (positional, `tupleFieldOffset`),
`index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over array/pointer/
many_pointer bases; slice/string bases bail), and `length` on an array value
(static `ArrayInfo.length`). Unit-tested: mixed tuple round-trip, `[3]i64`
gep/store + index_get sum (42), array `length` (3). 688 corpus green.
- **Sub-step 4b — DONE.** Slices + strings as `{ptr@0 (pointer_size), len@8 (i64)}`
fat pointers (`kindOf`: string/slice → aggregate). Ported `const_string` (materializes
text+NUL in comptime memory + a fat pointer), `length`/`data_ptr` (read len/ptr fields),
`array_to_slice`, `subslice`, indexing *through* a slice/string (`elemAddr` loads
`.ptr` first), and `str_eq`/`str_ne` (len+memcmp). Helpers `makeSlice`/`sliceLen`/
`sliceData`. Unit-tested: string length + str_eq/ne, array→slice + slice index +
slice length (23), array subslice (43). 688 corpus green.
- **Sub-step 4c — DONE (optionals + payloadless enums).** `kindOf`: `enum` → word;
`?T` → word if pointer-child (null==0) else `{T@0, i1@sizeof(T)}` aggregate. Ported
`optional_wrap`/`unwrap`/`has_value`/`coalesce` (with `optChildIsPtr`/`optHas`
helpers; `const_null` → `null_addr` reads as none), `enum_init` (payloadless: tag is
the value), `enum_tag` (payloadless/word). Unit-tested: non-pointer `?i64`
wrap/unwrap/coalesce (91), pointer `?*i64` null==0 (99), payloadless enum tag (11).
688 corpus green.
- **Sub-step 4d — partial (`addr_of`/`deref` DONE).** `addr_of` passes through (an
aggregate value already IS its address; a pointer is already an address — mirrors
the legacy); `deref` = `readField` through the pointer (`ins.ty` is the pointee).
Unit-tested (deref a `*i64` → 77; addr_of a struct value + field read → 80).
**Deferred to the wiring phase (intentionally, not ported blind):** tagged-union
payload (`enum_init` w/ payload, `enum_payload` — the legacy stores *untyped* Values
and `field_index` indexes payload sub-fields, not variants, so a byte model's
payload type is ambiguous without a real call site), `any` boxing, closures, and the
bitwise ops. These have subtleties best resolved against actual corpus cases — the
VM's loud `error.Unsupported` + `detail` will name exactly what each real eval needs.
- **Sub-step 1.5 — direct `call` DONE.** `Vm` gained `module: *const Module`
(resolves a callee `FuncId`) + a `depth`/`max_depth` recursion guard. `call`
marshals arg Refs → Reg words and recursively `run`s the callee; aggregate args/
results pass as their `Addr` over the SHARED comptime memory (no copy). **Stack-lifetime
change:** `Frame` no longer reclaims the machine on exit (a returned aggregate's
Addr would dangle) — a comptime eval's allocations live to `Vm.deinit`;
`Machine.mark`/`reset` stay for explicit use. Extern/builtin callees (no blocks)
bail loudly (1.5b). Unit-tested: direct call (`add(20,22)+100` → 142) and recursion
(`sum(0..n)` → 15/55). 688 corpus green.
- **Sub-step 1.5b — `Reg`↔`Value` boundary bridge DONE.** The builtin/`compiler_call`/
extern handlers are all coupled to the legacy `Interpreter` (e.g. `compiler_lib`
handlers take `*Interpreter`), so the VM can't call them directly — the wiring uses
WHOLE-FUNCTION fallback instead (VM runs pure functions; a bail re-runs the whole
eval in the legacy). That needs the boundary bridge: `valueToReg` (host `Value` arg →
VM `Reg`, materializing aggregates into comptime memory) + `regToValue` (VM result →
`Value`, deep-copied out). Covers scalars + strings + structs (other aggregate shapes
bail loudly; added as wiring surfaces them). Transitional — deleted once the VM owns
comptime end-to-end. Unit-tested with round-trips. 688 corpus green.
- **Then the wiring step** (below) — now unblocked.
### Decision (2026-06-17): pivot from blind op-porting to CALLS + hybrid wiring
The common leaf ops are ported (scalars, control flow, structs, tuples, arrays, slices,
strings, optionals, payloadless enums, deref/addr_of) and unit-tested. Continuing to
port the rarer ops (tagged-union payload, any, closures) in isolation risks subtle
bugs and has low signal. The higher-value path:
1. **Calls (sub-step 1.5)** — `call` (direct), then `call_builtin`/`compiler_call`. The
shared comptime memory makes aggregate args/results pass naturally (they're Addrs). The
one design point: **aggregate-return lifetime** — a callee's stack-reclaim would
dangle a returned struct Addr, so for comptime (bounded) the VM should stop
reclaiming per-frame and let the whole eval's allocations live until `Vm.deinit`
(keep `Machine.mark/reset` for explicit use; drop it from `Frame.deinit`).
2. **Hybrid wiring** — `-Dcomptime-flat` routes a comptime eval through the VM, falling
back to the legacy interp on `error.Unsupported`. This makes the VM run the REAL
corpus, proving parity incrementally and surfacing exactly which ops each real eval
needs — far better signal than more isolated unit tests.

207
current/PLAN-EXTERN-EXPORT.md
View File

@@ -1,207 +0,0 @@
# sx `extern` / `export` + `#foreign` retirement — Plan (FFI-linkage stream)
**One stream, two parts.** **Part A** adds `extern`/`export` (the linkage surface);
**Part B** migrates every `#foreign` onto it and purges `foreign` from the tree.
They are *one* plan: Part B can't start until Part A is a behavior-equivalent
superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant.
**Design rationale:** [design/inline-asm-design.md](../design/inline-asm-design.md) §II.2
(Deviation 6) + §II.10 #4 + the syntax evaluation.
**Decided syntax**
```sx
name :: (sig) -> Ret [callconv(.x)] [extern | export] [LIB] ["csym"] [;|{…}]; // functions
Name :: #objc_class("X") [extern | export] { … }; // aggregates (mirrors `struct #compiler`)
g : Type extern [LIB] ["csym"]; // extern global
```
- `extern` = import (no body, external linkage, C ABI, no sx ctx) — `#foreign`'s role.
- `export` = define **and** expose (body + external linkage + C ABI + no ctx) — **new**.
- `extern`/`export` imply `callconv(.c)`; write `callconv` only to override.
- Optional `LIB` (a `#library` alias) + `"csym"` rename mirror `#foreign LIB "csym"`,
so `extern` is a true `#foreign` **superset** (Gate A→B): carried on
`extern_lib`/`extern_name`. The `#library` declaration + build-flag linking
mechanism stays a separate axis — `extern` *references* a lib, it doesn't fold
in `#library` itself. (Revises the original "library fully separate" decision 4.)
> **END-STATE INVARIANT (hard requirement).** After this stream, `foreign` appears
> **nowhere** in the live tree — not the `#foreign` surface, and **not** internal
> identifiers. The extern AST is **not** named `foreign_expr`. Enforced by the
> Phase 9.4 grep gate. Scope today: 643 `foreign` lines / ~57 identifiers in `src/`
> + 28 in live docs — most of it the objc/jni **runtime-class** machinery.
**Naming constraint (so we can actually reach the invariant):** introduce
`extern`-named representations only — do **not** reuse or extend
`ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Carry extern/export on a new
`FnDecl.extern_export` modifier with a `;`/`{…}` body (so there is **no** `*_expr`
node for it) + `FnDecl.extern_lib`/`extern_name`; add `VarDecl.is_extern`/
`extern_lib`/`extern_name`. The IR is already extern-named (`Function.is_extern`,
`Builder.declareExtern`).
**Key finding (scopes Part A):** the IR + LLVM emit **already support everything** —
`Function.linkage` (`.external/.internal/.private`), `is_extern`, `call_conv`, and a
raw un-mangled symbol name are all emitted by `declareFunction`
(`emit_llvm.zig:1225-1300`). Part A is a **parser + lowering** job, no codegen change.
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior-lock).
`zig build && zig build test` after every step. Never regenerate snapshots while red.
---
# PART A — add `extern` / `export` (alongside `#foreign`)
## Phase 0 — tokens + parser plumbing
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | add `kw_extern`, `kw_export` (Tag enum + `StaticStringMap`, beside `kw_callconv` at `token.zig:45,282`); unit lex test | `src/token.zig` |
| 0.1 | lock | `parseOptionalExternExport()` (mirror `parseOptionalCallConv`, `parser.zig:3669`) + `ast.ExternExportModifier` enum + `FnDecl.extern_export` + `VarDecl.is_extern`/`extern_name` fields; **not yet consumed**; unit AST test | `src/parser.zig`, `src/ast.zig` |
## Phase 1 — `extern` (import; equivalent to lib-less `#foreign`)
| Step | Commit | What | Files |
|---|---|---|---|
| 1.0 | xfail | accept postfix `extern` after the callconv slot (`parser.zig:1950`); `examples/12xx-ffi-extern-fn.sx` extern-binds a libc symbol — red (lowering not wired) | `src/parser.zig` |
| 1.1 | green | lowering: `extern` ⇒ `is_extern`, `.external`, `callconv(.c)`, no ctx — route through `declareExtern` like a lib-less `#foreign` (anchors `decl.zig:1123,387,2110,2113`). Example green | `src/ir/lower/decl.zig` |
| 1.2 | green | optional `extern "csym"` rename + extern-global form `g : T extern;` (`parser.zig:425` path) | `src/parser.zig`, `src/ir/lower/decl.zig` |
## Phase 2 — `export` (define + expose; the NEW capability)
Fills the four export-gap conditions (all in `src/ir/lower/decl.zig`):
| Gap | Anchor | Fix |
|---|---|---|
| (i) linkage forced `.internal` | `:2382`, `:2514` | also `.external` when `extern_export == .export` |
| (ii) C ABI not promoted | `:2110` | also `.c` when `== .export` |
| (iii) no symbol-name override | `emit_llvm.zig:1226` raw name | parse optional `export "csym"`; map in the name map |
| (iv) ctx param not suppressed | `:387` `funcWantsImplicitCtx` | also suppress when `== .export` |
| Step | Commit | What | Files |
|---|---|---|---|
| 2.0 | xfail | multi-file test: an `export fn` called from a companion `.c` caller (same `XXXX-` prefix) — red (still internal) | `examples/12xx-ffi-export-fn.{sx,c}` + `expected/` |
| 2.1 | green | gaps (i),(ii),(iv): `export` ⇒ external + C-ABI + no-ctx on a **defined** fn (uses `beginFunction`, not `declareExtern`) | `src/ir/lower/decl.zig` |
| 2.2 | green | gap (iii): `export "csym"` symbol-name override | `src/parser.zig`, `src/ir/lower/decl.zig` |
## Phase 3 — aggregates (objc / jni runtime classes)
| Step | Commit | What | Files |
|---|---|---|---|
| 3.0 | xfail | `#objc_class("X") extern { … }` (import) + `… export { … }` (define) parse alongside legacy `#foreign #objc_class` | `src/parser.zig` (`tryParseForeignClassPrefix` :1305, `parseForeignClassDecl` :1369) |
| 3.1 | green | map postfix `extern`→reference, `export`→define+register; per-runtime tests (objc, jni) | `src/parser.zig`, `src/ir/lower/decl.zig`, `src/ir/lower/objc_class.zig` |
## Phase 4 — interplay, diagnostics, docs
`extern`+`callconv` stacking/redundancy; reject `extern`+`export` together;
`specs.md` documents `extern`/`export` (the three axes); `#foreign` still documented
until Part B cutover.
> **GATE A→B.** `extern`/`export` are a behavior-equivalent **superset** of
> `#foreign`. Lock with a unit test asserting `#foreign` and `extern` lower to
> identical IR for a sample fn / global / class. Do not start Part B before this.
---
# PART B — migrate `#foreign` → `extern`/`export`, then purge `foreign`
**Inventory (drives the batches):** `#foreign` = 466 uses. ~391 sx-code (308 fns
[207 lib / 196 rename], 75 classes [39 objc / 31 jni], 8 globals) + ~145 example
snapshots. 6 libs (`sqlib`98 `libc`61 `objc`22 `tlib`12 `raylib`7 `clib/pcaplib`3).
Hotspots: `vendors/sqlite`(98), `platform/{android,uikit,android_jni,sdl3}`,
`std/{socket,thread,fs,time}`, `ffi/{objc,raylib}`.
## Phase 5 — `#foreign` becomes an alias for `extern`
| Step | Commit | What | Files |
|---|---|---|---|
| 5.0 | lock | route the `#foreign` parser paths (`parser.zig:316,425,1305,1970`) to build the *same extern-named* AST as `extern`/`export`. Suite green, snapshots unchanged | `src/parser.zig` |
| 5.1 | lock | unit test: `#foreign` and `extern` produce identical IR (fn/global/class) | `src/ir/lower/decl.test.zig` |
## Phase 6 — migrate stdlib (behavior-preserving; snapshot diff must be EMPTY)
One commit per batch; rewrite `#foreign`→`extern` (fns/globals),
`#foreign #objc_class`→`#objc_class … extern`, defined classes → `… export`.
| Step | Batch | ~sites |
|---|---|---|
| 6.1 | `library/vendors/sqlite/` | 98 |
| 6.2 | `library/modules/platform/` (uikit/android/android_jni/sdl3) | ~95 |
| 6.3 | `library/modules/std/` (socket/thread/fs/time/process/…) | ~60 |
| 6.4 | `library/modules/ffi/` (objc/raylib/objc_block/…) | ~50 |
| 6.5 | remaining `library/` + `vendors/` | remainder |
## Phase 7 — migrate examples + issues (empty snapshot diff; review every diff)
| Step | Batch |
|---|---|
| 7.1 | `examples/12xx-ffi-*` (plain C) |
| 7.2 | `examples/13xx-ffi-objc-*` |
| 7.3 | `examples/14xx-ffi-jni-*` |
| 7.4 | `issues/*` repros + stragglers |
A non-empty diff ⇒ the alias wasn't behavior-equivalent — stop, fix Phase 5.
## Phase 8 — cutover
| Step | Commit | What |
|---|---|---|
| 8.0 | xfail | `examples/11xx-diagnostics-foreign-removed.sx` expects a "`#foreign` removed; use `extern`/`export`" diagnostic — still accepted (red) |
| 8.1 | green | parser hard-rejects `#foreign` (mirrors the variadic `name: ..T` cutover); `specs.md` drops `#foreign`, documents `extern`/`export` |
## Phase 9 — total `foreign` purge (the invariant)
`foreign` must not appear anywhere in the live tree, surface *or* internal. Each step
a mechanical, behavior-preserving rename commit (snapshots unchanged), small
per-file/subsystem commits — not one sweep.
| Step | What | Identifiers (count → new) |
|---|---|---|
| 9.0 | delete the surface | `hash_foreign`(11) + lexer entry + the 4 parse paths + the alias |
| 9.1 | rename **linkage** → `extern*` | `foreign_expr`(25) **eliminated** (folds into modifier) · `is_foreign`(39)→`is_extern` · `foreign_lib`/`foreign_name`→`extern_*` · `foreign_name_map`→`extern_name_map` · `callForeign`(8)→`callExtern` · `marshalForeignArg`→`marshalExternArg` · `is_foreign_c_api`(5)→`is_extern_c_api` · `dedupeForeignSymbol`→`dedupeExternSymbol` |
| 9.2 | rename **runtime-class** machinery → `runtime*` (decision 5) | `ForeignClassDecl`(65) · `ForeignMethodDecl`(31) · `ForeignClassMember`(20) · `ForeignFieldDecl`(15) · `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `foreign_path`(62) · `ForeignRuntime` · `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` · `findForeign{Method,Property}InChain` · `resolveForeign*` · `register*ForeignClass*` · `foreignClass*Type` · `*ForeignRefs` |
| 9.3 | purge **live docs** (28 lines) | `specs.md`/`readme.md`/`CLAUDE.md`: drop `#foreign`, document `extern`/`export`; fix file-roles + FFI/bundling notes |
| 9.4 | **acceptance gate** | `grep -rniE 'foreign' src/ library/ examples/ specs.md readme.md CLAUDE.md` → **0** |
---
## Open decisions
*Part A (ratified — recommendations stand):* 1. bare keywords (not `#extern`).
2. aggregate position postfix (`#objc_class(…) extern`, like `struct #compiler`).
3. `extern ⇒ callconv(.c)`. 4. **REVISED** (user, 2026-06-14): `extern` carries an
optional `LIB`+`"csym"` axis (`extern_lib`/`extern_name`), mirroring `#foreign LIB
"csym"`, so it's a true `#foreign` superset (Gate A→B). The `#library` declaration +
build-flag linking mechanism stays separate — `extern` references a lib, doesn't
fold in `#library`. (Was: "library fully separate / not on `extern`".)
*Part B:* 5. runtime-class rename target — **RATIFIED `Runtime*Class*`** (user, 2026-06-14;
it's the object-model axis, not linkage). 6. historical carve-out — **STILL OPEN** (user did
not confirm at the Part A milestone): keep `issues/*.md` (+ design-doc prose) as provenance &
gate only the live tree (recommended) vs purge everything. Confirm 6 before Phase 9.
## Relationship to ASM
`PLAN-ASM.md` Phase F (global asm) consumes `extern` (import the asm symbol) and
`export` (let asm call back into sx) — do it after **Part A Phase 2**.
---
## Kickoff prompt (paste into a fresh session to start Part A)
> Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint
> `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided
> syntax, Naming constraint, Key finding) and Part A; rationale is in
> `design/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4.
>
> **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix
> keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays
> untouched). Do NOT start Phase 2 (`export`) or Part B (migration).
>
> **Cadence (IMPASSIBLE):** no commit may both add a test and make it pass — lock
> behavior with a passing test, or land an xfail the next commit turns green.
> `zig build && zig build test` after every step.
>
> **Naming constraint (hard):** introduce only `extern`-named AST — do NOT reuse or
> extend `ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Use a new
> `FnDecl.extern_export` modifier (body `;` or `{…}`) and `VarDecl.is_extern`/
> `extern_name`. IR is already extern-named (`Function.is_extern`, `declareExtern`).
>
> Steps (commit after each; update the checkpoint each time):
> - 0.0 lock: `kw_extern`/`kw_export` tokens + map entries beside `kw_callconv`
> (`src/token.zig:45,282`) + unit lex test.
> - 0.1 lock: `parseOptionalExternExport()` (mirror `parseOptionalCallConv`,
> `parser.zig:3669`) + `ast.ExternExportModifier` + `FnDecl.extern_export` +
> `VarDecl.is_extern`/`extern_name` (parsed, unconsumed) + unit AST test.
> - 1.0 xfail: accept postfix `extern` after the callconv slot (`parser.zig:1950`);
> add `examples/12xx-ffi-extern-fn.sx` that extern-binds a libc symbol (red).
> - 1.1 green: in `src/ir/lower/decl.zig`, lower `extern` like a lib-less `#foreign`
> import — `is_extern`, `.external`, `callconv(.c)`, no ctx, via `declareExtern`
> (anchors :1123, :387, :2110, :2113). Example goes green.
> - 1.2 green: optional `extern "csym"` rename + extern-global `g : T extern;`
> (`parser.zig:425`).
>
> Stop at end of Phase 1. Verify: suite green; the `extern` libc binding runs;
> `#foreign` still works with no snapshot diffs. If you hit an unrelated compiler
> bug, follow the CLAUDE.md IMPASSIBLE RULE (file an issue, stop).

284
current/PLAN-FIBERS.md
View File

@@ -1,284 +0,0 @@
> **SUPERSEDED (2026-06-28).** The fiber-async layer described here — `sched.go` /
> `wait` / `cancel` over `Task($R)` (B1.4a) — was RETIRED and folded into the unified
> `context.io` async stack (`async` / `await` / `cancel` / `race`) in PLAN-IO-UNIFY
> Phase 5. See `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for the current
> design. The `Scheduler` ENGINE primitives this doc documents (swap_context, guarded
> mmap stacks, spawn / run / yield_now / suspend_self / wake / sleep / block_on_fd,
> virtual-time timers) REMAIN and are still accurate. Body below is a historical record.
# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
> **STATUS: ✅ COMPLETE.** The pure-sx M:1 async runtime is feature-complete end-to-end
> (`library/modules/std/sched.sx`, examples 1800–1817, suite 755/0): `abi(.naked)` context switch
> (aarch64 + x86_64/Win64), M:1 scheduler, suspending `go`/`wait`/`cancel`, deterministic virtual-time
> timers (`sleep`/`now_ms`), and real fd readiness via kqueue (`block_on_fd`). Five compiler bugs fixed
> en route (0151/0152/0153/0154/0156-P1/0157). Deferred non-blocking follow-ups: linux `epoll` twin,
> `Scheduler.deinit`, `Future(void)`/`timeout` (0150), `context.io`-routed async (M:N). Next carve:
> Stream B2 (channels / cancel / async stdlib). Historical step-status below.
>
> B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
> (`Io` interface + `async`/`await`/`cancel` over blocking `CBlockingIo`) ✅ · B1.3 (fiber
> runtime: naked `swap_context` + §10.7 stress gate + guarded `mmap` stacks, proven on aarch64
> AND x86_64/Win64) ✅ · **B1.5a (M:1 scheduler CORE — `std/sched.sx`: `spawn`/`yield_now`/
> `suspend_self`/`wake`/`run`) ✅** (fixed blocker 0154) · **B1.4a (suspending fiber-task async —
> `sched.go`/`wait`/`cancel` over `Task($R)`, nullary-thunk) ✅** (adversarially reviewed; fixed
> blockers 0156-Part1 + 0157 en route; locked `1813`).
> **B1.4b (deterministic virtual-time timers — sched.sleep/now_ms/timer-run) ✅** (reviewed; fixed a CRITICAL timer-vs-early-wake UAF; locked 1814/1815).
> **B1.4c (event-loop — real fd readiness via kqueue: `block_on_fd` + run-loop Mode 2) ✅** (reviewed; fixed a CRITICAL same-fd lost-wakeup hang; locked 1816). macOS only — linux epoll twin deferred.
> **B1.5 (end-to-end M:1 capstone — `go`/`wait`+`sleep`+scheduler, deterministic ordering) ✅** (locked 1817). **STREAM B1 COMPLETE.** Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
> deterministic timers live as `sched.*` methods (M:1), NOT routed through the erased `context.io` (avoids forcing sched.sx into every std consumer + the `_fib_tramp` dup-symbol
> trap); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready` stay reserved for M:N. Deferred:
> issue 0150 (`Future(void)`/`timeout`); 0156-Part2 (deferred `..` spread); the `::` callable-param
> feature.
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream B (§B1) + the
design-of-record [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md)
§4 (async), §7 steps 4–9, §8.1 (risks), §10 (testing). Progress in
[CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). Stream B2 (channels/cancel/stdlib) is a
separate carve ([PLAN-CHANNELS.md], when reached) and depends on this + atomics (✅).
**Goal:** the colorblind, stackful, **pure-sx** async runtime — fibers behind an `Io`
interface, an M:1 scheduler, blocking + deterministic-sim + event-loop `Io` impls. The
**compiler floor is small and net-new**: make `abi(.naked)` actually emit an LLVM `naked`
function (B1.0), and confirm/close the per-fiber `context` root (B1.1). **Everything
else — the context-switch asm, fiber bootstrap, `mmap` stacks, the scheduler, futures,
the `Io` vtables — is ordinary sx library code** (design §4, §4.4). The irreducible FFI
floor: the per-arch asm context-switch (in `.sx`), syscall `extern`s, and `mmap`.
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `18xx` concurrency. On an **unrelated** compiler bug → file
`issues/NNNN`, mark this checkpoint BLOCKED, STOP (CLAUDE.md). The in-session
worker-fix override (delegate a blocker to a worker) applies only with explicit user
authorization.
---
## Design (grounded against the tree)
### B1.0 — `abi(.naked)` codegen (the one genuinely net-new compiler piece in B1)
The design doc spells this `callconv(.naked)`; the **real sx surface is `abi(.naked)`** —
written in the postfix slot, `name :: (sig) -> Ret abi(.naked) { asm { … }; }` (cf.
`build_options :: () -> BuildOptions abi(.compiler);` in [build.sx:28](../library/modules/build.sx#L28)).
The sx-facing name is **`naked`** throughout (keyword, field `is_naked`, diagnostics) —
matching LLVM's `naked` attribute (the lowering mechanism) and the industry term
(Zig/Rust/GCC/Clang). The ABI variant was renamed `.pure → .naked`: "pure" universally
means *side-effect-free*, the opposite of a register-clobbering context switch.
**Grounding (verified — do not re-derive):**
- The `ABI` enum **already carries `.naked`** — `ABI = enum { default, c, compiler, naked }`
([ast.zig:142](../src/ast.zig#L142)), documented "naked function (inline asm
body), no calling-convention prologue/epilogue." So B1.0 is **NOT** "extend the enum."
- `.naked` is **inert today**: [type_resolver.zig:237](../src/ir/type_resolver.zig#L237)
maps `.compiler, .naked → .default` CC, and `emit_llvm` emits **no LLVM `naked`
attribute**. So the net-new work is exactly: **carry `abi == .naked` into the IR
`Function`, emit LLVM's `naked` attr, and skip the implicit-`Context` / prologue
lowering** so the body is just the asm block + its own `ret`.
- The IR `Function` struct ([inst.zig:605](../src/ir/inst.zig#L605)) carries `call_conv`
(default/c) + `is_compiler_domain`, but **no naked flag** — add one (`is_naked: bool`).
- Attribute API is in-tree: `nounwind` is set at
[emit_llvm.zig:1339](../src/ir/emit_llvm.zig#L1339) via
`LLVMGetEnumAttributeKindForName("nounwind", 8)` → `LLVMCreateEnumAttribute(ctx, id, 0)`
→ `LLVMAddAttributeAtIndex(func, func_idx_attr /* -1 */, attr)`. The LLVM `naked` attr
is the same shape: `LLVMGetEnumAttributeKindForName("naked", 5)`.
- The `.c` ABI **already skips the implicit ctx** at lowering — `lam.abi == .c` /
`fd.abi == .c` gates (closure.zig:171, [decl.zig:515](../src/ir/lower/decl.zig#L515)).
`.naked` must skip it **too** (a `.naked` fn gets no synthetic `__sx_ctx`, no stack frame,
no prologue — args arrive in ABI registers and are read directly from asm). The
implicit-return machinery (`lowerValueBody`) must also be bypassed: a `.naked` body has no
sx return (the asm rets itself), so lower its statements and cap the block with
`unreachable`.
- **Inline asm already works end-to-end** (lower→emit→JIT): aarch64
([examples/1645](../examples/1645-platform-asm-aarch64-add.sx)), x86_64
([examples/1651](../examples/1651-platform-asm-x86-syscall-write.sx)), global asm, JIT
([1653](../examples/1653-platform-asm-global-jit.sx)). `emitInlineAsm` /
`LLVMGetInlineAsm` at [ops.zig:915](../src/backend/llvm/ops.zig#L915). The `.naked` body
is a single asm block reusing this path.
**`.naked` ≠ `.c` (design §4.6 context-switch note):** a `.c` epilogue restores SP from the
frame; a context switch deliberately makes SP-in ≠ SP-out, so the `.c` epilogue would
restore from the *wrong* stack. `.naked` = no prologue/epilogue/frame — the asm emits its
own `ret`. This is *why* the switch must be `.naked`, not `.c`.
**Snapshot story (per the atomics precedent):** a `.naked` fn's *body is raw per-arch asm*
(it can't be portable — that's the point), while LLVM's `naked` attribute text is
arch-invariant. **B1.0a** (lock) needs only **one host example** locked to the emit bail —
the bail fires at the function level *before* any asm/instruction selection, so it is
host-independent (no `.build` target pin). **B1.0b** (green) adds emission, pins that
example aarch64 (`.build {"target": "aarch64-macos"}`, end-to-end on a matching host,
ir-only on a mismatch), and adds an x86_64 cross sibling — mirroring the existing asm
corpus split (1645 aarch64 / 1651 x86). The ir-only `.ir` (only producible once emission
lands in B1.0b) asserts the `naked` attribute + the asm body. State loudly: **the `.ir`
proves the `naked` keyword + asm emitted, NOT that any hand-written register save/restore
is correct** — that is the B1.3 switch-stress harness's job, never the corpus's.
### B1.1 — per-fiber `context` root (grounding says this is SMALL, likely library-only)
**Grounding (verified — closes the design doc's open sizing question):**
- `context` is an **implicit `*Context` parameter** (`__sx_ctx`, slot 0), threaded through
every default-conv sx call ([lower.zig:259](../src/ir/lower.zig#L259)) — **not raw TLS**.
Inside a function `current_ctx_ref = Ref.fromIndex(0)` (the param) → it **rides the fiber
stack frame for free**.
- `push Context.{…}` allocates the new `Context` with a **stack `alloca`** and rebinds
`current_ctx_ref` to that slot ([stmt.zig:1263](../src/ir/lower/stmt.zig#L1263)) — "No
global, no walk." So **push frames are fiber-local for free**.
- The **only shared root** is the `__sx_default_context` **global**, bound at
entry-points / `abi(.c)` fns *before any user code runs*
([decl.zig:2667](../src/ir/lower/decl.zig#L2667), :2815).
⇒ The design doc's "lower as swappable indirection, never raw TLS" guards a **non-problem**
(confirmed). The **real, now-sized** B1.1 work is purely a **library convention**: a
freshly-`spawn`ed fiber must take its root `Context` from the **spawner's snapshot** (passed
as the fiber-entry fn's `__sx_ctx` slot-0 arg by the spawn trampoline), **not** the
`__sx_default_context` global. That is sx-side (the trampoline already controls slot 0) —
**expected to be ZERO compiler change.** B1.1's first action is a probe confirming this; if
a fiber genuinely re-reads the global root mid-stack (it should not — entry binds once),
*then* and only then is there a compiler obligation. **Ground the probe before sizing any
compiler work.** Prerequisite of B1.3 (a fiber needs a valid root before it switches).
### B1.2–B1.5 — pure sx over the primitives (design §4)
- **B1.2 (A1):** `Io` interface + `context.io` + `Future` + `cancel()` — a protocol/vtable
threaded exactly like `Allocator` (which already lives at `Context` field 0; see
`allocViaContext` [call.zig:1214](../src/ir/lower/call.zig#L1214)). `Io` becomes another
`Context` field. No compiler change — protocols + context already carry it.
- **B1.3 (A2):** the fiber runtime — naked context-switch asm (per-arch), bootstrap, `mmap`
stacks **with mandatory guard pages**. All sx. **Highest corruption risk in the stream**
(§8.1.1) and **untestable by the deterministic `Io`** (which tests *scheduling*, not the
*switch*). Its **first deliverable, before the scheduler AND the deterministic `Io`**: a
standalone **2-fiber ping-pong switch-stress harness** (§10.7) — scribble every
callee-saved register + a stack canary before each suspend, deep/recursive chains, verify
all survive post-resume. This harness — not B1.4 — is A2's correctness gate.
- **B1.4 (A3):** `Io` impls in order **blocking → deterministic-sim (KEYSTONE) → event-loop**
(kqueue/epoll/io_uring). Build the deterministic `Io` right after blocking; **calibrate it
against blocking `Io`** before trusting it to gate everything async (§8.1.3, §10.7) — a
deterministic-but-wrong scheduler snapshots garbage. (Open, deferred: the event loop does
**not** yet cooperate with a platform UI run loop — CFRunLoop/ALooper; that's a §6
app-target gap, out of B1.)
- **B1.5 (A5·M:1):** the single-thread scheduler — validates the whole colorblind stack
end-to-end. `18xx` corpus runs under the deterministic `Io`, asserting a **program-emitted
ordering contract** (sequence markers), not raw interleaving, so scheduler-policy tweaks
don't churn every snapshot.
### Files the compiler floor touches (B1.0 only; B1.1–B1.5 are library + tests)
B1.0 (`.naked`) forces these plumbing sites:
- [ast.zig:142](../src/ast.zig#L142) — `ABI.naked` (exists; reference only).
- [inst.zig:605](../src/ir/inst.zig#L605) — add `is_naked: bool = false` to `Function`.
- [decl.zig](../src/ir/lower/decl.zig) — set `is_naked` from `fd.abi == .naked`; gate the
implicit-ctx off for `.naked` in `funcWantsImplicitCtx` (mirror the `.c` skip at
decl.zig:515) and bypass `lowerValueBody` for `.naked` bodies (lower statements + cap with
`unreachable`, in both body-lowering paths) — a `.naked` fn binds no ctx and has no sx
return.
- [type_resolver.zig:237](../src/ir/type_resolver.zig#L237) — leave CC `.default` (a `.naked`
fn-pointer type has no CC of its own; nakedness is a decl-level emit attribute).
- [emit_llvm.zig:402](../src/ir/emit_llvm.zig#L402) Pass 2 — **B1.0a:** bail loudly when
`func.is_naked` (build-gating). **B1.0b:** instead emit LLVM's `naked` attr (shape per
`nounwind` at emit_llvm.zig:1339) + the asm-only body (no prologue).
- Any `.op`/`Function`-field switch the Zig build flags — let the build tell you.
---
## Phases (xfail→green steps)
### B1.0 — `abi(.naked)` codegen — ✅ COMPLETE
- **B1.0a (lock) — ✅ DONE.** Carried `abi == .naked` into IR `Function.is_naked`; threaded
through `decl.zig` (`funcWantsImplicitCtx` skips `.naked` like `.c`; all body-lowering paths
bypass `lowerValueBody` for `.naked`, lowering the asm body + capping with `unreachable`) +
generic.zig + pack.zig; `emit_llvm` Pass 2 bailed loudly on `func.is_naked`. Locked by
`examples/1800-concurrency-naked-asm.sx` + the generic regression (review-found gap).
- **B1.0b (green) — ✅ DONE.** `emit_llvm` declaration pass adds LLVM `naked` + `noinline` +
`nounwind` for `func.is_naked` and skips `frame-pointer=all` (incompatible with a frameless
function); Pass 2 emits the body normally (`naked` ⇒ verbatim asm + own `ret`, no
prologue). `1800` pinned aarch64 → exit 42 + `.ir`; `1801-concurrency-naked-generic.sx`
(renamed from `-bail`) proves the generic path emits a naked body (exit 42);
`1802-concurrency-naked-asm-x86.sx` x86_64 cross sibling (ir-only here, `.ir` locks `naked`
+ `movl $42, %eax`). Unit test `emit: abi(.naked) function gets the naked attribute` asserts
`naked` present + `frame-pointer` absent. Suite green (724/0).
- **B1.0c (review-hardening) — ✅ DONE.** A param-bearing `.naked` fn emitted invalid LLVM
(loud verifier error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both
paths + generic.zig) so a naked fn's args stay in registers (read by the asm body) — this
*enables* B1.3's `swap_context(from, to)`. Locked by `1803-concurrency-naked-asm-param.sx`.
Pack `.naked` (variadic + naked, nonsensical) left unsupported → loud verifier error.
### B1.1 — per-fiber `context` root — ✅ COMPLETE (zero compiler change)
Probe confirmed the spawn convention works with ordinary language features: snapshot
`context` (`snap := context`), store it in a struct, and `push f.root { entry(args) }` from a
trampoline running under a different ambient context — the body reads the snapshot (via the
implicit slot-0 `*Context` param), not the ambient ctx, and `push` restores ambient on exit.
No path re-reads `__sx_default_context` mid-stack ⇒ **no compiler obligation**; this is a pure
library convention. Locked by `examples/1804-concurrency-context-snapshot.sx` (`fiber root:
42` / `ambient after: 99`). The design doc's "never raw TLS" guarded a non-problem.
### B1.2 — A1: `Io` interface + `context.io` + `Future` + `cancel()` API
Library-only. `Io` as a protocol added to `Context` (mirror `Allocator`). `Future`/`cancel`
API surface. xfail→green via an `18xx` example exercising the blocking `Io` default (real
suspend lands in B1.3). No compiler change expected; if a protocol-in-context gap appears,
file it.
### B1.3 — A2: fiber runtime (naked switch + bootstrap + guarded `mmap` stacks) — ✅ COMPLETE
- **B1.3a (switch-stress harness FIRST) — ✅** the §10.7 register/canary-survival gate (1807/1808),
validity proven by negative controls, adversarially reviewed.
- **B1.3b — ✅** fiber bootstrap + guarded `mmap` stacks (1809); the x86_64 sibling landed as Win64
on a real VM (1810, `0 0 P`). Switch proven on TWO arch/ABI pairs.
### B1.5a — M:1 scheduler CORE (`std/sched.sx`) — ✅ COMPLETE
The reusable scheduler wrapping `swap_context`: generic `Fiber`/`Scheduler`,
`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over guarded `mmap` stacks, one generic
`fib_dispatch` running any stored closure body. Adversarially reviewed + hardened; fixed blocker
bug 0154 (struct-field `null`/`---` over-store) en route. Locked by `1811` (round-robin) + `1812`
(suspend/wake). Built BEFORE the deterministic `Io` because FiberIo (B1.4a) needs it as substrate.
### B1.4a — suspending fiber-task async (`sched.go`/`wait`/`cancel`) — ✅ COMPLETE
`Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` in `sched.sx` (nullary-thunk;
self-contained). `go` spawns `work` as a fiber, `wait` parks the caller until it completes. Locked
by `1813`. Two compiler blockers fixed (0156-Part1, 0157) + adversarially reviewed/hardened.
### B1.4b/c — A3: `Io` impls (deterministic-sim KEYSTONE → event-loop)
Blocking exists (io.sx `CBlockingIo`). Next the deterministic-sim `Io`, **calibrated against
blocking** before any `18xx` test trusts it; then the event loop. The deterministic `Io` is the
test harness for *all* of B1.5 + Stream B2.
### B1.5 — A5: M:1 scheduler — ✅ COMPLETE
End-to-end validation of the colorblind stack. The `18xx` corpus asserts program-emitted ordering
contracts under the scheduler + deterministic timers; the capstone `1817` composes `go`/`wait` +
`sleep`/`now_ms` + the scheduler (three tasks complete in deadline order, deterministic sum). Stream
B1 is feature-complete.
---
## Gates
- **B1.0:** unit `emit_llvm.test.zig` (the `naked` attr present on a `.naked` fn); two
arch-gated examples (aarch64 + x86_64) run end-to-end on a matching host, ir-only on a
mismatch (assert `naked` + asm in `.ir`). **OUT of corpus scope, stated loudly:** the
*correctness* of any hand-written register save/restore — that's the B1.3 stress harness.
- **B1.1:** an `18xx` example locking context-carried-by-slot-0 behavior + a checkpoint note
on the spawn-trampoline convention.
- **B1.3:** the **switch-stress harness is A2's gate** (register/canary survival — §10.7),
NOT a run/snapshot test; plus arch-gated run tests.
- **B1.4:** deterministic `Io` **calibrated** against blocking `Io` (§8.1.3) before trusting
it; `18xx` under the deterministic `Io`.
- **B1.5:** `18xx` ordering-contract snapshots under the deterministic `Io`.
## Kickoff prompt (B1.0b — paste into a fresh session)
> Implement Stream B1 step **B1.0b** (`abi(.naked)` real emission) per
> `current/PLAN-FIBERS.md`. Verify `zig build && zig build test` is green first (B1.0a is
> already landed: `Function.is_naked` plumbed, `decl.zig` skips ctx + bypasses implicit-return
> for `.naked`, `emit_llvm` Pass 2 bails loudly, `examples/1800-concurrency-naked-asm.sx`
> locked to the bail). Then: (1) in `src/ir/emit_llvm.zig` Pass 2 (~line 402), REPLACE the
> `func.is_naked` bail with real emission — set LLVM's `naked` attribute on the function
> (`LLVMGetEnumAttributeKindForName("naked", 5)` → `LLVMCreateEnumAttribute(ctx, id, 0)` →
> `LLVMAddAttributeAtIndex(llvm_func, -1, attr)`; shape per the `nounwind` set at
> emit_llvm.zig:1339) and emit the `.naked` body as its asm block only, no prologue/epilogue
> (the body already lowers to the inline-asm op + an `unreachable` terminator). (2) Pin
> `examples/1800-concurrency-naked-asm.sx` aarch64 with a `.build` sidecar
> `{"target":"aarch64-macos"}`; on this aarch64 host it runs end-to-end (exit 42), capture
> `.ir` + regen (`-Dname=examples/1800-concurrency-naked-asm.sx -Dupdate-goldens`), review the
> diff (assert the `.ir` shows the `naked` attr + `mov x0, #42` / `ret`, NO stray error
> text). (3) Add `examples/1802-concurrency-naked-asm-x86.sx` (x86_64 body, `.build
> {"target":"x86_64-linux"}`, ir-only on this host — requires its `.ir`, now producible).
> (4) Add a unit test in `src/ir/emit_llvm.test.zig` asserting the `naked` attribute is
> present on an `abi(.naked)` function. Confirm `zig build test` green, commit. NOTE: the
> `.ir` proves the keyword + asm emitted, NOT register-save correctness (that's the B1.3
> switch-stress harness). If you hit an UNRELATED compiler bug, file `issues/NNNN`, mark
> `CHECKPOINT-FIBERS.md` BLOCKED, and STOP.

142
current/PLAN-METATYPE.md
View File

@@ -1,142 +0,0 @@
# PLAN-METATYPE — comptime type metaprogramming (`declare` / `define` + reflection)
## Goal
Comptime type metaprogramming with the smallest possible compiler surface:
- **`declare(name) -> Type`** — mint a NEW empty (undefined) nominal type NAMED
`name`, returned as a first-class `Type` handle. The compiler registers the
forward type at compile time, so the body can reference it (`*Name`).
- **`define(handle, info) -> Type`** — fill a declared handle's body from a
`TypeInfo` *value*, and return the handle (so the one-shot form chains).
- **`type_info($T) -> TypeInfo`** — reflect a type INTO data (the inverse of
`define`'s decode). *Done for enums* (`interp.zig:reflectTypeInfo`,
`examples/0619`); struct/tuple widening pending.
- **`field_type($T, i) -> Type`** — the i-th field / variant-payload / element
type of `$T`. *Done.*
These four `#builtin`s in `library/modules/std/meta.sx` are the **entire**
compiler surface. Every higher-level constructor is **plain sx built over
`declare`/`define`** — the compiler knows none of them by name:
```sx
// one-shot (non-recursive): declare + define chained, define returns the handle
T :: define(declare("T"), .enum(.{ variants = .[ … ] }));
// recursive: a ctor fn names the forward type via declare, references it as *Name
List :: make_list();
make_list :: () -> Type {
h := declare("List");
return define(h, .enum(.{ variants = .[
EnumVariant.{ name = "cons", payload = *List }, // self-reference
EnumVariant.{ name = "nil", payload = void } ] }));
}
// type-fns are ordinary sx (channel result types, etc.)
RecvResult :: ($T: Type) -> Type {
return define(declare("RecvResult"), .enum(.{ variants = .[
EnumVariant.{ name = "value", payload = T },
EnumVariant.{ name = "closed", payload = void } ] }));
}
```
This gates channel result types (`RecvResult($T)`) and `race`'s synthesized
tagged-union (design [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §7 step 3), and replaces a would-be `enum($T)` language feature.
## How it works (the locked design)
1. **Two comptime interp builtins.** `declare` mints an empty `tagged_union` slot
in the type table; `define` decodes the `TypeInfo` value (variant-name strings +
payload `Type`-tags) and completes the slot byte-identical to a source enum's
`buildEnumInfo` output, so it flows through enum codegen unmodified. The interp
mutates the type table via a `mint` handle the host sets (`setMintTable`).
2. **No syntactic constructor recognition.** A `::` binding or type-fn body that
calls a `Type`-returning fn is **comptime-evaluated** (`evalComptimeType`): the
expression runs through the interpreter, the `declare`/`define` builtins mint the
type, and the result `type_tag` is bound. `decl.zig` triggers on a non-generic
`-> Type` fn call; `instantiateTypeFunction` triggers on a type-fn body that
returns a `define(…)` call (or a bodied `-> Type` helper) — see
`generic.zig:returnExprMintsType`.
3. **Name on `declare`.** `declare("Name")` carries the name as a compile-time
string so `preregisterForwardTypes` (in `evalComptimeType`) can register the
forward type — and bind it as a type alias — BEFORE the body lowers. That's
what makes a `*Name` self-reference resolve (a `Name :: ctor()` decl makes
`Name` a const_decl author, so `*Name` resolves through the forward-ALIAS path;
the alias binding, not just the table registration, is what satisfies it). The
interp's `declare` returns the same slot by name; `define` fills it in place.
4. **Nominal identity** rides the existing type-fn mangled-name instantiation cache:
`RecvResult(i64)` at two sites memoizes to ONE `TypeId` (the body runs once;
`renameNominalType` re-keys the minted type to the mangled name).
5. **Comptime-only, JIT-free.** `declare`/`define` are interp ops; reaching them at
runtime / emit is a hard error.
6. **Undefined-until-defined.** `declare()` mints an undefined slot; *using* it
(construct / match / size) before its `define` is a loud diagnostic. A *pointer*
to an undefined slot (`*Self`) is fine — that's what self-reference needs.
## Key code anchors
- Builtins: `BuiltinId.declare` / `.define` (`src/ir/inst.zig`); lowering to
`callBuiltin` (`src/ir/lower/call.zig:tryLowerReflectionCall`); interp exec +
`defineEnum` + `decodeVariantElements` (`src/ir/interp.zig`); `mint` field +
`setMintTable`.
- Comptime evaluation: `evalComptimeType` / `renameNominalType`
(`src/ir/lower/comptime.zig`); decl trigger `fnReturnsTypeValue`
(`src/ir/lower/decl.zig`); type-fn trigger `returnExprMintsType` +
`instantiateTypeFunction` (`src/ir/lower/generic.zig`).
- Reflection: `field_type` → `fieldTypeOf` (`src/ir/lower/generic.zig`).
- Surface: `library/modules/std/meta.sx` (on-demand import — NOT the prelude, to
avoid shifting every `.ir` snapshot).
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior
lock). `zig build && zig build test` after every step. Never regenerate snapshots
while red. Examples: `06xx` (comptime), `11xx` (diagnostics).
## Status
- [x] `declare` / `define` comptime builtins + the `mint` plumbing.
- [x] Comptime evaluation of a `Type`-returning `::` RHS and type-fn body
(the only triggers; no constructor-name knowledge in the compiler).
- [x] Name-in-`TypeInfo`; nominal identity via the instantiation cache.
- [x] `field_type` reflection (`examples/0616`).
- [x] Examples green on the floor: `0614` (one-shot), `0615` (type-fn identity),
`0617` (channel result types).
- [x] **Self-reference** — recursive enums via `declare("Name")` + `*Name` in a
constructor fn (`preregisterForwardTypes` registers the forward type + alias
before the body lowers). `examples/0618` (recursive `*List`: construct, match
through the pointer, recursive traversal). Mutual recursion / by-value-self-ref
rejection fall out of the same mechanism (F5 adds the loud by-value check).
- [x] **`make_enum(name, variants: []EnumVariant)`** — the general enum constructor
over a COMPUTED (value, non-literal) variant list. Pure sx in `meta.sx`;
exercises `define` decoding a value-arg slice. `examples/0620` (array-literal
local) / `0624` (generic builder).
- [x] **Comptime slice over a non-string aggregate** — `arr[lo..hi]` over an array
yields a real slice value at comptime (`base_ty` threaded onto `Subslice`;
open-ended `hi` folded to the array's static length; `subsliceElements`).
`examples/0621`.
- [x] **`type_info($T) -> TypeInfo`** — reflect `enum`/`tagged_union`/`struct`/`tuple`
INTO a value (inverse of `define`'s decode); `define` decodes all three back
(`defineEnum`/`defineStruct`/`defineTuple`, dispatched on the TypeInfo tag).
Round-trips: `examples/0619` (enum) / `0622` (struct) / `0623` (tuple). The
reflect/construct triad is complete.
- [x] **Generic type-fn body locals** — a generic `($T) -> Type` comptime-evaluates
its FULL body (prelude statements + return), so a local before the return
resolves (`createComptimeFunctionWithPrelude` / `evalComptimeTypeBody`).
`examples/0624`.
- [x] **Validation + loud diagnostics** — by-value self-reference (`checkInfiniteSize`,
source `1178` + constructed `1182`; issue 0139), duplicate variant/field names
(`1180`), `declare()` never `define()`d (`1181`, was a `verifySizes` panic),
and the 0140 bail-surfacing (`1179`). use-before-define is subsumed by these
(no new check needed). Validation story COMPLETE.
- [ ] **Comptime `List` growth** (issue 0141, DEFERRED) — `List(T).append` at
comptime bails (two layers: null comptime allocator at scanDecls + `*T`
slot_ptr `struct_get`). Non-blocking; array-literal locals cover the use case.
## Risks / watch
- **Self-ref timing** — `define` for the two-statement form must complete before any
code uses the type's layout; a use-before-define must be a loud diagnostic, not a
silent empty enum.
- Keep `declare`/`define` **comptime-only**: reaching them at runtime is a hard error
(emit should bail loudly if one ever leaks into codegen).

156
current/PLAN-MULTIRET.md
View File

@@ -1,156 +0,0 @@
# PLAN-MULTIRET — bare-paren multi-value returns + named returns
## Why
sx already has multi-value returns, but only in a verbose spelling:
`-> Tuple(A, B)` / `-> Tuple(x: A, y: B)` types and `return .(a, b)` /
`return .(x = a, y = b)` tuple-literal returns. Destructuring (`a, b := f()`),
named/positional field access (`r.x` / `r.0`), and value-carrying failables
(`Tuple(A, B) !E`) all work on top of the existing `.tuple` TypeId.
The user wants the ergonomic, canonical surface:
```sx
a :: () -> () { } // () ≡ void
two :: () -> (i32, bool) { return 42, true; } // bare-paren type + bare comma return
b :: (f1: i32, f2: i32) -> (sum: i32, good: bool) { // named returns are in-scope locals
good = true;
sum = f1 + f2; // implicit return: all named slots set
}
b2 :: (f1: i32, f2: i32) -> (sum: i32, good: bool) {
return f1 + f2, f2 > 42; // bare comma return still works
}
read :: () -> (i32, bool, !) { ... } // error channel ALWAYS the last slot
```
Rules (from the user):
- **`() -> ()` ≡ `() -> void`.**
- **A multi-return signature is NOT a tuple — it just REUSES the tuple machinery.**
`-> (i32, bool)` / `-> (x: i32, y: bool)` mean "this function returns multiple
values", a DISTINCT thing from `-> Tuple(i32, bool)` (which returns one tuple
value). The bare-paren form is valid ONLY as a function/closure RETURN
signature — `x: (A, B)` (a variable/param/field annotation) stays REJECTED;
`Tuple(…)` is the spelling for an actual tuple value type.
- **Consumption — destructure OR single-bind (REVISED 2026-06-27).** A
multi-return result may be DESTRUCTURED (`s, g := b2()`) OR bound to a single
name and reached by field (`c := b2(); c.sum` / `c.0`). The earlier
destructure-only rule (single-bind = error) was REVERSED by the user — single
binding is allowed; the bound value behaves like a tuple of the value slots.
- **Failable: the error stays SEPARATE.** For `-> (sum, good, !)`, a bound
value (`c := f() catch …` / `try`) holds ONLY the value slots — the error
rides the `!` channel and is NEVER part of `c` (no `c.err`). This falls out of
the existing failable machinery (catch/try strip the error before binding).
- **Failable: the error channel is always the LAST slot** (`(A, B, !)`).
- **Bare comma return**: `return v1, v2;` maps positionally to the return slots —
no `.(…)` tuple literal needed.
- **Named returns are assignable locals.** With no explicit `return`, an implicit
return at end-of-body synthesizes the result from the named locals. **A named
return that is neither assigned on the path nor given a default is a COMPILE
ERROR.** A named slot may carry a default (`(sum: i32 = 0, good: bool)`); a
defaulted slot needn't be assigned.
## Representation (how "not a tuple, reuse machinery" is realized) — AS BUILT
- A dedicated AST node **`ReturnTypeExpr`** (`field_types` + optional
`field_names`, same shape as a tuple) is produced by the parser for a bare-paren
result list with **≥2 value slots** (`(A, B)`, `(x: A, y: B)`, `(A, B, !)`). A
single-value `(T, !)` stays a `tuple_type_expr` (a plain failable, `= -> T !`).
An EMPTY `()` parses to the `void` type.
- It resolves (type_resolver `internTupleLike`, shared with `tuple_type_expr`) to
a reused `.tuple` TypeId — full ABI / failable / destructure / field-access
machinery reuse. Its distinct MEANING lives in the AST node, not the TypeId.
- Position gating: the node is valid only in a return slot. `resolveParamType`
rejects a `ReturnTypeExpr` parameter annotation ("multi-return is return-only;
use Tuple(…)"). Being a distinct node, its mere appearance in a value-type
position is categorically an error (no flag to check) — exhaustive `switch`es
over `node.data` were forced to add a `.return_type_expr` arm (coverage).
- Consumption: destructure (`a, b := f()`) or single-bind + field access
(`c := f(); c.sum`). No single source of truth needed at call sites — the
result is just a tuple value.
- SCOPE: multi-return on `name :: (...) -> (…) { }` function declarations first.
Multi-return CLOSURE-TYPE values (`cb: Closure() -> (A, B)`) and lambda
literals are a later phase.
## What already exists (re-use, do NOT rebuild)
- `tuple_type_expr` → `.tuple` TypeId with optional `names` (type_resolver.zig
`resolveCompound`).
- Named + positional tuple field access `r.x` / `r.0` (expr.zig
`lowerFieldAccessOnType`).
- Destructuring `a, b := f()` (`DestructureDecl`, stmt.zig).
- Value-carrying failable assembly `(T1, …, !)` (error.zig
`lowerFailableSuccessReturn` / `emitTupleRet`) — error in the last slot.
- `return .(a, b)` / `return .(x = a, y = b)` tuple-literal returns (stmt.zig
`lowerReturn`).
- Generic inference through a failable/tuple closure return (this session's
parser `collectGenericNames` + generic.zig `extractTypeParam` tuple arms).
## Foundation already landed (uncommitted, suite-green)
- **parser.zig** — `collectGenericNames` descends tuple/optional/function nodes
(so `Closure() -> $R !` binds `$R`); the bare-paren result-list path builds a
failable `tuple_type_expr` when it ends in `!` (`(A, B, !)` parses).
- **generic.zig** — `extractTypeParam` / `matchTypeParam[Static]` handle the
`(value, !)` tuple so `$R` infers from a closure ARG's failable return.
## Phases (each: implement → lock with an example → `zig build test` green)
0. **`() -> ()` = void (parser).** Isolated, unambiguous. An empty `()` in the
paren type path resolves to `void`. Lock: `a :: () -> () { }`.
1. **Multi-return signatures `-> (A, B)` / `-> (x: A, y: B)` / `-> (A, B, !)`
(parser + AST + resolution).** Add the `multi_return_type` AST node; the parser
produces it for a bare-paren result list (return position). The return resolver
lowers it to a `.tuple` TypeId and sets `Function.multi_return`; the general
resolver rejects it (return-position only). Returns still use the existing
`return .(…)` literal in this phase (bare comma is Phase 2). Consumption is
destructuring `a, b := f()` (existing machinery). Lock: positional + named +
failable multi-return examples, each destructured.
2. **Destructure-only enforcement + bare comma `return v1, v2` (parser + lowering).**
(a) Reject using a multi-return call as a single value (`r := f()`, an arg, an
operand) — read `Function.multi_return` at the binding/use site; only
destructuring is allowed. (b) Extend the return statement to parse a
comma-separated value list and lower it to the same multi-slot return the
`.(…)` literal produces (error slot stays implicit for failables). Single-value
`return v` unchanged. Lock: `-> (i64, bool) { return 7, true; }`, a failable
variant, and a negative example (`r := f()` → diagnostic).
3. **Named-return locals + must-set rule (sema/lowering).** For a named return
`-> (x: A, y: B)`, bind each name as an in-scope assignable local (alloca). On
a path that reaches end-of-body with NO explicit `return`, synthesize the
implicit return from the named locals. Diagnose loudly if any named slot is
neither assigned on that path nor defaulted (no silent zero-fill). Explicit
`return v1, v2` / `return .(…)` still override. Lock: the
`b :: (...) -> (sum, good) { good = true; sum = ... }` example + a negative
example (unset slot → diagnostic).
4. **Named-return defaults `(sum: i32 = 0, good: bool)`.** A slot with a default
is exempt from the must-set rule; the default fills it at the implicit (or
partial explicit) return. Lock: an example mixing a defaulted + a required
slot.
## Open decisions (Decisions Log)
- **D1 — multi-return is NOT a tuple; return-position-only.** *Chosen* (user
directive). Realized via a distinct **`ReturnTypeExpr` AST node** (the user
preferred a dedicated node over a `TupleTypeExpr.is_multi_return` flag — it
makes "not a tuple" true at the AST level and makes position-gating
categorical) that resolves to a reused `.tuple` TypeId. A new `.tuple`-like
TypeInfo variant was rejected — it would ripple through every exhaustive type
switch for no ABI benefit. **Destructure-only was REVERSED** (see Rules):
single-binding a multi-return result is allowed (field access on the value
slots); the failable error stays on the separate `!` channel.
- **D2 (Phase 3) — storage for named-return locals.** Lean: an alloca per named
slot bound in the function scope under its name; the implicit return reads them
into the result tuple. Revisit if the must-set analysis wants SSA-style
definite-assignment instead of an alloca + per-path check.
- **D3 — multi-return closure-type values / lambda literals.** Deferred past the
function-decl phases (needs a `ClosureInfo.multi_return` flag). Phases 0–4 cover
named function declarations only.
## Validation (every phase)
- `zig build && zig build test` green (full corpus).
- New `examples/<category>/…` locked with snapshots; review the diff for `.ir`
churn only where expected (the prelude type table is untouched by this stream,
so churn should be minimal/none).
- Adversarial review of each phase before it lands.
## Category for examples
Multi-return is a core type/return feature — use the `types` block (`01xx`),
next free numbers, unless a better fit emerges.

145
current/PLAN-RACE.md
View File

@@ -1,145 +0,0 @@
> **SUPERSEDED (2026-06-28).** The `race` LOGIC described here shipped, then was
> RE-HOMED onto `*Future` + the `Io` protocol as `context.io.race` in PLAN-IO-UNIFY
> Phase 4. The Task-based `sched.race` over `*Task` documented below is RETIRED. See
> `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for the current design. The
> type-level machinery this doc documents (`RaceResult` / `make_variant` / tuple
> reflection) is UNCHANGED and still in use. Body below is a historical record.
# PLAN-RACE — Stream B2/A1: `race` over the M:1 fiber scheduler
Carved from the async roadmap ([../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md)
§4.5, §4.6, §7 step 3). The headline A1 feature still missing after Stream B1: **`race`** — start N
async tasks, return when the FIRST completes, and **structurally** cancel + join the losers before
returning. The result is a synthesized tagged-union mirroring the input named tuple's labels.
```sx
fa := s.go(() -> A => read_a(conn)); // *Task(A)
fb := s.go(() -> B => read_b(conn)); // *Task(B)
winner := s.race((a: fa, b: fb)); // RaceResult = enum { a: A; b: B }
if winner == {
case .a: (v) { handle_a(v); } // v : A (fb cancelled + joined)
case .b: (v) { handle_b(v); } // v : B (fa cancelled + joined)
}
// positional form: s.race((fa, fb)) → tags ._0 / ._1
```
## Design decisions (grounded against the tree, 2026-06-26)
- **Built over the M:1 `Task` layer, NOT `context.io`/`Future`.** The suspending async is
`sched.go`/`wait`/`cancel` over `*Task($R)` (B1.4a); `context.io.async`→`Future` is the BLOCKING
impl (workers run inline → racing is meaningless there). The roadmap's "Future" maps to our
`*Task`. `race` is a `Scheduler`/`Task` UFCS function in `library/modules/std/sched.sx`.
- **The result is a comptime-synthesized nominal tagged-union** (`RaceResult`), one variant per input
tuple element: variant NAME = the tuple label (positional → `_0`/`_1`), payload = the task's result
type. Synthesis uses the proven `declare`/`define`/`make_enum` + `field_count`/`field_name`/
`field_type` reflection (examples 0619–0623, 0646). The input arrives as an inferred type PARAMETER
`$T` (a named tuple of `*Task(..)`), which reflects correctly (issue 0195 fixed; the tuple-*alias*
gap is issue 0196, NOT on this path).
- **Cancellation rides the existing cooperative `Task.cancel`** (sets the flag + `.canceled` state).
`race` cancels every loser, then `wait`s each (joins) so no loser fiber outlives the `race` call —
structured. Reuses `suspend_self`/`wake`; no new scheduler machinery.
## The one net-new compiler primitive (step 1)
**`pointee($P: Type) -> Type #builtin`** — given a pointer type `*X`, return `X`. This is the only
missing reflection capability: `race` must project each tuple element `*Task(A)` to its result type
`A`, and there is currently NO way to get a pointer's target type at comptime (`field_count(*X)`=0,
`type_info` has no pointer variant). With it the projection is pure sx:
```sx
TaskResult :: ($P: Type) -> Type { // P = *Task(A) → A
return field_type(pointee(P), 0); // pointee → Task(A); field 0 = `value: A`
}
```
Small + generally useful (reflection is currently complete for aggregates but blind to pointer
targets). Mirror `field_type`'s `#builtin` plumbing (`src/ir/lower/call.zig` + `src/ir/calls.zig`),
backed by the pointer TypeInfo's pointee TypeId (`src/ir/types.zig`). Lock with a comptime example.
## Steps (each: implement → lock with an example → `zig build test` green → both platforms)
1. **`pointee` reflection builtin.** Add `pointee($P: Type) -> Type` (core.sx + compiler). Example:
`pointee(*i64)` = `i64`, `field_type(pointee(*Task(i64)), 0)` = the task value type. (worker+review)
2. **`RaceResult($T) -> Type` synthesis.** Type-fn: reflect the named-tuple `$T` of `*Task(..)`,
project each element via `TaskResult`, mint the tagged-union (labels → variants). Comptime-only
example asserting the minted type's `field_count`/`field_name` match the input tuple.
3. **`Task.Value` projection + result construction.** Confirm a winner's value can be boxed into the
minted variant by label/index (uses the existing variant-construction path).
4. **Runtime `race(tasks: $T) -> RaceResult(T)`.** Suspend the caller until the first task is
`.ready`; build the winner variant; then cancel + `wait`-join every loser before returning.
Single-winner (first by completion order; FIFO tiebreak). Example: 2 tasks, deterministic winner
via `sleep` ordering (like 1817), asserting the loser is cancelled + joined.
5. **Positional tuple form** (`._0`/`._1`) + edge cases (already-ready task → immediate, single-task
race, all-cancelled). Examples.
6. **Validate** every new example byte-identical on aarch64-macOS host AND aarch64-linux container;
full `zig build test` green; adversarially review each step.
## Status
Prereqs DONE (each committed + adversarially reviewed + suite-green):
- **issue 0195** (tuple/array/vector field reflection) fixed (`8ac6c573`). Tuple reflection works on
inline + `$T`-param forms. Issue 0196 (tuple *alias*) filed, not on the critical path.
- **`pointee($P) -> Type`** builtin added (`f1d29876`) — projects `*Task(A)` → `Task(A)`.
- **`field_count`/`size_of`/`align_of` fold as comptime constants** (`2a6ef398`) — so a generic
`($T) -> Type` builder can `inline for 0..field_count(T)` and size `[field_count(T)]EnumVariant`.
Verified: the variable-arity loop + array dim now work inside `RaceResult`.
- **comptime type-call composition** fixed (`eb18bbc6`) — a `field_type(...)`/`pointee(...)` result is
now usable as a `Type`-typed struct-field value, a generic `$P: Type` arg, and a nested type-call
arg (incl. with an `inline for` loop-var index). The **variable-arity `RaceResult` synthesis works
end-to-end** (proven by `examples/comptime/0649-comptime-typecall-composition.sx`: reflect a named
tuple of `*Box(..)` handles → mint a tagged-union with the tuple's labels, projecting `*Box(A)`→`A`).
- **`return` inside `inline if` fixed** (`84c2ae4f`) — the natural early-return-per-arm pattern (a
`return` in an `inline if`/comptime-`case` branch inside an `inline for`) no longer drops the
function's trailing statements. Lets `race` build the winner variant with a clean
`inline if i == { case 0: … else: … }` per-arm form.
- **`make_variant($E, idx, payload)`** added to `modules/std/meta.sx` (`1c26944e`) — the WRITE side of
the metatype triad: construct a minted tagged-union value by variant INDEX (the winner is chosen at
runtime; its label can't be a literal). Pure sx (writes the i64 tag@0 + payload@8). Verified for
complex payloads (struct / string / 40-byte struct). **This resolves the variant-construction gap.**
**GAP 1 — comptime-cursor indexing of a named-tuple VALUE — DONE** (`fee86adf`). `tasks[i]` with a
comptime cursor now reads the i-th element with its concrete type (option (a), a `structGet`).
**GAP 2 — surfaced during the runtime, all DONE** (`6a976287`): three compiler enablers the runtime
needed beyond the read path. (1) a named-tuple LITERAL passed directly as `$T` lost its element names
(`field_name` → ""), breaking `RaceResult`'s `make_enum`; (2) tuple-element L-VALUES by comptime index
(`tasks[i].waiter = …`) panicked at LLVM emit (an `index_gep` with `ptrTo(.unresolved)`); (3) a user
`($X) -> Type` call couldn't bind a `$E: Type` arg, blocking `make_variant(RaceResult(T), …)`. All
fixed + adversarially reviewed; OOB comptime tuple indices now diagnose loudly on every L-value path.
**Runtime in `sched.sx` — DONE** (`9099735e`).
- `RaceResult :: ($T) -> Type` over `*Task(..)` (the 0649 shape, with `Task`).
- `race :: ufcs (self: *Scheduler, tasks: $T) -> RaceResult(T)`: Phase 1 suspend until the FIRST
`.ready` (register waiter on all pending; on wake DEREGISTER from all; lowest-index winner). Phase 2
build the winner with `make_variant`. Phase 3 cancel + JOIN each loser one-at-a-time (only the joined
loser carries a waiter → no mid-join double-wake). Join rides a new `Task.finished` flag (set at the
end of the `go` body, checked before parking). Cooperative-cancel: a loser parked mid-`sleep` runs to
its natural end before `race` returns.
- Locked by `examples/concurrency/1821-concurrency-fiber-race.sx` (3 tasks i64/bool/f64, sleep 10/20/30,
shortest wins, losers cancelled + joined). Byte-identical on aarch64-macOS host AND aarch64-linux
container; full `zig build test` green (826/0).
**Remaining (step 5, future work):**
- POSITIONAL tuple form (`._0`/`._1`): `field_name` yields "" for an unnamed element → `make_enum`
rejects the duplicate. Needs a `_N` fallback in `RaceResult` (a comptime int→string) or a compiler
`field_name` default for positional tuples.
- Bare named-arg call form `s.race(a = ta, b = tb)` (no `.(…)` wrapper) — see the note below.
- Edge cases: already-ready-at-call (works today), all-cancelled (would deadlock-abort loudly).
- By-design caveat to document: `race` returns only when the SLOWEST loser finishes (cooperative
cancel can't preempt a mid-`sleep` loser; a loser blocked on `block_on_fd` that never fires blocks
the join forever — `cancel` doesn't unblock an fd waiter).
### Future work — bare named-arg call form `s.race(a = ta, b = tb)`
Today the result-tuple is passed as a named-tuple LITERAL: `s.race(.(a = ta, b = tb))`. The user asked
about dropping the `.(…)` so it reads `s.race(a = ta, b = tb)`. That is a CALL-SITE syntax/binding
feature, independent of the race runtime:
- The parser would have to accept `name = expr` call arguments and, for a `(tasks: $T)` parameter whose
type is inferred, MATERIALIZE them into a single named-tuple value bound to `$T` (rather than treating
each `name = expr` as a separate positional/keyword arg). Effectively "collect trailing `k = v` call
args into one anonymous named-tuple when the callee has a single inferred aggregate param".
- Risk: `name = expr` in call position currently has no meaning (or would collide with a future
keyword-argument feature). Decide whether `k = v` call args are (a) always tuple-materialized for an
inferred aggregate param, or (b) a dedicated `..` / sugar. Option (a) is the least new syntax.
- Once the call site produces the same named-tuple value, the entire race runtime is unchanged (it
already reflects `$T`). So this is purely front-end sugar — schedule it with the keyword-args work,
not the concurrency stream.