diff --git a/current/CHECKPOINT-ASM.md b/current/CHECKPOINT-ASM.md deleted file mode 100644 index c600c582..00000000 --- a/current/CHECKPOINT-ASM.md +++ /dev/null @@ -1,394 +0,0 @@ -# sx Inline Assembly — Checkpoint (ASM stream) - -Companion to `current/PLAN-ASM.md`; design in -[design/inline-asm-design.md](../design/inline-asm-design.md). Update after every -commit, one step at a time per the cadence rule (no commit may both add a test -and make it pass). - -## Last completed step -**G (indirect-memory `=*m` place outputs)** — the LAST substantive asm feature. -Unlike a write-through `=` output (which returns a value then stored), an -indirect output passes the place ADDRESS to the asm and the asm writes through -it — no return slot. `emitInlineAsm` (`src/backend/llvm/ops.zig`): indirect -outputs are excluded from the LLVM return type; their pointer is an opaque `ptr` -call arg placed **first** (arg-consuming constraint order = output-section -indirect pointers → inputs → read-write tied seeds); each gets an -`elementtype(T)` call-site attribute (required in the opaque-pointer era) via -`LLVMCreateTypeAttribute`/`LLVMAddCallSiteAttribute`; the store-back loop skips -them. New `asmIsIndirect(e, op)` helper. Lowering (`lowerAsmExpr`) stops -rejecting `*` (constraint kept verbatim, `=*m` reaches the constraint string -as-is). `asmOperandIndex` unchanged — indirect outputs still count as operands, -so `%[name]`→`${N}` holds. Verified by **running** on aarch64: store-through- -pointer (`str x9, %[out]` → 42, IR `"=*m,~{x9}"(ptr elementtype(i64) …)`) and a -mixed case (indirect + value output + input → `"=*m,=r,r"`, indirect ptr arg -first, `${0}/${1}/${2}` correct). Two commits per cadence: (1) -`examples/1652-platform-asm-indirect-mem.sx` locked the rejection; (2) implemented -+ flipped 1652 to a runnable aarch64-pinned example (`{ "target": "macos" }`, -ir-only elsewhere). `zig build test` green (661 corpus, 446 unit). Files: -`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1652-*`. - -Prior: **G (read-write `+` place outputs)** — a `+r` / `+{reg}` `-> @place` output is now -implemented. LLVM has no `+` constraint, so a -read-write place lowers to: an output **`=`** constraint (return slot, stored back -through the place after the call; the leading `+` rewritten to `=` in -`appendAsmConstraints`), **plus** a **tied input** (the decimal index of that -output) appended **after** the regular inputs, seeded with the place's loaded -value passed as a call arg. Tied inputs come **last** so existing operand indices -(`%[name]`→`${N}`) are undisturbed — `asmOperandIndex` unchanged. Lowering -(`lowerAsmExpr`) no longer rejects `+` (indirect `*` still rejected loudly). -`emitInlineAsm` (`src/backend/llvm/ops.zig`): grows arg/param arrays by the rw -count (`n_args = n_inputs + n_rw`), loads each seed (`asm.rw.seed`), emits the -tied constraint, and the existing store-back path writes the modified output back. -New `asmIsReadWrite(e, op)` helper. Verified by **running**: increment-in-place -(41→42, IR `"=r,0"`) and a mixed case (rw place + regular input + value output) → -textbook `"=r,=r,r,0"` with correct `${N}` indices and args `(input, seed)`. Two -commits per cadence: (1) `examples/1650-platform-asm-rw-place.sx` locked the -rejection; (2) implemented + flipped 1650 to a runnable aarch64-pinned example -(`{ "target": "macos" }`, ir-only elsewhere). `zig build test` green (658 corpus, -446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, -`examples/1650-*`. - -Prior: **2** — `-> @place` write-through outputs. An asm result can be **stored through -a place** (local / struct field) instead of returned; the place output does NOT -join the result tuple. Parser: `-> @place` parses the `@place` as an ordinary -address-of expression → an `out_place` operand (`src/parser.zig`). Lowering -(`lowerAsmExpr`): out_place operand = the lowered `@place` address, `out_ty` = -the pointee; read-write (`+`) and indirect-memory (`*`) constraints rejected -loudly (not yet implemented). Added `out_ty: TypeId` to the IR `AsmOperand` -(`src/ir/inst.zig`) so emit builds the **combined** return struct (ALL outputs). -`emitInlineAsm` rewrite (`src/backend/llvm/ops.zig`): the LLVM return type is now -built from every output's `out_ty`; after the call, out_place slots are -`store`d through their address and out_value slots rebuild the sx result — with a -**fast path** (no place outputs → the asm's struct return IS the result, so -pure-value asm IR is unchanged). Verified: write-to-local (`get42`→42), struct -field (`@p.b`), mixed value+place (`v=10 b=20`), `+` rejected. Locked with -`examples/1649-platform-asm-place-output.sx` (mixed, runs on aarch64). `zig build -test` green (657 corpus, 446 unit). Files: `src/parser.zig`, `src/ir/inst.zig`, -`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1649-*`. - -Prior: **F** — global (module-scope) asm. A top-level `asm { "tmpl", };` block (template -only) lowers to LLVM `module asm`, and a lib-less `extern` calls into the symbols -it defines. New `asm_global` AST node (`src/ast.zig`) + `parseAsmGlobal` -(`src/parser.zig`, dispatched from `parseTopLevel` on `kw_asm`) — rejects -`volatile` and any operands/clobbers. The node forced (and got) arms in the same -three `Node.Data` switches as `asm_expr` (`sema.zig` ×2, `semantic_diagnostics.zig`). -`Module` gains a `global_asm: ArrayList([]const u8)` (`src/ir/module.zig`); -`lowerMainAndComptime` captures each template (the dead `lowerDecls` is NOT the -top-level pass — `lowerRoot` Pass 2 uses `lowerMainAndComptime`); `emit_llvm.zig`'s -`emit()` appends each via `LLVMAppendModuleInlineAsm` (source order). Verified -end-to-end: an aarch64 `_my_add` global routine called via `extern` returns 42. -Locked with `examples/1648-platform-asm-global.sx` -(`.build { "aot": true, "target": "macos" }` → AOT build+run on aarch64, ir-only -elsewhere). `zig build test` green (656 corpus, 446 unit). **(Correction, later: -module asm ALSO runs under the JIT — `sx run` compiles to an in-memory object, -the integrated assembler assembles the `module asm` into it, ORC relocates and -runs it, so the symbol is resolvable at JIT main execution. The original "AOT -only" note was wrong; see 1653 for the JIT sibling. The genuine boundary is a -COMPILE-TIME `#run` call into a module-asm symbol, which fails loud via host -dlsym-miss — see 1654.)** Files: `src/ast.zig`, `src/parser.zig`, `src/sema.zig`, -`src/ir/semantic_diagnostics.zig`, `src/ir/module.zig`, `src/ir/lower/decl.zig`, -`src/ir/emit_llvm.zig`, `examples/1648-*`. - -Prior: **E** — multi-output tuples. **Inline asm now returns tuples.** Replaced the -N>1 bail with a shared `asmResultType` helper (`src/ir/lower/expr.zig`, mixed -into `Lowering`) that derives the result type from the `out_value` operands -(0→void, 1→T, N→named tuple, named via the §II.5 effective-name rule). The key -realization: `toLLVMType(tuple)` already produces a literal struct `{T1,…,Tn}` — -exactly LLVM's multi-output asm return — so **emit needed NO change**; building -the op with a tuple result type makes the asm call return the struct, which IS -sx's tuple value (destructured by the normal `tuple_get` path). `inferType`'s -`.asm_expr` arm now also delegates to `asmResultType` (single owner), so -`return asm`, `x := asm`, and `q, r := asm` all agree on the type. Verified -end-to-end on aarch64: `split(0x1234)`→`(lo=52, hi=18)`, a udiv/msub divmod→ -`(3, 2)`. IR is textbook: `call { i64, i64 } asm "divq ${4}", -"={rax},={rdx},{rax},{rdx},r,~{cc}"(…)` → extractvalue → tuple. Converted 1640 to -the x86_64 multi-output IR lock (ir-only) + added `1647-platform-asm-aarch64-multi` -(runs on aarch64). `zig build test` green (655 corpus, 446 unit). Files: -`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `src/ir/expr_typer.zig`, -`examples/164{0,7}-*`. - -Prior: **C.1 + D** — inline asm CODEGEN (lowering builds the op + LLVM emit). **Inline -assembly now runs end-to-end.** `lowerAsmExpr` (`src/ir/lower/expr.zig`) stops -bailing: it resolves each operand's effective name (§II.5 auto-naming), interns -template/constraints/clobbers, lowers input `Ref`s, derives the result `TypeId` -(0→void, 1→T), and builds the `inline_asm` op. Added a `%[name]`-references-a- -real-operand check (the last deferred validation). Multi-output (N>1) still bails -loudly ("Phase E"). `emitInlineAsm` (`src/backend/llvm/ops.zig`, port of Zig's -`airAssembly`): assembles the LLVM constraint string (outputs→inputs→`~{clobber}`, -`,`→`|`), rewrites the template (`%[name]`→`${N}`, `%%`→`%`, `$`→`$$`, `%=`→ -`${:uid}`), then `LLVMGetInlineAsm` + `LLVMBuildCall2` (AT&T). Dispatch wired -(`emit_llvm.zig`, replacing the C.0 `@panic`). **`llvm_shim.c`**: added -`LLVMInitializeNativeAsmParser()` — the JIT must assemble inline asm at run time. -Verified end-to-end: aarch64 `add`/`mov` run on the host (exit 42), `nop volatile` -runs (1642 now exit 0), IR is textbook (`call i64 asm "add ${0},${1},${2}", -"=r,r,r"(…)`). Locked with `examples/1645-platform-asm-aarch64-add.sx` (runs on -aarch64, ir-only elsewhere via `.build` + `.ir`). Also added the `inferType` -`.asm_expr` arm (`src/ir/expr_typer.zig`, 0→void / 1→T) — without it a bare -`x := asm {…-> T}` binding inferred `.unresolved` and silently produced 0; -regression-locked with `examples/1646-platform-asm-value-binding.sx`. Updated -1640 (now Phase-E bail) + 1642 (now runs). `zig build test` green (654 corpus, -446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, -`src/ir/emit_llvm.zig`, `src/ir/expr_typer.zig`, `llvm_shim.c`, -`examples/164{0,2,5,6}-*`. - -Prior: **C.0** — IR op `inline_asm` (lock; no behavior change). Added `inline_asm: -InlineAsm` to the IR `Op` union + the `InlineAsm` struct (`template: StringId`, -`operands: []const AsmOperand` {role/name/constraint/operand}, `clobbers: -[]const StringId`, `has_side_effects`) in `src/ir/inst.zig` — all strings -interned, operands in source order, result on `Inst.ty`. The new variant forced -(and got) arms in two exhaustive `Op` switches: `src/ir/interp.zig` (loud -`bailDetail` — inline asm is never comptime-evaluable) and `src/ir/print.zig` -(IR dump). `src/ir/emit_llvm.zig` gets a `@panic` **tripwire** — emit lands in -Phase D, and until then `lowerAsmExpr` still bails so no `inline_asm` op is ever -created (reaching emit would be a lowering-switched-over-too-early bug). Unit -test `inline_asm op shape` in `src/ir/inst.test.zig`. `zig build test` green -(652 corpus, 446 unit). Files: `src/ir/inst.zig`, `src/ir/interp.zig`, -`src/ir/print.zig`, `src/ir/emit_llvm.zig`, `src/ir/inst.test.zig`. - -Prior: **B.1** — operand-name validation (design §II.5 auto-naming rule). Extended -`lowerAsmExpr` with a `pinnedRegister(constraint)` helper (`"={eax}"`→`eax`, -`"+{rax}"`→`rax`, `"=r"`→null) and two checks: (1) **reject the echo form** -`[eax] "={eax}"` — a label identical to its own pinned register is redundant -(the operand is already auto-named after the register); (2) **reject duplicate -operand names** (ambiguous `%[name]` / result field). Locked with -`examples/1643-platform-asm-echo-name.sx` + `1644-platform-asm-duplicate-name.sx`. -`zig build test` green (652 corpus, 0 failed; 445 unit). Files: -`src/ir/lower/expr.zig`. - -Prior: **B.0** — asm shape validation (compile-path diagnostics). Restructured the -`.asm_expr` lowering arm into `lowerAsmExpr` (`src/ir/lower/expr.zig`, mixed into -`Lowering` in `src/ir/lower.zig`): it validates BEFORE the not-yet-implemented -codegen bail, so the user sees the real problem first. Two checklist items now -enforced with named diagnostics: (1) **template must be a compile-time-known -string** (`"..."` / `#string`); (2) **no value outputs ⇒ must be `volatile`** -(mirrors Zig — a result-less asm could be deleted). Valid shapes still bail with -the "codegen not yet implemented" message. Result-type derivation + auto-naming -stay deferred to a later step (observable only once Phase C produces a real IR -op). Locked with `examples/1641-platform-asm-missing-volatile.sx` (volatile -error) + `1642-platform-asm-nop-volatile.sx` (volatile no-output accepted → -codegen bail). `zig build test` green (650 corpus, 0 failed; 445 unit). Files: -`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `examples/164{1,2}-*`. - -Prior: **A.1** — parse `asm { … }` + loud lowering bail (folded A.1+A.2 into one honest -lock commit, since the loud bail IS current correct behavior — cadence option -(a)). Added `AsmExpr`/`AsmOperand` to `src/ast.zig` + the `asm_expr` `Node.Data` -arm; `parseAsmExpr` in `src/parser.zig` (`parsePrimary` `.kw_asm` dispatch) — -parses the template, flat operand list (`[name]? "constraint" -> Type` value -output / `= expr` input), and `clobbers(.…)`; `volatile`/`clobbers` recognized -contextually via `isContextualWord`. The new `asm_expr` tag forced (and got) -arms in three exhaustive `Node.Data` switches: `src/sema.zig` `analyzeNode` + -`findNodeAtOffset`, `src/ir/semantic_diagnostics.zig` `checkBindingNames` (all -recurse into template + operand payloads). Lowering bails LOUD + named in -`src/ir/lower/expr.zig` ("inline assembly codegen is not yet implemented…") via -an explicit `.asm_expr` arm (not the generic `unknown_expr` else) returning -`emitPlaceholder`. `-> @place` write-through is rejected with a clear "Phase 2" -parse error. Locked with `examples/1640-platform-asm-parse.sx` (multi-output -`divmod`, named operands, register pins, clobbers — parses then bails; called -from `main`). `zig build test` green (648 corpus, 0 failed; 445 unit). Files: -`src/ast.zig`, `src/parser.zig`, `src/sema.zig`, `src/ir/semantic_diagnostics.zig`, -`src/ir/lower/expr.zig`, `examples/1640-*`. - -Prior: **A.0** — `kw_asm` keyword (first compiler code). Added the `kw_asm` `Token.Tag` -variant + `.{ "asm", .kw_asm }` keyword-map entry in `src/token.zig`; `volatile` / -`clobbers` deliberately stay OUT of the global table (contextual). New exhaustive -`Tag` switch in `src/lsp/server.zig` `classifyToken` flagged the missing arm (the -intended coverage tripwire) — added `.kw_asm` to the keyword group. Lock test in -new `src/lexer.test.zig` (`asm`→`kw_asm`, `volatile`/`clobbers`→`identifier`), -wired into the `src/root.zig` barrel as `lexer_tests`. `zig build test` green (648 -corpus, 0 failed; 445 unit, 0 failed — +1). Files: `src/token.zig`, -`src/lexer.test.zig`, `src/root.zig`, `src/lsp/server.zig`. - -Prior: **0.2** — CLAUDE.md docs for `.build`; **Phase 0 COMPLETE**. -**0.1** — corpus runner **ir-only branch** for cross-target examples. Replaced -0.0's loud placeholder bail: when `cfg.target` doesn't match the host (`ir_only`), -`sweepRoot` skips run/build/exec and verifies via `sx ir --target` only — -asserting `.exit` (ir cmd) + `.ir` (normalized stdout) + `.stderr`, never -`.stdout` (write skipped in update mode, assertion skipped in verify mode). An -`.ir` snapshot is **required** in ir-only mode — its absence is a loud failure -("needs an .ir snapshot for ir-only mode"). Locked with -`examples/1639-platform-target-cross.sx` (asm-free `main :: () -> i64 { return 0; -}`), `.build` `{ "target": "x86_64-linux" }`, + checked-in `.ir`. Verified both -guards fire: corrupting the `.ir` → IR mismatch; deleting it → the require-failure. -`zig build test` green (647 corpus, 0 failed; 444 unit). Files: -`src/corpus_run.test.zig`, `examples/1639-*`. - -## Current state -**Inline assembly works end-to-end: 0, 1, and N value outputs (tuples).** Full -pipeline: lex (A.0) → parse (A.1) → validate (B.0/B.1 + `%[name]` check) → IR op -(C.0) → lower-builds-op + LLVM emit + JIT asm-parser init (C.1/D) → multi-output -tuples (E). Register-class + register-pinned operands, inputs, **symbol operands -(`"s"` → direct `bl`/`call` to a function/global by mangled name)**, clobbers, -`#string` multi-instruction templates, `%[name]`/`%%` rewriting, and the §II.5 -auto-naming rule all work and execute on the host JIT. Global `asm { … }` (Phase F) works via -lib-less `extern` under BOTH the JIT (`sx run` → 1653) and AOT (1648) — `sx run` -compiles to an object, so the integrated assembler bakes the `module asm` symbol -in and ORC resolves it. All three `-> @place` output forms now work and execute -on aarch64: **write-through** `=` (Phase 2), **read-write** `+` (tied input), and -**indirect-memory** `=*m` (pointer arg + `elementtype`, asm writes through it). -**Inline assembly is now feature-complete — no substantive features remain.** The -x86_64 syscall-write ir-only example is DONE (1651). Global asm runs under both -JIT (1653) and AOT (1648). `readme.md` now has an "Inline Assembly" section. - -Known orthogonal bug: **issue 0137** — `sx run` on a program with no `main` -segfaults (`src/target.zig:256-273`, unguarded JIT entry lookup). Pre-existing, -asm-independent; does NOT block the ASM stream (every example has a `main`). - -Phase E–F feasibility already confirmed against the live tree -(`LLVMGetInlineAsm` / `LLVMBuildCall2` / `LLVMAppendModuleInlineAsm` in LLVM@19 -`Core.h`; ERR-stream `extractvalue`→tuple in `emit_llvm.zig:726-927`; lib-less -`extern`, 60 sites; `--target` a global CLI flag). - -## Next step -**Inline assembly is feature-complete.** All substantive features are done: -0/1/N value outputs (tuples), register-class + pinned operands, inputs, clobbers, -`#string` templates, `%[name]`/`%%`/`$`/`%=` rewriting, §II.5 auto-naming, global -`asm { … }` (AOT), and all three `-> @place` output forms — write-through (`=`), -read-write (`+`), and indirect-memory (`=*m`). The x86_64 syscall-write ir-only -example (1651) and the output-to-`const` rejection (issue 0138) are also done. - -Global asm runs under BOTH the JIT (`sx run` → object → ORC; 1653) and AOT (1648) -— the earlier "AOT only / `sx run` mishandles module-asm" note was stale and has -been corrected. The one genuine boundary is a COMPILE-TIME `#run` into a -module-asm symbol: the interpreter resolves externs via host dlsym, the symbol -isn't linked yet, so it already fails loud (`comptime extern call: symbol not -found via dlsym`) — pinned by 1654. - -Remaining work, all **polish** (optional): -- None substantive. Possible niceties: tighten the `#run`-into-module-asm error - text to name module-asm specifically; broaden clobber validation to a checked - per-arch enum (design doc Phase 4). - -Orthogonal: **issue 0137** (no-`main` JIT segfault). - -Done since last: output-to-`const` rejection (issue 0138), x86_64 syscall-write -ir-only example (1651). - -Orthogonal: **issue 0137** (no-`main` segfault). - -## Log -- (init) Plan + design doc written; ASM stream opened. -- (0.0) Corpus runner target-gating: `.build` JSON config (replaces `.aot` - marker), `--target` threading, `hostMatchesTarget` execute-gate, loud - cross-target placeholder bail. Migrated 1226/1227 `.aot`→`.build`; locked with - 1638 fixture + unit tests. `zig build test` green. -- (0.1) ir-only branch: cross-target examples verify via `sx ir --target` only - (exit+ir+stderr, no stdout; `.ir` required). Locked with 1639 fixture; verified - corrupt-.ir → mismatch and missing-.ir → loud failure. `zig build test` green. -- (0.2) docs: CLAUDE.md documents `.build` JSON sidecar (aot + target + - ir-only gating), replacing stale `.aot` marker prose. **Phase 0 COMPLETE.** -- (A.0) `kw_asm` keyword in token.zig (+ map entry); LSP `classifyToken` switch - coverage; lock test in new `lexer.test.zig` (wired via root.zig). `volatile` / - `clobbers` stay contextual identifiers. `zig build test` green (445 unit, +1). -- (A.1) parse `asm { … }` → `AsmExpr` + loud lowering bail; `asm_expr` arms in 3 - exhaustive `Node.Data` switches; `-> @place` rejected (Phase 2). Adopted operand - auto-naming rule (design §II.5). Locked with 1640 fixture. Filed orthogonal - issue 0137 (no-`main` JIT segfault). `zig build test` green (648 corpus, 445 unit). -- (B.0) asm shape validation in `lowerAsmExpr`: comptime-string template + - no-output⇒volatile, with named diagnostics before the codegen bail. Locked with - 1641 (volatile error) + 1642 (volatile accepted). `zig build test` green (650 - corpus, 445 unit). -- (B.1) operand-name validation: `pinnedRegister` helper + reject echo form - (`[eax] "={eax}"`) and duplicate names. Locked with 1643 + 1644. `zig build - test` green (652 corpus, 445 unit). -- (C.0) IR op `inline_asm: InlineAsm` + interp `bailDetail` + print arm + emit - `@panic` tripwire (Phase D). No behavior change (lowering still bails). Unit - test `inline_asm op shape`. `zig build test` green (652 corpus, 446 unit). -- (C.1+D) CODEGEN — `lowerAsmExpr` builds the op (effective names, interned - strings, input Refs, 0/1 result type) + `%[name]` validation; `emitInlineAsm` - (constraint string + template rewrite + `LLVMGetInlineAsm`/`BuildCall2`, AT&T); - `inferType` arm; `LLVMInitializeNativeAsmParser` for the JIT. **Inline asm runs - end-to-end.** N>1 bails (Phase E). Locked with 1645 (aarch64 add, runs) + 1646 - (`:=` binding); updated 1640/1642. `zig build test` green (654 corpus, 446 unit). -- (E) multi-output tuples — `asmResultType` helper (0→void/1→T/N→named tuple), - shared by lowering + `inferType`. `toLLVMType(tuple)` == LLVM multi-output - struct, so emit unchanged; the asm struct return IS the sx tuple. Runs on - aarch64 (1647: `split`→`(lo,hi)`); 1640 → x86 multi-output IR lock (ir-only). - `zig build test` green (655 corpus, 446 unit). -- (F) global asm — `asm_global` AST node + `parseAsmGlobal` (top-level, rejects - volatile/operands); `Module.global_asm` captured in `lowerMainAndComptime`; - `emit()` appends via `LLVMAppendModuleInlineAsm`; call-into via lib-less - `extern`. AOT-verified (1648, `_my_add`→42). `zig build test` green (656 corpus). -- (docs) readme.md "Inline Assembly" section (b8800a2). -- (2) `-> @place` write-through — `out_place` operand; `out_ty` on the IR - AsmOperand; `emitInlineAsm` builds the combined output struct + splits - (out_place → store-through, out_value → result), fast-path when no places. - `+`/`*` rejected. Locked with 1649 (mixed, runs). `zig build test` green (657 - corpus, 446 unit). -- (G) read-write `+` place outputs — `+` lowers to an output `=` + a tied input - (output-index constraint) seeded with the place's loaded value, tied inputs - appended last (operand indices undisturbed). `appendAsmConstraints` rewrites - `+`→`=`; `emitInlineAsm` grows args by the rw count + loads seeds; - `asmIsReadWrite` helper. Lowering stops rejecting `+` (`*` still rejected). Two - commits (cadence): 1650 locked the rejection, then flipped to a runnable - aarch64 example (`"=r,0"` IR). `zig build test` green (658 corpus, 446 unit). -- (0138) output-to-`const` rejection — fixed the underlying general bug: scalar - `@const` (address-of a folded `::` constant) reinterpreted the value as a - pointer (`inttoptr`). `src/ir/lower/expr.zig` `.address_of` now diagnoses a - scalar const (local + module) instead of falling through; array/struct consts - keep storage. asm `-> @const` gets the clean diagnostic for free (same path). - Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`. Issue 0138 - RESOLVED. `zig build test` green (659 corpus, 446 unit). -- (x86 syscall) x86_64 Linux `write(2)` via raw `syscall` — locks the constraint - string `={rax},{rax},{rdi},{rsi},{rdx},~{rcx},~{r11},~{memory}` (register-pinned - inputs + pinned value output + pointer input + clobbers). ir-only on aarch64 - (`.ir` asserted), runs on x86_64-linux (hand-authored `"ok\n"` stdout). - `examples/1651-platform-asm-x86-syscall-write.sx`. Pure additive lock, no - compiler change. `zig build test` green (660 corpus, 446 unit). -- (G indirect) indirect-memory `=*m` place outputs — the place address is passed - as an opaque `ptr` arg (with an `elementtype(T)` call-site attr), placed before - inputs; asm writes through it; no return slot; store-back skips it. - `asmIsIndirect` helper; lowering stops rejecting `*`. Verified by running on - aarch64 (store-through → 42; mixed indirect+value+input → `"=*m,=r,r"`). Two - commits (cadence): 1652 locked the rejection, then flipped to a runnable aarch64 - example. **Inline asm now feature-complete.** `zig build test` green (661 corpus, - 446 unit). -- (jit) explored "asm in JIT": found it ALREADY works — `sx run` emits an - in-memory object (integrated assembler bakes in both in-function inline asm and - `module asm`), then ORC relocates+runs it. The stale "AOT only / `sx run` - mishandles module-asm" checkpoint prose was corrected. Locked global-asm-under- - JIT with `examples/1653-platform-asm-global-jit.sx` (`{ "target": "macos" }`, no - aot, → 42). `zig build test` green (662 corpus, 446 unit). -- (comptime guard) pinned the one genuine module-asm boundary: - `examples/1654-platform-asm-global-comptime-call.sx` — `#run` into a module-asm - symbol fails loud (`comptime extern call: symbol not found via dlsym`) because - the interpreter resolves externs via host dlsym before link. Arch-independent - (no `.build`). `zig build test` green (663 corpus, 446 unit). -- (round trip) `examples/1655-platform-asm-callback-into-sx.sx` — global-asm - trampoline that `bl _cb` back into an `export`ed sx function (sx→asm→sx, → 42). - Documented that `export` (external linkage + C symbol + C ABI) is what makes - the callback resolvable; `callconv(.c)` alone leaves it `internal` (DCE'd). - `zig build test` green (664 corpus, 446 unit). -- (symbol ops) symbol operands (`"s"`) — feed a function/global symbol; the - template emits its platform-mangled name so `bl %[fn]` is a DIRECT branch (one - fewer indirection than register-indirect `blr`, portable — no hardcoded `_`). - Emit passes the operand with its own llvm type (LLVMTypeOf), no coercion - (`asmIsSymbol` helper); lowering lowers the function RHS to `ptr @fn`. Decided - AGAINST mirroring Zig (which has no symbol operand — 483 std asm sites, none - call a function) because the direct `bl` matters. Two commits (cadence): 1656 - locked the rejection (replacing an LLVM-verifier crash), then implemented + - flipped to a runnable aarch64 example (objdump-confirmed direct `bl <_cb>`). - `zig build test` green (665 corpus, 446 unit). -- (x86 cross-arch) ir-only x86_64 siblings so each emit path is locked on BOTH - arches: 1657 read-write (`"incq ${0}","=r,0"`), 1658 indirect (`"movq $$42, - ${0}","=*m"`(ptr elementtype)), 1659 symbol (`"call ${2:P}"`, direct call). x86 - templates validated by cross-emitting an object (integrated assembler accepts; - objdump confirms 1659's direct `call` reloc). Pure additive locks. `zig build - test` green (668 corpus, 446 unit). -- (symbol portability) made `%[fn]` portable across arches — `renderAsmTemplate` - auto-injects LLVM's `:c` modifier (`${N}`→`${N:c}`) for symbol (`"s"`) operands - lacking an explicit modifier (`asmNamedIsSymbol` helper). Without it x86 renders - `$cb` (a bad `call` target needing a hand-written `:P`); aarch64 unaffected. - Verified `:c` ≡ `:P` for x86-64 calls (both → `R_X86_64_PLT32`). Explicit - `%[fn:X]` still wins (escape hatch). 1659 dropped its `:P` → same plain `%[fn]` - as aarch64 1656; both IRs regen to `${N:c}`. `zig build test` green (668 corpus, - 446 unit). - -## Known issues -- **0138** — RESOLVED. `@const` (address-of a `::` comptime constant) yielded a - wild pointer (`inttoptr (i64 to ptr)`). Fixed by diagnosing scalar - `@const` in `src/ir/lower/expr.zig` `.address_of` (no storage; array/struct - consts unaffected). Delivered the ASM "output-to-`const` rejection" for free. - Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`. -- **0137** — `sx run` on a program with no `main` segfaults (unguarded JIT entry - lookup, `src/target.zig:256-273`). Pre-existing, asm-independent. Filed - `issues/0137-jit-run-no-main-segfault.md`. Does not block A.1. diff --git a/current/CHECKPOINT-ATOMICS.md b/current/CHECKPOINT-ATOMICS.md deleted file mode 100644 index c859bbaf..00000000 --- a/current/CHECKPOINT-ATOMICS.md +++ /dev/null @@ -1,132 +0,0 @@ -# CHECKPOINT-ATOMICS — Stream A (atomics lowering) - -Companion to [PLAN-ATOMICS.md](PLAN-ATOMICS.md). Update after every step (one step at a -time, per the cadence rule). New corpus category: `17xx`. - -## Last completed step -**A.2 (CAS) — DONE** (A.2a lock + A.2b green). `compare_exchange`/`_weak` → LLVM `cmpxchg` -(result **`?T`, null = SUCCESS**; failure carries the actual value for retry). New IR op -`atomic_cmpxchg` + `AtomicCmpxchg{ptr, cmp, new, val_ty, success_ordering, failure_ordering, -weak}`. `emitAtomicCmpxchg`: `LLVMBuildAtomicCmpXchg` (success/failure orderings, singleThread=0) -→ `{T, i1}` pair; `LLVMSetWeak` for weak; `?T` result = `{ extractvalue 0 (actual), -xor(extractvalue 1, true) }` (has_value = NOT success). comptime_vm arm does real single-thread -CAS (read/compare/store-on-equal, build `?T`; weak == strong at comptime). Recognizer -(`atomic_cmpxchg`/`_weak`, 6 args) — CAS restricted to INTEGER T; BOTH orderings via -`atomicOrderingFromNode`; dual-ordering validation (failure may not be release/acq_rel nor -stronger than success, `atomicOrderingRank`). Methods `compare_exchange`/`_weak` on `Atomic($T)` -with comptime `$success`/`$failure: Ordering`. `examples/1702` green (CAS ok→20 / fail actual=20 / -weak retry loop 100→105); `examples/1186` locks a rejected ordering pair; unit test `emit: atomic -cmpxchg (strong + weak)` asserts `cmpxchg` + `cmpxchg weak`. Suite green (718/0). - -### A.1 (RMW) — DONE (A.1a lock + A.1b green) -`fetch_add/sub/and/or/xor` + `fetch_min/max` → LLVM `atomicrmw` (returns OLD value). New IR op -`atomic_rmw` + `RmwKind` (no `nand`); `LLVMBuildAtomicRMW` with binop from kind, signed/unsigned -`Min/Max` from `val_ty`. RMW restricted to INTEGER T (float fadd / pointer RMW out of scope, -rejected loudly); all five orderings valid for RMW. comptime_vm does real single-thread RMW. -`examples/1701` green; unit test locks `atomicrmw add` + signed `min` vs unsigned `umin`. - -## Next step -**A.3 — fence** (`atomic_fence($o: Ordering)` → LLVM `fence`), per PLAN-ATOMICS. No value -result; ordering must be acquire/release/acq_rel/seq_cst (relaxed is meaningless for a fence — -reject loudly). New IR op `atomic_fence` + dispatch/print/comptime_vm (no-op single-thread) + -`LLVMBuildFence`. Lock-then-green cadence as before. - -### Earlier — A.0c (guard hardening) -Adversarial review of A.0 found two CRITICAL silent-wrong defects (raw LLVM verifier errors -via the public intrinsics) + a latent align fallback; all fixed: scalar-kind allowlist + -per-op ordering validity (call.zig), `val_ty` align bail (ops.zig). Locked by examples -1130/1131. Suite green (713/0). - -### Earlier — A.0b (green) -Real atomic load/store emission: `LLVMBuildLoad2`/`LLVMBuildStore` -+ `LLVMSetOrdering` + mandatory `LLVMSetAlignment`, ordering via an explicit -sx-tag→`LLVMAtomicOrdering` switch (`llvmOrdering`). `examples/1700` green (7/42/43); IR -shows `load atomic i64, ptr … seq_cst, align 8` + `store atomic …`. Added unit test -`emit: atomic load/store (seq_cst, aligned)` in `emit_llvm.test.zig` (asserts `load -atomic`/`store atomic`/`seq_cst`/`align 8`). No fragile full-module `.ir` snapshot for 1700 -(it uses `print`); the unit test is the emission-shape gate. Suite green (710 + units). - -### Earlier — A.0a (lock commit) -Full atomic load/store plumbing with LLVM emission deliberately bailing loudly; -`examples/1700` locked to the bail diagnostic. -- `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (`init`/`load`/ - `store`, **seq_cst-only** — see capability gap below), `atomic_load`/`atomic_store` - `#builtin` decls. **Opt-in import**, NOT in the universal `std.sx` facade (mirrors - `trace`) — putting `Ordering` in the prelude grew every program's type table 378→380 and - churned 37 `.ir` snapshots; reverted. -- IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` (all 5) + structs (inst.zig); - print arms (print.zig); comptime_vm arms reuse load/store (single-thread correct); - recognizer `tryLowerAtomicIntrinsic` (call.zig) — const-ordering-literal guard + - scalar-size guard, both loud; emit dispatch arms (emit_llvm.zig) → `emitAtomicLoad`/ - `emitAtomicStore` (ops.zig) currently BAIL via `comptime_failed`. - -## A.0.5 — full ordering surface (DONE) -`Atomic($T).load($o: Ordering)` / `store(v, $o)` — ordering is a COMPTIME value param, -explicit (Rust-style, no default; design §4.6). `a.load(.acquire)` emits `load atomic … -acquire`; `a.store(v, .release)` emits `store atomic … release`; `a.load(.release)` is a -compile error (per-op validity guard fires through the method path). Recognizer -`atomicOrderingFromNode` now resolves a comptime-bound ordering identifier via -`comptimeIntNamed` (+ `atomicOrderingFromTag`, with the sx-Ordering ↔ IR-AtomicOrdering -declaration-order invariant documented). 1700 migrated to explicit orderings (output -unchanged 7/42/43). Suite green (715/0). - -**Unblocked by three comptime-value-param commits (workers):** enum (3c4305f), tagged_union -(d7a6857), generic-struct methods (d95ba0a). NOTE: default VALUES for comptime params on -generic-struct methods are NOT bound (orthogonal gap — free-fn defaults work); atomics -sidesteps it cleanly by requiring explicit ordering (matches the design). Candidate -follow-up, not an atomics blocker. - -## Known issues / capability gaps -- **RESOLVED:** comptime-constant ordering propagation — landed via comptime value params - (3c4305f / d7a6857 / d95ba0a); A.0.5 migrated the methods, no seq_cst-only legacy. -- **Orthogonal gap (not an atomics blocker):** default VALUES for comptime params don't bind - on generic-struct methods (free-fn defaults DO work). Atomics requires explicit ordering - (design-aligned), so it's unaffected. Candidate future fix. -- **Cosmetic:** an invalid ordering passed through a method (`a.load(.release)`) reports the - diagnostic at the lib forward site (`atomic.sx`), not the user's call. Loud + correct, but - the span could be improved by threading the call-site span. Polish. -- **Latent (observed, not yet filed):** calling an *unrecognized* bodiless `#builtin` - silently returns 0 / no-ops with exit 0 (that's how 1700 behaved before recognition - landed) — a silent-fallback footgun in the generic builtin-call path, independent of - atomics. Flag to user; candidate `issues/` entry. - -## Decisions (Stream A specifics; surface locked in design §4.6) -- `Atomic($T)` = pure-sx transparent 1-field struct (NO new IR type); ops = `#builtin` - intrinsics emitted as new IR ops. Minimal compiler surface. -- Ordering is compile-time-only (const enum literal), baked into the op as a Zig enum; - non-literal = loud diagnostic. sx tag → LLVM ordering via explicit switch (LLVM enum is - non-contiguous: 2/4/5/6/7). -- Atomic load/store REQUIRE explicit alignment (`LLVMSetAlignment`) — verifier mandate. -- Comptime VM treats atomics as ordinary load/store (single-thread ⇒ correct), not a bail. -- **Snapshot scope corrected:** `.ir` (LLVM IR) is arch-invariant for atomics → ONE host - `.ir` per op, not arch-gated x86/aarch64 pairs (they'd be byte-identical). Asm-level arch - divergence + weak-memory semantics are OUT of corpus scope (stress harness, Stream C). - -## Log -- **carve** — wrote PLAN-ATOMICS.md + CHECKPOINT-ATOMICS.md; grounded the intrinsic path, - switch sites, LLVM-C API (no `LLVMBuildAtomicLoad`; use `LLVMBuildLoad2`+`SetOrdering`+ - `SetAlignment`), and corrected the arch-`.ir` misconception (`sx ir` emits arch-invariant - LLVM IR). Stream ready; A.0a is the first implementation step. -- **A.0a** — landed lib (atomic.sx, opt-in import) + IR ops (atomic_load/atomic_store + - AtomicOrdering) + recognizer + print/vm arms + emit BAIL; locked `examples/1700` to the - bail diagnostic. Reverted a universal-facade wiring that churned 37 `.ir` snapshots - (Ordering would bloat every program's type table). Suite green (710/0). -- **A.0b** — real atomic load/store emission (LLVMBuildLoad2/Store + SetOrdering + - SetAlignment; explicit sx→LLVM ordering switch). 1700 green (7/42/43, `load atomic … - seq_cst, align 8`). Unit test added. Suite green (710 + units). -- **A.0c** — guard hardening from the adversarial review: scalar-kind allowlist + per-op - ordering validity (call.zig), val_ty align bail (ops.zig), + diagnostic examples - 1130/1131. Suite green (713/0). (comptime enum value params landed via worker 3c4305f.) -- **A.0.5** — full ordering surface: `Atomic($T).load/store($o: Ordering)` comptime ordering - (explicit). Recognizer resolves comptime-bound ordering via `comptimeIntNamed`. 1700 - migrated to explicit orderings (acquire/release/relaxed/seq_cst). Unblocked by - comptime-value-param workers (3c4305f/d7a6857/d95ba0a). Suite green (715/0). -- **A.1** — RMW: atomic_rmw op + RmwKind + recognizer (rmwKindFromName, integer-only) + - 7 fetch_* methods/intrinsics. A.1a bail-lock → A.1b real LLVMBuildAtomicRMW (signed|unsigned - min/max). comptime_vm real RMW. 1701 + unit test. Suite green (716/0). -- **A.2** — CAS: atomic_cmpxchg op + recognizer (dual-ordering validation) + emit (?T from - {actual,!success}) + comptime VM. compare_exchange/_weak methods. examples 1702 + 1186. - Review agent died; self-verified comptime↔runtime agreement, sub-8, ordering edges. - (Commits dca396e/79895be; A.2 has_value fix folded into A.3a.) -- **A.3** — swap (atomicrmw xchg) + fence (new atomic_fence op). A.3a bail-lock → A.3b real. - examples 1703/1704/1187 + unit test. Stream A feature-complete. Suite green (721/0). diff --git a/current/CHECKPOINT-COMPILER-API.md b/current/CHECKPOINT-COMPILER-API.md deleted file mode 100644 index 5b260930..00000000 --- a/current/CHECKPOINT-COMPILER-API.md +++ /dev/null @@ -1,1663 +0,0 @@ -# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`) - -Companion to the design-of-record -[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan -+ phased build order live there). This stream supersedes the metatype -`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute -with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step. - -## ⏯ Resume (fresh session) - -> **✅ P5.7 DONE (2026-06-19) — the comptime VM is the SOLE evaluator; ZERO legacy.** `interp.zig` (the legacy -> tagged-`Value` Interpreter) is DELETED; the `Value` result-DTO + `regToValue` materialization live in -> `comptime_value.zig` (the VM↔host boundary), `valueToReg` is gone. NO fallback anywhere — a VM bail is always a -> build-gating diagnostic (emit-time `#run`/const-init, lowering-time type-fn, `#insert`, inline comptime-call -> fold all run on the VM only). `#compiler` attribute + `compiler_call` IR op + `compiler_hooks` `Registry`/`HookFn` -> + all `hookXxx` are DELETED (`compiler_hooks.zig` keeps only `BuildConfig`/`BuildHooks`/`AssetDir`); `compiler_lib` -> is just the name registry now. The metatype `declare`/`define` are sx over the compiler-API (`declare_type`/ -> `register_type`); `type_info`/`field_type` stay builtins (documented). Bonus this arc: fixed issue 0141 (reject -> the silent `[*]T → []T` coercion; land the corrected List-grown form 0640 + diagnostic 1183); empty-member types -> are valid for all kinds (0641; never-defined `declare` still rejected 1179). 1654 reconciled to the VM wording. -> **709/0 corpus + 476/476 unit, green; commits on `reify` from `5d25e23` (Step A) through `5383496`.** -> REMAINING TAIL (P5.8, needs hardware/SDK): iOS-device + Android bundle validation + an `.apk` corpus smoke test. -> OPTIONAL follow-up: re-express `type_info` as sx (reflect-into-value; kept as builtin to protect the 0619/0622/ -> 0623 round-trips). See the newest `## Log` entries. -> -> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now -> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.** -> The **byte-weld + serialization/marshaling** approach is the wrong direction and is -> being **stripped**. New foundation: a **bytecode VM over byte-addressable -> memory** so comptime values are native bytes; then the compiler-API rides on it with -> direct memory access (no weld, no validation, no marshaling). Everything below this -> banner describes the now-superseded weld state (committed on `reify` through -> `40d075c`) and is kept only to scope the Phase 0 strip. Read -> `PLAN-COMPILER-VM.md` first. -> -> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as -> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)` -> reads the `Value` union's memory, not a struct. The weld tried to bridge that with -> hand-marshaling — exactly what the design set out to kill. Comptime memory makes comptime -> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it -> breaks cross-compilation — host vs target layout — and loses the sandbox. A -> comptime VM keeps both while getting native bytes + speed.) -> -> **Next action (2026-06-18) — the WHOLE metatype surface is VM-native (steps 7+8, committed through -> `d0ebc55`; step 8 uncommitted).** `declare`/`define`/`type_info` + tagged-union `enum_init` all run -> NATIVELY on the VM (`.call_builtin` exec arm → `callBuiltinVm`; `defineFromInfo` decodes a -> `TypeInfo` from comptime memory, `buildTypeInfo` reflects one INTO comptime memory — faithful ports of -> legacy `defineEnum`/`Struct`/`Tuple`/`reflectTypeInfo`). The ENTIRE metatype range `0614`–`0624` + -> `0632` runs **HANDLED with ZERO fallback** (incl. the `define(declare, type_info(T))` round-trips -> `0619`/`0622`/`0623`); VM output byte-matches legacy. `enum_init`/`define`/`type_info` bail loudly -> on a `backing_type` tagged union rather than silent-clobber. **697/0 BOTH gates + all unit tests.** -> **THE NEXT STEP — Phase 4D.3 (`compiler_call` / #compiler hooks on the VM).** Phase 4 (legacy-interp -> retirement) is PLANNED in `PLAN-COMPILER-VM.md`; **user decision: UNIFY** (the VM runs the post-link -> bundler too, `interp.zig` fully deleted). DONE this arc (all green): **4A.1** box_any/unbox_any + -> `.any` as a 16-byte aggregate (`1526d19`); **4D.0** comptime memory → an ARENA, `Addr` = real host -> pointer, no buffer/cap/move (`625ba0f`); **4D.1** general host-FFI escape — `Vm.callHostExtern` -> dlsym + host_ffi, any extern, args/returns pass untouched since Addr IS a host pointer (`e7a8708`, -> example 0636 `toupper`); **4D.2** slice/string args → NUL-term `char*` + float-arg/return guards -> (`6a7f690`, example 0637 `strlen([:0]u8)`). **699/0 BOTH gates.** -> -> **DIRECTION CORRECTION (2026-06-18, user): `#compiler` / `compiler_call` is DELETED, not bridged -> on the VM.** `BuildOptions` is RE-EXPRESSED as **`abi(.zig) extern compiler`** functions (the -> compiler-API surface the VM already dispatches via `callCompilerFn`); the `#compiler` attribute, the -> `compiler_call` IR op, and the `Value`-based hook `Registry` (`compiler_hooks.zig`) all go away. -> So there is **NO transitional `compiler_call`→hooks shim** on the VM (I started one — threading the -> legacy interp into `tryEval` for the hook registry — and reverted it; tree clean at `b05c74f`). -> `0602`/`0603` stay on legacy fallback until the BuildOptions migration lands. **Migration shape** -> (end-state, shares the `BuildConfig`-on-the-VM prerequisite with the bundler 4E): (1) each -> `BuildOptions` setter/getter becomes a `compiler` fn in `compiler_lib.bound_fns` + `Vm.callCompilerFn`, -> reading comptime args + a `*BuildConfig` threaded into the `Vm` (the same `BuildConfig` -> `main.zig` forwards); (2) `library/modules/build.sx` declares them `abi(.zig) extern compiler` -> instead of `struct #compiler`; (3) delete the `compiler_call` op + `compiler_hooks.zig` -> `HookFn`/`Registry` + the `#compiler` parse/lower path. See `PLAN-COMPILER-VM.md` Phase 4. -> -> **Corpus-driven remainder (independent of the BuildOptions migration):** ALL PURE ops are DONE: -> `switch_br`, `type_name`, `error_tag_name_get`, `global_addr`, `type_is_unsigned`. **`out` DONE (2026-06-19, -> newest Log entry):** removed the `out` builtin — it's a plain sx fn calling libc `write`, so the VM handles it -> via host-FFI (no buffer, no special arm; no double-print because there's no `out` op to bail-then-fallback on). -> `trace_resolve` PORTED (1035). 0613/1035/0522/1038 run VM-HANDLED. Remaining side-effect op: `interp_print_frames` -> (1034 — writes the comptime frame chain; could likewise become a plain sx fn over the trace runtime). -> · **4B VM diagnostics (1179/1180) — DONE** (strict renders the proper `comptime type construction failed:` -> diagnostic; VM-gap strict bails are now ONLY the 4 `compiler_call`) · **4C** `#insert`. **BuildOptions migration — design settled + -> foundation underway (2026-06-18, see the two newest Log entries):** `#compiler`/`compiler_call` is replaced -> by `abi(.compiler)` (a compiler-domain ABI — runs in the comptime evaluator, never in the binary). **S1+S2 -> DONE:** `abi(.compiler)` introduced, the old `abi(.zig) extern compiler` + `#library "compiler"` fully removed, -> all compiler-API examples migrated. **S3 DONE:** emit_llvm skips BODIED `abi(.compiler)` (compiler-domain) -> functions; a comptime-only call from a dead body emits `undef` (regression `0638`; 701/0 both gates). The -> earlier "runtime-reachability gating" blocker is MOOT — a compiler-domain callback isn't LLVM-emitted, so its -> `build_options()` calls never reach the `emitCall` gate. **S4 SKIPPED (optional ergonomics):** an -> `abi(.compiler)` function is type-compatible with a plain `() -> R` param (the ABI marks the *function*, not -> its *type*), so callbacks/registrars just declare themselves `abi(.compiler)` (S3) — no param-propagation -> needed. **S5a DONE:** `build_options` + `set_post_link_callback` → `abi(.compiler)`, `BuildConfig` threaded -> into the VM; `bundle_main` + the platform registrars marked `abi(.compiler)`; strict `compiler_call` bails -> 6→2 (0602/0603/1604/1611 HANDLED). **S5a is a GREEN INTERMEDIATE — do NOT extend it.** **DESIGN PIVOT -> (2026-06-18, user): the 37-hook BuildOptions port is DEAD — DRIVE THE BUILD PIPELINE FROM SX** (newest Log -> entry + `PLAN-COMPILER-VM.md` → Phase 5). `BuildConfig` becomes plain sx data; the compiler shrinks to a few -> `abi(.compiler)` primitives (`emit_object`/`link`/queries, explicit args, `-> !` not bool) + an `on_build` -> slot (stdlib `default_build`, user override `#run on_build = build;`). **P5.1 (= 4E) DONE (2026-06-19, newest -> Log entry):** `core.invokeByFuncId` (the post-link build-driver invocation) now runs the callback on the VM -> with **NO fallback** (a side-effecting callback can't double-execute); `BuildConfig` + `import_sources` threaded -> in; VM bail → hard build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). Smoke -> test `1661-platform-post-link-vm-list` (AOT) — a post-link callback that GROWS a `List` (0141: works on VM, -> bails on legacy with `struct_get`); build succeeds (exit 0) only via the VM. `flushInterpOutput` deleted (VM -> writes `out` direct via host-FFI). **702/0 both gates.** **P5.2 metadata queries DONE (2026-06-19, newest Log -> entry):** `c_object_paths() -> List(string)` + `link_libraries() -> List(string)` are `abi(.compiler)` primitives -> (new stdlib home `library/modules/compiler.sx`), serviced by `comptime_vm.callCompilerFn` reading `BuildConfig` -> fields `main.zig` forwards (`c_object_paths`/`link_libraries`). New reusable VM helper `makeStringList` builds a -> `List(string)` in comptime memory (target-aware via the result type's offsets); `invoke`/`callCompilerFn` now thread -> the call's result type (`ins.ty`). Legacy handlers bail loudly (VM-only by nature — post-link). Smoke test -> `1662-platform-build-pipeline-queries` (AOT, C companion → 1 object): a post-link callback checks the VM-built -> list is well-formed; build exit 0 ONLY if so (negative-probe verified: wrong count → "post-link callback -> returned false", exit 1). **`emit_object() -> string` ALSO landed** (a QUERY — the Zig driver emits eagerly, the -> primitive returns `BuildConfig.object_path`; NO vtable). So all three QUERY primitives are done. **703/0 both -> gates.** **P5.2b (`link` ACTION) DONE (2026-06-19, newest Log entry):** `link(objects, output, libraries, -> frameworks, flags, target)` dispatches through a host-installed `compiler_hooks.BuildHooks` vtable (`main.zig` -> `LinkHooksCtx` → `target.link`); **USER DECISION: the build callback is NOT fallible** — `link` is plain VOID, -> a failure BAILS (hard build error), no `-> !`/failable-tuple needed. New VM readers `readStringList`/ -> `readStringArg`. Smoke test `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's -> objects to a temp output via sx `link` — the relinked binary RUNS; negative-probe verified (bad path → bail → -> build exit 1). **P5.3 (`on_build` registrar) + P5.4 CORE DONE (2026-06-19, newest Log entries):** the whole -> build is sx-driven via `default_pipeline` (force-lowered + auto-invoked; NO Zig auto-emit/auto-link); -> `on_build(cb)` is the sole callback mechanism; `set_post_link_callback` deleted. **703/0 both gates.** -> **NEXT — the FULL MIGRATION (no legacy left), spec'd as Phase 5 steps P5.5–P5.8 in `PLAN-COMPILER-VM.md`:** -> **P5.5 DONE (2026-06-19, newest Log entry):** the 35 `BuildOptions` `#compiler` methods → VM-native -> `abi(.compiler)` arms (`comptime_vm.callBuildOptionFn`, NO legacy handler); setter strings duped into the -> persistent `Vm.gpa`; `#run`/const-init compiler-domain entries routed to the VM (`entryNeedsVm`, no fallback) -> so gate-OFF stays green; 5 bundle.sx helpers marked `abi(.compiler)`. BuildOptions `compiler_call` bails GONE -> (1609/1614/1615 strict-clean). 37 `.ir` regenerated (string-pool churn, behavior-identical). · **P5.6 prereq -> DONE (`994d649`):** ported `bit_and`/`bit_or`/`bit_xor`/`bit_not`/`shl`/`shr` into the VM (the 1616 `shr` gap); -> test `0639`. **704/0 BOTH gates.** · **P5.6 REMAINING (the big body, NEXT):** move `platform/bundle.sx`'s -> per-target logic into / called by `default_pipeline` (call `bundle()` after `link` when `bundle_path()` is set), -> reading the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI; remove the `--bundle`/`post_link_module` -> Zig shim (compiler keeps ONLY the linker primitive). ALL bundling + code signing for EVERY target -> (macOS/iOS-device/iOS-sim/Android) in -> the sx `default_pipeline` · P5.7 DELETE `#compiler`/`compiler_call`/`compiler_hooks`/`interp.zig` + the -> `regToValue` bridge + VM→legacy fallback (drop gate-OFF; VM is the SOLE evaluator) · P5.8 build -> `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test + add `.app`/`.apk` smoke tests. -> **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` + -> flip strict-to-default (remove the fallback) + delete `interp.zig`/`Value` + re-express `define`/`make_enum`. -> See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + top risks (bundler test coverage). -> Earlier landed: dedicated `Type` builtin TypeId (`6844fb9`/`94f60c5`/`554871b`); WRITE side -> declare_type/register_type/pointer_to VM-native (`66005af`); real lowering-time Context (`eb68d9e`); -> metatype construction declare/define/enum_init (`d0ebc55`). -> -> Done so far in Phase 3: -> - **READ side (7 readers, dual-path):** `find_type`/`type_kind`/`type_field_count`/ -> `type_nominal_name`/`type_field_name`/`type_field_type`/`type_field_value`, each backed by a -> `TypeTable` query both the legacy handler and the VM call (no drift). Examples 0628–0630. -> - **WRITE side (P3.3, legacy-only at lowering time):** `declare_type` + `pointer_to` + ONE -> kind-branching `register_type` (subsumes `define`'s per-kind dispatch; codes match -> `type_kind`: 1 struct · 2 actual `.@"enum"` · 3 tagged_union · 4 tuple). Idempotent re-fill -> (two-edge import). Plus two fixes (issue 0142): all-void enum → real `.@"enum"` (was a -> verifySizes panic); bare `EnumType.variant` qualified construction. Examples 0631–0635, 0187. -> - **Lowering-time VM (P3.4):** hardened the VM against malformed lowering-time IR (`refTy`, -> bailing `aggType`, bounds-checked branch targets — bails, never panics); wired `tryEval` -> into `runComptimeTypeFunc` behind the flag with legacy fallback; materialized a zeroed -> lowering-time `Context` (the global isn't built yet at lowering). All measured green. -> -> **THE WALL (next step):** a `Type` *value* is an 8-byte tid, but `.any` (the boxed-any) is a -> 16-byte `{tag,value}` — and they share one TypeId (`.any`). So a `Type` in an aggregate -> (`Member.ty`/`EnumVariant.payload`) is sized 16B while the value is 8B → every lowering-time -> type-fn bails at `const_type` / the Member-array build. Can't make `kindOf(.any)` a word: -> at EMIT time `.any` really is a 16B box (variadic any, 0603), so that would silently corrupt -> it. The correct fix is a **dedicated `Type` builtin TypeId (8B), distinct from `.any`** — -> measured at **~123 `.any` references across ~25 files** (pack.zig has 30), a ~100-touch-point -> cross-cutting change → its own focused session (USER CHOSE to pause rather than rush it). -> Rejected alternatives: a scoped "lowering-mode treats `.any` as a word" flag (silent-wrong on -> a real Any box in a reflection type-fn); scalar-only Type-fns (safe but no real corpus type-fn -> is scalar-only — they all build a Member/variant aggregate). -> -> **Decisions recorded:** `find_type` returns a non-optional `TypeId` using `unresolved`(0), NOT -> `?Type`; reader names use the `type_*` family (avoid colliding with std `field_name`/`type_name`); -> the write side is a single kind-branching `register_type`; the write side stays LEGACY-only -> until the VM runs at lowering time (needs the `Type` TypeId). End-state guarantee: ONE -> evaluator — `interp.zig` deleted; dual-path + fallback are transitional (see PLAN end state). -> Build/verify: `zig build && zig build test` (**697**, gate OFF). Run the corpus ON the VM: -> `zig build test -Dcomptime-flat` OR env `SX_COMPTIME_FLAT=1`. Coverage trace: -> `SX_COMPTIME_FLAT_TRACE=1` (now also prints lowering-time `type-fn` HANDLED/fallback lines). - -### (superseded) prior weld resume -Phase 1 done; Phase 2 welded structs were working via reflection + memory-order -validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even -earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in -the compiler type's MEMORY order; the compiler reflected the bound Zig type and -VALIDATED the header. **This whole mechanism is now being stripped — see the banner.** - -> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to -> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690 -> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots. -> See CLAUDE.md → Snapshot integrity. - -## Last completed step -**Phase 2 — welded structs by reflection + memory-order validation (byte-identical, -no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works -end-to-end as a byte-identical mirror of the bound Zig type. - -Design (locked, supersedes the byte-layout-override plan): -- The sx header declares fields in the compiler type's MEMORY order. The compiler - REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from - `@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is - maintained by hand; a `types.zig` change re-reflects on the next compiler build. -- On pass it's an ORDINARY struct whose natural layout already equals the Zig - layout → `@ptrCast` to the compiler type + deref is byte-identical. No - byte-blob, no index/remap tables, no reorder, no special LLVM path. -- Loud, precise diagnostics on any drift: *field not found* (+ memory order), - *wrong field order at position N* (+ expected memory order), *type layout - mismatch* (field size), *layout mismatch* (total size / count). - -What changed from the dropped plan: -- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes - `bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names. - Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks - the sx header against the memory-ordered registry. -- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics - (+ `weldedFieldOrderStr`). -- Examples: `0627` (StructInfo in memory order, byte-identical, usable); - `1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message - refreshed. -- `zig build` + `zig build test` green (692 corpus, unit tests pass). - -### Earlier — Phase 2.1 (weld-plan layout math, now removed) -**The weld-plan offset math + `StructInfo` registered.** Was the core of the -byte-layout-override engine; superseded by the reflection+validation design above. - -Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is -laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering + -padding included), so it passes to a Zig handler as raw memory with zero -marshalling. (The alternative — handlers reading interp `Value` aggregates -logically, no layout override — was rejected; welded types must also be usable as -runtime data, and the design wants the literal byte weld.) - -- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20, - `is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override - is genuinely required (`Field`'s two-u32 natural layout was the easy case). -- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second - `bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc, - fields, total)` — pure: orders fields by ascending byte offset, inserts padding - elements for gaps + the alignment tail, and builds the sx-field → LLVM-element - remap. This is what the LLVM type builder + struct-GEP sites will consume. -- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad); - `StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24, - pad@25..32]`, remap `[1,0,3,2]`. -- `zig build` + `zig build test` green. - -### Earlier — Phase 1 polish (comptime-only enforcement) -**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error -instead of an undefined-symbol link failure.** -- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded` - AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a - `#run`/`::` initializer wrapper whose LLVM body is dead), print a clear - "comptime-only … cannot be called at runtime" error and set - `comptime_failed` (the driver halts before object/JIT emission). The enclosing - `is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green. -- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)` - → clean error, exit 1, no link failure). -- `zig build` + `zig build test` green (458 unit + 690 corpus). - -### Earlier — fifth sub-step (host-call bridge) -**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to -its registered Zig handler instead of dlsym.** -- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`, - `bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the - string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value -> - Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)` - (the same mutable-table access the metatype mint path uses); `text_of` reads the - const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern). -- IR `Function` gained `compiler_welded: bool`. `declareFunction` - (`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES: - the bound lib must be `compiler` and the name must be on the function-export - list — else a build-gating `.err` (no silent fall-through to dlsym). -- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function - routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the - export list). -- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run - text_of(intern("hello, compiler"))` folds to a string constant → prints it); - `examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name → - build error). `findFn` lookup unit-tested. -- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a - RUNTIME call would emit a reference to a non-existent extern symbol → a loud - LINK error (not silent, but not a tidy diagnostic). The examples call welded fns - only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the - immediate follow-up. -- `zig build` + `zig build test` green (458 unit tests + 689 corpus). - -### Earlier — fourth sub-step (welded-struct layout validation) -**A `struct abi(.zig) extern compiler { … }` is validated against the binding -registry as a *header checked against the implementation*.** -- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns - the first `LayoutMismatch` (field count / name / size / total) or null. Plus - `lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes; - each drift flagged as the right variant). -- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`, - `validateWeldedStruct` checks the bound lib is `compiler`, the name is on the - export list (`findType`), and the sx layout (field names + `typeSizeBytes` + - total) matches the welded type — emitting a build-gating `.err` (good span into - the struct body) on any failure. No silent reinterpretation. -- `#library "compiler"` is the comptime-only internal surface, NOT a dylib — - `src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so` - load warning). -- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds, - validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld- - struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic). -- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it - isn't exercised by `Field` (two u32s = natural layout coincides with the weld). - It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound - offsets actually differ from the sx-natural ones. The validation already checks - per-field size + total, so a layout drift is caught even before the override - engine exists. -- `zig build` + `zig build test` green (456 unit tests + 687 corpus). - -### Earlier — third sub-step (binding registry) -**The binding registry (welded-type lookup, layout baked from the real Zig -type).** -- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the - curated safety boundary. `BoundType { sx_name, size, alignment, fields: - []FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real - Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a - sx-field-count mismatch is a `@compileError`, never a silent truncation). - `bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` — - two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl - resolution path will consult (returns null off the export list — clean boundary, - no silent default). -- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`. -- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real - `StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4); - an unexported name returns null. Break-verified (a wrong size → suite red, - named `ir.compiler_lib.test...`). -- `zig build` + `zig build test` green (454 unit tests). - -### Earlier — second sub-step (struct-decl parse) -**`abi(.zig) extern ` PARSES on a STRUCT decl (parse-only, no semantics).** -- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields. -- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler` - check), parse an optional `abi(...)` then optional `extern ` — same slot - order as fn decls — and thread them onto the node. Ordinary structs are - unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent). -- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern - compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib - == "compiler"`, field list intact; a plain struct leaves `abi == .default` / - `extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite - red, confirming the test runs). -- `zig build` + `zig build test` green. - -### Earlier — first sub-step (fn decls) + the syntax pivot -**`abi(.zig) extern ` PARSES on a fn decl (parse-only).** Plus the syntax -pivot it required. - -Syntax decision (locked 2026-06-17, supersedes the doc's original -`extern(.zig) ` single-qualifier form): the ABI/layout selector and the -linkage keyword are two orthogonal annotations. -- `abi(.x)` — ABI / calling-convention annotation in the slot **before** - `extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.** - `ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the - `compiler` library), `.naked` (naked asm), `.default` (unannotated). Can appear - standalone (no extern) on any fn / fn-type / lambda. -- `extern ` — linkage keyword + binding source (named library). - -So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`. - -What landed: -- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`; - the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`. -- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`, - keyword string `"callconv"` → `"abi"`. -- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi` - (parses `abi(.c|.zig|.naked)`); wired in the fn-decl postfix slot (before - `extern`/`export`), the function-type-expr slot, and the lambda slot; - `isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`. -- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`, - `closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the - function-type resolver maps `.zig`/`.naked` → IR `.default` (no fn-pointer-type - CC for those decl-level ABIs; neither occurs in a function-TYPE position yet). -- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the - user-facing text `callconv(.c)` → `abi(.c)`. -- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type - callback annotations — none in the fn-decl postfix slot, so no reordering). -- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 / - 1200) regenerated for the rename. -- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern ` - on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib == - "compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` / - `abi(.naked)`. lexer/sema tests updated. - -`zig build` + `zig build test` green (450/450 unit + 685 corpus). - -## Current state - -> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld -> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for -> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the -> weld *semantics* (layout reflection/validation, marshaling dispatch) go. - -- `compiler :: #library "compiler";` parses + is recognised as the comptime-only - internal surface (never dlopen'd). -- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry - (faithful → ok; drift → build-gating diagnostic). `Field` welds + usable. -- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to - their registered Zig handler (`intern`/`text_of` round-trip works); unexported - names rejected at declaration. Comptime-only. -- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only - enforcement at `emitCall`); the legitimate `#run`/`::` use stays green. -- The whole Phase 1 foundation (parse → registry → struct-layout validation → - function host-call bridge → comptime-only enforcement) is in place for the - two-u32 `Field` case + the two string readers. -- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts - (needed by `StructInfo`'s slice field, Phase 2). - -## Next step — execute `PLAN-COMPILER-VM.md` - -> The weld is being stripped. The next step is **Phase 0 of -> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize / -> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig` -> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics -> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then -> Phase 1 (byte-addressable value model). The weld-era "next step" below is **obsolete** — -> kept only as a record of what the weld surface was about to do. - -### (obsolete) weld-era next step -Welded structs were byte-identical mirrors, so the API surface was set to grow: - -- **Bind `register_struct` / `find_type`** over the host-call bridge - (`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct` - takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names, - kind well-formedness — the checks `define` does today). Because the welded - `StructInfo` is byte-identical, the handler can read it as the real Zig - `*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field — - the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table. - Prove: build a struct programmatically + round-trip a source one. -- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/ - `find_type`; migrate `examples/0622`; delete the bespoke struct interp arms - (`defineStruct` / the `reflectTypeInfo` struct path). - -Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo` -(optional fields → sentinels) — each just needs an sx header in the compiler -type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions` -to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete -`#compiler`. - -Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s -`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout -wrinkle — the sx header must mirror Zig's optional/union representation. Handle -when reached (sentinels or accessor fns; see the design doc Risks). - -## Known issues -- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime - `List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.) - -## Log -- **P5.8 — Android + iOS-sim validated end-to-end on emulator/simulator; first Android bundler corpus coverage - (2026-06-19).** The sx-driven build pipeline now validated on BOTH mobile targets via the local - emulator/simulator. **Android (Pixel_10_Pro emulator):** `sx build --target android --apk … -o lib….so` - produces a valid SIGNED `.apk` (AndroidManifest.xml + resources.arsc + `lib/arm64-v8a/lib….so` + classes.dex + - apksigner META-INF) via `default_pipeline → bundle_main` (javac/d8/aapt2/zip/zipalign/apksigner all on the - comptime VM); `adb install` + `am start` → the app **runs (no crash)** for the `super.onCreate(b)` example - (1424). **iOS-sim (iPhone 17 Pro simulator):** `sx build --target ios-sim --bundle …` → signed `.app`; - `simctl install` + `launch` → the sx AppDelegate fired (`[sx] application:didFinishLaunchingWithOptions: - called`), UIApplicationMain run loop alive. **THREE comptime-VM host-FFI gaps fixed to make the Android bundler - run on the VM** (`d8fb425`): (1) an extern returning an OPTIONAL-of-word (`getenv() -> ?cstring`) now wraps the - bare C payload word into the `{payload, has}` aggregate (present iff non-null), mirroring emit_llvm's - `char*`→`?cstring`; (2) `struct_init` of the builtin two-word aggregates `string`/`any` (e.g. - `from_cstring`'s `string.{ ptr=, len= }`); (3) the non-word-return bail now names the symbol + type. **`.apk` - corpus smoke test** (`2ba36f6`, `examples/1666-platform-android-apk-smoke`): a new `.build` `apk` directive - (cross-compile `--target android --apk`, build+inspect, NO execution) GATED on Android-SDK + real-JDK presence - — SKIPS cleanly on hosts without them (so normal `zig build test` stays green), RUNS+PASSES with `JAVA_HOME` - set (apk built + `AndroidManifest.xml`/`classes.dex`/`lib/arm64-v8a/` asserted, then cleaned up). Requires a - real JDK (the macOS `/usr/bin/javac` stub fails); Android Studio's JBR works. **709 ran + 1666 skipped (no - JDK) / 0 failed + 476/476 unit.** macOS `.app` (1665) + Android `.apk` (1666) now both cover the bundler. - > **REMAINING (true tail):** iOS-DEVICE codesigning/provisioning (needs a real device + signing identity — a - > simulator can't exercise the device-identity flow). Everything else in the stream is done. -- **Empty-member types are VALID for all kinds (user design decision, 2026-06-19).** A comptime-constructed type - with NO members now mints for every kind (empty `struct`, empty `tuple`/unit, empty `enum`, empty - `tagged_union`); ONLY a bare `declare("X")` never completed by a `define` stays rejected. `registerTypeVm` - dropped the blanket "a type with no members is never valid" bail (the per-kind loops are vacuous for an empty - list). To distinguish an EXPLICITLY-defined empty union from a never-defined `declare` PLACEHOLDER (both are - 0-field `tagged_union`s), added `defined: bool = true` to `TaggedUnionInfo` — default true for every real - construction (normal unions, error sets, `register_type` completion); set false ONLY at the two declare- - placeholder sites (`comptime_vm.declareNominal`, `lower/comptime.preregisterForwardTypes`). - `checkComptimeTypeResult` now rejects on `!defined` (not `fields.len == 0`). Codegen parity fix: - `typeSizeBytes(tagged_union)` floors the payload area at 8B when no field carries a payload, mirroring the LLVM - lowering (`backend/llvm/types.zig`) — fixes a `verifySizes` panic on an empty/all-void tagged_union. Tests: - `examples/1179` repurposed from "empty enum rejected" (now valid) to the never-defined `declare` case - (preserves its issue-0140 "diagnostic-not-panic" regression role); `examples/1180` (duplicate variant) - unchanged; new `examples/0641-comptime-empty-types-valid` constructs all four empty kinds. **709/0 corpus + - 476/476 unit** (`5383496`). -- **P5.7 Step D — re-expressed the metatype `declare`/`define` as sx over the compiler-API (2026-06-19).** - `declare(name)` → sx `{ return declare_type(name); }`; `define(handle, info)` → sx that matches the `TypeInfo` - union and calls `register_type(handle, kind, members)` (kinds 1/2/3/4). DELETED the bespoke `callBuiltinVm` - `.declare`/`.define` arms + the `defineFromInfo`/`decodeTypeSlice` helpers + the `declare`/`define` `BuiltinId` - enum members + their `tryLowerReflectionCall` interceptions. The metatype DSL now rides the ONE compiler-API - mechanism (`register_type`/`declare_type`, serviced by `callCompilerFn`), with the DSL in `meta.sx`. Supporting - VM change: tagged-union VALUE support so the sx `define` can `match` a `TypeInfo` (`kindOf` treats - `tagged_union` as a by-address aggregate; `enum_tag` reads the tag word; new `enum_payload` arm; all bail loudly - on a `backing_type` union). **KEPT as builtins (documented):** `type_info($T)` (reflects a type INTO a - byte-compatible `TypeInfo` value — `buildTypeInfo`; re-expressing risks the 0619/0622/0623 round-trips for no - proportional gain) and `field_type($T, idx)` (a LOWER-time fold in `generic.zig`, so it composes inside type-arg - slots — never was a `callBuiltinVm` arm). Diagnostics 1179/1180 regenerated (now name `register_type`). - **708/0 corpus + 476/476 unit** (`8850fcc`/`7b1d8ce`/`ccba704`). Net: 2 of 4 metatype builtins are now sx; the - bespoke Zig metatype surface shrank by two `callBuiltinVm` arms, two helpers, and two `BuiltinId`s. -- **P5.7 Step C — DELETED `interp.zig` (the legacy tagged-`Value` interpreter); the VM is the SOLE comptime - evaluator (2026-06-19).** Five green commits. **C1** (`#insert` → VM): `evalComptimeString` was the last - caller of `Interpreter.call`; routed through `comptime_vm.tryEval` (the VM bails-not-panics on malformed - lowering-time IR like 0737's `ret Ref.none`; `regToValue` dupes the result string into the lowering allocator). - **C2a** (ops.zig inline comptime-call fold → VM): the `emitCall` zero-arg comptime-callee fold now uses - `tryEval`. **C2b** (emit_llvm): dropped the `*const Interpreter` materialization param from `valueToLLVMConst`/ - `serializeAggregateValue` (it was used ONLY for the `.heap_ptr` data arm, which the VM's `regToValue` never - produces) + the dead `interp_inst`. **C3** (the atomic delete, done by a delegated worker + independently - verified): moved the `Value` result-DTO + `decodeVariantElements` into a new `src/ir/comptime_value.zig` - (the VM↔host materialization boundary type); repointed `comptime_vm`/`emit_llvm`/`ir.zig`-barrel `Value` to it - and `BuildConfig` to `compiler_hooks`; deleted the dead `valueToReg` bridge; slimmed `compiler_lib.zig` to just - the name registry (`BoundFn{sx_name}` + `bound_fns` + `findFn`, all names preserved — `weldedCompilerFn` only - validates names; deleted `FnHandler` + all `handle*` + the `Interpreter`/`Value`/`InterpError` imports); - simplified `main.printInterpBailDiag` to use only `comptime_vm.last_bail_reason`; dropped the unused - `interp_mod` import in `lower.zig`; **`rm src/ir/interp.zig` (2383 lines) + `src/ir/interp.test.zig` (844 - lines)** + their barrel entries. **DEVIATION from the plan's literal "delete Value":** `Value` is RELOCATED - (not eliminated) as the slim result/materialization DTO — the byte-addressable VM executes natively, and - `Value` survives ONLY at the VM→`valueToLLVMConst` boundary (the marshaling the pivot killed was at EXECUTION - time, which is gone). Eliminating it entirely (materializing LLVM consts straight from VM `Machine` bytes) is a - larger, riskier rewrite deferred as optional follow-up; the plan's PRIMARY goal — ONE evaluator, no legacy - interpreter, no fallback — is fully met. **706/0 corpus + 476/476 unit** (−24 from the deleted interp unit - tests + 1 `valueToReg` round-trip test). Also dropped dead `Value.asString`/`reflectTypeId` (no callers). - NEXT: Step D — re-express `define`/`make_enum` as sx over the compiler-API (they were legacy interp arms); - Step E — land the 0141 repro + finalize. -- **P5.7 Step B — deleted the `#compiler`/`compiler_call`/hook-Registry mechanism end-to-end (2026-06-19).** - All superseded by `abi(.compiler)` VM-native dispatch (P5.5) — no sx code emits any of it. Two green commits: - **B1** (`e2971f2`) removed the `compiler_call` IR op: the op variant + `CompilerCall` struct (`inst.zig`), the - `Builder.compilerCall` emitter (`module.zig`), the two dead producer blocks in `lower/call.zig` (the - `compiler_expr`-bodied free-fn + method dispatch), every consumer arm (`emit_llvm`, `ops.emitCompilerCall`, - `print`, the `interp.zig` hook-dispatch arm), and the `interp.hooks` field + init/deinit. Stripped - `compiler_hooks.zig` to its still-live `BuildConfig` / `BuildHooks` (link/emit vtable, P5.2b) / `AssetDir` — - deleted `HookError`/`HookFn`/`Registry`/`registerDefaults` + all 37 `hookXxx` fns + the now-unused - `interp`/`Value` imports. The two VM unit tests that used `compiler_call` as a sample unported op now use - `vec_splat`. **B2** removed the `#compiler` attribute + `compiler_expr` AST node: the `hash_compiler` token - (`token.zig`/`lexer.zig`/`lsp/server.zig`), the `is_compiler_struct` / `struct_default_compiler` parser - machinery + the two `compiler_expr` body-synthesis branches (`parser.zig`), the `compiler_expr: void` AST - variant (`ast.zig`), and every `.builtin_expr, .compiler_expr =>` arm / `== .compiler_expr` check across - sema/resolver/semantic_diagnostics/generic/decl/call/calls (dropped `.compiler_expr`, kept `.builtin_expr`). - `abi(.compiler)` (the NEW mechanism) is untouched. Deleted the obsolete `calls.test.zig` `#compiler`-dispatch - unit test. **500/500 unit (−1 obsolete test) + 706/0 corpus, no snapshot churn.** NEXT: Step C — delete - `interp.zig` + the `regToValue`/`valueToReg` bridge; move `#insert` (`evalComptimeString`) to the VM. -- **P5.7 Step A — the flip: VM is the SOLE comptime evaluator at the emit-time + type-fn sites; NO fallback - (2026-06-19).** Removed the `if (self.comptime_flat or need_vm)` gate + the `vm_result orelse fallback` - legacy-interp blocks from `emit_llvm.zig` (`runComptimeSideEffects` AND the const-init path in `emitGlobals`) - and from `comptime.zig` `runComptimeTypeFunc` (type-fn). The VM now ALWAYS runs; a bail is ALWAYS a - build-gating diagnostic (`comptime_vm.last_bail_reason`), never a fallback. Deleted `emit_llvm.entryNeedsVm` - (moot — every entry runs on the VM now). `runComptimeSideEffects` no longer creates an `Interpreter` at all - (VM writes `#run` `print` output direct to fd 1 via host-FFI); `emitGlobals` keeps a fresh `interp_inst` ONLY - as the helper context `valueToLLVMConst` uses to materialize the VM's result Value (it evaluates nothing) — - that's the `regToValue` bridge, removed in Step C with `interp.zig`. **`#insert` (`evalComptimeString`) still - uses the legacy interp** — intentionally deferred to Step C (interp.zig still exists); it only needs the - evaluator to bail without crashing (0737's real error is a lowering-time visibility diagnostic). The - `comptime_flat*` LLVMEmitter fields are now set-but-unused (harmless; cosmetic cleanup later). **Snapshot - reconcile:** only `1654` churned — the asm-global `#run` now reports the VM's clean `comptime init of - 'COMPUTED' failed: comptime extern call: symbol not found via dlsym …` instead of the legacy - `CannotEvalComptime (op=call: …)` wrapper (exit still 1); regenerated scoped via `-Dname`. `1179`/`1180` - unchanged (the VM-strict `comptime type construction failed:` wording already matched). **501/501 unit + - 706/0 corpus** (one gate now — `-Dcomptime-flat` is moot but still accepted). NEXT: Step B — delete the - `#compiler` attribute (parse+lower) + the `compiler_call` IR op + `compiler_hooks.zig`. -- **P5.7 Step 0 — strict sweep CLEAN; zero VM gaps to port (2026-06-19).** Gating prerequisite for deleting the - legacy fallback. Confirmed both gates 706/0 + 501/501 unit (gate-OFF and `-Dcomptime-flat`). Then ran every - corpus example (706) under `SX_COMPTIME_FLAT_STRICT=1` (VM, NO fallback) via `.sx-tmp/strict_sweep.sh` — - `sx run` (JIT), `sx build` (aot/bundle), or `sx ir --target` (cross-arch). Only **3** examples emit a VM-bail - signature, ALL expected-failures (strict exit == expected exit), NONE a real gap: **1179** - (`enum has no variants`) + **1180** (`duplicate variant name`) render the SAME `comptime type construction - failed:` diagnostic the VM-strict path in `comptime.zig` already emits (no snapshot churn at the flip); **1654** - (asm-global called at `#run`) — the VM bails cleanly via `callHostExtern` dlsym ("symbol not found … target- - specific binding called at compile time?"), only its `.stderr` WORDING changes (legacy `CannotEvalComptime - (op=call…)` → VM strict form) and must be reconciled at the flip. **Key conclusion (the prompt's flagged risk): - no SUCCESSFUL corpus example relies on the legacy VM→legacy fallback** — every passing example runs natively on - the VM; the only fallback users are the 3 expected-failures above. Removing the fallback is therefore safe. No - ops to port before flipping. NEXT: make `-Dcomptime-flat` permanent + delete the fallback (emit_llvm both sites - + comptime.zig) + remove `entryNeedsVm`, reconcile 1654. -- **P5.8 (partial) — real-project validation: m3te + distribution build with the new pipeline (2026-06-19).** - Acceptance test for the sx-driven build pipeline. **m3te** (`~/projects/m3te`, an SDL3 match-3 game): migrated - its `build.sx` off the deleted API — `configure_build :: ()` → `() abi(.compiler)`, `opts.set_post_link_callback( - bundle_main)` → `on_build(bundle_main)` (the ONLY source change needed). Then `sx build main.sx` produces a valid - SIGNED macOS `.app` (correct `Contents/{MacOS,Resources}` layout, `Resources/assets/` bundled, links Homebrew - SDL3, passes `codesign`); `sx build --target ios-sim main.sx` produces the flat iOS `.app` (DTPlatformName= - iPhoneSimulator; system-framework "not embedded" warnings are expected/benign). So BOTH the macOS and iOS-sim - bundle paths validate end-to-end on real project code. **distribution** (`~/projects/distribution`, a - server/CLI+sqlite project, NO bundling): `make build` clean (smoke + dist binaries via `default_pipeline`'s - emit+link with C objects + vendored sqlite); `make test` 24/25 (the 1 fail, `publish_happy.sx`, is a PRE-EXISTING - stale `#foreign` parse error — that keyword is gone from the parser; unrelated to the pipeline/this session). - **m3te's `build.sx` edit is in its working tree, uncommitted (the user's repo) — reported, not committed.** - > **macOS `.app` corpus smoke test — DONE (2026-06-19, `445ae97`):** added a `.build` `bundle` directive to the - > corpus runner (after an `aot` build, assert each `expect` entry under the produced `.app`, then `rm -rf` it; - > macOS-host only, skipped elsewhere). `examples/1665-platform-macos-bundle-smoke.sx` exercises - > `default_pipeline`'s auto-bundle end-to-end; passes on BOTH gates (bundler runs on legacy interp AND VM), - > negative-probe verified. **706/0 both gates.** This closes the stream's named top-risk gap (no bundler coverage). - > **REMAINING P5.8:** iOS-device + Android paths still unvalidated on this host (no device identity round-trip / - > Android SDK; an `.apk` corpus smoke test needs an Android SDK on the runner). m3te is the real-project macOS + - > iOS-sim acceptance test. -- **P5.6 (macOS path) — `default_pipeline` drives bundling; fix issue 0125 (2026-06-19).** `build.sx` now - `#import`s `platform/bundle.sx` and `default_pipeline` delegates to `bundle_main` when `bundle_path()` is set - (emit+link via the shared `emit_and_link` core, then wrap the `.app`/`.apk`); else just emit+link. **Removed the - Zig `--bundle`/`post_link_module` dispatch shim** (main.zig) — CLI bundle flags only feed `BuildConfig`, - `default_pipeline` branches on `bundle_path()`. **USER DECISION:** import the bundler directly (it's - `abi(.compiler)`, emit-skipped, never in the binary; the build↔bundle import cycle resolves like std↔build) — - NOT a registration slot. **Validated end-to-end on macOS** (the stream's top-risk gap — bundler had ZERO - coverage): `sx build --bundle App.app --bundle-id … plain.sx` AND auto-bundle from `set_bundle_path` both produce - a valid SIGNED `.app` (correct `Contents/MacOS/` layout, Info.plist, passes `codesign`, binary runs). Fixed a - pre-existing host-build bug: `target_triple` was empty for host builds → `is_macos()` false → wrong flat layout; - main.zig now exposes the host triple (`LLVMGetDefaultTargetTriple`) when `--target` is absent. `bundle_main` - dropped its redundant `build_options()` re-fetch (uses its `opts` param). **Fix issue 0125** (surfaced because - the bundler's `format("…{}…")` instantiates `any_to_string`, which over-materialized array types): the type-match - dispatcher (`lowerRuntimeDispatchCall`) unboxed each interned array tag to the concrete array type (whole-array - load) → LLVM scalarized to one DAG node/element (~12s / segfault at `[65536]u8`). Fix (route 1): `case array:` - arm calls `slice_to_string`, dispatcher builds a `{ptr,len}` slice VIEW of the payload pointer (`unbox_any → - [*]elem` = int-to-ptr, NO load) for an ARRAY tag bound to a SLICE param. Output byte-identical (`[a, b, c]`); - 0055 drops 12s→0.2s. Pinned `examples/0056-basic-large-array-format-no-blowup.sx`; issue 0125 marked RESOLVED. - 37 `.ir` regenerated (bundler types + array-format lowering); `.ir`-only. **705/0 both gates** (`48eb7bf`). - > **REMAINING P5.6 / next:** iOS-device / iOS-sim / Android paths in `bundle_main` exist + now run on the VM (the - > `shr` port let them through) but are NOT validated on this host (no iOS/Android SDK + no corpus bundle harness) - > — that's P5.8 (build `m3te`/`distribution` end-to-end + add `.app`/`.apk` corpus smoke tests). Then P5.7 (delete - > `#compiler`/`compiler_call`/`compiler_hooks`/`interp.zig` + the legacy fallback; VM becomes the sole evaluator). -- **P5.6 prerequisite — bitwise/shift ops ported into the VM (2026-06-19).** `comptime_vm` exec now handles - `bit_and`/`bit_or`/`bit_xor`/`bit_not`/`shl`/`shr` (new `bitwise` helper next to `arith`), mirroring the legacy - interp's i64 model EXACTLY: shift amount clamps to `@min(rhs, 63)`, `shr` is an ARITHMETIC right shift - (sign-extending). These were unported and bailed — the `shr` gap surfaced via the iOS-device bundler once P5.5 - let it run further (1616). With the port, 1616's strict VM run reaches the real bundler logic (no more `shr` - bail; it now stops only at the genuinely-unavailable iOS runtime on macOS — `_UIApplicationMain` / no linked - binary under `sx run`, expected). New focused corpus test `examples/0639-comptime-bitwise-shift.sx` (`::` consts - fold AND/OR/XOR/NOT/shl/shr/arith-shr; identical on both evaluators). **704/0 BOTH gates.** -- **P5.5 — the 35 `BuildOptions` accessors migrated off `struct #compiler` onto VM-native `abi(.compiler)` (2026-06-19).** - `BuildOptions :: struct #compiler { ...35 methods... }` → `BuildOptions :: struct { }` (an opaque - null-sentinel handle) + 35 free `ufcs (self: BuildOptions, …) abi(.compiler)` decls in - `library/modules/build.sx`, each serviced by a new `comptime_vm.callBuildOptionFn` arm (dispatched from - `callCompilerFn`). **NO legacy `compiler_lib` handler** (per the full-migration direction): the 35 names are - registered in `compiler_lib.bound_fns` only so `weldedCompilerFn` accepts them, with a single bailing stub - `handleBuildOptionsAccessor` (never reached). **String lifetime:** setters dupe the arg string into the - PERSISTENT `Vm.gpa` (the Compilation allocator threaded into both `tryEval` and `runBuildCallback` — NOT the - per-eval VM arena, whose bytes die at `Vm.deinit`), so a `#run`-set path survives to post-link. Setters - write/append the duped string to the threaded `BuildConfig` (`output_path`/`bundle_path`/…, the `link_flags`/ - `frameworks`/`asset_dirs` ArrayLists); string getters return the field (or `""`); bool getters compute from the - triple (`predIsMacOS`/`predIsIOS`/…, mirroring the legacy hooks); count/indexed getters read the `BuildConfig` - slices. **Dispatch routing (Option B, chosen at start):** a `#run` / const-init entry that directly calls a - compiler-domain / compiler-welded fn (`emit_llvm.entryNeedsVm`) is routed through the VM with NO legacy fallback - — regardless of the `-Dcomptime-flat` gate — so gate-OFF stays green without a legacy BuildOptions handler - (P5.7 retires the legacy interp entirely). The 5 `platform/bundle.sx` helpers that call getters - (`build_info_plist`/`embed_framework`/`android_bundle_main`/`build_android_manifest`/`compile_jni_main_sources`) - are marked `abi(.compiler)` too (they're comptime-only bundler code; without it their now-welded getter calls - trip the runtime-call gate). **Snapshots:** 37 `.ir` churned (std transitively imports build.sx → string-pool/ - type-table indices shift) — regen scoped via `-Dname`; verified ONLY `.ir` changed (zero behavior-stream diffs). - **703/0 BOTH gates.** Strict sweep: the BuildOptions `compiler_call` bails are GONE (1609/1614/1615 strict-clean); - 1616 now bails on `shr` (a pre-existing, separate VM gap — bitwise/shift ops `shl`/`shr`/`bit_and`/`bit_or`/ - `bit_xor`/`bit_not` are unported in `comptime_vm`, surfaced now that the iOS-device bundler runs further; 1616 is - unpinned + can't JIT-run on macOS anyway). **Also (per user): swept the outdated "flat memory" terminology** — - the comptime VM is byte-addressable, ARENA-backed memory where `Addr` is a REAL host pointer, NOT a flat - contiguous address space; "flat memory"/"flat-memory" → "comptime memory" / "byte-addressable" across - `comptime_vm.zig` + the plan/checkpoint/CLAUDE docs (flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept). - > **NEXT — P5.6 (ALL bundling + code signing in `default_pipeline`).** First likely sub-task: port the - > bitwise/shift ops (`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`) into `comptime_vm` so the real bundler - > path runs on the VM (the 1616 `shr` gap). Then move `platform/bundle.sx`'s per-target logic to read the - > migrated `abi(.compiler)` getters + `fs`/`process` host-FFI, call `bundle()` from `default_pipeline` after - > `link` when `bundle_path()` is set, and remove the `--bundle`/`post_link_module` Zig shim. -- **P5.4 CORE — the whole build is sx-driven via `default_pipeline`; no Zig auto-emit/auto-link (2026-06-19).** - The compiler's post-IR role is now: codegen → invoke the build callback. **There is NO auto-emit / auto-link.** - Commits (all green): (1) **core** (`d178454`) — `emit_object()` is an ACTION (verify+emit via a host - `BuildHooks` vtable; `main.BuildHooksCtx`); new query primitives `build_output`/`build_target`/ - `build_frameworks`/`build_flags` (read the merged `BuildConfig`); `library/modules/build.sx` imports - `compiler.sx` + defines `default_pipeline` (emit → gather c_objs → link); the compiler **force-lowers** - `default_pipeline` (well-known name, `decl.isDefaultBuildPipeline`, force-lowered after Pass 2) and - **auto-invokes** it post-codegen when no `on_build` override (`main` final fallback `invokeByName - "default_pipeline"`); the BUILD path **auto-imports `modules/build.sx`** (prepends a synthetic import node in - `compileWithTimer`) so prelude-less programs (asm tests) still get `default_pipeline`; removed the build cache - short-circuits (a future cache can live in `default_pipeline`). (2) **on_build-only** (`65ac370`) — migrated - all 9 `set_post_link_callback` callers to `on_build(cb)` (callback gains `opt: BuildOptions`); DELETED - `set_post_link_callback`. **Override semantics changed:** an `on_build` callback REPLACES the build (must - emit+link or `return default_pipeline(opt)` — delegation verified), unlike the old post-link callback that ran - AFTER the auto-link. Reworked tests: 1662 (queries) + 1664 (override+List-grow) DELEGATE to `default_pipeline`; - deleted 1661/1663 (primitives now exercised by EVERY AOT build). `sx run` (JIT) is UNTOUCHED (emits in-process, - never invokes `default_pipeline`). Benign `.ir` churn each step; **703/0 both gates.** - > **REMAINING P5.4 (the BuildOptions-surface migration — large, mechanical, dual-path, string-lifetime-sensitive; - > NOT YET DONE):** **FINAL DIRECTION (user 2026-06-19): FULL MIGRATION — NO LEGACY. Drop gate-OFF entirely; - > the VM is the SOLE evaluator; delete `interp.zig`. Migrate DIRECTLY to VM-native `abi(.compiler)` arms — NO - > legacy `compiler_lib` handlers, NO dual-path.** See `PLAN-COMPILER-VM.md` → Phase 5 steps **P5.5–P5.8** for - > the full spec. In brief: - > - **P5.5** — migrate all 36 `BuildOptions :: struct #compiler` methods → free `ufcs … abi(.compiler)` decls + - > `comptime_vm.callCompilerFn` arms (NO legacy handler). Setters dupe strings into a PERSISTENT allocator - > (thread `emit_llvm.alloc` via e.g. `BuildConfig.string_alloc`). Kills the 4 strict `compiler_call` bails. - > - **P5.6** — ALL bundling + code signing for EVERY target (macOS `.app`, iOS device/sim, Android `.apk`: - > Info.plist/codesign/provisioning/entitlements/framework-embed/AndroidManifest/javac/d8/aapt2/zipalign/ - > apksigner) runs in the sx `default_pipeline` (via the migrated getters + `fs`/`process` host-FFI). Remove the - > `--bundle`/`post_link_module` Zig shim. Compiler keeps ONLY the linker primitive (Option B). - > - **P5.7** — DELETE `#compiler` + `compiler_call` op + `compiler_hooks` (Registry/HookFn) + `interp.zig` - > (Interpreter/Value/reflectTypeInfo/callExtern) + `regToValue`/`valueToReg` + the VM→legacy fallback; make - > `-Dcomptime-flat` permanent. A VM bail is ALWAYS a build diagnostic now. Re-express `define`/`make_enum` as - > sx. Land the 0141 repro; reconcile 1654. - > - **P5.8** — build `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test that - > `default_pipeline` covers all targets; add `.app` + `.apk` bundle smoke tests (no corpus coverage today). -- **P5.3 (`on_build` registrar) — the build-callback registration mechanism; callback takes `BuildOptions` (2026-06-19).** - Per the user's design: `on_build(cb)` is the build-callback registrar (a FREE fn), generalizing - `set_post_link_callback` — the callback is `(opt: BuildOptions) -> bool abi(.compiler)` and the compiler invokes - it post-codegen WITH the opaque `BuildOptions` handle. **Key simplification:** the handle is a single - null-sentinel word, so passing it sidesteps the feared fat-`BuildConfig` marshaling. Changes: VM - `callCompilerFn` `on_build` arm + legacy `handleOnBuild` (both set `post_link_callback_fn` + a new - `BuildConfig.post_link_takes_options` flag); `comptime_vm` `runEntry`→`runEntryArgs(extra)` (implicit ctx + - explicit args) + a public `runBuildCallback(..., pass_options)`; `core.invokeByFuncId`/`invokeByName` now take - `pass_options` (was an always-empty args slice); `main.zig` passes `getPostLinkTakesOptions()`; `build.sx` - `on_build` decl. Smoke test `1664-platform-on-build-callback` (AOT). Benign 37-`.ir` churn (type table +1 for - the `on_build` fn type; behavior identical — verified only `.ir` streams changed). **705/0 both gates.** - > **CONSOLIDATED REMAINING PLAN (P5.4 — from the user's 2026-06-19 direction; large + coupled + re-churns - > snapshots; the bundler has NO corpus coverage = the stream's top risk):** - > 1. **Migrate to `on_build` ONLY** — convert every `set_post_link_callback(cb)` caller (`platform/bundle.sx` - > `bundle_main`, examples 1611/1614/1615/1616, 0602/0603) to `#run on_build(cb)` with `cb: (opt: - > BuildOptions) -> bool`; DELETE `set_post_link_callback` (build.sx + compiler_lib + VM arm). - > 2. **Bundle/Android config → sx data in the default script.** The `#compiler` accessors the user flagged — - > `set_bundle_path`/`bundle_path`/`bundle_id`/`codesign_identity`/`provisioning_profile`, - > `set_manifest_path`/`keystore_path`, `jni_main_count`/`jni_main_runtime_path_at`/`jni_main_java_source_at` - > — move into the sx `BuildConfig`/default script (sx-owned data), not compiler hooks. - > 3. **`default_pipeline` + override model.** `library/modules/build.sx` ships `#run on_build(default_pipeline)` - > (the stdlib default); a user's `#run on_build(custom)` in main.sx OVERRIDES it (LAST-WINS — already the - > behavior, since registration just overwrites `post_link_callback_fn`). `default_pipeline` calls - > `emit_object`/`c_object_paths`/`link_libraries`/`link` + the sx bundler. - > 4. **REMOVE the Zig driver's auto-emit/auto-link** (`main.compileWithTimer`) — COUPLED with (3): once - > `default_pipeline` drives emit+link, the driver must stop doing them or it double-links. Riskiest piece - > (whole build/bundle path; no corpus guard → needs dedicated bundle smoke tests). - > 5. **Delete `#compiler`/`compiler_call`/`compiler_hooks`** + the S5a `build_options` once config is sx data → - > kills the 4 strict `compiler_call` bails (1609/1614/1615/1616) → strict sweep green → `interp.zig` deletable. -- **P5.2b (`link` ACTION) — the sx `link` primitive links on the VM via a host-installed vtable; build callback de-failable'd (2026-06-19).** - Phase 5's one genuine ACTION primitive: `link(objects, output, libraries, frameworks, flags, target)` (in - `library/modules/compiler.sx`). **USER DECISION this step: drop fallibility from the build callback** — so - `link` is a plain VOID primitive (no `-> !`), and a link failure BAILS on the VM → hard build error (sidesteps - the failable-tuple-return construction entirely). **The vtable:** `comptime_vm.zig` can't depend on the driver - (`core`/`main`/`target`), so `link` dispatches through a new `compiler_hooks.BuildHooks { ctx, link_fn }` that - `main.zig` installs into `BuildConfig.build_hooks` before the post-link callback. The driver side is - `main.LinkHooksCtx` (holds allocator/io/base_config/has_jni_main; its `link` adapter unions the explicit - `flags` with the CLI ones and calls `target.link(objects[0], objects[1..], …)` — the linker treats first-vs-rest - as equal inputs). **New VM readers** (inverse of `makeStringList`): `readStringList` (a `List(string)` arg → - `[][]const u8`, element bytes are views into stable comptime arena) + `readStringArg` (a `string` arg). - Registered `link` on `bound_fns` (legacy stub bails — VM-only). **Smoke test** - `examples/1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's own objects (via - `c_object_paths` + `emit_object`) into a temp output through the sx `link` primitive — and the **relinked binary - is a FUNCTIONAL executable that runs** (verified manually). Build exit 0 only if the VM-driven link succeeds; - **negative-probe verified** (bad output path → `ld` fails → `ComptimeVmBail: comptime link: linking failed`, - build exit 1 — the P5.1 VM-reason diagnostic path). **The driver still auto-links too** (P5.2b does NOT remove - the Zig driver's `target.link`; the test links to a SEPARATE temp output) — removing the auto-link + having - `on_build` drive everything is P5.3/P5.4. **704/0 both gates.** -- **P5.2 (metadata queries) — `c_object_paths` / `link_libraries` compiler primitives + the VM `List(string)` builder (2026-06-19).** - Phase 5 step 2 (the read-only slice): two `abi(.compiler)` primitives that the sx build driver will pass to - `link` — `c_object_paths() -> List(string)` (the `#import c` companion `.o`s) and `link_libraries() -> List(string)` - (the `#library` names). They live in a NEW stdlib file `library/modules/compiler.sx` (the Phase 5 home the sx - `default_build` grows into) and are serviced by `comptime_vm.callCompilerFn` reading two new `BuildConfig` - fields (`c_object_paths`/`link_libraries`) that `main.zig` forwards before the post-link callback (alongside - `binary_path`/`target_triple`/…). **Reusable new piece:** `Vm.makeStringList(table, list_ty, items)` builds a - `List(string)` in comptime memory — backing array of `string` fat pointers + the `{items,len,cap}` struct, all laid - out from the RESULT type's field offsets/types (target-aware, no hardcoded layout). To get the result type, - `invoke`/`callCompilerFn` now thread the call instruction's `ins.ty` (the only call-result-type need so far). - Legacy (`compiler_lib`) handlers for these bail loudly (`handleBuildPipelineQuery`) — they're VM-only by nature - (the post-link callback always runs on the VM since P5.1), and a `List(string)` isn't faithfully buildable in - the legacy `Value` model (0141). Registered on `bound_fns` so `weldedCompilerFn` recognizes them. **Smoke test** - `examples/1662-platform-build-pipeline-queries` (AOT + a 1-line C `#source` → exactly one C object): a post-link - callback asserts `c_object_paths().len == 1`, `items[0].len > 0`, and iterates `link_libraries()` (liveness - touch) — build exit 0 only if the VM-built list is well-formed. **Negative-probe verified** a real guard (forcing - `len != 2` → "post-link callback returned false", build exit 1). **`emit_object() -> string` ALSO landed (same - step):** a QUERY, not an action — the compiler emits the object eagerly (the Zig driver, before the callback), - so the primitive just returns the path from a new `BuildConfig.object_path` field `main.zig` forwards (no - driver vtable needed). 1662's callback now also asserts `emit_object().len > 0`. So ALL THREE query primitives - (`emit_object`/`c_object_paths`/`link_libraries`) are done; only `link` (the genuine ACTION) remains. **No unit - test for `makeStringList`** — - constructing a `List(string)` `TypeId` in the test harness needs generic instantiation; the corpus test - exercises the real stdlib type end-to-end with a non-empty list + a negative guard instead. **`emit_object` + - `link` (the ACTIONS) deferred to P5.2b** — they must replace the Zig driver's auto-emit/auto-link (not duplicate - it), so they need the driver-restructuring + a host-installed callback vtable (the VM can't depend on - `core`/`main`/`target`). **703/0 both gates** + strict JIT run clean (no `compiler_call` bail). -- **P5.1 (= 4E) — the post-link build driver runs on the VM (NO fallback); smoke test 1661 (2026-06-19).** - Phase 5 step 1: `core.invokeByFuncId` — the post-codegen / post-link callback invocation `main.zig` fires after - `target.link` — now routes the callback through the **comptime VM** (`comptime_vm.tryEval`) instead of the - legacy `Interpreter`. **REQUIRED** because the sx build driver allocates/grows `List`s, which the legacy interp - can't do at comptime (issue 0141: `struct_get: base has no fields`); the VM can. **NO fallback** (user - directive): a side-effecting post-link callback can't safely re-run on a second evaluator (double execution), - so a VM bail is a HARD build error — `error.ComptimeVmBail`, with the reason in `comptime_vm.last_bail_reason` - (now surfaced by `main.printInterpBailDiag`, which previously only read the legacy interp's `last_bail_*` - statics). `BuildConfig` (`&emitter.build_config`) + `import_sources` are threaded into the VM call. Deleted the - now-dead `flushInterpOutput` (the VM writes `out` directly via host-FFI — no buffer to flush). Non-empty `args` - rejected loudly (`error.ComptimeVmArgsUnsupported`) — the `on_build(config)` arg-passing entry arrives in P5.3. - **Verification:** a probe with a List-growing post-link callback FAILS on the pre-change legacy path - (`sx build` exit 1, `OutOfBounds (op=struct_get)`) and SUCCEEDS after the change (exit 0). Formalized as - `examples/1661-platform-post-link-vm-list` (`{ "aot": true }`): the callback grows a `List` to 3 + returns - `len == 3`; the build links cleanly (exit 0) and the binary prints `runtime main`. AOT snapshots the binary's - streams (build stdout discarded), so the VM-success is pinned via exit 0 + `runtime main` — a legacy regression - would flip the build to exit 1 and mismatch. **No corpus example fires post-link** (none had AOT sidecars; the - platform examples register a callback at `#run` time but run JIT) — so `invokeByFuncId` was previously untested - by the corpus; 1661 is the first coverage. The 4 strict `compiler_call` bails (1609/1614/1615/1616) are - UNAFFECTED — they bail at `#run configure()` on still-`#compiler` accessors (`set_bundle_path` etc.), killed by - P5.4, not here. **702/0 both gates.** -- **4B (VM-native diagnostics) — the metatype negative tests (1179/1180) render proper diagnostics under strict; strict gap-bails now ONLY `compiler_call` (2026-06-19).** - The legacy and VM both BAIL on a `define()` validation failure with an identical detail string; only the - host's STRICT rendering differed (generic "bailed on the VM (strict)" vs the proper "comptime type - construction failed: " + span the non-strict legacy path emits). Fixed: (1) aligned the VM's `define` - messages with the legacy's exact text — `comptime define():` (was `comptime define:`), and the duplicate - variant/field cases now NAME the offender via a new `failFmt` helper (`'...' duplicate variant name 'value'`). - (2) The strict type-fn path (`lower/comptime.zig`) now emits `d.addFmt(.err, span, "comptime type construction - failed: {s}", .{vm_reason})` — the SAME diagnostic as the legacy fallback, so **1179/1180 produce their exact - expected `.stderr` under strict with NO legacy interp involved**. Left the const-init/`#run` strict paths on - the "bailed on the VM" wrapper ON PURPOSE — they still carry genuine VM-gap bails (`compiler_call`), so the - burndown sweep must keep distinguishing those. **701/0 both gates.** **STRICT GAP-BAILS NOW: only the 4 - `compiler_call` (1609/1614/1615/1616 → Phase 5 sx-build-pipeline)** + 1654 (a legitimate unresolvable-symbol - diagnostic — an asm global called at comptime; the legacy can't resolve it either; reconciles to VM wording - at the 4F flip). So: BuildOptions/Phase 5 is the ONLY thing between the VM and a green strict sweep. -- **`out` is now a PLAIN SX FUNCTION (libc `write`), NOT a builtin — VM handles it via host-FFI; `trace_resolve` ported; 0522 fixed (2026-06-19).** - Per user: removed the `out` `#builtin` entirely. `library/modules/std/core.sx` now defines - `libc_write :: (fd, [*]u8, usize) -> isize extern libc "write"` + `out :: (str: string) { libc_write(1, - str.ptr, xx str.len); }`. Deleted `BuiltinId.out` (`inst.zig`), the `resolveBuiltin` "out" mapping - (`call.zig`), the sema builtins-list entry (`sema.zig`), and BOTH `.out` arms (`interp.zig` buffered-append, - `ops.zig` LLVM `write` lowering). **At comptime `out` runs through the evaluator's host-FFI** (the VM's - dlsym `write` path / the interp's extern call) — so the VM HANDLES `out` with NO special arm. Benign prelude - `.ir` churn (`[*]u8` interned earlier + `@out`→`@write` + the `out` fn body) → regen'd 54 `.ir` snapshots - (verified: only string-table renumber + the intended decl/fn-body change; zero stdout/exit changes). - **This UNMASKED two latent VM gaps the `out`-bail was hiding** (the VM now runs past `out`): - (1) **`trace_resolve`** (1035) — PORTED to the VM (`comptime_vm.zig`): unpack the `(func_id<<32|offset)` - comptime frame, resolve func name + `file:line:col` + source line via a **`source_map` now threaded into the - VM** (new `tryEval` param, `&import_sources` from emit_llvm), build the `{file,line,col,func,line_text}` - `Frame` struct in comptime memory (`makeStringValue`/`writeField`/`fieldOffset`). (2) **0522** (bare-pack - `[]Any`) — was a CRASH (`reflectArgTypeId` `@intCast` of a garbage word) → hardened to a loud bail - (`typeIdxOf` checked cast; the VM must never panic). ROOT CAUSE: after the 0143 fix `$args` materializes as - `[]type_value` (8-byte), but the example declared `describe(args: []Any)` (16-byte) → every element past the - first read at the wrong stride; the legacy's loose Value model tolerated it, the byte-accurate VM didn't. The - bare-pack elements ARE `Type`s, so the fix is the honest type — `describe(args: []Type)` (output identical). - **Result: `out`/`trace_resolve`/the 0522 pack-reflection all run VM-HANDLED under strict** (0613/1035/0522/1038 - no longer bail). **701/0 BOTH gates + full suite.** (Build-pipeline relevance: the sx `default_build` driver - uses `out` for diagnostics — now VM-native; no compiler `out` builtin to special-case.) - **THEN `interp_print_frames` ported to the VM too** (1034): unlike `out` it needs the live evaluator - call-chain, so it's a VM arm (mirrors legacy `printInterpFrames`) — walks `call_stack` (skips the last frame), - writes ` at ` lines straight to fd 1 (consistent with `out`'s direct `write`). 1034 matches; 701/0. - **STRICT DELETION-GATE NOW DOWN TO 7 (all known categories):** `compiler_call` (4 — 1609/1614/1615/1616, the - still-`#compiler` BuildOptions accessors → Phase 5 sx-build-pipeline) · VM-diagnostic negatives (2 — - 1179/1180, the `define` bail IS the expected outcome → **4B**: surface as a proper build diagnostic) · - target-specific dlsym (1 — 1654, an asm global called at comptime; legacy can't resolve it either → a clean - diagnostic, not a bug). EVERY pure + side-effect op bail is cleared. -- **DESIGN PIVOT (2026-06-18, user) — DRIVE THE BUILD PIPELINE FROM SX; the 37-hook BuildOptions port is dead.** - Trigger: porting each `BuildOptions` accessor to an `abi(.compiler)` fn that delegates to a `compiler_hooks` - hook just re-encodes sx-level logic (setters/getters, `is_macos` triple-matching, list appends) as compiler - hooks — they need NOTHING from the compiler but the `BuildConfig` state. So instead: **`BuildConfig` becomes - plain sx data** (ordinary struct, sx-owned, no `#compiler`/hooks/shared-state/weld), and the **build pipeline - is an sx program** — the logical end of "bundling lives in sx". The compiler shrinks to a few `abi(.compiler)` - PRIMITIVES taking EXPLICIT args (`emit_object() -> !string`, `link(objects, output, libs, fws, flags, target) - -> !`, metadata queries) + an `on_build : (BuildConfig) -> ! abi(.compiler)` slot (stdlib default - `default_build`; user overrides via `#run on_build = build;`). **Chosen boundary: Option B** (compiler keeps - the Zig linker; sx owns config+orchestration+bundle); Option A (sx shells `cc`/`ld`) is a later refinement. - **NO bool** — failures are the error channel (`-> !`); VERIFIED on the current build: void `#run`, `-> !`/`-> !E` - failable `#run`, and a `raise` at `#run` fails the build with a return trace (+ suggests `#run … catch (e){…}`). - `on_build` GENERALIZES today's `post_link_callback_fn` (assignable typed global w/ default, vs a setter). - **Full design + step plan in `PLAN-COMPILER-VM.md` → Phase 5.** **S5a (below) is a green intermediate that the - sx-pipeline replaces wholesale** (don't extend it; P5.4 deletes `build_options`/`set_post_link_callback` + - all `#compiler`). **NEXT — P5.1 (= 4E):** route the post-codegen / `on_build` invocation through the VM - (`core.invokeByFuncId` → VM), REQUIRED because the sx driver allocates `List`s and the legacy interp can't - (0141, VERIFIED: comptime `List` growth works on the VM, fails on legacy with `struct_get: base has no - fields`). Add dedicated bundle smoke tests (no corpus coverage today). Both gates **701/0**. -- **S5a DONE — `build_options` + `set_post_link_callback` migrated off `#compiler` onto `abi(.compiler)`; `BuildConfig` threaded into the VM (2026-06-18).** - The corpus-covered slice of the BuildOptions migration. (1) `comptime_vm.zig` — `Vm.build_config: ?*BuildConfig`, - threaded via a new `tryEval` param (`&self.build_config` from emit_llvm's `#run`/const-init sites; `null` at - lowering-time type-fn). (2) Two `callCompilerFn` arms: `build_options` (returns the null-sentinel handle) + - `set_post_link_callback` (reads the cb `func_ref`, stores `post_link_callback_fn` on the threaded `BuildConfig`). - (3) `compiler_lib.zig` — matching legacy `handleBuildOptions`/`handleSetPostLinkCallback` (gate-OFF dual path). - (4) `build.sx` — `build_options :: () -> BuildOptions abi(.compiler);` and `set_post_link_callback` EXTRACTED - from the `struct #compiler` as a free `ufcs (…) abi(.compiler)` (so `opts.set_post_link_callback(cb)` still - resolves via UFCS); the other ~38 BuildOptions methods stay `#compiler` for now. (5) Registrars/callbacks that - call these are now compiler-domain: `platform/bundle.sx` `bundle_main :: () -> bool abi(.compiler)`, and the - six platform examples' `configure`/`configure_build` registrars marked `abi(.compiler)`; 0602/0603 reworked - the same way. **KEY learning:** every example transitively imports `build.sx` via the prelude, so the - `set_post_link_callback` method→free-function change is BENIGN `.ir` churn (declaration renumber + global - `@str`/`@tag.str` suffix shift) in all 37 examples that have `.ir` snapshots — verified line-by-line that NO - instruction/control-flow/payload changed (only auto-numbered global-name suffixes), then regen'd those 37 - snapshots scoped with `-Dname`. **Strict-VM `compiler_call` bail set dropped 6→2:** 0602/0603/1604/1611 now - fully VM-HANDLED; 1609/1615 still bail on the *other* (still-`#compiler`) BuildOptions methods they use → - **S5b** (migrate the remaining ~38 setters/getters). **701/0 BOTH gates + all unit tests.** -- **S3 DONE — emit_llvm skips BODIED `abi(.compiler)` (compiler-domain) functions; comptime-only calls emit `undef` (2026-06-18).** - A BODIED `abi(.compiler)` function is a user compiler-domain function (post-link callback / compiler helper): - the comptime evaluator runs its sx body, but it NEVER runs in the binary, so the backend skips it. Changes: - (1) IR `Function` gained `is_compiler_domain: bool` (`inst.zig`). (2) `decl.zig` — new `fnIsBodilessCompiler` - splits the API surface (bodiless → declare-only, `compiler_welded`, no implicit ctx — the S1 behavior) from a - bodied `abi(.compiler)` function (lowers its body for VM eval; flagged `is_compiler_domain` + `is_comptime`; - gets normal implicit-ctx). The four S1 guards now gate on `fnIsBodilessCompiler` not `fd.abi == .compiler`. - (3) `emit_llvm.zig` — Pass 2 skips `is_compiler_domain` bodies; Pass 1 declares them EXTERNAL-linkage (an - internal empty decl fails LLVM verification). (4) **KEY** `ops.zig` `emitCall` — a call to a comptime-only - callee (`compiler_welded` OR `is_compiler_domain`) from a dead comptime body now emits `undef` instead of a - real `call`; the runtime-call gate covers both. Without the undef, an AOT `sx build` left an undefined - `_double`/`_intern` symbol — this ALSO fixed a pre-existing, untested AOT breakage of the bodiless - compiler-API examples (the corpus runs them JIT). Diagnostic reworded "compiler-library" → "compiler-domain" - (1185 snapshot regen'd). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` - const → 84, absent from the binary via `nm`, JIT + AOT both run). **701/0 both gates + all unit tests.** - **NEXT: S4** — an `abi(.compiler)` function-TYPE param (`cb: () -> bool abi(.compiler)`) flags the bound - function compiler-domain (so a plain `bundle_main :: () -> bool { … }` becomes compiler-domain when passed to - `set_post_link_callback`). Then S5 (BuildOptions migration + delete `#compiler`/`compiler_call`/`compiler_hooks`). -- **S1+S2 DONE — `abi(.compiler)` replaces `abi(.zig) extern compiler` + `#library "compiler"` (clean cutover, no legacy path) (2026-06-18).** - Per the design pivot below, and the user's "no legacy paths": REMOVED the `.zig` ABI variant entirely (`ast.ABI` - is now `{ default, c, compiler, pure }`) and made `abi(.compiler)` the sole spelling for a compiler-domain / - compiler-API function — the ABI alone marks it, no `extern `, no fake `#library "compiler"`. Changes: - (1) `ast.zig` — `.zig` → `.compiler` (doc rewritten). (2) `parser.zig` — `parseOptionalAbi` accepts `.compiler` - (drops `.zig`); a **bodiless `abi(.compiler)` decl** (ends in `;`, no `extern`) is now accepted — synthesizes - the empty-block placeholder like an `extern` import (the Zig/VM handler is the impl). (3) `decl.zig` — - `weldedCompilerFn` keys off `fd.abi == .compiler` + export-list membership (no `extern_lib == "compiler"` - check); a bodiless `abi(.compiler)` decl lowers extern-like (`is_extern_decl`, and the two body-lowering paths - `lowerFunction`/`lazyLowerFunction` skip it) so it is declared-not-defined; `funcWantsImplicitCtx` returns - false for `abi == .compiler` (an implicit `__sx_ctx` prepend would shift args and break the handler arity — - this was the live bug surfaced + fixed). (4) `type_resolver.zig` — the function-TYPE CC switch handles - `.compiler` (sx-default CC). (5) Migrated ALL 8 compiler-API examples (0626/0628/0629/0630/0631/0633 + the - 1184/1185 negatives) `… abi(.zig) extern compiler;` → `… abi(.compiler);` and deleted every `compiler :: - #library "compiler";` line; regen'd the 1184 stderr snapshot (new "not a function exported by the compiler" - wording + shifted line). (6) Updated the two parser unit tests. **All 8 examples run HANDLED on the strict VM - with byte-correct output; 1184 (unexported name) + 1185 (runtime call) still error cleanly; gate-OFF legacy - still works.** **700/0 BOTH gates + all unit tests.** NOTE: the general `#library`/`extern ` PARSE paths - stay (used by `libc :: #library "c"` etc.) — only the compiler-API's USE of them is gone. `compiler_lib.lib_name` - + the `main.zig` dlopen-skip for a "compiler" lib are now dead defensive code (harmless; a `#library "compiler"` - is just meaningless now). The struct-`abi(...)` parse slot is vestigial (weld stripped) — parse-only test kept. - **NEXT: S3** — emit_llvm skips BODIED `abi(.compiler)` functions (Pass 2, like `is_extern`); thread an - `abi(.compiler)` flag onto the IR `Function` and refine the three "today every `abi(.compiler)` fn is bodiless" - guards in `decl.zig` (marked with `S3 NOTE`) to allow a bodied callback's body to lower for VM eval while NOT - emitting it. Then S4 (callback-param propagation) + S5 (BuildOptions migration). -- **DESIGN PIVOT (2026-06-18, user) — `abi(.compiler)` is the compiler-domain ABI; DROP the fake `#library "compiler"`.** - Supersedes both the `abi(.zig) extern compiler` + `#library "compiler"` binding mechanism AND the previous - "runtime-reachability gating" idea for the BuildOptions blocker (entry below). **The unifying concept:** a - function is *compiler-domain* (runs in the comptime evaluator, NEVER in the shipped binary) because its **ABI - says so** — `abi(.compiler)` — not because it's "extern" to an imaginary library. One annotation covers - BOTH roles: - 1. **Compiler-API surface** (`intern`, `text_of`, `find_type`, `declare_type`, `register_type`, - `build_options`, `set_post_link_callback`, …): bodiless `abi(.compiler)` decls (the Zig/VM handler IS the - impl). Replaces `… abi(.zig) extern compiler;` + the `compiler :: #library "compiler";` line — both GO AWAY. - 2. **User compiler-domain functions** (post-link callbacks like `platform.bundle.bundle_main`): BODIED - `abi(.compiler)` functions. emit_llvm does NOT lower them (skip in Pass 2, like `is_extern`); the comptime - VM/interp evaluates them. A callback PARAM type carries it too — `set_post_link_callback(self, cb: () -> bool - abi(.compiler))` — so the bound function is flagged compiler-domain. - **Why this dissolves the BuildOptions blocker:** the welded-call enforcement (`ops.zig` `emitCall`) only fired - because comptime-only callback bodies (`bundle_main`, 0602's `configure`) were being LLVM-emitted. A bodied - `abi(.compiler)` function is never emitted → its `build_options()`/`binary_path()` calls never reach `emitCall` - as runtime code → no enforcement, no undefined-symbol risk. **1185 stays correct**: `main` is an ordinary - runtime fn (not `abi(.compiler)`) calling a compiler-domain fn → still a clean build-gating error. (The - registrar half is independently fine via the idiomatic `#run { … }` block — the welded calls sit in the - `is_comptime` `__run` wrapper; 0602/0603 only tripped via an intermediate `configure()`, a test-shape artifact.) - **Staged plan (each its own step, both gates green):** - - **S1 — introduce `abi(.compiler)`** as a new `ABI` variant that marks a function `compiler_welded` (export-list - checked) WITHOUT requiring `extern compiler`/`#library`. Add it ALONGSIDE the existing `.zig extern compiler` - path so migration is incremental; prove with one example (0626 → `abi(.compiler)`). (`.zig` is a misnomer — - "we don't really have a zig abi"; it becomes `.compiler`, ultimately replacing `.zig` once all callers move.) - - **S2 — migrate the rest of the compiler-API decls** (0628–0633, 1184/1185) to `abi(.compiler)`; drop the - `#library "compiler"` lines; regen snapshots (the 1184 unexported-name + 1185 runtime-call diagnostics must - stay red with refreshed wording). Then retire the `.zig extern compiler` parse path + `#library "compiler"`. - - **S3 — emit_llvm skips bodied `abi(.compiler)` functions** (Pass 2 `continue`, like `is_extern`); thread the - `abi(.compiler)` flag onto the IR `Function`. Prove a bodied compiler-domain function isn't emitted. - - **S4 — callback-param propagation**: an `abi(.compiler)` function-type PARAM flags the bound function - compiler-domain. - - **S5 — BuildOptions migration** (now unblocked): `build_options`/`set_post_link_callback`/… become - `abi(.compiler)` (+ VM `callCompilerFn` arms / legacy `compiler_lib` handlers; `BuildConfig` threaded into the - VM — the bundler 4E shares this); callbacks declared/typed `abi(.compiler)`; delete `#compiler`/`compiler_call`/ - `compiler_hooks` Registry. Then **4E** bundler on the VM. - **Reusable facts from the reverted attempt:** only `build.sx` uses `#compiler`; VM dual-path bail-to-fallback - means the VM needs only corpus-covered fns; UFCS on a free fn needs the `ufcs` marker (composes with the ABI - annotation); the binding mechanism currently lives in `decl.zig` `weldedCompilerFn` (keys off `extern_lib == - "compiler"` — S1 makes it key off `abi == .compiler`). Mechanism files: `ast.zig` (`ABI` enum), `parser.zig` - (`parseOptionalAbi` + the extern-compiler postfix), `decl.zig` (`weldedCompilerFn`), `compiler_lib.zig` - (export list), `comptime_vm.zig` (`callCompilerFn`), `emit_llvm.zig` (Pass-2 skip), `ops.zig` (`emitCall` gate). -- **Phase 4 — BuildOptions→`abi(.zig) extern compiler` migration ATTEMPTED, then REVERTED; BLOCKER found: the comptime-only welded-call enforcement (2026-06-18).** - Scoped an incremental slice (migrate only the corpus-covered `build_options()` + `set_post_link_callback`, - leaving the 38 bundler accessors on `#compiler` → VM bails → legacy fallback). Built it end-to-end: - threaded `BuildConfig` into the `Vm` (`tryEval` gained a `?*BuildConfig` param, passed `&self.build_config` - from emit_llvm's `#run`/const-init sites); added `callCompilerFn` arms + legacy `compiler_lib` bound-handlers - for both; rewrote `build.sx` (`build_options` → `abi(.zig) extern compiler`; extracted `set_post_link_callback` - out of the `struct #compiler` as a free `ufcs (...) abi(.zig) extern compiler` fn so `opts.set_post_link_callback(cb)` - still resolves via UFCS; added `compiler :: #library "compiler";`). All COMPILED and the welded dispatch - fired. **BLOCKED at LLVM emission, NOT a bug — a design limitation the migration surfaces:** a - `compiler_welded` call inside a NON-`is_comptime` function is a hard build-gating error (`ops.zig` - `emitCall`, the Phase-1 enforcement guarding genuine runtime misuse — example 1185). But the post-link - callback idiom calls comptime-only-API functions (`build_options()`, `binary_path()`, `bundle_path()`, …) - **inside callback bodies** (`platform/bundle.sx`'s `bundle_main :: () -> bool`, and 0602's `configure`) that - run ONLY at comptime (post-link interp/VM) yet are still LLVM-emitted as real `() -> bool` bodies. The OLD - `#compiler`/`compiler_call` path emitted those as dead `undef` (`emitCompilerCall`), so no error; the welded - enforcement instead halts the build, and it CANNOT distinguish a dead comptime-reachable body from genuine - runtime use (1185, reachable from `main`) without runtime-reachability analysis. **Reverted the whole attempt** - (kept only the green pure-ops work); both gates back to **700/0**. **THE DECISION the next session must make - FIRST (before any BuildOptions migration):** how to emit a welded call in a comptime-only-but-LLVM-emitted - function. Recommended path **A — runtime-reachability gating:** in `emit_llvm`, mark functions reachable from - runtime roots (`main` / exported runtime fns); a welded call in an UNREACHABLE function emits `undef` (dead, - like `compiler_call` did) instead of erroring, while a reachable one still errors (1185 stays red). This is - also the right foundation for eventually NOT emitting comptime-only bodies at all. Rejected: (B) marking - callbacks `is_comptime` — can't statically identify which `func_ref`s become post-link callbacks; (C) blanket - softening to `undef` — would silently swallow genuine runtime misuse (1185). **Other migration facts confirmed - this attempt (reuse next session):** only `build.sx` uses `#compiler` (the `issues/*.md` hits are doc text); - the VM dual-path bail-to-fallback means the VM needs only the corpus-covered fns, the 38 bundler accessors can - ride legacy; UFCS on a free fn requires the `ufcs` marker, which composes with `abi(.zig) extern compiler`; - `build.sx` must declare `compiler :: #library "compiler";`. Do the reachability fix as its OWN step (verify - 1185 still errors + a comptime-only-body welded call now emits clean), THEN redo the BuildOptions slice on top. -- **Phase 4 burndown — three PURE comptime ops ported (`error_tag_name_get` + `global_addr` + `type_is_unsigned`); `interp_print_frames` correctly DEFERRED (2026-06-18).** - Also ported `type_is_unsigned` (a `BuiltinId` via `callBuiltinVm`): resolves the queried `TypeId` the - same way as `type_name` (a `.type_value` word, or an Any box `{tag@0,value@8}` whose tag IS the boxed - value's type) then returns `table.isUnsignedInt(tid)`. Extracted the shared resolution into a - `reflectArgTypeId` helper (VM-native `Value.reflectTypeId` mirror) so `type_name` + `type_is_unsigned` - can't drift. MATCH-verified by a new VM unit test (`type_is_unsigned(u32) - type_is_unsigned(i64) == 1`). - Strict sweep: 0600 `type_is_unsigned`→`out` (now its only remaining bail); no `type_is_unsigned` bails - remain in the corpus. **With this, all PURE comptime ops are ported** — the remaining strict bails are - side-effect (`out`/`interp_print_frames`), `compiler_call` (the BuildOptions migration), VM diagnostics - (1179/1180), and `#insert`/bundler. - Ported two side-effect-free ops onto the VM (`comptime_vm.zig` exec switch): (1) `error_tag_name_get` - — a runtime tag-id word → its name string via `table.getTagName` + `makeStringValue` (uses the table, - not the module, so it's unit-testable; `self.table == &module.types`); (2) `global_addr` — name-matches - `__sx_default_context` and returns the already-tested `materializeDefaultContext` Addr (an aggregate - value IS its address, so a downstream `load` sees the materialised Context), bailing for any other - global exactly like legacy. **MATCH verification:** `error_tag_name_get` locked in by a new VM unit - test (tag id → `"Bad"`, via `regToValue`); `global_addr` proven by the strict sweep (0600's first bail - moved past it) and reuses `materializeDefaultContext`, already exercised by every implicit-ctx comptime - call on the VM. **KEY CORRECTION to the handover's "three PURE ops" plan:** `interp_print_frames` - (1034) is NOT pure — it WRITES the comptime call-frame chain to the build output, a side effect in the - SAME bucket as `out` (the VM has no output buffer; output is direct-write, so a print-then-bail - double-prints under the legacy fallback). It must land atomically in the FINAL `out`/strict-default - step, NOT now. **Strict-sweep burndown:** 1035 `error_tag_name_get`→`out`; 0600 `global_addr`→ - `type_is_unsigned` (a NEW pure-op bail surfaced — still a known pure op, next to port); 1034 stays at - `interp_print_frames` (deferred, as it should). Also fixed the stale `comptime_vm.zig` header comment - (it still said "bump/stack allocator"; the memory model is an ARENA of stable host allocations since - 4D.0). **700/0 BOTH gates + all unit tests.** On `reify`. -- **Phase 4 burndown — issue 0143 FIXED (pack-as-`[]Type` stride) + regression test (2026-06-18).** - Root cause was a stale consequence of the `.type_value` migration: `buildPackSliceValue` - (`lower/pack.zig`) materialized a bare `$` `[]Type` slice as `[]Any` (16-byte elements) while - `const_type` now yields an 8-byte `.type_value` and `[]Type` resolves to `[]type_value` — so 8-byte - words sat in 16-byte slots and an 8-byte-stride reader got `[t0, pad, t1, …]`. Fixed by building the - array+slice as `.type_value` (8 bytes). Removed the stopgap `type_name` `.unresolved` guard (its - whole reason is gone; dropping it keeps any future stride bug VISIBLE as wrong output rather than a - silent fallback). Sibling `materialisePackSlice` checked — it genuinely boxes values into `[]Any` - (correct, not the same bug). Regression test `examples/0525-packs-pack-as-type-slice-arg`. **700/0 - both gates.** 0114 (and 0521/0522/0524) now bail ONLY at `out` (the deferred end-state op) — the - type bug is gone. issue 0143 RESOLVED. -- **Phase 4 burndown — switch_br + type_name ported; issue 0143 filed; KEY sequencing insight: `out` is end-state-only (2026-06-18).** - Ported two PURE comptime ops (`379ed05`): `switch_br` (i64-discriminant multi-way branch — enum/error - tag or `.type_value` index) and `type_name` (Type value / Any box → `table.typeName`, with an - `.unresolved`-bail guard). Correct in isolation; 0520–0524 run GREEN under strict. **Two blockers found:** - 1. **issue 0143 (FILED, OPEN) — pack-as-`[]Type` stride mismatch.** A `..$args` pack forwarded as a - `[]Type` ARGUMENT across a call is backed by a `[N x Any]` (16B) array but viewed as `[]type_value` - (8B) → half-stride reads (`[i64 string]` vs legacy `[i64 string bool]`). A LOWERING bug - the legacy's Value model masks; the byte-accurate VM exposes it. Blocks `examples/0114` from running - HANDLED. **Per CLAUDE.md: filed, NOT worked around** (the `type_name` `.unresolved` guard just makes - the VM decline rather than emit garbage). Repro + fix-prompt in `issues/0143-…md`. - 2. **`out` (comptime print) is an END-STATE op — it cannot land while the fallback exists.** Under the - legacy fallback, an eval that prints via `out` then BAILS double-prints (the VM wrote to fd 1, then - legacy re-runs the whole eval — no rewind). 0114 demonstrated it. So a direct-write `out` is only - safe once the fallback is GONE (strict-by-default). **Revised ordering:** land the PURE ops - (switch_br/type_name/type_is_unsigned/error_tag_name_get/global_addr/interp_print_frames) + the - BuildOptions migration + #insert + bundler FIRST; then in the FINAL step flip strict-to-default - (removing the fallback) AND add `out` together — at which point every `out`-using example flips - atomically with deletion. (Most of the gap-list examples print, so they stay on fallback until that - final flip — that's expected, not a regression.) 699/0 both default gates. -- **Phase 4 — STRICT no-fallback mode (the interp-retirement enumeration gate) + full gap list (2026-06-18).** - Added `-Dcomptime-flat-strict` / env `SX_COMPTIME_FLAT_STRICT` (implies `comptime_flat`): at all - THREE comptime sites (type-fn in `lower/comptime.zig`, const-init + `#run` in `emit_llvm.zig`) a VM - bail becomes a build-gating error naming the reason INSTEAD of falling back to legacy. This forces - every comptime eval onto the VM so the complete gap set is enumerable in one sweep; when the corpus - is green under strict mode AND every example MATCHES legacy, the VM handles everything and - `interp.zig` can be deleted (4F). Default behaviour unchanged — **699/0 both default gates**. - (Fixed a wiring bug: the type-fn site's local `comptime_flat` didn't include the strict flag, so - every type-fn falsely reported ``; now strict implies flat there too.) - **THE DELETION CHECKLIST (19 strict bails, swept via `SX_COMPTIME_FLAT_STRICT=1` over examples+issues; - 0103/0800 "WRONG" were false positives — raw heap-pointer addresses the corpus normalizes):** - - `switch_br` (5): 0114, 0521, 0522, 0524, 1035 — port the type-category multi-way branch (trivial - jump). **CAUTION:** porting it (+`type_name`) UNMASKS a silent-wrong in 0114 — a `[]Type` slice - materialized when a pack (`$args`) is passed ACROSS A CALL reads its `string` element as - ``. Must fix that VM pack-Type-materialization bug, not just add the op. - - `compiler_call` (6): 0602, 0603, 1604, 1609, 1611, 1615 — the **BuildOptions → `abi(.zig) extern - compiler`** migration (delete `#compiler`/`compiler_call`; thread `BuildConfig` into the VM). Big. - - `out` (2): 0613, 1038 — comptime print. Direct write to fd 1, BUT only safe when the WHOLE eval is - VM-handled (a print-then-bail double-prints under the legacy re-run — 0613). Flip atomically. - - `type_name` (1): 0520 — reflection reader (`.type_value` word / Any-box tag → `table.typeName`). - - `global_addr` (1): 0600 — only `&__sx_default_context` is materialised (mirror legacy). - - `interp_print_frames` (1): 1034 — return-trace frame printing. - - VM-native diagnostics (4B) (2): 1179, 1180 — NEGATIVE tests; the VM bail (`define: enum has no - variants` / `duplicate variant name`) IS the expected outcome → must surface as the proper - build-gating diagnostic, not the generic strict error. - - dlsym not found (1): 1654 — a target-specific `extern` (asm global) called at comptime; likely a - legitimately-unresolvable case → confirm it stays a clean diagnostic. - **Sweep command:** `SX_COMPTIME_FLAT_STRICT=1 ./zig-out/bin/sx run ` per example, diff vs legacy; - a strict bail prints `... bailed on the VM (strict, no fallback): `. -- **Phase 4D.2 (VM plan) — extern SLICE/string args (→ NUL-terminated `char*`) + float guards (2026-06-18).** - Extracted `marshalExternArg`: a scalar/pointer WORD passes verbatim (a `cstring` arg already works - as a pointer word via 4D.1); a `string`/slice `{ptr,len}` fat pointer is copied into a - NUL-terminated arena buffer and its `char*` passed (mirrors legacy `marshalExternArg` — what the - bundler's `popen(cmd: [:0]u8, …)` needs). Added FLOAT guards on args AND returns: floats are - `kindOf == .word` but the host_ffi trampolines have no float variant, so they bail loudly rather - than miscall through an integer register (the legacy interp doesn't support float FFI either, so - parity holds — no corpus float-FFI example exists). New example `0637-comptime-extern-slice-arg` - (`#run strlen("hello, world")` with a `[:0]u8` param → 12) runs **HANDLED on the VM**, byte-matching - legacy. **699/0 BOTH gates.** On `reify`. The FFI escape is now complete for scalar/pointer/cstring/ - slice args + scalar/pointer returns — enough for the bundler's libc surface. **Next (4D.3):** - `compiler_call` (#compiler hooks — 0602/0603), the last legacy-only role besides #insert/bundler. -- **Phase 4D.1 (VM plan) — general host-FFI escape: the VM calls any extern libc fn via dlsym + host_ffi (2026-06-18).** - Replaced the "extern not ported → bail" stub in `Vm.invoke` with `callHostExtern`: resolve the - symbol via `host_ffi.lookupSymbol` (dlsym RTLD_DEFAULT) and dispatch through the `host_ffi` - trampolines, exactly like the legacy `interp.callExtern`. **Marshalling is now trivial because - `Addr` is a real host pointer (4D.0):** every WORD-kind arg passes as `usize` verbatim — a - scalar's bits OR a pointer, no translation — and a pointer return is a valid `Addr`. Picks - `callPtrRet` (void*-ABI) for pointer-ish returns, `callIntRet` (i64-ABI) otherwise; honors - variadic (`is_variadic and args > fixed`). Non-word (aggregate/string/float) args+returns bail - loudly (no silent miscall — 4D.2 adds NUL-term cstring marshalling + float). NOT per-builtin: ONE - general mechanism for all externs. New example `0636-comptime-extern-libc` (`#run toupper(97)`/ - `tolower(90)` fold to 65/122) runs **HANDLED on the VM**, output byte-matching legacy. (`abs` - doesn't dlsym-resolve on macOS — a compiler builtin — and the VM fails identically to legacy, - confirming parity.) **698/0 BOTH gates** (one new example). On `reify`. **Next (4D.2):** - string/aggregate extern args (string→NUL-term cstring) + float args/returns, then `compiler_call` - (#compiler hooks, 4D.3). -- **Phase 4D.0 (VM plan) — comptime VM memory = an ARENA of stable host allocations; `Addr` = real host pointer (2026-06-18).** - Replaced the growable `ArrayList(u8)` flat buffer (which reallocs/MOVES on growth) with a - `std.heap.ArenaAllocator`: each `allocBytes` is a separate arena allocation that never moves and - is freed wholesale on `deinit` (no per-object free, no cap, no fixed buffer). **`Addr` is now the - allocation's absolute host pointer** (`@intFromPtr`), not an offset — so a comptime pointer and - an FFI-returned host pointer are the SAME kind of value, and the FFI bridge (4D.1) can pass them - to/from libc with ZERO translation and no per-call pinning (the original moving-buffer hazard is - gone by construction). `Machine.readWord/writeWord/bytes` deref the absolute pointer directly, - keeping the null-check bail (the malformed-IR / null-deref safety contract). Dropped the - offset-based upper-bounds check (can't bound an absolute pointer; the `Frame.bad_ref` guard still - catches the dominant malformed-IR vector) and the test-only `mark`/`reset` (the arena has no - cheap reset-to-mark; the VM never used them outside tests). Decision rationale (user): use a - GPA-like allocator, no artificial buffer limits. **697/0 BOTH gates + all unit tests** (rewrote - the two Machine tests: null-deref bail + arena-stability-across-grows). Pure refactor, no - comptime behavior change. **Next (4D.1):** extern-call dispatch in `Vm.invoke` — marshal args - (scalars by value, pointers as the host pointer they already are), call via `host_ffi` - trampolines, return scalars/pointers; a new `#run` libc example as the corpus guard. -- **Phase 4A.1 (VM plan) — `box_any`/`unbox_any` on the VM + `.any` as a 16-byte aggregate (2026-06-18).** - Ported the Any-boxing conversion pair: `box_any` allocates the 16-byte `{ type_tag@0, value@8 }` - box (tag = source TypeId index, matching the legacy comptime interp), writing a word source's - scalar via `writeField(source_type)` (so f32 round-trips) or an aggregate source's comptime - ADDR (the runtime pointer-in-value-slot shape); `unbox_any` reads the value slot back (word → - `readField`, aggregate → the stored ADDR). **Required making `.any` a first-class comptime - aggregate** (it was `kindOf → .unsupported`): `kindOf(.any) = .aggregate` (16B, by-address) + - `fieldOffset` special-cases `.any` to the `{@0, @8}` layout (shared with string/slice) — without - the latter, a `struct_get` on an Any panicked (`union field 'struct' while 'any' is active`), - caught + fixed (no crash; "never crash" upheld). Updated two unit tests that used `unbox_any` as - the "unported op" example → now `compiler_call`; added a box→unbox round-trip test. **697/0 BOTH - gates + all unit tests.** On `reify`. The 6 box_any examples (0114/0520–0524/1035) no longer bail - at box_any and produce VM output byte-matching legacy, but are not YET fully HANDLED — they now - fall back further at `switch_br` (comptime Any-tag type-switch), `type_name`, and `out`/print - (4A.2+/later steps). **Next (4A.2):** comptime `out`/print (VM output buffer + flush). -- **Phase 3 P3.4 step 8 (VM plan) — VM-native `type_info` REFLECTION → the whole metatype surface is HANDLED (2026-06-18).** - Ported `type_info($T)` into the VM (`callBuiltinVm` `.type_info` arm → new `buildTypeInfo`), the - inverse of step 7's `define`: reflect a type INTO a `TypeInfo` VALUE built in FLAT MEMORY (the - VM-native mirror of legacy `reflectTypeInfo`). Decodes the source type into a tag + members - (tagged-union/struct field & enum variant → `{ name, ty }`, a payloadless variant → `void`; - tuple → bare positional `Type`s), then lays out the nested value bottom-up using layouts derived - from the `TypeInfo` RESULT type (`ins.ty`, now threaded into `callBuiltinVm`): element array → - `{ptr,len}` slice → info struct (`EnumInfo`/`StructInfo`/`TupleInfo`) → `TypeInfo { tag, payload }` - tagged union (reusing step 7's tagged-union write). Variant/field names materialize via a - `makeStringValue` helper extracted from `text_of`. Same `backing_type` guard as step 7. **Result: - the ENTIRE metatype surface runs HANDLED on the VM with ZERO fallback** — `0614`–`0624` + `0632` - (0616 `field_type` folds at lower time, no comptime eval); the `define(declare, type_info(T))` - round-trips (`0619`/`0622`/`0623`) mint byte-identical copies on the VM. VM output byte-matches - legacy for all. **697/0 BOTH gates + all unit tests.** On `reify`. **Remaining VM fallbacks in the - comptime corpus are now genuinely-non-metatype** emit-time side effects: `print`/`out` (0613), - `global_addr` (0600), `compiler_call` #compiler hooks (0602/0603), and the inline-asm global - (1654). **Next:** port those (or confirm each is a legitimately-non-comptime case) to drive the - fallback list to empty, then — with user go-ahead — flip the VM to default + delete `interp.zig`. -- **Phase 3 P3.4 step 7 (VM plan) — VM-native metatype CONSTRUCTION: `declare`/`define` + tagged-union `enum_init` (2026-06-18).** - Ported the metatype type-CONSTRUCTION builtins into the VM so the construction examples run - HANDLED end-to-end (no `call_builtin` fallback). Three pieces: (1) **tagged-union `enum_init` - with payload** — the arm previously bailed; now allocates the value (zeroed), writes the tag at - offset 0 (`{ header(tag)@0, [N x i8] payload@tag_size }`, the LLVM `backend/llvm/types.zig` - layout) and copies the payload at `tag_size`. (2) A **`.call_builtin` exec arm** → new - `callBuiltinVm`, the VM-native mirror of the legacy `execBuiltinInner`: `declare(name)` mints an - empty forward nominal slot (shared `declareNominal` helper, also used by `declare_type`); - `define(handle, info)` reads the `TypeInfo` tagged-union VALUE from FLAT MEMORY (tag@0, active - payload `EnumInfo`/`StructInfo`/`TupleInfo` struct at `tag_size`, its single slice field) and - mints via `defineFromInfo`, a faithful port of legacy `defineEnum`/`defineStruct`/`defineTuple` - (all-void enum → real `.@"enum"` per issue 0142, dup-name rejection, `updatePreservingKey` vs - `replaceKeyedInfo`). (3) Refactored the `[]{name,ty}` decode out of `registerTypeVm` into a - shared `decodeMemberSlice` (+ `decodeTypeSlice` for bare-`Type` tuple elements), keyed to the - module-level `NamedMember`. Unmodeled builtins (`type_info`/`type_name`/…) return null → bail - with the builtin name → legacy fallback (dual-path parity). **Correctness guard (caught via - review):** `enum_init`/`define` assume a tag-headed layout, which is WRONG for a `backing_type` - tagged union (laid out as the backing struct) — both now bail loudly on `backing_type != null` - rather than silent-clobber. **Result:** examples `0614`/`0620`/`0621`/`0624`/`0632` run **fully - HANDLED** on the VM (define is the whole eval); `0622`/`0623` run define HANDLED then fall back - cleanly at the still-unported `type_info` reflection. VM output byte-matches legacy for all 7. - **697/0 BOTH gates + all unit tests (added: tagged-union `enum_init` payload layout).** On - `reify`. **Next:** port `type_info` (REFLECT a type → build a `TypeInfo` value in comptime memory, - the inverse — reuses the tagged-union `enum_init` write) so `0619`/`0622`/`0623` go fully HANDLED; - then the rest of the comptime corpus (drive the SX_COMPTIME_FLAT_TRACE fallback list toward the - genuinely-non-comptime cases) before the VM-default flip + legacy deletion. -- **Phase 3 P3.4 step 6 (VM plan) — REAL lowering-time Context: allocating + List-building type-fns now run HANDLED on the VM (2026-06-18).** - The VM can now evaluate a comptime type-fn that ALLOCATES at lowering time (the 0141 family) — - the legacy interp cannot. Four changes: (1) `runComptimeTypeFunc` (lower/comptime.zig) FORCES the - CAllocator→Allocator thunks to exist (`getOrCreateThunks`, idempotent, guarded by Allocator/ - CAllocator registered) BEFORE eval — a type-fn const runs at scanDecls (Pass 1), before Pass 1c - builds the default-context global + thunks, so the comptime allocator was otherwise null; - (2) `materializeDefaultContext` builds a REAL context at lowering time when the global is absent — - finds the two thunks by name (`findFuncByName`) and lays their func-refs into the inline - `Allocator` value `{ctx=null, alloc_fn@+ptr, dealloc_fn@+2*ptr}` at the head of `Context`, so - `context.allocator.alloc_bytes` dispatches `call_indirect` → thunk → native VM `malloc`; - (3) `aggType` now DEREFS a pointer `base_type` (the List write path emits `struct_gep` with - `base_type = *Struct` — `fieldOffset` panicked on the pointer; now derefs to the pointee, no - panic); (4) `subslice` handles a `[*]T` many-pointer / `*T` base (a List's `items` field — the - base IS the data pointer). **Verified end-to-end (manual probe):** a compiler-API type-fn that - builds its `[]Member` in a `List(Member)` (`.append` ×3, then `register_type(handle, kind, - vs.items[0..vs.len])`) runs **HANDLED on the VM** and mints correctly (`green=7`) — the exact - 0141 List-growth pattern, on the VM. **Can't be a corpus test yet** (gate-OFF/legacy still can't - allocate at lowering time — the dual-path bind), so locked in via VM unit tests instead - (many-pointer subslice; `struct_gep` with a pointer `base_type`). **697/0 BOTH gates + all unit - tests, EXIT=0.** On `reify`. **Remaining for the original 0141 repro (uses metatype `define`/ - `make_enum` → `call_builtin` → legacy fallback → legacy fails):** re-express the metatype over the - compiler-API so the whole type-fn runs on the VM (no `call_builtin`). THEN the repro works on the - VM — and the dual-path bind resolves only at the VM-default-flip + legacy-deletion end-state. -- **Phase 3 P3.4 — investigation: the "real lowering-time Context" is BLOCKED by issue 0141 (2026-06-18).** - Probed whether the VM needs a REAL lowering-time `Context` (CAllocator thunk func-refs) for - allocating type-fns. **Finding: lowering-time comptime ALLOCATION fails in the LEGACY interp - too** — a type-fn that calls `context.allocator.alloc_bytes` at lowering time bails in legacy - with `comptime call_indirect: callee is not a func_ref Value (raw fn-pointers from extern calls - aren't dispatchable in interp)`, and the VM bails at parity (`call_indirect through a null - function pointer`). This is exactly issue **0141**'s root cause (its analysis already notes "the - null allocator is the same story for the CAllocator thunks") — an OPEN deferred issue. So: - (1) the VM is CORRECT (parity — both bail; no regression); (2) the real-context work is - PREMATURE — its only consumer (allocating lowering-time type-fns) can't pass gate-OFF, so no - corpus test can validate it, and even a more-capable VM can't ship a divergence during the - dual-path phase. **Consequence for the metatype re-expression:** re-expressing `define`/`make_enum` - over the compiler-API needs to BUILD `[]Member` slices dynamically (allocation) — which is - blocked by 0141 at lowering time. The viable paths are: (a) avoid allocation by passing the - caller's existing slice through (needs `EnumVariant`/`StructField` to be usable AS `Member` — - they're layout-identical `{string, Type}`, but distinct nominal types — a metatype-API decision), - or (b) wait for 0141. **No code change this step** (the VM already bails correctly). Recorded so - the next session doesn't re-derive it. 697/0 both gates unchanged. -- **Phase 3 P3.4 step 5 (VM plan) — WRITE side ported to the VM → FIRST HANDLED lowering-time type-fns (2026-06-18).** - Ported `declare_type` / `pointer_to` / `register_type` into `Vm.callCompilerFn`, mirroring the - legacy `compiler_lib` handlers (mint via `@constCast(table)` — the same mutable access the - read-side `intern` uses; the lowering-time mint target IS `&module.types`). `register_type` - reads the `[]Member` slice from FLAT MEMORY: threaded `ref_types` through `invoke` → - `callCompilerFn` so the slice's element type (`Member = {name: string, ty: Type}`) gives the - field offsets + stride; decodes each `{name, ty}` and branches on `kind` (1 struct · 2 enum · - 3 tagged_union · 4 tuple) exactly as legacy (dup-name / payload-on-enum rejections, idempotent - re-fill via `nominalIdentOf`). **Key unblock:** the synthesized comptime type-fn wrapper - (`createComptimeFunction`/`…WithPrelude`) was built with return type `.any` → `regToValue` - bailed at the VM↔legacy boundary; changed to `.type_value` (the legacy path reads via `asTypeId` - regardless, so no legacy change). **Result: the compiler-API write type-fns now run HANDLED - end-to-end on the VM at LOWERING time** — `0631` (register-graph: 2 HANDLED, A↔B cycle via - forward handles + `pointer_to`) and `0635` (multi-edge import: 2 HANDLED), parity-correct. They - run on the ZEROED lowering-time context (fixed `.[…]` member arrays, no allocation). The - metatype `make_enum`/`define` examples (`0632`) still fall back CLEANLY through - `call_builtin(define)` (the separate metatype path — re-expressing it onto the compiler-API is - the other half of P3.4). **697/0 BOTH gates + EXIT=0.** On `reify`. **Next:** (optional, deferred) - a REAL lowering-time Context (CAllocator thunk func-refs) for List-growing type-fns; and - re-express the metatype `define`/`make_enum` over the compiler-API to delete the bespoke interp - arms (the end-state: ONE evaluator). -- **Phase 3 P3.4 step 4 (VM plan) — model `.type_value` natively in the comptime VM (2026-06-18).** - The VM now HANDLES Type values instead of bailing: `kindOf(.type_value)` → `.word`; a new - `const_type` exec arm → the word `TypeId.index()`; `regToValue` maps a `.type_value` word back - to a `.type_tag` Value at the legacy boundary (`valueToReg` already mapped `.type_tag` → - index). Surfaced + fixed a VM PANIC (forbidden): `struct_init` assumed a `.@"struct"` result - type and union-access-panicked on an ARRAY literal (`EnumVariant.[ … ]`, reached now that Type - args no longer bail early) — it's the generic aggregate-literal op, so it now dispatches on the - result kind (struct / array / tuple) and BAILS loudly on anything else, never panics. **697/0 - both gates** (the make_enum type-fns now run further on the VM, then bail cleanly at the - `define`/`make_enum` `call_builtin` → legacy mints — no mutation before the bail, parity holds). - VM unit test added (const_type → word → regToValue → `.type_tag`). On `reify`. **Next (the - payoff):** port the WRITE side (declare_type / register_type / pointer_to) into - `Vm.callCompilerFn` + give the lowering-time path a REAL Context (CAllocator thunk func-refs, - not zeroed) → the first HANDLED lowering-time type-fn end-to-end on the VM. -- **Phase 3 P3.4 step 3 (VM plan) — dedicated `Type` builtin TypeId: RESOLVER FLIPPED + `.any` migration (2026-06-18).** - Flipped `type_resolver:64` (`"Type"` → `.type_value`), `module.zig` `constType` (result type - → `.type_value`), and `emitConstType` (a bare i64 carrying `tid.index()`, NOT a 16-byte Any - box). Then migrated every `.any` reference that means "a Type value", classified per CLAUDE.md - (leave the real boxed-Any refs): (a) the "Any holds a Type" **meta-marker tag** moved `.any` → - `.type_value` at all 4 consumers — `reflectArgTypeId` (LLVM), `reflectTypeId` + the - `.type_tag`-as-struct-field comptime path (interp), and `resolveTypeCategoryTags("type")` - (generic.zig); (b) reflection-builtin RETURN types `.any` → `.type_value` (`type_of`/`declare`/ - `define`); the runtime `type_of(any)` now reads the tag AS a `.type_value` (no re-box); (c) - expr_typer infers a bare type-name expr as `.type_value` (with a `is_raw` backtick exemption — - `` `string `` is a value, never the reserved type); (d) `reflectionArgIsType` accepts - `.type_value` OR `.any` (a reflection arg can be a bare Type OR a boxed Any — the over-narrow - `==.type_value` was the catastrophic-regression cause, caught + fixed); (e) the comptime - `switch_br` accepts a `.type_tag` discriminant (type-category match); (f) a bare function name - in a `Type` slot now lowers to `const_type(its real function type)` instead of a func-ref - (fixed a JIT crash — was a func-ref word read as a TypeId), keeping the old string-box path only - for genuine `Any` params; (g) the field-not-found diagnostic + `formatTypeName` render - `.type_value` as "Type". Fixed 3 unit tests asserting the old `.any` Type behavior. - **697/0 BOTH gates** + all 494 unit tests (EXIT=0). Gate ON stays green because the VM's - `kindOf(.type_value)` → `.unsupported` → bails CLEANLY to legacy (no silent-wrong) — the VM - doesn't model `Type` values YET (next step), but parity holds. Regenerated 24 snapshots (22 - `.ir` const_type-shape; 2 `.stderr` Any→Type — diff reviewed, only the intended changes). On - `reify`. **Next:** model `.type_value` natively in the VM (`kindOf` → word, `const_type` → word - = `TypeId.index()`, `regToValue` word → `.type_tag`) for COVERAGE, then port the WRITE side into - `callCompilerFn` + a real lowering-time Context → the first HANDLED lowering-time type-fn. -- **Phase 3 P3.4 step 2 (VM plan) — dedicated `Type` builtin TypeId: FOUNDATION landed (dead/additive) (2026-06-18).** - Added `TypeId.type_value` (slot 19) + a matching `TypeInfo.type_value` variant + the builtins - init entry — an **8-byte type handle distinct from the 16-byte boxed `.any`** (THE WALL). All - `types.zig` layout handlers wired: `sizeOf`/`typeSizeBytes` → 8, `typeAlignBytes` → 8, - `typeName` → "Type", `hashTypeInfo`/`typeInfoEql` no-payload arms. Only ONE exhaustive switch - needed a new arm (`backend/llvm/types.zig` `toLLVMTypeInfo` → `cached_i64`); every other - `switch(TypeInfo)` site has an `else` (audited when the resolver flips). **`first_user` 19 → 100** - (per the user): slots 20–99 are RESERVED builtin headroom (infos padded with the `unresolved` - tripwire), so future builtins don't renumber user TypeIds / churn `sx ir` snapshots. Cost: - ~80 default entries in each binary's per-type reflection arrays (user opted in). **Still dead:** - `type_resolver.zig:64` STILL returns `.any` for "Type" — nothing produces `.type_value` yet, so - NO behavior change. Regenerated 22 IR snapshots (pure TypeId renumber to 100-base; `git diff - --name-only` confirmed ONLY `.ir` files + the 2 source files changed — no stdout/stderr/exit). - **697/0 both gates** (OFF and `-Dcomptime-flat`). **Next:** flip `type_resolver:64` → - `.type_value`, then migrate the `.any` refs that mean "a Type value" (const_type result / - reflection returns / metatype `Type` params / `.type_tag` checks) — leave the real boxed-Any - refs — file-by-file with a build after each. -- **Phase 3 P3.4 step 1 (VM plan) — lowering-time default context; first blocker cleared (2026-06-18).** - `materializeDefaultContext` now falls back to a ZEROED `Context` (found by name) when the - `__sx_default_context` global is absent — i.e. at LOWERING time, where the global isn't - emitted yet. A type-fn that never touches the allocator now runs past context setup; one - that allocates reads a null `alloc_fn` (zeroed) → `call_indirect` on the null func-ref - bails → legacy fallback (a REAL lowering-time context with the CAllocator thunk func-refs, - so allocating type-fns also run on the VM, is a follow-up). **Measurement: the bail moved - deeper** — metatype `make_enum` now bails at `const_type` (the `Type`-literal op, unported); - `register_type` type-fns bail at the welded write call (declare_type/register_type aren't in - `callCompilerFn`). No table mutation happens before either bail (the write fns bail before - minting), so parity holds: both gates **697/0**, no crashes. **Next blockers (the "model - Type" chunk):** (a) the `const_type` op → a word = `TypeId.index()`; (b) the Type-return - bridge (`regToValue` for a `Type`/`.any` word → `.type_tag`); (c) the VM-native write side - (declare_type/register_type/pointer_to in `callCompilerFn`) + a real lowering-time context. - Only once those land does a type-fn actually run end-to-end on the VM (a HANDLED case). -- **Phase 3 P3.4 (VM plan) — wire the VM at the LOWERING-time site + measure (2026-06-18).** - Routed `runComptimeTypeFunc` (the type-fn fold — the THIRD comptime call site) through - `comptime_vm.tryEval` behind `-Dcomptime-flat`/`SX_COMPTIME_FLAT` with legacy fallback, - mirroring the two emit-time folds. Extracted the shared post-check (`checkComptimeTypeResult` - — the declared-but-never-defined zero-field guard) so both paths use it. **Measurement - (SX_COMPTIME_FLAT_TRACE):** every metatype/compiler-API type-fn currently bails CLEANLY - with `no __sx_default_context global to materialize the implicit context` — at lowering - time the default-context global doesn't exist yet (it's built at emit time), so the VM bails - at context materialization, BEFORE running the body (no partial mint, no crash → legacy - mints). The hardening holds: **no crashes** across the corpus on the VM lowering-time path. - Both gates **697/0**. **So the FIRST lowering-time blocker is the implicit context, not - `Type` modeling** — the VM needs a way to materialize/skip the default context at lowering - time (most type-fns get an implicit ctx for potential `List`-growth alloc; many don't use - it). Next: materialize a lowering-time default context for the VM (or pass a null ctx + - bail only if the allocator is actually used), THEN model `Type` values + the VM-native write - side. This is near-pure fallback today — permanent scaffolding that lights up as those land. -- **Phase 3 P3.4-prep (VM plan) — harden the VM against malformed lowering-time IR (2026-06-18).** - Prerequisite for wiring the VM at the LOWERING-time comptime site (`runComptimeTypeFunc`), - where IR can be malformed (an unresolved name lowers to a dangling / `Ref.none` operand — - the 0737 crash). Closed the remaining panic vectors so the VM BAILS (→ legacy fallback) - instead of aborting: (1) a checked `Vm.refTy(ref_types, r)` replaces every raw - `ref_types[ref.index()]` in `exec` (the type-side companion to `Frame.get`'s `bad_ref` - value-side guard); (2) `aggType` is now a bailing method (`Error!TypeId`) using `refTy`; - (3) the block-dispatch loop bounds-checks the branch target before indexing - `func.blocks.items`. `global_get` was already guarded. No behavior change — gate OFF and - ON both **697/0**; unit test added (a `cmp_lt` with a `Ref.none` operand bails, not - panics). **Next:** wire `tryEval` into `runComptimeTypeFunc` behind the flag with legacy - fallback and measure (most minting type-fns will still bail at the welded-write call / - `Type`-result conversion until the VM models `Type` values + the VM-native write side land - — those are the steps that actually move lowering-time comptime onto the VM, toward - deleting legacy). -- **Phase 3 P3.3 (VM plan) — WRITE side: declare_type + pointer_to + ONE kind-branching register_type (2026-06-18).** - The mutating compiler-API: `declare_type(name) -> Type` (forward handle), `pointer_to(t) -> Type` - (build `*T`), and `register_type(handle, kind, members: []Member) -> Type` which branches on - `kind` IN THE COMPILER (subsuming define's per-kind dispatch). Take/return real `Type` values - (matching meta.sx declare/define). **Timing (per user): mint LAZILY at lowering time, single - pass** (the existing `runComptimeTypeFunc`), so the write side is **legacy-only** (`compiler_lib` - handlers) — the VM isn't wired at lowering time, no VM mirror needed; readers stay dual-path. - A non-generic `-> Type` builder is now flagged `is_comptime` (decl.zig) so its dead body permits - the welded calls. **Graph:** forward handles + `pointer_to` express mutually-recursive A↔B (`*A`, - `*B`, B-by-value); `register_type` is **idempotent** (re-fill a nominal slot reached via two - import edges — `nominalIdent`). `kind` codes match `type_kind` (1 struct · 2 actual `.@"enum"` · - 3 tagged_union · 4 tuple). **Fixed two bugs (issue 0142):** (a) a fully payloadless minted enum - was an all-void tagged_union → verifySizes panic; now a real `.@"enum"` (register_type kind 2 AND - metatype `defineEnum`); (b) bare `EnumType.variant` payloadless qualified construction wasn't - supported (failed for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`). - Examples 0631 (graph + actual enum + reflection), 0632 (make_enum all-void), 0633/0634/0635 - (namespaced / bare / multi-edge import of a minted type), 0187 (qualified variant construction). - **Parity 697/697** (gate ON and OFF); unit tests added. **Next (P3.4):** re-express - declare/define/type_info as sx over the compiler-API + delete the bespoke interp arms (needs the - VM hardened for lowering-time IR, or the metatype migrated onto the legacy compiler-API calls). -- **Phase 3 P3.2b (VM plan) — kind + enum-value readers: `type_kind` + `type_field_value`; READ side complete (2026-06-18).** - The last two read-only readers the metatype's `type_info(T)` needs (added to - `compiler_lib.bound_fns` AND `Vm.callCompilerFn`, each backed by a `TypeTable` query both - call): `type_kind(t) -> i64` (`kindCode` — a stable, compiler-owned discriminant: 0 other · - 1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · 7 vector · 8 error_set; - TOTAL, never bails) and `type_field_value(t, idx) -> i64` (`memberValue` — an enum variant's - explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for non-enum / - out-of-range). Example `0630-comptime-compiler-type-kind` reflects `Color` / `WindowFlags` - (flags) / `Point`. **The READ side is now COMPLETE** — `find_type` + `type_kind` + - `type_field_count` + `type_field_name`/`type_field_type`/`type_nominal_name` + - `type_field_value` cover everything `reflectTypeInfo` reads. VM unit test added. **Parity - 691/691** (gate ON and OFF). **Revised forward direction (per the user):** the WRITE side is - ONE `register_type(info)` fn that branches on the kind IN THE COMPILER (subsuming `define`'s - per-kind dispatch), not a per-kind `register_struct`. -- **Phase 3 P3.2 (VM plan) — field-level reflection readers: `type_nominal_name` + `type_field_name` + `type_field_type` (2026-06-18).** - Three more `compiler`-library readers on the same `TypeId`-handle shape (added to - `compiler_lib.bound_fns` AND `Vm.callCompilerFn`), each backed by a new `TypeTable` query - BOTH paths call (no drift): `nominalName` (a named type's own name handle; loud-bail for - unnamed types like `i64`/pointers), `memberName` (struct/union/tagged-union field, enum - variant, named-tuple element), `memberType` (struct/tuple/array/vector member type). All - loud-bail on out-of-range idx / no-member (no silent default). First MULTI-ARG compiler - fns — `callCompilerFn` reads arg 1 = idx; added `Vm.argHandle`/`argTypeId` (range-checked - u32/TypeId arg reads) and refactored `find_type`/`type_field_count` onto them. Named - `type_*` to avoid clashing with the std metatype builtins (`field_name`/`type_name` exist - in core.sx); `nominalName` (the TypeTable method) is distinct from the existing - `typeName(id) []const u8` display-string renderer. Example `0629-comptime-compiler-field-reflect` - reflects `Pair { lo: Point; hi: Point }` — each field name + the nominal name of a field's - type, all `#run`-folded, all VM-HANDLED natively. VM unit test added (type_field_name → "hi"; - type_nominal_name(type_field_type(Pair,0)) → "Point"). **Parity 690/690** (gate ON and OFF). -- **Phase 3 P3.1 (VM plan) — first read-only reflection readers: `find_type` + `type_field_count` (2026-06-18).** - Two more `compiler`-library fns, bound the same way as the `intern`/`text_of` seed (added - to `compiler_lib.bound_fns` for the legacy handler + the welded-decl export check, AND to - `Vm.callCompilerFn` for the native comptime path — NO marshaling). A **type handle is a - plain `u32` `TypeId`** (like `StringId`), so both keep the seed's clean scalar shape: - `find_type(name: StringId) -> TypeId` (`TypeTable.findByName`, `unresolved`/0 if absent) and - `type_field_count(t: TypeId) -> i64` (a NEW `TypeTable.memberCount` query — struct/union/ - tagged-union fields, enum variants, array/vector length — called by BOTH paths so they - can't drift; bails loudly, never a silent 0). New example `0628-comptime-compiler-find-type` - chains `intern → find_type → type_field_count` (and a not-found lookup → 0), both folded at - `#run`, both VM-HANDLED natively (trace confirms no fallback). VM unit test added - (`find_type` + `type_field_count`, struct found → 3 fields, missing → `unresolved`). - **Parity 689/689** (gate ON and OFF). **Decision (resolves the plan's `find_type → ?Type` - sketch):** return a NON-optional `TypeId` with the `unresolved` (0) sentinel for not-found, - NOT `?Type` — a `Type` value resolves to `.any` (which the comptime VM doesn't represent) - and an optional can't cross the legacy↔VM eval boundary; `unresolved` is the project-blessed - unmistakable "no type" marker. Forward (P3.2): more readers on the same handle shape - (`type_name`/`field_name`/`field_type`/kind), then `register_struct` (first mutating fn). -- **VM robustness — `Frame` bounds-check; lowering-time `#insert` wiring explored + reverted (2026-06-18).** - Explored wiring the VM at the LOWERING-time comptime site (`evalComptimeString`, the - `#insert` string fold). 12/13 `#insert` examples ran on the VM with parity, but `0737` - (an `#insert` of an unresolved `secret()`) CRASHED the VM (SIGABRT): lowering-time IR can - be malformed (a `ret Ref.none` from the unresolved name) and `Frame.get` panicked on the - out-of-range index. **Decision: reverted the lowering-time wiring** — unlike the emit-time - folds (fully lowered IR), lowering-time IR can be erroneous, and hardening the VM against - ALL malformed IR (every `ref_types[...]` / `aggType` access, not just `Frame`) is out of - scope here. The emit-time sites already give full corpus coverage. **KEPT** the defensive - fix regardless (CLAUDE.md "never crash"): `Frame.get`/`set` now bounds-check and flip a - `bad_ref` flag; the `run` loop bails (`badRef`) instead of panicking. Unit test added - (malformed `ret Ref.none` → bail, not crash). Parity **688/688** both ways. -- **Phase 3 SEED (VM plan) — compiler-call path: `intern`/`text_of` native on the VM (2026-06-18).** - `invoke` now dispatches a welded `compiler`-library fn (gated on `compiler_welded`) to - `Vm.callCompilerFn`, serviced NATIVELY on comptime memory (no legacy `Interpreter`): - `intern(string)->StringId` reads the comptime string bytes and `internString`s into the - const-cast table (pool-only — doesn't touch type layout, so cached sizes stay valid); - `text_of(StringId)->string` materializes the pooled text back into comptime memory. Unlocked - `0626`; the ONLY remaining const-init fallback is now the inline-asm global (`1654`). - Parity **688/688** (gate ON and OFF); unit test added. This is the mechanism Phase 3 grows - — the next compiler fns (`find_type`, `register_struct`, reflection readers) bind the same - way (comptime pointer in, handle/pointer out, no marshaling). -- **Phase 1.final step 9 (VM plan) — `-Dcomptime-flat` build flag (the "swap behind a build flag" step) (2026-06-18).** - Added the `-Dcomptime-flat` build option (build.zig → a `build_opts` options module on - `mod`; `emit_llvm.init` reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`). This is - the plan's "reach parity → swap behind a build flag → delete the old path" mechanism. - `zig build test -Dcomptime-flat` runs the FULL corpus on the VM (688/0). Verified the flag - toggles the binary: flag-built `sx` reports VM HANDLED with no env var; default-built does - not. Default OFF — `zig build test` unchanged (688/0). Env var still works for ad-hoc runs. - Next (forward): Phase 2 (bytecode) / Phase 3 (compiler-API on comptime memory); eventual - default-flip + legacy deletion. -- **Phase 1.final step 8 (VM plan) — wire the `#run` side-effect path + trace-clear-on-fallback (2026-06-18).** - Wired the SECOND comptime call site (`runComptimeSideEffects`, top-level `#run ;`) - through `tryEval` with legacy fallback, mirroring the const-init fold. `tryEval` now - handles void/noreturn entries (→ `.void_val`) so a void side-effect doesn't bail at the - result conversion. **Fixed a trace-corruption** the new site exposed (`1035`): a - side-effect that pushes return-trace frames and then bails (e.g. on `print`) had the - legacy re-run DOUBLE-push them (`sx_trace_push` is a side effect on the shared buffer). - Both wiring sites now `sx_trace_clear()` right before the legacy fallback, discarding the - VM's partial pushes. **Parity 688/688** (gate ON and OFF). Most side-effects still bail - (print/global_addr/call_builtin) → legacy, but the path is now uniform. All comptime - evaluation routes through the VM-with-fallback. -- **Phase 1.final step 7 (VM plan) — is_comptime + failable/error cluster + signed-load fix; coverage 31→36 (2026-06-18).** - `is_comptime` → 1 (unlocked `1030`). Ported the failable/error-channel cluster (`1037` - escape, `1038` handled): `kindOf(error_set)→word`, `regToValue` bridges TUPLES (the - failable `(value…,tag)` shape `checkComptimeFailable` reads), `trace_frame` packs - `(func_id<<32|span.start)` from a new `call_stack` (pushed by invoke/runEntry), and - `sx_trace_push`/`sx_trace_clear` serviced NATIVELY (the VM calls the real sx_trace.c - functions linked into the compiler, so the return-trace buffer is populated identically - to legacy). raise/catch/or now run on the VM. **Surfaced + fixed a real GENERAL bug:** - `readField` was ZERO-extending signed sub-64-bit loads, so a stored `i32 -1` reloaded as - `0xFFFFFFFF` (+4.29e9) and `< 0` was false — silently hiding `raise error.Bad`; now - SIGN-extends `i8`/`i16`/`i32`/`isize` (gate-ON parity confirms it's a strict fix; unit - test added). VM HANDLES **36** corpus const-inits (was 31); **parity 688/688** (gate ON - and OFF). Only **2 fallbacks** remain, both principled: `intern` (`0626`, welded - compiler-API fn — Phase 3) + inline-asm global (`1654`). Forward work: Phase 2 (bytecode), - Phase 3 (compiler-API on comptime memory). -- **Phase 1.final step 6 (VM plan) — real default context + call_indirect + func_ref + global_get; coverage 27→31 (2026-06-17).** - Per the user's direction ("the VM can set up a default context"), `runEntry` now - materializes the REAL default context instead of a zeroed one. The implicit-ctx param is - an opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global - and lays its initializer (`{ {null, alloc_fn, dealloc_fn}, null }`, the CAllocator thunk - func-refs) into comptime memory via a new recursive `layoutConst`. With `func_ref` (function - value encoded `FuncId.index()+1`, reserving word 0 for the null fn-ptr) and - `call_indirect` (decode word → FuncId → dispatch; 0 → bail) ported, the whole allocator - protocol runs on the VM: - `context.allocator.alloc_bytes` → call_indirect → thunk → `CAllocator.alloc_bytes` → - `libc_malloc` → native comptime malloc. Unlocked `0606` (string global). Also: `global_get` - lazily evaluates a comptime global's `comptime_func` (memoized) — unlocked `CT_CHAIN`; - field access (`fieldOffset`/`struct_get`) handles string/slice `{ptr@0,len@8}` fat - pointers (needed by `alloc_string`); `regToValue` maps function-typed words → `.func_ref` - (kept `1128`'s rejection byte-identical). Native `malloc` is still required (the thunk - bottoms out at it; a host pointer can't be used with comptime load/store). VM HANDLES - **31** corpus const-inits (was 27); **parity 688/688** (gate ON and OFF). Unit tests: - global_get, func_ref+call_indirect. Remaining fallbacks (7): `.unsupported` aggregates - (3× — `1037`/`1038`), extern/builtin `intern`+asm (2×), `trace_frame`, `is_comptime`. -- **Phase 1.final step 5 cont. (VM plan) — libc memory builtins + f32 fix; coverage 16→27 (2026-06-17).** - Identified the dominant fallback (`call to extern/builtin`) as **11× `malloc`** (0604) + - 1× `intern`. Modeled a curated set of libc MEMORY builtins natively on comptime memory - (`Vm.callMemBuiltin`): `malloc`/`calloc` → `allocBytes` (16-aligned, 256-MiB cap → bail), - `free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed (no host heap/dlsym), - target-aware; the computed result is byte-identical to legacy (which calls real libc). - This surfaced a **real latent f32 bug**: float registers hold f64 bits, but f32 MEMORY is - the 4-byte single — `readField`/`writeField` were truncating the f64 bits (writing zeros - for `1.0`); now they `@floatCast` on f32 load/store (mirrors legacy `storeAtRawPtr`). - Result: VM HANDLES **27** corpus const-inits (was 16); **parity 688/688** (gate ON and - OFF). Unit tests added (f32 round-trip; malloc → usable comptime memory). Next: the `kindOf` - `.unsupported` aggregates (3×), `global_get` (2×), the rest. -- **Phase 1.final step 5 (VM plan) — implicit-context materialization; coverage 0→16 (2026-06-17).** - `tryEval` now MATERIALIZES the implicit ctx instead of skipping it: a `has_implicit_ctx` - comptime entry (sole param `*Context`) gets a zeroed `Context` of the right size/align - in comptime memory, its address passed as arg 0. Const bodies that ignore the ctx run; a - body that uses the allocator hits unported `call_indirect` → bails → legacy. No func-ref - materialization needed (handled bodies don't read ctx contents; parity is the guard). - Fixed a real bug surfaced by the coverage pass: storing a `null` non-pointer optional - (the `null_addr` sentinel) into an aggregate slot OOB-bailed — `writeField` now ZEROES - the destination for a `null_addr` aggregate source (= none/empty); unit-test regression - added. Result: VM HANDLES **16** corpus const-inits (was 0); **parity 688/688 both - gate ON and OFF**. Next: port the ops the trace names — `call_builtin`/`compiler_call`/ - extern (13×, via the bridge), `kindOf` `.unsupported` aggregates (3×), `global_get` (2×), - func_ref / call_indirect / trace_frame / is_comptime. -- **Phase 1.final steps 1–4 (VM plan) — host wiring landed; coverage measured (2026-06-17).** - (1) **Hardening:** `Machine.readWord`/`writeWord`/`bytes` now return `error.OutOfBounds` - (null / out-of-range / oversized / overflow-safe) instead of `assert`-panicking; - `OutOfBounds` added to `Vm.Error`; `try` threaded through every helper + exec arm + the - bridge. New unit tests (accessor OOB returns; null-deref → `tryEval` null, not a crash). - (2) **Implicit context:** `tryEval` returns null for `has_implicit_ctx` funcs (legacy - fallback) — conservative; full ctx materialization deferred to step 5. (3) **Wiring:** - const-init fold in `emit_llvm.zig` `emitGlobals` is `(if comptime_flat) tryEval else - null) orelse interp.call(...)`, gated by env `SX_COMPTIME_FLAT` (read once into - `LLVMEmitter.comptime_flat`). Default OFF. (4) **Parity + coverage:** gate ON → full - corpus byte-identical (688, 0 failed) + manual 0605/0606/0607 byte-identical. - **Finding: 0 of 37 measured corpus const-inits are VM-handled — ALL are - `has_implicit_ctx`-gated.** Added a coverage-trace facility (`comptime_vm.last_bail_reason` - + env `SX_COMPTIME_FLAT_TRACE`). **Next: step 5 = implicit-context materialization** (the - unblocker), then port the deferred ops. 688 corpus green (gate OFF). -- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).** - `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on - the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back. - Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs - (1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of - crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind - `SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final. - 688 corpus green. -- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).** - Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the - wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs - the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg → - Reg, aggregates into comptime memory) + `regToValue` (VM result → Value, deep-copied). - Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip - unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through - the VM with legacy fallback). -- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).** - `Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals - arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared - comptime memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate - Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b). - Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next: - 1.5b (call_builtin/compiler_call/extern), then hybrid wiring. -- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).** - Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested - (deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops - (tagged-union payload/any/closures) blind — their byte semantics are ambiguous without - real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) + - HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so - the VM runs the real corpus and surfaces exactly what's needed. Key design point for - calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's - allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision - block. -- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).** - `kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}` - aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`; - const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91, - ?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any, - closures). -- **Phase 1 sub-step 4b (VM plan) — slices + strings on comptime memory (2026-06-17).** - `{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate). - Ported `const_string` (text+NUL + fat pointer in comptime memory), `length`/`data_ptr`, - `array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and - `str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23, - subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures). -- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on comptime memory (2026-06-17).** - `kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get` - (`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over - array/pointer/many_pointer; slice/string bases bail), `length` on array values. - Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next: - sub-step 4b (slices/strings, then optionals/enums/any/closures). -- **Phase 1 sub-step 3 (VM plan) — memory + structs on comptime memory (2026-06-17).** - `Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported - `alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out - at the table's natural offsets. Value model: scalar/pointer → register word; - struct → lives in comptime memory, its value IS its address (read→addr, write→memcpy), so - nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on - not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested - (struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next: - sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls). -- **Phase 1 sub-step 2 (VM plan) — comptime executor: scalars + control flow - (2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over - comptime frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy - interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic, - comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and - `block_param`; every other op bails loudly (`error.Unsupported` + op name in - `detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic, - cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus - untouched (688 green). Next: sub-step 3 (memory + aggregates on comptime memory, where - target-aware layout enters). -- **Phase 1 sub-step 1 (VM plan) — comptime machine substrate (2026-06-17).** - New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator - with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0 - reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on - deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested - (`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy - interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor + - scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule - from CLAUDE.md per request. -- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept - (2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig` - (`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/ - `LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr` - + the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and - examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function - host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3 - compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the - `emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples - `0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next: - Phase 1 (byte-addressable value model) per `PLAN-COMPILER-VM.md`. -- **DIRECTION CHANGE — pivot off the byte-weld to a byte-addressable bytecode VM - (2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong - direction (it hand-marshals onto a comptime value model that isn't bytes — exactly - what the design set out to kill). New foundation: a bytecode VM over comptime memory so - comptime values are native bytes; the compiler-API then rides on it via direct memory - (no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks - cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0 - strip → Phase 1 byte-addressable value model → Phase 2 bytecode → Phase 3 compiler-API on - comptime memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted - the session's uncommitted `register_struct`/`find_type` marshaling experiment back to - `reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action. -- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped - the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct / - byte-blob — all explored, all unnecessary). Instead: the sx header declares - fields in the compiler type's memory order; the compiler reflects the bound Zig - type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with - loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch). - On pass it's an ordinary byte-identical struct — cast + deref just works. - Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692). -- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math; - superseded by the reflection+validation design and deleted. -- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is - a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime` - so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite - green (458 unit, 690 corpus). -- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).** - `compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function` - `compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`); - `interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round- - trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean - rejection deferred (loud link error today). Build + suite green (458 unit, 689 - corpus). -- **Phase 1.1 fourth sub-step — welded-struct layout validation.** - `validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into - `registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against - the registry (lib == compiler, name exported, layout matches) with build-gating - diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful - Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to - Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit, - 687 corpus). -- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`: - the `compiler` lib's welded-type registry; `Field` welded to - `StructInfo.Field` with layout baked from the real Zig type - (`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test - (+ null off the export list). Standalone island — not yet consumed by lowering. - Build + suite green (454 unit tests). Break-verified. -- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl` - gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern ` - after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified. - Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete. -- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig) - extern ` on fn decls; unified `callconv` into `abi(.c|.zig|.naked)` (removed - the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs + - snapshots. Build + suite green. The original design's `extern(.zig)` single - qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern - ` (linkage + source) — recorded in the design doc's syntax-decision note. diff --git a/current/CHECKPOINT-EXTERN-EXPORT.md b/current/CHECKPOINT-EXTERN-EXPORT.md deleted file mode 100644 index f846589f..00000000 --- a/current/CHECKPOINT-EXTERN-EXPORT.md +++ /dev/null @@ -1,861 +0,0 @@ -# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream) - -Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds -`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after -every commit, one step at a time per the cadence rule. - -## Last completed step -**Phase 9 COMPLETE — total `foreign` purge; 9.4 GATE PASSES.** **THE ENTIRE -FFI-LINKAGE STREAM (Parts A + B, Phases 0–9) IS DONE.** Final commits: 9.0 token -delete (`dfae690`), 9.3 src/docs/example/library comment purge (`811a280`, `e99383f`, -`dc51c4b`, + the capital-Foreign sweep), 9.3 example filename renames + dedup -(`b52d424`), 9.3/9.4 issues/*.md purge + GATE (`b9cfe25`). -- **9.0:** deleted the `hash_foreign` token entirely (token/lexer/parser/lsp + the lex - test); `#foreign` now → a generic "expected ';'" parse error (accepted UX cost); - deleted the obsolete 1176 rejection test. -- **9.1/9.2:** all internal identifiers renamed (linkage→`extern`/`is_extern`, - runtime-class→`Runtime*`/`runtime_*` per Decision 5, `foreign_path`→`runtime_path` - across the build-hook boundary); `foreign_expr` node eliminated. -- **9.3:** purged every `foreign` COMMENT (src caps + lowercase, examples, docs incl. - the obsolete inline-asm Deviation 6, editors/vscode grammar) + renamed all 10 - `*-foreign*` example files (+ companions/expected/refs) to extern/runtime names - (dedup'd 1218↔1229, removed orphan 1620 dir) + rewrote 20 issues/*.md writeups + - renamed issues/0043. -- **9.4 GATE:** `grep -rniIE 'foreign' src/ library/ examples/ issues/ docs/ editors/ - specs.md readme.md CLAUDE.md` → **0**, excluding only the legitimate keeps: - `SQLITE_CONSTRAINT_FOREIGNKEY` (SQLite API const) + vendored `library/vendors/sqlite/ - c/*` (upstream third-party C). No `foreign`-named files in the tree (node_modules + - .sx-tmp are gitignored third-party/scratch). Suite green (644 corpus / 443 unit, 0 - failed). - -### Prior: Phase 9.3 — text/comment purge (src + docs + example comments) (commits -`e99383f` docs, `dc51c4b` src, + examples purge STAGED pending a classifier outage — -commit message ready; `git commit` the staged `examples/` changes when Bash is back). -`foreign` is now purged from: **all `src/` comments** (reworded to extern/runtime-class; -fixed 2 user-facing diagnostics — the type-annotation parse error no longer lists -`#foreign`, and the Android no-`#jni_main` help shows `#jni_class(…) extern`), **specs/ -readme/CLAUDE** ("Foreign Function Interface"→"C Interop", etc.), and **all example .sx -comments** (1219 stdout labels Foreign→Extern, snapshot regenerated). Suite green -(646/444) throughout; snapshot-neutral except the intentional 1219 regen. - -**What still contains `foreign` (the analyzed keep-list + the not-yet-done):** -- **KEEP (gate-exempt):** `src/` `hash_foreign` token + lexer entry + `lex hash_foreign` - test (`#foreignx`) + the 4 parser rejection messages ("`#foreign` has been removed…"); - `1176-diagnostics-foreign-removed.sx` (its `#foreign` decl + comments ARE the rejection - test); `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored `library/vendors/sqlite/c/*`. -- **NOT YET DONE:** example FILENAMES (`*-foreign*.sx` + the `0729`/`1205`/`1218`/`1219`/ - `1306`/`1318`/`1216`/`1217` families) and their `#import`/`#include`/`#source` path refs - + `expected/` files — needs a git-mv rename step; and **`issues/*.md`** (~20 writeups). - -### Prior: Phase 9.1 + 9.2 — internal IDENTIFIER purge COMPLETE (commits 9.1a `b838f63`, -9.1b `b78e7dd`, 9.1c `cd14794`, 9.1d `7ffdc7d`, 9.2a `3354446`, 9.2b `5c8af6e`, -9.2b-fix `a15a868`, 9.2c `d27be42`, 9.2d `8cca3b9`). **Every `foreign` IDENTIFIER in -`src/` is renamed** — the only `foreign` left in `src/` is COMMENTS + the kept token -(`hash_foreign` + its `#foreignx` lexer-boundary test) + the rejection-message string. -Suite green (646/444) at every commit. -- **9.1d** eliminated the `foreign_expr` AST node: migrated `c_import.zig` auto-synth - to the extern shape, deleted the node + `ForeignExpr` + all readers. -- **9.2a/b/c/d** ran the runtime-class family rename (Decision 5 → `Runtime*`): - types `ForeignClassDecl`→`RuntimeClassDecl` etc.; fns `parseForeignClassDecl`→ - `parseRuntimeClassDecl`, `lowerForeignMethodCall`→`lowerRuntimeMethodCall`, …; state - `foreign_class_map`→`runtime_class_map`, `foreign_class_decl` variant→ - `runtime_class_decl`; the extern-ref validators → `Extern` (linkage, `checkExternRefs`); - the reference flag → **`is_extern`** (per user: reuse existing terminology, not a new - `is_reference`); and `foreign_path`→`runtime_path` COUPLED across the hook boundary - (build.sx accessor `jni_main_runtime_path_at` + the registered hook string + - bundle.sx + specs.md), with 37 `.ir` snapshots regenerated for the renamed - `@BuildOptions.jni_main_runtime_path_at` declare stub (symbol-name change only). -- **9.1a/b/c** (linkage): 5 collision-free renames (callExtern, …); "foreign symbol" - diagnostic + panic → "extern symbol" (1172 regen); deleted dead VarDecl legacy fields. - -### Prior: Phase 9.1 (partial) — internal linkage-identifier purge (commits `b838f63` 9.1a, -`b78e7dd` 9.1b, `cd14794` 9.1c). **PHASE 9 STARTED.** Decision 6 = PURGE EVERYTHING, -scoped (user, 2026-06-15): purge `foreign` from **all `.sx` files + all documentation + -all our Zig (`src/`)**, analyzing each grep hit — **legitimate keeps stay** -(SQLITE_CONSTRAINT_FOREIGNKEY + other SQLite API constant names, vendored -`library/vendors/sqlite/c/*`, `1176-diagnostics-foreign-removed.sx` [the rejection test -MUST contain `#foreign`], the parser rejection-message string + `hash_foreign` token -[kept so `#foreign` keeps its friendly deprecation error]). -- **9.1a** (`b838f63`): 5 collision-free linkage renames — `callForeign`→`callExtern`, - `marshalForeignArg`→`marshalExternArg`, `dedupeForeignSymbol`→`dedupeExternSymbol`, - `foreign_name_map`→`extern_name_map`, `is_foreign_c_api`→`is_extern_c_api`. -- **9.1b** (`b78e7dd`): the "foreign symbol already bound" diagnostic (decl.zig) + - resolveFuncByName panic (call.zig) → "extern symbol". Intentional 1172 regen. -- **9.1c** (`cd14794`): **deleted** the dead `VarDecl.is_foreign`/`foreign_lib`/ - `foreign_name` fields (the global `#foreign` path rejects → write-dead; 3 coalescing - readers in decl.zig simplified to `vd.extern_name`/`vd.is_extern`). -All snapshot-neutral except the intentional 1172 regen; suite green (646/444). - -**COLLISION ANALYSIS (done — drives the rest of 9.1/9.2):** -- `is_foreign` lives on FnDecl?(no — flipped to `extern_export`), **VarDecl (deleted in - 9.1c)**, and **ForeignClassDecl (ast.zig:903 — STILL LIVE**, distinguishes runtime-class - reference vs define; renamed in 9.2, not 9.1). -- `is_extern`/`extern_lib`/`extern_name` already exist (VarDecl + IR insts) — so the - old `foreign_*` linkage names could NOT be blind-renamed onto them; 9.1c deleted the - dead VarDecl trio instead of renaming. -- `foreign_expr` (25) is **still BUILT by `c_import.zig` auto-synthesis** (`#import c - {#include}` synthesizes fn bodies as `foreign_expr`). To eliminate it: migrate that - synth path to build the extern shape (empty-block body + `extern_export = .extern_`), - exactly the Phase 5.0 fn-body flip but for auto-synth — THEN delete the `foreign_expr` - node + all readers. This is the last 9.1 item. - -### Prior: Phase 8 — CUTOVER: parser hard-rejects `#foreign` (`feat!` commit `3811311`, -preceded by the 8.0 xfail `8180faf` + 3 pre-cutover `refactor`s `2cce6a3`/`720556b`/ -`d132aab`). **PHASE 8 COMPLETE.** The prefix `#foreign` linkage directive is removed: -all four parse sites (const-with-type 316, data global 425, fn body 2065, runtime-class -prefix via caller 260) reject it with the migration message *"`#foreign` has been -removed; use the postfix `extern` (import) / `export` (define) linkage keyword -instead"*; added a span-aware `failAt` for the runtime-class case (the lookahead -consumes the token before the reject decision). New example **1176** -(`diagnostics-foreign-removed`) pins it. **Pre-cutover migrations** (all green, -behavior-preserving): the 7 identity `ffi-foreign-*` test DECLS (`2cce6a3`), the two -keyword-neutral diagnostic tests 1172 + 1228 with intentional snapshot regens -(`720556b`), and the 4 multi-file example companions Phase 7 missed (0729/a+b, 1617/c, -1623/mod — `d132aab`). **Deleted** obsolete tests 1174 (`#foreign`+postfix conflict, now -unreachable) + 1620 (`#foreign nosuchunit`, superseded by extern twin 1231), the GATE -A→B unit test + `lowerSrcToIr` helper (nothing left to compare), and converted the -in-source `parse void function with foreign body` parser test to postfix `extern`. -specs.md + readme.md document `extern`/`export` as the sole C-linkage surface. Suite -green (646 corpus / 444 unit, 0 failed). - -### Prior: Phase 7.4 — migrate straggler examples `#foreign`→`extern` (`refactor` commit -`1a8991a`). **PHASE 7 MIGRATABLE WORK COMPLETE (7.1–7.4 done).** Migrated 16 fn/global -examples across categories (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/ -1622/1628/1635/1636). Marker'd ones corpus-validated; the 3 unmarked uikit importers -(1607/1608/1616) verified byte-identical via `sx ir` probes. Empty snapshot diff; suite -green (647/444). - -**Phase 7 net result:** every example that uses `#foreign` *incidentally* (FFI plumbing, -output-preserving) is now on `extern`/`export`. The **24 files still holding `#foreign` -are exactly the intended keep-list**, all deferred to the Phase 8 cutover: -- **`foreign`-asserting diagnostics** (migrating changes a snapshot): 1172, 1174, 1219 - (stdout label), 1228 (equivalence test), 1620. -- **Identity `ffi-foreign-*` feature tests** (real decls; rename/dedup at cutover): - 1205-global, 1205-global-helper, 1207, 1218, 1219, 1306, 1318. -- **Comment-only / provenance prose** (decls=0; `#foreign` only in comments): 0716, 0729, - 1216, 1223, 1224, 1225, 1229, 1230, 1231, 1332, 1348, 1349, 1426, + issues/0030. -**Lesson (7.3):** the robust class-prefix transform is the GENERAL form -`s/#foreign\s+#(\w+)\((\"[^\"]*\")\)\s*\{/#$1($2) extern {/` — 1417 also used -`#jni_interface`/`#objc_protocol`/`#swift_class`/`#swift_struct`/`#swift_protocol`, and a -`#(objc|jni)_class`-only regex left `extern` in *prefix* position → parse error. All such -directives accept the postfix modifier (probed). Bare defined `#objc_class`/`#jni_class` -examples (no `#foreign`) were left untouched — not a purge target (define→export is an -optional consistency pass, deferrable). - -### Prior: Phase 7.1 — migrate incidental 12xx ffi examples (`refactor` commit `731fb8d`). -Migrated 12 plain-C examples (1200/1206/1209-1215/1220/1221/1222); established the -keep-list policy above. Phase 7.2 (`a68f7c2`): 18 13xx obj-c examples (prefix→postfix -classes). Phase 7.3 (`2888f6f`): 13 14xx jni examples incl. 1417 multi-runtime. - -### Prior: Phase 6.5 — migrate `gpu/` `#foreign`→`extern`; `library/` now `#foreign`-free -(`refactor` commit `32a7628`). **PHASE 6 COMPLETE.** Final batch: gpu/gles3.sx -(eglGetProcAddress + 1 comment) + gpu/metal.sx (MTLCreateSystemDefaultDevice), bare -fn markers → `extern`. Verified byte-identical `sx ir` on importers 1610 (gles3) + -1606 (metal). **Zero `#foreign` remains anywhere under `library/`** — verified by -`grep -rln '#foreign' library/` → no matches. Suite green (647 corpus / 444 unit, 0 -failed). - -### Prior: Phase 6.4 — migrate `ffi/` `#foreign`→`extern` (`refactor` commit `666a2e2`). -objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + objc.sx's 2 import runtime -classes (prefix→postfix `extern`). objc + objc_block validated by the 50 marked 13xx -corpus examples (incl. import classes 1300/1301 + defined classes 1339/1349); -raylib/ffi-sdl3/wasm verified by byte-identical `sx ir` probes pre/post. Empty snapshot -diff; suite green. - -### Prior: Phase 6.3 — migrate `std/` `#foreign`→`extern` (`refactor` commit `59f90d2`). -Pure source rename across 11 std modules (~60 sites): -cli/core/fmt/fs/log/net.kqueue/process/socket/thread/time/trace. All fn-decl markers -— bare `#foreign;`, `#foreign libc;`/`#foreign tlib;` (LIB ref), and -`#foreign libc "csym";` (LIB+rename) → the same `extern …` tail (`extern` carries the -identical `[LIB] ["csym"]` axis); plus 2 stale comment mentions (fmt/fs). No class -forms in std. These modules ARE host-corpus-exercised → empty snapshot diff is direct -validation. Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 batches: -6.4 ffi (~50, has runtime classes), 6.5 remainder. - -### Prior: Phase 6.2 — migrate `platform/` `#foreign`→`extern`/`export` (`refactor` commit -`2cd5d7b`). Pure source rename across uikit/android/android_jni/sdl3 (~64 sites): -30 fn `… #foreign;`→`… extern;`; 34 import runtime classes -`#foreign #objc_class/#jni_class("X") {`→`#…_class("X") extern {` (prefix→postfix); -4 defined `Sx*` obj-c classes `#objc_class("X") {`→`… export {`. Behavior-preserving; -empty snapshot diff. **Verification (these modules are largely uncompiled by the -host corpus** — bundle examples import `bundle.sx`, not the runtime modules; android.sx -only compiles under `OS==.android`): byte-identical `sx ir` on uikit importers 1610 + -1606 (which DO compile uikit incl. the 4 defined `Sx*` classes on host) and an sdl3 -direct-import probe; android.sx verified by an identical 4-error dedup set (host -pthread clashes — the keyword-neutral "foreign symbol already bound" dedup message is -unchanged, and the probe parsed all migrated `extern` jni classes + EGL fns cleanly -before hitting them). Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 -batches: 6.3 std (~60), 6.4 ffi (~50), 6.5 remainder. - -### Prior: Phase 6.1 — migrate `vendors/sqlite` `#foreign`→`extern` (`refactor` commit -`410a52e`). **PART B PHASE 6 STARTED.** Pure source rename: all 97 -`sqlite3_* … #foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ the one -stale header-comment reference, line 9). The `extern_lib` axis references the `sqlib` -`#import c` unit identically to `#foreign sqlib`, so IR/output is byte-identical — -empty snapshot diff (only `sqlite.sx` changed), and example 1624 -(`vendor-sqlite-module`) stdout byte-unchanged. Suite green (647 corpus / 444 unit, -0 failed). - -### Prior: Phase 5.1 — annotate A→B gate post-flip + add fn-rename case (`test` commit -`93e7b6f`). **PHASE 5 COMPLETE → PART B Phase 5 done.** The A→B gate -(`lower.test.zig`) already asserted `#foreign` ≡ `extern`/`export` byte-identical IR -for fn / global / Obj-C class; post-Phase-5.0 the fn-decl + data-global paths build -the SAME extern-named AST, so cases 1/2 are now STRUCTURALLY identical (guaranteed by -construction, not coincidence). Annotated the gate header to record this and keep it -as a regression tripwire (catches a future reader re-diverging the spellings, or a -revert of the flip); case 3 (runtime class) stays behaviorally — not structurally — -equal via the single `is_foreign_eff` field. Added a fn-rename case (case 2b, -`extern_name` axis: `c_abs` → `"abs"`) to broaden coverage beyond bare import -(verified IR-identical via `sx ir` probe before adding). Test-only, no snapshot churn. -Suite green (647 corpus / 444 unit, 0 failed). - -### Prior: Phase 5.0 — fn-decl `#foreign` body-marker FLIP (`refactor` commit `6b94bb6`). -**PHASE 5.0 PARSER ROUTING COMPLETE.** The fn-body `#foreign [LIB] ["csym"]` marker -now builds the SAME extern AST postfix `extern` produces (`extern_export = .extern_` -+ `extern_lib`/`extern_name` + empty-block body) instead of a `foreign_expr` body. -Behavior-preserving — all four prereqs (visibility, variadic, plain-free, lib-ref) -ensure every downstream reader coalesces `is_foreign` with `extern_export`, so IR + -runtime are byte-identical (full corpus + A→B gate green). Decision 7 churn realised: -example 1620's lib-ref error flips "#foreign library" → "extern library" (the only -snapshot moved; hand-edited, not regen). Parser unit test updated to assert the extern -shape. Spot-checked 1219/1218/0729 (foreign rename / cvariadic / same-name) end-to-end. -**All four `#foreign` parser paths now resolved:** global (`e5ddfbe`) + fn-body -(`6b94bb6`) flipped onto extern; const-with-type is dead (deferred); runtime-class is -already coalesced (`is_foreign_eff`). `c_import.zig` auto-synthesis STILL emits -`foreign_expr` bodies (Phase 6+), so both shapes coexist — every reader stays dual. -Suite green (647 corpus / 444 unit, 0 failed). - -### Prior: Phase 5.0 prereqs 3 & 4 — plain-free classification + extern lib-ref validation -(plain-free: xfail `2706521` → fix `3c94c14`; lib-ref: xfail `38c3240` → fix -`ad6aed3`). Two MORE extern/#foreign divergences found while de-risking the fn-path -flip, both now closed. **FOUR prereqs total done — the fn-decl flip fully de-risked.** -- **Prereq 3 (plain-free):** `isPlainFreeFn`/`isPlainFreeFnDecl` (resolver.zig:178, - generic.zig:815) excluded a `#foreign` body but classified an empty-block `extern` - fn as a plain free fn — so existing extern fns were wrongly counted in the bare-call - ambiguity verdict (example: two same-name `extern libc "abs"` authors errored - ambiguous, while the `#foreign` twin 0729 compiles). Both predicates now also - exclude `extern_export == .extern_`; `export` (real body) stays plain-free. Example - **1230**. -- **Prereq 4 (lib-ref validation):** `checkForeignRefs` (c_import.zig) validated only - `foreign_expr.library_ref`, so a bogus `extern nosuchunit "abs"` compiled silently - while `#foreign nosuchunit` errors (1620). Now reads the lib ref from EITHER spelling - and names the surface keyword in the diagnostic (so 1620 stays byte-unchanged). - Example **1231**. -- Two OTHER classifying sites probed and found BENIGN for extern (no flip prereq): - namespace/qualified dispatch (`registerQualifiedFn` decl.zig:2208, namespace gate - call.zig:729) — a namespaced `extern` fn resolves identically to its `#foreign` twin - (probe: `cm.c_abs(-9)` → 9 both ways; the registered qualified alias resolves to the - same extern symbol). - -### Prior: Phase 5.0 prereq — extern C-variadic tail (xfail `9a2c78d` → fix `0fdc821`) — the SECOND deferred fn-path prerequisite. **BOTH original fn-path prereqs done.** The C-variadic `...` handling was keyed on the `#foreign` (`foreign_expr`) -body shape at two sites — the `is_variadic` drop in `declareFunction` -(`decl.zig:2097`) and the call-site early-out in `packVariadicCallArgs` -(`pack.zig:302`). A variadic `extern` therefore kept its trailing slice param and -slice-packed the extras → garbage at the C ABI (probe: `sum_ints(3,10,20,30)` → -53316585, not 60). Both gates now also fire for `extern_export == .extern_`, so a -variadic `extern` drops the `..args: []T`, sets `is_variadic`, and passes extras -through the C `...` slot with default argument promotion — byte-identical to its -`#foreign` twin. New example **1229** (`1229-ffi-extern-cvariadic`, JIT `#source`, -int-sum + double-avg). Suite green (645 corpus / 444 unit, 0 failed). - -### Prior: Phase 5.0 prereq — visibility-gate equivalence (xfail `717c35d` → fix `7d8ba1a`) — the first of the two deferred fn-path prerequisites. -The non-transitive C-import visibility gate (`isVisible(.c_import_bare)`, -`decl.zig:2249`) used to recognise only the legacy `#foreign` body shape; a bare -`extern` fn (empty-block body + `extern_export == .extern_`) escaped the gate via -the `body != foreign_expr → return true` arm and was caught only by the general -`isNameVisible` gate — yielding the generic "not visible" wording instead of the -C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings -route to `visibleOverEdges`, and a library-bound `extern LIB` (like `#foreign LIB`) -stays unconditionally visible — so a future fn-decl `#foreign`→`extern` migration -is byte-identical at this gate. New cross-module example **1228** -(`examples/1228-ffi-extern-c-non-transitive`, main → b → c) pins the equivalence: -referencing c's lib-less `#foreign` AND `extern` twins transitively both produce -the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed). -**Empirical finding** (probe, not yet acted on): the bare-extern twin was NEVER a -silent visibility hole — the general `isNameVisible` gate already denied it; only -the *diagnostic wording* diverged. The fix aligns the wording + gate ownership. - -### Prior: Phase 5.0 (global path) (`refactor` lock, commit `e5ddfbe`) — **PART B STARTED.** -First of the four `#foreign` parser paths migrated onto the extern AST: the -data-global form `name : T #foreign [lib] ["csym"];` now builds the same -extern-named `VarDecl` (`is_extern`/`extern_lib`/`extern_name`) that postfix -`extern` already produces, instead of `is_foreign`/`foreign_lib`/`foreign_name`. -Behavior-preserving — lowering coalesces both forms identically -(`decl.zig:1119,1127,1141`), so zero snapshot churn. The fn-decl, const-with-type, -and runtime-class `#foreign` paths still build the legacy AST. - -### Prior: Phase 4 (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces: -(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) — -asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample -fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start -migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side -fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix -`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse -error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic -— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean -error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three -`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays -documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail). -NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a -harmless dup (both `.c`), and any non-`.c` callconv already errors on its own. - -### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/ -`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional -`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{` -body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime -class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx -class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same -`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/ -lowering change was needed** — the new surface reuses the existing reference-vs-define path. -Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via -JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`), -**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443 -unit, 0 fail). - -### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works: -external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename. -All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for -`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`, -`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc; -(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**: -`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn -stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name -+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226** -(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls -`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443 -unit, 0 fail). - -**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the -corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's -flat-namespace lookup), so an `expected/.aot` marker switches an example to a -`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted — -`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated). - -## Current state -Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`. -**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"` -axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag -linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.** -extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`); -data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`) -AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root. -All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored -but is a *reference* only — actual linking stays the `#library`/build-flag axis. -**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class` -(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the -**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A -COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` + -~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223 -(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn, -AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426 -(jni extern class), 1174/1175 (interplay diagnostics). - -## Next step -**NONE — the FFI-linkage stream is COMPLETE.** `extern`/`export` fully replace -`#foreign`; the keyword is rejected; zero `foreign` remains in the gated tree (Parts -A + B, Phases 0–9 all done; the 9.4 gate passes). This stream can be archived. - -Follow-ups (both DONE 2026-06-15, post-stream polish): -- ✅ Added `extern`/`export` to the editors/vscode tmLanguage keyword list as a - `storage.modifier.sx` pattern (`editors/vscode/syntaxes/sx.tmLanguage.json`). -- ✅ Dropped the vestigial `RuntimeClassPrefix.is_extern` field + - `parseRuntimeClassDecl`'s `is_extern` param (always-false dead path; the postfix - `extern`/`export` keyword is the sole reference-vs-define decider). Suite green - (644 corpus / 442 unit, 0 failed). - ---- (historical: the finish-Phase-9 plan, now done) --- -**PART B — finish Phase 9: example FILENAME renames + `issues/*.md` + 9.0/9.4.** -(All `src/` identifiers + AST node + all comments/docs/example-comments are DONE.) - -0. **FIRST: commit the staged `examples/` comment purge** (a classifier outage blocked - the commit; changes are `git add`ed). Message: "refactor(ffi-linkage): Phase - 9.3-examples — purge 'foreign' from example .sx comments". -1. **Example filename rename** (git-mv step, snapshot-careful): rename the `*-foreign*` - example files to extern/runtime names and update every `#import`/`#include`/`#source` - ref + the `expected/.*` companions. Families: `0729-modules-flat-same-name-foreign` - (+ `/a.sx`,`/b.sx` dir), `1205-ffi-foreign-global`(+`-helper`), `1207-ffi-foreign-global-from-helper`, - `1216-ffi-08-foreign-in-method`(+`.h`/`.c`), `1217-ffi-09-foreign-result-chain`(+`.h`/`.c`), - `1218-ffi-foreign-cvariadic`(+`.c`), `1219-ffi-foreign`, `1306-ffi-objc-foreign-class-chained-dispatch`, - `1318-ffi-objc-property-foreign`. ⚠ A renamed file with an `.ir`/`.stderr` snapshot that - echoes its own path will need that snapshot regenerated (intentional). Pick new names - that drop "foreign" (e.g. `…-extern-global`, `…-extern-in-method`, `…-runtime-class-chained-dispatch`). - NOTE: keep `1176-diagnostics-foreign-removed.sx` name (it's the rejection test — fine to keep "foreign"). -2. **issues/*.md** (~20) — rewrite writeup prose `#foreign`/`foreign`→`extern`/`runtime-class`. -2b. **`docs/*.md`** — ALSO in the gate scope (was missed; the gate areas are now - `src/ library/ examples/ issues/ docs/ specs.md readme.md CLAUDE.md`). `docs/debugger.md` - referenced the renamed `callForeign` (fixed → `callExtern`, UNCOMMITTED with the staged - batch); sweep all of `docs/` for stale renamed-identifier refs + `foreign` prose. -3. **9.0 surface decision — RATIFIED (user, 2026-06-15): DELETE the `hash_foreign` token.** - The user explicitly flagged token.zig:121 + lsp/server.zig:1693 "this also needs to - go" — total purge, accept `#foreign`→generic error (no friendly migration hint). This - is the LAST src change; it is load-bearing → needs a build + test + 1176 regen (do it - when mutating Bash is back). Steps: - - token.zig: remove `hash_foreign` enum (121). - - lexer.zig: remove the `.{ "#foreign", Tag.hash_foreign }` map entry (91), drop - `#foreign` from the directive-list comment (72), DELETE the `lex hash_foreign` test - (626-631, incl. `#foreignx`). - - parser.zig: remove the 4 `self.current.tag == .hash_foreign` rejection sites (268 - caller / 327 / 419 / 2024) + their messages, AND the 2 lookahead refs (`hasFnBody…` - ~3658 + ~3676). ⚠ Decide what `#foreign` lexes to with no keyword entry (likely an - error/unknown-directive token) and confirm the parser surfaces a sane error. - - lsp/server.zig: remove the `.hash_foreign,` arm (1693). - - **1176-diagnostics-foreign-removed**: its expected stderr is the now-deleted - "`#foreign` has been removed…" message → it WILL change. Regen 1176's snapshot to - whatever the generic post-deletion error is (intentional), OR delete 1176 entirely - (its purpose — a friendly rejection — no longer exists). Recommend: keep 1176 as a - "`#foreign` is no longer a directive" regression, regen its snapshot. NOTE: after - this, 1176 may still contain `#foreign` in its SOURCE (the rejected token) — that's - the only legitimately-remaining `foreign` in `.sx`, OR rename/rework it to avoid even - that if the gate must be absolute. -4. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` → 0 (no keep-list - left except possibly 1176's source token + `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored C). - ---- (historical: the prose-purge plan, now mostly done) --- -**PART B — finish Phase 9: the COMMENT / DOC / issues text purge** (all `src/` -identifiers + the AST node are already done; remaining is prose). Lower-risk than the -renames (text only, mostly snapshot-neutral) but needs per-instance reading — NOT a -blind sed. Footprint: `src/` ~205 (all comments now), `examples/*.sx` ~100 comments, -`issues/*.md` ~20 files, docs (specs/readme/CLAUDE). - -Order: -1. **src/ comments** (~200) — reword `foreign`→`extern`/`runtime-class` to match the - renamed identifiers. KEEP: the rejection-message string, the `hash_foreign` token + - its `#foreignx`/`lex hash_foreign` test, and any comment that legitimately explains - the cutover (it must name `#foreign` to say it's removed). The ast.zig FnDecl comment - still says "mirroring `#foreign LIB "csym"` (foreign_lib/foreign_name)" — reword. -2. **examples/*.sx comments** — the deferred provenance comments (full list in the prior - Next-step revision / git). ⚠ Many CONTRAST `#foreign` vs `extern` — reword to stay - coherent. ⚠ `1219-ffi-foreign.sx` prints `"foreign-rename: {}"`/`"=== 15. Foreign ==="` - to STDOUT — changing those regens its snapshot (intentional). `1176`/`1216` legitimately - discuss `#foreign` removal — keep minimal `#foreign` mentions where the test IS about it. -3. **issues/*.md** (~20) — rewrite writeup prose to `extern`/`export`/`runtime-class`. -4. **docs** — specs.md (rename "Foreign Function Interface" heading → "C Interop"; the - `#import c` "foreign declarations" prose; the comptime "foreign function calls" line; - §3344 "foreign code can't observe the error channel"), readme.md (211-212 the `#import - c` exemption prose), CLAUDE.md (host_ffi `#foreign("c")` ref → `extern`; "foreign calls"). -5. **9.0 surface decision** (recommend KEEP `hash_foreign` token + rejection for a good - deprecation — then it + the message + 1176 + `#foreignx` are permanent gate-exempt keeps). -6. **9.4 gate** — `grep -rniIE 'foreign'` over `.sx` + docs + `src/` minus the keep-list → 0. - KEEP-LIST (gate-exempt): `SQLITE_CONSTRAINT_FOREIGNKEY` + SQLite API names, vendored - `library/vendors/sqlite/c/*`, the `hash_foreign` token + `#foreignx` test + rejection - message string, `1176-diagnostics-foreign-removed.sx` (rejection test must contain it). -**Gate (scoped per user 2026-06-15):** `grep -rniIE 'foreign'` → 0 across `.sx` files, -all docs, and our `src/` Zig — EXCLUDING the legitimate keeps listed in Last completed -step (SQLite API names, vendored C, the rejection test/message + `hash_foreign` token). - -Remaining, in suggested (dependency-safe) order: -1. **9.1d — eliminate `foreign_expr`** (last linkage item): migrate `c_import.zig` - auto-synthesis to build the extern shape instead of a `foreign_expr` body (the Phase - 5.0 fn-body flip applied to auto-synth), then delete the `foreign_expr` AST node + - `ForeignExpr` + all readers (25). Snapshot-neutral; verify full corpus (the `#import c` - examples 1215/1216/1217 + sqlite 1624 exercise it). -2. **9.2 — runtime-class family rename → `Runtime*` (Decision 5).** The BIG one, do as - small per-identifier commits with `zig build` after each (snapshot-neutral). Targets - (counts): `ForeignClassDecl`(65)→`RuntimeClassDecl` · `foreign_path`(62)→`runtime_path` - · `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `ForeignMethodDecl`(31) - · `foreign_class_decl`(30) · `foreign_expr`-gone-by-now · `ForeignClassMember`(20) · - `ForeignFieldDecl`(15) · `ForeignClassDecl.is_foreign`(the live one)→e.g. `is_reference` - · `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` · - `findForeign*InChain` · `resolveForeign*` · `register*ForeignClass*` · `*ForeignRefs` · - `ForeignRuntime` · `current_foreign_class`/`_method`. ⚠ COUPLED .sx↔.zig hook names: - `jni_main_foreign_path_at`/`jni_main_foreign_paths`/`hookJniMainForeignPathAt`/ - `foreignPathToJavaName`/`splitForeignPath` span build.sx + bundle.sx + compiler_hooks.zig - + specs.md (2975/3049) — rename all four sites together. -3. **9.x-src-comments** — the ~200 bare-`foreign` comments in `src/` (rename last, since - many reference identifiers that 9.1d/9.2 rename; do AFTER those so the comment text - matches the new names). -4. **9.3-examples comments** — the deferred `.sx` provenance comments (0716, 0729, 1205/ - 1207/1216/1218/1219/1220, 1223-1231, 1306/1308/1315/1318/1320/1321/1331/1332/1348/1349, - 1412/1414/1417/1418/1419/1426, 0117/0415, 1140/1141/1125, issues/0030.sx). ⚠ Many - CONTRAST `#foreign` vs `extern` ("no `#foreign`, no `#library`") or reference renamed - internals — rewrite each to stay coherent (NOT blind sed). ALSO: `1219-ffi-foreign.sx` - prints `"foreign-rename: {}"` to STDOUT — changing it regens the snapshot (intentional). -5. **9.3-issues** — `issues/*.md` writeups (~20 files) → rewrite `#foreign`/`foreign` to - `extern`/`export`/`runtime-class` per the renames. -6. **9.3-docs** — specs.md (12: rename "Foreign Function Interface" heading → "C Interop"; - the `#import c` "foreign declarations" prose; the jni_main_foreign_path_at refs with #2), - readme.md (2), CLAUDE.md (2: host_ffi `#foreign("c")` ref + "foreign calls"). -7. **9.0 surface decision** — keep `hash_foreign` token + rejection (recommended: good - deprecation) vs delete it. If kept, the token + the rejection-message string + 1176 are - permanent legitimate keeps; the gate excludes them. -8. **9.4 gate** — `grep -rniIE 'foreign'` over the gated set minus the keep-list → 0. -- **6.2 verification note (carry forward):** the `platform/` runtime modules - (uikit/android/android_jni) are NOT compiled by any marker'd host corpus test — verify - future platform-adjacent migrations via direct `sx ir` on importers (1610/1606 compile - uikit on host) or import probes, not the corpus alone. -- **Phases 6–7** (`refactor` batches, empty snapshot diff per batch): migrate the - stdlib + examples from `#foreign` spelling to `extern`. Because the AST is already - unified, this is a pure SOURCE rename (`… #foreign LIB "sym";` → `… extern LIB "sym";` - for fns; the global/const forms similarly), and IR/output must be byte-identical per - batch. NOTE: `c_import.zig` auto-synthesis (`#import c {#include}`) still BUILDS - `foreign_expr` bodies internally — that's a compiler-internal path, migrated separately - (likely Phase 8/9 area), not a source-spelling change. -- **Then Phase 8** (cutover: hard-reject the `#foreign` keyword) and **Phase 9** (purge - all `foreign` identifiers — needs Decision 5 [done, `Runtime*Class*`] + Decision 6 - [open, historical carve-out]). - -**Watch items carried forward:** -- `c_import.zig:262` auto-synthesis still emits `foreign_expr` — both shapes coexist - until that path is migrated; keep every `body.data == .foreign_expr` reader dual - (checked exhaustively this stream). -- const-with-type `#foreign` parser path (`parser.zig:316`) is still on `foreign_expr` - but DEAD (registers no const); migrate or delete it at the Phase 8 cutover. -- The `decl.zig:2055` "foreign symbol … already bound" dedupe message is keyword-neutral - and fires for both forms — no churn, but reword to "extern" at cutover for consistency. Route the fn-decl `#foreign` path so a -`#foreign` fn builds the SAME extern AST that postfix `extern` already produces, -instead of a `foreign_expr` body. This is the highest-value path (the bulk of -`#foreign` usage). Key sub-questions to resolve before/while routing: -- The `foreign_expr` node carries `library_ref` + `c_name`; the `extern` fn carries - `extern_export = .extern_` + `extern_lib` + `extern_name` on the FnDecl with an - empty-block body. Migration = the parser's fn-body `#foreign` arm - (`parser.zig:~2062`) builds the extern shape (set `extern_export`, map - `library_ref→extern_lib`, `c_name→extern_name`) rather than a `foreign_expr`. -- Lowering ALREADY coalesces the two at every fn site checked this stream - (`decl.zig` 2088/2124/2132/2156/2324/2531 read `is_foreign OR extern_export`), - and the two prereq gates (visibility `decl.zig:2249`, variadic `decl.zig:2097` + - `pack.zig:302`) now do too — so the migration should be behavior-preserving with - ZERO snapshot churn. VERIFY with the A→B gate test (`lower.test.zig`) + a full - `zig build test` after routing; any churn means a site still reads `foreign_expr` - structurally and must be coalesced first. -- ⚠ This ALSO migrates the **const-with-type** path implicitly IF it shares the same - `foreign_expr`→extern reshape (it builds `const_decl{value=foreign_expr}`). Decide: - reshape the const path's value node alongside, or leave the dead const path on - `foreign_expr` until Phase 8 cutover. The const path is dead (see findings below), - so leaving it is acceptable; but the parser arm is shared-ish — check whether the - fn-body arm change touches it. -- Cadence: because the migration is behavior-preserving (no churn), it's a single - `refactor`/lock commit (like the 5.0 global-path commit `e5ddfbe`), NOT an - xfail→fix pair. - -**Investigation findings (this session — reorder the remaining paths):** -- **const-with-type** (`parser.zig:316`, `name :: type_expr #foreign …`) is a - **DEAD path**: it builds `const_decl{value = foreign_expr}`, but - `registerTypedModuleConst` (`decl.zig:848-851`) bails on a `foreign_expr` value - (`else => return`), so it registers no const and emits no symbol — a probe - (`g_abs :: FP #foreign "abs";`) returns `unresolved 'g_abs'` at the use site, and - the form is used NOWHERE in `library`/`examples`/`issues`. Its migration target is - ambiguous because the `foreign_expr` value node is SHARED with the fn-decl path, - which isn't migrated yet. **Decision (user, 2026-06-14): defer it — migrate it - alongside the fn-decl path once `foreign_expr`'s extern shape is decided.** The - checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the - "confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it). -- **runtime-class prefix** (`parser.zig:~1351`, `#foreign #objc_class/#jni_class`) is - **ALREADY coalesced**: both prefix `#foreign` and postfix `extern` feed the single - `is_foreign_eff`→`is_foreign` field on `foreign_class_decl` (`parser.zig:1421-1432`), - so there is NO Phase 5.0 AST change for it — only the Phase 9.2 `Runtime*Class*` - rename remains. Drop it from the Phase 5.0 path list. - -So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then -the fn-decl `#foreign` body-marker migration. const-with-type + runtime-class need -no standalone Phase 5.0 commit. - -Then Phase 5.1 (`lock`): unit test that `#foreign` and `extern` produce identical IR (the -A→B gate already covers fn/global/class — extend or reuse `lowerSrcToIr`). Then Phases 6–7 -migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject -`#foreign`), Phase 9 total `foreign` purge. - -**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target -(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep -`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames; -the plan says confirm before Phase 9, but worth raising with the user before sinking Part B -effort. **Also pick up the two Deferred items below at the start of Part B** (the -visibility-gate equivalence in particular needs a cross-module example). - -**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session -(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects -directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a -dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a -~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples -still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime -`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC -`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C -constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*` -examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c` -(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own -milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test -path and works today. Spike fully reverted (target.zig/main.zig at HEAD). - -**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md` -document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) ~~visibility-gate -equivalence~~ — **DONE** (`717c35d`/`7d8ba1a`): the `c_import_bare` gate now polices a -lib-less `extern` fn identically to its lib-less `#foreign` twin (same C-specific -diagnostic); a library-bound `extern LIB` stays unconditionally visible. Locked by the -cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the -general `isNameVisible` gate already denied it; only the diagnostic wording diverged.) - -## Open decisions -Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B: -- **Decision 5 RATIFIED** (user, 2026-06-14): runtime-class rename target = `Runtime*Class*` - (object-model axis, not linkage). Drives the Phase 9.2 identifier renames. -- **Decision 6 RATIFIED** (user, 2026-06-15): **PURGE EVERYTHING** — the Phase 9.4 gate is - absolute, including `issues/*.md` writeups (NOT the recommended keep-provenance default). - Every `#foreign`/`foreign` reference in the gated tree (`src/ library/ examples/ issues/ - specs.md readme.md CLAUDE.md`) is rewritten to `extern`/`export`; provenance lives in git - history + `(Regression issue NNNN)` notes, not the keyword spelling. -- **Decision 7 RATIFIED** (user, 2026-06-15): **accept the churn** — `#foreign`-spelled - decls produce `extern`-worded diagnostics; example 1620 regenerated (only snapshot moved). - Aligns with Part B's extern-only end state; the interim oddity is cosmetic and removed at - the Phase 8 cutover. Landed in the fn-body flip `6b94bb6`. (Original framing below.) - — interim diagnostic wording for `#foreign`-spelled decls (gated the fn-body flip). Once the flip lands, a `#foreign`-spelled fn builds the extern AST, so any - diagnostic that reads the unified AST can no longer tell the user wrote `#foreign` vs - `extern`. Concretely, example 1620's lib-ref error flips "#foreign library…" → - "extern library…". Options: **(A, recommended)** accept the narrow churn — regen 1620 as - intentional; it aligns with Part B's `extern`-only end state and the interim oddity - (`#foreign` source → "extern" message) is cosmetic and short-lived (Phase 8 cutover - removes `#foreign`). **(B)** retain a one-bit surface marker on `FnDecl` (`wrote_foreign`) - so interim diagnostics stay keyword-accurate (zero churn, small extra plumbing, marker - deleted at cutover). Affects only diagnostic wording — IR/behavior identical either way. - -## Log -- (9.0 + 9.3 + 9.4) **PHASE 9 COMPLETE — STREAM DONE; 9.4 GATE PASSES.** Deleted the - hash_foreign token (9.0, `dfae690`); purged all `foreign` comments incl. capital-F - (src/examples/docs/editors); renamed 10 `*-foreign*` example files + dedup'd 1218 - (`b52d424`); rewrote 20 issues/*.md + renamed 0043 (`b9cfe25`). Gate: zero `foreign` - in the gated tree except `SQLITE_CONSTRAINT_FOREIGNKEY` + vendored sqlite c/. Suite - green (644/443). User flagged several leftover areas mid-purge (docs/, editors/, - capital-Foreign comments, the token) — all addressed. -- (9.3 src capital-Foreign) Fixed the case-sensitivity gap — my earlier src verify grep - was case-sensitive, missing ~21 capital `Foreign`/`FOREIGN` comments (Foreign-class→ - Runtime-class, Foreign path→Runtime path, Foreign decls→Extern decls, FOREIGN function→ - extern function, etc.) across calls/inst/ffi_objc/jni_descriptor/emit_llvm/c_import/ - lower.* /ops.zig. All reworded via Edit (comments only — no build impact). UNCOMMITTED - (mutating Bash blocked by a classifier outage). After this, src `foreign` = ONLY the - `hash_foreign` token machinery + 4 rejection messages (the 9.0-delete targets). -- (9.0 RATIFIED) User: DELETE the hash_foreign token (total purge). Pending build+regen. -- (9.3 text purge) Purged `foreign` from all `src/` comments (`dc51c4b`), specs/readme/ - CLAUDE (`e99383f`), and all example .sx comments (STAGED, commit pending a classifier - outage). Fixed 2 user-facing diagnostics (type-annotation error, Android jni_main help). - 1219 stdout labels Foreign→Extern (regen). Suite green (646/444). Remaining: example - FILENAMES + issues/*.md + the 9.0 token decision + 9.4 gate. -- (9.2a-d) **RUNTIME-CLASS IDENTIFIER PURGE COMPLETE** (Decision 5 → `Runtime*`). - 9.2a types (`3354446`), 9.2b fns+state+`is_extern` flag (`5c8af6e`, fixed `a15a868` - per user: reuse `is_extern` not new `is_reference`), 9.2c extern-ref validators → - `Extern` (`d27be42`), 9.2d `foreign_path`→`runtime_path` coupled across the build-hook - boundary + 37 `.ir` regens (`8cca3b9`). `src/` now has ZERO `foreign` identifiers - (only comments + the kept token/message remain). Suite green throughout. -- (9.1d) Eliminated the `foreign_expr` AST node — migrated `c_import.zig` auto-synth to - the extern shape, deleted the node + all readers. `refactor` `7ffdc7d`. -- (9.1c) Deleted dead `VarDecl.is_foreign`/`foreign_lib`/`foreign_name` (global `#foreign` - rejects → write-dead); 3 decl.zig readers simplified to `vd.extern_name`/`vd.is_extern`. - Snapshot-neutral; suite green (646/444). `refactor` `cd14794`. -- (9.1b) "foreign symbol already bound" diagnostic + resolveFuncByName panic → "extern - symbol"; intentional 1172 regen. Suite green. `refactor` `b78e7dd`. -- (9.1a) **PHASE 9 STARTED.** 5 collision-free linkage renames (callForeign→callExtern, - marshalForeignArg, dedupeForeignSymbol, foreign_name_map→extern_name_map, - is_foreign_c_api). Snapshot-neutral; suite green. `refactor` `b838f63`. Decision 6 - scoped by user: purge `.sx` + docs + our `src/` Zig, keep legitimate hits (SQLite API - names, vendored C, the rejection test/message + hash_foreign token). -- (8.1 cutover) **PHASE 8 COMPLETE.** Parser hard-rejects `#foreign` at all 4 sites - (const/global/fn-body via `self.fail`; runtime-class via `self.failAt` at the caller, - new helper); greens xfail 1176. Deleted obsolete 1174 + 1620, the GATE A→B test + - `lowerSrcToIr` helper; converted the in-source parser test to postfix `extern`; - `extern_export` → `const`. specs.md + readme.md drop `#foreign`. Suite green - (646/444). `feat!` `3811311`. -- (8.0 xfail) Added `1176-diagnostics-foreign-removed.sx` pinning the desired rejection. - RED (still accepted). `test`/xfail `8180faf`. -- (8 pre-cutover) Migrated the 4 multi-file example companions Phase 7 missed - (0729/a+b, 1617/c, 1623/mod). `refactor` `d132aab`. -- (8 pre-cutover) Migrated keyword-neutral diagnostics 1172 (decl→extern, message stays - internal "foreign symbol") + 1228 (→ two foreign-free extern symbols c_abs_one/_two), - intentional snapshot regens reviewed. `refactor` `720556b`. -- (8 pre-cutover) Migrated the 7 identity `ffi-foreign-*` test decls to extern/export - (decls only; comments left for Phase 9.3). `refactor` `2cce6a3`. -- (7.4 stragglers) **PHASE 7 MIGRATABLE WORK COMPLETE.** Migrated 16 fn/global examples - (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/1622/1628/1635/1636) `#foreign`→ - `extern`; 1607/1608/1616 (unmarked) verified by `sx ir` probes. 24-file keep-list remains - by design (deferred to Phase 8). Suite green (647/444). `refactor` `1a8991a`. -- (7.3 14xx) Migrated 13 jni examples (1410-1419/1423/1424/1425). 1417 (all-runtimes) hit - a parse-error trap: a `#(objc|jni)_class`-only regex left `extern` in PREFIX position on - `#jni_interface`/`#objc_protocol`/`#swift_*` lines → fixed with the GENERAL - `#foreign #(\w+)("X") {`→`#$1("X") extern {` rewrite (all such directives accept the - postfix modifier, probed). Kept 1426 (comment-only). Suite green. `refactor` `2888f6f`. -- (7.2 13xx) Migrated 18 obj-c examples (1308/1311-1321/1341-1347): prefix→postfix import - classes + fn markers. Kept identity 1306/1318, comment-only 1332/1348/1349. No 13xx - snapshot asserts on foreign. Suite green. `refactor` `a68f7c2`. -- (7.1 12xx) **PHASE 7 STARTED.** Migrated 12 incidental plain-C examples - (1200/1206/1209-1215/1220/1221/1222) `#foreign`→`extern`; output byte-identical, - empty snapshot diff, corpus-validated. Established the keep-list policy (see Last - completed step): kept 1172/1174/1620/1228 + ffi-foreign-* (1205/1207/1216/1218/1219) - + comment-only 1223/1229/1230/1231 for Phase 8. Suite green (647/444). `refactor` - `731fb8d`. -- (6.5 gpu) **PHASE 6 COMPLETE.** Migrated `gpu/gles3.sx` + `gpu/metal.sx` (3 sites); - `library/` now `#foreign`-free (`grep -rln '#foreign' library/` → 0). Verified - byte-identical `sx ir` on importers 1610/1606. Suite green (647/444). `refactor` - `32a7628`. -- (6.4 ffi) Migrated `ffi/` objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + - objc.sx's 2 import classes (prefix→postfix `extern`). objc/objc_block validated by 50 - marked 13xx examples; raylib/ffi-sdl3/wasm by `sx ir` probes pre/post. Empty snapshot - diff; suite green (647/444). `refactor` `666a2e2`. -- (6.3 std) Migrated 11 `std/` modules (~60 sites): cli/core/fmt/fs/log/net.kqueue/ - process/socket/thread/time/trace. All fn-decl markers (bare / `libc`|`tlib` LIB ref / - `libc "csym"` rename) → `extern …` + 2 comment mentions; no class forms. Host-corpus- - exercised → empty snapshot diff validates. Suite green (647/444). `refactor` `59f90d2`. -- (6.2 platform) Migrated `platform/` (uikit/android/android_jni/sdl3, ~64 sites): - 30 fn `#foreign;`→`extern;`, 34 import classes prefix `#foreign #objc/jni_class`→ - postfix `… extern {`, 4 defined `Sx*` objc classes → `… export {`. Behavior- - preserving, empty snapshot diff. Verified byte-identical `sx ir` on uikit importers - 1610/1606 + sdl3 probe; android via identical 4-error dedup set (host-only module). - Suite green (647/444). `refactor` `2cd5d7b`. NOTE: these runtime modules aren't in - the marker'd host corpus — verified out-of-band. -- (6.1 sqlite) **PHASE 6 STARTED.** Migrated `vendors/sqlite/sqlite.sx`: 97 - `… #foreign sqlib "csym";` fn decls → `extern sqlib "csym";` (+ line-9 comment). - `extern_lib` references the `sqlib` `#import c` unit like `#foreign sqlib`; IR - byte-identical, empty snapshot diff, example 1624 stdout unchanged. Suite green - (647/444). `refactor` `410a52e`. -- (5.1 gate annotate) **PHASE 5 COMPLETE.** Annotated the A→B gate header - (`lower.test.zig`) to record that post-Phase-5.0 the fn/global `#foreign` paths - build the same extern-named AST → cases 1/2 are structurally (not coincidentally) - identical; the gate stays as a regression tripwire. Added fn-rename case 2b - (`c_abs` → `"abs"`, `extern_name` axis), IR-identical per a `sx ir` probe. - Test-only, no snapshot churn. Suite green (647/444). `test` `93e7b6f`. -- (5.0 fn-body flip) **PHASE 5.0 PARSER ROUTING COMPLETE.** Flipped the fn-body - `#foreign` parser arm (`parser.zig:~2062`) onto the extern AST (empty-block body + - `extern_export = .extern_` + extern_lib/extern_name); `extern_export` made `var` so - the body arm can route onto it. Updated the parser unit test to assert the extern - shape. Behavior-preserving via the four prereqs; only example 1620's lib-ref message - churned ("#foreign library"→"extern library", Decision 7, hand-edited). Suite green - (647 corpus / 444 unit). `refactor` `6b94bb6`. -- (5.0 prereq plain-free xfail) Added `1230-ffi-extern-same-name-authors` (two flat - authors of `absval` via `extern libc "abs"`; the `extern` twin of `#foreign` 0729). - RED — extern authors wrongly counted as ambiguous (646/1 fail). `test`/xfail `2706521`. -- (5.0 prereq plain-free fix) `isPlainFreeFn`/`isPlainFreeFnDecl` now also exclude - `extern_export == .extern_` (external C symbol, no sx body; name-keyed first-wins like - `#foreign`); `export` stays plain-free. 1230 green (`absval = 7`). Suite green (646/444). - `fix`/green `3c94c14`. -- (5.0 prereq lib-ref xfail) Added `1231-ffi-extern-undeclared-lib` (`extern nosuchunit - "abs"` — bogus lib ref). RED — compiles silently (extern lib ref unvalidated). - `test`/xfail `38c3240`. -- (5.0 prereq lib-ref fix) `checkForeignRefs` (c_import.zig) now reads the lib ref from - either spelling (foreign_expr.library_ref OR extern_lib) and names the surface keyword, - so 1620 (#foreign) is byte-unchanged and 1231 (extern) gets "extern library … not - declared". 1231 green. Suite green (647/444). `fix`/green `ad6aed3`. **ALL FOUR fn-path - prereqs DONE → fn-body flip de-risked; awaiting Decision 7 (interim wording).** -- (5.0 prereq variadic xfail) Added `1229-ffi-extern-cvariadic` (JIT `#source`, - int-sum + double-avg, `extern` C-variadic). Expected snapshot pins the DESIRED - correct output. RED (variadic `extern` slice-packs extras → garbage: - `sum_ints(3,10,20,30)` → 53316585; doubles → 0.0). `test`/xfail `9a2c78d`. -- (5.0 prereq variadic fix) Extended the two C-variadic gates — the `is_variadic` - drop in `declareFunction` (`decl.zig:2097`) and the early-out in - `packVariadicCallArgs` (`pack.zig:302`) — to fire for `extern_export == .extern_` - as well as a `foreign_expr` body. 1229 green (`60` / `2.000000`). Suite green - (645 corpus / 444 unit, 0 failed). `fix`/green `0fdc821`. **BOTH fn-path prereqs - DONE → fn-decl `#foreign` body-marker migration unblocked.** -- (5.0 prereq vis xfail) Added cross-module example `1228-ffi-extern-c-non-transitive` - (main → b → c). Main references c's lib-less `#foreign` + `extern` twins - transitively; expected snapshot pins the DESIRED equivalent C-specific - diagnostic for both. RED (extern twin gets the generic "not visible" wording — - 443/444). `test`/xfail commit `717c35d`; the fix greens it. -- (5.0 prereq vis fix) Extended `isVisible(.c_import_bare)` (`decl.zig:2249`) to - switch on the body: a `foreign_expr` body OR an `extern_export == .extern_` decl - with no lib both route to `visibleOverEdges`; a library-bound decl stays - unconditionally visible. 1228 green — both twins emit "C function not visible". - Suite green (644 corpus / 444 unit, 0 failed). `fix`/green commit `7d8ba1a`. - **Deferred prereq (b) CLOSED.** Investigation this session also found - const-with-type is a DEAD parser path (defer per user) and the runtime-class - prefix is already coalesced (no Phase 5.0 change) — see Next step. -- (5.0 global) **PART B STARTED.** Routed the `#foreign` data-global parser path - (`parser.zig:425`) onto the extern-named `VarDecl` (`is_extern`/`extern_lib`/ - `extern_name`) — the same AST postfix `extern` builds. Behavior-preserving - (lowering coalesces both at `decl.zig:1119,1127,1141`); zero snapshot churn. Suite - green (444/444 unit, 643 corpus). `refactor` lock, commit `e5ddfbe`. Remaining - Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic - prereqs), runtime-class prefix (1305). -- (init) Plan written; FFI-linkage stream opened. -- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint. -- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword - classification + `lex linkage keywords` test. Suite green; no identifier collisions - in the corpus. `lock` commit. -- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` + - `VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2 - parser unit tests. Suite green (443/633). `lock` commit. -- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) + - added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback - (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering. - Suite green (443/633). `lock` commit. -- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots. - RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it. -- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing - mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc + - `lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green; - `declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit. -- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) + - hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse - error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also - recovered a formatter-clobbered `parser.zig` — see Known issues.) -- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into - `extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR - extern_name → declare under C name, map sx→C) and extends the dedupe guard to - extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443). - `green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis). -- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`, - mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse - error: var-decl `extern` not accepted). `xfail`; 1.2d greens it. -- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` → - `is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue` - consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green - (636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work. -- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run` - (JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC - JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling - (`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future - milestone (see Next step). Spike reverted; no commit. -- (2.0) Added the **AOT corpus mode** (`expected/.aot` → `sx build` + execute) to - `corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md - updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT - link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it. -- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both - define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower - export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green - (637/443). `green` commit. -- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls - `triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref - undefined). `xfail`; 2.2b greens it. -- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name + - sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so - the body promotes into the C-named function (`define @triple_c`). sx-side call sites - resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite - green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works. -- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`, - new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the - directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it. -- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl` - (optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed - `is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses - the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443). - `green` commit. **PHASE 3 COMPLETE.** -- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`, - parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class, - `counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per - the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens` - newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr - 0-byte per repo convention; runner normalizes both.) -- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`: - `#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class. - Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the - equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three). - Suite green (641/444). `test` commit. -- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign` - + postfix `export` on an aggregate previously surfaced a confusing internal - "emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`: - `parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green. -- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the - `#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit. -- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on - one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`: - `parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite - green (643/444). **PHASE 4 COMPLETE → PART A DONE.** -- (golden-fix) **`-Dupdate-goldens` churn RESOLVED.** Root cause was NOT a code bug: - `writeGolden` always writes `content + "\n"` (empty → canonical 1-byte `\n`, used by 484 - of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte - *outliers* (verify trims trailing `\n` so both forms passed, but regen always rewrote them - to 1-byte). Conformed all 5 to the 1-byte form → `-Dupdate-goldens` is now idempotent, no - more churn. (Separately: a flaky `0712-sha256-streaming` >10s timeout appears only under - concurrent `zig build` load — not a real failure; re-run serially.) - -## Known issues -- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the - working-tree `src/parser.zig` between commits — it reformatted one-liners AND - silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse - error. Recovered with `git checkout src/parser.zig` (HEAD had the correct, - committed version). **After any Edit-tool change to a file the IDE may have open, - rebuild + run the affected example before trusting the edit.** diff --git a/current/CHECKPOINT-FIBERS.md b/current/CHECKPOINT-FIBERS.md deleted file mode 100644 index cd278f8f..00000000 --- a/current/CHECKPOINT-FIBERS.md +++ /dev/null @@ -1,980 +0,0 @@ -> **SUPERSEDED (2026-06-28).** The fiber-async API tracked here — `Task` / `go` / `wait` -> / `cancel` — was RETIRED and folded into the unified `context.io` async stack in -> PLAN-IO-UNIFY Phase 5. See `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for -> the current design. The `Scheduler` ENGINE primitives (swap_context, mmap stacks, -> spawn / run / yield_now / suspend_self / wake / sleep / block_on_fd, virtual-time -> timers) REMAIN and are still accurate. Below is a historical record. - -# CHECKPOINT-FIBERS — Stream B1 (fibers + Io + M:1 scheduler) - -Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step at a time, -per the cadence rule). New corpus category: `18xx` concurrency. - -## Last completed step -**B1.6 — aarch64-LINUX port of the M:1 fiber runtime (sched.sx).** `library/modules/std/sched.sx` -now runs end-to-end on aarch64-linux as well as aarch64-macOS, validated **byte-identical** on both -via Apple `container` (static ELF, no emulation). The per-OS bits are comptime-branched: -- `MAP_AP` (mmap MAP_ANON flag) — `inline if OS == { case .linux: 0x22 case .macos: 0x1002 }`, - exhaustive on the supported OSes (no default → a new target fails loud on use). -- The fd-readiness backend — kqueue on darwin, **epoll on linux**. The `epoll` import is scoped to - the linux branch (`inline if OS == .linux { ep :: #import "modules/std/net/epoll.sx" }`) so darwin - never pulls epoll types into the concurrency examples (the std-barrel-drift rule). `block_on_fd`, the - run-loop Mode-2 drain, and `cancel_io_waiter_for` each branch kqueue/epoll; epoll additionally - `EPOLL_CTL_DEL`s on fire + on early-wake (EPOLLONESHOT only DISABLES, kqueue EV_ONESHOT auto-removes). -- The first-entry trampoline was redesigned from a per-OS hand-written global-asm symbol to a **naked - sx fn** `fib_tramp` (`mov x0, x19; br x20`) + register-indirect dispatch (spawn presets - `regs[1] == x20 == &fib_dispatch`), so no per-OS `.global _fib_tramp`/`fib_tramp` symbol literal is - needed. This sidesteps a compiler bug (wrapped top-level `asm` dropped — now **issue 0194**, OPEN). - -**Bug fixed en route (issue 0193 Bug A):** the tramp redesign initially bus-errored on the 1817 -go/wait/sleep capstone (both OSes) because `fib_dispatch` was no longer pinned to the C-ABI (the -original pinned it via `export "fib_dispatch"`, which the redesign dropped). Without a C-ABI pin -`fib_dispatch` uses sx's internal ABI (x0 = implicit `context`, `self` shifted to x1), but the -trampoline hands `self` in x0 (C-ABI) → on first entry the body runs (x1 happens to alias `self`) but -the closure then loads `regs[1] == &fib_dispatch` as its first capture and recurses forever → stack -overflow. **Fix: annotate `fib_dispatch` `abi(.c)`** (pins it to C-ABI, `self` in x0, with no public -symbol — it's reached only by address through the trampoline). Root cause found -via lldb on an AOT macOS build; confirmed by an adversarial source review (`src/ir/lower/decl.zig`). -The 1817 capstone in the suite guards the fix. Suite GREEN **817/0**; 1811/1814/1816/1817 byte-identical -macOS host ↔ aarch64-linux container. - -### Earlier — B1 follow-up — `Scheduler.deinit` (close the bounded leaks). Post-B1 non-blocking cleanup: a -terminal `deinit` on `library/modules/std/sched.sx`'s `Scheduler` releases the resources B1 documented -as leaked. Frees, in order: (1) any fibers still enqueued ready (leak-safety net for `spawn`/`go` -without `run()` — `munmap` stack + free struct; a suspended off-queue fiber is unreachable, but a clean -`run()` aborts on orphans so none survive it); (2) every heap `*Task` from `go` — newly tracked via a -`task_allocs: List(*void)` field appended in `go` (the scheduler otherwise has no handle on its generic -`Task($R)`s); (3) the three `List` backings (`task_allocs`/`timers`/`io_waiters`, all grown through -`own_allocator`); (4) the lazily-opened kqueue fd (`close`, reset to `-1`). NOT freed (unchanged -language limitation): the per-`spawn`/`go` closure env (sx exposes no env-free). Idempotent (rests on -`List.deinit` nulling `items` + the `kq`/`ready_head` resets); TERMINAL contract — no scheduler-owned -handle (`*Task`, `*Fiber`, the scheduler) is usable after `deinit`. -- Added a canonical `close :: (i32) -> i32 extern libc` (matches the dedupe-canonical signature 1816 - already uses) + the `task_allocs` field. -- Locked by `examples/concurrency/1820-concurrency-fiber-scheduler-deinit.sx` (aarch64-macOS `.build - {"target":"macos"}`, runs end-to-end): one run touches every freed resource — a SLEEPER (`timers`), a - pipe READER `block_on_fd` + WRITER (kqueue fd + `io_waiters`), two `go` tasks (`Task`s + `task_allocs`) - — then `deinit`. Verified by a tracking `GPA`: `freed by deinit: 5`, `live after deinit: 5` (the - RESIDUAL = the 5 documented closure envs, not a bug), `kq open after run: true` → `kq after deinit: - -1` (the genuinely-open kqueue fd is closed), `read: 3 [97 98 99]` (the fd path actually ran). Counts - captured into locals BEFORE printing (`print` allocates format temporaries through the same GPA). -- **Adversarially reviewed (worker):** no real memory-safety bug in the supported (deinit-after-`run`) - path — reap-loop reads `f.next` before freeing `f`, the three freed List backings + Tasks + kq are all - disjoint + scheduler-owned, no over-free, idempotent. The one CRITICAL it raised was a DOC contradiction - (step-(1) defensive reap vs step-(2) "post-run only"), reconciled by spelling out the terminal contract. - Its 0154-over-store concern (`.{}`→`List` writes in `init` could clobber `kq`) was PROBED and cleared: - `kq == -1` immediately after `init`, all fields clean. Suite GREEN **759/0**. - -### Earlier — B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE -A single capstone exercises the whole -colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async -`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)` -`swap_context` on guarded `mmap` stacks (B1.0–B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`: -a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in -SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in -DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual -clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**. - -**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the -`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler, -suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via -kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx` -(1800–1817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1, -zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a; -0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`), -0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`; -routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env -/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve. - -### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS) -`library/modules/std/sched.sx` now lets a -fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready. -Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes), -`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT` -constants) rather than re-deriving the FFI — sched.sx imports it as `kqb`. Added to `Scheduler`: -- `kq: i32` (LAZY — `-1` in `init`, opened by the first `block_on_fd`, so a pure-compute / - virtual-timer scheduler never opens a kqueue fd; leaks one fd at exit once opened, same class as the - documented spawn-env / go-Task leaks — no deinit yet); -- `io_waiters: List(IoWaiter)` (`IoWaiter :: struct { fd: i32; fiber: *Fiber; }`, grown through - `own_allocator` per the long-lived-container rule); -- `block_on_fd(self, fd, want_read)` — lazily opens `kq`, arms a one-shot `EVFILT_READ` registration, - records an `IoWaiter{fd, current}`, then `suspend_self()`. Guards a null `current` (loud abort, like - `sleep`); `want_read=false` (write-readiness) is not wired yet → loud abort rather than silently - arming a read filter. -- Run-loop: after the ready queue drains, **Mode 1 (virtual time)** fires the earliest pending timer - (takes precedence — a program uses `sleep` OR fds, documented non-unification limitation); **Mode 2 - (real fd)** — if `io_waiters` is non-empty, BLOCK on `kq_wait(kq, evbuf, MAXEV=16, -1)` (null - timeout), then for each fired event match `ev.ident` back to its waiter, evict it, and `wake` the - fiber; **else** break. Orphan-deadlock check unchanged in spirit but now correct: an fd waiter is NOT - an orphan (while `io_waiters.len > 0` the loop blocks on kqueue rather than reaching the check), and - a genuine no-timer/no-fd suspend still aborts loudly (verified with a probe: exit 134). -- `wake` now also evicts a pending fd-waiter (`cancel_io_waiter_for`, mirror of `cancel_timer_for`) — - same UAF reasoning: a fiber woken by another path must not leave a stale `IoWaiter` pointing at a - reaped `*Fiber`. The kqueue registration is `EV_ONESHOT` so we never `EV_DELETE` (a never-fired - one-shot lingers harmlessly; the drain ignores an unmatched ident; closing the fd auto-removes it). -- DE-RISK probe (run first, no scheduler): confirmed `size_of(Kevent) == 32`, the pipe roundtrip - (`kq_wait` returned 1 with `out.ident == read_fd`, `out.filter == -1` (EVFILT_READ), `out.data == 1` - byte readable) — the struct layout reads back the fd correctly. -- Locked by `examples/concurrency/1816-concurrency-fiber-io-pipe.sx`: a `pipe`; a reader fiber spawned - FIRST blocks on the empty read end, then a writer fiber writes `a b c` → the run loop blocks on - kqueue, wakes the reader, which reads the 3 bytes. Output `log: wrote read 3 [97 98 99]` / - `n_suspended: 0` (the "wrote" before "read" ordering proves the reader actually blocked then woke via - kqueue readiness). `.build` `{ "target": "macos" }` (matches host arch → runs end-to-end; ir-only on - a mismatch, like 1814/1815 — no `.ir` snapshot needed since it runs here). The example declares its - own `read`/`write`/`close` externs with the CANONICAL signatures std already binds - (`(i32,[*]u8,usize)->isize` / `(i32)->i32`) — a divergent re-binding is rejected by the extern dedupe. -- **Adversarial review (worker) of the run-loop change — found 2 CRITICALs:** - - **(1) two fibers on the SAME fd → lost wakeup + permanent hang.** macOS `EV_ADD` for an existing - `(ident, filter)` REPLACES the registration (doesn't stack), so two waiters share one registration: - the fd fires once, one wakes, the other is stranded in `io_waiters` and the next `kq_wait(-1)` blocks - forever. FIXED: `block_on_fd` now enforces one-waiter-per-fd with a loud abort (the model already - assumed it). Verified: dup-fd → `sched: block_on_fd: fd N already has a waiter`, not a hang. - - **(2) an fd-waiter on a never-ready fd hangs instead of the timer path's loud abort.** Re-examined: - this is CORRECT event-loop semantics — blocking on I/O until ready (possibly forever, like a server - idling on a socket) is the point; the scheduler cannot know an fd will never become ready, so it must - keep waiting. NOT a scheduler deadlock. Fixed the MISLEADING comment that implied the orphan check - covers fd-waiters: it does not, by design (it covers only pure `suspend_self` parks). No code change — - the "hang" is a caller-side logic issue (waiting on input that never arrives), not a bug to abort on. - - Review CLEARED: the IoWaiter UAF parity (early-wake evicts the waiter; a lingering one-shot that later - fires hits no match → clean no-op), ident width/sign, `kq_wait` EINTR/error handling, timer-vs-io - precedence (timer wins; no hang). All probed safe. -- Suite GREEN **754/0** (incl. the dup-fd guard, no new example needed — the abort is host-fragile to - pin like 1809's guard-firing). Next: **B1.5** (end-to-end M:1 validation under the deterministic timers - / fd readiness); a linux epoll twin of `block_on_fd` (mirror via `std/net/epoll`, the OS-neutral facade - is `std.event`) is future work. - -### Earlier — B1.4b — deterministic VIRTUAL-TIME timer scheduling (the KEYSTONE) — landed + adversarially -reviewed (caught a CRITICAL UAF, fixed). -`library/modules/std/sched.sx` gained a virtual clock + -sleep timers so fibers schedule in reproducible simulated time (no real clock): `clock_ms` (advances -ONLY as timers fire), a `timers: List(Timer)` (insertion-order, linear min-scan, FIFO tiebreak), -`now_ms()`, `sleep(ms)` (arm `{clock_ms+ms, current}` + `suspend_self`), and a timer-driven `run` -(drain ready → fire earliest timer → advance clock → wake → repeat; orphan-deadlock check preserved -for a genuine no-timer suspend). Locked by `1814` (5 fibers sleep 30/10/20/15/15 → wake order -B@10, D@15, E@15 (FIFO), C@20, A@30 — deadline order, not spawn order; `now_ms()` reads each virtual -deadline; final clock 30). §8.1.3 calibration note in the header: the deterministic wake ORDER -equals what real `sleep`s produce, reproducing blocking semantics' observable ordering without real -time. The deterministic-sim `Io` is realized at the scheduler level (`sleep`/`now_ms`/timer-`run`), -not as an erased `Io`-protocol impl (same erasure reason as FiberIo). -- **Adversarial review (worker) of the run-loop change: found a CRITICAL use-after-free** — a fiber - that armed a `sleep` timer but was woken EARLY by another path (a manual/`Task` `wake`) ran to - completion + was reaped (stack `munmap`'d, `Fiber` freed) while its `Timer` still held a dangling - `*Fiber`; a later fire would `wake` freed memory (silent-corruption: "passes" only because the - freed slot coincidentally read `state != .suspended`). FIXED: `wake` now evicts the woken fiber's - pending timer (`cancel_timer_for`) — every re-ready path funnels through `wake` (the timer-fire in - `run` already removed the fired timer, so it's a harmless re-scan there), so no stale timer can - outlive its fiber. Regression `1815-concurrency-fiber-timer-early-wake.sx` (early wake → `clock: 0`, - the stale 100ms timer evicted, not fired). Review CLEARED: `n_suspended` accounting, - orphan-deadlock false-positives, timer-list integrity (re-arm during fire), clock monotonicity, - termination — all traced/probed safe. -- Suite GREEN (count below). Next: **B1.4c** (event-loop `Io` — real fd readiness, kqueue/epoll). - -### Earlier — B1.4a — a truly-SUSPENDING fiber-task async layer (`go`/`wait`/`cancel`) -landed + adversarially reviewed; cleared two more compiler blockers en route. `library/modules/std/sched.sx` -now carries `Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` (a `ufcs` layer over -the M:1 scheduler). `s.go(work)` runs the nullary thunk `work` as a REAL fiber; `t.wait()` SUSPENDS -the caller until it completes (vs io.sx's blocking `context.io.async`, which runs inline). Locked by -`examples/concurrency/1813-concurrency-fiber-async-suspend.sx`: two tasks interleave (A yields -mid-body so B runs first → `1 2 3`), awaited values `42`/`100`, and a canceled task's `wait` raises -`.Canceled` → `or -99` → `sequence: 1 2 3 42 100 -99`. -- **Design: a NULLARY thunk, not `async(worker, ..args)`.** A comptime variadic pack can't cross a - deferred (fiber) boundary — `..args` captured into a closure re-expands from the spawner's - now-gone locals (issue 0156 Part 2). So `go` takes `work: Closure() -> $R`; the user captures - inputs in the lambda at the call site (the `go func(){…}()` idiom). **Self-contained in sched.sx** - (NOT io.sx): io.sx importing sched.sx duplicates the `_fib_tramp` global asm when a program also - imports sched.sx directly (global asm emits per import-path) — so the Io-protocol - `spawn_raw`/`suspend_raw`/`ready` hooks stay reserved for the future M:N model; M:1 uses - `go`/`wait` directly. Heap `*Task` (must outlive `go`'s frame; leak documented). `TaskErr` is - LOCAL (the `!` failable detection doesn't see through io.sx's `IoErr` re-export alias). -- **Two compiler blockers hit + FIXED (user-authorized in-session):** - - **issue 0156 Part 1** — a single-type generic `$R` (parsed as `comptime_pack_ref`) used as a - type-arg (`Box($R)`, `size_of(Box($R))`) inside a pack-fn body hit a missing arm in - `resolveTypeWithBindings` → `.unresolved` → LLVM panic. Fix: mirror `resolveTypeArg`'s - `comptime_pack_ref` arm (look up `type_bindings`, else a loud diagnostic). Regression - `examples/generics/0216-generics-typearg-in-pack-fn-body.sx`. (Part 2 — deferred `..` spread - crashes — reframed OPEN/non-blocking, `issues/0156`.) - - **issue 0157** — a user generic `ufcs` method whose name collides with a stdlib re-export - (`cancel` on `*Task` vs io.sx's `cancel` on `*Future`) resolved via last-wins `fn_ast_map` with - NO receiver filtering → wrong overload → `$R` unbound → LLVM panic. Fix - (`src/ir/lower/call.zig` `selectUfcsGenericByReceiver`): every generic-ufcs dispatch enumerates - ALL module authors (`module_decls`), keeps receiver-binding ones, picks the most - receiver-SPECIFIC (concrete > bare `$T`), dedups re-exports, and flags a genuine 2-specific tie - as a deterministic "ambiguous — qualify" diagnostic (never a silent order-dependent pick). - Regression `examples/generics/0217-generics-ufcs-method-name-collides-stdlib.sx`. -- **Adversarial review (worker) of the 0157 fix + Task layer.** Caught the determinism CRITICAL - (fixed: always-run selection + specificity + ambiguity), `wait`-outside-a-fiber null-deref (fixed: - loud guard in `suspend_self`/`yield_now`), and cancel-doesn't-skip-work (fixed: worker skips - `work()` if already canceled). Lost-wakeup / cancel-after-complete / reap traced safe. Also - simplified `1812` (`**Fiber` shared handle → a `Sh.parked` field; output identical). -- Suite GREEN 751/0 (749 + 1813 + 0217). Next: **B1.4b** (deterministic-sim `Io`). - -### Earlier — B1.5a — the M:1 cooperative fiber scheduler CORE — landed + adversarially reviewed -The hand-bootstrapped ping-pong (1807-1810) is now a reusable scheduler API in pure sx: -`library/modules/std/sched.sx` — a generic `Fiber` (`body: Closure() -> void`) + `Scheduler` -with `init`/`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over the proven `swap_context` on -guarded `mmap` stacks. The ONE generic dispatch (`fib_dispatch`, reached from the `_fib_tramp` -trampoline) runs ANY stored closure body on a fresh stack — replacing the fixed `bl _fib_body`. -Reaping `munmap`s the stack + frees the heap `Fiber` on completion; an intrusive FIFO gives -round-robin order. -- **Foundational design de-risked by probe before building:** a fiber can store + call a - `Closure() -> void` on its fresh stack via the generic dispatch; outputs flow OUT through - pointers captured in the closure (capture-by-value does NOT write back — pushed onto the user). -- **Hit + FIXED a blocker compiler bug — issue 0154** (user-authorized in-session fix). `null` / - `---` assigned to a struct field picked up a leaked enclosing `target_type` (the function's - RETURN type, set for the whole body at decl.zig:2691) and built a WHOLE-STRUCT-typed null → - an oversized `zeroinitializer` store through the field's GEP that overran the field's slot and - clobbered the saved x29/x30, so the fn `ret`'d to 0x0. This was EXACTLY the `Scheduler.init()` - by-value-return shape (`sched_ctx: [13]u64` before `current: *Fiber`). Fix: added - `.null_literal, .undef_literal` to the `needs_target` switch in `lowerAssignment` - (`src/ir/lower/stmt.zig`) so the field's type is used. Repro → regression test - `examples/types/0193-types-sret-array-before-pointer.sx`; `issues/0154-*.md` RESOLVED. -- **Adversarial review (worker): asm/bootstrap/lifetime SOUND** (the headline closure-env-lifetime - fear was disproven — envs are heap-promoted, survive the spawn frame). Found **1 CRITICAL** + - robustness gaps, ALL hardened: (CRITICAL) `wake` re-enqueued an already-queued fiber → - FIFO corruption/segfault → now GUARDED on `.suspended` (spurious/double/stale wake = safe - no-op); orphan-suspend leak/deadlock → `n_suspended` accounting + a loud `run()`-drain - diagnostic+abort; `mmap` `MAP_FAILED` (=-1, not null) / `mprotect` / Fiber-OOM → loud bails - (per §8.1.1 the guard is mandatory); the per-fiber closure-env leak (sx exposes no env-free) → - documented as a KNOWN LIMITATION (bounded by spawn count; invisible under the default GPA). -- **Locked two `18xx` examples** (aarch64-macos `.build`-pinned, ir-only on a mismatch): - `1811-concurrency-fiber-scheduler.sx` (3 fibers round-robin via `yield_now` → ordering contract - `sequence: 0 1 2 0 1 2 0 1 2`, all `.done`) + `1812-concurrency-fiber-suspend-wake.sx` (park via - `suspend_self`, resumed by another fiber's `wake`, + the spurious-wake no-op — the CRITICAL-fix - regression → `log: 10 20 21 11` / `suspended-left: 0`). -- **Filed issue 0155 (NON-blocking, NOT fixed)** — found incidentally in the review: indexing a - scalar pointer (`pc[0]`, `pc: *i64`) panics codegen (`.unresolved` reaching LLVM emission). The - scheduler uses array-field indexing + `.*`, never this, so it's filed for its own session. -- Suite GREEN **748/0** (746 base + 1811 + 1812 + 0193 regression). Next: **B1.4a** (FiberIo — - wire `Io.spawn_raw`/`suspend_raw`/`ready` onto the scheduler so `async`/`await` truly suspend). - -### Earlier — B1.3b-1 — the x86_64 / Win64 `swap_context` sibling — VALIDATED on real hardware -The -context switch is now proven on a SECOND architecture + ABI. A Win64 `swap_context` saves the -COMPLETE Win64 callee-saved set — 8 GP (rbx, rbp, rdi, rsi, r12-r15) + rsp **and xmm6-xmm15** -(10 XMM, 128-bit via `movups` — Win64 has callee-saved XMM, unlike SysV/aarch64) — plus a Win64 -`scribble_verify` (32-byte shadow + 16-align at each `call`, COFF symbols, rsp-carried return -addr). Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`, -ir-only here): the 2-fiber mutual scribble printed **`0 0 P`** when built `--target -x86_64-windows-gnu --self-contained` and **run on a Windows 7 x64 VM (UTM)** — every GP + XMM -callee-saved survived. **Adversarially reviewed before the VM run** (worker emitted the real `.s` -and verified every `call` alignment, the 264-byte frame offsets, the rsp/return-addr round-trip, -swap ordering, and COFF naming against the Win64 ABI — no critical/minor bugs). The build→VM→run -loop was set up this session (cross-build needs `--self-contained`; output via the Win32 -`WriteFile` boundary, the 1660 pattern). Suite green. Note: this is the GOOD-swap-only mutual -scribble (self-validating by construction; the in-process negative control was dropped to avoid an -sx fn-ptr-convention rabbit hole — the detection of this exact logic was negative-controlled on -aarch64 in 1808). The SysV/Linux x86_64 sibling (different reg set: no callee-saved XMM, args -rdi/rsi) remains for a Linux x86_64 host. - -### Earlier — B1.3b-2 — mmap guard-page stacks (commit `dd532ab`) -Fiber stacks are `mmap`'d with a `PROT_NONE` GUARD PAGE at the low end (§8.1.1: a -fixed stack without a guard silently corrupts neighbors on overflow). `mmap` the `[guard | -usable]` region, `mprotect` the low 16KB page `PROT_NONE`; SP descends into the guard and faults -loudly at the boundary instead of corrupting a neighbor. Locked by -`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1` -(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields). -- **Guard FIRING validated** (manually, not corpus-pinned — a deliberate overflow crash is - host-fragile): a fiber recursing past its 128KB stack faults with `Bus error` at the guard page - (`region+GUARD`); the sx crash handler turns it into exit 134. Documented in the example header. -- **x86_64 sibling:** was deferred here (couldn't run x86_64 on this arm64 host), then DONE as - Win64 once a Windows 7 x64 VM became available — see B1.3b-1 above (`examples/1810`, `0 0 P`). - -### Earlier — B1.3a-2 — the context-switch STRESS GATE (design §10.7) — DONE + adversarially reviewed -The explicit every-callee-saved-register scribble that B1.3a-1 owed. `swap_context` now saves the -COMPLETE AAPCS64 callee-saved set — integer x19-x28 + fp/lr + sp AND FP **d8-d15** (per §6.1.2 -only the low 64 bits of v8-v15 are callee-saved, so `d8-d15` is exactly sufficient; x18 is Apple's -reserved platform reg, untouched). A naked `scribble_verify(self_ctx, peer, base)` loads a unique -sentinel into all 18 callee-saved regs, yields, and on resume counts the ones that didn't survive -(honoring its own caller ABI via a 176-byte frame that saves+restores the caller's callee-saved; -base reloaded from the frame post-swap; the original lr round-trips through the swap). The gate is -a **2-fiber MUTUAL scribble** (A and B scribble DISTINCT sentinels into the same physical regs, so -each survives only if `swap_context` saved+restored it — a lone fiber yielding to an idle peer -would NOT exercise preservation). Locked by `examples/1808-concurrency-fiber-switch-stress.sx` -(aarch64-pinned): `A mismatches: 0` / `B mismatches: 0`. -- **Validity proven by NEGATIVE controls:** dropping the d8-d15 save/restore → 8/8 mismatches - (exactly the FP regs); dropping x27/x28 → 2/2. The gate genuinely catches a broken switch. -- **Adversarial review (worker, per the plan): no CRITICAL bugs.** Verified the callee-saved set - is complete + correct, all frame offsets/16-alignment, the lr/sp dance, and swap read-ordering - against AAPCS64. Applied its one recommendation: `boot` now zeroes the FP ctx slots [13..20] so a - first switch-to loads 0 (not garbage) into d8-d15. Residual gaps it flagged (all spec-correct - for a call-boundary swap, documented in the example header): NZCV/FPSR not swapped; **FPCR** - (rounding mode — thread-global, bleeds across fibers if changed) and **TPIDR_EL0/TLS** (errno, - allocator thread-caches — shared by same-thread fibers) not swapped; fp=0 bootstrap blocks - unwind/signal walking past a fiber trampoline. These bite at the N×M:1 / signals stages, not the - single-thread switch. -- Suite green **734/0**, master clean. WIP probes: `.sx-tmp/scribble2.sx` (+ `_broken`/`_gp`). - -### Earlier — B1.3a-1 — the foundational stackful context switch (commit `b234b7d`) -Pure sx over `abi(.naked)`: naked `swap_context` (GP-only 13-slot save) + by-hand fiber bootstrap -(SP = `alloc_bytes` stack top, LR = global-asm trampoline, x19 = `*Fiber`). Locked by -`examples/1807-concurrency-fiber-context-switch.sx`: 2-fiber ping-pong (`rounds: 6` / `canary -fails: 0`) + 64-frame deep recursion (`frames verified: 64` / `depth fails: 0`). Indirect -register/stack survival; 1808 supersedes its switch with the complete GP+FP save area + the -explicit gate. - -### Earlier — B1.2 COMPLETE — the async surface works end-to-end -All three surface blockers (0151, 0152, 0153) FIXED + committed; async examples landed + green. -- **0151 fixed** (`362674f`): generic `$T` infers through generic-struct / pointer / UFCS-pack - params. Regression `0214` + `0215`. -- **0152 fixed** (`e5586f6`): `Atomic(bool)` load/store byte-promoted to `i8` in the codegen - emitters. Regression `1705`. -- **0153 fixed** (`68c1991`): `inferGenericReturnType` now pins return-type resolution to the - fn's DEFINING module (mirroring `monomorphizeFunction`), so a re-exported value-failable's - `!E` resolves to the real `.error_set` TypeId — the failable channel survives the re-export - alias. Regression `1058-errors-reexport-value-failable-channel.sx`. -- **Async examples landed:** `examples/1805-concurrency-io-blocking-async.sx` - (`context.io.async((a,b)->i64 => a+b, 40, 2).await() or {…}` → `sum: 42` / `double: 42` / - `clock ok`) + `examples/1806-concurrency-io-cancel.sx` (`f.cancel()` → `await` raises - `.Canceled` → `or` default; `ok: 7` / `canceled: -99`). Both green, snapshots captured. - -### Earlier — the three B1.2 surface fixes (committed) -Generic `$T` inference, `Atomic(bool)` byte-promotion, and re-export failable-channel pin — -details below. -- **0151 fix (committed):** four gaps closed on the inference + UFCS-dispatch path — - (1) `extractTypeParam`/`matchTypeParam(Static)` got a `parameterized_type_expr` arm - (recover the arg instance's recorded per-param bindings via `struct_instance_bindings` + - the template's ordered `type_params`, recurse positionally; this also fixes `*Box($T)` — - it recurses into its `Box($T)` pointee); (2) the `pointer_type_expr` arm now falls through - to match the pointee against a non-pointer arg (auto-address-of: a `*Box($T)` param accepts - a by-value `Box($T)`, e.g. a UFCS receiver `b.m()`); (3) `ExprTyper.inferType` got a - `.lambda` arm building the closure type from the lambda's annotations (the UFCS binder types - args from the raw AST before they're lowered, so it can now bind `Closure(..) -> $R` from - the worker's declared return type); (4) a pack UFCS target routes through the SAME - `lowerPackFnCall` the direct call uses, with the receiver spliced in as `args[0]`. -- Regression tests: `examples/0214-generics-ufcs-closure-return-pack.sx` (direct + UFCS - closure-return pack) + `examples/0215-generics-infer-through-pointer.sx` (by-value / - pointer / multi-param / nested / UFCS-auto-ref struct-head inference). Issue 0151 marked - RESOLVED; repro moved into the suite. - -### Earlier — B1.2 (Io capability) — LANDED + adversarially reviewed -Commits `a1b14f0` (lock) + `45d869d` (Io capability) + `3eeb965` (issue 0151 lock). -- **LANDED + review-confirmed correct** (commit 45d869d): `Io :: protocol #inline` - (spawn_raw/suspend_raw/ready/poll/now_ms/arm_timer) + `io` field on `Context` - (`{allocator; data; io}`, io LAST); BOTH `__sx_default_context` materializers - (protocol.zig + comptime_vm.zig) build an identical CBlockingIo→Io vtable (review verified - byte-for-byte agreement; `context.io.now_ms()` dispatches at runtime AND comptime); the - `push Context.{…}` omitted-field-**inherits-ambient** fix (review: correct, right fix, no - bad blast radius); `library/modules/std/io.sx` (`Future($R)`, `CBlockingIo`, - `async`/`await`/`cancel`); the `!`-protocol-impl-lint suppression; 37 `.ir` regens - (review: pure layout/type-table, no error text, zero .exit/.stdout/.stderr change). -- **BLOCKED — async surface non-functional:** `await`/`cancel` take `*Future($R)` and are - **uncallable in EVERY form** (not just UFCS) — sx can't infer a generic `$T` from a - pointer-wrapped arg (`*Future($R)`). `async(...)` (create) works via explicit call and - produces a correct `.ready` Future, but you can't `await` it. Root bug = **issue 0151 - (WIDENED)**: infer `$T` from `*T`-wrapped params + closure-return-via-pack + UFCS dispatch. - Minimal repro: `unbox :: (b: *Box($T)) -> $T` fails to infer `T`. -- **No async example in the corpus** (1805 was removed because it needs the blocked surface) - → the green suite does NOT cover async. Restore `1805` (async/await) + add `1806` (cancel) - once 0151 is fixed. - -### Earlier — B1.1 (per-fiber `context` root) — DONE. Zero compiler change (confirmed by probe). -The fiber-spawn context convention works end-to-end with ordinary language features: -- `snap := context` captures the spawner's `Context` as a value; -- the snapshot is stored in a struct (the stand-in `Fiber`); -- a trampoline running under a *different* ambient context installs the fiber's stored root - with `push f.root { … }`, and the body reads the snapshot — not the trampoline's ambient - context — because `context` is an implicit slot-0 `*Context` param (call-carried, rides the - callee's own stack) and `push` allocates on the caller frame (no global, no TLS). -- Locked by `examples/1804-concurrency-context-snapshot.sx`: prints `fiber root: 42` (the - installed snapshot wins over ambient 99) + `ambient after: 99` (the `push` scope restores - the ambient context on exit). No fiber runtime yet (that's B1.3) — this proves the plumbing - it will build on. No `.build` pin (pure sx, host-independent). -- **Probe result:** the design doc's "lower as swappable indirection, never raw TLS" guarded - a non-problem — context was already param-carried, never TLS. No path re-reads - `__sx_default_context` mid-stack, so there is **no compiler obligation** here. -- `zig build && zig build test` green: **726 ran, 0 failed**. - -### Earlier — B1.0 (`abi(.naked)` codegen) — complete -Replaced the emit bail with real LLVM `naked` emission: -- `emit_llvm` declaration pass: for `func.is_naked`, add the LLVM `naked` + `noinline` + - `nounwind` attributes and **skip** the `frame-pointer=all` attribute (incompatible with a - frameless function). Pass 2 now emits the `.naked` body normally — `naked` makes the - backend emit it verbatim (the inline asm + its own `ret`) with no prologue/epilogue. -- IR shape (verified): `; Function Attrs: naked noinline nounwind` / `define internal i64 - @answer() #0 { entry: call void asm sideeffect "…ret…", ""() unreachable }` / - `attributes #0 = { naked noinline nounwind }`. The caller invokes it as an ordinary - `() -> i64` call (`.naked` is `call_conv == .default`). -- `examples/1800-concurrency-naked-asm.sx` — now GREEN, aarch64-pinned (`.build {"target": - "macos"}`): runs end-to-end → **exit 42** on this host, ir-only on a mismatch; `.ir` - snapshot captured. -- `examples/1801-concurrency-naked-generic.sx` (renamed from `-bail`) — the generic `.naked` - now emits a correct naked `answer__i64` (exit 42), proving generic.zig produces a naked - body, not a framed one. aarch64-pinned. -- `examples/1802-concurrency-naked-asm-x86.sx` — x86_64 cross sibling (`.build {"target": - "x86_64-linux"}`, ir-only here): `.ir` locks `naked` + `movl $42, %eax` / `ret`. -- Unit test `emit: abi(.naked) function gets the naked attribute (no frame-pointer)` in - `emit_llvm.test.zig` (asserts `naked` present, `frame-pointer` absent). -- **B1.0c (review-hardening):** a param-bearing `.naked` fn emitted invalid LLVM (loud - verifier error "cannot use argument of naked function") because the param-alloca loop - wasn't gated. Fixed forward (this *enables* the B1.3 context-switch use case rather than - rejecting it): gated the param-alloca loop on `fd.abi != .naked` in decl.zig (both paths) + - generic.zig; a naked fn's args stay in registers (read by asm), declared-but-unused in - LLVM. Locked by `examples/1803-concurrency-naked-asm-param.sx` (`add(a,b)` → x0+x1 → 42). -- `zig build && zig build test` green: **725 ran, 0 failed** + unit tests. - -### Earlier — B1.0a (lock + review hardening) -Plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites + generic.zig + -pack.zig); `funcWantsImplicitCtx` skips `.naked` (no synthetic ctx, like `.c`); all -body-lowering paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx -return); `emit_llvm` Pass 2 bailed loudly (since flipped to real emission). Adversarial -review caught the generic/pack `is_naked` gap (a generic `.naked` silently shipped a framed -body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positive -(unparseable — `isLambda` breaks on the `abi` keyword). - -## Current state -**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime: -the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task -async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b: -`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`, -`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded -`mmap` stacks (B1.0–B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw -suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). A terminal -`deinit` (B1 follow-up) closes the previously-documented leaks: heap `Task`s (tracked via `task_allocs`), -the `timers`/`io_waiters`/`task_allocs` List backings, and the kqueue fd; the per-`spawn`/`go` closure -env remains unfreeable (language limitation). Locked by `18xx` 1800–1820 (naked-asm, context-snapshot, -blocking async, the switch + §10.7 stress gate + guarded stacks + Win64 sibling, scheduler round-robin, -suspend/wake, async go/wait/cancel, sim-timer ordering, timer early-wake eviction, kqueue pipe I/O, the -**1817 end-to-end capstone**, sleep-negative/double-wait guards, and **1820 scheduler-deinit**). Suite -GREEN **817/0**, committed. **B1.6: now also runs on aarch64-linux** (epoll fd-backend + comptime-branched -`MAP_AP` + naked-fn trampoline) — validated byte-identical to macOS in an Apple `container`. - -Future work (none blocking B1): routing the suspending async through -the erased `context.io` (forces sched.sx into every std consumer — deferred to the M:N model, where -the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/ -`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task / -closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream -B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started. - -### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists -`library/modules/std/sched.sx` drives N fibers -(generic `Closure() -> void` bodies) cooperatively over the proven `swap_context`, on guarded -`mmap` stacks: `spawn` / `yield_now` (round-robin) / `suspend_self` + `wake` (off-queue park/resume) -/ `run` (drives to drain, reaps on `.done`). Adversarially reviewed + hardened (wake guarded, loud -mmap/mprotect/OOM/deadlock bails, env-leak documented). Locked by `1811` (round-robin ordering -contract) + `1812` (suspend/wake park-resume + spurious-wake guard). Suite GREEN **748/0**. - -The remaining B1.4 work wires this scheduler under the `Io` capability: **B1.4a (FiberIo)** makes -`context.io` route `spawn_raw`/`suspend_raw`/`ready` onto the `Scheduler` so `async`/`await` truly -SUSPEND (today's `CBlockingIo` runs the worker to completion inline); **B1.4b** the deterministic-sim -`Io` (virtual clock + timer queue, calibrated against blocking — the KEYSTONE test harness); -**B1.4c** the event-loop `Io` (kqueue/epoll). Then **B1.5** is the end-to-end M:1 validation under -the deterministic `Io`. - -### Earlier — B1.2 COMPLETE -The full async surface (Io capability on Context + `async`/`await`/`cancel` + -blocking `CBlockingIo`) works end-to-end. Master GREEN (732/0), installed `sx` clean. All four -B1.2 surface bugs resolved or deferred: -- **0151 fixed** (`362674f`): generic `$T` through generic-struct / pointer / UFCS-pack params. - Regression `0214` + `0215`. -- **0152 fixed** (`e5586f6`): `Atomic(bool)` byte-promoted to `i8` in the load/store emitters. - Regression `1705`. -- **0153 fixed** (`68c1991`): `inferGenericReturnType` pins return-type resolution to the fn's - defining module, so a re-exported value-failable keeps its `!` channel. Regression `1058`. -- Issue **0150** (`void` struct field → SIGTRAP) DEFERRED — only `Future(void)` / `timeout`, - which are B1.4. - -The async examples are landed + green: `1805` (`async`/`await` + `now_ms` → `sum: 42` / -`double: 42` / `clock ok`) + `1806` (`cancel` → `await` raises `.Canceled` → `or` default). -The `18xx` concurrency category now covers naked-asm (1800-1803), context-snapshot (1804), and -the async surface (1805-1806). - -### B1.2 Io capability — what is LANDED + verified (commit 45d869d) -- `Io :: protocol #inline { spawn_raw; suspend_raw -> !; ready; poll; now_ms; arm_timer; }` - in `core.sx` next to `Allocator`, with `SpawnOpts{ pin: PinTarget }` + `ParkToken{ handle }`. - Six methods, each justified by a downstream consumer (B1.3-B1.5). -- `Context :: struct { allocator; data; io: Io; }` — `io` appended LAST so `allocator` stays - index 0 (the `call.zig:1229` hardcode) and `data` keeps index 1 (minimal VM-fallback churn). -- Both `__sx_default_context` materializers updated in lockstep + verified: `protocol.zig` - `emitDefaultContextGlobal` (extended `ctx_fields` 2→3, built the `CBlockingIo→Io` inline - 7-word vtable `{null-ctx, fn0..fn5}` via `getOrCreateThunks("Io","CBlockingIo")`) and - `comptime_vm.zig` `materializeDefaultContext` fallback (wrote the 6 thunk func-refs at - `io_base = addr + 4*ps`, offset `+ (i+1)*ps`). The global path auto-followed the 3-field - Context type. **`context.io.now_ms()` printed `clock ok` live — the capability threads + the - vtable dispatches correctly.** -- Stateless `CBlockingIo :: struct {}` + `impl Io for CBlockingIo` (mirror of `CAllocator`): - blocking semantics — `spawn_raw`/`ready`/`poll`/`arm_timer` no-op/0, `now_ms` → `time.mono_ms()`. -- **push-inherit-omitted fix** (`stmt.zig` `lowerPush`): a `push Context.{...}` now SEEDS the - new slot from the ambient context (load+store), then overwrites ONLY the literal's named - fields — so omitted fields (now incl. `io`) are INHERITED, never zero-inited to a null - vtable. Eliminates the omitted-field footgun globally (zero per-site churn across the 17 - partial-literal sites). This is the correct capability-bag semantics; it compiled clean. -- **`!`-protocol-method warning fix** (`error_analysis.zig` + a new `Lowering.impl_method_names` - set populated in `protocols.zig` `registerImplBlock`): a protocol impl method may be declared - `!` by contract (e.g. `Io.suspend_raw`) yet never raise; the "declared `!` but never errors — - drop the `!`" hint is a false positive for impl methods, now suppressed for them. - -Status of the blockers that originally stopped B1.2: -- **issue 0151 — FIXED this session** (generic `$T` through generic-struct / pointer / - UFCS-pack params). `async`/`await`/`cancel` are callable. See "Last completed step". -- **issue 0152 — NEW, the current blocker** (`Atomic(bool)` → sub-byte i1 atomic; LLVM reject). - Blocks the async examples via `Future.canceled: Atomic(bool)`. Filed; codegen-level fix. -- **issue 0150** — `void` struct field SIGTRAP; only `Future(void)`/`timeout` (B1.4). DEFERRED. - -Per the IMPASSABLE STOP rule: 0151 fix shipped (suite green 728/0), 0152 filed, STOPPED. -Resume B1.2's async examples once 0152 lands. - -### Earlier — B1.0 + B1.1 complete -Stream A (atomics) is feature-complete (✅). Stream B1: **B1.0 + B1.1 complete.** The two -compiler-floor preconditions for the fiber runtime are in place: (1) `abi(.naked)` emits a -real LLVM `naked` function end-to-end (decl, generic, pack paths) — the context-switch -substrate; (2) per-fiber `context` root needs **no compiler change** — the spawn convention -(snapshot `context`, store, `push` it from the trampoline) is pure library sx. No -fibers/Io/scheduler code yet. Grounded floor facts: -- `context` is an implicit slot-0 `*Context` param + `push Context` is a stack `alloca` ⇒ - **fiber-local for free** (confirmed by the B1.1 probe — never TLS, never re-read from the - `__sx_default_context` global mid-stack). A spawn passes the snapshot as the fiber-entry - fn's slot-0 ctx via `push f.root { entry(args) }`. Locked by `1804-...-context-snapshot`. -- Inline asm works end-to-end (lower→emit→JIT, aarch64 + x86_64) — the `.naked` body reuses it. -- **`.naked` with PARAMS works** (B1.0c, the B1.3 substrate): the param-alloca loop is gated - on `fd.abi != .naked` in decl.zig (both paths) + generic.zig — a naked fn's args stay in - ABI registers (read by the asm body), declared-but-unused in LLVM (verifier-legal). - Example `1803-concurrency-naked-asm-param.sx` (`add(a,b)` reads x0/x1). **Unsupported (loud, - not silent):** a `.naked` *variadic-pack* fn (pack.zig's param loop is intertwined with - comptime-param/`#insert` handling, and a naked fn can't read a runtime-sized pack from - registers anyway) → loud LLVM-verifier error for that nonsensical construct. Acceptable - boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker. - -## Next step -**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature- -complete and committed (1800–1820 green, 759/0), now WITH a `Scheduler.deinit` closing the bounded -leaks. Pick up **Stream B2** (channels / structured cancel / async stdlib) as a fresh carve -(PLAN-CHANNELS.md), OR one of the remaining non-blocking follow-ups: the linux `epoll` twin of -`block_on_fd`, `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the -erased `context.io` for the M:N model. (`Scheduler.deinit` — DONE, see Last completed step.) None of -these block B1. The closure-env leak survives `deinit` (no language affordance to free a closure env); -revisit if/when sx grows closure-env ownership. - -**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS -kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via -`std/net/epoll` and the OS-neutral facade is `std.event` — keep the two as separate run modes inside -`run`, branching on the platform, exactly as the timer-vs-fd modes are kept separate now. Documented -non-unification: virtual-time timers and real kqueue timeouts are NOT merged — `run` fires a pending -timer before ever blocking on kqueue (a program uses `sleep` OR fds); a true "fd-or-real-timeout" wants -a kqueue `EVFILT_TIMER`, future work. - -> **▶ LINUX EPOLL — in progress (2026-06-26), via `std.event.Loop` (the OS-neutral facade).** -> Chosen over the sched.sx `block_on_fd` twin because the facade is the named home for epoll, is pure -> sx + libc (zero compiler change), is consumed by http.sx, and has a runnable darwin sibling. Landed: -> (A) **`library/modules/std/net/epoll.sx`** — raw bindings, the linux twin of `std/net/kqueue.sx`. -> `epoll_event` is modelled as an **arch-branched struct** (`{events, data_lo, data_hi}` u32 fields → -> 12 B x86_64 packed / 16 B aarch64), so layout is byte-exact with NO packed attribute, NO unaligned -> access, NO scalar-pointer indexing (issue 0155) — the struct-per-arch approach the user flagged as -> better than raw byte poking. Self-contained (libc only — NO build.sx import; the top-level `inline if -> ARCH` resolves via the compiler's flatten pre-pass, keeping the IR small). Locked by -> `examples/event/1633-event-epoll-bindings-linux.sx` (ir-only x86_64-linux, durable 244-line .ir; -> aarch64 16 B layout also probe-verified). (B) **`std.event.Loop` branched on `inline if OS`** into two -> top-level OS-selected structs (sx has no conditional struct fields): the kqueue Loop unchanged -> (darwin, runs — 1632 green), a new epoll Loop (linux) with the per-fd registration table (combined -> EPOLLIN/OUT mask via ADD/MOD/DEL), eventfd wake channel, and EPOLLRDHUP→eof. **RUNTIME-VALIDATED on -> real Linux:** a static `aarch64-linux` build of the 1632-equivalent Loop test (+ the eventfd wake path) -> ran **6/6 green inside an Apple `container` Linux VM** (kernel 6.18 aarch64) — add_read, idle-timeout, -> readable+fd+udata, the MOD-mask add_write path, the eventfd wake channel, and EPOLLRDHUP/HUP eof all -> behave identically to kqueue (lone documented difference: `nbytes` is 0 on epoll). Also lowers clean for -> both linux arches; the ABI is corpus-locked by 1633. NOT corpus-snapshotted (the corpus runner is -> host-based, not container-aware; a Loop example drags the std barrel → ~18k-line brittle IR). -> **The epoll deliverable is COMPLETE.** Re-validation recipe in the event.sx VALIDATION note. Optional -> follow-on: route sched.sx `block_on_fd` through `std.event` (still needs the linux sched.sx port — mmap -> consts, tramp symbol, errno, x86_64 SysV switch). - -> **✅ issue 0192 FIXED (2026-06-26) — epoll work UNBLOCKED.** A qualified-import-member const -> (`m.EV_SIZE`) now folds as a compile-time constant in every position the bare/flat form does -> (array dim, arithmetic, Vector lane, generic value-param, inline-for) — so the clean -> `[MAXEV * ep.EV_SIZE]u8` event buffer the bindings want will work. Fix: a `lookupQualifiedConst` -> ctx hook resolving the namespace alias → target module's per-source const, wired into the int/float -> const folders (`src/ir/program_index.zig` + `src/ir/lower/comptime.zig`). Regression: -> `examples/modules/0842-modules-qualified-import-const-comptime.sx`. The hint stands for the rebuild: -> **a struct-per-arch `EpollEvent` (arch-branched u32 fields, 12 B x86_64 / 16 B aarch64) beat raw -> byte access** — idiomatic field reads, no issue-0155 scalar-pointer indexing, no unaligned u64. -> Resume: rebuild `std/net/epoll.sx`, branch `std.event.Loop` on `inline if OS`, lock with a darwin run -> + ir-only linux example. - -> **⛔ (HISTORICAL) BLOCKED on issue 0192 (filed 2026-06-26).** Started the epoll work: chose the `std.event.Loop` -> backend (pure sx + libc externs, zero compiler change — per "do this in sx as much as possible") as -> the first deliverable, since event.sx already names epoll as its linux backend and it's runnable -> (darwin via kqueue) + ir-only-verifiable (linux). De-risked four landmines by probe — arch-dependent -> layout const via module-scope `inline if ARCH` (folds + validates in linux IR), slice-based byte access -> (sidesteps issue 0155), no unaligned u64 (store the 32-bit fd in epoll `data`), and comptime-dead linux -> externs don't break the darwin corpus (just an unreferenced `declare`). Then hit a compiler bug while -> sizing the event buffer: a **qualified-import-member const is not a compile-time constant** — -> `[m.CAP]u8` / `A :: m.CAP` fail (a *flat*-imported const works). Root cause located: -> `evalConstIntExpr` (`src/ir/program_index.zig:325`) has no namespace-member-const arm. Per the STOP -> rule the half-built `std/net/epoll.sx` (which used a struct-based layout to route around the bug) was -> **removed**, not landed — the unblock session rebuilds it cleanly with the fix in hand. Repro + -> investigation prompt: `issues/0192-qualified-import-const-not-comptime.{md,sx}`. - -Design note carried forward: an event-loop `Io` needs a current-`Scheduler` handle. `sched.*` methods -thread it via `self`/the `Task`; if B1.4c wants the capability-threaded `context.io` form it'll need -an ambient current-scheduler accessor in sched.sx (still deferred — the `sched.*`-method form -suffices). The `Io` protocol's `poll`/`arm_timer` map onto this when/if that wiring is built. - -**Side thread (optional, low priority): the SysV/Linux x86_64 sibling.** A THIRD switch variant -for `x86_64-linux`: SysV callee-saved = rbx, rbp, r12-r15 + rsp (6 GP + sp; **no** callee-saved -XMM, unlike Win64) — a 7-slot ctx, args rdi/rsi/rdx, the rsp-carried return addr. Needs a Linux -x86_64 host (or a working cross-run) to RUN + the mutual-scribble gate. Not blocking — the switch -is already validated on two arch/ABI pairs. - -**Deferred (do NOT block on these):** issue **0150** (`void` struct field SIGTRAP) — only -`Future(void)`/`timeout` (B1.4). The **`::` callable-parameter feature** (named-fn async workers -`async(read_a, conn)`) — WIP at `.sx-tmp/wip-callable-params/patch.diff` (parser done, inference -incomplete); a dedicated effort; lambda workers are the idiom meanwhile. - -`Context` layout settled: `{ allocator; data; io; }` (allocator index 0 fixed by -`call.zig:1229`, io last). Io protocol + materializers + push-inherit are LANDED + reviewed. - -## Known issues / capability gaps -- **issue 0157 (OPEN, BLOCKING B1.4a)** — a user-defined generic ufcs method whose NAME collides - with a stdlib re-export (`cancel`, re-exported by `std.sx` from `io.sx` as `ufcs (f: *Future($R))`), - called via UFCS on a different generic struct (`*Task($R)`), leaves `$R` unresolved → `.unresolved` - reaches LLVM emission → panic (`src/backend/llvm/types.zig:196`). Renaming → works; the non-UFCS - call form already diagnoses `cannot infer generic type parameter 'R'`, so the UFCS path skips that - diagnostic. Surfaced by `cancel :: ufcs (t: *Task($R))` in `std/sched.sx`. Minimal repro (no - fibers/closures): `issues/0157-ufcs-generic-method-name-collides-stdlib-unresolved.{md,sx}`. -- **✅ issue 0154 — FIXED** (`null`/`---` to a struct field over-stored a whole-struct null when - the function's return type leaked as `target_type`, corrupting the frame → `ret` to 0x0; - surfaced building `Scheduler.init()`'s by-value return). Fix: `.null_literal`/`.undef_literal` - added to `needs_target` in `lowerAssignment` (`src/ir/lower/stmt.zig`). Regression: - `examples/types/0193`. -- **issue 0155 (OPEN, NON-blocking)** — indexing a scalar pointer (`pc[0]`, `pc: *i64`) panics - codegen (`.unresolved` reaching LLVM emission, `src/backend/llvm/types.zig:196`). Found in the - B1.5a review; the scheduler doesn't use it (array-field index + `.*` only). Filed for its own - session: `issues/0155-scalar-pointer-index-llvm-panic.{md,sx}`. -- **✅ issue 0158 — FIXED** — a plain `union` struct-literal (`b : Overlay = .{ f = 3.14 }`) fell - through the generic struct-literal path (`getStructFields` empty for a union → malformed - `structInit`, overlapping zero-fill clobbered the member → silent `0.0`). Fix: `lowerStructLiteral` - detects a plain-union target → new `lowerUnionLiteral` (`src/ir/lower/stmt.zig`) writes each named - member into a union-sized slot via the assignment-path lvalue resolver, then loads it back. - Single-arm only (one direct member, or same-arm promoted members); overlapping/different-arm/ - positional literals are diagnosed. specs.md updated. Regressions: `examples/types/0194` + - `examples/diagnostics/1191`. -- **✅ issue 0157 — FIXED** (B1.4a) — a user generic `ufcs` method whose name collides with a - stdlib re-export resolved via last-wins `fn_ast_map` with no receiver filtering → wrong overload → - `$R` unbound → LLVM panic. Fix: `selectUfcsGenericByReceiver` (`src/ir/lower/call.zig`) — most - receiver-specific binding author across ALL module authors, deterministic, ambiguity-diagnosing. - Regression: `examples/generics/0217`. -- **✅ issue 0156 Part 1 — FIXED** (B1.4a) — single-type generic `$R` as a type-arg in a pack-fn - body (`Box($R)`/`size_of(Box($R))`) → `.unresolved` → panic. Fix: `comptime_pack_ref` arm in - `resolveTypeWithBindings`. Regression: `examples/generics/0216`. - - **Part 2 (OPEN, NON-blocking)** — a deferred `..` spread (a comptime pack captured into a - closure, or a tuple `..t` spread) crashes instead of working/diagnosing. The fiber async layer - avoids it by design (nullary thunks), so it's filed for its own session: `issues/0156`. -- **Heap leaks in the fiber runtime (documented limitations, NOT bugs):** `spawn`'s closure env + - `go`'s heap `Task` are never freed (sx exposes no closure-env free; Task ownership is deferred). - Bounded by spawn/go count, invisible under the default GPA. Revisit for a long-running - arena-backed scheduler. -- **✅ issue 0153 — FIXED** (re-exported generic value-failable `($R, !E)` kept its `!` channel: - `inferGenericReturnType` now pins return-type resolution to the fn's defining module). - Regression: `examples/1058`. Was the LAST B1.2 surface blocker. -- **✅ issue 0152 — FIXED** (`Atomic(bool)` sub-byte i1 atomic → byte-promoted to i8 in the - load/store emitters). Regression: `examples/1705`. Unblocked `Future.canceled`. -- **✅ issue 0151 — FIXED** (generic `$T` through generic-struct / pointer / UFCS-pack params). - Regression: `examples/0214` + `0215`. Was the original B1.2 surface blocker. -- **issue 0150** (deferred) — a `void` struct field crashes the compiler (unsized-type SIGTRAP - in LLVM `getTypeSizeInBits`). Blocks `Future(void)` → `timeout` (B1.4). Repro: `issues/0150-...`. - - (Note: **issue 0149**, filed by another session against an earlier dirty binary, was a - manifestation of the pre-fix 0151 — now moot.) -- **Orthogonal (not a B1 blocker):** default VALUES for comptime params don't bind on - generic-struct methods (free-fn defaults DO work) — inherited from Stream A. Only matters - if a B2 lib type wants a defaulted comptime param; atomics/fibers require explicit, so - unaffected. -- **Issue 0144 (open, independent):** calling an unrecognized bodiless `#builtin` silently - returns 0 / exit 0 — a silent-fallback footgun in the generic builtin-call path. Filed; - leave for its own fix session unless prioritized. Not a B1 blocker. -- **Deferred design gap (documented):** the B1.4 event-loop `Io` does not yet cooperate with - a platform UI run loop (CFRunLoop/NSRunLoop/ALooper); pinning gives thread-affinity, not - run-loop integration — a §6 app-target concern, out of B1 scope. - -## Decisions (Stream B1 specifics; surface locked in design §4 / §4.6) -- **The async runtime is sx LIBRARY code.** The compiler provides only: the general - primitives (inline asm ✅, `abi(.naked)` naked [B1.0], atomics ✅) + fiber-safe codegen - (`context` already fiber-local — B1.1). Schedulers, fibers, channels, futures, `Io` - vtables, `mmap` stacks are all sx. -- **`abi(.naked)` is the real spelling of the design's `callconv(.naked)`** — postfix slot, - `name :: (sig) -> Ret abi(.naked) { asm { … }; }`. B1.0 = carry it into IR + emit LLVM - `naked` + skip prologue/ctx (mirror the existing `.c` skip), NOT extend the enum (it's - already there, just inert). -- **`.naked` ≠ `.c`:** a `.c` epilogue would restore SP from the wrong stack across a context - switch (SP-in ≠ SP-out by design). `.naked` = no prologue/epilogue/frame; the asm emits its - own `ret`. This is why the switch must be `.naked`. -- **Naming:** sx-facing name is **`naked`** (keyword `abi(.naked)`, field `is_naked`, the - diagnostic), matching LLVM's `naked` attribute and the industry term (Zig/Rust/GCC/Clang). - The ABI variant was renamed `.pure → .naked` (user direction): "pure" universally means - *side-effect-free*, the opposite of a register-clobbering context switch. -- **B1.0 snapshot scope:** a `.naked` body is raw per-arch asm; LLVM's `naked` attr text is - arch-invariant. **B1.0a** = one host example locked to the emit bail (host-independent — - fires before instruction selection; no `.build` pin). **B1.0b** = pin aarch64 + add an - x86_64 cross sibling (`.build` target-gated, ir-only on mismatch), like the asm corpus - split. The `.ir` proves the `naked` attr + asm emitted, NOT register-save correctness - (that's B1.3's stress harness). -- **B1.1 — per-fiber context is library-only (CONFIRMED by probe):** push frames are - stack-`alloca`'d and the implicit ctx rides slot 0, so the spawn convention — snapshot - `context`, store it, `push f.root { entry(args) }` from the trampoline — installs the - fiber's root with no compiler change. Verified: the body reads the snapshot over a different - ambient context, and `push` restores ambient on exit (`1804-...-context-snapshot`). The - design doc's "never raw TLS" guarded a non-problem (context was never TLS). -- **Test keystones (design §10):** the **B1.3 switch-stress harness** gates the - context-switch (the one piece the deterministic `Io` can't test — §8.1.1, §10.7); the - **B1.4 deterministic-sim `Io`** (calibrated against blocking `Io` — §8.1.3) gates all - scheduling tests. Both must exist + be calibrated before the async tests they gate are - trusted. `18xx` asserts program-emitted ordering contracts, not raw interleaving. - -## Log -- **B1.6 — aarch64-linux port of sched.sx.** Comptime-branched the per-OS bits: `MAP_AP` (linux - `0x22` / macOS `0x1002`), the fd-readiness backend (epoll on linux, kqueue on darwin — epoll import - scoped to the linux branch; `block_on_fd` / run-loop Mode-2 / `cancel_io_waiter_for` each branch, - epoll `EPOLL_CTL_DEL`s on fire + early-wake), and the first-entry trampoline (per-OS global-asm - symbol → naked sx fn `fib_tramp` + register-indirect `br x20` to `&fib_dispatch` preset in - `regs[1]`). **Fixed issue 0193 Bug A:** the tramp redesign bus-errored on 1817 (both OSes) until - `fib_dispatch` was annotated `abi(.c)` (C-ABI pin; the original pinned it via `export`, which the - redesign dropped) — without the pin the fn uses sx's internal ABI (x0 = implicit - `context`, `self` → x1) while the trampoline supplies `self` in x0, so the closure loads - `regs[1] == &fib_dispatch` as its first capture and recurses forever → stack-overflow bus error. - Root cause found via lldb (AOT macOS build) + an adversarial source review. **Bug B** (wrapped - top-level `asm` dropped) carved to **issue 0194** (OPEN; no live trigger — the naked-fn tramp - sidesteps it). Validated byte-identical on aarch64-macOS host AND aarch64-linux Apple `container` - for 1811/1814/1816/1817; full suite GREEN **817/0**. -- **B1 follow-up — `Scheduler.deinit`.** Closes the bounded leaks B1 documented. Added a `task_allocs: - List(*void)` field (appended in `go` so the scheduler can reach its generic `Task($R)`s) + a canonical - `close` extern, then a terminal idempotent `deinit`: reap leftover ready fibers (`munmap` + free) → - free tracked Tasks → `List.deinit` the 3 backings → `close` the lazy kqueue fd (reset `-1`). Closure - envs stay unfreeable (documented). Probe-observed the accounting under a tracking GPA (deinit drives - live allocs 7→3 in a spawn+sleep+2×go run; residual = envs). Locked by - `1820-concurrency-fiber-scheduler-deinit.sx` (one run hits timers + kqueue fd + Tasks; `freed by - deinit: 5`, `live after deinit: 5` (env residual), `kq open after run: true`→`kq after deinit: -1`, - `read: 3 [97 98 99]`), `.build {"target":"macos"}`. Adversarial review: no real UAF/over-free in the - supported deinit-after-`run` path; reconciled a doc contradiction (terminal-contract wording); 0154 - over-store concern probed + cleared (`kq == -1` right after `init`). Suite GREEN **759/0**. -- **B1.4c — real fd-readiness blocking via kqueue (macOS).** De-risked first with a no-scheduler probe - (confirmed `size_of(Kevent)==32` and the pipe→kevent roundtrip: `kq_wait` returned 1, `out.ident == - read_fd`, `out.filter == -1`, `out.data == 1` — the struct layout reads the fd back correctly). Then - added to `library/modules/std/sched.sx` (importing the existing verified `std/net/kqueue.sx` as `kqb` - rather than re-deriving the FFI): a lazy `kq: i32` (-1 until first use), `io_waiters: List(IoWaiter)`, - `block_on_fd(fd, want_read)` (arm one-shot `EVFILT_READ`, record waiter, `suspend_self`), a run-loop - Mode 2 (block on `kq_wait(kq, evbuf, MAXEV=16, -1)` when only fd waiters remain, wake the fiber whose - fd fired), and `wake` now also evicts a stale fd-waiter (`cancel_io_waiter_for`, the same UAF guard as - `cancel_timer_for`). Timers keep precedence over fds (documented non-unification). Orphan-deadlock - check still fires for a genuine no-timer/no-fd suspend (probed: exit 134). Locked by - `1816-concurrency-fiber-io-pipe.sx` (reader blocks on empty pipe → writer writes `a b c` → kqueue - wakes reader → reads 3 bytes; `log: wrote read 3 [97 98 99]`, `n_suspended: 0`), `.build` - `{ "target": "macos" }`, runs end-to-end on host. The example's `read`/`write`/`close` externs use the - canonical signatures std already binds (extern-dedupe rejects a divergent re-binding). Suite GREEN - **754/0**. Next: B1.5 (end-to-end M:1 validation); linux epoll twin deferred. -- **carve** — wrote PLAN-FIBERS.md + CHECKPOINT-FIBERS.md. Grounded the B1 compiler floor: - `ABI.naked` inert (type_resolver.zig:237), IR `Function` has no naked flag (inst.zig:605), - attribute API pattern (emit_llvm.zig:1339 nounwind), `.c` ctx-skip precedent - (decl.zig:515), `push Context` stack-alloca + slot-0 implicit ctx (stmt.zig:1263, - lower.zig:259), `__sx_default_context` root (decl.zig:2667/2815), inline-asm corpus - (1645/1651). Corrected the design's `callconv(.naked)` → real `abi(.naked)` spelling and - the B1.0 snapshot story. B1.1 grounded as likely library-only. Baseline green (721/0). -- **B1.0a** — plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites); - `funcWantsImplicitCtx` skips `.naked` (no implicit ctx, like `.c`); both body-lowering - paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx return); - `emit_llvm` Pass 2 bails loudly on `func.is_naked`. `examples/1800-concurrency-naked-asm.sx` - locked to the bail (exit 1 + diagnostic). Suite green (722/0). (ABI variant later renamed - `.pure → .naked` — see the Naming decision above — so all `is_*`/`abi(.*)`/example names - here read `naked`.) -- **B1.0a review-hardening** — adversarial review found generic/pack Function-creation paths - left `is_naked` false (silent framed body for a generic `.naked` instance — returned 42 but - corrupted the stack). Fixed generic.zig + pack.zig (set `is_naked` + asm-only `unreachable` - cap); locked by `examples/1801-concurrency-naked-generic-bail.sx`. The review's `.naked`- - lambda CRITICAL was a false positive (unparseable — `isLambda` breaks on `abi`). Suite - green (723/0). -- **B1.0b** — real `naked` emission: emit_llvm declaration pass adds LLVM `naked`/`noinline`/ - `nounwind` + skips `frame-pointer` for `func.is_naked`; Pass 2 emits the body verbatim (no - prologue). `1800` green aarch64-pinned (exit 42 + `.ir`); renamed `1801` → `-generic` - (generic `.naked` emits a naked body, exit 42); added x86_64 sibling `1802` (ir-only, `.ir` - locks `naked` + `movl $42, %eax`). Unit test asserts `naked` present + `frame-pointer` - absent. Suite green (724/0). -- **B1.0c** — review-hardening: param-bearing `.naked` emitted invalid LLVM (loud verifier - error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both paths + generic.zig) - — naked args stay in registers, read by the asm body (the B1.3 context-switch shape). - Locked by `examples/1803-concurrency-naked-asm-param.sx`. Pack `.naked` left unsupported - (loud, nonsensical). **B1.0 complete.** Suite green (725/0). -- **rename** — ABI variant `.pure → .naked` (keyword, `Function.is_naked`, diagnostics, - examples 1800-1803 `*-pure-* → *-naked-*`, docs). "pure" universally means side-effect-free - — wrong for a register-clobbering switch; "naked" matches LLVM/Zig/Rust/GCC/Clang. Pure - cosmetics, no semantic change. Suite green (725/0). -- **B1.1** — per-fiber `context` root: **zero compiler change** (probe-confirmed). The spawn - convention (snapshot `context` → store in a struct → `push f.root { entry() }` from the - trampoline) installs the fiber's root via the implicit slot-0 `*Context` param; the body - reads the snapshot, not the trampoline's ambient ctx, and the `push` scope restores ambient - on exit. Locked by `examples/1804-concurrency-context-snapshot.sx` (prints `fiber root: 42` - / `ambient after: 99`). Suite green (726/0). **Next: B1.2 (Io interface + context.io).** -- **B1.2 (BLOCKED)** — built the full `Io` capability (protocol on `Context`, stateless - `CBlockingIo` blocking default, both `__sx_default_context` materializers, push-inherit-omitted - fix, `!`-impl-method warning fix) and VERIFIED the core works live (`context.io.now_ms()` → - `clock ok`). Two independent compiler bugs blocked the `async`/`await`/`timeout` layer: - **0150** (`void` struct field → unsized SIGTRAP, blocks `Future(void)`) and **0151** (type-var - from a fn-ptr param's return type not bound in the body, blocks `async`'s `Future(R)`). Both - filed with standalone repros + investigation prompts. Per the STOP rule: reverted ALL B1.2 - working changes (master green again, 726/0; the dirty binary had broken the photo project — - see the now-moot 0149), saved WIP to `.sx-tmp/b12-wip/`, STOPPED. Resume after 0150 + 0151. -- **0151 FIXED** — generic inference now binds `$T` through a generic-struct param head, a - pointer (`*Box($T)`, incl. UFCS auto-ref), and a closure-return-via-pack on the UFCS path. - Four gaps closed: `parameterized_type_expr` arm in `extractTypeParam`/`matchTypeParam(Static)` - (recovers the arg instance's recorded per-param bindings, recurses positionally); pointer arm - falls through to match a value arg (auto-address-of); `ExprTyper.inferType` `.lambda` arm - (closure type from annotations — UFCS types args from raw AST pre-lowering); pack UFCS target - routes through `lowerPackFnCall` with the receiver spliced in as `args[0]`. Issue 0151 marked - RESOLVED; repro → `examples/0214-generics-ufcs-closure-return-pack.sx`; widened cases → - `examples/0215-generics-infer-through-pointer.sx`. Suite green 728/0. The now-callable async - surface immediately exposed a SEPARATE codegen bug — **issue 0152** (`Atomic(bool)` → sub-byte - i1 atomic, LLVM reject; `Future.canceled` hits it). Filed with standalone repro + fix prompt. - Per the STOP rule: shipped the 0151 fix, filed 0152, STOPPED. Resume the async examples - (1805/1806) after 0152. -- **0152 FIXED** — the atomic load/store emitters (`src/backend/llvm/ops.zig`) byte-promote a - sub-byte (`bool`→`i1`) access to its `i8` storage type and `trunc`/`zext` the value at the - boundary (new `atomicByteType` helper). rmw/cmpxchg left as-is (a `bool` rmw/CAS is rejected - at the sx level — integer-only — so a sub-byte element never reaches them; comments record - this). Regression `examples/1705-atomics-bool-byte-promoted.sx` (load/store round-trip). Issue - 0152 marked RESOLVED. Suite green 729/0. With `Atomic(bool)` working, the async surface - exposed the TRUE remaining blocker — **issue 0153**: a re-exported generic value-failable - `($R, !E)` loses its `!` channel at the call site (the earlier "secondary `or` PHI" symptom - was this, NOT an `Atomic` cascade — confirmed it persists after 0152). Narrowed to the - generic+re-export co-requirement (non-generic re-export OK; direct generic import OK; only the - combination drops `!`). Root cause: the monomorphized return-type's error-set, reached via the - re-export alias, resolves to a non-`.error_set` TypeId, so `errorChannelOf` - (`lower/error.zig:148`) misses the channel. Filed `issues/0153-...` with a minimal co-located - 2-file repro + a single-file stdlib-`await` repro + investigation prompt. Per the STOP rule: - shipped the 0152 fix, filed 0153, STOPPED. Resume the async examples after 0153. -- **0153 FIXED → B1.2 COMPLETE** — `inferGenericReturnType` (`src/ir/generics.zig`) resolved the - return-type AST in the CALL-SITE module, so a re-exported error set (`LE :: lib.LE`) resolved - to a non-`.error_set` alias and the planned call-result was a plain tuple (channel lost). Fix: - pin the source to `fd.body.source_file` around the return-type resolution, exactly as - `monomorphizeFunction` does — the `!E` now resolves to the real `.error_set`. One-function - change; full suite green (732/0), no regression. Issue 0153 RESOLVED; repro → - `examples/1058-errors-reexport-value-failable-channel.sx` (+ companion `lib.sx`). With the - channel preserved, landed the async examples: **`1805`** (`async`/`await` + `now_ms` → `sum: - 42` / `double: 42` / `clock ok`) + **`1806`** (`cancel` → `await` raises `.Canceled` → `or` - default; `ok: 7` / `canceled: -99`). **B1.2 (Io capability + M:1 async surface) is COMPLETE.** - Next: B1.3 (fiber runtime) on the `.naked` context-switch substrate. -- **B1.3a-1 — context switch works.** Implemented the stackful switch in pure sx over - `abi(.naked)`: `swap_context(from, to)` (save callee-saved x19-x28 + fp/lr + sp into `*from`, - load from `*to`, `ret` onto `to`'s stack) + by-hand fiber bootstrap (SP = top of an - `alloc_bytes` stack, LR = a `.global _fib_tramp` global-asm trampoline that does `mov x0, x19; - bl _fib_body`, x19 = `*Fiber`). Proven via a probe (main↔fiber), then locked by - `examples/1807-concurrency-fiber-context-switch.sx` (aarch64-pinned): a 2-fiber ping-pong - (`rounds: 6`, `canary fails: 0` — a per-fiber stack canary survives every switch) + a 64-frame - deep recursive chain suspended at the bottom and resumed (`frames verified: 64` / `depth fails: - 0`). The `bl _fib_body` reaches the sx body via `export "fib_body"` (the 1655 asm→sx pattern); - runs under JIT, ir-only on a non-arm host (`.ir` captured — `swap_context` shows `naked noinline - nounwind`). Suite green 733/0. **Honest scope:** indirect register/stack survival only; the - EXPLICIT every-callee-saved + FP scribble (§10.7) is B1.3a-2, still owed. Next: B1.3a-2. -- **B1.3a-2 — the §10.7 stress gate, adversarially reviewed.** Extended `swap_context` to the - COMPLETE AAPCS64 callee-saved set (added FP d8-d15 → 21-slot ctx) and wrote a naked - `scribble_verify` that loads a unique sentinel into all 18 callee-saved regs, yields, and counts - non-survivors on resume (176-byte frame saves/restores the caller's callee-saved + base; lr - round-trips the swap). The gate is a 2-fiber MUTUAL scribble (each clobbers the other's regs, so - survival ⇒ the switch saved+restored them). Locked by - `examples/1808-concurrency-fiber-switch-stress.sx` (`A/B mismatches: 0`). Validity proven by - negative controls (drop d8-d15 → 8/8; drop x27/x28 → 2/2). **Spawned an adversarial-review - worker (per the plan + user request): NO critical bugs** — callee-saved set complete (x18 rightly - excluded; d8-d15 suffices per §6.1.2), offsets/alignment/lr-sp dance all verified. Applied its - one rec: `boot` zeroes FP ctx slots so first-entry loads 0, not garbage. Honest residual gaps - (spec-correct for a call-boundary swap; in the example header): FPCR/FPSR/NZCV + TPIDR/TLS not - swapped, fp=0 blocks unwind — relevant at N×M:1 / signals, not here. Suite green 734/0. - Next: B1.3b (x86_64 sibling + mmap guard-page stacks). -- **B1.3b — mmap guard-page stacks (x86_64 sibling deferred).** Fiber stacks now `mmap` a - `[guard | usable]` region and `mprotect` the low 16KB page `PROT_NONE`, so a stack overflow - faults at the guard boundary instead of silently corrupting a neighbor (§8.1.1). Locked by - `examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1` - (`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields). - Guard FIRING validated manually (overflow → `Bus error` at `region+GUARD`, exit 134 via the sx - crash handler) — not corpus-pinned because a deliberate-overflow crash is host-fragile (and a - mere "child faulted" fork test wouldn't prove the BOUNDARY catch). The x86_64 `swap_context` - sibling was DEFERRED: `--target x86_64-macos` mislinks on this arm64 host and `x86_64-linux` - can't run here, so it could only ship un-run/un-negative-controlled — which §10.7 forbids for the - highest-risk asm. SysV target notes (rbx/rbp/r12-r15/rsp, no callee-saved XMM, rsp-carried return - addr) recorded in Next step. Suite green **735/0**. Next: x86_64 sibling (needs an x86_64 host) - OR B1.4 (`Io` impls / scheduler) on the proven aarch64 substrate. -- **B1.3b-1 — x86_64 / Win64 switch sibling VALIDATED on real hardware.** The user provided a - Windows 7 x64 VM (UTM), so the x86_64 switch became RUNNABLE (as Win64). Validated the - cross-build→VM→run loop (`--target x86_64-windows-gnu --self-contained` → PE32+; output via the - Win32 `WriteFile` boundary, the 1660 pattern). Wrote a Win64 `swap_context` (8 GP rbx/rbp/rdi/ - rsi/r12-r15 + rsp + **xmm6-xmm15** via `movups` — Win64 has callee-saved XMM) + a Win64 - `scribble_verify` (264-byte frame, 32-byte shadow + 16-align at each `call`, COFF symbols, - rsp-carried return addr) driving the 2-fiber mutual scribble. **Adversarially reviewed (worker - emitted the real `.s`, verified every alignment/offset/round-trip against the Win64 ABI — no - critical/minor bugs), THEN run on the VM → `0 0 P`** (all 8 GP + 10 XMM callee-saved survived). - Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`, - ir-only on this host; the VM run is the runtime-correctness provenance). Good-swap-only (the - in-process negative control was dropped to avoid an sx fn-ptr-convention rabbit hole; the - detection of this exact logic was negative-controlled on aarch64 in 1808). Suite green **736/0**. - The B1.3 context switch is now proven on TWO arch/ABI pairs. Next: **B1.4** (Io impls / M:1 - scheduler) on the proven substrate. (Side thread: the SysV/Linux x86_64 sibling, when a Linux - x86_64 host is available.) -- **B1.5a — M:1 scheduler CORE + a fixed blocker bug.** Built `library/modules/std/sched.sx`: a - generic `Fiber`/`Scheduler` over `swap_context` on guarded `mmap` stacks. `spawn` heap-allocs a - fiber, bootstraps its ctx, enqueues it; the ONE generic dispatch (`fib_dispatch` via `_fib_tramp`) - runs ANY stored `Closure() -> void` on a fresh stack (replacing the fixed `bl _fib_body`); - `yield_now` round-robins, `suspend_self`/`wake` park/resume off-queue, `run` drives to drain + - reaps `.done` fibers (`munmap` + free). **De-risked first by probe** (closure-on-fiber + output - via captured pointer). **Hit blocker bug 0154** (user-authorized fix): `null`/`---` to a struct - field over-stored a whole-struct null when the fn return type leaked as `target_type`, corrupting - the frame (`ret` 0x0) — exactly the `Scheduler.init()` by-value-return shape. Fixed in `stmt.zig` - (`needs_target` += `null`/`undef` literals); regression `examples/types/0193`; `0154` RESOLVED. - **Adversarial review:** asm/bootstrap/lifetime sound (env-lifetime fear disproven — heap-promoted); - 1 CRITICAL (`wake` re-enqueue → FIFO segfault) + robustness gaps ALL hardened (wake guarded on - `.suspended`, `n_suspended` deadlock diagnostic+abort, loud mmap/mprotect/OOM bails, env-leak - documented). Locked `1811` (round-robin `0 1 2 ×3`) + `1812` (suspend/wake + spurious-wake guard, - `log: 10 20 21 11`). Filed NON-blocking `0155` (scalar-pointer index panics codegen — review - incidental, unused by sched). Suite GREEN **748/0**. Next: **B1.4a** (FiberIo). -- **B1.4a (truly-suspending fiber-task async, nullary-thunk design) — BLOCKED on issue 0157.** - Implemented the async layer SELF-CONTAINED in `library/modules/std/sched.sx` (kept its lone - `#import "modules/std.sx"` to avoid the duplicate-`_fib_tramp` trap): `TaskState`, a LOCAL - `TaskErr :: error { Canceled }` (the re-exported `IoErr` alias is NOT seen through by the - `raise`/failable-type check — verified), `Task($R)`, and `go`/`wait`/`cancel` ufcs. Design is - the validated nullary-thunk (`.sx-tmp/pnullary.sx` → `log: 1 2 3 42 100`): `work` is a - `Closure() -> $R`, user captures inputs at the call site, NO `..args` crosses the fiber boundary - (deliberately sidesteps 0156). `go`+`wait` run correctly; both wake-orderings traced. Wrote the - example `examples/concurrency/1813-concurrency-fiber-async-suspend.sx` (+ `{ "target": "macos" }` - `.build`) but its `cancel` ufcs surfaced a NEW compiler bug — issue **0157**: a user generic - ufcs whose name collides with a stdlib re-export (`cancel` from io.sx) is mis-resolved on UFCS - call over a different generic struct, leaving `$R` unresolved → LLVM panic. Bisected to a minimal - no-fiber repro (name is the sole trigger; non-UFCS form diagnoses correctly). Example NOT seeded - into the corpus (no `.exit` marker) — do NOT regen its goldens until 0157 lands. Per the STOP - rule: filed `issues/0157-*.{md,sx}`, marked state BLOCKED, paused. -- **B1.4a COMPLETE (this session) — suspending fiber-task async + two compiler fixes.** Built the - `Task($R)` + `go`/`wait`/`cancel` layer in `sched.sx` (nullary-thunk design; self-contained to - avoid the `_fib_tramp` duplicate-symbol trap). Locked `1813` (`sequence: 1 2 3 42 100 -99`). - FIXED the two blockers the worker had filed: **0156 Part 1** (`comptime_pack_ref` arm in - `resolveTypeWithBindings`; regression `0216`) and **0157** (receiver-driven UFCS overload - selection `selectUfcsGenericByReceiver`; regression `0217`). Adversarial review of the 0157 fix + - Task layer found a determinism CRITICAL (always-run selection + specificity + ambiguity - diagnostic), a `wait`-outside-fiber null-deref (loud guard), and cancel-not-skipping-work (skip - if pre-canceled) — all fixed. Simplified `1812` (`**Fiber` → `Sh.parked`). 0156 Part 2 reframed - OPEN/non-blocking. Suite GREEN **751/0**. Next: B1.4b (deterministic-sim `Io`, the KEYSTONE). -- **B1.4b COMPLETE (this session) — deterministic virtual-time timers + a CRITICAL UAF fix.** Added - `clock_ms`/`timers`/`now_ms`/`sleep` + a timer-driven `run` to `sched.sx` (worker-built): fibers - sleep in reproducible simulated time, waking in deadline order (FIFO tiebreak). Locked `1814` - (5 fibers, wake order B@10/D@15/E@15/C@20/A@30). Adversarial review of the run-loop change found a - CRITICAL use-after-free — a fiber woken EARLY (manual/Task `wake`) before its `sleep` timer fired - was reaped while its `Timer` kept a dangling `*Fiber`; a later fire dereferenced freed memory - (silent "pass" only by luck). Fixed: `wake` evicts the fiber's pending timer (`cancel_timer_for`); - regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended - accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination. - Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll). -- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a - lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx` - (worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd - lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a - never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment - corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0. -- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack - (`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order - (task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async - runtime is feature-complete end-to-end (1800–1817), all adversarially reviewed. Suite GREEN - **755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in - B1.2). Next carve: Stream B2 (channels / cancel / async stdlib). -- **Post-review hardening (this session) — 6 findings from an adversarial review of the B1 commits.** - Fixed: **P1-a** the UFCS generic PLANNER (`calls.zig`) used the last-wins `fn_ast_map` winner while - lowering reselected by receiver → plan/lowering could disagree and MISBOX the result; now both share - `selectUfcsGenericByReceiver`. **P1-b** the selection scanned `module_decls` globally, flagging a - transitively-hidden same-named overload as a FALSE ambiguity; now two-tier — directly-visible authors - first (ambiguity only among those), global fallback for receiver-reachable namespaced methods (e.g. - `Task.cancel`) that defers to `fd0` on a hidden tie. **P2-b** boolean specificity tied `*$T` with - `*Box($T)`; now peels pointer layers so the structurally-narrower receiver wins. **P1-c** a second - concurrent `Task.wait` overwrote the single waiter slot → silent deadlock; now one-awaiter-per-task - loud abort. **P2-c** `sleep(negative)` rewound the virtual clock; now rejected loudly. (**P2-a** - non-generic-winner-hides-generic did not reproduce — the non-generic arm already falls through.) - Regressions: `examples/generics/0218` (receiver specificity + plan/lowering agreement), - `examples/concurrency/1818` (negative-sleep abort), `1819` (double-wait abort). Suite GREEN **758/0**. diff --git a/current/CHECKPOINT-METATYPE.md b/current/CHECKPOINT-METATYPE.md deleted file mode 100644 index dead75cc..00000000 --- a/current/CHECKPOINT-METATYPE.md +++ /dev/null @@ -1,276 +0,0 @@ -# CHECKPOINT-METATYPE — comptime type metaprogramming (`declare` / `define`) - -Companion to [PLAN-METATYPE.md](PLAN-METATYPE.md). Update after every step (one -step at a time, per the cadence rule). - -## Last completed step -**`type_info` / `define` widened to TUPLE types — reflect/construct triad -complete.** `TypeInfo` gained a `` `tuple(TupleInfo) `` variant (`TupleInfo{ -elements: []Type }`, positional/unnamed). `reflectTypeInfo` builds `.tuple` -(tag 2) as bare `type_tag` elements; `defineTuple` decodes `[]Type` and completes -the declare slot as a structural `.tuple` via `replaceKeyedInfo` (tuples are -structural, so the declared name is vestigial, but the slot is completed in place -so `define` returns the handle like enum/struct). `call.zig`'s `type_info` guard -admits `.tuple`. `examples/0623` (programmatic `Pair` + source-tuple round-trip). -Suite green (684). All three TypeInfo shapes now reflect + construct + round-trip -(`0619` enum, `0622` struct, `0623` tuple). - -## Earlier — struct widening -**`type_info` / `define` widened to STRUCT types.** `TypeInfo` gained a -`` `struct(StructInfo) `` variant (`StructField{ name, type }`); the metatype -system now reflects AND constructs structs, not only enums. -- `meta.sx`: `StructField` / `StructInfo` / `` `struct `` TypeInfo variant. -- `interp.zig`: `reflectTypeInfo` builds `.struct` (tag 1) for a source - `@"struct"`; `define` dispatches on the TypeInfo tag (`defineType` → - `defineEnum` (0) / `defineStruct` (1)). `defineStruct` mirrors `defineEnum` - (duplicate-field-name check included) but completes the declare slot AS a - struct via `replaceKeyedInfo` — a KIND change re-keys the intern map, whereas - `updatePreservingKey` (the enum path) asserts the key is unchanged. -- `lower/call.zig`: the lower-time `type_info` guard now admits `@"struct"`. -- `examples/0622`: programmatic `Vec2` via `.struct(.{ fields = … })` + a - source-struct round-trip `define(declare("RowCopy"), type_info(Row))`. Enum - path (`0619`) unchanged. Suite green (683). Tuple is the last shape (Next step). - -## Earlier — make_enum -**`make_enum(name, variants: []EnumVariant) -> Type`** — the general enum -constructor over `declare`/`define`, minting a nominal enum from a variant list -passed as a VALUE. Pure sx in `meta.sx`. `examples/0620` assembles the list in a -local then mints, exercising `define`'s value-arg SLICE decode. - -## Prior step -**`type_info($T)` reflection — enum round-trip.** Reflect a type INTO a `TypeInfo` -value (the inverse of `define`'s decode), so `define(declare(n), type_info(T))` -mints a byte-identical copy with NO literal variant list. - -- `inst.zig`: new `BuiltinId.type_info` (comptime-only, alongside `declare`/`define`). -- `lower/call.zig:tryLowerReflectionCall`: the old "not yet implemented" bail is - gone. Resolve `$T` at lower time, reject a non-`enum`/non-`tagged_union` arg - loudly (good span: `"type_info: 'X' is not an enum …"`), else emit - `callBuiltin(.type_info, [const_type], TypeInfo)`. -- `interp.zig:reflectTypeInfo`: builds the exact nested-aggregate Value - `defineEnum` decodes — variant `{name, payload}`, slice `{data, len}`, EnumInfo - `{variants}`, TypeInfo `{tag0, EnumInfo}`. A `tagged_union` reflects each - `field.ty` (tagless variants already carry `void`); a payloadless `` `enum `` - reflects `void` per variant. Round-trips both source enums AND constructed - (declare/define) enums. -- emit unchanged — `type_info` is always comptime-evaluated; the existing - comptime-only `else` arm in `emitCallBuiltin` (shared with declare/define) - never fires. -- Scope: **enum-only** (the symmetric inverse of `define`'s current capability). - Struct/tuple `TypeInfo` widening is a separate later step. - -`examples/0619` locks it (source enum `circle:f64 / rect:i64 / empty` reflected → -reconstructed → constructs + matches). Full suite green (676 examples + units). - -## Earlier step -**Self-reference — recursive enums via `declare("Name")` + `*Name`.** The -`declare`/`define` floor now supports self-referential types. - -- `declare(name) -> Type` mints an empty (undefined) nominal slot NAMED `name`; - `define(handle, info) -> Type` decodes the `TypeInfo` value (variant names + - payload Type-tags), fills the slot byte-identical to a source enum, and returns - the handle (one-shot form chains: `T :: define(declare("T"), info)`). Interp - executes both against a `mint` TypeTable handle; `defineEnum` + - `decodeVariantElements` in `interp.zig`. -- **Self-reference:** `evalComptimeType`'s `preregisterForwardTypes` scans the - comptime expression (and a called ctor fn's body) for `declare("Name")` calls - and, before the body lowers, registers each as an empty forward nominal type AND - binds it as a type alias. The alias is essential — a `Name :: ctor()` decl makes - `Name` a const_decl author, so a `*Name` self-reference resolves through the - forward-ALIAS path (`type_aliases_by_source`), which a bare `findByName` - registration alone does NOT satisfy (it returns a pending empty-struct stub). The - interp's `declare` returns that same slot; `define` fills it. -- A `::` binding or type-fn body calling a `Type`-returning fn is - **comptime-evaluated** (`evalComptimeType`) — no constructor-name knowledge. - `decl.zig` trigger = `fnReturnsTypeValue`; type-fn trigger = `returnExprMintsType`. -- Nominal identity rides the type-fn instantiation cache (`renameNominalType`). -- The type NAME is on `declare(name)` (compile-time string), not `EnumInfo`. - -Examples green: `0614` (one-shot), `0615` (type-fn identity), `0617` (channel -results), **`0618` (recursive `*List`: construct, match through pointer, recursive -traversal)**; `field_type` reflection `0616`. Full suite green (674 examples). - -## Current state -- `modules/std/meta.sx`: `EnumVariant` / `EnumInfo{ name, variants }` / `TypeInfo` - data types; `declare` / `define` / `type_info` / `field_type` `#builtin`s; - `RecvResult($T)` / `TryResult($T)` + the general `make_enum(name, variants)` sx - constructors over `define(declare(), …)`. -- Compiler primitives only: `declare`/`define` (construction), `field_type` - (reflection). No constructor-name knowledge anywhere in the compiler — every - named constructor is sx. `declare(name)` carries the type name (compile-time - string) for forward-type registration. -- `type_info($T)` reflects an `enum`/`tagged_union`/`struct`/`tuple` INTO a - `TypeInfo` value (`call.zig` emits `callBuiltin(.type_info)`; - `interp.zig:reflectTypeInfo` builds the Value). `define` decodes `.enum` → - tagged_union, `.struct` → struct, `.tuple` → tuple (the last via - `replaceKeyedInfo`). `examples/0619` (enum) / `0622` (struct) / `0623` (tuple) - round-trip. All three TypeInfo shapes ship. - -## Decision (kept) -**Meta lives in `modules/std/meta.sx`, not the prelude.** Declaring its data types -in the always-loaded prelude interns them into every module's type table and -shifts every `.ir` snapshot. On-demand import keeps the prelude clean. - -## Next step -The reflect/construct triad is COMPLETE — `` `enum `` (`0619`), `` `struct `` -(`0622`), `` `tuple `` (`0623`) all reflect AND construct + round-trip. Remaining -METATYPE work is ONE deferred enhancement, a clean diagnostic rather than a crash -— filed as **issue 0141** (repro `issues/0141-*.sx` + full two-layer writeup + -investigation prompt): -- **Comptime `List` growth** — `List(T).append` at comptime bails ("struct_get: - base has no fields"). Doesn't block anything: array-literal locals already build - variant lists (`examples/0620`/`0624`). Probe `.sx-tmp/probe_makeenum.sx` / - `probe_li64.sx`. **Investigated — it's TWO layers** (both reproduce with plain - `List(i64)`, not metatype-specific; List works via `#run` because that evaluates - at EMIT time, after everything is lowered, while a metatype `::` const evaluates - at `scanDecls` time): - 1. **Null comptime allocator.** `interp.zig:defaultContextValue` builds the - comptime `context.allocator` by looking up `__thunk_CAllocator_Allocator_alloc_bytes` - by name in the module's functions — but at `scanDecls` time those protocol - thunks aren't lowered yet, so `alloc_fn`/`dealloc_fn` are `.null_val` and any - comptime allocation fails. FIX (tried, works for this layer): call - `self.getOrCreateThunks("Allocator", "CAllocator")` (guarded by the same - Context/Allocator/CAllocator-registered check `emitDefaultContextGlobal` uses) - before the interp runs in `comptime.zig:runComptimeTypeFunc`. - `createProtocolThunk` saves/restores builder state, so calling it mid-lowering - is safe. After this, `alloc_fn=func_ref` — but layer 2 still bails. - 2. **`struct_get` through a `*T` slot_ptr chain.** A `*List` struct receiver - (`vs.append(…)` → `append(self: *List, …)`) lands in the interp as a slot - whose contents are a slot_ptr to the actual value — `self.field` does - `struct_get` on `base=slot_ptr field_index=1` and bails. The auto-deref in - `interp.zig:.struct_get` does a single `loadSlot`; a chain-resolve loop did - NOT fix it (the final loaded value is a field-pointer aggregate that - `resolveFieldLoad` turns back into a slot_ptr — List's comptime representation - uses field-pointers + slot_ptrs the struct_get path doesn't fully resolve). - This is the deep part: comptime pointer/struct/slot resolution for `*T` - receivers, its own focused effort. Both speculative fixes were REVERTED (no - end-to-end testable win without layer 2). -The metatype surface (declare/define/type_info/field_type + make_enum) is -feature-complete for the locked design; generic type-fn body locals now work too. -- ~~**Validation + loud diagnostics**~~ — COMPLETE. duplicate variant names - (`examples/1180`); `declare()` never `define()`d (`examples/1181`, was a - `verifySizes` panic); by-value self-reference for both source (`1178`) and - CONSTRUCTED (`1182`) types via `checkInfiniteSize`. **use-before-define needs no - new check** — it's subsumed by the existing guards: a by-value cycle → - `checkInfiniteSize` ("infinitely sized"); an unfinished slot → declare-never- - defined; a bad/non-Type payload → a 0140 clean bail; a forward reference resolves - correctly via in-place slot mutation (`updatePreservingKey`); a `*Name` pointer - needs no layout. Probes `.sx-tmp/probe_ubd{1..4}.sx` confirmed: no remaining - crash or silent-corruption, only clean diagnostics / correct results. - -### make_enum follow-ups (deferred capability gaps — NOT crashes; clean diagnostics) -`make_enum` itself is DONE (see Last completed step). Remaining adjacent -capabilities would let the variant list be built more freely; both error cleanly -(post-0140) rather than crash, so they're enhancements, not blockers: -- ~~Comptime slice over a non-string aggregate~~ — DONE. `arr[lo..hi]` over a - `[]EnumVariant` array now yields a real slice value at comptime (was: bailed, - string-only). Fix threaded `base_ty` onto the `Subslice` op so the interp tells - an array from a `{ptr,len}` slice, folded open-ended `hi` to a fixed array's - static length at lower time (no runtime/.ir change), and added - `interp.zig:subsliceElements`. `examples/0621` locks it. -- **Comptime `List` growth** (issue 0141). `List(T).append` at comptime bails - ("struct_get: base has no fields"). Investigated — two layers (null comptime - allocator at scanDecls + `struct_get` through a `*T` slot_ptr chain); see the - detailed writeup under "Next step" and `issues/0141-*.md`. Layer 1 has a known - fix; layer 2 is deep. Probe `.sx-tmp/probe_makeenum.sx`. -- ~~Generic type-fn body locals~~ — DONE. A generic `($T) -> Type` now - comptime-evaluates its FULL body (prelude statements + return), so a local - before the return resolves. `createComptimeFunctionWithPrelude` + - `evalComptimeTypeBody`; no-prelude bodies stay on the old path. `examples/0624`. - -## Known issues -- issue 0141 (OPEN, deferred enhancement — not a blocker) — `List(T).append` at - comptime bails in a type-construction `::` (two layers: null comptime allocator - + `*T` slot_ptr `struct_get`). Workaround: array-literal locals - (`examples/0620`/`0624`). Full writeup + investigation prompt in - `issues/0141-*.md`. -- issue 0140 — comptime type-construction bail panicked instead of diagnosing — - RESOLVED. `evalComptimeType` now clears `last_bail_detail` before the interp - call and, on the `catch`, emits a build-gating `.err` at the construction span - ("comptime type construction failed: {detail}") before returning the - `.unresolved` poison — so the reason is shown and no unresolved type reaches - emission unannounced. `examples/1179` locks it. -- issue 0139 — by-value self-reference segfault — RESOLVED (`checkInfiniteSize` - Pass 1g emits a loud "infinitely sized" diagnostic + breaks the cycle; - `examples/1178` locks it). - -## Log -- **Generic type-fn body locals.** A generic `($T) -> Type` comptime-evaluated - only its return EXPRESSION, so a local before the return was unresolved. Now a - body with a prelude (statements before the return) has its FULL body evaluated: - `createComptimeFunctionWithPrelude` lowers the pre-return statements into the - comptime function's scope, then the return expr. No-prelude bodies (RecvResult - etc.) stay on the old path → zero regression. `examples/0624`. Suite green (685). -- **Tuple widening done — reflect/construct triad complete.** `TypeInfo` gained - `` `tuple(TupleInfo) `` (positional `[]Type`); `reflectTypeInfo` reflects a - `.tuple` (bare type_tags, tag 2), `defineType` dispatches tag 2 → `defineTuple` - (completes the slot as a structural tuple via `replaceKeyedInfo`), and the - lower-time `type_info` guard admits `.tuple`. `examples/0623`. Suite green (684). - enum/struct/tuple all reflect + construct + round-trip. -- **Struct widening done.** `TypeInfo` gained `` `struct(StructInfo) ``; `define` - dispatches on the tag (`defineType` → `defineEnum`/`defineStruct`), `reflectTypeInfo` - reflects a `@"struct"`, and the lower-time `type_info` guard admits structs. - `defineStruct` uses `replaceKeyedInfo` (kind change: tagged_union declare slot → - struct). `examples/0622` (programmatic build + source round-trip). Suite green - (683). Tuple is the last remaining shape. -- **Validation story COMPLETE.** use-before-define needs no new check — subsumed - by `checkInfiniteSize` (by-value cycles), declare-never-defined (unfinished - slots), 0140 bails (bad payloads), and in-place slot mutation (forward refs); - `*Name` pointer use needs no layout. Probed `.sx-tmp/probe_ubd{1..4}.sx`: all - clean diagnostics / correct results, no crash. `examples/1182` locks the - by-value self-ref rejection for CONSTRUCTED enums (companion to source `1178`). -- **declare()-never-defined validation.** A bare `declare("X")` with no `define` - left a zero-field nominal slot that panicked at codegen (`verifySizes`). - `evalComptimeType` now detects a zero-variant `tagged_union` result and emits a - clean diagnostic naming the type. Self-reference (declared slot completed by - `define`) is unaffected. `examples/1181` locks it. Suite green (681). -- **Duplicate variant-name validation.** Two same-named variants in a constructed - enum used to silently succeed (ambiguous construction/match). `defineEnum` now - bails naming the duplicate; `evalComptimeType` renders it (post-0140). - `examples/1180` locks it. Suite green (680). -- **Comptime subslice over non-string aggregates.** `arr[lo..hi]` at comptime - used to bail (interp `.subslice` was string-only) and the open-ended `hi` came - from a `.length` op that misread a 2-elem array as a `{ptr,len}` fat pointer. - Fix (interp-only; runtime already correct via `LLVMTypeOf`): thread `base_ty` - onto the `Subslice` op, fold open-ended `hi` to a fixed array's static length at - lower time (no IR/.ir change), add `subsliceElements`. `examples/0621` mints an - enum from `dirs[0..2]`. Suite green (679). -- **`make_enum` done.** General enum constructor `make_enum(name, variants: - []EnumVariant) -> Type` in `meta.sx` (pure sx over declare/define). A non-generic - builder assembles the variant list in a local, then mints from it — - `examples/0620` exercises `define`'s value-arg SLICE decode. No compiler change. - Suite green (678). Deferred free-form gaps (subslice/List at comptime, - generic-type-fn locals) noted under Next step — all clean diagnostics now, not - crashes (post-0140), so enhancements rather than blockers. -- **issue 0140 fixed.** A comptime type-construction bail (`declare`/`define`/ - reflection) used to panic at LLVM emission ("unresolved type reached LLVM - emission") or hide behind a cascade — `evalComptimeType` swallowed the interp's - `last_bail_detail`. Now it clears the detail before the call and renders a - build-gating `.err` at the construction span on the `catch`. `examples/1179` - locks the empty-variants case. Suite green (677). Unblocks make_enum (its - computed-slice decode failures now surface cleanly). -- **`type_info($T)` reflection done (enum round-trip).** New `BuiltinId.type_info`; - `lower/call.zig` resolves `$T`, rejects non-enum loudly, emits the builtin; - `interp.zig:reflectTypeInfo` constructs the exact nested-aggregate Value - `defineEnum` decodes (variant `{name,payload}` / slice `{data,len}` / EnumInfo / - TypeInfo `.enum`). `tagged_union` reflects `field.ty`; payloadless `` `enum `` - reflects `void`. Round-trips source AND constructed enums. Enum-only; - struct/tuple widening deferred. `examples/0619` locks it. Suite green (676). -- **By-value self-reference rejected (issue 0139, F5 partial).** New - `checkInfiniteSize` pass (Pass 1g) detects by-VALUE containment cycles (source + - comptime types, direct + mutual), emits a loud "infinitely sized" diagnostic, - and breaks the cycle (was a `typeSizeBytes` stack-overflow segfault). `*Self` - (pointer) stays valid. `examples/1178` locks the message. Suite green (675). -- **Self-reference done.** `declare(name)` + `preregisterForwardTypes` (forward - type + alias before body lowers) → `*Name` resolves; recursive `*List` enum - constructs, matches through the pointer, and traverses recursively. `0618` locks - it. `declare` gained its `name` arg; `EnumInfo.name` dropped. Suite green (674). -- **declare/define floor established.** The comptime type-construction surface is - two primitives (`declare`/`define`); all named constructors are sx. A `::` binding - or type-fn body that calls a `Type`-returning fn is comptime-evaluated (the - builtins mint the type) — no syntactic constructor recognition in the compiler. - Examples 0614 (one-shot) / 0615 (type-fn identity) / 0617 (channel results) on the - floor; `field_type` reflection (0616) unchanged. -- **Stream carved (earlier).** Selected as the first async-first foundation: gates - channel result types (`RecvResult($T)`) and `race`'s synthesized union, fully - validated, self-contained, testable in isolation (`06xx` comptime). diff --git a/current/CHECKPOINT-MULTIRET.md b/current/CHECKPOINT-MULTIRET.md deleted file mode 100644 index b9d411de..00000000 --- a/current/CHECKPOINT-MULTIRET.md +++ /dev/null @@ -1,153 +0,0 @@ -# CHECKPOINT-MULTIRET — bare-paren multi-value + named returns - -Plan: `current/PLAN-MULTIRET.md`. Branch: `feat/multi-return`. - -## Last completed step -**Phases 0–3 implemented** (final suite + snapshot capture in progress). Examples -renumbered to the free `types` block 0202–0206 (0130/0131 already had duplicate -existing owners). -- **Phase 0** — empty `()` in the type path → `void`. (0202) -- **Phase 1** — multi-return SIGNATURES `(A, B)` / `(x: A, y: B)` / `(A, B, !)` - (≥2 value slots) parse to a `tuple_type_expr` tagged `is_multi_return`; a - single-value `(T, !)` is a plain failable (= `-> T !`). Return resolver yields - the reused tuple TypeId; `resolveParamType` rejects a multi-return tuple - (return-position-only). Consumed by destructuring. (0203, 0204) -- **Phase 2** — bare comma `return v1, v2` (positional) / `return x = v, y = w` - (named): the return parser builds the same `tuple_literal` the `.(…)` form - produces. Single positional `return v` unchanged. (used throughout 0203–0205) -- **Phase 3** — NAMED-return slots are in-scope assignable LOCALS: bound as - zero-init allocas (`bindNamedReturnSlots`), the implicit return is synthesized - from them (`synthesizeNamedReturn` → reuses `lowerReturn`), and the MUST-SET - rule errors on an unset/undefaulted slot (`bodyAssignsTo`, path-insensitive - MVP). Works with the failable error channel too. (0205 positive, 0206 negative) - -Earlier foundation: parser `collectGenericNames` descends tuple/optional/function -nodes; generic.zig `extractTypeParam` handle the `(value, !)` tuple. - -## Current state (works, verified by probes) -- `() -> ()` ≡ void; `-> (A, B)` / `-> (x: A, y: B)` / `-> (A, B, !)` multi-return. -- `return a, b` / `return x = a, y = b` bare comma; named-return locals + implicit - return + must-set diagnostic; failable named multi-return. -- `(A, B)` in a PARAM slot → loud diagnostic. -- Representation: `TupleTypeExpr.is_multi_return` flag + `Lowering.named_return_names` - state (reuses tuple ABI; no new TypeInfo variant; multi-return-ness derivable - from the FnDecl AST). - -## Post-Phase-3 changes (this session) -- **Representation refactored to a dedicated `ReturnTypeExpr` AST node** (user - preferred it over the `TupleTypeExpr.is_multi_return` flag). Resolves to a - reused `.tuple` TypeId via the shared `internTupleLike` helper. Forced - `.return_type_expr` arms onto the exhaustive `node.data` switches (sema, - semantic_diagnostics) — the coverage benefit. Param-position reject + the - named-return-locals / must-set sites now key off `.return_type_expr`. -- **Destructure-only enforcement REVERSED** (user): single-binding a multi-return - is ALLOWED. `c := f(); c.sum` works (the result is a tuple of the value slots). - For a failable multi-return, `c := f() catch …` binds only the value slots — - the error stays on the `!` channel (verified). The `callIsMultiReturn` reject - was removed. Examples 0203/0204 updated to show single-bind + field access - (output byte-identical, snapshots unchanged). - -## Phase 4 — named-return DEFAULTS (done, suite pending) -`-> (sum: i32 = 0, good: bool)`: the parser parses `= ` per slot into -`ReturnTypeExpr.field_defaults`; `bindNamedReturnSlots` seeds a defaulted slot -with its (lowered+coerced) default; a defaulted slot is EXEMPT from the must-set -rule. Also fixed `hasFnBodyAfterArrow` (the fn-def-vs-type-const lookahead) to -skip `=` + literal tokens in the return-type scan — otherwise a `=` made the decl -misread as a bodyless type-const ("expected ';'" at the body `{`). Lock: 0207. - -## Adversarial-review fixes (this session) -An adversarial review found 8 issues; fixed the soundness + silent-wrong ones: -- **#1 (segfault on a conditionally-assigned non-scalar slot)** → must-set is now - PATH-SENSITIVE definite-assignment (`definitelyAssigns`, stmt.zig): a slot not - assigned on every non-diverging path (and undefaulted) is a COMPILE ERROR, not - a runtime garbage read. `return`/`raise` count as divergence; `if` needs both - branches; `push` bodies count; `match` needs an else arm + all arms. -- **#2 (wrong-type default → segfault)** → `bindNamedReturnSlots` type-checks a - default via the coercion classifier (`.none` ⇒ diagnostic). (NOTE: the same - silent bitcast/segfault exists for ANY annotated assignment `x: i32 = "hi"` — a - broader PRE-EXISTING type-checking gap, not multi-return-specific.) -- **#3 / #8 (return arity garbage)** + **#4 (named elements ignored)** → - `validateMultiReturn` (stmt.zig, called from `lowerReturn`): rejects a bare - value where ≥2 are required, wrong arity, a comma list from a single-value fn, - and named elements out of slot order. (Reordering-by-name is a future nicety; - for now a mismatch is a loud error, never silent-wrong.) -- **#5 (slot shadows param)** → collision diagnostic in `bindNamedReturnSlots`. -- **#7 (push/defer false must-set error)** → subsumed by the DA rewrite (push - bodies count; defer correctly does NOT, as it runs after the implicit return). - -## Known limitations / next -- ~~**#6 (design gap)**: `ReturnTypeExpr` silently accepted in non-return positions~~ - — **DONE** (2026-06-27): generic-type-arg position now rejected - (`rejectMultiReturnValueType` at both `instantiateGenericStruct` arg-resolution - sites, generic.zig). Param / field / variable already rejected. Type-alias - `T :: (A,B)` is value-parsed → already rejected. Closure-RETURN `(A,B)` is a - legitimate return position → see D3 below (works as a multi-return closure). - Lock: 0215 (negative generic-arg). -- ~~**Reordering named return elements by name** (vs requiring slot order)~~ — - **DONE** (2026-06-27): `reorderNamedReturn` (stmt.zig) permutes a fully-named - multi-return list to slot order by name (value-only AND full-failable-tuple - forms); errors on unknown / duplicate / missing-slot names; positional & mixed - lists pass through unchanged. `validateMultiReturn`'s old slot-order check was - removed. Adversarial review caught a silent mis-permute in the full-failable- - tuple named form (now reordered/validated, not positionally dropped). Lock: - 0210 (positive reorder, incl. failable) + 0214 (negative: unknown / duplicate). -- ~~**PRE-EXISTING**: annotated-assignment type mismatch (`x: i32 = "hi"`) segfaults~~ - — **RESOLVED** as issue 0197 (2026-06-27): width-mismatch guard - (`checkAssignable` / `noneReinterpretIsUnsafe`, coerce.zig) at every - annotated-slot store site; the named-return-default guard now shares it. Locked - by `examples/diagnostics/1205` + `1206`. -- ~~Multi-return CLOSURE-TYPE values / lambda literals deferred (D3).~~ — - **RESOLVED** (2026-06-27): they ALREADY WORK via the reused tuple machinery. A - `Closure() -> (A, B)` value's call result destructures (`a, b := cb()`), - single-binds + field-accesses (`c := cb(); c.0`), and a `() => { return v1, v2; }` - lambda literal satisfies a multi-return closure param — verified identical to - the function-decl surface. NO `ClosureInfo.multi_return` marker needed (the - destructure-only rule was reversed, so there's nothing extra to enforce). Lock: - 0216. -- **Generic multi-return (Task 2d): DONE.** POSITIONAL works — `(a: $T, b: $U) -> (T, U)` - (inferred) and `($T: Type, …) -> (T, U)` (explicit); lock 0217. NAMED-slot - implicit-return form now works too (issue **0200 RESOLVED** — - `monomorphizeFunction` now calls `bindNamedReturnSlots`; covers free fns + - generic struct methods, defaults, failable); lock 0218. -- Docs: readme.md / specs.md not yet updated for multi-return (docs-track rule). - -## Known issues -- ~~**issue 0198**: implicit `Any → T` unbox unchecked (segfault / silent garbage)~~ - — **RESOLVED** (2026-06-27): implicit `Any → T` is now a compile error - (`coerceMode` `.unbox_any` arm, mode == .implicit); `xx` + match dispatch - unaffected. Locked by `examples/diagnostics/1207`. -- ~~**issue 0199**: `Any == ` aborts the LLVM verifier~~ — **RESOLVED** - (2026-06-27): the `Any`-shaped `==`/`!=` arm (expr.zig) now fires when EITHER - operand is `.any`, boxing the concrete side first. Lock 0654. -- ~~**issue 0200**: NAMED generic multi-return implicit-return "produces no value"~~ - — **RESOLVED** (2026-06-27): `monomorphizeFunction` now calls - `bindNamedReturnSlots` (it previously bound params but skipped named-return - slots). Covers generic free fns + struct methods, defaults, failable. Lock 0218. - -## Log -- **2026-06-27 session** (handover: issue 0197 → finish multi-return → Io Phase 3): - - **issue 0197 RESOLVED** — width-mismatch guard at every annotated-slot store - site (var/const-decl, single + multi assignment for identifier/field/index/ - element/deref, named-return defaults). Examples 1205 + 1206. Adversarial review - caught & fixed: a bare-fn-ref false-positive (size-discriminator via - `typeSizeBytes`, not the wrong fn-ref typing) and an aggregate-overrun - false-negative (sx-padded `sizeOf` → LLVM-accurate `typeSizeBytes`); cascade - suppression via `externalErrorsExist` (guard tallies its own diagnostics). - - **issue 0198 RESOLVED** — implicit `Any → T` unbox is now a compile error - (reviewer-confirmed sound). Example 1207. **issue 0199 FILED** (Any==concrete - LLVM-verify abort, loud, open). - - **multi-return Task 2 DONE** (2a reorder 0210/0214; 2b reject in generic-arg - 0215; 2c D3 closures already work 0216; 2d positional generic works 0217 + - named-generic gap filed as 0200). Multi-return feature surface complete. - - **REMAINING** (next session): **Task 3 Io-unification Phase 3** (the - capture-typing blocker below + true cancellation — needs fresh context + both - macOS & aarch64-linux validation per PLAN-IO-UNIFY.md). (0198/0199/0200 all - resolved this session; no open multi-return/type-check issues remain.) -- Pivoted here from the Io-unification Phase 3 (true cancellation), which is - PAUSED at its blocker: capturing a failable closure into a nested closure loses - its failability (`worker() catch` → operand type 'unresolved'; repro - `.sx-tmp/pD.sx`/`pE.sx`). That capture-typing gap is unrelated to multi-return - and waits for a later session. The Io-Phase-3 stdlib edits (core/sched/io + - example 1825) were REVERTED to keep the tree green; the multi-return-relevant - compiler changes were kept. -- Foundation landed + suite green; plan + checkpoint written. diff --git a/current/PLAN-ASM.md b/current/PLAN-ASM.md deleted file mode 100644 index 3e1fcd5c..00000000 --- a/current/PLAN-ASM.md +++ /dev/null @@ -1,167 +0,0 @@ -# sx Inline Assembly — Implementation Plan (ASM stream) - -**Design source of truth:** [design/inline-asm-design.md](../design/inline-asm-design.md). -This plan turns that doc's §II.7 stage-map + §II.8 phasing into ordered, -commit-sized, testable steps. Read the design doc first — this file is the -*how/when*, not the *what/why*. - -**Surface (decided):** -`asm volatile { "template", "=r" -> T, "r" = expr, clobbers(.cc, .memory) }` -— brace block; `->` output / `=` input; `clobbers(.…)` dot-name list; N `-> Type` -outputs return a tuple; templates are pure AT&T (via LLVM). - -**Feasibility (confirmed):** sx links LLVM@19; `src/llvm_api.zig` `@cImport`s -`llvm-c/Core.h`, so `llvm_api.c.*` already exposes `LLVMGetInlineAsm` (9-arg), -`LLVMInlineAsmDialectATT`, `LLVMBuildCall2`, `LLVMAppendModuleInlineAsm`. No shim. - -**Relationship to other streams:** -- Phases A–E (the inline-asm *expression*) are independent of EXTERN-EXPORT. -- Phase F (global asm) consumes `extern`/`export` to import/expose asm symbols — - do it **after** `PLAN-EXTERN-EXPORT.md` Phase 2. - -## Cadence (IMPASSIBLE) -No commit may both add a test AND make it pass. Each feature step is either a -behavior-locking PASSING test, or an xfail test the *next* commit turns green. -Arch-pinned tests live in `examples/16xx-platform-asm-*` and declare their target -via the `expected/.target` sidecar marker (Phase 0). Never regenerate -snapshots while red. - -## Phase 0 — corpus target-gating (test-infra prerequisite; no compiler code) -**Why first.** The flagship v1 examples are `x86_64` (syscall-write, divmod, -cpuid) but the dev host is `aarch64`-Darwin, and the corpus runner -([src/corpus_run.test.zig](../src/corpus_run.test.zig)) currently (a) never threads -a per-example `--target` and (b) has no host-arch gate — its only skip is "marker -has no `.sx`". So D.0's `…-syscall-write` markers asserting exit/stdout describe -output the harness *cannot* produce on this host, which would violate the cadence -rule (the "next commit turns it green" can never happen). Phase 0 closes that gap. -It touches **only the runner + two fixtures** — zero compiler code, zero risk to -A–E, and unblocks every arch-pinned asm example. - -**Marker taxonomy (the cleanup).** The runner currently spreads per-example -*directives* across standalone boolean/value sidecars (`.aot` now, `.target` -proposed, more later). Replace that sprawl with **one optional config file, -`expected/.build`**, holding all build/run directives; the output snapshots -(`.exit` / `.stdout` / `.stderr` / `.ir`) stay separate — they are -machine-regenerated data, not config. `.exit` remains the **test-discovery key** -(every test has one; `.build` is optional). - -**`.build` format** — JSON, parsed with `std.json`: -```json -{ "aot": true, "target": "x86_64-linux" } -``` -Parse via `std.json.parseFromSlice(BuildConfig, …)` into -`struct { aot: bool = false, target: ?[]const u8 = null }`. Field defaults cover -omitted keys; `std.json`'s default `ignore_unknown_fields = false` makes an -**unknown key a loud `error.UnknownField`** (surfaced as a runner failure, never a -silent ignore — CLAUDE.md no-silent-default rule). Extensible: future `"cpu"`, -`"link"`, `"cwd"` are just new optional struct fields, no new sidecar file and no -custom parser. - -**What the directives do:** - -1. **`target = `** threads `--target ` into every `sx` - invocation for that example (`run` / `build` / `ir` — `--target` is a global - flag, confirmed [main.zig:39](../src/main.zig#L39)), AND **host-match selects - the mode.** The runner parses the leading `arch` + `os` tokens of the resolved - triple and compares them to `@import("builtin").target` (normalizing - `arm64`→`aarch64`): - - **match** → *execute* exactly as today (`sx run`, or `aot` build+exec) with - the target threaded, plus the `.ir` diff if an `.ir` snapshot exists. ⇒ an - x86_64 example gives **real end-to-end coverage on an x86_64 CI runner**. - - **mismatch** → **ir-only**: run *only* `sx ir --target `; assert - `.exit` (the ir command's exit), `.ir` (normalized stdout), and `.stderr` - (diagnostics, normally empty). Do **not** run/build/exec; do **not** assert - `.stdout`. An `.ir` snapshot is **required** in ir-only mode — its absence is - a loud runner failure ("arch-pinned : ir-only mode requires an .ir - snapshot"), never a silent pass. Robust even if `sx ir` treats `--target` as - a partial no-op: the `inline_asm` op carries the template + constraint string - verbatim, so the IR snapshot still locks the exact thing §II.11 flags as - silently-miscompiling (the constraint assembler + template rewrite). -2. **`aot`** is the existing JIT-vs-build+exec switch, just relocated from the - standalone `.aot` marker into `.build`. - -**Negative compile-error examples need NO `.build`.** `…-missing-volatile` -(no-output-without-`volatile`) is a Sema diagnostic raised before codegen/JIT, so -plain `sx run` reports it identically on any host — it stays a normal example with -no config file. - -**update-goldens interaction:** in ir-only mode, `-Dupdate-goldens` writes `.exit` -(ir exit) + `.ir` (+ `.stderr` if non-empty) and skips `.stdout`. Execute mode -(incl. `aot`) is unchanged. `.build` is hand-authored — update-goldens never -writes it. - -| Step | Commit | What | Files | -|---|---|---|---| -| 0.0 | lock | Add `BuildConfig` + `std.json` parse of `expected/.build` (unknown-key ⇒ `error.UnknownField`); **migrate** the 2 existing `.aot` markers → `.build` (content `{ "aot": true }`) and delete them; thread `target`'s `--target` into the spawned argv; add `hostMatchesTarget(value) bool` (arch+os token parse, `arm64`→`aarch64`) gating the **execute** path. Lock with `examples/16xx-platform-target-host.sx` (trivial `main`) + a `.build` `{ "target": "" }` (still runs+passes) and unit `test`s for the JSON parse + `hostMatchesTarget`. | `src/corpus_run.test.zig`, `examples/expected/1226-*.{aot→build}`, `…/1227-*`, + fixture | -| 0.1 | lock | Implement the **mismatch ⇒ ir-only** branch (skip run/build/exec; assert `.exit`+`.ir`+`.stderr` from `sx ir --target`; require `.ir`). Lock with `examples/16xx-platform-target-cross.sx` (asm-free `() -> i64 { return 0; }`) + `.build` `{ "target": "x86_64-linux" }` + a checked-in `.ir` snapshot — exercises ir-only on the arm64 host. | `src/corpus_run.test.zig` + fixture | -| 0.2 | docs | Update CLAUDE.md §"Test layout"/§"Testing" to document `.build` (format + `aot`/`target` keys) replacing the standalone `.aot` marker prose (lines ~435, ~492). | `CLAUDE.md` | - -Both 0.0 and 0.1 are **lock** commits: the runner change and the fixture that -exercises it land together and pass the moment they land (the mechanism works -immediately — nothing is left red), which is the cadence rule's "lock in current -behavior" flavor, not a feature red→green. No asm lowering is gated on either. - -**Phase 0 verification:** `zig build test` green; deliberately corrupt the -cross-target `.ir` fixture and confirm the runner reports an IR mismatch (proves -ir-only actually asserts, isn't a no-op); delete it and confirm the -"requires an .ir snapshot" failure fires. - -**Estimated runner delta:** ~70–90 lines (sidecar read + `--target` argv threading -+ `hostMatchesTarget` + the ir-only branch + update-mode tweak). Within the -"no step > ~500 new lines" rule; well under the read budget. - -## Phase A — keyword + AST + parser (parses; no codegen) -| Step | Commit | What | Files | -|---|---|---|---| -| A.0 | lock | add `kw_asm` keyword + map entry; unit lex test `asm → kw_asm` | `src/token.zig`, `src/lexer.zig` + `.test.zig` | -| A.1 | xfail | parse `asm { … }` → `AsmExpr`/`AsmOperand` in `parsePrimary`; pin an AST/`sx ir` parse snapshot; lowering still `bailDetail("inline asm codegen unimplemented")` | `src/ast.zig` (:85 union arm, :721 structs), `src/parser.zig` (parsePrimary), `src/ir/interp.zig` | -| A.2 | green | parse-shape snapshot lands green; the unimplemented bail is loud + named | — | - -## Phase B — sema / typing -| Step | Commit | What | Files | -|---|---|---|---| -| B.0 | xfail | result-type rule (0→`void` / 1→`T` / N→named-or-positional tuple) + checklist (no-output⇒`volatile`, layout, comptime-string template) — pin error messages | `src/ir/expr_typer.zig` | -| B.1 | green | typing + diagnostics implemented; `.unresolved` sentinel on failure (no silent default) | `src/ir/expr_typer.zig`, `src/ir/semantic_diagnostics.zig` | - -## Phase C — IR op + lowering -| Step | Commit | What | Files | -|---|---|---|---| -| C.0 | lock | add `inline_asm: InlineAsm` to `Op` + `AsmOperand` (role/name/constraint/operand) + interp `bailDetail` arm; unit tests for the IR shape | `src/ir/inst.zig` (:80), `src/ir/interp.zig` | -| C.1 | xfail→green | `lowerAsmExpr` in `lowerExpr` dispatch — interns template/constraints/clobber-names, lowers input `Ref`s, sets result `TypeId` | `src/ir/lower/expr.zig` | - -## Phase D — LLVM emit (single value-output; the core) -| Step | Commit | What | Files | -|---|---|---|---| -| D.0 | xfail | `examples/16xx-platform-asm-syscall-write.sx` + `…-register-read.sx` + `…-no-output-volatile.sx` + `…-missing-volatile.sx` (expected compile error) — all red | examples + `expected/` markers | -| D.1 | green | `emitInlineAsm`: **port `FuncGen.airAssembly`** — constraint-string assembler (outputs `=`/`+`, inputs, `clobbers(.name)`→`~{name}`), `%[name]`→`${N}` / `%%` / `%=` template rewriter, `LLVMGetInlineAsm`+`LLVMBuildCall2`, `sideeffect=volatile`, AT&T dialect | `src/ir/emit_llvm.zig` (emitInst dispatch + handler) | -| D.2 | green | lock the template-rewrite + constraint string via an `expected/*.ir` snapshot on `…-template-subst.sx` | examples | - -**Phase D verification:** `zig build test`; the syscall example runs on -`x86_64-linux`; IR snapshot matches the design doc's worked `sys_write` lowering. - -## Phase E — multi-return tuples + `clobbers(.…)` -| Step | Commit | What | Files | -|---|---|---|---| -| E.0 | xfail | `…-asm-multi-return.sx` (`divmod`→`(quot,rem)`, `cpuid`→4-tuple) red | examples | -| E.1 | green | N `out_value` → LLVM struct return + `extractvalue i` → sx tuple (named when operands named); `clobbers(.name)` dot-name lowering finalized | `src/ir/emit_llvm.zig`, `src/ir/lower/expr.zig` | - -## Phase F — global asm (needs EXTERN-EXPORT Phase 2) -| Step | Commit | What | Files | -|---|---|---|---| -| F.0 | xfail | top-level `asm { … }` decl parsed (reject operands/`volatile`); `…-asm-global.sx` (defines a symbol, imported via `extern`) red | `src/parser.zig`, `src/ast.zig` | -| F.1 | green | lower `asm_global` → `c.LLVMAppendModuleInlineAsm`; comptime-call guard (dlsym-miss is loud); blocks concatenate in source order | `src/ir/lower/decl.zig`, `src/ir/emit_llvm.zig`, `src/ir/interp.zig` | - -## Phase G — later (own steps when scheduled) -`-> @place` write-through + read-write (`"+r" -> @place`) + indirect-memory -(`"=*m"`) outputs · `%=` unique-id · output-to-const rejection · Intel-dialect -opt-in · naked functions (`callconv(.naked)`, coordinate with EXTERN-EXPORT). - -## Open decisions (design doc §II.10) -Dialect (AT&T-only v1, recommended) · `volatile` contextual-keyword (recommended) -· brace separator comma (recommended) · `clobbers(.name)` dot-name sugar now → -checked per-arch `Clobber` enum later (Phase 4 of the design doc). - -## End-to-end verification (per phase) -`zig build && zig build test`; for arch-pinned examples confirm they run on a -matching host or assert on `sx ir`/`.s` snapshots. After intentional output -changes only: `zig build test -Dupdate-goldens`, then review the diff. diff --git a/current/PLAN-ATOMICS.md b/current/PLAN-ATOMICS.md deleted file mode 100644 index 3f79e79b..00000000 --- a/current/PLAN-ATOMICS.md +++ /dev/null @@ -1,225 +0,0 @@ -# PLAN-ATOMICS — Stream A (atomics lowering) - -> **STATUS: ✅ COMPLETE (feature-complete).** All phases A.0 → A.3 landed + green. -> Surface shipped: `Atomic($T)` `load`/`store`/`fetch_add`/`sub`/`and`/`or`/`xor`/`min`/`max`/ -> `swap`/`compare_exchange`/`compare_exchange_weak` (all comptime `$o: Ordering`) + free -> `fence(.ordering)`. IR ops `atomic_load`/`store`/`rmw`/`cmpxchg`/`fence`. Both LLVM emit -> AND the comptime VM implemented (verified to agree). Enabled by net-new comptime value -> params (enum/tagged_union/generic-struct methods — 3 commits). Corpus `17xx` (1700-1704) + -> `11xx` diagnostics (1130/1131/1186/1187). Commits: 22af404, 64c7db5, 8144a88, acf3183, -> 718f27e, 0531164, 68ed732, dca396e, 79895be, fca4304, b65544a (+ comptime-param 3c4305f, -> d7a6857, d95ba0a). **Unblocks Stream B2 channels + Stream C parallel schedulers.** -> -> Deferred (documented, NOT legacy — intentional scope): RMW/CAS/swap are integer-only -> (float fadd / pointer atomics out of scope); fence/orderings explicit (no defaults — the -> comptime-default-on-generic-method gap is orthogonal). Asm-level arch divergence + -> weak-memory *semantics* remain OUT of corpus scope (Stream-C stress harness). - -Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream A + the design-of-record -[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §3 (N1) -+ §4.6 (locked surface). Progress in [CHECKPOINT-ATOMICS.md](CHECKPOINT-ATOMICS.md). - -**Goal:** net-new LLVM atomic codegen. Surface = a pure-sx `Atomic($T)` generic struct + -an `Ordering` enum (ordinary sx), with the actual atomic operations recognized as -`#builtin` intrinsics at lower-time and emitted as new IR ops. This is **100% net-new** — -no atomics scaffolding exists (the only `lower.zig` "ordering" is *comparison* ordering -`< <= >=`, unrelated to memory ordering — do not mistake it for groundwork). - -**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then -flip to green); `zig build && zig build test` green after every step; never regen snapshots -while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff. -New corpus category: `17xx` atomics. - ---- - -## Design (grounded against the tree) - -### Representation — minimal compiler surface -- **`Ordering`** is an ordinary sx enum, zero compiler coupling: - ```sx - Ordering :: enum { relaxed; acquire; release; acq_rel; seq_cst; } // tags 0..4 - ``` -- **`Atomic($T)`** is an ordinary sx **generic struct** (mirrors `List :: struct ($T: Type)` - at [list.sx:5](../library/modules/std/list.sx#L5)), a transparent 1-field wrapper — - atomicity is a property of the *operation*, not the storage, so `Atomic(i64)` has the - exact layout/size/align of `i64`. NO new IR *type*, NO type-system coupling: - ```sx - Atomic :: struct ($T: Type) { - value: T; - init :: (v: T) -> Atomic(T) { return .{ value = v }; } - load :: (self: *Atomic(T), o: Ordering) -> T { return atomic_load(T, @self.value, o); } - store :: (self: *Atomic(T), v: T, o: Ordering) { atomic_store(T, @self.value, v, o); } - } - ``` -- The **operations** are `#builtin` intrinsic free functions, recognized by name at - lower-time (the established pattern — `size_of`/`type_info` in - [`tryLowerReflectionCall`](../src/ir/lower/call.zig#L1672), recognized BEFORE arg lowering): - ```sx - atomic_load :: ($T: Type, ptr: *T, o: Ordering) -> T #builtin; - atomic_store :: ($T: Type, ptr: *T, v: T, o: Ordering) #builtin; - ``` - Explicit `$T` first arg follows the `size_of($T)` / `field_name($T, idx)` mixed - type+value precedent (lowest-risk; the reflection path already resolves type args). - -### Ordering is compile-time-only by construction — and that forces a capability gap -LLVM atomic ordering is an **instruction attribute**, not a runtime operand, so the -ordering MUST be known at emit time. The lower-time handler reads the ordering arg's -variant name statically (it must be a **constant enum literal** `.seq_cst`) and bakes it -into the IR op as a Zig enum field (`AtomicOrdering`). A non-literal ordering is a **loud -diagnostic**, never a silent default (REJECTED-PATTERNS). - -**Discovered gap (grounded):** a generic `Atomic(T)` method `load(self, o: Ordering)` would -forward `o` — a *runtime parameter* — to the intrinsic, where it is NOT a literal. And -**comptime enum value params don't exist** (`$o: Ordering` → `o` is "unresolved" in the -body; `resolveValueParamArg` folds integer constraints only). A runtime dispatch hack -(`if o == { case .acquire: atomic_load(…, .acquire) … }`) also fails: `load` with a -`release`/`acq_rel` ordering is *invalid LLVM*, so the arms can't be uniform. Therefore the -**full ordering surface is blocked on a net-new capability** (comptime-constant ordering -propagation — either comptime enum value params, or compiler-recognized `Atomic` method -calls). That capability is its **own step (A.0.5)**, sequenced before ordering-bearing ops. - -### sx tag → LLVM ordering is EXPLICIT (non-contiguous!) -LLVM's `LLVMAtomicOrdering` is **not** 0..4: `Monotonic=2, Acquire=4, Release=5, -AcquireRelease=6, SequentiallyConsistent=7` ([Core.h:338-354]). The sx `Ordering` tags -(relaxed=0…seq_cst=4) map via an explicit `switch`, never an identity cast: -`relaxed→Monotonic, acquire→Acquire, release→Release, acq_rel→AcquireRelease, -seq_cst→SequentiallyConsistent`. - -### LLVM-C API (verified present in `llvm-c/Core.h`, no new extern decls needed) -- Atomic load = `LLVMBuildLoad2` + `LLVMSetOrdering(v, ord)` + `LLVMSetAlignment(v, size)` - (**alignment is mandatory** on atomic load/store — LLVM verifier rejects atomics without - it). There is **no** `LLVMBuildAtomicLoad`/`Store` (the Explore agent was wrong). -- Atomic store = `LLVMBuildStore` + `LLVMSetOrdering` + `LLVMSetAlignment`. -- (Later) `LLVMBuildAtomicRMW(B, op, ptr, val, ord, singleThread)`, - `LLVMBuildAtomicCmpXchg(B, ptr, cmp, new, succOrd, failOrd, singleThread)`, - `LLVMBuildFence(B, ord, singleThread, name)`, `LLVMSetWeak`. -- `singleThread = 0` (multi-thread / cross-thread ordering). Atomic-eligible `T` = - integer / pointer / float of size 1·2·4·8(·16). **Reject non-scalar / bad-size `T` - loudly** (diagnostic), do not silently emit. - -### Comptime VM treats atomics as ordinary load/store -Comptime is single-threaded, so seq_cst is trivially satisfied — the -[`comptime_vm`](../src/ir/comptime_vm.zig#L659) arms for `atomic_load`/`atomic_store` -reuse the ordinary `load`/`store` paths (correct, NOT a bail). `sx run` JITs via LLVM so -runtime atomics execute the real ops; the VM arm only matters for `#run`/const-init. - -### Files the new IR op variants force (exhaustive switches) -`atomic_load` / `atomic_store` variants must be handled in every `Op` switch or the Zig -build fails (this is the desired tripwire): -- [inst.zig:159](../src/ir/inst.zig#L159) — add `atomic_load: AtomicLoad`, `atomic_store: AtomicStore` + the structs (mirror `Store` at [inst.zig:286](../src/ir/inst.zig#L286)). -- [lower/call.zig:1672](../src/ir/lower/call.zig#L1672) — recognize the intrinsics, emit the ops (new `tryLowerAtomicIntrinsic`, called alongside `tryLowerReflectionCall` at [call.zig:80](../src/ir/lower/call.zig#L80)). -- [print.zig:231](../src/ir/print.zig#L231) — print arms (sx-IR / `ir-dump`). -- [emit_llvm.zig:1566](../src/ir/emit_llvm.zig#L1566) — dispatch arms → ops.zig. -- [backend/llvm/ops.zig:325](../src/backend/llvm/ops.zig#L325) — `emitAtomicLoad`/`emitAtomicStore` (mirror `emitLoad`/`emitStore`). -- [comptime_vm.zig:659](../src/ir/comptime_vm.zig#L659) — arms reusing load/store. -- Any other `.op` switch the Zig compiler flags (module.zig / program_index.zig) — let the build tell you. - -### Test snapshots — the arch-`.ir` requirement is a MISCONCEPTION for atomics -`sx ir` = [`emitIR`](../src/main.zig#L210), which emits **LLVM IR** (respects `--target`); -`sx ir-dump` is the sx-IR printer. At the **LLVM-IR level**, `load atomic i64, ptr %x -seq_cst, align 8` is **arch-invariant** — identical text for x86_64 and aarch64. The -x86-`lock`/MOV vs aarch64-`ldar`/`stlr` divergence happens only at *instruction selection* -(`sx asm`), which the corpus does **not** snapshot. So: -- **A single host `.ir` snapshot** proves the achievable gate (the `load atomic ` - keyword + correct ordering + alignment emitted). PLAN-POST §A / design §10.3's - "arch-gated x86_64 + aarch64 `.ir`" would capture **byte-identical** files — drop it. -- Optionally add ONE cross-arch ir-only example (`.build {"target":"x86_64-linux"}` on an - aarch64 host) purely as a **cross-target-emission-doesn't-crash** smoke — note in its - header that the IR body is identical to host. -- **State loudly (out of snapshot scope, parallel to the ordering-semantics caveat):** - asm-level arch lowering AND weak-memory ordering *semantics* are NOT proven by `.ir`; - those need the Stream-C stress harness, not the corpus. - ---- - -## Phases - -### A.0 — `Atomic($T)` + `Ordering` + **`seq_cst`-only** `load`/`store` ← START HERE -**Scope (descoped per the discovered gap above):** ship the net-new atomic load/store -codegen with a **`seq_cst` literal baked in the method bodies** — `load(self) -> T` / -`store(self, v)` (NO ordering param yet). The intrinsic still carries the full -`AtomicOrdering` field (always `.seq_cst` here); the recognizer + emit handle all five -orderings already, so A.0.5 only has to plumb the *constant* through. Explicit orderings -(`a.load(.acquire)`) land in A.0.5. seq_cst-only is correct (conservative-strongest), not a -silent fallback. - -Two-commit cadence (lock-to-bail → green): - -- **A.0a (lock)** — land the lib + IR plumbing with emit deliberately bailing: - 1. New `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (value + - `init`/`load`/`store`), `atomic_load`/`atomic_store` `#builtin` decls. **Opt-in import - (`#import "modules/std/atomic.sx"`), NOT carried by the universal `std.sx` facade** — - mirrors `trace`. Rationale (grounded): adding the concrete `Ordering` enum to the - universal prelude registers it into EVERY program's global type table, growing - `@__sx_type_is_unsigned` (378→380) and shifting all string-global numbering → churned - 37 unrelated `.ir` snapshots + bloats every binary. Atomics is a deliberate concurrency - capability, so consumers import it explicitly. - 2. Add IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` + the two op structs - (inst.zig); print arms; comptime_vm arms (reuse load/store); lower recognition - (`tryLowerAtomicIntrinsic`) incl. the const-ordering-literal guard + non-scalar-`T` - reject. - 3. emit_llvm/ops.zig arms **bail loudly** for now: `emitAtomicLoad`/`Store` call the - emitter's bail-with-diagnostic path ("atomic load/store LLVM emission not yet - implemented") so the Zig build is exhaustive but the example is red-by-diagnostic. - 4. Add `examples/1700-atomics-load-store.sx` (construct `Atomic(i64).init`, `store`, - `load`, `print`). Seed marker; capture snapshot = the emit-bail diagnostic (nonzero - exit). `zig build && zig build test` green (matches the locked bail snapshot). Commit. -- **A.0b (green)** — replace the emit bail with real emission: - `LLVMBuildLoad2`+`LLVMSetOrdering`+`LLVMSetAlignment` / `LLVMBuildStore`+`LLVMSetOrdering` - +`LLVMSetAlignment`, ordering via the explicit sx-tag→LLVM `switch`. Regen `1700` to - success output + capture its host `.ir` (asserts `load atomic`/`store atomic` + ordering). - Add a unit test in `emit_llvm.test.zig` (correct op + ordering + alignment emission). - Review the diff (no stray error text). Commit. - -### A.0.5 — comptime-constant ordering propagation (the capability gap) -Enable `a.load(.acquire)` etc. — i.e. an `Ordering` that reaches the intrinsic as a -compile-time constant through a method. Two candidate designs (pick at pickup): -- **(a) comptime enum value params** — make `$o: Ordering` resolve in the body to its - variant tag (extend `comptime_value_bindings`/the typer beyond integers). General, - reusable; larger typer change. -- **(b) compiler-recognized `Atomic` methods** — special-case `Atomic(T).load/store/…` - calls (read the literal ordering arg at the method call site), bounded coupling to the - std `Atomic` type (cf. how `Vector` is special-cased). Smaller; less general. -Also enforce per-op ordering validity (load: relaxed/acquire/seq_cst; store: -relaxed/release/seq_cst; CAS's dual orderings) as **compile errors**, which is exactly what -the constant-ordering path buys. Retrofit the ordering param onto `load`/`store` here. - -### A.1 — RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`) -One IR op `atomic_rmw` carrying an `RmwKind` (maps to `LLVMAtomicRMWBinOp*`). Signed vs -unsigned min/max picks `Max/Min` vs `UMax/UMin` from `T`'s signedness. Same lock→green -cadence; `17xx` examples. - -### A.2 — `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**) -`atomic_cmpxchg` op (ptr, cmp, new, success_ord, failure_ord, weak). LLVM `cmpxchg` -returns `{T, i1}`; lower to `?T` where **null = success** (extract the i1, invert). -**Validate the two orderings in the compiler** (design §4.6): failure ordering may not be -`release`/`acq_rel` nor stronger than success — loud diagnostic. `_weak` sets `LLVMSetWeak`. - -### A.3 — `swap` + `fence(.ordering)` -`swap` = `atomic_rmw` with `Xchg` kind (folds into A.1's op). `fence` = a new `atomic_fence` -op (ordering only) → `LLVMBuildFence`. `17xx` examples. - ---- - -## Gates (per the corrected snapshot story) -- **unit** `emit_llvm.test.zig`: each op emits the right LLVM builder + ordering + alignment. -- **corpus** `17xx` single-thread deterministic (`sx run`, JIT executes real atomics). -- **host `.ir`** snapshot per op proves the keyword/ordering/alignment lowered. -- **OUT of snapshot scope, stated loudly:** asm-level arch divergence (`sx asm`) and - weak-memory ordering *semantics* — Stream-C stress harness territory, not the corpus. - -## Kickoff prompt (A.0a — paste into a fresh session) -> Implement Stream A step A.0a (atomics lock commit) per `current/PLAN-ATOMICS.md`. Verify -> `zig build && zig build test` is green first. Then: (1) create -> `library/modules/std/atomic.sx` with the `Ordering` enum, `Atomic($T)` struct, and -> `atomic_load`/`atomic_store` `#builtin` decls; wire into `library/modules/std.sx`'s tail. -> (2) Add the `atomic_load`/`atomic_store` IR ops + `AtomicOrdering` + op structs in -> `src/ir/inst.zig`; handle them in every exhaustive `Op` switch the Zig build flags -> (print.zig, comptime_vm.zig reuse load/store, emit_llvm dispatch). (3) Add -> `tryLowerAtomicIntrinsic` in `src/ir/lower/call.zig` (recognize the two builtins, bake the -> const ordering literal into the op, loud-reject non-literal ordering AND non-scalar/bad-size -> `T`). (4) Make `emitAtomicLoad`/`emitAtomicStore` in `src/backend/llvm/ops.zig` BAIL loudly -> ("not yet implemented") this commit. (5) Add `examples/1700-atomics-load-store.sx`, seed the -> marker, capture the bail diagnostic as the locked snapshot, confirm `zig build test` green, -> commit. STOP — A.0b (real emission) is the next step. Do NOT implement emission in the same -> commit that adds the example. diff --git a/current/PLAN-COMPILER-VM.md b/current/PLAN-COMPILER-VM.md deleted file mode 100644 index 1bcfaf25..00000000 --- a/current/PLAN-COMPILER-VM.md +++ /dev/null @@ -1,741 +0,0 @@ -# PLAN — Comptime Bytecode VM + comptime memory (then re-home the compiler-API on it) - -> **Direction change (2026-06-17).** The comptime compiler-API stream pivots off the -> **byte-weld**. The weld (sx structs whose layout is validated to mirror the -> compiler's Zig types) + the **serialization / marshaling** bridge at the call -> boundary is the wrong direction — it bolts a parallel layout regime and hand-built -> byte-copies onto a comptime value model that fundamentally isn't bytes. We strip it -> and build the right foundation: a **bytecode VM over byte-addressable -> memory**, where comptime values ARE native bytes (like runtime). On that base the -> compiler-API needs no weld, no validation, no marshaling — the compiler's own types -> are read/built directly as memory and its functions take/return real pointers. -> -> Supersedes the build order in `design/comptime-compiler-api.md` (kept for history). -> This is the active plan for the stream. Branch: `reify`. - -## Why - -`src/ir/interp.zig` is a tree-walking interpreter over the SSA IR that represents -every value as a tagged `Value` union (`int`, `float`, `aggregate: []const Value`, -`type_tag`, `heap_ptr`, …). Two consequences: - -1. **Slow.** Per-value boxing in a tagged union; per-op `switch` over `Inst`; an - aggregate is a heap `[]const Value`, walked element-by-element. -2. **Not native memory.** A struct value is `[]const Value` (tagged unions), NOT the - struct's bytes. So a comptime `@ptrCast(*StructInfo)` reads the `Value` union's - memory, not a `StructInfo` — which forced the whole weld+marshal detour. - -Make comptime values **native bytes in byte-addressable memory** and both problems dissolve: -structs/arrays/slices are their bytes at natural layout (no weld), the compiler's own -records are directly addressable (no marshal), and a bytecode loop over comptime memory is -fast. - -## End state - -- Comptime execution = a **bytecode VM** over a **byte-addressable memory** (real - host-allocated bytes; layout is **target-aware** via the type table's sizes). Values - are bytes at addresses plus a scalar register file. No tagged `Value` union. -- The comptime compiler-API: the compiler **exposes its real types + functions** to - comptime sx. sx reads/builds them as native memory and calls compiler functions by - pointer. No `abi(.zig)` weld, no `validateStructLayout`, no `register_struct` - field-by-field marshaling — gone. -- `declare`/`define`/`type_info` and `#compiler`/`BuildOptions` ride this one - mechanism; the bespoke interp arms are deleted. -- **ONE evaluator at the end — non-negotiable.** The legacy tagged-`Value` interpreter - (`interp.zig`) is **DELETED**. We do NOT ship both permanently. "Dual-path" - (a compiler-API fn with both a legacy `compiler_lib` handler AND a VM-native impl) and - the emit-time legacy fallback are **transitional only** — scaffolding while the VM - reaches parity at BOTH comptime sites (emit time AND lowering time). The flag - `-Dcomptime-flat` is the swap mechanism; once the VM runs everywhere with parity, the - flag, the fallback, and `interp.zig` all go. Any "VM-only at emit, legacy at lowering" - split is a waypoint, never the destination. - -## Principles (hold at every step) - -- **Green at every step.** `zig build && zig build test` pass after each sub-step. The - existing tagged-`Value` interpreter stays the live evaluator until the VM reaches - corpus parity; swap behind a build flag, then delete the old path. -- **Target-aware, not host-baked.** Flat-memory layout uses the type table's target - sizes (`pointer_size`, `typeSizeBytes`/offsets), NEVER host `@sizeOf`. This is what - keeps cross-compilation correct (the JIT-comptime alternative could not). -- **Sandboxed.** Flat-memory accesses are bounds-checked; step/call-depth budgets - remain; an OOB / bad access traps to a build-gating diagnostic with a source span — - never a compiler-process crash. -- **No silent fallbacks** (per CLAUDE.md): an unhandled op / shape bails loudly with a - named reason, never a zero/default that looks like success. - -## Phases - -### Phase 0 — Strip the weld / serialize / marshal machinery -Delete the wrong-direction code so the VM builds on a clean base. Pure removal + -corpus rebaseline; suite green. - -- `src/ir/compiler_lib.zig`: the reflection (`weldStruct` / `bound_types` / - `FieldLayout` / `BoundType`), the layout validation (`validateStructLayout` / - `LayoutMismatch` / `SxField`). Decide the fate of the `bound_fns` host-call registry - (`intern`/`text_of` handlers) — it is likely subsumed by the VM's compiler-call path - in Phase 3, but `intern`/`text_of` may survive as the first such calls. -- `src/ir/lower/nominal.zig`: `validateWeldedStruct` + `weldedFieldOrderStr` + the - `sd.abi == .zig` validation call in `registerStructDecl`. -- `src/ir/interp.zig`: the `compiler_welded` dispatch branch. -- `src/backend/llvm/ops.zig`: the `emitCall` comptime-only gate keyed on - `compiler_welded` (re-derive the comptime-only guard from a non-weld signal if still - needed). -- Corpus: retire / convert the weld examples + diagnostics — `0625`, `0627` (welded - struct), `1183`, `1186` (weld-layout diagnostics), `1184`/`1185` (welded-fn). Keep - `0626` (`intern`/`text_of` round-trip) only if it survives the new call path. -- **Keep (re-evaluate in Phase 3), independent of the weld semantics:** the - `#library "compiler"` decl, the `abi(.x)` annotation + `extern ` syntax, and the - `callconv → abi` unification. These are surface syntax that may still serve the - compiler-API; only the *weld semantics* are stripped here. - -**Verification:** `zig build test` green with the weld machinery gone; the surviving -syntax still parses (parser unit tests). - -### Phase 1 — Flat-memory value model (still IR-walking, no bytecode yet) -Introduce comptime memory and move comptime values onto it, **decoupled from bytecode** so -the value-model change is isolated. Each sub-step ports one op group and keeps the -corpus green; the OLD tagged path stays behind a build flag (`-Dcomptime-flat`) until -all groups land, then the shim is deleted. - -1. **Machine + scalars.** A comptime memory region (host `[]u8`) with a stack (frames) + - bump-allocated heap, and a scalar register file. Port `int`/`float`/`bool`/`undef` - and arithmetic/compare/branch. Aggregates still go through a compat shim to the old - representation. -2. **Aggregates.** Structs/arrays/tuples laid out in comptime memory at **target** layout; - port `struct_init` / `struct_get` / `array` / `index_gep` to read/write bytes at - computed offsets. -3. **Slices / strings.** `{ptr, len}` fat pointers in comptime memory. -4. **Optionals / enums / tagged unions.** Tag + payload bytes. -5. **Pointers.** `alloca` / `store` / `load` / GEP unified onto comptime addresses; retire - `slot_ptr` / `heap_ptr` / `byte_ptr` in favor of comptime addresses. -6. **Closures.** Fn id + captured env materialized in comptime memory. -7. **Extern / host calls.** A struct arg is already bytes → pass its address; this - removes most of `marshalExternArg`. -8. **Reflection / minting.** `declare` / `define` / `type_info` read comptime - values; type-table mutation copies escaping data into compiler-owned memory at the - boundary (lifetime), as today. - -**Verification:** with `-Dcomptime-flat` the full corpus (currently 692) is byte-for- -byte identical to the tagged path; then make the VM the default and delete the shim. - -### Phase 2 — Bytecode -Compile a comptime function's IR → a compact bytecode and execute the bytecode instead -of walking `Inst`. Pure encoding/speed; semantics identical to Phase 1. Land at least a -minimal register-bytecode loop (the stream's stated goal is a *bytecode* VM); a -fragment cache is optional follow-up. - -**Verification:** corpus identical to Phase 1; comptime throughput measurably improved -on a heavy-comptime micro-benchmark. - -### Phase 1.final — host wiring (the remaining integration) -The wiring ENTRY POINT exists: `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a -comptime function entirely on the VM and returns a legacy `Value`, or `null` to fall -back. Unit-tested (pure `6*7` → 42; unsupported → null). Remaining to actually route the -host through it: -1. **Panic→error hardening (prerequisite).** `Machine.readWord`/`writeWord`/`bytes` - currently `assert` (debug panic) on null/OOB. For arbitrary host functions to be - safe, make them return `error.OutOfBounds` so a malformed run BAILS (→ null → legacy) - instead of crashing the compiler. Ripples through `readField`/`writeField`/slice - helpers (add `try`). -2. **Implicit context.** Host comptime functions may have `has_implicit_ctx` (param 0 = - `*Context`); the legacy `run` materializes a default ctx. The VM `run` does not — so - either materialize it too, or only route `tryEval` at funcs without implicit ctx. -3. **Wire one site** behind a flag/env (`SX_COMPTIME_FLAT`, → `-Dcomptime-flat` later): - the const-init fold in `emit_llvm.zig` `emitGlobals` (`result = tryEval(...) orelse - interp.call(...)`). Default off → corpus unaffected. -4. **Parity + coverage.** Run the corpus with the flag ON; results must be byte-identical - to legacy. Measure how many comptime evals the VM already handles; the bail `detail`s - name what to port next (tagged-union payload / any / closures / builtins). -5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge) - until the VM is the default and the legacy path is deleted. - -**Status (2026-06-17): steps 1–4 DONE; step 5 = the next session.** -- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return - `error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of - asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through - `readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and - every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a - null-deref function → `tryEval` returns `null` (legacy fallback), not a panic. -- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a - conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with - `has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the - right size/align allocated in comptime memory, its address passed as arg 0. The common - const body never reads the ctx; a body that USES the allocator loads a fn from it and - `call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed: - handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed) - empirically confirms no divergence. (A body that read+branched on a null allocator fn - could in principle diverge; none does — parity is the guard.) -- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat) - tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT` - (a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`). - Default OFF → corpus unaffected (688 green). -- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed); - manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace - facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`, - printing HANDLED / fallback+reason per init). -- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.** - Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity - stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx - materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer - optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the - destination = none/empty; unit-test regression); curated libc MEMORY builtins on comptime - memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped, - `free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed, target-aware, - result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32 - storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single — - `readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had - written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added). -- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage - **27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF. - Per the user's direction ("the VM can set up a default context"), `runEntry` now - materializes the REAL default context (not a zeroed one): the implicit-ctx param is an - opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global - and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying - the CAllocator thunk func-refs) into comptime memory via a new recursive `layoutConst`. - With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays - reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect` - (decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body - that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` → - `context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` → - `libc_malloc` → the VM's native comptime `malloc`. Unlocked `0606` (string global via - the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func` - (memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/ - `struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by - `alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to - `.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s - rejection diagnostic byte-identical). Unit tests added (global_get, func_ref + - call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk - bottoms out at libc `malloc`, and the VM can't use a host pointer with comptime - load/store, so comptime `malloc` must allocate from comptime memory. The default context - lets the allocator PROTOCOL run; native `malloc` is its final step. -- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage - **31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF. - - **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`. - - **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set) - → word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)` - shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 | - span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push` - / `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions — - linked into the compiler — so the return-trace buffer the host reads is populated - identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now. - - **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):** - `readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a - stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently - hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded; - gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests - pass via 1037/1038 in the corpus). - - **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):** - `intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm - global call (`1654`, never comptime-evaluable). Every other measured corpus const-init - is handled on the VM. - At this point the comptime VM handles essentially the entire real comptime corpus - (scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls + - recursion, the implicit context + allocator protocol, globals, failables + return - traces). Phase 2 (bytecode) and Phase 3 (compiler-API on comptime memory) are the forward - work; flipping the VM to default + deleting the legacy path awaits those. -- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second - comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run ;`) now - routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields - `.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed - (`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy - re-run double-push them — both sites now `sx_trace_clear()` right before the legacy - fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All - comptime evaluation now routes through the VM-with-fallback (uniform). -- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM - gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init` - reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test - -Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity - gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var; - default-built does not). This is the prerequisite to eventually making the VM default + - deleting the legacy path (which still awaits Phase 2/3 + broader confidence). -- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.** - `invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is - the safety boundary) via `Vm.callCompilerFn` — natively on comptime memory, NO legacy - `Interpreter`: `intern(s: string) -> StringId` reads the string bytes from comptime memory and - `internString`s into the (const-cast) table (pool-only, never touches type layout, so the - VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back - into comptime memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback - is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688** - both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next - compiler functions (`find_type`, `register_struct`, the reflection readers) are added the - same way — comptime pointer in, handle/pointer out, no marshaling. - -**Phase 3 progress (2026-06-18):** -- **(P3.1) First read-only reflection readers — `find_type` + `type_field_count` (DONE).** - Two more `compiler`-library fns bound the same way as the `intern`/`text_of` seed - (added to `compiler_lib.bound_fns` AND `Vm.callCompilerFn`, native on comptime memory, no - marshaling). A **type handle is a plain `u32` `TypeId`** (exactly like `StringId`), so - both calls keep the seed's clean scalar shape — handle in, scalar out: - `find_type(name: StringId) -> TypeId` (`TypeTable.findByName`) and - `type_field_count(t: TypeId) -> i64` (a new `TypeTable.memberCount` query — struct/union/ - tagged-union fields, enum variants, array/vector length — that BOTH the legacy handler - and the VM call, so the two paths can't drift). Example `0628` chains - `intern → find_type → type_field_count` and a not-found lookup, both folded at `#run`, - both VM-HANDLED natively (no fallback). Parity **689/689** (gate ON and OFF); VM unit test - added. - - **Decision (resolves the plan's `find_type → ?Type` sketch):** `find_type` returns a - NON-optional `TypeId`, using the codebase's dedicated `unresolved` (0) sentinel for - not-found — NOT an `?Type`. Rationale: a `Type` value resolves to `.any` - (`type_resolver.zig`), which the comptime VM does not represent; and an optional - return can't cross the legacy↔VM eval boundary (`regToValue` bridges only - word/string/struct/tuple). `unresolved` is the project-blessed unmistakable "no type" - marker (see CLAUDE.md REJECTED PATTERNS — a dedicated sentinel is the required shape), - so the caller checks the handle against 0. This keeps the reader a clean scalar mirror - of `intern`/`text_of` and defers `.any`/optional plumbing to when it's actually needed. -- **(P3.2) Field-level reflection readers — `type_nominal_name` + `type_field_name` + - `type_field_type` (DONE).** Three more readers on the same `TypeId`-handle shape (each - backed by a new `TypeTable` query that BOTH the legacy handler and the VM call, so no - drift): `type_nominal_name(t: TypeId) -> StringId` (`nominalName` — a named type's own - name; loud-bail for unnamed types), `type_field_name(t: TypeId, idx: i64) -> StringId` - (`memberName` — struct/union/tagged-union field, enum variant, named-tuple element), and - `type_field_type(t: TypeId, idx: i64) -> TypeId` (`memberType` — struct/tuple/array/vector - member type). All loud-bail on out-of-range idx / no-member (no silent default). These are - the first MULTI-ARG compiler fns (the VM's `callCompilerFn` now reads arg 1 = idx); added - `Vm.argHandle`/`argTypeId` helpers (range-checked u32/TypeId arg reads). Naming uses the - `type_*` family so nothing collides with the std metatype builtins (`field_name`/`type_name` - exist in `core.sx`). Example `0629` reflects `Pair { lo: Point; hi: Point }` — reads each - field name and the nominal name of a field's type, all folded at `#run`, all VM-HANDLED - natively. Parity **690/690** (gate ON and OFF); VM unit test added. -- **(P3.2b) Kind + enum-value readers — `type_kind` + `type_field_value` (DONE).** The last - two read-only readers the metatype's `type_info(T)` needs, completing the READ side: a - comptime sx fn can now fully reflect a struct/enum/tagged-union/tuple into data with no - `#builtin`. `type_kind(t: TypeId) -> i64` (`TypeTable.kindCode` — a stable, compiler-owned - discriminant: 0 other · 1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · - 7 vector · 8 error_set; TOTAL — never bails, an unnamed/non-aggregate type reads `other`) - and `type_field_value(t: TypeId, idx: i64) -> i64` (`TypeTable.memberValue` — an enum - variant's explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for - a non-enum / out-of-range idx). Example `0630` reflects `Color`/`WindowFlags`(flags)/`Point`. - Parity **691/691** (gate ON and OFF); VM unit test added. - - **READ side now complete:** `find_type` + `type_kind` + `type_field_count` + - `type_field_name` + `type_field_type` + `type_nominal_name` + `type_field_value` cover - everything `reflectTypeInfo` reads. -- **(P3.3) WRITE side — `declare_type` + `pointer_to` + ONE kind-branching `register_type` (DONE).** - The mutating side is a SINGLE `register_type(handle, kind, members)` that branches on `kind` - IN THE COMPILER (subsuming `define`'s `defineStruct`/`defineEnum`/`defineTuple`), plus - `declare_type(name) -> Type` (forward handle) and `pointer_to(t) -> Type` (build `*T` - references). They take/return real `Type` values (matching meta.sx's declare/define). - - **Timing decision (per the user):** mint LAZILY at LOWERING time (single pass, NOT a - pre-emit phase, NOT two-pass) — the existing `runComptimeTypeFunc` path. So the write - side is **legacy-only** (`compiler_lib` handlers); the VM isn't wired at lowering time, so - no VM mirror is needed (the read-side readers stay dual-path for emit-time reflection). A - non-generic `-> Type` builder is now flagged `is_comptime` (`decl.zig`) so its dead body - permits the welded calls (the comptime-only gate). - - **Graph support:** forward `declare_type` handles + `pointer_to` express a - mutually-recursive A↔B graph (`*A`, `*B`, B-by-value) before bodies are filled. - `register_type` is **idempotent** — re-filling a nominal slot (same module reached via two - import edges) re-mints identically instead of erroring (`nominalIdent` reads identity from - any nominal kind). `kind` codes match `type_kind`: 1 struct · 2 enum (actual `.@"enum"`) · - 3 tagged_union · 4 tuple. - - **Two bugs fixed en route** (issue 0142): (a) a fully payloadless comptime-minted enum - was minted as an all-void `tagged_union` → `verifySizes` panic; now mints a real - `.@"enum"` (both `register_type` kind 2 AND the metatype `defineEnum`). (b) bare - `EnumType.variant` qualified construction of a payloadless variant wasn't supported (failed - for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`). - - Examples: `0631` (graph + actual-enum + reflection), `0632` (make_enum all-void), - `0633`/`0634`/`0635` (namespaced / bare / multi-edge import of a minted type), `0187` - (qualified variant construction). Parity 697/697 (gate ON and OFF); unit tests added. -- **Next (P3.4):** re-express `declare`/`define`/`type_info` as sx over the read+write - compiler-API and DELETE the bespoke interp arms — needs the VM hardened against malformed - lowering-time IR first (the metatype runs at lowering time), so either harden + wire the VM - there, or migrate the metatype onto the legacy compiler-API calls first. Decide when reached. - Phase 2 (bytecode) is the orthogonal speed work. - -### Phase 3 — Compiler-API on comptime memory (resume the stream — no weld) -With native-byte comptime values, re-home the compiler-API: - -- **Expose the compiler's real types.** Register the actual `types.zig` records - (`StructInfo`, `EnumInfo`, `Field`, …) into the comptime type table under sx-visible - names, with their **real (host) layout** — the type IS the compiler's, so there is - nothing to validate or keep in sync. (This is the projection that *replaces* the - weld's reflection — owned by the compiler, not declared in sx.) -- **Expose the compiler's functions.** `register_struct`, `find_type`, `intern`, - `text_of`, and the reflection readers operate on comptime pointers / handles - directly (no marshaling — the bytes already ARE the record). -- **Re-express** `declare` / `define` / `type_info` as sx over these; delete the - bespoke interp arms (`defineStruct` / `defineEnum` / `defineTuple` / `reflectTypeInfo`); - migrate `examples/0622` (struct), `0619`/`0620`/`0623` (enum/tuple). -- **Migrate `BuildOptions`** off `#compiler` onto this mechanism; **delete `#compiler`**. - -**Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over -the exposed compiler-API; full corpus green. - -### Phase 4 — Retire the legacy interp (the ONE-evaluator end state) - -The metatype CONSTRUCTION + REFLECTION surface is VM-native (steps 7/8 — `0614`–`0624`, -`0632` all HANDLED). This phase moves EVERYTHING ELSE off `interp.zig` and deletes it. - -**What the legacy interp is still used for (audited 2026-06-18) — five roles:** - -| Role | Wired to VM? | Site | -|------|--------------|------| -| **A. Comptime folds** (type-fn / `::` const-init / `#run`) | ✅ VM + legacy fallback | `comptime.zig:530`, `emit_llvm.zig:871`/`971` | -| **B. `#insert` string eval** | ❌ legacy-only (VM wiring reverted — 0737 malformed-IR crash) | `comptime.zig:634` | -| **C. Post-link bundler** (`platform.bundle` — Info.plist/codesign/process/fs) | ❌ legacy-only | `core.zig:invokeByFuncId` ← `main.zig:769` | -| **D. `#compiler` hooks** (`compiler_call` — BuildOptions/bundling) | ❌ legacy-only; `Value`-based ABI | `compiler_hooks.zig`, `interp.zig:1130` | -| **E. Bail diagnostics** (`Interpreter.last_bail_*` statics) | n/a | `main.zig:464` | - -Shared substrate everything traffics in: the **`Value`** tagged union (the -`regToValue`/`valueToReg` bridge + the hooks + `core.zig`) and the **host-FFI bridge** -(`host_ffi.zig` + `interp.callExtern` — dlsym + cdecl trampolines for real libc). - -**DECISION (2026-06-18, user): UNIFY.** The VM gains a host-FFI escape + real-pointer -translation and runs BOTH sandboxed comptime folds AND the unsandboxed post-link bundler. -`interp.zig` is fully deleted — true ONE evaluator, two modes (sandboxed / host-effects). - -**Remaining comptime-fold gaps** (full corpus fallback inventory — 15 examples; 1179/1180 -are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan artifact): -`box_any`/`unbox_any` (6), `out`/print (2), `global_addr` (1), trace frames (1), -`compiler_call` (2 — role D). - -**Sub-phases (dependency order; each its own session, both gates 697/0 after each):** - -- **4A — finish comptime ops (small, parity-guarded).** Drive the fold fallback list to - empty except `compiler_call`: - - **4A.1** `box_any`/`unbox_any`. Word case = alloc 16B `{tag@0, value@8}`, tag = - `source_type.index()` (matches legacy comptime; note runtime `anyTag` normalizes - arbitrary-width ints), value via `writeField(source_type)` (so f32 etc. round-trip); - unbox = `readField(addr+8, target)`. Aggregate-Any payload needs the runtime - pointer-in-value-slot shape (`coerceToI64` alloca+ptrtoint) — implement or bail loudly. - - **4A.2** `out`/print → add a VM output buffer; flush through the same path as - `core.flushInterpOutput`. - - **4A.3** `global_addr` (address-of a global in comptime memory). - - **4A.4** trace frames (`sx_trace_*` / `interp_print_frames`). -- **4B — VM-native diagnostics (role E). MUST land before deleting legacy.** Today a VM - bail silently falls back; with legacy gone the VM bail IS the user-facing build-gating - diagnostic. Surface the VM's `detail`/span/file into what `main.zig` renders; turn - 1179/1180-style bails into proper diagnostics. No diagnostic may regress. -- **4C — `#insert` on the VM (role B).** Re-wire `evalComptimeString` through `tryEval`; - the lowering-time-IR hardening that forced the 0737 revert is already in place. Verify - the `#insert` corpus parity. -- **4D — host FFI on the VM (role D substrate). DONE.** Solved by a better allocator, not a - pin/tag scheme: the comptime memory is now an **arena** of stable host allocations and `Addr` - IS a real host pointer (`4D.0`, `625ba0f`), so a comptime pointer and an FFI-returned host - pointer are the same value — no translation, no realloc hazard. `Vm.callHostExtern` - (`4D.1`, `e7a8708`) dispatches ANY extern via `host_ffi` dlsym + trampolines (args/returns pass - untouched); `4D.2` (`6a7f690`) adds slice/string args (→ NUL-term `char*`) + float guards. - Examples 0636/0637. **(Superseded sub-note:** the earlier "pin the buffer / comptime↔host translate" - hazard is moot — the arena never moves an allocation.) -- **`#compiler` / `compiler_call` — DELETED, replaced by the `abi(.compiler)` ABI (decision 2026-06-18, - REVISED from the earlier `abi(.zig) extern compiler` shape).** A function is *compiler-domain* — it runs in - the comptime evaluator (VM/interp), NEVER in the shipped binary — because its **ABI says so**: `abi(.compiler)`. - No `extern `, no fake `#library "compiler"`. One annotation covers BOTH roles: (a) the **compiler-API - surface** (`intern`/`find_type`/`build_options`/`set_post_link_callback`/… — bodiless decls whose Zig/VM - handler is the impl, on `compiler_lib`'s export list, dispatched by `Vm.callCompilerFn`); (b) **user - compiler-domain functions** like post-link callbacks (`bundle_main` — BODIED `abi(.compiler)`, lowered for VM - eval but emit-skipped). The `#compiler` struct attribute + the `compiler_call` IR op + the `Value`-based hook - `Registry` (`compiler_hooks.zig`) all **go away**. **Why this is cleaner than the welded-fn approach:** the - former runtime-call enforcement blocker (a `build_options()` call inside an LLVM-emitted callback body) is - MOOT — a compiler-domain function is never emitted, so its compiler-API calls never reach `emitCall`. - **Staged build (each its own step, both gates green):** - - **S1+S2 — DONE (2026-06-18):** introduced `abi(.compiler)`, REMOVED the `.zig` ABI + `abi(.zig) extern - compiler` + `#library "compiler"` (clean cutover, no legacy); migrated all compiler-API examples. The - binding now keys off `fd.abi == .compiler` (`decl.zig` `weldedCompilerFn`); a bodiless `abi(.compiler)` - decl lowers extern-like (declared-not-defined) with no implicit ctx. **700/0 both gates.** - - **S3 — DONE (2026-06-18):** emit_llvm skips BODIED `abi(.compiler)` function bodies. Added an - `is_compiler_domain` flag to the IR `Function`; a bodied `abi(.compiler)` function LOWERS its body (for VM - eval) + is flagged `is_comptime` but is NOT emitted (Pass 2 skip; declared external-linkage so the empty - decl verifies). KEY fix: a call to a comptime-only callee (compiler-API `compiler_welded` OR - `is_compiler_domain`) inside a dead comptime body now emits `undef` instead of a real `call` (`ops.zig` - `emitCall`) — the old `compiler_call` did this; without it an AOT link leaves an undefined `_double`/`_intern` - reference (this also fixed a pre-existing untested AOT breakage of the bodiless compiler-API examples). - `fnIsBodilessCompiler` distinguishes the API surface (declare-only) from a compiler-domain callback (lowered, - emit-skipped). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` const, - absent from the binary, JIT+AOT). **701/0 both gates.** - - **S4 — callback-param propagation: OPTIONAL / DEFERRED (ergonomics only).** Verified 2026-06-18: an - `abi(.compiler)` function is TYPE-compatible with a plain `() -> R` param (the ABI marks the *function* — - `is_compiler_domain` — not its *type*, which stays `() -> R` CC-default). So a callback that needs to be - compiler-domain just declares itself `abi(.compiler)` (S3) and passes to a plain param fine; auto-propagation - from an `abi(.compiler)` PARAM type is a nicety, not a prerequisite for S5. Skipped for now. - - **S5a — DONE (2026-06-18):** the corpus-covered slice. `build_options` + `set_post_link_callback` → - free `abi(.compiler)` functions (VM `callCompilerFn` arms + legacy `compiler_lib` handlers); **`BuildConfig` - threaded into the VM** via a `tryEval` param (the same one `main.zig` forwards — shared with 4E). `build.sx` - extracts `set_post_link_callback` from the `struct #compiler` as a free `ufcs` fn; `bundle_main` + the - platform registrars (`configure`) are `abi(.compiler)`. 37 examples' `.ir` snapshots regen'd (benign: - declaration renumber + `@str` suffix shift — every example imports build.sx via the prelude). Strict - `compiler_call` bails 6→2; 0602/0603/1604/1611 HANDLED. **701/0 both gates.** - - **S5b/S5c (port the ~37 hooks) — SUPERSEDED 2026-06-18 by the sx-driven build pipeline (below).** - Porting each `BuildOptions` accessor to an `abi(.compiler)` function that delegates to a `compiler_hooks` - hook just re-encodes sx-level logic (string setters/getters, `is_macos` triple-matching, list appends) as - compiler hooks. The hooks need NOTHING from the compiler except the `BuildConfig` state. So instead of 37 - hooks, **drive the whole build pipeline from sx** (the logical end of "bundling lives in sx"). S5a stays as - a green intermediate; the sx-build-pipeline replaces `build_options`/`set_post_link_callback`/the whole - `#compiler` surface wholesale. - -### Phase 5 — sx-driven build pipeline (replaces the BuildOptions hooks; decision 2026-06-18, user) - -**The build pipeline becomes an sx program.** `BuildConfig` is plain sx data (an ordinary struct, sx-owned -end-to-end — no `#compiler`, no hooks, no shared Zig state, no weld/offset access). The compiler shrinks to -a few `abi(.compiler)` PRIMITIVES that take **explicit args** (so nothing is shared by memory), and an sx -`build()` driver orchestrates configure → emit → link → bundle. **Chosen boundary: Option B** — the compiler -keeps the proven Zig linker as a primitive; sx owns config + orchestration + bundle. (Option A — sx shells -`cc`/`ld` itself — is a later refinement once the per-target link-line logic is ported to sx.) - -**File split (user decision 2026-06-19):** the low-level compiler-API PRIMITIVES live in -`library/modules/compiler.sx` (the comptime `compiler` library — renamed from the interim `std/build.sx`); the -default `build` IMPLEMENTATION (`default_build` + the `on_build` slot + the sx `BuildConfig`) lives in -`library/modules/build.sx` alongside the existing `BuildOptions` DSL. So `compiler.sx` = primitives, `build.sx` = -orchestration/default impl. **Build-callback fallibility was DROPPED (user 2026-06-19):** the primitives + the -build callback are NOT `-> !` — a failed action (e.g. `link`) BAILS on the VM (hard build error). So the shapes -below shed their `-> !`. - -Shape (build-callback fallibility dropped 2026-06-19): -```sx -// library/modules/compiler.sx (the comptime `compiler` library — PRIMITIVES) -emit_object :: () -> string abi(.compiler); // emitted .o path (query) -link :: (objects: List(string), output: string, libraries: List(string), - frameworks: List(string), flags: List(string), target: string) abi(.compiler); // void; bails on failure -c_object_paths :: () -> List(string) abi(.compiler); // metadata queries -link_libraries :: () -> List(string) abi(.compiler); - -// library/modules/build.sx (the build DSL — DEFAULT IMPLEMENTATION + slot) -BuildConfig :: struct { output: string; target: string; flags: List(string); - frameworks: List(string); bundle_path: string; bundle_id: string; ... } -default_build :: (config: BuildConfig) abi(.compiler) { // the default pipeline (void) - obj := emit_object(); objs := c_object_paths(); objs.append(obj); - link(objs, config.output, link_libraries(), config.frameworks, config.flags, config.target); - if config.bundle_path.len > 0 { bundle_app(config); } } // bundle_app = today's sx bundler -on_build : (BuildConfig) abi(.compiler) = default_build; // the override slot -// user overrides: build :: (config: BuildConfig) abi(.compiler) { ... } #run on_build = build; -``` -The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfig` → read `on_build` → invoke -`on_build(config)` on the VM; a `raise` fails the build. Plain `sx run` fires none of it. - -**Steps (each its own green step; depends on 4E first):** -- **P5.1 — 4E prereq — DONE (2026-06-19).** `core.invokeByFuncId` routes the post-link callback through the - **VM** (`comptime_vm.tryEval`), NO fallback (a side-effecting callback can't double-execute): a bail is a hard - build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). `BuildConfig` + - `import_sources` threaded in; `flushInterpOutput` deleted (VM `out` writes direct via host-FFI). Smoke test - `examples/1661-platform-post-link-vm-list` (AOT): a post-link callback GROWS a `List` (0141 — works on the VM, - bails on legacy with `struct_get`), so the build succeeds (exit 0) only via the VM. Non-empty callback `args` - rejected loudly (the `on_build(config)` arg-marshaling entry is P5.3). **702/0 both gates.** -- **P5.2 — primitives.** Split: the read-only **metadata queries are DONE (2026-06-19)** — `c_object_paths() -> - List(string)` + `link_libraries() -> List(string)` as `abi(.compiler)` fns (stdlib `library/modules/compiler.sx`), - serviced by `comptime_vm.callCompilerFn` over `BuildConfig` fields `main.zig` forwards; new VM `makeStringList` - builds the `List(string)` in comptime memory from the call's result type (`ins.ty` now threaded through - `invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both - gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits - the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all - three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER - DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard - build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks` - vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM - readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test - `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output — - the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0. -- **P5.3 — `on_build` registrar — DONE (2026-06-19).** `on_build(cb)` registers the build callback - (`cb: (opt: BuildOptions) -> bool abi(.compiler)`); the compiler force-lowers + auto-invokes the well-known - `default_pipeline` when no override. (Implemented as a registrar, not an assignable slot — the opaque - `BuildOptions` handle is one word, so arg-passing needs no struct marshaling.) -- **P5.4 core — DONE (2026-06-19).** `default_pipeline` in `build.sx` drives the whole build; NO Zig - auto-emit/auto-link; `emit_object`/`link` are sx-called actions via the `BuildHooks` vtable; - `set_post_link_callback` deleted (all callers on `on_build`). Build-path auto-imports `modules/build.sx`. - 703/0 both gates. - -### THE FINAL DIRECTION (user, 2026-06-19): FULL MIGRATION — NO LEGACY LEFT. - -**Decision: DROP gate-OFF entirely.** The VM becomes the SOLE comptime evaluator; `-Dcomptime-flat` is made -permanent then removed; `interp.zig` (the legacy tagged-`Value` `Interpreter`) is DELETED. There is no -dual-path, no legacy `compiler_lib` handler, no `regToValue`/`valueToReg` bridge, no VM→legacy fallback. We -migrate the BuildOptions surface DIRECTLY to VM-native `abi(.compiler)` arms (no legacy handler — there is no -legacy to handle). **All bundling + code signing for EVERY target lives in the sx `default_pipeline`.** - -- **P5.5 — DONE (2026-06-19).** The 35 `BuildOptions :: struct #compiler` methods migrated to VM-native - `abi(.compiler)`: `BuildOptions :: struct { }` (opaque null-sentinel handle) + 35 free - `ufcs (self: BuildOptions, …) abi(.compiler)` decls in `build.sx`, serviced by a new - `comptime_vm.callBuildOptionFn` arm off `callCompilerFn` — **NO legacy `compiler_lib` handler** (names - registered in `bound_fns` with a single bailing stub only so `weldedCompilerFn` accepts them). Setters dupe the - arg string into the PERSISTENT `Vm.gpa` (the Compilation allocator — threaded into both `tryEval` and - `runBuildCallback` — NOT the per-eval VM arena) and write/append to the threaded `BuildConfig`; string getters - return the field (or `""`); bool getters compute from the triple (`predIsMacOS`/…); count/index getters read the - `BuildConfig` slices. **Dispatch routing (Option B):** a `#run`/const-init entry that directly calls a - compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with NO legacy fallback regardless of the - `-Dcomptime-flat` gate → gate-OFF stays green without a legacy BuildOptions handler. 5 `platform/bundle.sx` - getter-calling helpers marked `abi(.compiler)` (comptime-only bundler code). 37 `.ir` regenerated (string-pool - churn; behavior-identical, verified `.ir`-only). **703/0 BOTH gates.** BuildOptions `compiler_call` bails GONE - (1609/1614/1615 strict-clean); 1616 now bails on `shr` — a SEPARATE unported bitwise/shift VM gap - (`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`), to port FIRST in P5.6 (1616 is unpinned + can't JIT-run on - macOS regardless). Also swept the outdated "flat memory" terminology → "comptime/byte-addressable" (the VM is - arena-backed, `Addr` = real host pointer; flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept). -- **P5.6 — ALL bundling + code signing in `default_pipeline` (every target).** `default_pipeline` (or a - `bundle()` it calls, in `platform/bundle.sx`) performs, after `link`, the full per-target bundle when - `bundle_path()` is set — branching on `is_macos`/`is_ios_device`/`is_ios_simulator`/`is_android`: - - **macOS `.app`** — `Contents/{MacOS,Resources,Frameworks}`, `Info.plist`, embed `-framework` dylibs + - `install_name_tool` fixups, `codesign` (ad-hoc or with `codesign_identity`). - - **iOS device `.app`** — device slice, embedded `.mobileprovision` (`provisioning_profile`), entitlements, - `codesign` with the real identity; **iOS simulator `.app`** — sim slice, no provisioning, ad-hoc sign. - - **Android `.apk`** — `AndroidManifest.xml` (or `manifest_path` override), asset tree (`add_asset_dir`), - `#jni_main` Java → `javac` → `d8` → `classes.dex`, `aapt2` package, `zipalign`, `apksigner` with the - debug/`keystore_path` keystore. - All of it runs on the VM via the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI (the existing - `platform/bundle.sx` logic, now reading the VM-native accessors instead of `#compiler` hooks). The compiler - keeps ONLY the linker as a primitive (Option B). Remove the `--bundle`/`post_link_module` Zig shim — bundling - is `default_pipeline`'s job; CLI flags feed `BuildConfig` and `default_pipeline` branches on it. -- **P5.7 — DELETE all legacy.** Remove the `#compiler` attribute (parse + lower), the `compiler_call` IR op - (`inst.zig` + every switch arm + the `interp.zig:1130` dispatch), `compiler_hooks.zig` - (`HookFn`/`Registry`/all hooks). Make `-Dcomptime-flat` permanent (VM always) and **delete `interp.zig`** - (`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`/`last_bail_*`); drop the - `regToValue`/`valueToReg` bridge and the VM→legacy fallback in `emit_llvm` (`#run`/const-init) and - `comptime.zig` (type-fn / `#insert`) — a VM bail is now ALWAYS a build-gating diagnostic (4B wiring), never a - fallback. `core.invokeByFuncId` is already VM-only. Re-express `define`/`make_enum` as sx over the - compiler-API. Land the 0141 repro as a corpus test. Reconcile 1654 (asm-global at comptime) to the VM wording. -- **P5.8 — real-project validation (integration).** Build `~/projects/m3te` and `~/projects/distribution` with - the new pipeline end-to-end (their real bundle/codesign/target configs) — these are the acceptance test that - `default_pipeline` covers all targets. Fix gaps surfaced there. Add dedicated bundle smoke tests (min `.app` + - `.apk`) to the corpus (the bundler still has no `zig build test` coverage — the stream's top risk). - -**End state:** ONE evaluator (the VM); ZERO legacy; the entire build — emit, link, and all bundling + code -signing for macOS/iOS-device/iOS-sim/Android — is sx in `default_pipeline`, overridable via `#run on_build(...)`. -The compiler is: parse → IR → codegen → invoke `on_build`/`default_pipeline` on the VM (which calls back into -the linker primitive). `m3te` + `distribution` build clean. - -**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ; -FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F. -**Top risks:** (1) the bundler has no corpus guard (4E needs dedicated tests); (2) deleting -`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API (`abi(.compiler)`) touches the -whole build/bundle path — stage it behind real bundle builds; (3) S3's emit-skip relies on DCE dropping the -unreferenced compiler-domain declaration — verify no stray runtime reference keeps it alive (link error). - -## Open questions (resolve as reached, record decisions here) - -- **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed - records are host-laid-out; user comptime types are target-laid-out. The comptime - model must carry both regimes (a per-type ABI tag on layout queries). Confirm the - boundary where a comptime pointer to a compiler record is handed to host Zig code - uses host layout. -- **Exposing compiler types to sx.** Mechanism for projecting `types.zig` records into - the comptime type table with real offsets (the non-weld replacement) — a registry the - compiler owns, keyed by sx-visible name → real Zig type's layout + a host-call ABI. -- **Bytecode shape.** IR-derived vs a fresh ISA; register vs stack; fragment caching. -- **Pointer escape / lifetime.** Flat-memory pointers stored into the persistent type - table must be copied into compiler-owned memory at the boundary (as today). -- **Old-path retirement.** Keep the tagged interpreter until Phase 1 parity, then - delete — confirm no non-comptime consumer depends on `Value`. - -## File map (current → touched) - -| Area | File | Phase | -|------|------|-------| -| Comptime evaluator | `src/ir/interp.zig` | 0 (strip weld dispatch), 1–2 (rebuild) | -| Weld registry | `src/ir/compiler_lib.zig` | 0 (strip), 3 (replace with type/fn exposure) | -| Weld validation | `src/ir/lower/nominal.zig` | 0 (strip `validateWeldedStruct`) | -| Comptime-only gate | `src/backend/llvm/ops.zig` | 0 (re-derive without weld signal) | -| Host-FFI marshalling | `src/ir/host_ffi.zig` | 1 (struct-by-pointer trims it) | -| Metatype arms | `src/ir/interp.zig` (`defineStruct`/…/`reflectTypeInfo`) | 3 (delete, re-express in sx) | -| `#compiler` / BuildOptions | `library/modules/build.sx`, `src/ir/compiler_hooks.zig` | 3 (migrate, delete `#compiler`) | -| Surface syntax | `src/parser.zig`, `src/ast.zig` (`abi`/`extern`/`#library`) | kept; revisited Phase 3 | - -## Status - -- **Phase 0 — DONE (2026-06-17).** The struct-weld machinery is stripped: - `compiler_lib.zig` lost the type registry (`weldStruct`/`bound_types`/`BoundType`/ - `FieldLayout`/`findType`/`SxField`/`LayoutMismatch`/`validateStructLayout`); - `nominal.zig` lost `validateWeldedStruct`/`weldedFieldOrderStr` + the - `sd.abi == .zig` call; the struct-weld unit tests + examples `0625`/`0627`/`1183`/ - `1186` are removed. **Decision (recorded):** the `intern`/`text_of` function - host-call bridge is KEPT — it is a clean scalar dispatch (string→handle), not - weld/serialize/marshal, and is the seed Phase 3 grows the compiler-call path from. - So the `compiler_welded` dispatch (`interp.callExtern` is unchanged at HEAD — the - pre-branch in `call()`), `weldedCompilerFn` (decl.zig), the `emitCall` comptime-only - gate (ops.zig), and examples `0626`/`1184`/`1185` stay. The `#library`/`abi`/`extern` - SYNTAX stays. `zig build test` green (688 corpus, 0 failed; unit tests pass). -- **Phase 1 — in progress.** - - **Sub-step 1 — DONE.** `src/ir/comptime_vm.zig`: the comptime `Machine` - (linear byte memory + bump/stack allocator with `mark`/`reset` reclamation + - scalar `readWord`/`writeWord` (1/2/4/8, little-endian) + `bytes` views; addr 0 - reserved as `null_addr`) and `Frame` (register file indexed by Ref + stack - reclamation on `deinit`). A register `Reg` is a raw u64 — immediate scalar OR - `Addr`. Standalone + unit-tested (`comptime_vm.test.zig`, in the barrel); does - NOT touch the live interpreter, so the corpus stays green (688). No op execution - yet. - - **Sub-step 2 — DONE.** The executor (`Vm` in `comptime_vm.zig`): walks the SAME - IR `Inst` over comptime frames, mirroring the legacy interp's scalar semantics - (i64 wrapping/signed + f64 register words, keyed off the result/operand `TypeId`). - Ported: constants (`const_int`/`float`/`bool`/`null`/`undef`), arithmetic - (`add`/`sub`/`mul`/`div`/`mod`/`neg`), comparison (`cmp_*`), logical - (`bool_and`/`or`/`not`), conversions (`widen`/`narrow`/`bitcast` passthrough, - `int_to_float`/`float_to_int`), terminators (`br`/`cond_br`/`ret`/`ret_void`) and - `block_param` (branch args passed as Refs — the same frame persists, SSA-safe). - Any other op bails loudly (`error.Unsupported` + `detail = @tagName(op)`). - Unit-tested on hand-built IR (`Fb` builder): integer add, f64 arithmetic, cond_br - branch selection, a block-param loop summing i..1, div-by-zero + unsupported-op - bails. Corpus untouched (688 green) — the executor is exercised by unit tests only, - not yet wired to real comptime eval. - - **Sub-step 3 — DONE.** Memory + structs on comptime memory. `Vm` gained an optional - `table: *const TypeTable` (target-aware layout). Ported `alloca`/`load`/`store` - (over comptime addresses, `Store.val_ty` drives width) and `struct_init`/`struct_get`/ - `struct_gep` (structs laid out at the table's natural offsets). The value model: a - `Kind.word` (scalar/pointer ≤8B) sits in a register; a `Kind.aggregate` (struct) - lives in comptime memory and its "value" IS its address (read returns the address, - write memcpys), so nested structs compose and `struct_gep` is just base+offset (no - field-pointer dance). `kindOf` bails loudly on the not-yet-ported types - (slice/string/any/optional/enum/array/tuple/…). The Addr-based value model survives - allocator realloc (offsets are stable; slices are only materialized transiently). - Unit-tested: struct_init+get round-trip, alloca+gep+store+load, nested-struct - aggregate copy + nested read. Corpus untouched (688 green). - - **Sub-step 4a — DONE.** Tuples + arrays. `kindOf` widened (`tuple`/`array` → - aggregate). Ported `tuple_init`/`tuple_get` (positional, `tupleFieldOffset`), - `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over array/pointer/ - many_pointer bases; slice/string bases bail), and `length` on an array value - (static `ArrayInfo.length`). Unit-tested: mixed tuple round-trip, `[3]i64` - gep/store + index_get sum (42), array `length` (3). 688 corpus green. - - **Sub-step 4b — DONE.** Slices + strings as `{ptr@0 (pointer_size), len@8 (i64)}` - fat pointers (`kindOf`: string/slice → aggregate). Ported `const_string` (materializes - text+NUL in comptime memory + a fat pointer), `length`/`data_ptr` (read len/ptr fields), - `array_to_slice`, `subslice`, indexing *through* a slice/string (`elemAddr` loads - `.ptr` first), and `str_eq`/`str_ne` (len+memcmp). Helpers `makeSlice`/`sliceLen`/ - `sliceData`. Unit-tested: string length + str_eq/ne, array→slice + slice index + - slice length (23), array subslice (43). 688 corpus green. - - **Sub-step 4c — DONE (optionals + payloadless enums).** `kindOf`: `enum` → word; - `?T` → word if pointer-child (null==0) else `{T@0, i1@sizeof(T)}` aggregate. Ported - `optional_wrap`/`unwrap`/`has_value`/`coalesce` (with `optChildIsPtr`/`optHas` - helpers; `const_null` → `null_addr` reads as none), `enum_init` (payloadless: tag is - the value), `enum_tag` (payloadless/word). Unit-tested: non-pointer `?i64` - wrap/unwrap/coalesce (91), pointer `?*i64` null==0 (99), payloadless enum tag (11). - 688 corpus green. - - **Sub-step 4d — partial (`addr_of`/`deref` DONE).** `addr_of` passes through (an - aggregate value already IS its address; a pointer is already an address — mirrors - the legacy); `deref` = `readField` through the pointer (`ins.ty` is the pointee). - Unit-tested (deref a `*i64` → 77; addr_of a struct value + field read → 80). - **Deferred to the wiring phase (intentionally, not ported blind):** tagged-union - payload (`enum_init` w/ payload, `enum_payload` — the legacy stores *untyped* Values - and `field_index` indexes payload sub-fields, not variants, so a byte model's - payload type is ambiguous without a real call site), `any` boxing, closures, and the - bitwise ops. These have subtleties best resolved against actual corpus cases — the - VM's loud `error.Unsupported` + `detail` will name exactly what each real eval needs. - - - **Sub-step 1.5 — direct `call` DONE.** `Vm` gained `module: *const Module` - (resolves a callee `FuncId`) + a `depth`/`max_depth` recursion guard. `call` - marshals arg Refs → Reg words and recursively `run`s the callee; aggregate args/ - results pass as their `Addr` over the SHARED comptime memory (no copy). **Stack-lifetime - change:** `Frame` no longer reclaims the machine on exit (a returned aggregate's - Addr would dangle) — a comptime eval's allocations live to `Vm.deinit`; - `Machine.mark`/`reset` stay for explicit use. Extern/builtin callees (no blocks) - bail loudly (1.5b). Unit-tested: direct call (`add(20,22)+100` → 142) and recursion - (`sum(0..n)` → 15/55). 688 corpus green. - - **Sub-step 1.5b — `Reg`↔`Value` boundary bridge DONE.** The builtin/`compiler_call`/ - extern handlers are all coupled to the legacy `Interpreter` (e.g. `compiler_lib` - handlers take `*Interpreter`), so the VM can't call them directly — the wiring uses - WHOLE-FUNCTION fallback instead (VM runs pure functions; a bail re-runs the whole - eval in the legacy). That needs the boundary bridge: `valueToReg` (host `Value` arg → - VM `Reg`, materializing aggregates into comptime memory) + `regToValue` (VM result → - `Value`, deep-copied out). Covers scalars + strings + structs (other aggregate shapes - bail loudly; added as wiring surfaces them). Transitional — deleted once the VM owns - comptime end-to-end. Unit-tested with round-trips. 688 corpus green. - - **Then the wiring step** (below) — now unblocked. - -### Decision (2026-06-17): pivot from blind op-porting to CALLS + hybrid wiring -The common leaf ops are ported (scalars, control flow, structs, tuples, arrays, slices, -strings, optionals, payloadless enums, deref/addr_of) and unit-tested. Continuing to -port the rarer ops (tagged-union payload, any, closures) in isolation risks subtle -bugs and has low signal. The higher-value path: -1. **Calls (sub-step 1.5)** — `call` (direct), then `call_builtin`/`compiler_call`. The - shared comptime memory makes aggregate args/results pass naturally (they're Addrs). The - one design point: **aggregate-return lifetime** — a callee's stack-reclaim would - dangle a returned struct Addr, so for comptime (bounded) the VM should stop - reclaiming per-frame and let the whole eval's allocations live until `Vm.deinit` - (keep `Machine.mark/reset` for explicit use; drop it from `Frame.deinit`). -2. **Hybrid wiring** — `-Dcomptime-flat` routes a comptime eval through the VM, falling - back to the legacy interp on `error.Unsupported`. This makes the VM run the REAL - corpus, proving parity incrementally and surfacing exactly which ops each real eval - needs — far better signal than more isolated unit tests. diff --git a/current/PLAN-EXTERN-EXPORT.md b/current/PLAN-EXTERN-EXPORT.md deleted file mode 100644 index 70a34445..00000000 --- a/current/PLAN-EXTERN-EXPORT.md +++ /dev/null @@ -1,207 +0,0 @@ -# sx `extern` / `export` + `#foreign` retirement — Plan (FFI-linkage stream) - -**One stream, two parts.** **Part A** adds `extern`/`export` (the linkage surface); -**Part B** migrates every `#foreign` onto it and purges `foreign` from the tree. -They are *one* plan: Part B can't start until Part A is a behavior-equivalent -superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant. - -**Design rationale:** [design/inline-asm-design.md](../design/inline-asm-design.md) §II.2 -(Deviation 6) + §II.10 #4 + the syntax evaluation. - -**Decided syntax** -```sx -name :: (sig) -> Ret [callconv(.x)] [extern | export] [LIB] ["csym"] [;|{…}]; // functions -Name :: #objc_class("X") [extern | export] { … }; // aggregates (mirrors `struct #compiler`) -g : Type extern [LIB] ["csym"]; // extern global -``` -- `extern` = import (no body, external linkage, C ABI, no sx ctx) — `#foreign`'s role. -- `export` = define **and** expose (body + external linkage + C ABI + no ctx) — **new**. -- `extern`/`export` imply `callconv(.c)`; write `callconv` only to override. -- Optional `LIB` (a `#library` alias) + `"csym"` rename mirror `#foreign LIB "csym"`, - so `extern` is a true `#foreign` **superset** (Gate A→B): carried on - `extern_lib`/`extern_name`. The `#library` declaration + build-flag linking - mechanism stays a separate axis — `extern` *references* a lib, it doesn't fold - in `#library` itself. (Revises the original "library fully separate" decision 4.) - -> **END-STATE INVARIANT (hard requirement).** After this stream, `foreign` appears -> **nowhere** in the live tree — not the `#foreign` surface, and **not** internal -> identifiers. The extern AST is **not** named `foreign_expr`. Enforced by the -> Phase 9.4 grep gate. Scope today: 643 `foreign` lines / ~57 identifiers in `src/` -> + 28 in live docs — most of it the objc/jni **runtime-class** machinery. - -**Naming constraint (so we can actually reach the invariant):** introduce -`extern`-named representations only — do **not** reuse or extend -`ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Carry extern/export on a new -`FnDecl.extern_export` modifier with a `;`/`{…}` body (so there is **no** `*_expr` -node for it) + `FnDecl.extern_lib`/`extern_name`; add `VarDecl.is_extern`/ -`extern_lib`/`extern_name`. The IR is already extern-named (`Function.is_extern`, -`Builder.declareExtern`). - -**Key finding (scopes Part A):** the IR + LLVM emit **already support everything** — -`Function.linkage` (`.external/.internal/.private`), `is_extern`, `call_conv`, and a -raw un-mangled symbol name are all emitted by `declareFunction` -(`emit_llvm.zig:1225-1300`). Part A is a **parser + lowering** job, no codegen change. - -## Cadence (IMPASSIBLE) -No commit may both add a test AND make it pass (xfail-then-green, or a behavior-lock). -`zig build && zig build test` after every step. Never regenerate snapshots while red. - ---- - -# PART A — add `extern` / `export` (alongside `#foreign`) - -## Phase 0 — tokens + parser plumbing -| Step | Commit | What | Files | -|---|---|---|---| -| 0.0 | lock | add `kw_extern`, `kw_export` (Tag enum + `StaticStringMap`, beside `kw_callconv` at `token.zig:45,282`); unit lex test | `src/token.zig` | -| 0.1 | lock | `parseOptionalExternExport()` (mirror `parseOptionalCallConv`, `parser.zig:3669`) + `ast.ExternExportModifier` enum + `FnDecl.extern_export` + `VarDecl.is_extern`/`extern_name` fields; **not yet consumed**; unit AST test | `src/parser.zig`, `src/ast.zig` | - -## Phase 1 — `extern` (import; equivalent to lib-less `#foreign`) -| Step | Commit | What | Files | -|---|---|---|---| -| 1.0 | xfail | accept postfix `extern` after the callconv slot (`parser.zig:1950`); `examples/12xx-ffi-extern-fn.sx` extern-binds a libc symbol — red (lowering not wired) | `src/parser.zig` | -| 1.1 | green | lowering: `extern` ⇒ `is_extern`, `.external`, `callconv(.c)`, no ctx — route through `declareExtern` like a lib-less `#foreign` (anchors `decl.zig:1123,387,2110,2113`). Example green | `src/ir/lower/decl.zig` | -| 1.2 | green | optional `extern "csym"` rename + extern-global form `g : T extern;` (`parser.zig:425` path) | `src/parser.zig`, `src/ir/lower/decl.zig` | - -## Phase 2 — `export` (define + expose; the NEW capability) -Fills the four export-gap conditions (all in `src/ir/lower/decl.zig`): -| Gap | Anchor | Fix | -|---|---|---| -| (i) linkage forced `.internal` | `:2382`, `:2514` | also `.external` when `extern_export == .export` | -| (ii) C ABI not promoted | `:2110` | also `.c` when `== .export` | -| (iii) no symbol-name override | `emit_llvm.zig:1226` raw name | parse optional `export "csym"`; map in the name map | -| (iv) ctx param not suppressed | `:387` `funcWantsImplicitCtx` | also suppress when `== .export` | - -| Step | Commit | What | Files | -|---|---|---|---| -| 2.0 | xfail | multi-file test: an `export fn` called from a companion `.c` caller (same `XXXX-` prefix) — red (still internal) | `examples/12xx-ffi-export-fn.{sx,c}` + `expected/` | -| 2.1 | green | gaps (i),(ii),(iv): `export` ⇒ external + C-ABI + no-ctx on a **defined** fn (uses `beginFunction`, not `declareExtern`) | `src/ir/lower/decl.zig` | -| 2.2 | green | gap (iii): `export "csym"` symbol-name override | `src/parser.zig`, `src/ir/lower/decl.zig` | - -## Phase 3 — aggregates (objc / jni runtime classes) -| Step | Commit | What | Files | -|---|---|---|---| -| 3.0 | xfail | `#objc_class("X") extern { … }` (import) + `… export { … }` (define) parse alongside legacy `#foreign #objc_class` | `src/parser.zig` (`tryParseForeignClassPrefix` :1305, `parseForeignClassDecl` :1369) | -| 3.1 | green | map postfix `extern`→reference, `export`→define+register; per-runtime tests (objc, jni) | `src/parser.zig`, `src/ir/lower/decl.zig`, `src/ir/lower/objc_class.zig` | - -## Phase 4 — interplay, diagnostics, docs -`extern`+`callconv` stacking/redundancy; reject `extern`+`export` together; -`specs.md` documents `extern`/`export` (the three axes); `#foreign` still documented -until Part B cutover. - -> **GATE A→B.** `extern`/`export` are a behavior-equivalent **superset** of -> `#foreign`. Lock with a unit test asserting `#foreign` and `extern` lower to -> identical IR for a sample fn / global / class. Do not start Part B before this. - ---- - -# PART B — migrate `#foreign` → `extern`/`export`, then purge `foreign` - -**Inventory (drives the batches):** `#foreign` = 466 uses. ~391 sx-code (308 fns -[207 lib / 196 rename], 75 classes [39 objc / 31 jni], 8 globals) + ~145 example -snapshots. 6 libs (`sqlib`98 `libc`61 `objc`22 `tlib`12 `raylib`7 `clib/pcaplib`3). -Hotspots: `vendors/sqlite`(98), `platform/{android,uikit,android_jni,sdl3}`, -`std/{socket,thread,fs,time}`, `ffi/{objc,raylib}`. - -## Phase 5 — `#foreign` becomes an alias for `extern` -| Step | Commit | What | Files | -|---|---|---|---| -| 5.0 | lock | route the `#foreign` parser paths (`parser.zig:316,425,1305,1970`) to build the *same extern-named* AST as `extern`/`export`. Suite green, snapshots unchanged | `src/parser.zig` | -| 5.1 | lock | unit test: `#foreign` and `extern` produce identical IR (fn/global/class) | `src/ir/lower/decl.test.zig` | - -## Phase 6 — migrate stdlib (behavior-preserving; snapshot diff must be EMPTY) -One commit per batch; rewrite `#foreign`→`extern` (fns/globals), -`#foreign #objc_class`→`#objc_class … extern`, defined classes → `… export`. -| Step | Batch | ~sites | -|---|---|---| -| 6.1 | `library/vendors/sqlite/` | 98 | -| 6.2 | `library/modules/platform/` (uikit/android/android_jni/sdl3) | ~95 | -| 6.3 | `library/modules/std/` (socket/thread/fs/time/process/…) | ~60 | -| 6.4 | `library/modules/ffi/` (objc/raylib/objc_block/…) | ~50 | -| 6.5 | remaining `library/` + `vendors/` | remainder | - -## Phase 7 — migrate examples + issues (empty snapshot diff; review every diff) -| Step | Batch | -|---|---| -| 7.1 | `examples/12xx-ffi-*` (plain C) | -| 7.2 | `examples/13xx-ffi-objc-*` | -| 7.3 | `examples/14xx-ffi-jni-*` | -| 7.4 | `issues/*` repros + stragglers | -A non-empty diff ⇒ the alias wasn't behavior-equivalent — stop, fix Phase 5. - -## Phase 8 — cutover -| Step | Commit | What | -|---|---|---| -| 8.0 | xfail | `examples/11xx-diagnostics-foreign-removed.sx` expects a "`#foreign` removed; use `extern`/`export`" diagnostic — still accepted (red) | -| 8.1 | green | parser hard-rejects `#foreign` (mirrors the variadic `name: ..T` cutover); `specs.md` drops `#foreign`, documents `extern`/`export` | - -## Phase 9 — total `foreign` purge (the invariant) -`foreign` must not appear anywhere in the live tree, surface *or* internal. Each step -a mechanical, behavior-preserving rename commit (snapshots unchanged), small -per-file/subsystem commits — not one sweep. -| Step | What | Identifiers (count → new) | -|---|---|---| -| 9.0 | delete the surface | `hash_foreign`(11) + lexer entry + the 4 parse paths + the alias | -| 9.1 | rename **linkage** → `extern*` | `foreign_expr`(25) **eliminated** (folds into modifier) · `is_foreign`(39)→`is_extern` · `foreign_lib`/`foreign_name`→`extern_*` · `foreign_name_map`→`extern_name_map` · `callForeign`(8)→`callExtern` · `marshalForeignArg`→`marshalExternArg` · `is_foreign_c_api`(5)→`is_extern_c_api` · `dedupeForeignSymbol`→`dedupeExternSymbol` | -| 9.2 | rename **runtime-class** machinery → `runtime*` (decision 5) | `ForeignClassDecl`(65) · `ForeignMethodDecl`(31) · `ForeignClassMember`(20) · `ForeignFieldDecl`(15) · `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `foreign_path`(62) · `ForeignRuntime` · `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` · `findForeign{Method,Property}InChain` · `resolveForeign*` · `register*ForeignClass*` · `foreignClass*Type` · `*ForeignRefs` | -| 9.3 | purge **live docs** (28 lines) | `specs.md`/`readme.md`/`CLAUDE.md`: drop `#foreign`, document `extern`/`export`; fix file-roles + FFI/bundling notes | -| 9.4 | **acceptance gate** | `grep -rniE 'foreign' src/ library/ examples/ specs.md readme.md CLAUDE.md` → **0** | - ---- - -## Open decisions -*Part A (ratified — recommendations stand):* 1. bare keywords (not `#extern`). -2. aggregate position postfix (`#objc_class(…) extern`, like `struct #compiler`). -3. `extern ⇒ callconv(.c)`. 4. **REVISED** (user, 2026-06-14): `extern` carries an -optional `LIB`+`"csym"` axis (`extern_lib`/`extern_name`), mirroring `#foreign LIB -"csym"`, so it's a true `#foreign` superset (Gate A→B). The `#library` declaration + -build-flag linking mechanism stays separate — `extern` references a lib, doesn't -fold in `#library`. (Was: "library fully separate / not on `extern`".) -*Part B:* 5. runtime-class rename target — **RATIFIED `Runtime*Class*`** (user, 2026-06-14; -it's the object-model axis, not linkage). 6. historical carve-out — **STILL OPEN** (user did -not confirm at the Part A milestone): keep `issues/*.md` (+ design-doc prose) as provenance & -gate only the live tree (recommended) vs purge everything. Confirm 6 before Phase 9. - -## Relationship to ASM -`PLAN-ASM.md` Phase F (global asm) consumes `extern` (import the asm symbol) and -`export` (let asm call back into sx) — do it after **Part A Phase 2**. - ---- - -## Kickoff prompt (paste into a fresh session to start Part A) - -> Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint -> `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided -> syntax, Naming constraint, Key finding) and Part A; rationale is in -> `design/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4. -> -> **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix -> keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays -> untouched). Do NOT start Phase 2 (`export`) or Part B (migration). -> -> **Cadence (IMPASSIBLE):** no commit may both add a test and make it pass — lock -> behavior with a passing test, or land an xfail the next commit turns green. -> `zig build && zig build test` after every step. -> -> **Naming constraint (hard):** introduce only `extern`-named AST — do NOT reuse or -> extend `ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Use a new -> `FnDecl.extern_export` modifier (body `;` or `{…}`) and `VarDecl.is_extern`/ -> `extern_name`. IR is already extern-named (`Function.is_extern`, `declareExtern`). -> -> Steps (commit after each; update the checkpoint each time): -> - 0.0 lock: `kw_extern`/`kw_export` tokens + map entries beside `kw_callconv` -> (`src/token.zig:45,282`) + unit lex test. -> - 0.1 lock: `parseOptionalExternExport()` (mirror `parseOptionalCallConv`, -> `parser.zig:3669`) + `ast.ExternExportModifier` + `FnDecl.extern_export` + -> `VarDecl.is_extern`/`extern_name` (parsed, unconsumed) + unit AST test. -> - 1.0 xfail: accept postfix `extern` after the callconv slot (`parser.zig:1950`); -> add `examples/12xx-ffi-extern-fn.sx` that extern-binds a libc symbol (red). -> - 1.1 green: in `src/ir/lower/decl.zig`, lower `extern` like a lib-less `#foreign` -> import — `is_extern`, `.external`, `callconv(.c)`, no ctx, via `declareExtern` -> (anchors :1123, :387, :2110, :2113). Example goes green. -> - 1.2 green: optional `extern "csym"` rename + extern-global `g : T extern;` -> (`parser.zig:425`). -> -> Stop at end of Phase 1. Verify: suite green; the `extern` libc binding runs; -> `#foreign` still works with no snapshot diffs. If you hit an unrelated compiler -> bug, follow the CLAUDE.md IMPASSIBLE RULE (file an issue, stop). diff --git a/current/PLAN-FIBERS.md b/current/PLAN-FIBERS.md deleted file mode 100644 index 108d8457..00000000 --- a/current/PLAN-FIBERS.md +++ /dev/null @@ -1,284 +0,0 @@ -> **SUPERSEDED (2026-06-28).** The fiber-async layer described here — `sched.go` / -> `wait` / `cancel` over `Task($R)` (B1.4a) — was RETIRED and folded into the unified -> `context.io` async stack (`async` / `await` / `cancel` / `race`) in PLAN-IO-UNIFY -> Phase 5. See `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for the current -> design. The `Scheduler` ENGINE primitives this doc documents (swap_context, guarded -> mmap stacks, spawn / run / yield_now / suspend_self / wake / sleep / block_on_fd, -> virtual-time timers) REMAIN and are still accurate. Body below is a historical record. - -# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler) - -> **STATUS: ✅ COMPLETE.** The pure-sx M:1 async runtime is feature-complete end-to-end -> (`library/modules/std/sched.sx`, examples 1800–1817, suite 755/0): `abi(.naked)` context switch -> (aarch64 + x86_64/Win64), M:1 scheduler, suspending `go`/`wait`/`cancel`, deterministic virtual-time -> timers (`sleep`/`now_ms`), and real fd readiness via kqueue (`block_on_fd`). Five compiler bugs fixed -> en route (0151/0152/0153/0154/0156-P1/0157). Deferred non-blocking follow-ups: linux `epoll` twin, -> `Scheduler.deinit`, `Future(void)`/`timeout` (0150), `context.io`-routed async (M:N). Next carve: -> Stream B2 (channels / cancel / async stdlib). Historical step-status below. -> -> B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2 -> (`Io` interface + `async`/`await`/`cancel` over blocking `CBlockingIo`) ✅ · B1.3 (fiber -> runtime: naked `swap_context` + §10.7 stress gate + guarded `mmap` stacks, proven on aarch64 -> AND x86_64/Win64) ✅ · **B1.5a (M:1 scheduler CORE — `std/sched.sx`: `spawn`/`yield_now`/ -> `suspend_self`/`wake`/`run`) ✅** (fixed blocker 0154) · **B1.4a (suspending fiber-task async — -> `sched.go`/`wait`/`cancel` over `Task($R)`, nullary-thunk) ✅** (adversarially reviewed; fixed -> blockers 0156-Part1 + 0157 en route; locked `1813`). -> **B1.4b (deterministic virtual-time timers — sched.sleep/now_ms/timer-run) ✅** (reviewed; fixed a CRITICAL timer-vs-early-wake UAF; locked 1814/1815). -> **B1.4c (event-loop — real fd readiness via kqueue: `block_on_fd` + run-loop Mode 2) ✅** (reviewed; fixed a CRITICAL same-fd lost-wakeup hang; locked 1816). macOS only — linux epoll twin deferred. -> **B1.5 (end-to-end M:1 capstone — `go`/`wait`+`sleep`+scheduler, deterministic ordering) ✅** (locked 1817). **STREAM B1 COMPLETE.** Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async + -> deterministic timers live as `sched.*` methods (M:1), NOT routed through the erased `context.io` (avoids forcing sched.sx into every std consumer + the `_fib_tramp` dup-symbol -> trap); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready` stay reserved for M:N. Deferred: -> issue 0150 (`Future(void)`/`timeout`); 0156-Part2 (deferred `..` spread); the `::` callable-param -> feature. - -Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream B (§B1) + the -design-of-record [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) -§4 (async), §7 steps 4–9, §8.1 (risks), §10 (testing). Progress in -[CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). Stream B2 (channels/cancel/stdlib) is a -separate carve ([PLAN-CHANNELS.md], when reached) and depends on this + atomics (✅). - -**Goal:** the colorblind, stackful, **pure-sx** async runtime — fibers behind an `Io` -interface, an M:1 scheduler, blocking + deterministic-sim + event-loop `Io` impls. The -**compiler floor is small and net-new**: make `abi(.naked)` actually emit an LLVM `naked` -function (B1.0), and confirm/close the per-fiber `context` root (B1.1). **Everything -else — the context-switch asm, fiber bootstrap, `mmap` stacks, the scheduler, futures, -the `Io` vtables — is ordinary sx library code** (design §4, §4.4). The irreducible FFI -floor: the per-arch asm context-switch (in `.sx`), syscall `extern`s, and `mmap`. - -**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then -flip to green); `zig build && zig build test` green after every step; never regen snapshots -while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff. -New corpus category: `18xx` concurrency. On an **unrelated** compiler bug → file -`issues/NNNN`, mark this checkpoint BLOCKED, STOP (CLAUDE.md). The in-session -worker-fix override (delegate a blocker to a worker) applies only with explicit user -authorization. - ---- - -## Design (grounded against the tree) - -### B1.0 — `abi(.naked)` codegen (the one genuinely net-new compiler piece in B1) - -The design doc spells this `callconv(.naked)`; the **real sx surface is `abi(.naked)`** — -written in the postfix slot, `name :: (sig) -> Ret abi(.naked) { asm { … }; }` (cf. -`build_options :: () -> BuildOptions abi(.compiler);` in [build.sx:28](../library/modules/build.sx#L28)). -The sx-facing name is **`naked`** throughout (keyword, field `is_naked`, diagnostics) — -matching LLVM's `naked` attribute (the lowering mechanism) and the industry term -(Zig/Rust/GCC/Clang). The ABI variant was renamed `.pure → .naked`: "pure" universally -means *side-effect-free*, the opposite of a register-clobbering context switch. - -**Grounding (verified — do not re-derive):** -- The `ABI` enum **already carries `.naked`** — `ABI = enum { default, c, compiler, naked }` - ([ast.zig:142](../src/ast.zig#L142)), documented "naked function (inline asm - body), no calling-convention prologue/epilogue." So B1.0 is **NOT** "extend the enum." -- `.naked` is **inert today**: [type_resolver.zig:237](../src/ir/type_resolver.zig#L237) - maps `.compiler, .naked → .default` CC, and `emit_llvm` emits **no LLVM `naked` - attribute**. So the net-new work is exactly: **carry `abi == .naked` into the IR - `Function`, emit LLVM's `naked` attr, and skip the implicit-`Context` / prologue - lowering** so the body is just the asm block + its own `ret`. -- The IR `Function` struct ([inst.zig:605](../src/ir/inst.zig#L605)) carries `call_conv` - (default/c) + `is_compiler_domain`, but **no naked flag** — add one (`is_naked: bool`). -- Attribute API is in-tree: `nounwind` is set at - [emit_llvm.zig:1339](../src/ir/emit_llvm.zig#L1339) via - `LLVMGetEnumAttributeKindForName("nounwind", 8)` → `LLVMCreateEnumAttribute(ctx, id, 0)` - → `LLVMAddAttributeAtIndex(func, func_idx_attr /* -1 */, attr)`. The LLVM `naked` attr - is the same shape: `LLVMGetEnumAttributeKindForName("naked", 5)`. -- The `.c` ABI **already skips the implicit ctx** at lowering — `lam.abi == .c` / - `fd.abi == .c` gates (closure.zig:171, [decl.zig:515](../src/ir/lower/decl.zig#L515)). - `.naked` must skip it **too** (a `.naked` fn gets no synthetic `__sx_ctx`, no stack frame, - no prologue — args arrive in ABI registers and are read directly from asm). The - implicit-return machinery (`lowerValueBody`) must also be bypassed: a `.naked` body has no - sx return (the asm rets itself), so lower its statements and cap the block with - `unreachable`. -- **Inline asm already works end-to-end** (lower→emit→JIT): aarch64 - ([examples/1645](../examples/1645-platform-asm-aarch64-add.sx)), x86_64 - ([examples/1651](../examples/1651-platform-asm-x86-syscall-write.sx)), global asm, JIT - ([1653](../examples/1653-platform-asm-global-jit.sx)). `emitInlineAsm` / - `LLVMGetInlineAsm` at [ops.zig:915](../src/backend/llvm/ops.zig#L915). The `.naked` body - is a single asm block reusing this path. - -**`.naked` ≠ `.c` (design §4.6 context-switch note):** a `.c` epilogue restores SP from the -frame; a context switch deliberately makes SP-in ≠ SP-out, so the `.c` epilogue would -restore from the *wrong* stack. `.naked` = no prologue/epilogue/frame — the asm emits its -own `ret`. This is *why* the switch must be `.naked`, not `.c`. - -**Snapshot story (per the atomics precedent):** a `.naked` fn's *body is raw per-arch asm* -(it can't be portable — that's the point), while LLVM's `naked` attribute text is -arch-invariant. **B1.0a** (lock) needs only **one host example** locked to the emit bail — -the bail fires at the function level *before* any asm/instruction selection, so it is -host-independent (no `.build` target pin). **B1.0b** (green) adds emission, pins that -example aarch64 (`.build {"target": "aarch64-macos"}`, end-to-end on a matching host, -ir-only on a mismatch), and adds an x86_64 cross sibling — mirroring the existing asm -corpus split (1645 aarch64 / 1651 x86). The ir-only `.ir` (only producible once emission -lands in B1.0b) asserts the `naked` attribute + the asm body. State loudly: **the `.ir` -proves the `naked` keyword + asm emitted, NOT that any hand-written register save/restore -is correct** — that is the B1.3 switch-stress harness's job, never the corpus's. - -### B1.1 — per-fiber `context` root (grounding says this is SMALL, likely library-only) - -**Grounding (verified — closes the design doc's open sizing question):** -- `context` is an **implicit `*Context` parameter** (`__sx_ctx`, slot 0), threaded through - every default-conv sx call ([lower.zig:259](../src/ir/lower.zig#L259)) — **not raw TLS**. - Inside a function `current_ctx_ref = Ref.fromIndex(0)` (the param) → it **rides the fiber - stack frame for free**. -- `push Context.{…}` allocates the new `Context` with a **stack `alloca`** and rebinds - `current_ctx_ref` to that slot ([stmt.zig:1263](../src/ir/lower/stmt.zig#L1263)) — "No - global, no walk." So **push frames are fiber-local for free**. -- The **only shared root** is the `__sx_default_context` **global**, bound at - entry-points / `abi(.c)` fns *before any user code runs* - ([decl.zig:2667](../src/ir/lower/decl.zig#L2667), :2815). - -⇒ The design doc's "lower as swappable indirection, never raw TLS" guards a **non-problem** -(confirmed). The **real, now-sized** B1.1 work is purely a **library convention**: a -freshly-`spawn`ed fiber must take its root `Context` from the **spawner's snapshot** (passed -as the fiber-entry fn's `__sx_ctx` slot-0 arg by the spawn trampoline), **not** the -`__sx_default_context` global. That is sx-side (the trampoline already controls slot 0) — -**expected to be ZERO compiler change.** B1.1's first action is a probe confirming this; if -a fiber genuinely re-reads the global root mid-stack (it should not — entry binds once), -*then* and only then is there a compiler obligation. **Ground the probe before sizing any -compiler work.** Prerequisite of B1.3 (a fiber needs a valid root before it switches). - -### B1.2–B1.5 — pure sx over the primitives (design §4) -- **B1.2 (A1):** `Io` interface + `context.io` + `Future` + `cancel()` — a protocol/vtable - threaded exactly like `Allocator` (which already lives at `Context` field 0; see - `allocViaContext` [call.zig:1214](../src/ir/lower/call.zig#L1214)). `Io` becomes another - `Context` field. No compiler change — protocols + context already carry it. -- **B1.3 (A2):** the fiber runtime — naked context-switch asm (per-arch), bootstrap, `mmap` - stacks **with mandatory guard pages**. All sx. **Highest corruption risk in the stream** - (§8.1.1) and **untestable by the deterministic `Io`** (which tests *scheduling*, not the - *switch*). Its **first deliverable, before the scheduler AND the deterministic `Io`**: a - standalone **2-fiber ping-pong switch-stress harness** (§10.7) — scribble every - callee-saved register + a stack canary before each suspend, deep/recursive chains, verify - all survive post-resume. This harness — not B1.4 — is A2's correctness gate. -- **B1.4 (A3):** `Io` impls in order **blocking → deterministic-sim (KEYSTONE) → event-loop** - (kqueue/epoll/io_uring). Build the deterministic `Io` right after blocking; **calibrate it - against blocking `Io`** before trusting it to gate everything async (§8.1.3, §10.7) — a - deterministic-but-wrong scheduler snapshots garbage. (Open, deferred: the event loop does - **not** yet cooperate with a platform UI run loop — CFRunLoop/ALooper; that's a §6 - app-target gap, out of B1.) -- **B1.5 (A5·M:1):** the single-thread scheduler — validates the whole colorblind stack - end-to-end. `18xx` corpus runs under the deterministic `Io`, asserting a **program-emitted - ordering contract** (sequence markers), not raw interleaving, so scheduler-policy tweaks - don't churn every snapshot. - -### Files the compiler floor touches (B1.0 only; B1.1–B1.5 are library + tests) -B1.0 (`.naked`) forces these plumbing sites: -- [ast.zig:142](../src/ast.zig#L142) — `ABI.naked` (exists; reference only). -- [inst.zig:605](../src/ir/inst.zig#L605) — add `is_naked: bool = false` to `Function`. -- [decl.zig](../src/ir/lower/decl.zig) — set `is_naked` from `fd.abi == .naked`; gate the - implicit-ctx off for `.naked` in `funcWantsImplicitCtx` (mirror the `.c` skip at - decl.zig:515) and bypass `lowerValueBody` for `.naked` bodies (lower statements + cap with - `unreachable`, in both body-lowering paths) — a `.naked` fn binds no ctx and has no sx - return. -- [type_resolver.zig:237](../src/ir/type_resolver.zig#L237) — leave CC `.default` (a `.naked` - fn-pointer type has no CC of its own; nakedness is a decl-level emit attribute). -- [emit_llvm.zig:402](../src/ir/emit_llvm.zig#L402) Pass 2 — **B1.0a:** bail loudly when - `func.is_naked` (build-gating). **B1.0b:** instead emit LLVM's `naked` attr (shape per - `nounwind` at emit_llvm.zig:1339) + the asm-only body (no prologue). -- Any `.op`/`Function`-field switch the Zig build flags — let the build tell you. - ---- - -## Phases (xfail→green steps) - -### B1.0 — `abi(.naked)` codegen — ✅ COMPLETE -- **B1.0a (lock) — ✅ DONE.** Carried `abi == .naked` into IR `Function.is_naked`; threaded - through `decl.zig` (`funcWantsImplicitCtx` skips `.naked` like `.c`; all body-lowering paths - bypass `lowerValueBody` for `.naked`, lowering the asm body + capping with `unreachable`) + - generic.zig + pack.zig; `emit_llvm` Pass 2 bailed loudly on `func.is_naked`. Locked by - `examples/1800-concurrency-naked-asm.sx` + the generic regression (review-found gap). -- **B1.0b (green) — ✅ DONE.** `emit_llvm` declaration pass adds LLVM `naked` + `noinline` + - `nounwind` for `func.is_naked` and skips `frame-pointer=all` (incompatible with a frameless - function); Pass 2 emits the body normally (`naked` ⇒ verbatim asm + own `ret`, no - prologue). `1800` pinned aarch64 → exit 42 + `.ir`; `1801-concurrency-naked-generic.sx` - (renamed from `-bail`) proves the generic path emits a naked body (exit 42); - `1802-concurrency-naked-asm-x86.sx` x86_64 cross sibling (ir-only here, `.ir` locks `naked` - + `movl $42, %eax`). Unit test `emit: abi(.naked) function gets the naked attribute` asserts - `naked` present + `frame-pointer` absent. Suite green (724/0). -- **B1.0c (review-hardening) — ✅ DONE.** A param-bearing `.naked` fn emitted invalid LLVM - (loud verifier error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both - paths + generic.zig) so a naked fn's args stay in registers (read by the asm body) — this - *enables* B1.3's `swap_context(from, to)`. Locked by `1803-concurrency-naked-asm-param.sx`. - Pack `.naked` (variadic + naked, nonsensical) left unsupported → loud verifier error. - -### B1.1 — per-fiber `context` root — ✅ COMPLETE (zero compiler change) -Probe confirmed the spawn convention works with ordinary language features: snapshot -`context` (`snap := context`), store it in a struct, and `push f.root { entry(args) }` from a -trampoline running under a different ambient context — the body reads the snapshot (via the -implicit slot-0 `*Context` param), not the ambient ctx, and `push` restores ambient on exit. -No path re-reads `__sx_default_context` mid-stack ⇒ **no compiler obligation**; this is a pure -library convention. Locked by `examples/1804-concurrency-context-snapshot.sx` (`fiber root: -42` / `ambient after: 99`). The design doc's "never raw TLS" guarded a non-problem. - -### B1.2 — A1: `Io` interface + `context.io` + `Future` + `cancel()` API -Library-only. `Io` as a protocol added to `Context` (mirror `Allocator`). `Future`/`cancel` -API surface. xfail→green via an `18xx` example exercising the blocking `Io` default (real -suspend lands in B1.3). No compiler change expected; if a protocol-in-context gap appears, -file it. - -### B1.3 — A2: fiber runtime (naked switch + bootstrap + guarded `mmap` stacks) — ✅ COMPLETE -- **B1.3a (switch-stress harness FIRST) — ✅** the §10.7 register/canary-survival gate (1807/1808), - validity proven by negative controls, adversarially reviewed. -- **B1.3b — ✅** fiber bootstrap + guarded `mmap` stacks (1809); the x86_64 sibling landed as Win64 - on a real VM (1810, `0 0 P`). Switch proven on TWO arch/ABI pairs. - -### B1.5a — M:1 scheduler CORE (`std/sched.sx`) — ✅ COMPLETE -The reusable scheduler wrapping `swap_context`: generic `Fiber`/`Scheduler`, -`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over guarded `mmap` stacks, one generic -`fib_dispatch` running any stored closure body. Adversarially reviewed + hardened; fixed blocker -bug 0154 (struct-field `null`/`---` over-store) en route. Locked by `1811` (round-robin) + `1812` -(suspend/wake). Built BEFORE the deterministic `Io` because FiberIo (B1.4a) needs it as substrate. - -### B1.4a — suspending fiber-task async (`sched.go`/`wait`/`cancel`) — ✅ COMPLETE -`Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` in `sched.sx` (nullary-thunk; -self-contained). `go` spawns `work` as a fiber, `wait` parks the caller until it completes. Locked -by `1813`. Two compiler blockers fixed (0156-Part1, 0157) + adversarially reviewed/hardened. - -### B1.4b/c — A3: `Io` impls (deterministic-sim KEYSTONE → event-loop) -Blocking exists (io.sx `CBlockingIo`). Next the deterministic-sim `Io`, **calibrated against -blocking** before any `18xx` test trusts it; then the event loop. The deterministic `Io` is the -test harness for *all* of B1.5 + Stream B2. - -### B1.5 — A5: M:1 scheduler — ✅ COMPLETE -End-to-end validation of the colorblind stack. The `18xx` corpus asserts program-emitted ordering -contracts under the scheduler + deterministic timers; the capstone `1817` composes `go`/`wait` + -`sleep`/`now_ms` + the scheduler (three tasks complete in deadline order, deterministic sum). Stream -B1 is feature-complete. - ---- - -## Gates -- **B1.0:** unit `emit_llvm.test.zig` (the `naked` attr present on a `.naked` fn); two - arch-gated examples (aarch64 + x86_64) run end-to-end on a matching host, ir-only on a - mismatch (assert `naked` + asm in `.ir`). **OUT of corpus scope, stated loudly:** the - *correctness* of any hand-written register save/restore — that's the B1.3 stress harness. -- **B1.1:** an `18xx` example locking context-carried-by-slot-0 behavior + a checkpoint note - on the spawn-trampoline convention. -- **B1.3:** the **switch-stress harness is A2's gate** (register/canary survival — §10.7), - NOT a run/snapshot test; plus arch-gated run tests. -- **B1.4:** deterministic `Io` **calibrated** against blocking `Io` (§8.1.3) before trusting - it; `18xx` under the deterministic `Io`. -- **B1.5:** `18xx` ordering-contract snapshots under the deterministic `Io`. - -## Kickoff prompt (B1.0b — paste into a fresh session) -> Implement Stream B1 step **B1.0b** (`abi(.naked)` real emission) per -> `current/PLAN-FIBERS.md`. Verify `zig build && zig build test` is green first (B1.0a is -> already landed: `Function.is_naked` plumbed, `decl.zig` skips ctx + bypasses implicit-return -> for `.naked`, `emit_llvm` Pass 2 bails loudly, `examples/1800-concurrency-naked-asm.sx` -> locked to the bail). Then: (1) in `src/ir/emit_llvm.zig` Pass 2 (~line 402), REPLACE the -> `func.is_naked` bail with real emission — set LLVM's `naked` attribute on the function -> (`LLVMGetEnumAttributeKindForName("naked", 5)` → `LLVMCreateEnumAttribute(ctx, id, 0)` → -> `LLVMAddAttributeAtIndex(llvm_func, -1, attr)`; shape per the `nounwind` set at -> emit_llvm.zig:1339) and emit the `.naked` body as its asm block only, no prologue/epilogue -> (the body already lowers to the inline-asm op + an `unreachable` terminator). (2) Pin -> `examples/1800-concurrency-naked-asm.sx` aarch64 with a `.build` sidecar -> `{"target":"aarch64-macos"}`; on this aarch64 host it runs end-to-end (exit 42), capture -> `.ir` + regen (`-Dname=examples/1800-concurrency-naked-asm.sx -Dupdate-goldens`), review the -> diff (assert the `.ir` shows the `naked` attr + `mov x0, #42` / `ret`, NO stray error -> text). (3) Add `examples/1802-concurrency-naked-asm-x86.sx` (x86_64 body, `.build -> {"target":"x86_64-linux"}`, ir-only on this host — requires its `.ir`, now producible). -> (4) Add a unit test in `src/ir/emit_llvm.test.zig` asserting the `naked` attribute is -> present on an `abi(.naked)` function. Confirm `zig build test` green, commit. NOTE: the -> `.ir` proves the keyword + asm emitted, NOT register-save correctness (that's the B1.3 -> switch-stress harness). If you hit an UNRELATED compiler bug, file `issues/NNNN`, mark -> `CHECKPOINT-FIBERS.md` BLOCKED, and STOP. diff --git a/current/PLAN-METATYPE.md b/current/PLAN-METATYPE.md deleted file mode 100644 index b45b8e54..00000000 --- a/current/PLAN-METATYPE.md +++ /dev/null @@ -1,142 +0,0 @@ -# PLAN-METATYPE — comptime type metaprogramming (`declare` / `define` + reflection) - -## Goal - -Comptime type metaprogramming with the smallest possible compiler surface: - -- **`declare(name) -> Type`** — mint a NEW empty (undefined) nominal type NAMED - `name`, returned as a first-class `Type` handle. The compiler registers the - forward type at compile time, so the body can reference it (`*Name`). -- **`define(handle, info) -> Type`** — fill a declared handle's body from a - `TypeInfo` *value*, and return the handle (so the one-shot form chains). -- **`type_info($T) -> TypeInfo`** — reflect a type INTO data (the inverse of - `define`'s decode). *Done for enums* (`interp.zig:reflectTypeInfo`, - `examples/0619`); struct/tuple widening pending. -- **`field_type($T, i) -> Type`** — the i-th field / variant-payload / element - type of `$T`. *Done.* - -These four `#builtin`s in `library/modules/std/meta.sx` are the **entire** -compiler surface. Every higher-level constructor is **plain sx built over -`declare`/`define`** — the compiler knows none of them by name: - -```sx -// one-shot (non-recursive): declare + define chained, define returns the handle -T :: define(declare("T"), .enum(.{ variants = .[ … ] })); - -// recursive: a ctor fn names the forward type via declare, references it as *Name -List :: make_list(); -make_list :: () -> Type { - h := declare("List"); - return define(h, .enum(.{ variants = .[ - EnumVariant.{ name = "cons", payload = *List }, // self-reference - EnumVariant.{ name = "nil", payload = void } ] })); -} - -// type-fns are ordinary sx (channel result types, etc.) -RecvResult :: ($T: Type) -> Type { - return define(declare("RecvResult"), .enum(.{ variants = .[ - EnumVariant.{ name = "value", payload = T }, - EnumVariant.{ name = "closed", payload = void } ] })); -} -``` - -This gates channel result types (`RecvResult($T)`) and `race`'s synthesized -tagged-union (design [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §7 step 3), and replaces a would-be `enum($T)` language feature. - -## How it works (the locked design) - -1. **Two comptime interp builtins.** `declare` mints an empty `tagged_union` slot - in the type table; `define` decodes the `TypeInfo` value (variant-name strings + - payload `Type`-tags) and completes the slot byte-identical to a source enum's - `buildEnumInfo` output, so it flows through enum codegen unmodified. The interp - mutates the type table via a `mint` handle the host sets (`setMintTable`). -2. **No syntactic constructor recognition.** A `::` binding or type-fn body that - calls a `Type`-returning fn is **comptime-evaluated** (`evalComptimeType`): the - expression runs through the interpreter, the `declare`/`define` builtins mint the - type, and the result `type_tag` is bound. `decl.zig` triggers on a non-generic - `-> Type` fn call; `instantiateTypeFunction` triggers on a type-fn body that - returns a `define(…)` call (or a bodied `-> Type` helper) — see - `generic.zig:returnExprMintsType`. -3. **Name on `declare`.** `declare("Name")` carries the name as a compile-time - string so `preregisterForwardTypes` (in `evalComptimeType`) can register the - forward type — and bind it as a type alias — BEFORE the body lowers. That's - what makes a `*Name` self-reference resolve (a `Name :: ctor()` decl makes - `Name` a const_decl author, so `*Name` resolves through the forward-ALIAS path; - the alias binding, not just the table registration, is what satisfies it). The - interp's `declare` returns the same slot by name; `define` fills it in place. -4. **Nominal identity** rides the existing type-fn mangled-name instantiation cache: - `RecvResult(i64)` at two sites memoizes to ONE `TypeId` (the body runs once; - `renameNominalType` re-keys the minted type to the mangled name). -5. **Comptime-only, JIT-free.** `declare`/`define` are interp ops; reaching them at - runtime / emit is a hard error. -6. **Undefined-until-defined.** `declare()` mints an undefined slot; *using* it - (construct / match / size) before its `define` is a loud diagnostic. A *pointer* - to an undefined slot (`*Self`) is fine — that's what self-reference needs. - -## Key code anchors - -- Builtins: `BuiltinId.declare` / `.define` (`src/ir/inst.zig`); lowering to - `callBuiltin` (`src/ir/lower/call.zig:tryLowerReflectionCall`); interp exec + - `defineEnum` + `decodeVariantElements` (`src/ir/interp.zig`); `mint` field + - `setMintTable`. -- Comptime evaluation: `evalComptimeType` / `renameNominalType` - (`src/ir/lower/comptime.zig`); decl trigger `fnReturnsTypeValue` - (`src/ir/lower/decl.zig`); type-fn trigger `returnExprMintsType` + - `instantiateTypeFunction` (`src/ir/lower/generic.zig`). -- Reflection: `field_type` → `fieldTypeOf` (`src/ir/lower/generic.zig`). -- Surface: `library/modules/std/meta.sx` (on-demand import — NOT the prelude, to - avoid shifting every `.ir` snapshot). - -## Cadence (IMPASSIBLE) - -No commit may both add a test AND make it pass (xfail-then-green, or a behavior -lock). `zig build && zig build test` after every step. Never regenerate snapshots -while red. Examples: `06xx` (comptime), `11xx` (diagnostics). - -## Status - -- [x] `declare` / `define` comptime builtins + the `mint` plumbing. -- [x] Comptime evaluation of a `Type`-returning `::` RHS and type-fn body - (the only triggers; no constructor-name knowledge in the compiler). -- [x] Name-in-`TypeInfo`; nominal identity via the instantiation cache. -- [x] `field_type` reflection (`examples/0616`). -- [x] Examples green on the floor: `0614` (one-shot), `0615` (type-fn identity), - `0617` (channel result types). -- [x] **Self-reference** — recursive enums via `declare("Name")` + `*Name` in a - constructor fn (`preregisterForwardTypes` registers the forward type + alias - before the body lowers). `examples/0618` (recursive `*List`: construct, match - through the pointer, recursive traversal). Mutual recursion / by-value-self-ref - rejection fall out of the same mechanism (F5 adds the loud by-value check). -- [x] **`make_enum(name, variants: []EnumVariant)`** — the general enum constructor - over a COMPUTED (value, non-literal) variant list. Pure sx in `meta.sx`; - exercises `define` decoding a value-arg slice. `examples/0620` (array-literal - local) / `0624` (generic builder). -- [x] **Comptime slice over a non-string aggregate** — `arr[lo..hi]` over an array - yields a real slice value at comptime (`base_ty` threaded onto `Subslice`; - open-ended `hi` folded to the array's static length; `subsliceElements`). - `examples/0621`. -- [x] **`type_info($T) -> TypeInfo`** — reflect `enum`/`tagged_union`/`struct`/`tuple` - INTO a value (inverse of `define`'s decode); `define` decodes all three back - (`defineEnum`/`defineStruct`/`defineTuple`, dispatched on the TypeInfo tag). - Round-trips: `examples/0619` (enum) / `0622` (struct) / `0623` (tuple). The - reflect/construct triad is complete. -- [x] **Generic type-fn body locals** — a generic `($T) -> Type` comptime-evaluates - its FULL body (prelude statements + return), so a local before the return - resolves (`createComptimeFunctionWithPrelude` / `evalComptimeTypeBody`). - `examples/0624`. -- [x] **Validation + loud diagnostics** — by-value self-reference (`checkInfiniteSize`, - source `1178` + constructed `1182`; issue 0139), duplicate variant/field names - (`1180`), `declare()` never `define()`d (`1181`, was a `verifySizes` panic), - and the 0140 bail-surfacing (`1179`). use-before-define is subsumed by these - (no new check needed). Validation story COMPLETE. -- [ ] **Comptime `List` growth** (issue 0141, DEFERRED) — `List(T).append` at - comptime bails (two layers: null comptime allocator at scanDecls + `*T` - slot_ptr `struct_get`). Non-blocking; array-literal locals cover the use case. - -## Risks / watch - -- **Self-ref timing** — `define` for the two-statement form must complete before any - code uses the type's layout; a use-before-define must be a loud diagnostic, not a - silent empty enum. -- Keep `declare`/`define` **comptime-only**: reaching them at runtime is a hard error - (emit should bail loudly if one ever leaks into codegen). diff --git a/current/PLAN-MULTIRET.md b/current/PLAN-MULTIRET.md deleted file mode 100644 index f77822fc..00000000 --- a/current/PLAN-MULTIRET.md +++ /dev/null @@ -1,156 +0,0 @@ -# PLAN-MULTIRET — bare-paren multi-value returns + named returns - -## Why -sx already has multi-value returns, but only in a verbose spelling: -`-> Tuple(A, B)` / `-> Tuple(x: A, y: B)` types and `return .(a, b)` / -`return .(x = a, y = b)` tuple-literal returns. Destructuring (`a, b := f()`), -named/positional field access (`r.x` / `r.0`), and value-carrying failables -(`Tuple(A, B) !E`) all work on top of the existing `.tuple` TypeId. - -The user wants the ergonomic, canonical surface: - -```sx -a :: () -> () { } // () ≡ void -two :: () -> (i32, bool) { return 42, true; } // bare-paren type + bare comma return -b :: (f1: i32, f2: i32) -> (sum: i32, good: bool) { // named returns are in-scope locals - good = true; - sum = f1 + f2; // implicit return: all named slots set -} -b2 :: (f1: i32, f2: i32) -> (sum: i32, good: bool) { - return f1 + f2, f2 > 42; // bare comma return still works -} -read :: () -> (i32, bool, !) { ... } // error channel ALWAYS the last slot -``` - -Rules (from the user): -- **`() -> ()` ≡ `() -> void`.** -- **A multi-return signature is NOT a tuple — it just REUSES the tuple machinery.** - `-> (i32, bool)` / `-> (x: i32, y: bool)` mean "this function returns multiple - values", a DISTINCT thing from `-> Tuple(i32, bool)` (which returns one tuple - value). The bare-paren form is valid ONLY as a function/closure RETURN - signature — `x: (A, B)` (a variable/param/field annotation) stays REJECTED; - `Tuple(…)` is the spelling for an actual tuple value type. -- **Consumption — destructure OR single-bind (REVISED 2026-06-27).** A - multi-return result may be DESTRUCTURED (`s, g := b2()`) OR bound to a single - name and reached by field (`c := b2(); c.sum` / `c.0`). The earlier - destructure-only rule (single-bind = error) was REVERSED by the user — single - binding is allowed; the bound value behaves like a tuple of the value slots. - - **Failable: the error stays SEPARATE.** For `-> (sum, good, !)`, a bound - value (`c := f() catch …` / `try`) holds ONLY the value slots — the error - rides the `!` channel and is NEVER part of `c` (no `c.err`). This falls out of - the existing failable machinery (catch/try strip the error before binding). -- **Failable: the error channel is always the LAST slot** (`(A, B, !)`). -- **Bare comma return**: `return v1, v2;` maps positionally to the return slots — - no `.(…)` tuple literal needed. -- **Named returns are assignable locals.** With no explicit `return`, an implicit - return at end-of-body synthesizes the result from the named locals. **A named - return that is neither assigned on the path nor given a default is a COMPILE - ERROR.** A named slot may carry a default (`(sum: i32 = 0, good: bool)`); a - defaulted slot needn't be assigned. - -## Representation (how "not a tuple, reuse machinery" is realized) — AS BUILT -- A dedicated AST node **`ReturnTypeExpr`** (`field_types` + optional - `field_names`, same shape as a tuple) is produced by the parser for a bare-paren - result list with **≥2 value slots** (`(A, B)`, `(x: A, y: B)`, `(A, B, !)`). A - single-value `(T, !)` stays a `tuple_type_expr` (a plain failable, `= -> T !`). - An EMPTY `()` parses to the `void` type. -- It resolves (type_resolver `internTupleLike`, shared with `tuple_type_expr`) to - a reused `.tuple` TypeId — full ABI / failable / destructure / field-access - machinery reuse. Its distinct MEANING lives in the AST node, not the TypeId. -- Position gating: the node is valid only in a return slot. `resolveParamType` - rejects a `ReturnTypeExpr` parameter annotation ("multi-return is return-only; - use Tuple(…)"). Being a distinct node, its mere appearance in a value-type - position is categorically an error (no flag to check) — exhaustive `switch`es - over `node.data` were forced to add a `.return_type_expr` arm (coverage). -- Consumption: destructure (`a, b := f()`) or single-bind + field access - (`c := f(); c.sum`). No single source of truth needed at call sites — the - result is just a tuple value. -- SCOPE: multi-return on `name :: (...) -> (…) { }` function declarations first. - Multi-return CLOSURE-TYPE values (`cb: Closure() -> (A, B)`) and lambda - literals are a later phase. - -## What already exists (re-use, do NOT rebuild) -- `tuple_type_expr` → `.tuple` TypeId with optional `names` (type_resolver.zig - `resolveCompound`). -- Named + positional tuple field access `r.x` / `r.0` (expr.zig - `lowerFieldAccessOnType`). -- Destructuring `a, b := f()` (`DestructureDecl`, stmt.zig). -- Value-carrying failable assembly `(T1, …, !)` (error.zig - `lowerFailableSuccessReturn` / `emitTupleRet`) — error in the last slot. -- `return .(a, b)` / `return .(x = a, y = b)` tuple-literal returns (stmt.zig - `lowerReturn`). -- Generic inference through a failable/tuple closure return (this session's - parser `collectGenericNames` + generic.zig `extractTypeParam` tuple arms). - -## Foundation already landed (uncommitted, suite-green) -- **parser.zig** — `collectGenericNames` descends tuple/optional/function nodes - (so `Closure() -> $R !` binds `$R`); the bare-paren result-list path builds a - failable `tuple_type_expr` when it ends in `!` (`(A, B, !)` parses). -- **generic.zig** — `extractTypeParam` / `matchTypeParam[Static]` handle the - `(value, !)` tuple so `$R` infers from a closure ARG's failable return. - -## Phases (each: implement → lock with an example → `zig build test` green) - -0. **`() -> ()` = void (parser).** Isolated, unambiguous. An empty `()` in the - paren type path resolves to `void`. Lock: `a :: () -> () { }`. - -1. **Multi-return signatures `-> (A, B)` / `-> (x: A, y: B)` / `-> (A, B, !)` - (parser + AST + resolution).** Add the `multi_return_type` AST node; the parser - produces it for a bare-paren result list (return position). The return resolver - lowers it to a `.tuple` TypeId and sets `Function.multi_return`; the general - resolver rejects it (return-position only). Returns still use the existing - `return .(…)` literal in this phase (bare comma is Phase 2). Consumption is - destructuring `a, b := f()` (existing machinery). Lock: positional + named + - failable multi-return examples, each destructured. - -2. **Destructure-only enforcement + bare comma `return v1, v2` (parser + lowering).** - (a) Reject using a multi-return call as a single value (`r := f()`, an arg, an - operand) — read `Function.multi_return` at the binding/use site; only - destructuring is allowed. (b) Extend the return statement to parse a - comma-separated value list and lower it to the same multi-slot return the - `.(…)` literal produces (error slot stays implicit for failables). Single-value - `return v` unchanged. Lock: `-> (i64, bool) { return 7, true; }`, a failable - variant, and a negative example (`r := f()` → diagnostic). - -3. **Named-return locals + must-set rule (sema/lowering).** For a named return - `-> (x: A, y: B)`, bind each name as an in-scope assignable local (alloca). On - a path that reaches end-of-body with NO explicit `return`, synthesize the - implicit return from the named locals. Diagnose loudly if any named slot is - neither assigned on that path nor defaulted (no silent zero-fill). Explicit - `return v1, v2` / `return .(…)` still override. Lock: the - `b :: (...) -> (sum, good) { good = true; sum = ... }` example + a negative - example (unset slot → diagnostic). - -4. **Named-return defaults `(sum: i32 = 0, good: bool)`.** A slot with a default - is exempt from the must-set rule; the default fills it at the implicit (or - partial explicit) return. Lock: an example mixing a defaulted + a required - slot. - -## Open decisions (Decisions Log) -- **D1 — multi-return is NOT a tuple; return-position-only.** *Chosen* (user - directive). Realized via a distinct **`ReturnTypeExpr` AST node** (the user - preferred a dedicated node over a `TupleTypeExpr.is_multi_return` flag — it - makes "not a tuple" true at the AST level and makes position-gating - categorical) that resolves to a reused `.tuple` TypeId. A new `.tuple`-like - TypeInfo variant was rejected — it would ripple through every exhaustive type - switch for no ABI benefit. **Destructure-only was REVERSED** (see Rules): - single-binding a multi-return result is allowed (field access on the value - slots); the failable error stays on the separate `!` channel. -- **D2 (Phase 3) — storage for named-return locals.** Lean: an alloca per named - slot bound in the function scope under its name; the implicit return reads them - into the result tuple. Revisit if the must-set analysis wants SSA-style - definite-assignment instead of an alloca + per-path check. -- **D3 — multi-return closure-type values / lambda literals.** Deferred past the - function-decl phases (needs a `ClosureInfo.multi_return` flag). Phases 0–4 cover - named function declarations only. - -## Validation (every phase) -- `zig build && zig build test` green (full corpus). -- New `examples//…` locked with snapshots; review the diff for `.ir` - churn only where expected (the prelude type table is untouched by this stream, - so churn should be minimal/none). -- Adversarial review of each phase before it lands. - -## Category for examples -Multi-return is a core type/return feature — use the `types` block (`01xx`), -next free numbers, unless a better fit emerges. diff --git a/current/PLAN-RACE.md b/current/PLAN-RACE.md deleted file mode 100644 index da003544..00000000 --- a/current/PLAN-RACE.md +++ /dev/null @@ -1,145 +0,0 @@ -> **SUPERSEDED (2026-06-28).** The `race` LOGIC described here shipped, then was -> RE-HOMED onto `*Future` + the `Io` protocol as `context.io.race` in PLAN-IO-UNIFY -> Phase 4. The Task-based `sched.race` over `*Task` documented below is RETIRED. See -> `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for the current design. The -> type-level machinery this doc documents (`RaceResult` / `make_variant` / tuple -> reflection) is UNCHANGED and still in use. Body below is a historical record. - -# PLAN-RACE — Stream B2/A1: `race` over the M:1 fiber scheduler - -Carved from the async roadmap ([../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) -§4.5, §4.6, §7 step 3). The headline A1 feature still missing after Stream B1: **`race`** — start N -async tasks, return when the FIRST completes, and **structurally** cancel + join the losers before -returning. The result is a synthesized tagged-union mirroring the input named tuple's labels. - -```sx -fa := s.go(() -> A => read_a(conn)); // *Task(A) -fb := s.go(() -> B => read_b(conn)); // *Task(B) -winner := s.race((a: fa, b: fb)); // RaceResult = enum { a: A; b: B } -if winner == { - case .a: (v) { handle_a(v); } // v : A (fb cancelled + joined) - case .b: (v) { handle_b(v); } // v : B (fa cancelled + joined) -} -// positional form: s.race((fa, fb)) → tags ._0 / ._1 -``` - -## Design decisions (grounded against the tree, 2026-06-26) - -- **Built over the M:1 `Task` layer, NOT `context.io`/`Future`.** The suspending async is - `sched.go`/`wait`/`cancel` over `*Task($R)` (B1.4a); `context.io.async`→`Future` is the BLOCKING - impl (workers run inline → racing is meaningless there). The roadmap's "Future" maps to our - `*Task`. `race` is a `Scheduler`/`Task` UFCS function in `library/modules/std/sched.sx`. -- **The result is a comptime-synthesized nominal tagged-union** (`RaceResult`), one variant per input - tuple element: variant NAME = the tuple label (positional → `_0`/`_1`), payload = the task's result - type. Synthesis uses the proven `declare`/`define`/`make_enum` + `field_count`/`field_name`/ - `field_type` reflection (examples 0619–0623, 0646). The input arrives as an inferred type PARAMETER - `$T` (a named tuple of `*Task(..)`), which reflects correctly (issue 0195 fixed; the tuple-*alias* - gap is issue 0196, NOT on this path). -- **Cancellation rides the existing cooperative `Task.cancel`** (sets the flag + `.canceled` state). - `race` cancels every loser, then `wait`s each (joins) so no loser fiber outlives the `race` call — - structured. Reuses `suspend_self`/`wake`; no new scheduler machinery. - -## The one net-new compiler primitive (step 1) - -**`pointee($P: Type) -> Type #builtin`** — given a pointer type `*X`, return `X`. This is the only -missing reflection capability: `race` must project each tuple element `*Task(A)` to its result type -`A`, and there is currently NO way to get a pointer's target type at comptime (`field_count(*X)`=0, -`type_info` has no pointer variant). With it the projection is pure sx: -```sx -TaskResult :: ($P: Type) -> Type { // P = *Task(A) → A - return field_type(pointee(P), 0); // pointee → Task(A); field 0 = `value: A` -} -``` -Small + generally useful (reflection is currently complete for aggregates but blind to pointer -targets). Mirror `field_type`'s `#builtin` plumbing (`src/ir/lower/call.zig` + `src/ir/calls.zig`), -backed by the pointer TypeInfo's pointee TypeId (`src/ir/types.zig`). Lock with a comptime example. - -## Steps (each: implement → lock with an example → `zig build test` green → both platforms) - -1. **`pointee` reflection builtin.** Add `pointee($P: Type) -> Type` (core.sx + compiler). Example: - `pointee(*i64)` = `i64`, `field_type(pointee(*Task(i64)), 0)` = the task value type. (worker+review) -2. **`RaceResult($T) -> Type` synthesis.** Type-fn: reflect the named-tuple `$T` of `*Task(..)`, - project each element via `TaskResult`, mint the tagged-union (labels → variants). Comptime-only - example asserting the minted type's `field_count`/`field_name` match the input tuple. -3. **`Task.Value` projection + result construction.** Confirm a winner's value can be boxed into the - minted variant by label/index (uses the existing variant-construction path). -4. **Runtime `race(tasks: $T) -> RaceResult(T)`.** Suspend the caller until the first task is - `.ready`; build the winner variant; then cancel + `wait`-join every loser before returning. - Single-winner (first by completion order; FIFO tiebreak). Example: 2 tasks, deterministic winner - via `sleep` ordering (like 1817), asserting the loser is cancelled + joined. -5. **Positional tuple form** (`._0`/`._1`) + edge cases (already-ready task → immediate, single-task - race, all-cancelled). Examples. -6. **Validate** every new example byte-identical on aarch64-macOS host AND aarch64-linux container; - full `zig build test` green; adversarially review each step. - -## Status - -Prereqs DONE (each committed + adversarially reviewed + suite-green): -- **issue 0195** (tuple/array/vector field reflection) fixed (`8ac6c573`). Tuple reflection works on - inline + `$T`-param forms. Issue 0196 (tuple *alias*) filed, not on the critical path. -- **`pointee($P) -> Type`** builtin added (`f1d29876`) — projects `*Task(A)` → `Task(A)`. -- **`field_count`/`size_of`/`align_of` fold as comptime constants** (`2a6ef398`) — so a generic - `($T) -> Type` builder can `inline for 0..field_count(T)` and size `[field_count(T)]EnumVariant`. - Verified: the variable-arity loop + array dim now work inside `RaceResult`. - -- **comptime type-call composition** fixed (`eb18bbc6`) — a `field_type(...)`/`pointee(...)` result is - now usable as a `Type`-typed struct-field value, a generic `$P: Type` arg, and a nested type-call - arg (incl. with an `inline for` loop-var index). The **variable-arity `RaceResult` synthesis works - end-to-end** (proven by `examples/comptime/0649-comptime-typecall-composition.sx`: reflect a named - tuple of `*Box(..)` handles → mint a tagged-union with the tuple's labels, projecting `*Box(A)`→`A`). - -- **`return` inside `inline if` fixed** (`84c2ae4f`) — the natural early-return-per-arm pattern (a - `return` in an `inline if`/comptime-`case` branch inside an `inline for`) no longer drops the - function's trailing statements. Lets `race` build the winner variant with a clean - `inline if i == { case 0: … else: … }` per-arm form. -- **`make_variant($E, idx, payload)`** added to `modules/std/meta.sx` (`1c26944e`) — the WRITE side of - the metatype triad: construct a minted tagged-union value by variant INDEX (the winner is chosen at - runtime; its label can't be a literal). Pure sx (writes the i64 tag@0 + payload@8). Verified for - complex payloads (struct / string / 40-byte struct). **This resolves the variant-construction gap.** - -**GAP 1 — comptime-cursor indexing of a named-tuple VALUE — DONE** (`fee86adf`). `tasks[i]` with a -comptime cursor now reads the i-th element with its concrete type (option (a), a `structGet`). - -**GAP 2 — surfaced during the runtime, all DONE** (`6a976287`): three compiler enablers the runtime -needed beyond the read path. (1) a named-tuple LITERAL passed directly as `$T` lost its element names -(`field_name` → ""), breaking `RaceResult`'s `make_enum`; (2) tuple-element L-VALUES by comptime index -(`tasks[i].waiter = …`) panicked at LLVM emit (an `index_gep` with `ptrTo(.unresolved)`); (3) a user -`($X) -> Type` call couldn't bind a `$E: Type` arg, blocking `make_variant(RaceResult(T), …)`. All -fixed + adversarially reviewed; OOB comptime tuple indices now diagnose loudly on every L-value path. - -**Runtime in `sched.sx` — DONE** (`9099735e`). -- `RaceResult :: ($T) -> Type` over `*Task(..)` (the 0649 shape, with `Task`). -- `race :: ufcs (self: *Scheduler, tasks: $T) -> RaceResult(T)`: Phase 1 suspend until the FIRST - `.ready` (register waiter on all pending; on wake DEREGISTER from all; lowest-index winner). Phase 2 - build the winner with `make_variant`. Phase 3 cancel + JOIN each loser one-at-a-time (only the joined - loser carries a waiter → no mid-join double-wake). Join rides a new `Task.finished` flag (set at the - end of the `go` body, checked before parking). Cooperative-cancel: a loser parked mid-`sleep` runs to - its natural end before `race` returns. -- Locked by `examples/concurrency/1821-concurrency-fiber-race.sx` (3 tasks i64/bool/f64, sleep 10/20/30, - shortest wins, losers cancelled + joined). Byte-identical on aarch64-macOS host AND aarch64-linux - container; full `zig build test` green (826/0). - -**Remaining (step 5, future work):** -- POSITIONAL tuple form (`._0`/`._1`): `field_name` yields "" for an unnamed element → `make_enum` - rejects the duplicate. Needs a `_N` fallback in `RaceResult` (a comptime int→string) or a compiler - `field_name` default for positional tuples. -- Bare named-arg call form `s.race(a = ta, b = tb)` (no `.(…)` wrapper) — see the note below. -- Edge cases: already-ready-at-call (works today), all-cancelled (would deadlock-abort loudly). -- By-design caveat to document: `race` returns only when the SLOWEST loser finishes (cooperative - cancel can't preempt a mid-`sleep` loser; a loser blocked on `block_on_fd` that never fires blocks - the join forever — `cancel` doesn't unblock an fd waiter). - -### Future work — bare named-arg call form `s.race(a = ta, b = tb)` -Today the result-tuple is passed as a named-tuple LITERAL: `s.race(.(a = ta, b = tb))`. The user asked -about dropping the `.(…)` so it reads `s.race(a = ta, b = tb)`. That is a CALL-SITE syntax/binding -feature, independent of the race runtime: -- The parser would have to accept `name = expr` call arguments and, for a `(tasks: $T)` parameter whose - type is inferred, MATERIALIZE them into a single named-tuple value bound to `$T` (rather than treating - each `name = expr` as a separate positional/keyword arg). Effectively "collect trailing `k = v` call - args into one anonymous named-tuple when the callee has a single inferred aggregate param". -- Risk: `name = expr` in call position currently has no meaning (or would collide with a future - keyword-argument feature). Decide whether `k = v` call args are (a) always tuple-materialized for an - inferred aggregate param, or (b) a dedicated `..` / sugar. Option (a) is the least new syntax. -- Once the call site produces the same named-tuple value, the entire race runtime is unchanged (it - already reflects `$T`). So this is purely front-end sugar — schedule it with the keyword-args work, - not the concurrency stream.