Compare commits

...

236 Commits

Author SHA1 Message Date
agra
bdf83db4c8 Merge remote-tracking branch 'origin/master' 2026-06-21 16:03:48 +03:00
agra
66bdc70bf1 test: group examples into per-category folders
Move examples/*.sx and their expected/ snapshots into per-category
subfolders (examples/<category>/...). Folder = leading filename token,
with ffi-objc/ffi-jni kept whole; filenames are unchanged. The corpus
runner and LSP sweep now discover each category's expected/ dir, while
issues/ stays flat. Example 1058's repo-root-relative companion import
is made file-relative. Path strings embedded in 164 snapshots were
regenerated (path-only changes). Test-layout docs in CLAUDE.md updated.
2026-06-21 14:41:34 +03:00
agra
6d1409bc1f test: remove resolved/fixed issue writeups
Delete the issues/*.md whose writeup carries a RESOLVED or FIXED banner;
only the open issues (0030, 0148) remain.
2026-06-21 14:41:18 +03:00
agra
e95f7c448a ... 2026-06-21 11:20:00 +03:00
agra
6b0ebdd92b lang: require explicit receiver in protocol method declarations
Protocol method declarations now declare their receiver explicitly as the first
parameter — 'self: *Self' (or 'self: Self') — matching the impl method signature,
instead of the old implicit-receiver form where the listed params were only the
extra args. That asymmetry repeatedly caused confusion over whether the first
param was the receiver or an argument.

The parser validates the first param is 'self' typed Self/*Self, then strips it,
so all downstream lowering and the dispatch ABI are unchanged (impl blocks and
call sites are unaffected). A protocol method missing the receiver is now a parse
error.

Migrated all 129 protocol method signatures across library + examples (+ one
inline-sx test in sema.zig) to the explicit form. Updated specs.md + readme.md.

New: examples/0418-protocols-explicit-receiver.sx (feature),
examples/1190-diagnostics-protocol-missing-receiver.sx (negative/diagnostic).
2026-06-21 11:02:16 +03:00
agra
eb93c63c45 docs(issue 0148): record root cause, working mechanism, and blast-radius findings
Attempted the canonicalize-path fix (realpath + cwd-relativization for display);
it fixes the e2e absolute-entry duplication but ripples path-identity changes
into ~8 import-subsystem unit tests + 2 cosmetic snapshots. Reverted as too broad
for a drive-by rework; documented a minimal repro and a recommended deliberate
approach (lexical normalize, single chokepoint, batch test updates). Issue stays
OPEN.
2026-06-21 09:49:55 +03:00
agra
21d91e6718 fix: resolve module-alias-qualified type in reflection arg slot (issue 0147)
size_of(sel.Selection) and the other reflection builtins rejected a
module-alias-qualified type: in argument position it parses as a .field_access
expression (not the dotted .type_expr a declaration produces), and neither
isStaticTypeArg nor resolveTypeArg had a .field_access arm. Add both: a pure
namespace-decl scan in isStaticTypeArg, and resolution via namespaceAliasTarget
+ resolveNominalLeaf in the target module context in resolveTypeArg (mirroring
the value-position lowerFieldAccess path). No fabricated-stub fallback.

Regression: examples/0192-types-size-of-qualified-alias.sx
2026-06-21 09:33:46 +03:00
agra
c21b683b08 docs(issues): mark 17 already-fixed issues RESOLVED with verified banners
Each banner was re-verified against the current binary (repro now behaves
correctly) and cites the actual fix location in current src/** plus the covering
regression example. Closes the stale-but-fixed backlog: 0019, 0042-0056, 0131.
No compiler change.
2026-06-21 09:25:52 +03:00
agra
4fc5411cd9 fix: allow void (zero-sized) struct/tuple fields instead of crashing (issue 0150)
A struct/tuple/?T with a void field crashed the compiler: the field lowered to
LLVM's unsized 'void' type, which traps getTypeSizeInBits. Lower a void field to
a SIZED zero-byte [0 x i8] (fieldLLVMType) so the enclosing aggregate stays sized
with identical element indices, and skip inserting a value for a void field in
emitStructInit (the i64 placeholder would type-mismatch the [0 x i8] slot and
corrupt the aggregate constant -> runtime bus error). Future(void) now works.

Regression: examples/0190-types-void-struct-field-zero-sized.sx
2026-06-21 09:21:18 +03:00
agra
7057175fb6 fix: promote mismatched comparison operands before emitting cmp (issue 0146)
A comparison with int-vs-float (or two float widths) operands emitted cmp on
the raw operands with no promotion, unlike the arithmetic arms -- producing a
mixed-type compare the LLVM verifier rejects / mis-evaluates. lowerBinaryOp now
coerces each operand to the promoted common type (from arithResultType) via
coerceToType (SIToFP / FPExt) for the ordering/equality arms when the promoted
type is a float, so LLVM gets a well-typed fcmp.

Regression: examples/0189-types-int-float-compare-promote.sx
2026-06-21 09:11:52 +03:00
agra
d4edf4b4b0 fix: method on array-index/deref receiver mutates the live place (issue 0145)
A *self method called directly on arr[i] (or a deref place) fell through to an
alloca+store-of-value, so the callee mutated a throwaway copy and the live slot
was never written. fixupMethodReceiver now takes the real address of
.index_expr/.deref_expr receivers via lowerExprAsPtr (normalized to *T),
mirroring the explicit-argument path. A comptime-pack index (xs[i] where xs is
a pack) is excluded -- a pack has no runtime storage to address -- so it keeps
flowing through the general copy path.

Regression: examples/0188-types-method-array-index-receiver.sx
2026-06-21 09:11:44 +03:00
agra
333f57026c fix: give error-set decls per-decl nominal identity (issue 0134)
A local 'error { ... }' set with the same name as an imported one collapsed
onto the import, losing its own tags, because registerErrorSetDecl deduped via
the flat findByName path while struct/enum/union use E6a per-decl identity.
Build the .error_set TypeInfo (new buildErrorSetInfo helper factored from
resolveInlineErrorSet) and intern via internNamedTypeDecl with shadowNominalId;
reserve a distinct shadow slot in scanDecls; consult per-decl type_decl_tids in
namedRefTid before findByName. The inline/anonymous findByName short-circuit is
preserved.

Regression: examples/1059-errors-same-name-error-set-own-wins.sx (moved from
issues/0134).
2026-06-21 09:11:06 +03:00
agra
ad45ae07ef fix: diagnose unknown generic #builtin instead of silently returning 0 (issue 0144)
A bodiless #builtin with a $T: Type param routes through monomorphization.
When resolveBuiltin returned null for an unrecognized name, the builtin-body
branch fell through to ensureTerminator's constInt(0) -- a silent-fallback
default the CLAUDE.md REJECTED PATTERNS forbid. Emit a loud
'error: unknown #builtin <name>' diagnostic instead.

Regression: examples/1189-diagnostics-unknown-builtin.sx
2026-06-21 09:10:38 +03:00
agra
6ed29621ad fix: diagnose missing 'main' instead of segfaulting on 'sx run' (issue 0137)
A program with no 'main' reached the JIT entry-point call with a garbage
address (ORC reports lookup success but leaves main_addr degenerate), then
called it -> SIGSEGV. Add a pre-JIT entry-point check in main.zig that emits
'error: no main function found' and exits non-zero before codegen, plus a
defensive main_addr==0 guard in target.zig runJITFromObject as a backstop.

Regression: examples/1188-diagnostics-run-no-main.sx
2026-06-21 09:10:30 +03:00
agra
11dc6a3299 fibers: drop redundant async_void — the variadic async covers nullary workers
async_void :: ufcs (io, worker: Closure() -> $R) -> Future($R) was redundant:
the variadic async :: ufcs (io, worker: Closure(..$args) -> $R, ..$args) binds
$args to the empty pack, so context.io.async(() -> $R => ...) already calls
worker() and returns Future($R). The name was also misleading — it returns
Future($R), not void (a true void form is Future(void), separate, blocked by
issue 0150).

Removed the definition (std/io.sx) + the std.sx re-export; nothing else
referenced it. Locked the nullary path in examples/1805 (prints nullary: 42) so
the coverage async_void provided is not lost. Suite green 736/0.
2026-06-21 07:54:45 +03:00
agra
2437cf5e59 fibers B1.3b-1: x86_64 / Win64 swap_context sibling, validated on a Win7 x64 VM
The context switch is now proven on a second arch/ABI pair. A Win64 swap_context
saves the complete Win64 callee-saved set: 8 GP (rbx,rbp,rdi,rsi,r12-r15) + rsp
AND xmm6-xmm15 (10 XMM, 128-bit via movups -- Win64 has callee-saved XMM, unlike
SysV/aarch64), plus a Win64 scribble_verify (264-byte frame, 32-byte shadow +
16-align at each call, COFF symbols, rsp-carried return address) driving the
2-fiber mutual scribble.

Built --target x86_64-windows-gnu --self-contained (PE32+, output via the Win32
WriteFile boundary -- the 1660 pattern) and run on a Windows 7 x64 VM (UTM):
printed '0 0 P' -- every GP + XMM callee-saved register survived the switch.
Adversarially reviewed before the VM run (worker emitted the real .s and
verified every call alignment, the frame offsets, the rsp/return-address
round-trip, swap ordering, and COFF naming against the Win64 ABI -- no
critical/minor bugs).

Locked by examples/1810-concurrency-fiber-switch-win64.sx (pinned
x86_64-windows-gnu, ir-only on this non-Windows host; the VM run is the
runtime-correctness provenance). Good-swap-only mutual scribble (self-validating
by construction; the in-process negative control was dropped to avoid an sx
fn-ptr-convention issue -- detection of this exact logic was negative-controlled
on aarch64 in 1808).

Suite green 736/0. The B1.3 switch is proven on aarch64 + x86_64/Win64. Next:
B1.4 (Io impls / M:1 scheduler).
2026-06-21 07:35:51 +03:00
agra
dd532ab7b2 fibers B1.3b: mmap guard-page fiber stacks (x86_64 switch sibling deferred)
Fiber stacks are now mmap'd with a PROT_NONE guard page at the low end: mmap a
[guard | usable] region and mprotect the low 16KB page PROT_NONE, so a stack
overflow faults at the guard boundary instead of silently corrupting a neighbor
(design 8.1.1 — fixed stacks without a guard corrupt silently on overflow).

Locked by examples/1809-concurrency-fiber-guard-stack.sx (aarch64-macos-pinned):
guard armed: 1 (mprotect -> 0) + sum: 20100 (a fiber runs real recursion on the
guarded stack and yields). The guard FIRING is validated manually (a fiber
recursing past its 128KB stack faults with Bus error at region+GUARD, exit 134
via the sx crash handler) — not corpus-pinned, since a deliberate-overflow crash
is host-fragile and a 'child faulted' fork test would not prove the boundary
catch specifically.

The x86_64 swap_context sibling is DEFERRED: sx build --target x86_64-macos
mislinks on this arm64 host (object x86_64, link step arm64) and x86_64-linux
can't run here, so it could only ship IR-only / unrun. For the highest-
corruption-risk asm, shipping un-run / un-negative-controlled code violates the
design 10.7 'correctness not existence' rule. SysV target notes (rbx/rbp/r12-r15
/rsp, no callee-saved XMM, rsp-carried return address) recorded for a future
x86_64 host. Suite green 735/0.
2026-06-21 06:51:29 +03:00
agra
ed1b6c396d fibers B1.3a-2: context-switch stress gate (explicit callee-saved scribble) + adversarial review
The design section-10.7 correctness gate the foundational switch owed: explicitly
scribble EVERY callee-saved register, switch, and verify each survived.

- Extended swap_context to the COMPLETE AAPCS64 callee-saved set: integer
  x19-x28 + fp/lr + sp AND the FP regs d8-d15 (21-slot context). Per AAPCS64
  6.1.2 only the low 64 bits of v8-v15 are callee-saved, so d8-d15 is exactly
  sufficient; x18 is Apple's reserved platform register, untouched.
- naked scribble_verify(self_ctx, peer, base): loads a unique sentinel into all
  18 callee-saved regs, bl swap_context to yield, and on resume counts the regs
  that did not survive. Honors its own caller ABI via a 176-byte frame that
  saves+restores the caller's callee-saved; base reloaded post-swap (x2 not
  preserved); the original lr round-trips through the swap.
- The gate is a 2-fiber MUTUAL scribble (A and B scribble DISTINCT sentinels into
  the same physical regs), so a value survives only if swap_context saved and
  restored it. A lone fiber yielding to an idle peer would NOT exercise
  preservation.

Locked by examples/1808-concurrency-fiber-switch-stress.sx (aarch64-pinned):
A/B mismatches: 0. Validity proven by negative controls: dropping the d8-d15
save/restore reports 8/8 mismatches (the FP regs); dropping x27/x28 reports 2/2.

Adversarial review (worker): no critical bugs — callee-saved set complete and
correct, all frame offsets / 16-alignment / the lr-sp dance verified against
AAPCS64. Applied its one recommendation: boot zeroes the FP ctx slots so a first
switch-to loads 0, not garbage, into d8-d15. Residual gaps (spec-correct for a
call-boundary swap, documented in the header): FPCR/FPSR/NZCV + TPIDR/TLS are not
swapped, fp=0 blocks unwind past a fiber trampoline — these matter at the N×M:1 /
signals stages, not the single-thread switch.

Suite green 734/0. Next: B1.3b (x86_64 sibling + mmap guard-page stacks).
2026-06-21 06:38:02 +03:00
agra
b234b7df6f fibers B1.3a-1: stackful context switch (naked swap_context + fiber bootstrap)
The first piece of the B1.3 fiber runtime — the stackful context switch, pure
sx over abi(.naked). swap_context(from, to) saves the callee-saved registers +
SP/LR into *from and loads them from *to, then rets onto to's stack (SP-in !=
SP-out by design — why it must be .naked). Fibers are bootstrapped by hand: the
saved context starts with SP = top of an alloc_bytes stack, LR = a global-asm
trampoline (mov x0, x19; bl _fib_body, reaching the sx body via export), and
x19 = the *Fiber.

Locked by examples/1807-concurrency-fiber-context-switch.sx (aarch64-pinned):
- 2-fiber ping-pong (A <-> B, 3 rounds each): rounds: 6, and a per-fiber stack
  canary held live across every suspend survives (canary fails: 0);
- a 64-frame deep recursive chain suspended at the bottom and resumed, verifying
  every frame's stack-local on the unwind (frames verified: 64, depth fails: 0).

Scope (honest): exercises register/stack preservation INDIRECTLY (compiler-
allocated live values + the canary). The EXPLICIT every-callee-saved GP
(x19-x28) + FP (d8-d15) sentinel scribble — the full design-section-10.7 gate —
is B1.3a-2, still owed. x86_64 sibling + mmap guard-page stacks are B1.3b.

Suite green 733/0. Runs under JIT, ir-only on a non-arm host.
2026-06-21 06:16:58 +03:00
agra
37d68e72be fibers B1.2 COMPLETE: async/await/cancel examples (1805/1806)
With the three surface blockers fixed (0151 generic inference, 0152
Atomic(bool), 0153 re-export failable channel), the M:1 async surface works
end-to-end on the blocking Io default. Landed the corpus examples:

- 1805-concurrency-io-blocking-async.sx: context.io.async(lambda, ..args)
  runs the worker inline, await() or {…} yields the result; context.io.now_ms()
  reads the monotonic clock. Prints sum: 42 / double: 42 / clock ok.
- 1806-concurrency-io-cancel.sx: f.cancel() marks the future canceled so a
  later await() raises error.Canceled out of its (R, !IoErr) channel, caught
  with or. Prints ok: 7 / canceled: -99.

B1.2 (Io capability on Context + async/await/cancel + blocking CBlockingIo) is
complete. Suite green 732/0. Next: B1.3 (fiber runtime).
2026-06-21 05:59:04 +03:00
agra
68c1991e11 issue 0153 RESOLVED: pin generic return-type resolution to the fn's defining module
inferGenericReturnType resolved a generic call's return-type AST ($R, !E) in
the CALL-SITE module context. For a re-exported fn the error-set name (LE /
IoErr, re-exported as LE :: lib.LE) resolved through the call-site alias to a
TypeId NOT tagged .error_set, so the planned result was a tuple whose last
field wasn't an error set — errorChannelOf saw a plain tuple and the value-
failable's ! channel was lost (try/or rejected it / built a malformed i1 PHI).

monomorphizeFunction already pins the source to the fn's defining module
before resolving the return type; inferGenericReturnType did not, so the
planned call-result type disagreed with the instance's real signature. Fix:
pin the source to fd.body.source_file around the return-type resolution
(binding-build stays in the call-site context — its args are typed there).

Regression test examples/1058-errors-reexport-value-failable-channel.sx
(+ companion lib.sx). Suite green 732/0.
2026-06-21 05:55:14 +03:00
agra
a7499d5f51 fibers B1.2: 0152 fixed → Atomic(bool) works; blocked on 0153 (re-export value-failable loses ! channel)
With 0151 + 0152 fixed, the async surface is callable and Atomic(bool) works.
Building the async examples isolated the TRUE remaining blocker (the earlier
'secondary or PHI' symptom, confirmed NOT an Atomic cascade): a re-exported
generic value-failable ($R, !E) fn loses its ! error channel at the call site
— the result types as a plain tuple, so await(...) or { ... } / try ...await()
fail / build a malformed i1 PHI. await/IoErr are re-exported via std.sx, so the
async surface hits it.

Narrowed to the generic + re-export co-requirement (non-generic re-export OK;
direct generic import OK). Filed issues/0153 with a minimal co-located 2-file
repro + a single-file stdlib-await repro + investigation prompt (root cause:
the monomorphized return-type's error-set, reached via the re-export alias,
resolves to a non-.error_set TypeId, so errorChannelOf misses the channel).
Per the STOP rule, paused B1.2's async examples pending the 0153 fix.
2026-06-21 05:45:27 +03:00
agra
e5586f61b8 issue 0152 RESOLVED: byte-promote sub-byte (Atomic(bool)) atomic load/store
LLVM rejects a sub-byte atomic memory access (must be byte-sized), so
Atomic(bool) — bool lowers to i1 — failed verification on load/store. The
atomic emitters in src/backend/llvm/ops.zig now perform a sub-byte access in
its byte storage type (i8) and trunc/zext the value at the boundary (new
atomicByteType helper: i8 for .bool, null otherwise). rmw/cmpxchg are left
as-is on purpose — a bool rmw/CAS is rejected at the sx level (integer-only),
so a sub-byte element never reaches those emitters.

Regression test examples/1705-atomics-bool-byte-promoted.sx. Suite green 729/0.
Unblocks Future.canceled: Atomic(bool) in the B1.2 async layer.
2026-06-21 05:42:48 +03:00
agra
ea1faf7b69 fibers B1.2: 0151 fixed → async surface callable; blocked on 0152 (Atomic(bool) i1 atomic)
issue 0151 (generic $T inference through generic-struct / pointer / UFCS-pack
params) is fixed and committed, so io.sx's async/await/cancel are now callable
in every form. Building the async examples then tripped a SEPARATE codegen bug:
Atomic(bool) emits a sub-byte (i1) atomic load/store that fails LLVM
verification (must be byte-sized). Future.canceled: Atomic(bool) hits it.

Filed issues/0152 with a standalone repro + investigation prompt (codegen fix
in src/backend/llvm/ops.zig — promote sub-byte atomics to i8 storage). Per the
STOP rule, paused B1.2's async examples (1805/1806) pending the 0152 fix.
Checkpoint updated: 0151 RESOLVED, async surface BLOCKED on 0152.
2026-06-21 05:27:41 +03:00
agra
362674f04d issue 0151 RESOLVED: infer generic $T through generic-struct / pointer / UFCS-pack params
The generic-inference engine could not bind a $T from a generic-struct
argument head. Four gaps, all on the inference + UFCS dispatch path:

- extractTypeParam / matchTypeParam(Static) gained a parameterized_type_expr
  arm: recover the arg instance's recorded per-param bindings
  (struct_instance_bindings + the template's ordered type_params via
  struct_instance_author) and recurse positionally, so $T binds from
  Box($T) <=> Box(i64) like it does from []$T <=> []i64. This also fixes
  the pointer case — *Box($T) recurses into its Box($T) pointee.
- The pointer_type_expr arm now falls through to match the pointee against a
  non-pointer arg (auto-address-of: a *Box($T) param accepts a by-value
  Box($T), e.g. the UFCS receiver b.m()).
- ExprTyper.inferType gained a .lambda arm building the closure type from the
  lambda's annotations, so the UFCS binder (which types args from the raw AST
  before they are lowered) can bind a Closure(..) -> $R from the worker's
  declared return type.
- A pack UFCS target (worker: Closure(..) -> $R, ..$args) now routes through
  the same lowerPackFnCall the direct call uses, with the receiver spliced in
  as args[0] (lowerPackFnCall reads only call_node.args, never the callee).

Regression tests: examples/0214 (direct + UFCS closure-return pack) and
examples/0215 (by-value / pointer / multi-param / nested / UFCS-auto-ref
generic-struct-head inference). Suite green 728/0.
2026-06-21 05:25:39 +03:00
agra
0ab26c8a40 fibers B1.2: record review findings — async surface blocked on 0151 (widened)
Adversarial review of 45d869d: the Io infrastructure (both materializers,
push-inherit, 37 .ir regens, !-lint) is correct + landed; but await/cancel
(*Future($R)) are uncallable in EVERY form because sx can't infer a generic
$T from a pointer-wrapped arg. Widened issue 0151 to that root (repro:
unbox(b: *Box($T)) -> $T). Checkpoint: B1.2 partially landed; next = fix 0151
generic inference -> make await/cancel callable -> add 1805/1806 -> B1.3.
2026-06-21 00:43:09 +03:00
agra
3eeb965925 issue 0151: UFCS dot-call leaves $R inferred from a closure return type via a pack unresolved
A generic free fn whose `$R` is inferred from a worker `Closure(..$args) -> $R`
(+ trailing `..$args`) and which returns a type built from `$R` (`-> Wrap($R)`)
monomorphizes correctly when called directly (`f(recv, worker, ..)`) but leaves
`$R` UNRESOLVED when called via UFCS dot syntax (`recv.f(worker, ..)`) — the
unresolved type reaches LLVM emission and trips the `.unresolved` tripwire
(SIGTRAP). Distinct from RESOLVED issue 0119 (UFCS `$T` from receiver/slice).

Blocks the B1.2 user-facing async idiom `context.io.async((a,b) -> R => ..., x, y)`
(a UFCS call inferring $R from the worker closure's return type). The Io/async
library + compiler plumbing are in place and correct (landed in the prior
commit); only the UFCS call form hits this inference gap. Repro depends on no
project symbols beyond modules/std.sx; unpinned (no expected/ marker) so it
does not run in the corpus.
2026-06-20 22:21:38 +03:00
agra
45d869da41 fibers B1.2: Io capability + context.io + blocking impl + Future/async/await/cancel
Threads an `Io` capability onto `Context` exactly like `Allocator`: a
`protocol #inline` whose process-wide default is a stateless `CBlockingIo`
(the mirror of `CAllocator`), installed in `__sx_default_context`.

Library (library/modules/std):
- core.sx: `Io` protocol (spawn_raw / suspend_raw / ready / poll / now_ms /
  arm_timer) + `SpawnOpts` / `PinTarget` / `ParkToken`; `Context` gains an
  `io: Io` field LAST (allocator stays index 0, data stays index 1).
- io.sx (new): `CBlockingIo` + `impl Io` (blocking M:1 semantics — now_ms is
  a real monotonic clock, the rest are no-ops/0; suspend never called);
  `Future($R)` { value; state: FutureState; err: IoErr; park; task; canceled:
  Atomic(bool) } with `Value :: R`; the async ergonomic layer
  `async` / `async_void` / `await` (value-carrying `(R, !IoErr)`) / `cancel`.
  Built with the verified `= ---` + field-assign + `Closure(..$args) -> $R` +
  `..$args` idiom (NON-void $R only — Future(void) is deferred per issue 0150).
- std.sx: re-export the Io surface + the io.sx tail.

Compiler (src/ir):
- protocol.zig `emitDefaultContextGlobal` + comptime_vm.zig
  `materializeDefaultContext`: both materializers of `__sx_default_context`
  now build the inline CBlockingIo->Io vtable (7 words) at the new field.
- stmt.zig `lowerPush`: `push Context.{...}` now INHERITS omitted fields from
  the ambient context (seed the slot from current_ctx_ref, overwrite only the
  literal's named fields) — correct capability-bag semantics, so the partial
  `push Context.{ allocator = X }` sites don't zero a null `io` vtable.
- protocols.zig + lower.zig + error_analysis.zig: record protocol-impl method
  names so the "declared `!` but never errors" lint skips a conforming impl
  whose `!` is dictated by the protocol contract (e.g. Io.suspend_raw).

37 `.ir` snapshots regenerated: layout-only (the Context type now carries the
Io field, shifting type-table numbering); no stdout/stderr/exit changes.

The blocking Io + now_ms + Future/async work when `async` is called with the
receiver passed explicitly; the user-facing UFCS form `context.io.async(...)`
is blocked on a separate UFCS generic-inference bug (filed next).

Suite: 726 ran, 0 failed.
2026-06-20 22:21:27 +03:00
agra
a1b14f0c0f fibers B1.2: lock async/await example 1805 (RED)
Adds the blocking-Io async/await example with a seeded empty .exit marker.
The example fails today (Io protocol + context.io + async/await not yet
implemented); the next commit lands the Io interface + blocking impl +
both __sx_default_context materializers + push-inherit fix to turn it green.

Worker is a lambda with annotated params (the verified B1.2 idiom);
named-fn workers are deferred pending a :: callable-param feature.
2026-06-20 21:55:52 +03:00
agra
eee905c73c fibers B1.2: lambda-only async (named-fn :: feature deferred)
User decision: ship B1.2 async with lambda workers (works today, zero
compiler change); defer named-fn workers, which need a new :: callable-
parameter language feature (3 failed worker attempts; partial WIP saved
at .sx-tmp/wip-callable-params/). Records the resolved lambda async idiom
+ resume plan; no compiler/library code changed.
2026-06-20 21:51:01 +03:00
agra
7bf65565bd fibers B1.2: UNBLOCKED — remove invalid issue 0151, correct the async idiom
The B1.2 "blockers" were not real:
- Issue 0151 was INVALID: its repro used the non-idiomatic `($A) -> $R`
  bare-fn-ptr form. The canonical higher-order pack idiom
  `Closure(..$args) -> $R` + `..$args` (see examples/0543-packs-canonical-map)
  infers $R fine and runs today with no compiler change. Removed 0151.
- The correct async idiom is verified working live (42 42 for homo + hetero
  args): async :: (io, worker: Closure(..$args) -> $R, ..$args) -> Future($R)
  with a lambda worker (annotated params) + a `result = ---; result.v = ...`
  build form. No compiler change needed.

Issue 0150 (void struct field -> SIGTRAP exit 133) IS a real bug but is only
reached via Future(void) (void-returning worker / timeout) — deferred to B1.4;
B1.2 supports non-void workers.

Updates the PLAN/CHECKPOINT B1.2 status to UNBLOCKED with the corrected idiom
and the resume plan. No compiler/library code changed in this commit.
2026-06-20 20:00:36 +03:00
agra
f0a918f3c8 fibers B1.2: record async-args = variadic pack (..$args: []Type) correction
User correction: async's args are a variadic heterogeneous comptime pack
(..$args: []Type, specs.md:1383), not a single $A. Orthogonal to 0151
(return type-var binding). Recorded for the B1.2 resume.
2026-06-20 19:03:43 +03:00
agra
b97da83e8b fibers: commit the abi(.naked) example bodies (rename staging miss)
The .pure->.naked rename (a7fe165) git-mv'd examples 1800-1803 to their
naked names but the perl content edit (abi(.pure) -> abi(.naked) in the
bodies/comments) was never re-staged, so HEAD carried the renamed files
with stale abi(.pure) bodies — which the compiler now rejects ("unknown
ABI"). The working tree had the correct .naked bodies uncommitted; this
commits them so HEAD parses + builds clean.
2026-06-20 18:57:14 +03:00
agra
e78320637f fibers B1.2: BLOCKED on compiler bugs 0150 + 0151 (Io design proven)
Stream B1 B1.2 (Io capability + context.io + Future + cancel) is blocked on
two newly-discovered, independent compiler bugs, both with standalone repros:

- 0150: a `void` struct field crashes the compiler with an unsized-type
  SIGTRAP in LLVM getTypeSizeInBits. Blocks `Future(void)` -> `timeout`.
- 0151: a type-var inferred from a fn-pointer parameter's RETURN type is not
  bound as a usable type in the function body (`unknown type 'R'`). Blocks the
  central `async(io, worker: ($A)->$R, arg)` free-fn's `Future(R)`.

The B1.2 design itself is validated end-to-end (the Io protocol threaded on
Context like Allocator, the stateless blocking CBlockingIo default, both
__sx_default_context materializers, and `context.io.now_ms()` all work live).
Only the async/await/timeout ergonomic layer hits the two bugs. Per the
IMPASSABLE STOP rule, all B1.2 working changes were reverted (master green,
726/0) and the work paused pending fixes; WIP is saved at .sx-tmp/b12-wip/.

Checkpoint + plan updated to mark B1.2 BLOCKED with full resume notes.
2026-06-20 18:54:04 +03:00
agra
bab4886346 fibers B1.1: per-fiber context root is library-only (no compiler change)
A fiber needs its own root Context (the spawner's snapshot), not the
ambient one. Probed whether that needs compiler support: it does not.
context is an implicit slot-0 *Context param (call-carried, rides the
callee's own stack) and push Context allocates on the caller frame —
never TLS, never re-read from the __sx_default_context global mid-stack.
So the spawn convention is pure library sx:

  snap := context;            // snapshot the spawner's context
  f := Fiber.{ root = snap }; // store it
  push f.root { entry(args) } // trampoline installs it as the fiber root

examples/1804-concurrency-context-snapshot.sx locks it: a trampoline
running under ambient ctx 99 installs a stored snapshot (42); the body
reads 42, and the push scope restores 99 on exit. No fiber runtime yet
(B1.3) — this proves the plumbing it builds on.

The design doc's "lower context as swappable indirection, never raw
TLS" guarded a non-problem — context was already param-carried.

Suite green (726/0).
2026-06-20 17:09:26 +03:00
agra
a7fe165684 fibers: rename ABI variant .pure -> .naked
"pure" universally means side-effect-free (GCC __attribute__((pure)),
FP purity, D's pure) — the opposite of a register-clobbering context
switch. The concept is "naked": no compiler-generated prologue/epilogue,
body is raw asm that emits its own ret. That is the established term
everywhere (LLVM's naked function attribute — which we literally emit —
plus Zig callconv(.naked), Rust #[naked], GCC/Clang __attribute__
((naked))). Rename the keyword + everything keyed off it so concept,
surface, field, and the emitted LLVM attribute all agree.

- ast.zig: ABI enum variant pure -> naked (+ doc).
- parser: accept abi(.naked); error text updated.
- IR Function.is_pure -> is_naked; type_resolver/decl/generic/pack/
  emit_llvm references updated; diagnostics say abi(.naked).
- examples 1800-1803 renamed *-pure-* -> *-naked-* (source + expected/
  snapshots; .ir/.exit/.stdout/.stderr are byte-identical — the emitted
  IR is unchanged, only the keyword spelling differs).
- docs (PLAN-FIBERS, CHECKPOINT-FIBERS, PLAN-POST-METATYPE, the design
  roadmap, the compiler-API checkpoint/design) updated; the naming
  rationale now records why .naked over .pure.

No semantic change — pure cosmetics. Suite green (725/0).
2026-06-20 17:01:09 +03:00
agra
b631590574 fibers B1.0c: support params in abi(.pure) (read from registers)
Adversarial review of B1.0b found a param-bearing abi(.pure) function
emitted invalid LLVM ("cannot use argument of naked function" — loud
verifier error, not silent) because the param-alloca loop spilled the
args to stack slots, which a naked function cannot have.

Fixed forward — this ENABLES the B1.3 context-switch use case rather
than rejecting it: gate the param-alloca loop on fd.abi != .pure in
decl.zig (both body-lowering paths) and generic.zig. A naked function's
args stay in their ABI registers and are read directly by the asm body
(e.g. swap_context reads from/to from x0/x1); the LLVM args are
declared-but-unused, which the verifier allows.

examples/1803-concurrency-pure-asm-param.sx: naked add(a, b) reads x0/x1
(add x0, x0, x1; ret) -> 40 + 2 = 42. aarch64-pinned.

Pack abi(.pure) (variadic + naked — nonsensical, can't read a runtime
pack from registers) left unsupported: pack.zig's param loop is
intertwined with comptime-param/#insert handling, so that case still
hits the loud verifier error. Documented in the checkpoint.

Also updates PLAN-FIBERS / CHECKPOINT-FIBERS for B1.0 completion.
B1.0 complete. Suite green (725/0).
2026-06-20 16:36:31 +03:00
agra
4b384788e6 fibers B1.0b: abi(.pure) emits a real LLVM naked function (green)
Flip the B1.0a emit bail to real emission. The emit_llvm declaration
pass now adds LLVM's naked + noinline + nounwind attributes for an
is_pure function and skips frame-pointer=all (incompatible with a
frameless function); Pass 2 emits the body normally, and the naked
attribute makes the backend emit it verbatim (the inline asm + its own
ret) with no prologue/epilogue.

IR shape verified:
  ; Function Attrs: naked noinline nounwind
  define internal i64 @answer() #0 {
  entry:
    call void asm sideeffect "...ret...", ""()
    unreachable
  }
The caller invokes it as an ordinary () -> i64 call (.pure is
call_conv == .default).

- examples/1800-concurrency-pure-asm.sx: now green, aarch64-pinned
  (.build macos) -> exit 42 + .ir snapshot.
- examples/1801-concurrency-pure-generic.sx (renamed from -bail): the
  generic .pure now emits a correct naked answer__i64 (exit 42),
  proving generic.zig produces a naked body, not a framed one.
- examples/1802-concurrency-pure-asm-x86.sx: x86_64 cross sibling
  (.build x86_64-linux, ir-only here); .ir locks naked + movl $42,%eax.
- unit test in emit_llvm.test.zig asserts the naked attribute is present
  and frame-pointer absent on an abi(.pure) function.

Suite green (724/0).
2026-06-20 16:36:12 +03:00
agra
40424df1b8 fibers B1.0a: close generic/pack is_pure gap (review)
Adversarial review of dd363ca found is_pure was set only at the two
declareFunction decl sites. Generic monomorphization (generic.zig) and
pack expansion (pack.zig) create the IR Function via a different path
and left is_pure false, so a generic abi(.pure) instance bypassed the
emit bail and silently shipped a framed body — it returned 42 but
leaked the prologue's stack adjustment (the exact SP-in != SP-out
corruption the lock exists to prevent).

Both paths now set is_pure and route .pure bodies through the asm-only
+ unreachable cap, mirroring the decl path. Locked by
examples/1801-concurrency-pure-generic-bail.sx (generic .pure reaches
the loud bail).

The review's other CRITICAL (a .pure lambda) is a false positive:
isLambda's return-type scan (parser.zig:3652) breaks on the abi
keyword, so a .pure lambda is unparseable and parseLambda's abi
handling is never reached. Latent isLambda/parseLambda inconsistency,
not a B1 concern.

Suite green (723/0).
2026-06-20 14:45:29 +03:00
agra
dd363ca877 fibers B1.0a: plumb abi(.pure), emit bails (lock)
First implementation step of Stream B1 (fibers). Make the inert abi(.pure)
ABI carry an is_pure flag through lowering, with LLVM emission deliberately
bailing loudly until B1.0b — the lock half of the lock->green cadence.

- IR Function.is_pure, set from fd.abi == .pure at both declareFunction
  decl sites.
- funcWantsImplicitCtx skips .pure (no synthetic __sx_ctx, mirroring the
  .c skip): a pure fn reads args from ABI registers, an implicit ctx would
  occupy a register slot the asm doesn't expect.
- both body-lowering paths bypass lowerValueBody for .pure: lower the asm
  body as statements + cap with unreachable. A pure body has no sx return
  (the asm rets itself), so the implicit-return diagnostic must not fire.
- emit_llvm Pass 2 bails loudly when func.is_pure (build-gating nonzero
  exit) rather than emit a framed body, whose epilogue would corrupt a
  context switch's deliberate SP-in != SP-out.

examples/1800-concurrency-pure-asm.sx: one host example (no .build pin --
the bail fires before instruction selection, so it is host-independent),
locked to the bail snapshot. B1.0b flips emit to LLVM's naked attribute +
asm-only body and pins the example per-arch.

The sx-facing name is "pure" throughout (field, diagnostic); LLVM's naked
attribute is only the B1.0b lowering mechanism. Suite green (722/0).
2026-06-20 14:34:53 +03:00
agra
7044b8133b fibers: carve Stream B1 (PLAN-FIBERS + CHECKPOINT-FIBERS)
Carve the async-runtime fibers stream off PLAN-POST-METATYPE Stream B,
mirroring the atomics carve. Grounds the B1 compiler floor against the
tree:

- abi(.pure) exists in the ABI enum but is inert (type_resolver maps it
  to .default CC, emit emits no naked attr) -> B1.0 makes it emit LLVM
  naked + skip prologue/ctx. Corrected the design's callconv(.naked)
  spelling to the real abi(.pure).
- context is already an implicit *Context param (slot 0) + push Context
  is a stack alloca -> fiber-local for free; only shared root is the
  __sx_default_context global. B1.1 grounded as likely library-only
  (probe-first).
- B1.0 snapshot story corrected: naked body is raw per-arch asm -> two
  arch-gated examples (aarch64 + x86_64), not one host .ir.

Full xfail->green step detail + a B1.0a kickoff prompt. Baseline green
(721/0). No code change; first implementation step is B1.0a.
2026-06-20 14:16:39 +03:00
agra
3fad2d5a21 issue 0144: unrecognized $T-param #builtin silently returns 0
A bodiless #builtin with a $T: Type parameter that no recognizer matches folds
to 0 (exit 0) instead of erroring — while a non-type-param #builtin link-errors
loudly. Discovered during the atomics stream (Atomic methods ran to 0 before
recognition existed). The reflection/type-arg lowering path defaults instead of
rejecting (REJECTED-PATTERNS silent-fallback class). Repro + investigation prompt
in the issue. Open (unpinned — not added to the suite, since the repro currently
exits 0 by the bug).
2026-06-20 14:09:41 +03:00
agra
9bcb4159ef atomics: close out Stream A (feature-complete)
Final whole-stream adversarial review came back CLEAN (no CRITICAL/MEDIUM/LOW).
Close the one informational gap it noted: extend examples/1703 with a #run
comptime swap so swap's comptime VM arm is locked (742, matches runtime) — every
op now has comptime↔runtime corpus coverage.

Docs: PLAN-ATOMICS.md status banner (COMPLETE); PLAN-POST-METATYPE.md Stream A
marked done (unblocks B2-channels + C-parallel); readme.md gains a user-facing
Atomics section. Suite green (721/0).
2026-06-20 14:02:41 +03:00
agra
b65544a68c atomics A.3b: real swap (xchg) + fence emission + unit test (green)
emitAtomicRmw xchg arm (swap) and emitAtomicFence (LLVMBuildFence) now real.
examples/1703 (swap old=7/now=42, 'atomicrmw xchg') + 1704 (fence release/acquire/
seq_cst) green. Unit test 'emit: atomic swap (xchg) + fence'. Stream A
(atomics) is feature-complete: load/store, RMW (add/sub/and/or/xor/min/max),
compare_exchange[_weak], swap, fence. Suite green (721/0).
2026-06-20 13:51:36 +03:00
agra
fca4304f83 atomics A.3a: swap + fence ops + recognizer, emit bails (lock)
swap (atomicrmw xchg) and a standalone fence wired end-to-end except LLVM
emission (both bail loudly; A.3b makes them real).
- RmwKind += xchg; atomic_swap intrinsic + swap method reuse the atomic_rmw op.
- new atomic_fence op (+ AtomicFence) — ordering-only, void; fence($o)/atomic_fence
  intrinsic; recognizer rejects .relaxed (LLVM has no monotonic fence).
- comptime_vm: xchg = store operand/return old; fence = no-op (single-thread).
- examples 1703 (swap) + 1704 (fence) locked to bails; 1187 (relaxed-fence reject).
- 1186 converted to a direct-intrinsic call → stable user-file diagnostic span
  (the lib-forward-site span shifted when atomic.sx grew — fragile-snapshot fix).

Also fixes a latent A.2 comptime-CAS bug found while here: the success/null
has_value write was 'writeWord(addr, SIZE=0, val=1)' — a 0-byte no-op, correct
ONLY because allocBytes zero-inits (REJECTED-PATTERNS 'coincidentally correct').
Now writes the flag explicitly (size=1, val=0). Suite green (721/0).
2026-06-20 13:47:08 +03:00
agra
79895be401 atomics A.2b: real CAS emission (cmpxchg) + unit test (green)
emitAtomicCmpxchg: LLVMBuildAtomicCmpXchg (success/failure orderings,
singleThread=0) returns a {T, i1} pair; LLVMSetWeak for the weak variant. The
sx ?T result (null = SUCCESS) is built as { extractvalue 0 (actual value),
xor(extractvalue 1 (success), true) } -- has_value = NOT success. Integer-only
(recognizer guard), so never a pointer/niche optional.

examples/1702 green: successful CAS returns null (value updated), failing CAS
returns the actual value (unchanged), weak retry loop increments a counter
(100 -> 105). LLVM IR shows `cmpxchg ... acq_rel acquire` and `cmpxchg weak`.
Unit test `emit: atomic cmpxchg (strong + weak)` locks `cmpxchg` + the weak
marker. Suite green (718/0).
2026-06-20 10:57:01 +03:00
agra
dca396ed1f atomics A.2a: CAS ops + recognizer + methods, emit bails (lock)
compare_exchange/_weak wired end-to-end except LLVM emission (bails loudly;
A.2b makes it real). New IR op atomic_cmpxchg + AtomicCmpxchg{ptr, cmp, new,
val_ty, success_ordering, failure_ordering, weak}; result type = ?T (null =
SUCCESS, failure carries the actual value for retry). print arm; emit dispatch
-> emitAtomicCmpxchg (BAILS). comptime_vm arm does real single-thread CAS (read
actual / compare / store-on-equal / build ?T: success->none, failure->some;
weak == strong at comptime). Recognizer extended (atomic_cmpxchg/_weak, 6 args)
-- CAS restricted to INTEGER T (loud reject); BOTH orderings resolved via
atomicOrderingFromNode; dual-ordering validation (failure may not be
release/acq_rel nor stronger than success, via atomicOrderingRank). Methods
compare_exchange/_weak on Atomic($T) with comptime $success/$failure: Ordering.
examples/1702 locked to the bail; examples/1186 locks a rejected ordering pair.
Suite green (718/0).
2026-06-20 10:44:31 +03:00
agra
68ed732b79 atomics A.1c: fix comptime signed fetch_min/max (was unsigned compare)
Adversarial review CRITICAL: the comptime VM's atomic_rmw min/max arm called
@max/@min directly on Reg (=u64) values for SIGNED types, doing an UNSIGNED
compare — so comptime fetch_min/max on negatives diverged from the runtime LLVM
atomicrmw min/max (signed). Fix: reinterpret as i64 in the signed branch before
comparing, bitcast back (mirrors the unsigned branch + the emit-side signedness).

Closes the coverage gap that hid it: extend examples/1701 with signed min/max on
a negative at BOTH comptime (#run) and runtime — they now agree (3 / -5). Suite
green (716/0).
2026-06-20 10:32:50 +03:00
agra
05311646aa atomics A.1b: real RMW emission (atomicrmw) + unit test (green)
emitAtomicRmw: LLVMBuildAtomicRMW (binop from RmwKind; signed Min/Max vs
unsigned UMin/UMax from val_ty; singleThread=0; LLVM supplies ABI alignment).
examples/1701 green (add/sub/and/or/xor/min/max return old values, results
verified). Unit test 'emit: atomic rmw (add + signed/unsigned min)' locks
'atomicrmw add' + signed 'min' vs unsigned 'umin'. Suite green (716/0).
2026-06-20 10:19:44 +03:00
agra
718f27e27f atomics A.1a: RMW ops + recognizer + methods, emit bails (lock)
fetch_add/sub/and/or/xor/min/max wired end-to-end except LLVM emission (bails
loudly; A.1b makes it real). New IR op atomic_rmw + RmwKind (no nand) +
AtomicRmw{ptr, operand, val_ty, ordering, kind}. print arm; comptime_vm arm
implements real single-thread RMW (load/compute/store/return-old, signed|unsigned
min/max from val_ty). Recognizer extended (rmwKindFromName) — RMW restricted to
integer T (float fadd / pointer RMW out of scope, rejected loudly); all orderings
valid for RMW. Methods fetch_* on Atomic($T) with comptime $o: Ordering.
examples/1701 locked to the bail. Suite green (716/0).
2026-06-20 10:14:49 +03:00
agra
acf31839ea atomics A.0.5: full ordering surface (comptime $o: Ordering)
Migrate Atomic methods from seq_cst-only to the explicit ordering surface now
that comptime value params work on generic-struct methods (workers 3c4305f /
d7a6857 / d95ba0a):

- atomic.sx: load/store take a comptime $o: Ordering (explicit, Rust-style; no
  default, matching design 4.6). a.load(.acquire) -> 'load atomic .. acquire'.
- call.zig: atomicOrderingFromNode resolves a comptime-bound ordering identifier
  via comptimeIntNamed (+ atomicOrderingFromTag); documents the sx-Ordering <->
  IR-AtomicOrdering declaration-order invariant. The per-op validity guard fires
  through the method path (a.load(.release) is a compile error).
- 1700 migrated to explicit orderings (output unchanged 7/42/43).

Suite green (715/0).
2026-06-20 10:04:39 +03:00
agra
d95ba0a937 comptime value params: bind on generic-struct methods
A free function's $o comptime value param binds via lowerComptimeCall →
bindComptimeValueParams. The generic-struct-instance method path
(b.pick(.b)) took a different dispatch route: genericInstanceMethod →
ensureGenericInstanceMethodLowered emitted a plain call to the
monomorphized FuncId, never checking hasComptimeParams — so the method's
$o was never bound and lowered to 'unresolved o'.

Fix: when the selected generic-instance method declares comptime params,
route through the new lowerComptimeGenericInstanceMethod, which composes
the two mechanisms — installs the struct instance's type_bindings (so T /
*Box(T) resolve), pre-binds the receiver self as a normal pointer-param
alloca (so self.field reads work in the inlined body), then routes the
remaining ($) params through lowerComptimeCallArgsSkip(skip_params=1).
That reuses bindComptimeValueParams, so comptimeIntNamed /
comptimeValueRefNamed resolve the value param inside the method body,
identically to the free-function path.

lowerComptimeCall is refactored into lowerComptimeCallArgs(Skip) cores
parameterized over the effective arg-node slice + a leading skip count;
the original free-call entry point is unchanged behaviorally.

Loud-diagnostic behavior preserved: a non-constant / unknown-variant arg
still emits the value-param diagnostic, never a silent default. Int value
params ($n: i64) remain unbound — a pre-existing limitation shared with
free functions, orthogonal to this fix.

Locks examples/0642 (enum + tagged-union comptime value params on a
generic-struct method, incl. self.field read and comptimeIntNamed via a
type-position [o]i64).
2026-06-20 09:57:15 +03:00
agra
d7a6857ee1 comptime value params: generalize to tagged_union (+ aggregate hook)
`$s: <TaggedUnion>` now binds a constant variant-literal argument as a
compile-time-known value and resolves it in the inlined body — the
payload-bearing generalization of the enum value param (3c4305f). A bare
variant (`.point`) or a payload variant (`.circle(5.0)`) both bind:

  * the variant TAG goes into `comptime_value_bindings` (i64), so
    `comptimeIntNamed`/`if s == .circle` keep working and the param is
    readable in a TYPE position (`[s]i64`);
  * the full materialized `enum_init(tag, payload)` value goes into a new
    `comptime_value_ref_bindings` (param -> Ref) AND is scoped, so a
    payload read off the bound value (`s.rect`) resolves. A new
    lowering-time accessor `comptimeValueRefNamed(param)` reads it.

`bindEnumValueParams` is generalized to `bindComptimeValueParams`, which
switches on the constraint kind: `.@"enum"` -> tag-only bind,
`.tagged_union` -> tag + value bind. Other value kinds (struct/array
aggregates) are left with an explicit `else` (no silent default) and a
comment marking where the aggregate-const arm goes when a repro lands; a
non-constant arg / unknown variant is a loud, well-spanned diagnostic.

Locked by examples/0640-comptime-tagged-union-value-param.sx (bare +
payload variants, tag comparison, tag-as-dimension, payload read).
0627 (enum) stays green.
2026-06-20 09:39:10 +03:00
agra
8144a88a21 atomics A.0c: harden guards (scalar-kind, ordering validity, align bail)
Adversarial review of A.0 found two silent-wrong defects reachable via the public
atomic_load/atomic_store intrinsics (raw LLVM verifier errors, not clean sx
diagnostics) + a latent alignment fallback. All fixed:

- scalar-kind allowlist (call.zig): the size-only T guard admitted same-sized
  aggregates ([8]u8, 8-byte structs) -> invalid 'load atomic [8 x i8]'. Now an
  allowlist switch (integer/float/bool/pointer/enum/vector) rejects loudly.
- per-op ordering validity (call.zig): load cannot release/acq_rel, store cannot
  acquire/acq_rel -> loud diagnostic instead of invalid LLVM.
- val_ty align fallback (ops.zig): the 'else .i64' (align 8) default would
  over-align a sub-8 store -> now bails loudly on a missing val_ty.

Locked by examples 1130 (non-scalar) + 1131 (bad ordering). Suite green (713/0).
2026-06-20 09:26:53 +03:00
agra
3c4305f78f comptime enum value params: $o: EnumType binds+resolves variant tag
A comptime value param whose constraint is a plain enum ($o: Ord) now
binds its enum-literal argument to the variant tag during inlined
comptime-call lowering. The tag is recorded in comptime_value_bindings
(readable downstream via comptimeIntNamed / direct map lookup, and as an
array-dim style const-int leaf) AND the param is bound into scope as an
enum_init value so body comparisons like 'if o == .a' lower as ordinary
enum comparisons. Distinct ordering args monomorphize the inlined body
per value.

A non-constant argument or an unknown variant emits a loud diagnostic
and binds nothing — never a silent default.

Locked by examples/0627-comptime-enum-value-param.sx.
2026-06-20 09:19:18 +03:00
agra
64c7db5eb1 atomics A.0b: real seq_cst load/store emission (green)
Replace the A.0a emit bail with real LLVM atomic codegen:
- emitAtomicLoad: LLVMBuildLoad2 + LLVMSetOrdering + LLVMSetAlignment
- emitAtomicStore: LLVMBuildStore + LLVMSetOrdering + LLVMSetAlignment (value
  coerced to the pointee type, mirroring emitStore)
- llvmOrdering: explicit sx AtomicOrdering -> LLVMAtomicOrdering map (LLVM's enum
  is non-contiguous; never an identity cast)

examples/1700 now prints 7/42/43; IR is 'load atomic i64, ptr .. seq_cst, align 8'
+ 'store atomic ..'. Unit test 'emit: atomic load/store (seq_cst, aligned)' locks
the emission shape (load atomic/store atomic/seq_cst/align 8) without a fragile
full-module .ir snapshot. Suite green (710 examples + units).
2026-06-20 09:08:05 +03:00
agra
22af40413d atomics A.0a: lib + IR ops + recognizer, emit bails (lock commit)
Stream A (atomics) foundation. Net-new atomic load/store codegen path, wired
end-to-end except LLVM emission, which deliberately bails loudly so the example
locks to a clean diagnostic (A.0b turns it green — cadence: no commit both adds a
test and makes it pass).

- library/modules/std/atomic.sx: Ordering enum, Atomic($T) transparent wrapper
  (init/load/store, seq_cst-only for now), atomic_load/atomic_store #builtin
  intrinsics. Opt-in import, NOT in the universal std facade (Ordering in the
  prelude grows every program's type table + churns 37 .ir snapshots).
- IR: atomic_load/atomic_store ops + AtomicOrdering (all 5) + structs (inst.zig);
  print arms; comptime_vm arms reuse load/store (single-thread correct);
  recognizer tryLowerAtomicIntrinsic (const-ordering + scalar-size guards, both
  loud); emit dispatch -> emitAtomicLoad/Store bail via comptime_failed.
- examples/1700-atomics-load-store.sx locked to the bail diagnostic.

Full ordering surface (a.load(.acquire)) blocked on comptime-constant ordering
propagation (comptime enum value params) — A.0.5, migrated not legacy.
2026-06-20 08:47:07 +03:00
agra
ad1687c692 plan: correct grounded errors + harden async streams (post-metatype review)
Fold the adversarial-review corrections into the program plan + design-of-record:
- atomics is 100% net-new (no scaffolding; lower.zig 'ordering' is comparison-only)
- context is already an implicit *Context param (not TLS) — B1.1 rescoped
- abi(.pure) exists but is inert (no naked emission) — B1.0 rescoped
- B1.3 switch-stress harness is the first deliverable + mandatory stack guards
- Stream C gated on a named TSan/ASan + run-N stress harness, not a footnote
2026-06-20 08:47:07 +03:00
agra
f81d101fae checkpoint: P5.8 — Android + iOS-sim validated on emulator/simulator 2026-06-19 22:32:32 +03:00
agra
2ba36f6562 P5.8: add an Android .apk bundle smoke test to the corpus (first Android bundler coverage)
Mirror the macOS .app smoke test (1665) for Android. New `.build` `apk`
directive (ApkCheck = { out, bundle_id, expect }) cross-compiles via
`sx build --target android --apk ... --bundle-id ... -o lib*.so`, then
asserts the produced APK's zip entries (AndroidManifest.xml, classes.dex,
lib/arm64-v8a/) via `unzip -l`. Build+inspect only — aarch64-linux-android
can't execute on the host, so no exit/stdout/stderr snapshot; the apk
branch is self-contained and never falls through to stream comparison.

Gated on the Android SDK ($ANDROID_HOME / $ANDROID_SDK_ROOT /
~/Library/Android/sdk) AND a real JDK (`javac -version` exit 0 — the
macOS /usr/bin/javac stub fails the gate). Missing either → skip cleanly,
so a bare-host `zig build test` stays green. Cleanup rm -rf's the apk,
staged .so, .stage dir, .unaligned/.aligned intermediates, and the
apksigner .idsig sidecar.

Verified: default `zig build test` skips 1666 (709 examples ran, 0 failed;
476/476 unit). With JAVA_HOME set to Android Studio's jbr, 1666 RUNS and
PASSES (apk built + all three entries found).
2026-06-19 22:28:56 +03:00
agra
d8fb42501d comptime VM: support optional-of-word extern returns + string/any struct_init
Two host-FFI gaps surfaced by the sx Android bundler running on the VM
(default_pipeline calls env() -> getenv() -> ?cstring, and from_cstring builds
a string literal):

- callHostExtern: an extern returning an OPTIONAL whose child is a single
  register word (e.g. getenv() -> ?cstring) now wraps the bare C payload word
  into the {payload@0, has@sizeof(child)} optional aggregate (present iff
  non-null), mirroring emit_llvm's char*->?cstring handling. Previously bailed
  'non-word return'. The non-word bail now names the symbol + return type.
- struct_init: the builtin two-word aggregates string ({ptr,len}) and any
  ({tag,value}) can now be struct_init'd (e.g. string.{ ptr=, len= } in
  from_cstring). Previously bailed 'struct_init at a builtin result type'.

These let the full Android .apk bundling pipeline (javac/d8/aapt2/zipalign/
apksigner) run on the comptime VM. 709/0 corpus + 476/476 unit.
2026-06-19 22:14:57 +03:00
agra
3014c61236 docs: note [*]T does not implicitly coerce to []T (slice with ptr[0..len]) 2026-06-19 21:45:56 +03:00
agra
310461f651 checkpoint: P5.7 done — comptime VM is the sole evaluator, zero legacy
Update CHECKPOINT-COMPILER-API: Resume banner + Log entries for Step D
(metatype declare/define re-expressed as sx over the compiler-API) and the
empty-member-types-valid change. 709/0 corpus + 476/476 unit.
2026-06-19 21:45:27 +03:00
agra
538349611e comptime: empty-member types are valid for all kinds; keep never-defined declare rejected
A comptime-constructed type with NO members is now VALID for every kind
(empty struct, empty tuple, empty enum, empty tagged_union) — only a bare
`declare("X")` placeholder that is never completed by a matching `define`
stays rejected (it would panic codegen).

- comptime_vm.zig registerTypeVm: drop the blanket "a type with no members
  is never valid" rejection. The per-kind loops are vacuous for an empty
  member list and the dup-name checks stay correct.
- types.zig TaggedUnionInfo: add `defined: bool = true`. Every real
  construction (normal unions, error sets, register_type completion) is
  "defined" by default; only the two declare-PLACEHOLDER sites set it false:
  comptime_vm.declareNominal and lower/comptime.preregisterForwardTypes.
- lower/comptime.checkComptimeTypeResult: reject on `!defined` (never-defined
  placeholder) instead of `fields.len == 0`, so an explicitly-defined empty
  union passes through while a never-completed declare is still gated.
- types.zig typeSizeBytes(tagged_union): floor the payload area at 8 bytes
  when no field carries a payload, mirroring the LLVM lowering — fixes a
  verifySizes panic on an empty/all-void tagged_union (IR sized to tag-only,
  LLVM laid out tag + [8 x i8]).

Tests:
- examples/1179: repurposed from "empty enum rejected" (now valid) to the
  never-defined `declare` case (the remaining rejection); preserves its
  issue-0140 regression role.
- examples/1180 (duplicate variant): still rejected, unchanged output.
- examples/0641 (new): construct empty struct/tuple/enum/tagged_union via
  define/declare; instantiate the constructible ones; exit 0.
2026-06-19 21:41:07 +03:00
agra
ccba704378 P5.7 Step D: delete dead .define builtin arm, defineFromInfo, decodeTypeSlice
Now that define() is sx over register_type, remove the bespoke metatype define
surface from the comptime VM: the .define callBuiltinVm arm, the defineFromInfo
helper (kind-branching minting), and decodeTypeSlice (its only caller). Remove the
BuiltinId.define enum member. The .declare/.define interceptions in lowering and
their BuiltinIds are now gone; only type_info/field_type remain as metatype
builtins. register_type/decodeMemberSlice stay (shared by the sx define and the
compiler-API graph builder).
2026-06-19 21:14:05 +03:00
agra
7b1d8ceb83 P5.7 Step D: re-express metatype define() as sx over register_type
define(handle, info) is now an ordinary sx fn in modules/std/meta.sx: it matches
the TypeInfo union and calls the abi(.compiler) register_type primitive with the
matching kind code, decoding the variant/field/element list into []Member. An
all-void enum variant set registers as kind 2 (actual enum); any payload variant
as kind 3 (tagged_union).

To support matching the TypeInfo VALUE in the comptime VM, added tagged-union
value support: kindOf now treats tagged_union as a by-address aggregate, enum_tag
reads the tag word at offset 0, and a new enum_payload arm reads the active
payload at tag_size (both bail loudly on backing_type unions, whose layout
differs). register_type's duplicate-name diagnostics now include the offending
name. Dropped the define interception in tryLowerReflectionCall; the .enum(...)
arg infers TypeInfo from the sx fn's param type via the ordinary call path.

Regenerated 1179/1180 diagnostic snapshots (same span/line; the message now
names register_type instead of define()). define/type_info builtins still exist
pending dead-code removal.
2026-06-19 21:09:18 +03:00
agra
8850fcce70 P5.7 Step D: re-express metatype declare() as sx over declare_type
declare(name) is now an ordinary sx fn in modules/std/meta.sx that calls the
abi(.compiler) declare_type primitive — both mint/find the same forward nominal
slot. Removed the bespoke .declare arm from callBuiltinVm and the BuiltinId.declare
member; dropped the declare interception in tryLowerReflectionCall (the call now
routes to the sx fn). preregisterForwardTypes still scans for the literal
declare("Name") spelling so *Name self-references forward-register before the
body lowers (0618). define/type_info/field_type remain builtins.
2026-06-19 20:58:34 +03:00
agra
61f5700a36 P5.7 Step E: fix issue 0141 (reject silent [*]T -> []T coercion); land regression
The 0141 repro relied on a silent-wrong coercion: passing List.items (a
[*]T many-pointer, no length) to a []T parameter passed the bare 8-byte
pointer into a 16-byte {ptr,len} slot — garbage .len, at comptime a segfault
in the VM slice decoder (decodeMemberSlice), at runtime an LLVM verify failure.

Fix (root cause): classify [*]T -> []T as many_to_slice_reject in
conversions.zig and emit a build-gating diagnostic in coerce.zig telling the
user to slice with a length (ptr[0..len]). Guard runComptimeTypeFunc to skip
VM eval once diagnostics.hasErrors() — a type-fn body that failed coercion
holds malformed comptime data (a real host Addr) that would fault the VM's
Ref-level guards.

Land the corrected feature as examples/0640 (List-grown comptime enum via
vs.items[0..vs.len] -> green=7) and the rejection as
examples/1183-diagnostics-many-pointer-to-slice-rejected. Mark issue 0141
RESOLVED.

708/0 corpus + 476/476 unit.
2026-06-19 20:40:21 +03:00
agra
7b8be86834 P5.7 Step C: delete interp.zig — the comptime VM is the sole evaluator
The legacy tagged-Value Interpreter is gone. Relocate the Value result-DTO
+ decodeVariantElements into a new comptime_value.zig (the VM<->host
materialization boundary); repoint comptime_vm/emit_llvm/ir-barrel Value to
it and BuildConfig to compiler_hooks; delete the dead valueToReg bridge;
slim compiler_lib.zig to just the name registry (BoundFn{sx_name} + bound_fns
+ findFn — weldedCompilerFn only validates names); simplify printInterpBailDiag
to comptime_vm.last_bail_reason; drop the unused interp_mod import in lower.zig.
rm src/ir/interp.zig + interp.test.zig.

Value is relocated (not eliminated): it survives only as the slim result DTO
at the VM->valueToLLVMConst boundary; the execution-time marshaling the VM
pivot targeted is gone. Drop dead Value.asString/reflectTypeId.

706/0 corpus + 476/476 unit.
2026-06-19 20:05:57 +03:00
agra
103a156b26 P5.7 Step C2b: drop the Interpreter materialization context from emit_llvm
valueToLLVMConst / serializeAggregateValue took a *const Interpreter only to
resolve .heap_ptr data fields via interp.heapSlice — but the VM's regToValue
never produces .heap_ptr (it's interp-internal). Drop the param, remove the
dead interp_inst in emitGlobals, and drop the .heap_ptr fat-pointer data arm
(falls through to the loud bail). Remove the now-unused Interpreter alias.

500/500 unit + 706/0 corpus.
2026-06-19 19:52:44 +03:00
agra
cd8608c10c P5.7 Step C2a: fold inline comptime calls on the VM (ops.zig)
The emitCall inline comptime-call fold (zero-arg comptime callee -> constant)
was the last backend use of the legacy Interpreter. Route it through
comptime_vm.tryEval; a bail falls through to the normal call path. Drop the
interp_mod/Interpreter imports from ops.zig.

500/500 unit + 706/0 corpus.
2026-06-19 19:38:54 +03:00
agra
4d9f73f506 P5.7 Step C1: evaluate #insert on the comptime VM (sole evaluator)
evalComptimeString (the #insert lowering-time site) was the last user of
the legacy Interpreter.call. Route it through comptime_vm.tryEval instead:
the VM is hardened to bail (never panic) on malformed lowering-time IR
(0737's ret Ref.none), and regToValue dupes the result string into the
lowering allocator so it outlives the VM arena. Drop the now-unused
interp_mod / build_opts imports from comptime.zig.

500/500 unit + 706/0 corpus.
2026-06-19 17:25:12 +03:00
agra
64eb01918a P5.7 Step B2: remove the #compiler attribute + compiler_expr AST node
The #compiler struct attribute + #compiler-suffixed bodyless methods were
fully superseded by abi(.compiler) (P5.5) — no sx code uses them.

Remove the hash_compiler token (token/lexer/lsp), the is_compiler_struct /
struct_default_compiler parser machinery + the two compiler_expr body-
synthesis branches, the compiler_expr AST variant, and every
.builtin_expr/.compiler_expr switch arm + == .compiler_expr check across
sema/resolver/semantic_diagnostics/generic/decl/call/calls (kept .builtin_expr).
abi(.compiler) is untouched. Delete the obsolete calls.test.zig dispatch test.

500/500 unit + 706/0 corpus.
2026-06-19 17:18:45 +03:00
agra
e2971f272c P5.7 Step B1: remove the compiler_call IR op + the hook Registry
The compiler_call op + #compiler hook mechanism was fully superseded by
abi(.compiler) VM-native dispatch (P5.5) — no sx code emits it anymore.

Remove: the compiler_call op variant + CompilerCall struct (inst.zig); the
Builder.compilerCall emitter (module.zig); the two dead producer blocks in
lower/call.zig (compiler_expr-bodied free fns + methods); every consumer
switch arm (emit_llvm, ops.emitCompilerCall, print, interp dispatch); the
interp.hooks field + init/deinit. Strip compiler_hooks.zig down to the still-
live BuildConfig / BuildHooks / AssetDir (delete HookError/HookFn/Registry/
registerDefaults + all hookXxx, and the now-unused interp/Value imports).

Test refs that used compiler_call as a sample unported op now use vec_splat.

501/501 unit + 706/0 corpus.
2026-06-19 16:54:38 +03:00
agra
5d25e23143 P5.7 Step A: VM is the sole comptime evaluator at emit-time + type-fn sites (no fallback)
Remove the comptime_flat/need_vm gate and the vm_result-orelse-legacy
fallback from emit_llvm.zig (runComptimeSideEffects + emitGlobals const-init)
and comptime.zig (runComptimeTypeFunc). The comptime VM now always runs;
a bail is always a build-gating diagnostic, never a fallback. Delete the
now-moot entryNeedsVm. runComptimeSideEffects drops the Interpreter entirely
(VM writes #run output direct to fd 1); emitGlobals keeps a fresh interp_inst
only as the valueToLLVMConst materialization context (the regToValue bridge,
removed with interp.zig in a later step).

#insert (evalComptimeString) still routes through the legacy interp — deferred
until interp.zig deletion.

Reconcile 1654: the comptime asm-global #run now reports the VM's clean dlsym
bail instead of the legacy CannotEvalComptime wrapper (exit still 1).

501/501 unit + 706/0 corpus.
2026-06-19 16:44:52 +03:00
agra
ab8f0d41bb checkpoint: macOS .app corpus smoke test done (706/0); top-risk bundler-coverage gap closed 2026-06-19 16:06:20 +03:00
agra
445ae9705c P5.8: add a macOS .app bundle smoke test to the corpus (closes the no-bundler-coverage gap)
The corpus had ZERO bundler coverage (the stream's named top risk). Add a `.build`
`bundle` directive to the corpus runner: after a successful `aot` build it asserts
each `expect` entry exists under the produced `.app` (repo-relative), then `rm -rf`s
it. macOS-host only — the `.app` + codesign are Apple-specific, so the example is
skipped on other hosts.

`examples/1665-platform-macos-bundle-smoke.sx` sets `bundle_path`/`bundle_id` via a
`#run` config; `default_pipeline` auto-bundles (build.sx imports the bundler, no
explicit `on_build` needed). The directive asserts `Contents/MacOS`,
`Contents/Info.plist`, `Contents/_CodeSignature`. Verified: passes on BOTH gates
(the bundler runs on the legacy interp AND the VM), the `.app` is cleaned up, and a
bad `expect` entry correctly fails (the check is not vacuous). Unit test +
CLAUDE.md `.build`-directive docs updated. 706/0 both gates.
2026-06-19 16:06:02 +03:00
agra
224478fabf checkpoint: P5.8 partial — m3te + distribution validated with the new build pipeline 2026-06-19 15:40:51 +03:00
agra
a91b6e8ae0 checkpoint: P5.6 macOS bundling via default_pipeline + 0125 fix done; remaining iOS/Android validation 2026-06-19 15:32:39 +03:00
agra
48eb7bf48a P5.6 (macOS): default_pipeline drives bundling; fix issue 0125 (array-format blowup)
build.sx now `#import`s the sx bundler and `default_pipeline` delegates to its
`bundle_main` when a bundle was requested (emit + link, then wrap the binary into
the `.app`/`.apk`); otherwise it just emit+links via the shared `emit_and_link`
core. The Zig `--bundle`/`post_link_module` dispatch shim is removed — the CLI
bundle flags only feed `BuildConfig`, and `default_pipeline` branches on
`bundle_path()`. Validated end-to-end on macOS: `sx build --bundle App.app
--bundle-id … foo.sx` on a plain program AND auto-bundle from `set_bundle_path`
both produce a valid signed `.app` (correct `Contents/MacOS/` layout, Info.plist,
passes `codesign`, binary runs). Also fixed a pre-existing host-build bug:
target_triple was left empty for host builds → `is_macos()` false → wrong flat
layout; main.zig now exposes the host triple when `--target` is absent.

bundle_main no longer re-calls `build_options()` (the handle is already its `opts`
param).

Fix issue 0125 (root cause): the type-match dispatcher unboxed each interned array
tag to the concrete array type — a whole-array load — and passed it to
`array_to_string` by value, which LLVM scalarized into one SelectionDAG node per
element (~12s / segfault at [65536]u8). The bundler's `format("…{}…")` instantiates
`any_to_string`, so importing it into the prelude surfaced 0125 for any large-array
program. Fix (route 1): `any_to_string`'s `case array:` arm calls `slice_to_string`,
and `lowerRuntimeDispatchCall` detects an ARRAY tag bound to a SLICE param and builds
a `{ptr,len}` slice VIEW of the payload pointer (`unbox_any → [*]elem` is an
int-to-ptr with NO load, paired with the array length) instead of loading the array.
Output is byte-identical (`[a, b, c]`). Pinned as
examples/0056-basic-large-array-format-no-blowup.sx; 0055 drops 12s → 0.2s.

37 `.ir` snapshots regenerated (build.sx now pulls in the bundler's types + the
array-format lowering changed); verified `.ir`-only, zero behavior-stream diffs.
705/0 both gates.
2026-06-19 15:32:07 +03:00
agra
88730aa337 checkpoint: P5.5 + P5.6 bitwise/shift prereq done; record remaining P5.6 bundler-restructure 2026-06-19 14:21:03 +03:00
agra
994d6498fc P5.6 prereq: port bitwise/shift ops into the comptime VM
`comptime_vm` exec now handles `bit_and`/`bit_or`/`bit_xor`/`bit_not`/`shl`/`shr`
(a new `bitwise` helper next to `arith`), mirroring the legacy interp's i64 model
exactly: the shift amount clamps to `@min(rhs, 63)` and `shr` is an arithmetic
right shift (sign-extending).

These were unported and bailed; the `shr` gap surfaced via the iOS-device bundler
once P5.5 let it run further (1616). With the port, 1616's strict VM run reaches
the real bundler logic and stops only at the genuinely-unavailable iOS runtime on
macOS (`_UIApplicationMain` / no linked binary under `sx run`), as expected.

New corpus test `examples/0639-comptime-bitwise-shift.sx` folds AND/OR/XOR/NOT/
shl/shr/arith-shr as `::` consts — identical on both evaluators. 704/0 both gates.
2026-06-19 14:20:37 +03:00
agra
ba28488d99 P5.5: migrate the 35 BuildOptions accessors off #compiler to VM-native abi(.compiler)
`BuildOptions :: struct #compiler { ...35 methods... }` becomes
`BuildOptions :: struct { }` (an opaque null-sentinel handle) plus 35 free
`ufcs (self: BuildOptions, …) abi(.compiler)` decls in build.sx, each serviced
by a new `comptime_vm.callBuildOptionFn` arm (off `callCompilerFn`). No legacy
`compiler_lib` handler: the names are registered in `bound_fns` with a single
bailing stub only so `weldedCompilerFn` accepts them.

- String lifetime: setters dupe the arg into the persistent `Vm.gpa` (the
  Compilation allocator, threaded into both `tryEval` and `runBuildCallback` —
  not the per-eval VM arena) and write/append to the threaded `BuildConfig`.
  Getters read the field/slice or compute the target predicate from the triple.
- Dispatch routing (Option B): a `#run`/const-init entry that directly calls a
  compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with no
  legacy fallback regardless of the `-Dcomptime-flat` gate, so gate-OFF stays
  green without a legacy BuildOptions handler (P5.7 retires the legacy interp).
- Mark the 5 `platform/bundle.sx` getter-calling helpers `abi(.compiler)` (they
  are comptime-only bundler code; otherwise their now-welded getter calls trip
  the runtime-call gate).
- 37 `.ir` snapshots regenerated (std transitively imports build.sx → string-
  pool/type-table indices shift); verified `.ir`-only, zero behavior-stream diffs.

BuildOptions `compiler_call` strict bails gone (1609/1614/1615 strict-clean);
1616 now bails on a separate, pre-existing unported bitwise/shift VM gap (`shr`),
to port first in P5.6. 703/0 both gates.

Also sweep the outdated "flat memory" terminology to "comptime/byte-addressable"
across comptime_vm + the plan/checkpoint/CLAUDE docs: the comptime VM is
arena-backed, byte-addressable memory where `Addr` is a real host pointer, not a
flat contiguous address space (flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
2026-06-19 13:21:09 +03:00
agra
af32c3823c plan: final direction — full migration, no legacy; all bundling/codesign in default_pipeline
Records the user's decision: drop gate-OFF entirely (VM is the sole comptime
evaluator; delete interp.zig); migrate BuildOptions directly to VM-native
abi(.compiler) arms with NO legacy handlers; ALL bundling + code signing for
every target (macOS/iOS-device/iOS-sim/Android) lives in the sx default_pipeline;
validate against ~/projects/m3te + ~/projects/distribution. Phase 5 steps
P5.5-P5.8 in PLAN-COMPILER-VM.md.
2026-06-19 09:49:17 +03:00
agra
37c982467b checkpoint: P5.4 core done; record remaining BuildOptions-migration plan 2026-06-19 09:42:45 +03:00
agra
65ac370683 P5.4: migrate all callers to on_build; delete set_post_link_callback
on_build is now the sole post-build callback mechanism. Migrated the 9 callers
(0602/0603/1611/1614/1615/1616 + the platform bundle_main) from
opts.set_post_link_callback(cb) to on_build(cb), giving each callback the
(opt: BuildOptions) param. Deleted set_post_link_callback from build.sx,
compiler_lib (bound_fns + handleSetPostLinkCallback), and the VM arm.

Reworked the P5 smoke tests for the new semantics: an on_build override REPLACES
the build (must emit+link or delegate), unlike the old post-link callback which
ran after the auto-link. 1662 (queries) + 1664 (override+List-grow) now delegate
to default_pipeline for the real build; deleted 1661/1663 (the primitives are now
exercised by every AOT build). bundle_main invoked with pass_options=true.

Benign 37-.ir churn (build.sx shrank). 703/0 both gates.
2026-06-19 09:37:05 +03:00
agra
d178454841 P5.4 core: drive the whole build from sx default_pipeline (no auto-emit/link)
The compiler's post-IR role shrinks to: codegen -> invoke the build callback.
There is NO Zig auto-emit / auto-link anymore; emit + link are sx-called actions.

- emit_object() is now an ACTION (verify + emit via a host BuildHooks vtable),
  returning the object path. New query primitives build_output/build_target/
  build_frameworks/build_flags (data reads from the merged BuildConfig).
- library/modules/build.sx imports compiler.sx and defines default_pipeline:
  emit_object -> gather c_object_paths -> link(objs, output, libs, fws, flags,
  target). The std<->build import cycle is handled by the resolver.
- The compiler FORCE-LOWERS default_pipeline (well-known name) and AUTO-INVOKES
  it post-codegen when no on_build/set_post_link_callback override was
  registered (driver's final fallback: invokeByName default_pipeline).
- Prelude-less programs (e.g. asm tests) don't import build.sx, so the BUILD
  path auto-imports modules/build.sx (idempotent if already transitive) so
  default_pipeline is always available. JIT sx run is untouched (emits in-process).
- Removed the build cache short-circuits (incompatible with the always-run sx
  driver; a future cache can live in default_pipeline).

Benign 37-.ir churn (build.sx grew); zero behavior changes (verified diff is
.ir-only). 705/0 both gates.
2026-06-19 09:22:54 +03:00
agra
1f796e92ec checkpoint: record P5.3 on_build + consolidated P5.4 plan 2026-06-19 08:48:32 +03:00
agra
9cbee5e4bd P5.3: on_build(cb) build-callback registrar; callback takes BuildOptions
Per user design: on_build(build) is the build-callback registrar (a free fn),
generalizing set_post_link_callback — the callback is (opt: BuildOptions) ->
bool and the compiler invokes it post-codegen WITH the BuildOptions handle.

- VM: callCompilerFn 'on_build' arm + legacy handleOnBuild, both set
  post_link_callback_fn + a new BuildConfig.post_link_takes_options flag.
- comptime_vm: runEntry refactored to runEntryArgs(extra) (implicit ctx +
  explicit args); new public runBuildCallback(..., pass_options) passes the
  opaque BuildOptions handle (one word) after the ctx. The fat-config
  marshaling fear is moot — the handle is a single null-sentinel word.
- core.invokeByFuncId/invokeByName take pass_options (was an unused args
  slice); main.zig passes comp.getPostLinkTakesOptions().
- build.sx: on_build decl (set_post_link_callback kept for now).

Smoke test examples/1664-platform-on-build-callback (AOT): #run on_build(build)
with build :: (opt: BuildOptions) -> bool; the callback is invoked with the
handle arg (runEntryArgs param-count match) and runs the primitives.

Benign .ir churn (37 snapshots: type table +1 for the on_build fn type +
global renumber; behavior identical). 705/0 both gates.
2026-06-19 08:47:05 +03:00
agra
d8affd45e8 rename std/build.sx -> modules/compiler.sx (the compiler-API surface)
Per user direction: the low-level abi(.compiler) primitive surface is the
comptime 'compiler' library, so name the file compiler.sx (a peer of build.sx)
instead of the interim std/build.sx — which also frees the 'build' name for the
default build IMPLEMENTATION (default_build + on_build slot), which will live in
modules/build.sx alongside the BuildOptions DSL.

Updated the two example imports + the plan's Phase 5 file-split note. 704/0
both gates.
2026-06-19 08:17:35 +03:00
agra
f7362ee013 P5.2b: link() build-pipeline action on the VM via a host vtable
The one genuine action primitive: link(objects, output, libraries, frameworks,
flags, target) in library/modules/std/build.sx. Per the user decision to drop
fallibility from the build callback, link is plain VOID — a link failure bails
on the VM (hard build error), no -> ! / failable-tuple needed.

comptime_vm.zig can't depend on the driver (core/main/target), so link
dispatches through a new compiler_hooks.BuildHooks { ctx, link } vtable that
main.zig installs into BuildConfig.build_hooks before the post-link callback.
The driver side is main.LinkHooksCtx (unions explicit + CLI link flags, calls
target.link). New VM readers readStringList / readStringArg (inverse of
makeStringList) decode the List(string)/string args from flat memory.

Smoke test examples/1663-platform-build-pipeline-link (AOT): a post-link
callback re-links the build's own objects (c_object_paths + emit_object) into a
temp output via sx link — the relinked binary is a functional executable that
runs. Negative-probe verified (bad path -> ld fails -> ComptimeVmBail -> build
exit 1). The Zig driver still auto-links; removing that is P5.4.

704/0 both gates.
2026-06-19 08:11:36 +03:00
agra
83de0fa04d P5.2: emit_object() -> string query primitive
The compiler emits the sx object eagerly (the Zig driver, before the post-link
callback), so emit_object is a QUERY (not an action): it returns the path from
a new BuildConfig.object_path field main.zig forwards — no driver vtable. This
completes the build-pipeline QUERY primitives (emit_object / c_object_paths /
link_libraries); only link (the genuine action) remains for the vtable step.

Extended examples/1662 to also assert emit_object().len > 0. 703/0 both gates.
2026-06-19 07:58:59 +03:00
agra
44dfdcddf9 P5.2 metadata queries: c_object_paths / link_libraries on the VM
Two abi(.compiler) build-pipeline primitives the sx driver will pass to link:
- c_object_paths() -> List(string)  (#import c companion objects)
- link_libraries() -> List(string)  (#library names)

They live in a new stdlib home library/modules/std/build.sx and are serviced
by comptime_vm.callCompilerFn reading two new BuildConfig fields that main.zig
forwards before the post-link callback. New reusable VM helper makeStringList
builds a List(string) in flat memory from the call's result type offsets
(target-aware); invoke/callCompilerFn now thread ins.ty for that. Legacy
handlers bail loudly (VM-only by nature — post-link; List(string) isn't
faithfully buildable in the legacy Value model, 0141).

Smoke test examples/1662-platform-build-pipeline-queries (AOT + a 1-line C
#source → one object): a post-link callback verifies the VM-built list is
well-formed; build exit 0 only if so (negative-probe confirmed a real guard).

emit_object + link (the actions) deferred to P5.2b — they replace the Zig
driver's auto-emit/auto-link and need a host-installed callback vtable.

703/0 both gates.
2026-06-19 07:42:27 +03:00
agra
7cba33ea6d P5.1: post-link build driver runs on the comptime VM (no fallback)
core.invokeByFuncId routes the post-link callback through comptime_vm.tryEval
instead of the legacy Interpreter. REQUIRED because the sx build driver
allocates/grows Lists, which the legacy interp can't do at comptime (issue
0141: struct_get: base has no fields); the VM can. No fallback (a
side-effecting post-link callback can't double-execute): a VM bail is a hard
build error (comptime_vm.last_bail_reason, surfaced by printInterpBailDiag).
BuildConfig + import_sources threaded in; non-empty args rejected loudly.
flushInterpOutput deleted (VM out writes direct via host-FFI).

Smoke test examples/1661-platform-post-link-vm-list (AOT): a post-link
callback grows a List to 3 + returns len==3, so the build succeeds (exit 0)
only via the VM. First corpus coverage of the post-link path.

702/0 both gates.
2026-06-19 07:20:42 +03:00
agra
2060373c16 comptime VM arc: abi(.compiler) ABI, out as sx fn, VM-native diagnostics, BuildConfig threaded
Lands the full VM/compiler-API arc on branch reify (701/0 both gates):
- abi(.compiler) ABI replaces abi(.zig) extern compiler + the fake
  #library "compiler"; bodiless decl = compiler-API surface, bodied =
  user compiler-domain fn (lowered for VM eval, emit-skipped).
- out is a plain sx fn (libc write) — the out builtin deleted; the VM
  handles it via host-FFI. trace_resolve + interp_print_frames ported.
- 4B VM-native diagnostics: 1179/1180 render proper comptime type
  construction failed: under strict.
- S5a: build_options/set_post_link_callback on abi(.compiler) with
  BuildConfig threaded into the VM (green intermediate).
- 0522 fixed (describe(args: []Type)); regression 0638.

Strict deletion-gate down to 4 compiler_call bails (1609/1614/1615/1616)
+ 1654 (legitimate unresolvable-symbol diagnostic).
2026-06-19 07:04:10 +03:00
agra
fdc4ee2331 CHECKPOINT-COMPILER-API: record issue 0143 RESOLVED 2026-06-18 19:43:18 +03:00
agra
f807436f04 fix issue 0143: pack-as-[]Type built as []Any — build it as []type_value
buildPackSliceValue (lower/pack.zig) materialized a bare `$<pack>` []Type slice
as []Any (16-byte elements) — a stale mapping from before the dedicated Type
builtin (.type_value, 8 bytes) replaced Type -> .any. It stored 8-byte const_type
words into 16-byte slots, so a []Type reader (8-byte stride) read [t0, pad, t1, ...]
instead of [t0, t1, ...]. The legacy interp's tagged-Value model hid it; the
byte-accurate comptime VM exposed it (a `..$args` pack forwarded as a []Type
argument across a call read its elements shifted/garbled).

Fix: build the pack-slice array + slice as .type_value (8 bytes). Removed the
stopgap type_name .unresolved guard (379ed05) now that the root cause is fixed.

Regression test examples/0525-packs-pack-as-type-slice-arg.sx (outer(42,"hi",true)
-> inner($args: []Type) -> "i64 string bool"). 700/0 both gates; issue 0143 RESOLVED.
2026-06-18 19:42:42 +03:00
agra
a446550013 issues: file 0143 (pack-as-[]Type stride mismatch) + record out is end-state-only
0143: a ..$args pack forwarded as a []Type argument across a call is backed by a
[N x Any] (16B) array but viewed as []type_value (8B) -> half-stride reads. A
lowering bug the legacy Value model masks; the byte-accurate VM exposes it. Blocks
examples/0114 on the VM. Filed per CLAUDE.md (not worked around; the type_name
.unresolved guard only makes the VM decline rather than emit garbage).

Checkpoint also records the sequencing insight: comptime `out` (print) can only
land once the fallback is removed (a print-then-bail double-prints under the legacy
re-run), so side-effecting ops + fallback-removal are the FINAL step; pure ops +
migrations land first.
2026-06-18 19:28:04 +03:00
agra
379ed05495 comptime VM: switch_br + type_name (pure reflection ops); guard unresolved type reads
Port two pure (side-effect-free) comptime ops toward the no-fallback gate:
- switch_br: multi-way branch on an i64 discriminant (enum/error tag, or a
  .type_value whose word is its TypeId index) — mirrors the legacy
  asInt-orelse-asTypeId switch.
- type_name(x): a Type value (.type_value word) or Any box ({tag,value}; tag ==
  type_value means the boxed Type's id is in the value slot) -> table.typeName.
  Guards an .unresolved TypeId (bail, not "<unresolved>") to surface a bad
  slice/pack read instead of emitting garbage (see issue 0143).

These are correct in isolation (0520-0524 run green under strict mode) but flip
nothing yet because their examples also print via `out`, which can only land at
the end state: under the legacy fallback a print-then-bail double-prints (no
re-run rewind), so `out` is deferred until the fallback is removed. 699/0 both
default gates.
2026-06-18 19:26:59 +03:00
agra
dcb1392255 comptime VM: strict no-fallback mode — the interp-retirement enumeration gate (Phase 4)
Add -Dcomptime-flat-strict / env SX_COMPTIME_FLAT_STRICT (implies comptime_flat):
at all three comptime sites (type-fn in lower/comptime.zig, const-init + #run in
emit_llvm.zig) a VM bail becomes a build-gating error naming the reason INSTEAD of
falling back to legacy. Forces every comptime eval onto the VM so the complete gap
set is enumerable in one sweep; when the corpus is green under strict mode AND every
example matches legacy, interp.zig can be deleted.

Default behaviour unchanged (699/0 both default gates). Fixed a wiring bug: the
type-fn site's local comptime_flat didn't include the strict flag (every type-fn
falsely reported <unknown>); strict now implies flat there too.

Swept the gap list (19 strict bails): switch_br (5, + unmasks a []Type-across-call
silent-wrong in 0114), compiler_call (6, = the BuildOptions->abi(.zig) extern
compiler migration), out (2), type_name (1), global_addr (1), interp_print_frames
(1), 2 negative-test diagnostics (1179/1180), 1 dlsym (1654). Recorded as the
deletion checklist in CHECKPOINT-COMPILER-API.md.
2026-06-18 19:06:51 +03:00
agra
da6a8423c7 plan/checkpoint: #compiler/compiler_call is DELETED not bridged — BuildOptions → abi(.zig) extern compiler
Direction correction (user): the VM does NOT get a transitional compiler_call →
hook-registry shim. BuildOptions is re-expressed as abi(.zig) extern compiler
functions (the compiler-API the VM already dispatches via callCompilerFn), and the
#compiler attribute + compiler_call op + Value-based hook Registry are deleted.

Also record that 4D (host FFI) is DONE via the arena/absolute-pointer allocator
(the earlier pin/tag hazard is moot — the arena never moves an allocation), and
that 0602/0603 stay on legacy fallback until the BuildOptions migration lands.
No code change (reverted the speculative compiler_call bridge).
2026-06-18 18:19:44 +03:00
agra
b05c74f2f1 CHECKPOINT-COMPILER-API: record 4D.0-4D.2 done + precise 4D.3 (compiler_call) restart notes
Update the resume banner: box_any/unbox_any (4A.1), arena allocator with absolute
host-pointer Addr (4D.0), general host-FFI escape (4D.1), slice/string args + float
guards (4D.2) all landed and green (699/0 both gates). Next is 4D.3 (#compiler
hooks / compiler_call) with the hook-ABI + build_config findings recorded for a fast
restart.
2026-06-18 18:11:25 +03:00
agra
6a7f6902b8 comptime VM: extern slice/string args (-> NUL-term char*) + float guards (Phase 4D.2)
Extract marshalExternArg: a scalar/pointer word passes verbatim (a cstring arg
already works as a pointer word via 4D.1); a string/slice {ptr,len} fat pointer is
copied into a NUL-terminated arena buffer and its char* passed -- mirrors the legacy
marshalExternArg, and is what the bundler's popen(cmd: [:0]u8, ...) needs.

Add float guards on args AND returns: floats are kindOf == .word but the host_ffi
trampolines have no float variant, so bail loudly rather than miscall through an
integer register (the legacy interp doesn't support float FFI either -> parity).

New example 0637-comptime-extern-slice-arg (#run strlen("hello, world") with a
[:0]u8 param -> 12) runs HANDLED on the VM, byte-matching legacy. 699/0 both gates.
The FFI escape now covers scalar/pointer/cstring/slice args + scalar/pointer returns.
2026-06-18 18:09:46 +03:00
agra
e7a8708287 comptime VM: general host-FFI escape — call any extern libc fn via dlsym + host_ffi (Phase 4D.1)
Replace the "extern not ported -> bail" stub in Vm.invoke with callHostExtern:
resolve the symbol via host_ffi.lookupSymbol (dlsym RTLD_DEFAULT) and dispatch
through the host_ffi trampolines, like the legacy interp.callExtern.

Marshalling is trivial now that Addr is a real host pointer (4D.0): every WORD-kind
arg passes as usize verbatim (a scalar's bits OR a pointer, no translation), and a
pointer return is a valid Addr. Picks callPtrRet (void*-ABI) for pointer-ish
returns, callIntRet (i64-ABI) otherwise; honors variadic. Non-word
(aggregate/string/float) args+returns bail loudly (4D.2 adds them). One general
mechanism for all externs, not per-builtin special cases.

New example 0636-comptime-extern-libc (#run toupper(97)/tolower(90) -> 65/122) runs
HANDLED on the VM, output byte-matching legacy. 698/0 both gates.
2026-06-18 18:00:07 +03:00
agra
625ba0fb27 comptime VM: memory = arena of stable host allocations; Addr = real host pointer (Phase 4D.0)
Replace the growable ArrayList(u8) flat buffer (reallocs/MOVES on growth) with a
std.heap.ArenaAllocator. Each allocBytes is a separate arena allocation that never
moves and is freed wholesale on deinit -- no per-object free, no cap, no fixed
buffer. Addr is now the allocation's ABSOLUTE host pointer (@intFromPtr), not an
offset, so a flat-memory pointer and an FFI-returned host pointer are the same kind
of value -- the FFI bridge (4D.1) passes them to/from libc with zero translation and
no per-call pinning (the moving-buffer hazard is gone by construction).

readWord/writeWord/bytes deref the absolute pointer with a null-check bail (the
malformed-IR / null-deref safety contract). Dropped the offset-based upper-bounds
check (can't bound an absolute pointer; Frame.bad_ref still catches the dominant
malformed-IR vector) and the test-only mark/reset (arena has no reset-to-mark; the
VM never used them outside tests).

697/0 both gates + all unit tests (rewrote the two Machine tests). Pure refactor, no
comptime behavior change.
2026-06-18 17:51:49 +03:00
agra
1526d198e2 comptime VM: box_any/unbox_any + .any as a 16-byte flat-memory aggregate (Phase 4A.1)
Ported the Any-boxing conversion pair:
- box_any: alloc the 16-byte { type_tag@0, value@8 } box, tag = source TypeId
  index (matches the legacy comptime interp; runtime anyTag also normalizes
  arbitrary-width ints). Value slot holds a word source's scalar bytes (via
  writeField(source_type) so f32 round-trips) or an aggregate source's
  flat-memory ADDR (the runtime pointer-in-value-slot shape).
- unbox_any: read the value slot back (word -> readField; aggregate -> the
  stored ADDR).

Required promoting .any to a first-class flat-memory aggregate (was
kindOf -> .unsupported): kindOf(.any) = .aggregate (16B, by-address) and
fieldOffset special-cases .any to the {@0, @8} layout (shared with
string/slice). Without the latter a struct_get on an Any panicked
(union field 'struct' while 'any' is active) -- caught + fixed, no crash.

Updated two unit tests that used unbox_any as the "unported op" example ->
compiler_call; added a box->unbox round-trip test. 697/0 both gates + all
unit tests. The 6 box_any examples no longer bail at box_any (output matches
legacy) but fall back further at switch_br/type_name/out (later 4A steps).
2026-06-18 16:56:50 +03:00
agra
3283effa97 plan: Phase 4 — retire the legacy interp (ONE-evaluator end state)
Audited the 5 roles interp.zig still serves (A comptime folds, B #insert,
C post-link bundler, D #compiler hooks, E bail diagnostics) and the shared
substrate (Value tagged union + host_ffi bridge).

User decision: UNIFY — the VM gains a host-FFI escape + real-pointer
translation and runs the post-link bundler too; interp.zig fully deleted.

Dependency-ordered sub-phases recorded in PLAN-COMPILER-VM.md:
4A finish comptime ops (box_any/unbox_any, out/print, global_addr, trace) ->
4B VM-native diagnostics -> 4C #insert -> 4D host FFI + #compiler hooks ->
4E post-link bundler (+ dedicated bundle tests) -> 4F flip default + delete
interp.zig/Value + re-express define/make_enum over the compiler-API.

No code change — planning + checkpoint only.
2026-06-18 16:45:25 +03:00
agra
736f64e664 comptime VM: VM-native type_info REFLECTION — whole metatype surface HANDLED (P3.4 step 8)
Ported type_info($T) into the VM (callBuiltinVm .type_info arm -> new
buildTypeInfo), the inverse of step 7's define: reflect a type INTO a TypeInfo
VALUE built in flat memory (VM-native mirror of legacy reflectTypeInfo).

- Decodes the source type into a tag + members (tagged-union/struct field &
  enum variant -> {name, ty}, payloadless variant -> void; tuple -> bare
  positional Types), then lays the nested value out bottom-up using layouts
  derived from the TypeInfo RESULT type (ins.ty, now threaded into
  callBuiltinVm): element array -> {ptr,len} slice -> info struct
  (EnumInfo/StructInfo/TupleInfo) -> TypeInfo {tag, payload} tagged union
  (reusing step 7's tagged-union write).
- Variant/field names materialize via makeStringValue, extracted from text_of.
- Same backing_type guard as step 7 (bail rather than mis-read the tag).

The ENTIRE metatype surface now runs HANDLED on the VM with ZERO fallback:
0614-0624 + 0632 (0616 field_type folds at lower time). The
define(declare, type_info(T)) round-trips (0619/0622/0623) mint byte-identical
copies on the VM; VM output byte-matches legacy for all. 697/0 both gates + all
unit tests. Remaining VM fallbacks in the comptime corpus are now genuinely
non-metatype emit-time side effects (print/global_addr/compiler_call/inline-asm).
2026-06-18 15:57:11 +03:00
agra
d0ebc55f99 comptime VM: VM-native metatype CONSTRUCTION — declare/define + tagged-union enum_init (P3.4 step 7)
The metatype type-construction builtins now run natively on the flat-memory
VM, so the construction examples run HANDLED end-to-end (no call_builtin
fallback to the legacy interp).

- Tagged-union enum_init WITH payload: allocate zeroed, write the tag at
  offset 0, copy the payload at tag_size ({ header, [N x i8] } layout).
- New .call_builtin exec arm -> callBuiltinVm (VM-native mirror of the legacy
  execBuiltinInner): declare(name) mints an empty forward nominal slot (shared
  declareNominal, also used by declare_type); define(handle, info) reads the
  TypeInfo tagged-union VALUE from flat memory and mints via defineFromInfo,
  a faithful port of legacy defineEnum/defineStruct/defineTuple (all-void enum
  -> real .enum per issue 0142, dup-name rejection, updatePreservingKey vs
  replaceKeyedInfo). Unmodeled builtins bail -> legacy fallback (dual-path).
- Refactored the []{name,ty} decode out of registerTypeVm into a shared
  decodeMemberSlice (+ decodeTypeSlice for bare-Type tuple elements).
- Correctness guard: enum_init/define assume a tag-headed layout, wrong for a
  backing_type tagged union (laid out as the backing struct) — both now bail
  loudly on backing_type != null rather than silent-clobber.

Examples 0614/0620/0621/0624/0632 run fully HANDLED on the VM; 0622/0623 run
define HANDLED then fall back at the still-unported type_info. VM output
byte-matches legacy for all 7. 697/0 both gates + all unit tests (added:
tagged-union enum_init payload layout).
2026-06-18 15:48:48 +03:00
agra
eb68d9ed94 comptime VM: real lowering-time Context — allocating + List-building type-fns run on the VM (issue 0141)
The VM can now evaluate a comptime type-fn that allocates at lowering time (the
0141 family) — the legacy interp cannot. Four changes:

- runComptimeTypeFunc (lower/comptime.zig): force the CAllocator->Allocator thunks
  to exist (getOrCreateThunks, idempotent, guarded) BEFORE eval. A type-fn const
  runs at scanDecls (Pass 1), before Pass 1c builds the default-context global +
  thunks, so the comptime allocator was otherwise null.
- materializeDefaultContext: build a REAL context at lowering time when the global
  is absent — find the two thunks by name and lay their func-refs into the inline
  Allocator value at the head of Context, so context.allocator.alloc_bytes
  dispatches call_indirect -> thunk -> native VM malloc.
- aggType: deref a pointer base_type (the List write path emits struct_gep with
  base_type = *Struct; fieldOffset panicked on the pointer — now derefs, no panic).
- subslice: handle a [*]T many-pointer / *T base (a List's items field — the base
  IS the data pointer).

Verified end-to-end (manual probe): a compiler-API type-fn building its []Member in
a List(Member) runs HANDLED on the VM and mints (green=7) — the 0141 List-growth
pattern. Can't be a corpus test yet (gate-OFF/legacy can't allocate at lowering
time — the dual-path bind), so locked in via VM unit tests (many-pointer subslice;
struct_gep with a pointer base_type). 697/0 both gates + all unit tests.
2026-06-18 15:04:55 +03:00
agra
3c0e0852a8 issue 0141: re-refine root cause — IR is correct; it's a legacy slot_ptr chain + null comptime allocator (VM has neither failure mode) 2026-06-18 14:41:33 +03:00
agra
c085840964 CHECKPOINT-COMPILER-API: record that the real lowering-time Context is blocked by issue 0141 2026-06-18 14:32:59 +03:00
agra
5a0f8393c4 CHECKPOINT-COMPILER-API: fill commit hash in resume banner 2026-06-18 14:24:37 +03:00
agra
66005af478 comptime VM: port the WRITE side (declare_type/pointer_to/register_type) -> first HANDLED lowering-time type-fns
declare_type / pointer_to / register_type are now serviced natively in
Vm.callCompilerFn, mirroring the legacy compiler_lib handlers (mint via
@constCast(table) — the lowering-time mint target is &module.types). register_type
reads the []Member slice from flat memory: ref_types is threaded through invoke ->
callCompilerFn so the slice element type (Member = {name: string, ty: Type}) gives
the field offsets + stride; each {name, ty} is decoded and minted with the same
kind branching + dup/payload rejections + idempotent re-fill as legacy.

Key unblock: the synthesized comptime type-fn wrapper was built with return type
.any, so regToValue bailed at the VM<->legacy boundary; changed to .type_value
(the legacy path reads via asTypeId regardless). The compiler-API write type-fns
(0631 register-graph, 0635 multi-edge import) now run HANDLED end-to-end on the VM
at lowering time — parity-correct, on the zeroed lowering-time context (fixed
member arrays, no allocation). The metatype make_enum/define examples still fall
back cleanly through call_builtin(define).

697/0 both gates + EXIT=0.
2026-06-18 14:19:54 +03:00
agra
7b1212b41e CHECKPOINT-COMPILER-API: THE WALL broken — dedicated Type TypeId wired end-to-end 2026-06-18 14:05:50 +03:00
agra
554871ba0b comptime VM: model .type_value natively (word); harden struct_init vs arrays
kindOf(.type_value) -> .word; new const_type exec arm -> word = TypeId.index();
regToValue maps a .type_value word back to a .type_tag Value at the legacy
boundary. The VM now runs comptime evals involving Type values instead of
bailing.

This reached a latent VM panic: struct_init assumed a .@"struct" result type and
union-access-panicked on an array literal (EnumVariant.[...]). It is the generic
aggregate-literal op, so it now dispatches on the result kind (struct/array/
tuple) and bails loudly on anything else — never panics (CLAUDE.md no-panic).

697/0 both gates (make_enum type-fns run further on the VM, then bail cleanly at
the define call_builtin -> legacy mints; no mutation before bail). VM unit test
added (const_type -> word -> regToValue -> .type_tag).
2026-06-18 14:05:16 +03:00
agra
94f60c51c0 comptime VM: flip Type to .type_value; migrate the .any refs that mean a Type value
type_resolver "Type" -> .type_value; const_type result + emitConstType now a
bare 8-byte i64 handle (not a 16-byte Any box). Migrated every .any ref meaning
"a Type value", leaving real boxed-Any refs:

- "Any holds a Type" meta-marker tag .any -> .type_value at all 4 consumers
  (reflectArgTypeId, reflectTypeId, the comptime type_tag-as-struct path,
  resolveTypeCategoryTags "type").
- reflection-builtin return types (type_of/declare/define) -> .type_value;
  runtime type_of(any) reads the tag as a .type_value (no re-box).
- expr_typer: a bare type-name expr is .type_value (backtick is_raw exempt).
- reflectionArgIsType accepts .type_value OR .any (a reflection arg can be a
  bare Type or a boxed Any).
- comptime switch_br accepts a .type_tag discriminant (type-category match).
- a bare function name in a Type slot -> const_type(its function type), not a
  func-ref (fixes a JIT crash); old string-box kept only for genuine Any params.
- field-not-found diagnostic + formatTypeName render .type_value as "Type".

Fixed 3 unit tests asserting the old .any behavior. 697/0 both gates (gate ON
bails cleanly to legacy since the VM doesn't model Type values yet) + 494 unit
tests. 24 snapshots regenerated (22 .ir const_type shape; 2 .stderr Any->Type).
2026-06-18 13:54:56 +03:00
agra
6844fb90e7 comptime VM: dedicated Type builtin TypeId (8B), distinct from .any — foundation (dead)
Add TypeId.type_value (slot 19) + matching TypeInfo.type_value variant: an
8-byte type handle, distinct from the 16-byte boxed .any. All types.zig layout
handlers wired (size/align 8, display "Type", hash/eql); toLLVMTypeInfo -> i64.

Reserve builtin headroom: first_user 19 -> 100 (slots 20-99 padded with the
unresolved tripwire) so future builtins don't renumber user TypeIds / churn
sx ir snapshots. 22 IR snapshots regenerated (pure renumber to 100-base).

type_resolver still returns .any for "Type" — nothing produces .type_value
yet, so no behavior change. 697/0 both gates.
2026-06-18 13:03:21 +03:00
agra
7d59b5eeb6 CHECKPOINT-COMPILER-API: refresh resume banner — Phase 3 read+write done, lowering-time VM wired; next is the dedicated Type TypeId
Records the current state (read side, write side P3.3, lowering-time hardening +
wiring + zeroed context P3.4) and pins the next focused step: a dedicated Type
builtin TypeId (8B) distinct from .any (16B box) — ~123 .any refs across ~25 files,
a cross-cutting change to run as its own session. Paused here at a clean, green
boundary (697/697 both gates) per the decision to not rush it.
2026-06-18 12:30:35 +03:00
agra
6473a4e227 comptime VM: lowering-time default context (P3.4 step 1)
materializeDefaultContext now falls back to a zeroed Context (found by name) when
the __sx_default_context global is absent — i.e. at lowering time, where the global
isn't emitted yet. A type-fn that never touches the allocator runs past context
setup; one that allocates reads a null alloc_fn (zeroed) and call_indirect on the
null func-ref bails to legacy (a real lowering-time context with the CAllocator
thunk func-refs is a follow-up).

Measurement (SX_COMPTIME_FLAT_TRACE): the bail moved deeper — make_enum now bails
at const_type (the Type-literal op, unported); register_type type-fns bail at the
welded write call. No table mutation before either bail (write fns bail before
minting), so parity holds: both gates 697/0, no crashes.

Next: model the const_type op + the Type-return bridge + the VM-native write side,
which together let a type-fn run end-to-end on the VM.
2026-06-18 12:12:10 +03:00
agra
9d041b5136 comptime VM: wire the VM at the lowering-time site + measure (P3.4)
Route runComptimeTypeFunc (the type-fn fold — the third comptime call site)
through comptime_vm.tryEval behind -Dcomptime-flat/SX_COMPTIME_FLAT with legacy
fallback, mirroring the two emit-time folds. Extract the shared post-check
(checkComptimeTypeResult — the declared-but-never-defined zero-field guard) so the
VM and legacy paths share it.

Measurement (SX_COMPTIME_FLAT_TRACE): every metatype/compiler-API type-fn bails
CLEANLY at "no __sx_default_context global to materialize the implicit context" —
at lowering time the default-context global doesn't exist yet (it's built at emit
time), so the VM bails at context materialization, before running the body (no
partial mint, no crash -> legacy mints). The hardening holds: no crashes across
the corpus on the lowering-time VM path.

So the first lowering-time blocker is the implicit context, not Type modeling.
Both gates 697/0. Near-pure fallback today — permanent scaffolding that lights up
as the default-context handling + Type modeling + VM-native write side land.
2026-06-18 11:55:59 +03:00
agra
34734d415b comptime VM: harden against malformed lowering-time IR (P3.4-prep)
Prerequisite for wiring the VM at the lowering-time comptime site
(runComptimeTypeFunc), where IR can be malformed (an unresolved name lowers to a
dangling / Ref.none operand — the 0737 crash). Close the remaining panic vectors
so the VM bails (-> legacy fallback) instead of aborting:

- Vm.refTy(ref_types, r): a bounds-checked accessor replacing every raw
  ref_types[ref.index()] in exec — the type-side companion to Frame.get's
  bad_ref value-side guard.
- aggType is now a bailing method (Error!TypeId) routed through refTy.
- the block-dispatch loop bounds-checks the branch target before indexing
  func.blocks.items (a malformed br target). global_get was already guarded.

No behavior change: gate OFF and -Dcomptime-flat both 697/0. Unit test added
(a cmp_lt with a Ref.none operand bails, not panics).
2026-06-18 11:45:40 +03:00
agra
9ae3934f0f PLAN-COMPILER-VM: record non-negotiable end state — ONE evaluator, legacy deleted
Dual-path + emit-time legacy fallback are transitional scaffolding only; the VM
must reach parity at BOTH comptime sites (emit time AND lowering time), after
which the -Dcomptime-flat flag, the fallback, and interp.zig are all removed.
We do not ship both evaluators permanently.
2026-06-18 11:35:59 +03:00
agra
9e3aabcf76 comptime VM: Phase 3 — register_type write side + payloadless-enum fixes
The mutating compiler-API, minting types LAZILY at lowering time (single pass,
the existing runComptimeTypeFunc path — so the write side is legacy-only; the
VM isn't wired at lowering time, and the read-side readers stay dual-path):

  declare_type(name) -> Type            forward nominal handle (≈ declare)
  pointer_to(t) -> Type                 build *T references
  register_type(handle, kind, members)  ONE kind-branching fill (≈ unified define)

register_type branches on kind IN THE COMPILER (subsuming define's per-kind
dispatch); codes match type_kind: 1 struct, 2 actual .@"enum", 3 tagged_union,
4 tuple. Members are {name: string, ty: Type}. A non-generic `-> Type` builder is
now flagged is_comptime (decl.zig) so its dead body permits the welded calls.

Graph support: forward declare_type handles + pointer_to express a mutually-
recursive A<->B graph (*A, *B, B-by-value) before bodies are filled. register_type
is idempotent — re-filling a nominal slot (a minting module reached via two import
edges) re-mints identically rather than erroring (nominalIdent reads identity from
any nominal kind).

Fixes (issue 0142):
- A fully payloadless comptime-minted enum was minted as an all-void tagged_union,
  whose IR size disagrees with its LLVM size -> verifySizes panic. Now mints a real
  .@"enum" (register_type kind 2 AND the metatype defineEnum).
- Bare `EnumType.variant` qualified construction of a payloadless variant wasn't
  supported (failed for hand-written enums too — the type name lowered to a Type
  value). Added in lowerFieldAccess via isPayloadlessVariant; payload-carrying
  variants keep their call form.

Examples: 0631 (graph + actual enum + reflection), 0632 (make_enum all-void),
0633/0634/0635 (namespaced / bare / multi-edge import of a minted type), 0187
(qualified variant construction). Unit tests added.

Parity 697/697 (gate OFF and -Dcomptime-flat).
2026-06-18 10:47:36 +03:00
agra
27bc301651 comptime VM: Phase 3 — type_kind + type_field_value readers (read side complete)
The last two read-only readers the metatype's type_info(T) needs, each backed by
a TypeTable query both the legacy handler and the VM call (no drift):

  type_kind(t: TypeId) -> i64            (kindCode; stable discriminant, total — never bails)
  type_field_value(t: TypeId, idx) -> i64 (memberValue; enum explicit value or ordinal)

kindCode codes (compiler-owned, stable): 0 other / 1 struct / 2 enum /
3 tagged_union / 4 tuple / 5 union / 6 array / 7 vector / 8 error_set.

With these, the READ side is complete: find_type + type_kind + type_field_count +
type_field_{name,type} + type_nominal_name + type_field_value cover everything
reflectTypeInfo reads — a comptime sx fn can fully reflect a struct/enum/tuple
into data with no #builtin.

Example 0630 reflects Color / WindowFlags(flags) / Point. VM unit test added.

Revised forward direction: the write side will be ONE register_type(info) fn that
branches on the kind in the compiler (subsuming define's per-kind dispatch), not a
per-kind register_struct.

Parity 691/691 (gate OFF and -Dcomptime-flat).
2026-06-18 09:47:23 +03:00
agra
d23e208430 comptime VM: Phase 3 — field-level reflection readers
Three more read-only compiler-API readers on the TypeId-handle shape, each backed
by a new TypeTable query that both the legacy handler and the VM call (no drift):

  type_nominal_name(t: TypeId) -> StringId     (nominalName; loud-bail for unnamed types)
  type_field_name(t: TypeId, idx: i64) -> StringId   (memberName)
  type_field_type(t: TypeId, idx: i64) -> TypeId     (memberType)

All loud-bail on out-of-range idx / no-member — no silent default. First multi-arg
compiler fns (callCompilerFn now reads arg 1 = idx); added Vm.argHandle/argTypeId
range-checked arg readers and moved find_type/type_field_count onto them. Names use
the type_* family to avoid colliding with the std metatype builtins (field_name /
type_name in core.sx); the new TypeTable.nominalName is distinct from the existing
typeName(id) display-string renderer.

Example 0629 reflects Pair { lo: Point; hi: Point } — each field name + the nominal
name of a field's type, #run-folded, VM-HANDLED natively. VM unit test added.

Parity 690/690 (gate OFF and -Dcomptime-flat).
2026-06-18 09:34:36 +03:00
agra
a9302a8b50 comptime VM: Phase 3 — find_type + type_field_count reflection readers
First read-only compiler-API reflection readers, bound the same way as the
intern/text_of seed (compiler_lib.bound_fns + Vm.callCompilerFn, native on flat
memory, no marshaling). A type handle is a plain u32 TypeId (like StringId), so
both stay clean scalar host-calls:

  find_type(name: StringId) -> TypeId          (TypeTable.findByName; unresolved/0 if absent)
  type_field_count(t: TypeId) -> i64           (new TypeTable.memberCount; loud-bail, no silent 0)

memberCount is the single source both the legacy handler and the VM read, so the
two paths can't drift. find_type returns a non-optional TypeId using the
unresolved(0) sentinel for not-found rather than ?Type — a Type value is
.any-typed (which the flat-memory VM does not represent) and an optional can't
cross the legacy<->VM eval boundary; unresolved is the project-blessed "no type"
marker.

Example 0628 chains intern -> find_type -> type_field_count (+ a not-found
lookup), folded at #run, VM-HANDLED natively. VM unit test added.

Parity 689/689 (gate OFF and -Dcomptime-flat).
2026-06-18 09:25:26 +03:00
agra
0367d96d9b comptime VM: host wiring, full corpus parity, build flag, Phase 3 seed
Phase 1.final of the flat-memory comptime VM — wire the host through it,
reach corpus parity, and gate it behind a build flag — plus the first
Phase 3 (compiler-API) step. Default OFF; legacy interpreter unchanged.

Host wiring + hardening:
- Machine accessors return error.OutOfBounds (no debug panic) on bad
  addresses; Frame.get/set bounds-check and bail (no panic) on a malformed
  operand ref (e.g. a ret Ref.none from an unresolved name).
- tryEval routed at both comptime call sites in emit_llvm — the const-init
  fold and the #run side-effect path — with per-eval legacy fallback;
  yields .void_val for void/noreturn entries. Both sites sx_trace_clear()
  before the legacy fallback so a partial VM run that pushed trace frames
  doesn't double-push on re-run.

VM coverage (all corpus const-inits except the inline-asm global):
- Implicit context materialized from the __sx_default_context global; the
  full allocator protocol runs on the VM (context.allocator.alloc ->
  call_indirect -> CAllocator thunk -> libc_malloc -> native flat malloc).
- Native libc memory builtins (malloc/calloc/free/memcpy/memmove/memset)
  on flat memory; f32 stored/loaded as the 4-byte single; signed sub-64-bit
  loads sign-extended; global_get (lazy + memoized); func_ref/call_indirect
  (func-ref encoded fid+1, 0 reserved for null); string/slice fat-pointer
  field access; is_comptime; the failable/error cluster (error_set tuples,
  trace_frame + native sx_trace_push/clear -> raise/catch/or + return traces).

Build flag + Phase 3 seed:
- -Dcomptime-flat (build_opts module) OR SX_COMPTIME_FLAT env enables the VM;
  zig build test -Dcomptime-flat runs the full corpus on the VM (688/0).
- intern/text_of serviced natively on flat memory via Vm.callCompilerFn
  (compiler_welded boundary) — the seed the rest of the compiler-API grows on.

Parity 688/688 gate ON and OFF. Unit tests added throughout. The
lowering-time #insert wiring was explored and reverted (lowering-time IR can
be malformed; full malformed-IR hardening is a prerequisite, deferred).
2026-06-18 08:27:58 +03:00
agra
b8f3d6fd78 comptime VM: flat-memory machine + executor + Reg<->Value bridge + tryEval
Phase 1 of the flat-memory comptime VM (current/PLAN-COMPILER-VM.md),
built standalone + unit-tested with the legacy interpreter still live and
the corpus untouched (688 green).

src/ir/comptime_vm.zig:
- Machine: one linear byte memory (comptime stack+heap) with a bump/stack
  allocator (mark/reset), scalar readWord/writeWord (1/2/4/8 LE) + byte
  views; addr 0 reserved as null_addr. Frame: a Ref-indexed register file
  (Reg = raw u64: immediate scalar bits OR an Addr). Target-aware layout
  comes from the type table, so cross-compilation stays correct.
- Vm executor over the SAME SSA IR, mirroring the legacy interp's scalar
  semantics (i64 wrapping/signed, f64). Ported: constants, arithmetic,
  comparison, logical, conversions, control flow (br/cond_br/ret + block
  params); structs (alloca/load/store/struct_init/get/gep at target
  offsets); tuples; arrays (index_get/gep, length); slices+strings as
  {ptr,len} fat pointers (const_string, data_ptr, subslice,
  array_to_slice, str_eq/ne, index-through-slice); optionals (pointer and
  {T,i1} shapes); payloadless enums; deref/addr_of; direct + recursive
  call over the shared flat memory (depth-guarded). The value model: a
  word for scalars/pointers, by-address for aggregates (a struct's value
  IS its Addr). Any unported op bails loudly (error.Unsupported + detail).
- Reg<->Value boundary bridge (valueToReg / regToValue) + tryEval, the
  hybrid-wiring entry point: run a comptime fn on the VM, return a legacy
  Value or null to fall back. Transitional, for the legacy interop edge.

Registered in the ir.zig barrel.
2026-06-17 19:29:50 +03:00
agra
18af8eb845 comptime-API: strip the byte-weld; pivot to a flat-memory comptime VM
The byte-weld (sx structs whose layout was validated to mirror the
compiler's Zig records) plus the serialization/marshaling bridge was the
wrong direction: it bolted a parallel layout regime and hand-built
byte-copies onto a comptime value model that fundamentally isn't bytes.

Strip the struct-weld machinery:
- compiler_lib.zig loses the type registry (weldStruct / bound_types /
  BoundType / FieldLayout / findType / SxField / LayoutMismatch /
  validateStructLayout); it is now just the intern/text_of function
  host-call bridge (kept as the Phase-3 compiler-call seed).
- nominal.zig loses validateWeldedStruct / weldedFieldOrderStr + the
  sd.abi == .zig validation call.
- Remove the struct-weld unit tests and examples 0625/0627 (welded
  structs) + 1183/1186 (weld-layout diagnostics).
- The #library / abi / extern syntax stays.

Record the new direction: a bytecode VM over flat, byte-addressable
memory so comptime values are native bytes (no weld/validation/marshal),
target-aware (preserves cross-compilation) and sandboxed. See
current/PLAN-COMPILER-VM.md (Phase 0 strip -> Phase 1 flat-memory value
model -> Phase 2 bytecode -> Phase 3 compiler-API on flat memory).
design/comptime-compiler-api.md gets a SUPERSEDED banner. Also drop the
"~500 lines / split the step" rule from CLAUDE.md.
2026-06-17 19:29:36 +03:00
agra
40d075ca98 compiler-API: welded structs by reflection + memory-order validation
Replace the explored byte-layout-override engine (offset-ordered LLVM structs /
weld plans / byte-blobs — all unnecessary) with a much simpler design: a welded
`struct abi(.zig) extern compiler { … }` is a bodied header declaring its fields
in the bound compiler type's MEMORY order. The compiler reflects the real Zig
type (field names via @typeInfo, offsets via @offsetOf, size via @sizeOf —
nothing hand-maintained) and validates the header matches, with loud diagnostics.

On pass it is an ordinary struct whose natural layout already equals the Zig
layout — no reorder, no padding, no index/remap tables, no special LLVM path — so
@ptrCast'ing it to the compiler's own type and dereferencing is byte-identical.
When types.zig shifts, the header stops matching and the developer gets a specific
message to fix it.

- compiler_lib.zig: weldStruct reflects field names and bakes bound_types fields
  in ascending-offset (memory) order; deleted computeWeldPlan/WeldPlan/WeldElement.
- nominal.zig validateWeldedStruct: precise diagnostics — field-not-found,
  wrong-field-order (+ expected memory order), type-layout (size) mismatch,
  total-size mismatch.
- Examples: 0627 (StructInfo in memory order, byte-identical, usable),
  1186 (source-order StructInfo -> wrong-field-order diagnostic); 1183 refreshed.
- Design doc + checkpoint updated.
2026-06-17 15:45:23 +03:00
agra
88c4cbcfa5 test harness: add -Dname to scope the corpus to specific examples
`zig build test -Dname=examples/0625-foo.sx[,examples/0626-bar.sx]` runs ONLY the
named example(s) — full repo-relative .sx paths, comma-separated (a leading `./`
is tolerated). Empty = run everything (unchanged default).

Why: a full `-Dupdate-goldens` re-runs and rewrites all ~690 snapshots, so one
flaky/host-divergent example (AOT links, cross-arch `target` examples) can clobber
a good snapshot. `-Dname` regenerates only the named example(s) and touches
nothing else. It also busts the cached test-run result — the corpus enumerates
.sx/expected files at runtime, so a bare snapshot edit alone is otherwise served
from cache.

- build.zig: new `name` option threaded onto corpus_paths.
- corpus_run.test.zig: `nameMatchesFilter` + a per-example skip in the run loop.
- CLAUDE.md: document the targeted-regen workflow under Snapshot integrity.
2026-06-17 14:55:06 +03:00
agra
0b4c50b187 compiler-API: resume scaffolding for a fresh session
Add the COMPILER-API stream to CLAUDE.md's session-start router and a
`## ⏯ Resume` block to CHECKPOINT-COMPILER-API.md (next action = sub-step 2.2,
read order, build/verify, and the cross-arch snapshot-regen gotcha).
2026-06-17 13:32:33 +03:00
agra
cd5b958d19 comptime compiler-API: Phase 1 foundation + Phase 2.1 weld plan
Introduce the welded comptime `compiler` library (`#library "compiler"` +
`abi(.zig) extern compiler`), per design/comptime-compiler-api.md, and unify
`callconv(...)` into the new `abi(...)` annotation.

abi(...) replaces callconv(...):
- New ABI enum { default, c, zig, pure }; `abi(.c|.zig|.pure)` parses in the
  postfix slot before extern/export (and standalone). `kw_callconv` -> `kw_abi`.
- Migrated 52 sx files, the call-convention-mismatch diagnostic, and docs
  (readme/specs) from `callconv(.c)` to `abi(.c)`.

Phase 1 — welded compiler library (parse -> registry -> validation -> bridge):
- `abi(.zig) extern compiler` parses on fn decls (carries abi/extern_lib) and
  struct decls (StructDecl.abi/extern_lib).
- `#library "compiler"` is the comptime-only internal surface — never dlopen'd.
- src/ir/compiler_lib.zig: the binding registry (the safety boundary). `Field`
  welded to StructInfo.Field with layout baked from the real Zig type
  (@offsetOf/@sizeOf); `findType`/`findFn`. Welded structs are layout-validated
  at registration (field set + total size) as a header checked against the impl.
- Host-call bridge: a `fn abi(.zig) extern compiler` dispatches under the
  comptime interp to its registered Zig handler (intern/text_of round-trip),
  never dlsym. IR Function.compiler_welded; validated in declareFunction.
- Comptime-only enforcement: a runtime call to a welded fn is a clean
  build-gating error (emitCall), not an undefined-symbol link failure.

Phase 2.1 — byte-layout weld foundation:
- Decision: full byte-layout weld (sx struct laid out byte-identically to the
  bound Zig type). Registered StructInfo (first non-natural / Zig-reordered
  layout). `computeWeldPlan` — pure offset-ordered element plan + padding +
  sx-field->LLVM-element remap; unit-tested. Emit/interp wiring is the next
  sub-step (2.2+, see current/CHECKPOINT-COMPILER-API.md).

Examples: 0625/0626 (welded struct + fn round-trip), 1183/1184/1185
(layout-mismatch, unexported-fn, runtime-call diagnostics).
2026-06-17 13:31:11 +03:00
agra
3a9b508502 design: drop 'Co-designed' attribution line 2026-06-17 10:11:38 +03:00
agra
7a37fe33ce design: ground compiler-API build order in code anchors
#library already lexes/parses (library_decl node); extern/export are
keywords. Phase 1 new work pinned to concrete sites: parser (extern(.zig)
postfix at the #builtin/#compiler positions), AST binding field,
compiler_hooks.zig as the registry, types/llvm layout emission, host_ffi
comptime bridge. First testable sub-step: extern(.zig) <lib> parses on a
fn decl.
2026-06-17 10:04:10 +03:00
agra
2b43af4f8a upgrade llvm@22 2026-06-17 09:58:43 +03:00
agra
08b0a35758 design: comptime compiler API — #library "compiler" + extern(.zig)
Unified sx<->compiler binding that subsumes the metatype declare/define
primitives AND the #compiler struct attribute. A named 'compiler' library
exposes the compiler's real types (layout-welded via extern(.zig), offsets
queried from the Zig type at compiler-build time + a build-time equality
assertion) and functions (comptime-only, host-call bridged). declare/
define/type_info become sx library code over register_*/find_type; the
projected meta.sx TypeInfo + hand marshaling are deleted; BuildOptions
migrates onto it and #compiler is removed. Includes the safety boundary
(curated export list, guarded mutators, comptime-only), the honest limit
(the ordering law stays, but stops leaking as 'weird stages' — dissolving
the 0141 class), a phased suite-green build order, and the open risks
(union(enum) welding, optional fields, LLVM offset emission).
2026-06-17 09:38:00 +03:00
agra
e2b2e22fa7 issue(0141): Direction 2 (defer eval) ruled out by experiment; Direction 1 is the path
Wired a minimal deferral (eval at a new Pass 1c' after the CAllocator
thunks exist) — the List repro STILL bailed with struct_get, and it
destabilized examples/0620. So deferring past the thunks isn't the cause
of the wrong IR; the field-access lowering only emits struct_gep at
body-lowering/emit time. No single pass slot satisfies both 'body lowers
correctly' and 'layout ready before use'. Pivot to Direction 1 (robust
*Struct field-access lowering). Experiment reverted; tree clean.
2026-06-17 08:34:50 +03:00
agra
a448f50f7f issue(0141): refine root cause — wrong IR (struct_get vs struct_gep) at scanDecls
Instrumentation shows List.append lowers list.len/list.cap to struct_gep
(correct) at #run/emit time but struct_get (wrong, value access on a *T
receiver) at scanDecls/metatype time — same source, different IR. The
function IS lowered both ways, just to wrong IR at scanDecls due to
incomplete generic-instantiation context. So an interp-side lazy-lower
hook can't fix it (IR is wrong before the interp runs); the fix is either
robust field-access lowering or deferring the comptime type-construction
eval to a complete-world pass (like #run). Supersedes the two-layer framing.
2026-06-17 08:21:55 +03:00
agra
86feced560 docs(metatype): refresh PLAN Status — surface complete, 0141 deferred 2026-06-17 08:08:56 +03:00
agra
0f88525884 issue(0141): comptime List growth in type construction (two-layer)
File the last METATYPE deferred enhancement: List(T).append at comptime
bails ('struct_get: base has no fields') in a type-construction ::.
Standalone repro + two-layer root cause (null comptime allocator at
scanDecls; *T slot_ptr struct_get) + investigation prompt. Non-blocking:
array-literal locals already build variant lists (examples/0620/0624).
Checkpoint + Known issues reference 0141.
2026-06-17 08:07:11 +03:00
agra
85c1b85f8b docs(metatype): comptime List growth — two-layer root cause documented
Investigated the last deferred enhancement. List(T).append at comptime
fails in two independent layers (both reproduce with plain List(i64);
List works via #run because that evaluates at emit time, after lowering):
1. null comptime allocator — defaultContextValue looks up the
   CAllocator->Allocator thunks by name, but they aren't lowered at
   scanDecls time. Fixable by forcing getOrCreateThunks before the interp
   runs in runComptimeTypeFunc (tried, works for this layer).
2. struct_get through a *T slot_ptr chain (the *List receiver) — the
   deep part; comptime pointer/struct/slot resolution, its own session.
Speculative fixes reverted (no end-to-end win without layer 2).
2026-06-17 07:58:34 +03:00
agra
c7e997043f docs(metatype): generic type-fn body locals done; only List growth remains 2026-06-17 07:45:08 +03:00
agra
32bbfdecc1 test(metatype): generic type-fn body local (examples/0624)
Locks the generic-type-fn prelude eval (d87d86d): make_status($T)
assembles a variant list in a local then mints, with the ok payload = T.
2026-06-17 07:44:15 +03:00
agra
d87d86df8a feat(metatype): comptime-eval generic type-fn body locals
A generic ($T) -> Type type-fn comptime-evaluated only its return
EXPRESSION, so a local declared before the return ('vs := …; return
make_enum(…, vs)') was unresolved. Now a body with a prelude (statements
before the return) has its full body evaluated: createComptimeFunction-
WithPrelude lowers the pre-return statements into the comptime function's
scope before the return expr, so the locals resolve.

- comptime.zig: createComptimeFunctionWithPrelude (prelude stmts +
  expr); evalComptimeTypeBody (extract prelude + return expr, scan the
  whole body for declare() forward types); runComptimeTypeFunc factored
  out of evalComptimeType (shared bail/declare-never-defined handling).
- generic.zig: route a type-fn body WITH a prelude through
  evalComptimeTypeBody; no-prelude bodies stay on evalComptimeType (zero
  change for RecvResult/TryResult etc.).

Non-generic builders (whole body already evaluated) and the List-growth
path are unaffected. Suite green (684).
2026-06-17 07:40:09 +03:00
agra
60293bf5dd docs(metatype): tuple done; reflect/construct triad complete 2026-06-17 07:10:45 +03:00
agra
14cfb64874 test(metatype): tuple construct + round-trip (examples/0623)
Locks the tuple widening (9f3f746): programmatic Pair build via
.tuple(.{elements}) + a source-tuple round-trip via type_info. Completes
the reflect/construct triad (enum 0619, struct 0622, tuple 0623).
2026-06-17 07:10:01 +03:00
agra
9f3f746c4b feat(metatype): widen type_info/define to tuple types
TypeInfo gains a `tuple(TupleInfo) variant (TupleInfo{elements: []Type},
positional/unnamed) — completing the reflect/construct triad with enum
and struct.

- meta.sx: TupleInfo + `tuple TypeInfo variant.
- interp: reflectTypeInfo builds .tuple (tag 2) as bare type_tag elements
  (no name pairs); defineType dispatches tag 2 -> defineTuple, which
  decodes []Type and completes the declare slot as a structural .tuple
  via replaceKeyedInfo (kind change). Tuples are structural so the
  declared name is vestigial, but the slot is still completed in place so
  define returns the handle (consistent with enum/struct).
- call.zig: the lower-time type_info guard now admits .tuple.

define(declare("P"), .tuple(.{elements=.[i64,f64]})) builds a tuple, and
define(declare("T"), type_info((i64,bool,f64))) round-trips one. Suite
green (683).
2026-06-17 07:05:55 +03:00
agra
d83f5fa90d docs(metatype): struct widening done; tuple is the last shape 2026-06-17 06:59:17 +03:00
agra
8f03349279 test(metatype): struct construct + round-trip (examples/0622)
Locks the struct widening (aaac019): programmatic Vec2 build via
.struct(.{fields}) and a source-struct round-trip via type_info.
2026-06-17 06:58:28 +03:00
agra
aaac019715 feat(metatype): widen type_info/define to struct types
TypeInfo gains a `struct(StructInfo) variant (StructField{name,type});
the metatype system now reflects AND constructs structs, not just enums.

- meta.sx: StructField / StructInfo / `struct TypeInfo variant.
- interp: reflectTypeInfo builds .struct (tag 1) for a source @"struct";
  define dispatches on the TypeInfo tag (defineType) -> defineEnum (0) /
  defineStruct (1). defineStruct mirrors defineEnum (dup-field-name check
  included) but completes the declare slot AS a struct via replaceKeyedInfo
  (a kind change re-keys the intern map; updatePreservingKey asserts no
  key change, true only for the enum path).
- call.zig: the lower-time type_info guard now admits @"struct".

define(declare("P"), .struct(.{ fields = .[ … ] })) builds a struct, and
define(declare("C"), type_info(SrcStruct)) round-trips one. Suite green
(682); enum path (0619) unchanged.
2026-06-17 06:54:17 +03:00
agra
afb1fee252 docs(metatype): validation story complete (use-before-define subsumed) 2026-06-17 06:42:27 +03:00
agra
dcdf1dd318 test(metatype): lock by-value self-ref rejection for constructed enums (1182)
Constructed-type companion to examples/1178 (source form): a declare/
define enum whose variant references itself BY VALUE is rejected by the
same checkInfiniteSize guard ('infinitely sized'). Pins the use-before-
define corner of the validation story — by-value self-reference is the
one self-ref shape that isn't legal; *L (pointer) is fine (see 0618).
No compiler change (locks existing behavior).
2026-06-17 06:42:02 +03:00
agra
fe6799545a docs(metatype): declare()-never-defined validation done 2026-06-17 06:34:26 +03:00
agra
c185dbdd13 test(metatype): lock declare()-never-defined rejection (examples/1181)
Diagnostics example for the bare-declare guard (14f30f3): an unfinished
declare("Undef") -> build-gating error naming the type, exit 1.
2026-06-17 06:33:31 +03:00
agra
14f30f341c fix(metatype): reject declare() never completed by define()
A bare declare("X") with no define left a zero-field nominal slot that
panicked at codegen (verifySizes: llvm_size != ir_size). evalComptimeType
now detects a zero-variant tagged_union result and emits a clean
build-gating diagnostic naming the type — a zero-variant enum is never a
legitimate construction result (defineEnum rejects empty variant lists
too). Self-reference (a declared slot completed by define) is unaffected.
2026-06-17 06:29:23 +03:00
agra
c573e4befb docs(metatype): duplicate-variant validation done; tidy Next step list 2026-06-17 05:27:17 +03:00
agra
e291034e46 test(metatype): lock duplicate-variant-name rejection (examples/1180)
Diagnostics example for the define duplicate-name guard (b2db2c5): two
'value' variants -> build-gating error naming the duplicate, exit 1.
2026-06-17 05:26:35 +03:00
agra
b2db2c54ed fix(metatype): reject duplicate variant names in define
Two same-named variants in a constructed enum silently succeeded —
construction (.a) and matching would ambiguously pick one. defineEnum
now bails when a variant name repeats, naming it. The name is dynamic so
it sets last_bail_detail directly (bailDetail takes a comptime string);
evalComptimeType renders it as a build-gating diagnostic.
2026-06-17 05:22:23 +03:00
agra
964ddeb73a docs(metatype): comptime aggregate subslice gap resolved 2026-06-17 05:16:08 +03:00
agra
60471b3a2c test(metatype): comptime subslice over an aggregate (examples/0621)
make_enum from dirs[0..2] — mints Axis from a comptime SUBSLICE of a
local EnumVariant array. Locks the interp subslice-over-non-string-
aggregate fix (d22037c); previously bailed.
2026-06-17 05:15:43 +03:00
agra
d22037c4a7 fix(interp): comptime subslice over non-string aggregates
`arr[lo..hi]` at comptime bailed for any non-string base — the interp's
.subslice op only handled string-backed values. Worse, the open-ended
`hi` came from a .length op that misread a 2-element array as a {ptr,len}
fat pointer (returning the 2nd element, not the count), so even lo/hi
weren't valid ints.

Fix, interp-only (runtime already handles arrays via LLVMTypeOf):
- Thread the base operand's IR type onto the Subslice op (base_ty); the
  interp uses it to tell a bare array (elements = aggregate fields) from a
  {data,len} slice (elements in the data field) — indistinguishable by
  Value shape alone.
- Fold an open-ended slice's hi to the array's static length for fixed
  arrays at lower time (runtime emitLength folds the same constant, so the
  IR result is unchanged — no snapshot churn — but the comptime interp no
  longer hits the ambiguous .length op).
- subsliceElements() resolves the element list (array/slice, inline or
  slot_ptr-backed) and subslice returns a proper {data,len} slice value.

Suite green (678), no .ir changes.
2026-06-17 05:11:33 +03:00
agra
4e8075491d docs(metatype): make_enum done; note deferred free-form-construction gaps 2026-06-17 04:56:56 +03:00
agra
2250652ba5 feat(metatype): make_enum — general enum constructor over a []EnumVariant value
make_enum(name, variants: []EnumVariant) -> Type mints a nominal enum
from a variant list passed as a VALUE, not a hardcoded literal — the
open-ended form the channel-result constructors are special cases of.
Pure sx over declare/define; no compiler machinery.

Because variants is an ordinary comptime value, a non-generic builder
can ASSEMBLE it in a local before minting. examples/0620: build_level
fills a local array, then make_enum mints Level from it — exercising
define decoding a value-arg SLICE (decodeVariantElements' slice branch),
vs. the inline .[ … ] array the 0614-0618 examples pass directly.

No compiler change (locks existing capability). Suite green (678).
2026-06-17 04:55:48 +03:00
agra
0cb1aa270e docs(metatype): issue 0140 resolved; make_enum unblocked 2026-06-17 04:36:51 +03:00
agra
4da6add334 test(0140): pin comptime type-construction bail diagnostic (examples/1179)
Move the issue 0140 repro into the feature suite as a regression test.
Asserts the build-gating diagnostic 'comptime type construction failed:
comptime define(): enum has no variants' at the construction site, exit
1 — locking out the prior 'unresolved type reached LLVM emission' panic.
2026-06-17 04:36:09 +03:00
agra
37ec3da8cb fix(0140): surface comptime type-construction bail as a diagnostic
evalComptimeType did `interp.call(...) catch return null`, dropping the
interp's last_bail_detail; callers poisoned to .unresolved with no
diagnostic, so the sentinel reached LLVM emission and panicked
("unresolved type reached LLVM emission"), or hid behind a downstream
cascade.

Clear last_bail_detail before the call; on the catch emit a build-gating
.err at the construction expr's span ("comptime type construction
failed: {detail}", mirroring the #run surfacing in emit_llvm.zig), then
return null to keep the .unresolved poison — now gated by a real message
so no unresolved type reaches emission unannounced.

Empty-variant define now prints 'comptime define(): enum has no
variants' and exits 1 (no panic); make_enum-style computed-slice
failures show their root reason at the construction site.
2026-06-17 04:31:38 +03:00
agra
3a062780f7 issue(0140): comptime type-construction bail panics instead of diagnosing
A failing declare/define (e.g. empty variant list) bails correctly in
the interp, but evalComptimeType swallows last_bail_detail via
`catch return null`; the decl poisons to .unresolved with no diagnostic
and reaches LLVM emission -> panic ("unresolved type reached LLVM
emission"), or hides behind a misleading downstream cascade.

Pre-existing (plain define path), surfaced while starting the make_enum
step. Blocks make_enum's computed (pointer-backed) []EnumVariant slice
decode. Repro + investigation prompt filed; CHECKPOINT-METATYPE marked
BLOCKED. Session paused pending fix per CLAUDE.md IMPASSABLE rule.
2026-06-16 22:59:49 +03:00
agra
52b0dc2a9a docs(metatype): type_info($T) enum reflection done; update plan/checkpoint 2026-06-16 22:53:52 +03:00
agra
1ffda415c2 feat(metatype): implement type_info($T) reflection (enum round-trip)
type_info reflects an enum / tagged-union INTO a TypeInfo value — the
inverse of define's decode — so define(declare(n), type_info(T)) mints
a byte-identical copy with NO literal variant list.

- inst.zig: new BuiltinId.type_info (comptime-only, like declare/define).
- lower/call.zig: replace the 'not yet implemented' bail. Resolve $T at
  lower time, reject non-enum/non-tagged-union loudly with a good span,
  emit callBuiltin(.type_info, [const_type], TypeInfo).
- interp.zig: reflectTypeInfo builds the exact nested-aggregate Value
  defineEnum decodes — variant {name,payload}, slice {data,len}, EnumInfo
  {variants}, TypeInfo {tag0, EnumInfo}. tagged_union reflects field.ty
  (tagless already void); payloadless `enum` reflects void per variant.
- emit: unchanged — type_info is always comptime-evaluated, the existing
  comptime-only else arm (shared with declare/define) never fires.

0619 turns green: a source enum (circle:f64 / rect:i64 / empty) reflected
and reconstructed, constructs and matches like the original.
2026-06-16 22:52:53 +03:00
agra
3805a051cc test(metatype): lock type_info round-trip example (currently bails)
type_info($T) is still unimplemented, so the round-trip
define(declare("ShapeCopy"), type_info(Shape)) bails with
"type_info is not yet implemented" plus the downstream
enum-inference cascade. Snapshot pins that current behavior;
the next commit implements type_info and turns this green.
2026-06-16 22:42:26 +03:00
agra
d0a2967f18 docs(metatype): issue 0139 resolved; by-value self-ref rejection done 2026-06-16 22:25:11 +03:00
agra
2f0905b407 fix(0139): reject by-value self-referential types loudly (was a segfault)
A nominal aggregate that contains itself (or a mutual peer) BY VALUE has no
finite layout and infinite-recursed typeSizeBytes into a stack overflow —
for SOURCE enums/structs as well as comptime-constructed types.

New `checkInfiniteSize` pass (lower/decl.zig, Pass 1g — after type
registration, before body lowering): walks the by-VALUE containment graph
(pointer/slice/optional payloads break the cycle, so `*Self` stays valid);
on a back-edge it emits a loud diagnostic — "type 'X' is infinitely sized
(it contains itself by value); use a pointer ('*X') to break the cycle" —
and poisons the offending field to `.unresolved` so sizing can't recurse
before the build halts on the error. Covers source + declare/define types,
direct + mutual recursion.

examples/1178 locks the diagnostic; issue 0139 marked RESOLVED. This also
completes METATYPE PLAN F5's by-value-self-reference rejection. Full suite
green (675).
2026-06-16 22:24:31 +03:00
agra
f845fc6413 issue(0139): by-value self-referential type segfaults (typeSizeBytes recursion)
Discovered while testing metatype self-reference: a by-VALUE self-ref
(`payload = List`, not `*List`) infinite-loops typeSizeBytes → segfault
instead of a loud "infinite size" diagnostic. PRE-EXISTING — a hand-written
source enum `enum { node: Bad; leaf }` crashes identically, so it's a
general type-system gap (the comptime F5 by-value-rejection inherits the
fix). Filed per the IMPASSABLE rule; metatype checkpoint notes it.
2026-06-16 22:10:53 +03:00
agra
a7dde2efd1 docs(metatype): declare(name) + self-reference done; update plan/checkpoint 2026-06-16 22:08:50 +03:00
agra
2a9ffd25a8 test(metatype): self-reference regression example (recursive *List enum)
examples/0618 mints a recursive `List` enum (`cons: *List; nil`) via
declare("List")/define, builds a 3-node list, matches the pointer payload
directly and via deref, and counts it recursively. Locks the self-reference
capability. Full suite green (674).
2026-06-16 22:07:02 +03:00
agra
7a9db03bcc green(metatype): declare(name) + self-reference (recursive enums via *Name)
declare now takes the type's NAME — `declare(name) -> Type` — because the
compiler needs it at compile time to register the forward type, which is
what makes self-reference resolve. EnumInfo drops `name` (it lives on
declare now); define completes the handle's body in place (the slot is
already named).

Self-reference mechanism (evalComptimeType): before lowering a comptime
type expression, preregisterForwardTypes scans it (and a called ctor fn's
body) for `declare("Name")` calls and registers each as an empty forward
nominal type AND binds it as a type alias. The alias is essential: a
`Name :: ctor()` decl makes `Name` a const_decl author, so a `*Name`
self-reference resolves through the forward-ALIAS path
(type_aliases_by_source), which a bare findByName registration doesn't
satisfy. With both in place `*Name` resolves to the forward slot at lower
time; the interp's declare returns that same slot; define fills it.

  List :: make_list();
  make_list :: () -> Type {
      h := declare("List");
      return define(h, .enum(.{ variants = .[
          EnumVariant.{ name = "cons", payload = *List },
          EnumVariant.{ name = "nil",  payload = void } ] }));
  }

Verified: cons/nil construct + match (direct and through the pointer),
multi-node list traversal via a recursive `count(*List)`. meta.sx
RecvResult/TryResult + examples 0614/0615/0617 updated to declare(name);
full suite green (673).
2026-06-16 22:02:48 +03:00
agra
12e2ff7ef4 docs+rename: erase the reify name everywhere — stream is METATYPE
The compiler concept is declare/define (comptime type construction); the
old "reify" framing is gone from the entire repo.

- Rename: PLAN-REIFY → PLAN-METATYPE, CHECKPOINT-REIFY → CHECKPOINT-METATYPE,
  PLAN-POST-REIFY → PLAN-POST-METATYPE (both rewritten around declare/define);
  examples 0614/0615/0617 → comptime-metatype-* (+ their expected/ triplets),
  headers rewritten.
- Scrub reify from design/execution-evolution-roadmap.md (§7 step 3 contracts,
  §8.1, §9 decisions, §10 gates) → declare/define / comptime type construction.
- core.sx prelude pointer + parser.test.zig surface lock updated to the
  declare/define builtins (define(handle, info) -> Type; EnumInfo.name).

No behavior change; renamed examples match their renamed snapshots. Full
suite green (673), all unit tests pass. Zero `reify` tokens remain in
src/docs/sx/examples.
2026-06-16 21:23:05 +03:00
agra
5f2419854e green: erase the sx reify sugar — declare/define are the only constructors
Per the directive to strip reify entirely: the sx `reify(info)` one-shot is
removed. `define(handle, info)` now RETURNS the (completed) handle, so the
one-shot constructor chains as a single expression:
    T :: define(declare(), .enum(.{ name = "T", variants = ... }));

- meta.sx: drop reify; RecvResult/TryResult use `define(declare(), …)`.
- interp .define returns the handle type_tag (was void); call.zig lowers it
  with `Type` result and sets the info arg's target type to TypeInfo so the
  intercepted call still infers the `.enum(…)` literal.
- returnExprMintsType: a type-fn body that returns `define(…)` (or a bodied
  non-generic Type-returning sx helper) is comptime-evaluated.
- examples 0614 (direct) + 0615 (type-fn) use `define(declare(), …)`.

Full suite green (673). Files/docs still carry the old reify naming — the
rename sweep is the next commit.
2026-06-16 21:12:32 +03:00
agra
8ae655687a green(reify): type-fn bodies comptime-evaluated; reify fully removed from the compiler
Second slice of the re-architecture — the compiler now has ZERO type-
construction code beyond declare/define.

- instantiateTypeFunction: a type-fn body returning a computed Type (a call
  to a non-generic, bodied, Type-returning fn) is comptime-evaluated with the
  type bindings active, then renamed to the mangled instantiation name for
  identity (renameNominalType). Replaces the old reify-call pattern-matching.
- DELETED: reifyType (lower/nominal.zig), findReturnReifyCall (lower/generic.zig),
  and the stale inline-position reify gate in resolveTypeCallWithBindings.
- evalComptimeType (was evalComptimeTypeNamed): pure eval, no rename; the
  type-fn caller renames explicitly. renameReifiedType → renameNominalType.
- The TYPE NAME now travels in the data: EnumInfo gains `name`, and define()
  names the slot from it (the compiler derives no name from a binding LHS).
  examples/0614/0615 carry `name = "..."`; RecvResult/TryResult set it too.
- field_type stays a reflection #builtin (reads a type); only construction
  moved out. All reify mentions stripped from compiler source.

examples 0614/0615/0617 run on the floor. Full suite green (673).
2026-06-16 21:03:16 +03:00
agra
442a70b8c9 green(reify): declare/define floor — reify is sx; E :: reify(...) comptime-evaluated
First slice of the re-architecture. The compiler gains two comptime
type-construction builtins — declare() (mint an empty/undefined nominal
slot) and define(handle, info) (decode a TypeInfo VALUE + complete the
slot) — executed by the interpreter against a new `mint` TypeTable handle
(setMintTable). reify becomes PLAIN sx in meta.sx:
  reify :: (info) -> Type { h := declare(); define(h, info); return h; }

`E :: f(...)` where f is a non-generic Type-returning fn (reify, and later
make_enum) is now comptime-evaluated via evalComptimeTypeNamed: wrap the
call in a throwaway comptime fn, run it through the interp with the mint
table enabled so declare/define mint the type, read back the type_tag, and
rename the anonymous slot to the binding name. The compiler has ZERO reify
knowledge at the decl site — the old `E :: reify` hook is deleted.

examples/0614 (inline reify) now runs on this floor. Full suite green (673).

INTERMEDIATE: reifyType + findReturnReifyCall still serve the type-fn path
(0615/0617) and will be deleted in the next slice (type-fn body
comptime-eval), after which the compiler has no reify code at all.
2026-06-16 20:39:02 +03:00
agra
ae27cffe9d plan(reify): F1 findings + lock the zero-compiler-reify end state
Record the verified pass-order / define-timing / parse / dispatch findings
from F1 investigation, and make explicit that the floor work MUST delete
reifyType + the E :: reify decl hook + findReturnReifyCall (reify lives only
in meta.sx). Removal can't precede the floor, so they land together; suite
never left red across a session boundary.
2026-06-16 20:15:21 +03:00
agra
e5d1d0de39 plan(reify): re-architect onto declare/define as the only compiler primitive
User-directed redirection. The compiler should expose ONLY declare() and
define(handle, info) as comptime type-table primitives; reify / make_enum /
RecvResult / TryResult all become plain sx in meta.sx (reify ==
{ h := declare(); define(h, info); return h; }). The AST-walking reifyType
and every syntactic reify recognition (decl.zig E :: reify hook, generic.zig
findReturnReifyCall routing) are to be DELETED, replaced by generic comptime
evaluation of a Type-returning expression.

PLAN-REIFY gains a RE-ARCHITECTURE section: the irreducible compiler floor
(declare = empty nominal slot; define = decode a TypeInfo VALUE + fill via
updatePreservingKey; comptime-eval a Type-returning ::-RHS/type-fn body),
the resolved naming/identity story (declare mints anonymous, the binding site
names it; identity via the existing instantiation cache), and an F1-F5 phase
table that re-greens 0614/0615/0617 on the floor.

No code change in this commit — the in-session Phase 3.2 attempt (make_enum +
eval-decode reader) was reverted (reset to 9306ad5) so the floor is built
first. Checkpoint records the revert + sets next step = F1.
2026-06-16 20:08:17 +03:00
agra
9306ad570d green(reify): RecvResult/TryResult channel result types over reify
REIFY Phase 3.1. Add RecvResult($T) and TryResult($T) to meta.sx as
type-fns over reify (value-or-closed; value-or-empty-or-closed). They
need NO new compiler machinery — reify-of-a-literal in a type-fn body is
exactly the Phase 1 path — so the channel result types are pure sx
library code. examples/0617 green (both construct + match, incl.
payload-less .closed / .empty). Suite green (673 examples, 447 unit).

make_enum(variants) (3.2) and type_info (2.2) remain — both blocked on a
generalized reify reader (reifyType currently AST-walks a literal
TypeInfo). Plan/checkpoint updated.
2026-06-16 19:15:26 +03:00
agra
6627f7348b xfail(reify): RecvResult/TryResult channel result types over reify
REIFY Phase 3.0. Add examples/0617 using RecvResult(i64) / TryResult(i64)
(construct + match, plus payload-less .closed / .empty). Seed an empty
expected/*.exit marker. RED by design — the type-fns aren't defined yet
("unresolved RecvResult"); Phase 3.1 adds them to meta.sx as type-fns
over reify and turns this green.
2026-06-16 19:10:34 +03:00
agra
ac8c689518 green(reify): field_type($T, i) -> Type over the type table
REIFY Phase 2.1. fieldTypeOf (lower/generic.zig, re-exported on Lowering)
returns the i-th member type of T: struct field / tagged-union + union
variant payload (.void for a tagless variant) / tuple element / array +
vector element. Out-of-range and memberless types poison to .unresolved
with a loud diagnostic (never a silent default). Wired into
resolveTypeCallWithBindings (replacing the Phase-2 bail); since it folds
to a TypeId at lower time it composes inside type_eq / type_name / any
type-arg slot.

examples/0616 green: struct fields (name via field_name + type via
field_type), type_eq fold, tagged-union payloads incl. quit -> void.
Suite green (672 examples, 447 unit).

type_info($T) -> TypeInfo (reflect into a value, inverse of reify) is
NOT done — still bails loudly; it's the larger Phase 2.2 step (widen the
TypeInfo data model + comptime value construction). Plan/checkpoint updated.
2026-06-16 19:06:57 +03:00
agra
bd139dc09c xfail(reify): field_type — read struct/enum member types by index
REIFY Phase 2.0. Add examples/0616: reflect a struct's fields (name via
field_name, type via field_type) and a tagged-union's variant payloads,
including field_type composed inside type_eq / type_name. Seed an empty
expected/*.exit marker. RED by design — field_type still bails ("not yet
implemented"); Phase 2.1 implements it over the type table and turns
this green.
2026-06-16 19:01:03 +03:00
agra
18a4f9dd54 green(reify): type-fn over reify memoizes by mangled name (identity)
REIFY Phase 1.1 (Phase 1 complete). instantiateTypeFunction detects a
type-fn body that returns reify(...) (findReturnReifyCall) and routes it
to reifyType under the instantiation's name — mangled for inline use,
the alias name for `Foo :: Box(i64)` — with the type-arg bindings active
so reify payloads (`payload = T`) resolve against the instantiation args.
Placed before the general case, whose resolveTypeWithBindings would
route the reify call to the inline-position loud bail.

Registering under the mangled name lets the top-of-instantiation cache
return the SAME TypeId on a second instantiation, so Box(i64) resolved
at two independent sites is ONE type (Contract 1). examples/0615 green
(build()->consume() cross-site + `b : Box(i64) = .none`). Suite green
(671 examples, 447 unit).
2026-06-16 18:54:11 +03:00
agra
e4d24476a9 xfail(reify): typefn identity — Box($T) over reify, two sites one type
REIFY Phase 1.0. Add examples/0615: a type-fn `Box :: ($T)->Type {
return reify(...) }` used at two independent sites (a return type and a
parameter type); consume(build()) typechecks only if both sites resolve
to ONE TypeId. RED by design — reify in a type-fn body still bails
("only supported in a :: binding"). Phase 1.1 routes a reify-returning
type-fn body through reifyType under the mangled instantiation name so
identity holds, turning this green.
2026-06-16 18:47:55 +03:00
agra
04e833a825 docs(reify): name Phase 4 self-reference pair declare()/define()
User picked the declaration-vs-definition split over reserve/complete.
declare() returns a forward nominal Type handle (named from the :: LHS);
define(handle, info) fills its body. reify(info) stays the one-shot
sugar. Updated PLAN-REIFY Phase 4 + Contract 5 + CHECKPOINT-REIFY.
2026-06-16 18:45:08 +03:00
agra
ae5de1e687 docs(reify): Phase 4 self-reference = explicit reserve()/complete()
User-directed API decision: replace the reify_rec((self)=>...) closure
with an explicit reserve() -> Type handle + complete(handle, info) pair.
reserve() returns a forward nominal Type usable freely in any later
TypeInfo (*List, []List, and across types for mutual recursion the
one-self closure couldn't express); reify(info) stays as the one-shot
sugar. Maps onto existing reserve->complete machinery. Captured in
PLAN-REIFY Phase 4 + Contract 5 + CHECKPOINT-REIFY.
2026-06-16 18:38:49 +03:00
agra
353109206b green(reify): implement reify(.enum) — mint a flat enum from TypeInfo
REIFY Phase 0.2 (Phase 0 complete). Lowering.reifyType (lower/nominal.zig)
reads the flat-enum TypeInfo literal off the AST, synthesizes an
ast.EnumDecl, and feeds it through the SAME type_bridge.buildEnumInfo
path source enums use — so the minted type is byte-identical to a
hand-written `enum { value: i64; closed; }` and flows through enum
codegen (layout / construct / match) UNMODIFIED (Contract 2).

Wired at the `E :: reify(...)` const-decl hook in lower/decl.zig
(replacing the Phase-0.0 loud bail). Unsupported argument shapes bail
loudly via reifyBail — never a silent default. The generic.zig inline
reify path now reports it's only supported in a `::` binding (Phase 0).

examples/0614 green: reify a {value: i64, closed} enum, construct
.value(3) and .closed, match both -> "value 3" / "closed". Full suite
green (670 examples, 447 unit).
2026-06-16 18:32:05 +03:00
agra
b25a2f60d6 feat(parser): reserved keyword as member name after .
After a leading `.` (enum literal `.enum`, field access `x.enum` /
`E.struct`, match arm `case .enum:`) a reserved keyword is unambiguously
the member/variant NAME — the dot rules out the keyword reading — so no
backtick escape is needed. A declaration of such a variant still needs
the backtick (enum { `enum: i64 }), since the decl site has no dot.

Adds Parser.dotMemberName() (identifier OR identifier-shaped keyword)
and routes the leading-dot enum-literal and postfix field-access sites
through it. readme updated. The reify example 0614 now uses the cleaner
reify(.enum(...)) spelling (still xfail — reify lands next commit).
2026-06-16 18:22:21 +03:00
agra
1bec54d0c4 xfail(reify): examples/0614-comptime-reify-enum — reify a flat enum
REIFY Phase 0.1. Add the end-to-end Phase-0 example: reify a flat enum
(value: i64, closed) from a TypeInfo literal, construct E.value(3) /
E.closed, and match both arms. Seed an empty expected/*.exit marker.

RED by design (reify still bails -> "unparseable expected exit"); the
next commit (0.2) implements reify and turns it green. Satisfies the
no-commit-both-adds-a-test-and-passes cadence.
2026-06-16 18:08:00 +03:00
agra
81669c72b7 lock(reify): meta.sx surface + bodyless #builtin decls + loud bails
REIFY Phase 0.0. Add the comptime type-metaprogramming surface as the
on-demand module modules/std/meta.sx (NOT the prelude — declaring its
data types in always-loaded core.sx interns them into every module's
type table and shifts every .ir snapshot):

  - EnumVariant / EnumInfo / TypeInfo data types. TypeInfo's variant uses
    the backtick raw escape `enum so it reads as the keyword.
  - reify / type_info / field_type as bodyless #builtin decls.

Each builtin bails LOUDLY when reached unimplemented (no silent default):
  - reify(...) in a :: type-alias position -> decl.zig .call branch
    (also the Phase 0.2 construction hook); poisons the alias .unresolved.
  - reify / field_type in any other type position ->
    generic.zig resolveTypeCallWithBindings.
  - type_info(...) in expression position -> call.zig tryLowerReflectionCall.

Unit test src/parser.test.zig (registered in root.zig) locks that the
decls parse. zig build test green (447 unit, 669 examples).
2026-06-16 17:44:19 +03:00
agra
ded106333b docs(design): execution-model roadmap + reify implementation stream
Add the async-first execution-model roadmap (comptime JIT spine, colorblind
fibers/Io, atomics, hot-reload) with all seven decisions resolved and
three-way reviewed, and carve the first stream: comptime type_info/reify
(PLAN-REIFY + checkpoint) — the codebase-validated foundation for channel
result types and race's synthesized tagged union.
2026-06-16 16:43:29 +03:00
agra
b6a7378af4 feat(dist): bundled-zig link backend for hermetic macOS/Linux/Windows builds
Drive a bundled `zig` as `zig cc` for the AOT link step, supplying lld + CRT
+ libc (musl/glibc/mingw) so `sx build` produces native binaries with no host
toolchain. Default Linux output is static musl (portable-anywhere).

- src/zig_backend.zig: discover zig ($SX_ZIG / bundled-next-to-exe / PATH);
  bundled-vs-PATH provenance gates auto-activation.
- src/target.zig: selectZigLinker + emitZigLinkArgv + zigTargetTriple, dispatched
  before the per-OS branches; macOS/Linux/Windows in scope.
- src/ir/emit_llvm.zig: LLVMNormalizeTargetTriple so vendor-less zig triples
  (e.g. x86_64-windows-gnu) parse to the correct OS/object format (COFF not ELF).
- src/main.zig: --self-contained / --no-self-contained; linux-musl, linux-musl-arm,
  windows-gnu shorthands; de-vendor linux/linux-arm to match the corpus runner.
- examples/1660: Windows Win32 print-42 + exit(0) via kernel32 (ir-only off-Windows).

Auto-activates only for a bundled zig; a PATH-only zig engages under
--self-contained, so native dev/CI builds are never silently rerouted.

Docs: readme Cross-Compilation, design/bundled-zig-link-backend-design.md, current/PLAN-DIST.md.
2026-06-16 15:56:06 +03:00
agra
0e0ee40528 docs(asm): symbol refs are portable — explain the auto-:c mechanism
Updates the symbol-operand guide: x86 now uses the same plain %[fn] as
aarch64, and a 'How the portability works' note explains the mechanism
(compiler auto-injects LLVM's :c modifier for "s" operands, equivalent
to GCC :P/%P0 for x86 calls, no-op on aarch64, overridable). Drops the
stale per-arch :P guidance; checkpoint updated.
2026-06-16 09:05:15 +03:00
agra
066ba54346 feat(asm): portable symbol refs — auto-inject :c operand modifier
A `%[name]` that references a symbol ("s") operand without an explicit
modifier now lowers to `${N:c}` (LLVM 'bare constant — no punctuation')
instead of `${N}`. This makes `bl %[fn]` / `call %[fn]` portable across
targets with no per-arch knowledge: x86 would otherwise render `$cb`
(an invalid call target, requiring a hand-written `:P`); aarch64 is
unaffected. Verified `:c` is equivalent to `:P` for x86-64 calls (both
emit R_X86_64_PLT32), and correct for branch targets, RIP-relative
addressing, and `$`-prefixed absolute immediates.

renderAsmTemplate injects `:c` only for symbol operands lacking an
explicit modifier (asmNamedIsSymbol helper); an explicit `%[name:X]`
still wins (escape hatch). x86 example 1659 drops its `:P` for the same
plain `%[fn]` as aarch64 1656. Snapshots regen to `${N:c}`. zig build
test green (668 corpus, 446 unit).
2026-06-16 09:04:23 +03:00
agra
79042ab9ab docs(asm): note x86 %[fn:P] call modifier + checkpoint x86 coverage 2026-06-16 08:37:09 +03:00
agra
17e3b91eb9 test(asm): x86_64 cross-arch siblings for place + symbol operands
Adds ir-only x86_64 examples mirroring the aarch64 feature examples, so
each emit path is locked on both arches:
- 1657 read-write `+`  → "incq ${0}", "=r,0" (tied input)
- 1658 indirect `=*m`   → "movq $$42, ${0}", "=*m"(ptr elementtype i64)
- 1659 symbol `"s"`     → "call ${2:P}", direct call to an exported sx fn

Each is x86-pinned (ir-only on this aarch64 host — the .ir is the
assertion; runs on x86_64-linux, main returns 0 on success / 1 if the
asm misbehaved). x86 templates validated by cross-emitting an object
(LLVM's integrated assembler accepts them; objdump confirms 1659 is a
direct `call` reloc to cb). Note: x86 direct calls need the `P` operand
modifier (`%[fn:P]`); aarch64 `bl %[fn]` needs none. Pure additive
locks, no compiler change. zig build test green (668 corpus, 446 unit).
2026-06-16 08:36:33 +03:00
agra
a0face7571 docs(asm): document symbol operands ("s") + checkpoint
Adds a 'Symbol inputs — "s" = fn' section to docs/inline-assembly.md
(direct bl/call, portability, the export-vs-callconv linkage point) and
logs the symbol-operand + round-trip work in CHECKPOINT-ASM.
2026-06-16 08:26:22 +03:00
agra
10f4137cbd feat(asm): symbol operands ("s") — direct call/branch to a function
A `"s"` input operand feeds a function/global symbol; the template's
%[name] emits the platform-mangled name, so `bl %[fn]` / `call %[fn]`
branches DIRECTLY to it (PC-relative, no register load — one fewer
indirection than register-indirect `blr`).

Lowering: an `"s"` input lowers its RHS normally (a function name →
`ptr @fn`); the rejection added last commit is removed. Emit: a symbol
operand is passed with its OWN llvm type (LLVMTypeOf) and no coercion —
the function value is a `ptr`, and the old coerce-to-register-int path
mistyped it and failed the verifier. New asmIsSymbol helper.

Verified on aarch64: examples/1656 (sx → asm → bl _cb → sx → 42); the
emitted asm is a direct `bl <_cb>` (objdump-confirmed), IR constraint
`...,s,...`(ptr @cb). Flipped 1656 from the rejection lock to a runnable
aarch64 example. zig build test green (665 corpus, 446 unit).
2026-06-16 08:24:53 +03:00
agra
c187122531 test(asm): reject symbol "s" operands cleanly + lock (symbol-op prep)
A symbol operand (constraint "s") feeds a function/global symbol whose
mangled name the template emits — enabling a DIRECT `bl %[fn]` (one
fewer indirection than register-indirect `blr`). Until now `"s" = fn`
fell through to emit and produced an LLVM-verifier crash (param type
mismatch). Reject it at lowering with a clear diagnostic instead, and
lock that with examples/1656-platform-asm-symbol-operand.sx. The next
commit implements it and flips the example to run (→ 42).
2026-06-16 08:19:18 +03:00
agra
1346a2d020 test(asm): round-trip example — asm calls back into an sx function
Adds examples/1655-platform-asm-callback-into-sx.sx: a global-asm
trampoline (_caller) that `bl _cb` back into an `export`ed sx function.
Demonstrates the sx → asm → sx round trip and that `export` (external
linkage + stable C symbol + C ABI) is what makes the callback symbol
resolvable — `callconv(.c)` alone leaves it internal and it DCE's away.
Runs under the JIT on aarch64-macos (→ 42); ir-only elsewhere. Locks
current behavior; no compiler change.
2026-06-16 07:55:05 +03:00
agra
e7eeecc0f3 docs: move inline-asm design doc to a top-level design/ folder
Moves docs/inline-asm-design.md -> design/inline-asm-design.md (the
internal design record now lives under design/, separate from the
user-facing docs/). Updates all links: current/CHECKPOINT-ASM.md,
current/PLAN-ASM.md, current/PLAN-EXTERN-EXPORT.md (../docs -> ../design)
and docs/inline-assembly.md (same-dir -> ../design).
2026-06-16 07:46:01 +03:00
agra
b4d1ce78c3 docs(asm): add user-facing inline-assembly guide
Adds docs/inline-assembly.md — a how-to guide for inline assembly in the
docs/error-handling.md style: mental model, operands (inputs / value
outputs / naming + auto-naming rule), the result-type table, volatile,
clobbers, all three `-> @place` forms (write-through / read-write /
indirect-memory), multi-instruction `#string` templates, global asm +
lib-less extern, the JIT/AOT-yes vs `#run`-no execution model, a
cookbook (read-register, x86_64 syscall, divmod), and rules of thumb.
All aarch64 snippets are verified to run; x86_64 ones are labeled. The
design doc (docs/inline-asm-design.md) stays as the internal rationale;
this guide is the user-facing companion, linked from readme.md.
2026-06-16 07:41:14 +03:00
agra
73f5f0ed11 docs(asm): checkpoint comptime-call guard (1654) 2026-06-16 07:29:56 +03:00
agra
ab7fc393b6 test(asm): pin loud failure of #run into a module-asm symbol
Adds examples/1654-platform-asm-global-comptime-call.sx — the comptime
guard. A module-asm symbol only exists after assemble+link; the comptime
interpreter resolves extern calls via host dlsym, where it's absent, so
`#run my_add(…)` fails with a clear diagnostic ("comptime extern call:
symbol not found via dlsym") rather than misfiring. Runtime calls work
(1648/1653). dlsym-miss precedes asm assembly, so arch-independent — no
.build. Locks current behavior; no compiler change.
2026-06-16 07:29:38 +03:00
agra
66e1e39418 docs(asm): correct stale 'AOT only' module-asm prose (JIT works)
sx run compiles to an object before ORC relocation, so module asm is
assembled in and its symbols resolve at JIT main execution. Corrected
the Phase F note, Current state, and Next step; the only real boundary
is a compile-time #run into a module-asm symbol (loud dlsym-miss).
2026-06-16 07:25:32 +03:00
agra
e954f044d8 test(asm): global asm runs under the JIT (sx run), not just AOT
Adds examples/1653-platform-asm-global-jit.sx — a module-scope asm { … }
block executed via `sx run` (no `aot`). sx run compiles the module to an
in-memory object (the integrated assembler assembles the `module asm`
into it), then ORC relocates and runs it, so a module-asm symbol IS
resolvable at JIT main execution — the long-assumed "AOT only" limit was
stale. Sibling of 1648 (same feature via AOT). Locks current behavior
(exit 42); no compiler change.
2026-06-16 07:24:09 +03:00
agra
d5aee7a222 docs(asm): checkpoint indirect-memory =*m — inline asm feature-complete 2026-06-16 07:10:31 +03:00
agra
cb6c032c58 feat(asm): indirect-memory =*m place outputs
Implements indirect-memory (`=*m`) `-> @place` outputs — the last
substantive asm feature. Unlike a write-through `=` output (which
returns a value that is then stored), an indirect output passes the
place ADDRESS to the asm and the asm writes through it; there is no
return slot.

emitInlineAsm:
  - indirect outputs are excluded from the LLVM return type;
  - their pointer is passed as an opaque `ptr` call arg, placed FIRST
    (the arg-consuming constraint order is: output-section indirect
    pointers, then inputs, then read-write tied seeds);
  - each indirect arg gets an `elementtype(T)` call-site attribute
    (required in the opaque-pointer era), T = the pointee type;
  - the store-back loop skips indirect outputs (already written).
New asmIsIndirect helper. Lowering stops rejecting `*` (constraint kept
verbatim; `=*m` reaches the constraint string as-is). asmOperandIndex
is unchanged — indirect outputs still count as operands, so `%[name]`
${N} numbering holds.

Verified by running on aarch64: store-through-pointer (str x9, %[out]
→ 42, IR `=*m,~{x9}` with `ptr elementtype(i64)`) and a mixed case
(indirect + value output + input → `=*m,=r,r`, indirect ptr arg first,
${0}/${1}/${2} correct). 1652 flipped from the rejection lock to a
runnable aarch64 example (ir-only elsewhere). zig build test green
(661 corpus, 446 unit).
2026-06-16 07:09:17 +03:00
agra
2a43713d7f test(asm): lock indirect-memory =*m rejection (Phase G prep)
Adds examples/1652-platform-asm-indirect-mem.sx exercising a `=*m -> @x`
indirect-memory place output. Currently rejected loudly at lowering
("not yet implemented"); this locks that behavior as a passing test.
The next commit implements indirect-memory outputs and flips this
example to run end-to-end (store-through-pointer → 42).
2026-06-16 07:05:05 +03:00
agra
59469f2b2f docs(asm): checkpoint x86_64 syscall-write example (1651) 2026-06-16 06:39:14 +03:00
agra
cdd920b692 test(asm): x86_64 Linux syscall-write example (ir-only lock)
Adds examples/1651-platform-asm-x86-syscall-write.sx — the canonical
inline-asm use case: `write(2)` via a raw x86_64 `syscall` (SYS_write
in rax, fd/buf/count pinned to rdi/rsi/rdx, rcx+r11+memory clobbered,
byte count returned in rax). Exercises register-pinned inputs, a pinned
value output, a pointer input (*u8 -> rsi), and clobbers(.…) lowering
together.

x86-pinned via .build { "target": "x86_64-linux" }: ir-only on this
aarch64 host (the .ir snapshot locks the exact constraint string
`={rax},{rax},{rdi},{rsi},{rdx},~{rcx},~{r11},~{memory}` — the §II.11
silent-miscompile risk zone), runs natively on x86_64-linux printing
"ok\n" (hand-authored .stdout, asserted only in execute mode).

Pure additive test coverage — no compiler change (lock commit).
zig build test green (660 corpus, 446 unit).
2026-06-16 06:38:13 +03:00
agra
9e7661b915 docs(asm): checkpoint 0138 resolved — output-to-const rejection done 2026-06-16 06:30:22 +03:00
agra
2a954ceeb6 fix(0138): diagnose @scalar-const address-of (no storage)
A scalar `::` constant folds to its value and has no storage. The
unary `.address_of` lowering (src/ir/lower/expr.zig) skipped the
alloca path (is_alloca == false) and resolveGlobalRef (scalar consts
get no storage global), falling through to the generic addr_of arm,
which reinterpreted the folded value as a pointer:
`inttoptr (i64 <value> to ptr)`. That wild pointer segfaulted on
deref and emitted invalid stores for inline-asm `-> @const`.

Diagnose instead, in the address_of(identifier) path: a non-alloca,
non-ref-capture, non-pack-elem scope binding (local scalar const) and
a module_const_map name not backed by storage (module scalar const)
both report "cannot take the address of constant '<name>' — a scalar
'::' constant has no storage …" and return a placeholder Ref. Chose
diagnose over materializing read-only storage (consistent with the
fold-only scalar model). Array/struct consts keep real storage and
stay addressable (@K/@LIT unchanged).

Also gives the ASM stream's planned output-to-const rejection for
free — asm `-> @const` lowers through the same path. Regression:
examples/1177-diagnostics-addr-of-const-rejected.sx. Resolves 0138.
2026-06-16 06:29:36 +03:00
agra
c760b92548 issue(0138): @const address-of yields wild pointer; ASM output-to-const BLOCKED
Filed issues/0138: `@const` (address-of a `::` comptime constant) lowers
to `inttoptr (i64 <value> to ptr)` — segfaults on deref, invalid store for
asm `-> @const`. Root cause in src/ir/lower/expr.zig .address_of (not asm).
Marked CHECKPOINT-ASM Next step BLOCKED on 0138 for the output-to-const
rejection item.
2026-06-15 23:18:37 +03:00
agra
97a4050462 docs(asm): checkpoint Phase G — read-write + place outputs 2026-06-15 23:08:24 +03:00
agra
4128416d48 feat(asm): read-write + place outputs
Implements read-write (`+r` / `+{reg}`) `-> @place` outputs. LLVM has
no `+` constraint, so a read-write place lowers to:

  - an output `=` constraint (return slot, stored back through the
    place after the call), with the leading `+` rewritten to `=`; plus
  - a TIED input constraint (the decimal index of that output) appended
    after the regular inputs, seeded with the place's loaded value
    passed as a call arg.

Tied inputs are appended last so existing operand indices (%[name] ->
${N}) are undisturbed; asmOperandIndex stays correct. Lowering no longer
rejects `+` (indirect `*` still rejected). emitInlineAsm grows the
arg/param arrays by the rw count, loads each seed, and emits the tied
constraint.

Verified by running: increment-in-place (41 -> 42) and a mixed case
(rw place + regular input + value output) producing the textbook
"=r,=r,r,0" constraint with correct ${N} indices. 1650 flipped from
the rejection lock to a runnable aarch64-pinned example (ir-only
elsewhere). zig build test green (658 corpus, 446 unit).
2026-06-15 23:07:38 +03:00
agra
335ac52374 test(asm): lock read-write + place-output rejection (Phase G prep)
Adds examples/1650-platform-asm-rw-place.sx exercising a `+r -> @x`
read-write place output. Currently rejected loudly at lowering
("not yet implemented"); this locks that behavior as a passing test.
The next commit implements read-write outputs and flips this example
to run end-to-end (increment-in-place → 42).
2026-06-15 23:00:48 +03:00
agra
967005621a feat(asm): Phase 2 — -> @place write-through outputs
An asm result can be STORED through a place (a local / struct field) instead of
returned; the place output does not join the result tuple.

- parser.zig: `-> @place` parses `@place` as an ordinary address-of expression
  → an out_place operand (the in-function form; reuses the existing `@` prefix).
- inst.zig: AsmOperand gains out_ty (the output slot's value type) so emit can
  build the combined return struct without re-deriving from Inst.ty.
- lower/expr.zig: out_place operand = the lowered @place address, out_ty = the
  pointee. Read-write (`+`) and indirect-memory (`*`) constraints rejected loudly
  (not yet implemented) rather than miscompiled.
- ops.zig emitInlineAsm: the LLVM return type is built from ALL outputs
  (out_value + out_place); after the call, out_place slots are stored through
  their address and out_value slots rebuild the sx result. Fast path when there
  are no place outputs (the struct return IS the result — pure-value asm IR
  unchanged).

Verified: write-to-local (42), struct field, mixed value+place (v=10 b=20), `+`
rejected. Locked with 1649-platform-asm-place-output (mixed, runs on aarch64).

zig build test green (657 corpus, 446 unit).
2026-06-15 22:47:34 +03:00
agra
b8800a234c docs(asm): add Inline Assembly section to readme
Documents the `asm { … }` expression (template + `-> Type` / `= expr` operands +
clobbers), the §II.5 auto-naming rule (register pin → implicit name; echo form
rejected), the result-shape rule (0→void+volatile / 1→T / N→tuple), `#string`
multi-instruction templates, and top-level global asm + lib-less `extern`
call-into. Per the docs-track-changes rule (inline asm is a landed user-facing
feature). Examples are ones verified running in the corpus.
2026-06-15 22:28:10 +03:00
agra
4d75b9323c feat(asm): Phase F — global (module-scope) asm
A top-level `asm { "tmpl", };` block (template only) lowers to LLVM `module asm`;
a lib-less `extern` declaration calls into the symbols it defines (the import
direction reuses the existing C-FFI extern path — no new surface).

- ast.zig: asm_global node (AsmGlobal { template }).
- parser.zig: parseAsmGlobal, dispatched from parseTopLevel on kw_asm — rejects
  `volatile` and any operands/clobbers (template only). The in-function asm
  expression form stays in parsePrimary.
- module.zig: Module.global_asm list; lower/decl.zig captures each template in
  lowerMainAndComptime (the real top-level pass — lowerDecls is dead for
  top-level); emit_llvm.zig emit() appends each via LLVMAppendModuleInlineAsm in
  source order.
- the new node forced asm_global arms in sema.zig (analyzeNode +
  findNodeAtOffset) and semantic_diagnostics.zig (checkBindingNames).

Verified end-to-end: an aarch64 `_my_add` global routine, called via `extern`,
returns 42 — AOT only (the ORC JIT doesn't link module-asm symbols; global-asm
symbols live in the final linked binary). Locked with 1648-platform-asm-global
({ "aot": true, "target": "macos" } → AOT build+run on aarch64, ir-only else).

zig build test green (656 corpus, 446 unit).
2026-06-15 22:22:29 +03:00
agra
d3c6ffed5a feat(asm): Phase E — multi-output asm returns tuples
Replaces the N>1 "Phase E" bail with a shared asmResultType helper (lowering +
inferType) that derives the result type from the out_value operands: 0→void,
1→T, N→a named tuple (fields named via the §II.5 effective-name rule).

Key realization: toLLVMType(tuple) already produces a literal struct {T1,…,Tn} —
exactly what LLVM's multi-output inline asm returns — so emit needs NO change.
Building the op with a tuple result type makes the asm call return the struct,
which IS sx's tuple value (destructured by the normal tuple_get path).

inferType's .asm_expr arm now also delegates to asmResultType (single owner), so
`return asm`, `x := asm`, and `q, r := asm` all agree on the type.

Verified end-to-end on aarch64: split(0x1234)→(lo=52,hi=18), a udiv/msub
divmod→(3,2). IR: `call { i64, i64 } asm "divq ${4}",
"={rax},={rdx},{rax},{rdx},r,~{cc}"(…)` → extractvalue → tuple.

1640 → the x86_64 multi-output IR lock (ir-only); 1647 → a multi-output example
that runs on aarch64.

zig build test green (655 corpus, 446 unit).
2026-06-15 21:55:38 +03:00
agra
5a5e04c6d5 feat(asm): Phase C.1 + D — inline asm codegen (runs end-to-end)
lowerAsmExpr stops bailing and builds the inline_asm op: resolves each operand's
effective name (§II.5 — explicit [name] else the {reg} pin), interns
template/constraints/clobbers, lowers input Refs, derives the result TypeId
(0→void, 1→T). Adds the last deferred validation (every %[name] must name an
operand). Multi-output (N>1) bails with a named "Phase E" diagnostic.

emitInlineAsm (backend/llvm/ops.zig) ports Zig's airAssembly: assembles the LLVM
constraint string (outputs → inputs → ~{clobber}, ',' → '|'), rewrites the
template (%[name]→${N}, %%→%, $→$$, %=→${:uid}), then LLVMGetInlineAsm +
LLVMBuildCall2 (AT&T dialect). Dispatch wired in emit_llvm.zig (replacing the C.0
@panic tripwire).

inferType gains an .asm_expr arm (expr_typer.zig) so a bare `x := asm {…-> T}`
binding types correctly — without it the binding inferred .unresolved and
silently produced 0.

llvm_shim.c: LLVMInitializeNativeAsmParser() — the JIT must assemble inline asm
at run time.

Verified end-to-end on the aarch64 host: `mov`/`add` with register-class inputs
and a value output run (exit 42/99), `nop volatile` runs (exit 0). IR is
textbook: `call i64 asm "add ${0},${1},${2}", "=r,r,r"(…)`.

Locked with 1645 (aarch64 add, runs; ir-only on non-aarch64) + 1646 (:= binding).
Updated 1640 (now Phase-E bail) + 1642 (now runs).

zig build test green (654 corpus, 446 unit).
2026-06-15 21:39:54 +03:00
agra
6c08de8ec1 feat(asm): Phase C.0 — add inline_asm IR op (lock, no behavior change)
Adds the `inline_asm: InlineAsm` opcode to the IR Op union (inst.zig): interned
template + operand list (role/name/constraint/operand) + interned clobber names
+ has_side_effects; the result rides on Inst.ty (void / scalar / tuple).

The new variant forces coverage in the exhaustive Op switches:
- interp.zig: loud bailDetail — inline asm is never comptime-evaluable.
- print.zig: an IR-dump arm.
- emit_llvm.zig: a @panic TRIPWIRE — emit lands in Phase D, and until then
  lowerAsmExpr still bails, so no inline_asm op is ever created. Reaching emit
  would mean lowering switched over before emit was ready; crash loudly rather
  than miscompile.

No behavior change: lowering still bails, the op is constructed only in the new
`inline_asm op shape` unit test (inst.test.zig).

zig build test green (652 corpus, 446 unit).
2026-06-15 21:00:12 +03:00
agra
5f444aae26 feat(asm): Phase B.1 — operand-name validation (echo + duplicates)
Extends lowerAsmExpr with a pinnedRegister(constraint) helper and two §II.5
operand-naming checks, in the compile path before the codegen bail:

- reject the echo form `[eax] "={eax}"` — a label identical to the register its
  own constraint pins is redundant (the operand is already auto-named after the
  register); the useful form is a label that differs (`[quot] "={rax}"`);
- reject duplicate operand names (ambiguous %[name] / result field).

Locked with 1643-platform-asm-echo-name and 1644-platform-asm-duplicate-name.

zig build test green (652 corpus, 445 unit).
2026-06-15 20:41:41 +03:00
agra
1040b8c776 feat(asm): Phase B.0 — validate asm shape in the compile path
Restructures the .asm_expr lowering arm into lowerAsmExpr, which validates the
asm shape with specific named diagnostics BEFORE the not-yet-implemented codegen
bail, so the user sees the real problem first. Two checklist items enforced:

- template must be a compile-time-known string ("..." or #string), not a
  runtime expression;
- an asm with no value outputs must be `volatile` (else its effects could be
  deleted) — mirrors Zig's rule.

Valid shapes still bail with the "codegen not yet implemented" message. Result-
type derivation + the operand auto-naming rule stay deferred to Phase C, where a
real IR op makes the result type observable/testable.

Locked with 1641-platform-asm-missing-volatile (the volatile error) and
1642-platform-asm-nop-volatile (no-output + volatile accepted → codegen bail).

zig build test green (650 corpus, 445 unit).
2026-06-15 20:35:43 +03:00
agra
f8e029d719 feat(asm): Phase A.1 — parse asm { … } into AsmExpr; loud lowering bail
`asm volatile? { "tmpl", [name]? "constraint" (-> Type | = expr), …,
clobbers(.…) }` now parses into a flat-operand AsmExpr/AsmOperand (ast.zig +
parser.zig parseAsmExpr, dispatched from parsePrimary on .kw_asm). `volatile`
and `clobbers` are recognized contextually (not reserved). `-> @place`
write-through is rejected with a clear "Phase 2" parse error.

Codegen is not implemented yet (IR op + LLVM emit are Phases C–E), so lowering
bails LOUD + named via an explicit .asm_expr arm in lower/expr.zig (not the
generic unknown_expr else) — emitPlaceholder makes hasErrors() abort the build
on the message.

The new asm_expr tag forced (and got) arms in three exhaustive Node.Data
switches: sema.zig analyzeNode + findNodeAtOffset, semantic_diagnostics.zig
checkBindingNames — each recurses into template + operand payloads.

Design: adopted the operand auto-naming rule (design §II.5) — name auto-derived
from a {reg} pin, explicit [name] only when it differs or for register-class
operands, echo form rejected. Typing-stage rule; parser stores name: ?[]const u8.

Locked with examples/1640-platform-asm-parse.sx (multi-output divmod: named
operands, register pins, clobbers — parses then bails, called from main).

Also files issue 0137 (pre-existing, orthogonal: `sx run` with no `main`
segfaults via an unguarded JIT entry lookup in target.zig — not an asm bug).

zig build test green (648 corpus, 445 unit).
2026-06-15 20:21:25 +03:00
agra
3c9ecd0b42 feat(asm): Phase A.0 — add kw_asm keyword + lex test
`asm` now lexes as a dedicated `kw_asm` keyword (Token.Tag + keyword map entry).
`volatile` and `clobbers` stay out of the global keyword table — they are
recognized contextually only inside an `asm { … }` body (PLAN-ASM Deviation 4).

- token.zig: kw_asm tag + `.{ "asm", .kw_asm }` map entry.
- lsp/server.zig: classifyToken exhaustive switch gained the .kw_asm arm
  (the new enum value forced coverage — intended tripwire).
- lexer.test.zig (new, wired into root.zig barrel): locks `asm`->kw_asm and
  `volatile`/`clobbers`->identifier.

Lock commit (behavior-locking passing test). zig build test green (445 unit).
2026-06-15 18:32:34 +03:00
agra
c92d11e748 docs(asm): Phase 0.2 — document <name>.build sidecar; Phase 0 complete
CLAUDE.md §Testing + §Test-layout now describe the optional `<name>.build` JSON
config (aot + target keys, ir-only arch-gating, unknown-key-is-error) and list
it among the `expected/` files, replacing the stale standalone `.aot` marker
prose. Closes Phase 0 (corpus target-gating); next is Phase A (kw_asm keyword).
2026-06-15 18:20:33 +03:00
agra
0095584105 test(asm): Phase 0.1 — corpus ir-only branch for cross-target examples
When a `.build` target doesn't match the host, the runner can't execute the
example here, so it verifies via `sx ir --target` only: asserts exit + the `.ir`
snapshot (stdout) + diagnostics (stderr), never `.stdout`. An `.ir` snapshot is
REQUIRED in ir-only mode — its absence is a loud failure, never a silent pass.

- corpus_run.test.zig: ir_only flag (target set & !hostMatchesTarget); first
  dispatch arm runs `sx ir`, sets act_exit/act_err/act_ir; skip stdout in both
  update and verify modes; require ir_raw.
- lock fixture 1639-platform-target-cross (asm-free main, target x86_64-linux,
  checked-in .ir). Verified: corrupt .ir => IR mismatch; delete .ir => require
  failure.

Test-infra only; no compiler code. zig build test green (647 corpus, 444 unit).
2026-06-15 18:19:17 +03:00
agra
c88f4fbcef test(asm): Phase 0.0 — corpus target-gating + .build JSON config
Adds per-example build/run directives to the corpus runner via an optional
`expected/<name>.build` JSON sidecar (`BuildConfig { aot, target }`), replacing
the standalone `.aot` marker. Threads `--target` into the run/build/ir spawns
and gates the execute path on host arch+os match; a cross-target example fails
loudly ("ir-only mode not yet implemented") pending Phase 0.1.

- corpus_run.test.zig: BuildConfig + std.json parse (unknown-key => error),
  hostMatchesTarget (shorthand-expand + arch/os token match, arm64->aarch64),
  withTarget argv helper; unit tests for both.
- migrate 1226/1227 `.aot` markers -> `.build` { "aot": true }.
- lock fixture 1638-platform-target-host (`.build` { "target": "macos" }).

Test-infra only; no compiler code. zig build test green (646 corpus, 444 unit).
2026-06-15 17:37:35 +03:00
3598 changed files with 679399 additions and 298066 deletions

125
CLAUDE.md
View File

@@ -339,21 +339,30 @@ Five active workstreams run in parallel — **IR** (the language compiler),
overhaul, mem.sx + protocol expansion), **LANG** (user-facing language overhaul, mem.sx + protocol expansion), **LANG** (user-facing language
features — diagnostics renderer, heterogeneous variadic packs), and features — diagnostics renderer, heterogeneous variadic packs), and
**ERR** (error handling: separate-channel `!` errors, `try` / `catch` / **ERR** (error handling: separate-channel `!` errors, `try` / `catch` /
`or` / `onfail`, return traces). They touch mostly disjoint files; `or` / `onfail`, return traces), and **COMPILER-API** (the comptime `compiler`
any can be advanced independently. library that supersedes the metatype `declare`/`define` `#builtin`s and the
`#compiler` attribute — **pivoted 2026-06-17** off the byte-weld to a **byte-addressable
bytecode comptime VM** as its foundation; see `current/PLAN-COMPILER-VM.md`). They
touch mostly disjoint files; any can be advanced independently.
1. Read all five checkpoints to see where each stream is paused: 1. Read all checkpoints to see where each stream is paused:
- `current/CHECKPOINT.md` — IR progress tracker. - `current/CHECKPOINT.md` — IR progress tracker.
- `current/CHECKPOINT-FFI.md` — FFI progress tracker. - `current/CHECKPOINT-FFI.md` — FFI progress tracker.
- `current/CHECKPOINT-MEM.md` — MEM progress tracker + issues log. - `current/CHECKPOINT-MEM.md` — MEM progress tracker + issues log.
- `current/CHECKPOINT-LANG.md` — LANG progress tracker. - `current/CHECKPOINT-LANG.md` — LANG progress tracker.
- `current/CHECKPOINT-ERR.md` — ERR progress tracker. - `current/CHECKPOINT-ERR.md` — ERR progress tracker.
- `current/CHECKPOINT-COMPILER-API.md` — COMPILER-API progress tracker
(has a `## ⏯ Resume` block; **pivoted to the comptime VM** — Phase 0 strip
pending, branch `reify`).
2. Read the plan that corresponds to the stream the user wants to advance: 2. Read the plan that corresponds to the stream the user wants to advance:
- `current/PLAN.md` — IR implementation plan. - `current/PLAN.md` — IR implementation plan.
- `current/PLAN-FFI.md` — FFI ceremony reduction plan. - `current/PLAN-FFI.md` — FFI ceremony reduction plan.
- `~/.claude/plans/tidy-doodling-cray.md` — MEM (mem.sx) implementation plan. - `~/.claude/plans/tidy-doodling-cray.md` — MEM (mem.sx) implementation plan.
- `current/PLAN-LANG.md` — LANG implementation plan. - `current/PLAN-LANG.md` — LANG implementation plan.
- `current/PLAN-ERR.md` — ERR implementation plan. - `current/PLAN-ERR.md` — ERR implementation plan.
- `current/PLAN-COMPILER-VM.md`**COMPILER-API active plan** (byte-addressable bytecode
comptime VM, then re-home the compiler-API on it). `design/comptime-compiler-api.md`
is the SUPERSEDED weld design, kept only for history + to scope the Phase 0 strip.
3. Read `specs.md` if you need to understand language behavior. 3. Read `specs.md` if you need to understand language behavior.
4. Pick up from the next incomplete step in the relevant `CHECKPOINT*.md`. 4. Pick up from the next incomplete step in the relevant `CHECKPOINT*.md`.
If the user hasn't said which stream to work on, ask before picking. If the user hasn't said which stream to work on, ask before picking.
@@ -391,7 +400,6 @@ any can be advanced independently.
- **Never modify `src/codegen.zig` in Phases 01.** It is the safety net. - **Never modify `src/codegen.zig` in Phases 01.** It is the safety net.
- In Phase 3, only read specific sections of codegen.zig (grep for the relevant handler). - In Phase 3, only read specific sections of codegen.zig (grep for the relevant handler).
- No step should require reading more than ~1,000 lines of existing code. If it does, split it. - No step should require reading more than ~1,000 lines of existing code. If it does, split it.
- No step should produce more than ~500 lines of new code. If it does, split it.
- If Claude gets confused mid-step, stop, update `current/CHECKPOINT.md` with partial progress, and tell the user to start a new session. - If Claude gets confused mid-step, stop, update `current/CHECKPOINT.md` with partial progress, and tell the user to start a new session.
## Context management ## Context management
@@ -431,10 +439,12 @@ After any compiler change:
- A test is still keyed off its `expected/<name>.exit` marker, so seed an - A test is still keyed off its `expected/<name>.exit` marker, so seed an
empty marker first for a brand-new example (see "Adding a feature"). empty marker first for a brand-new example (see "Adding a feature").
`zig build test` is the only way to run the corpus — there is no standalone `zig build test` is the only way to run the corpus — there is no standalone
shell runner (the legacy `tests/run_examples.sh` was removed). An shell runner (the legacy `tests/run_examples.sh` was removed). Per-example
`expected/<name>.aot` marker switches an example from JIT `sx run` to a build/run directives live in an optional `expected/<name>.build` **JSON** sidecar
`sx build` + execute flow (needed to exercise a C-ABI symbol exported FROM sx (see "Test layout" below): `{ "aot": true }` switches an example from JIT `sx run`
a JIT-resident symbol is invisible to a dlopen'd C dylib). to a `sx build` + execute flow (needed to exercise a C-ABI symbol exported FROM sx
— a JIT-resident symbol is invisible to a dlopen'd C dylib); `{ "target":
"x86_64-linux" }` threads `--target` and arch-gates the example.
### Test layout ### Test layout
@@ -442,23 +452,53 @@ Examples and pinned issue repros use the `XXXX-category-test-name` scheme — a
4-digit number in per-category 100-blocks: `basic` 00xx, `types` 01xx, `generics` 4-digit number in per-category 100-blocks: `basic` 00xx, `types` 01xx, `generics`
02xx, `closures` 03xx, `protocols` 04xx, `packs` 05xx, `comptime` 06xx, `modules` 02xx, `closures` 03xx, `protocols` 04xx, `packs` 05xx, `comptime` 06xx, `modules`
07xx, `memory` 08xx, `optionals` 09xx, `errors` 10xx, `diagnostics` 11xx, `ffi` 07xx, `memory` 08xx, `optionals` 09xx, `errors` 10xx, `diagnostics` 11xx, `ffi`
12xx, `ffi-objc` 13xx, `ffi-jni` 14xx, `vectors` 15xx, `platform` 16xx. 12xx, `ffi-objc` 13xx, `ffi-jni` 14xx, `vectors` 15xx, `platform` 16xx. (Newer
categories have grown past 16xx — `atomics` 17xx, `concurrency` 18xx — and some
share 16xx; the **category is the leading name token**, not the number block.)
Expected output lives in an `expected/` directory **next to the test file**, `examples/` is organized into **per-category subfolders** — the folder name is
split into three streams (no more merged `2>&1`) plus an optional IR snapshot: the leading token of the filename (`ffi-objc`/`ffi-jni` kept whole). The full
`XXXX-category-...` filename is unchanged; the folder just groups it. Each
category folder has its own `expected/` directory holding the snapshots, split
into three streams (no more merged `2>&1`) plus an optional IR snapshot:
``` ```
<root>/XXXX-category-name.sx examples/<category>/XXXX-category-name.sx
<root>/expected/XXXX-category-name.exit # process exit code examples/<category>/expected/XXXX-category-name.exit # process exit code
<root>/expected/XXXX-category-name.stdout # normalized stdout examples/<category>/expected/XXXX-category-name.stdout # normalized stdout
<root>/expected/XXXX-category-name.stderr # normalized stderr examples/<category>/expected/XXXX-category-name.stderr # normalized stderr
<root>/expected/XXXX-category-name.ir # optional `sx ir` snapshot examples/<category>/expected/XXXX-category-name.ir # optional `sx ir` snapshot
examples/<category>/expected/XXXX-category-name.build # optional JSON build/run directives
``` ```
A test is any `<name>.sx` with an `expected/<name>.exit` marker. The runner `issues/` stays **flat** (`issues/<name>.sx` + `issues/expected/<name>.exit`).
scans two roots: `examples/` (the feature suite) and `issues/` (pinned bug A test is any `<name>.sx` with a sibling `expected/<name>.exit` marker. The
repros). Multi-file tests keep companions (`.c`/`.h`, imported `.sx`, fixture runner scans two roots — `examples/` (the feature suite, recursing one level
dirs) under the same `XXXX-` prefix. into category folders) and `issues/` (pinned bug repros) — discovering every
`expected/` directory under each. Multi-file tests keep companions (`.c`/`.h`,
imported `.sx`, fixture dirs) under the same `XXXX-` prefix **in the same
category folder**, and reference them with file-relative imports (e.g.
`#import "XXXX-foo/lib.sx"`), never a repo-root-relative `examples/...` path.
The optional `<name>.build` JSON sidecar carries per-example directives
(unknown keys are a hard error — never silently ignored):
- `"aot": true` — build a native binary and execute it instead of JIT `sx run`.
- `"target": "<triple|shorthand>"` — thread `--target` into every `sx`
invocation and gate on the host. If the target's arch+os **match** the host,
the example runs normally; if they **mismatch** (e.g. `x86_64-linux` on an
aarch64 host), the runner switches to **ir-only** mode — it skips
run/build/exec and asserts only `.exit` + `.ir` + `.stderr` from
`sx ir --target` (`.stdout` is not asserted). An `.ir` snapshot is **required**
in ir-only mode (its absence is a loud failure). This is how arch-pinned
examples (e.g. x86_64 inline-asm) are tested on a non-matching dev host while
still running end-to-end on a matching CI runner.
- `"bundle": { "app": "<rel .app path>", "expect": ["Contents/MacOS", ...] }`
bundle smoke test (requires `"aot": true`). After the `sx build` (which runs the
sx bundler via `default_pipeline`) the runner asserts each `expect` entry exists
under `app` (repo-relative), then `rm -rf`s the `app`. **macOS-host ONLY** — on any
other host the example is SKIPPED (the `.app` + `codesign` are Apple-specific).
Example: `examples/1665-platform-macos-bundle-smoke.sx`.
### Snapshot integrity ### Snapshot integrity
@@ -467,17 +507,34 @@ dirs) under the same `XXXX-` prefix.
Safe workflow: Safe workflow:
1. Fix the code until `zig build test` passes against the **existing** snapshots. 1. Fix the code until `zig build test` passes against the **existing** snapshots.
2. Only run `zig build test -Dupdate-goldens` when you've intentionally changed output (new feature, new test, changed formatting). 2. Only run `zig build test -Dupdate-goldens` when you've intentionally changed output (new feature, new test, changed formatting).
3. After regenerating, review the diff (`git diff examples/expected/ issues/expected/`) to confirm no error messages or empty output were captured. 3. After regenerating, review the diff (`git diff examples/ issues/expected/`) to confirm no error messages or empty output were captured.
**Scope a regen to specific examples with `-Dname`.** A *full* `-Dupdate-goldens`
re-runs and rewrites all ~690 snapshots, so a single flaky/host-divergent example
(AOT links, cross-arch `target` examples, anything that intermittently fails) can
silently clobber a good snapshot. To capture just the example(s) you added, pass
their full repo-relative `.sx` path(s) — now including the category folder —
comma-separated; this rewrites ONLY those and touches nothing else:
```sh
zig build test -Dname=examples/comptime/0625-comptime-weld-struct-field.sx -Dupdate-goldens
zig build test -Dname=examples/comptime/0625-foo.sx,examples/types/0126-bar.sx # verify just these
```
`-Dname` also busts the test-run cache (the corpus enumerates `.sx`/`expected/`
files at RUNTIME, so editing a snapshot alone does NOT force a re-run — a plain
`zig build test` may be served a cached result). Changing `-Dname` — or any
compiler source — forces a fresh run.
### Adding a new language feature ### Adding a new language feature
There is no monolithic smoke file — each feature is its own focused example. There is no monolithic smoke file — each feature is its own focused example.
1. Create `examples/XXXX-<category>-<name>.sx` (next free number in the matching 1. Create `examples/<category>/XXXX-<category>-<name>.sx` (next free number in
category block). the matching category block, in that category's folder).
2. Run it: `./zig-out/bin/sx run examples/XXXX-<category>-<name>.sx` 2. Run it: `./zig-out/bin/sx run examples/<category>/XXXX-<category>-<name>.sx`
3. Seed the marker and capture expected output: 3. Seed the marker and capture expected output:
`: > examples/expected/XXXX-<category>-<name>.exit` then `: > examples/<category>/expected/XXXX-<category>-<name>.exit` then
`zig build test -Dupdate-goldens` `zig build test -Dupdate-goldens`
4. Verify all tests still pass: `zig build test` 4. Verify all tests still pass: `zig build test`
@@ -485,9 +542,9 @@ There is no monolithic smoke file — each feature is its own focused example.
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `examples/XXXX-category-name.sx` | Focused feature example — one feature per file. | | `examples/<category>/XXXX-category-name.sx` | Focused feature example — one feature per file, in its category folder. |
| `examples/expected/XXXX-category-name.{exit,stdout,stderr}` | Expected exit code + the two output streams. Regenerate with `zig build test -Dupdate-goldens`. | | `examples/<category>/expected/XXXX-category-name.{exit,stdout,stderr}` | Expected exit code + the two output streams. Regenerate with `zig build test -Dupdate-goldens`. |
| `examples/expected/XXXX-category-name.ir` | Optional `sx ir` snapshot — present only where lowering shape is locked. | | `examples/<category>/expected/XXXX-category-name.ir` | Optional `sx ir` snapshot — present only where lowering shape is locked. |
| `issues/NNNN-slug.md` | Open-issue / bug-report writeup (mark RESOLVED in a banner when fixed; the `.md` stays). | | `issues/NNNN-slug.md` | Open-issue / bug-report writeup (mark RESOLVED in a banner when fixed; the `.md` stays). |
| `issues/NNNN-slug.sx` (+ `issues/NNNN-slug/`) | The issue's minimal repro, co-located with the `.md`. A repro with an `issues/expected/NNNN-slug.exit` marker runs in the suite; unpinned ones don't. | | `issues/NNNN-slug.sx` (+ `issues/NNNN-slug/`) | The issue's minimal repro, co-located with the `.md`. A repro with an `issues/expected/NNNN-slug.exit` marker runs in the suite; unpinned ones don't. |
| `src/corpus_run.test.zig` | The corpus runner inside `zig build test` — spawns `sx` per example, diffs stdout/stderr/exit (+ optional IR); regenerates snapshots under `-Dupdate-goldens`. | | `src/corpus_run.test.zig` | The corpus runner inside `zig build test` — spawns `sx` per example, diffs stdout/stderr/exit (+ optional IR); regenerates snapshots under `-Dupdate-goldens`. |
@@ -509,10 +566,12 @@ All Zig unit tests live in separate `*.test.zig` files alongside the source they
### Creating a new standalone test ### Creating a new standalone test
1. Create `examples/XXXX-<category>-<name>.sx` (focused example) **or**, for an 1. Create `examples/<category>/XXXX-<category>-<name>.sx` (focused example)
open bug, `issues/NNNN-slug.{md,sx}` (repro co-located with the writeup). **or**, for an open bug, `issues/NNNN-slug.{md,sx}` (repro co-located with
the writeup).
2. Run it: `./zig-out/bin/sx run <path>.sx` 2. Run it: `./zig-out/bin/sx run <path>.sx`
3. Seed the marker (`: > <root>/expected/<name>.exit`) and capture expected: 3. Seed the marker (`: > <dir>/expected/<name>.exit`, where `<dir>` is the
example's category folder or `issues/`) and capture expected:
`zig build test -Dupdate-goldens` `zig build test -Dupdate-goldens`
4. Verify: `zig build test` 4. Verify: `zig build test`
@@ -521,8 +580,8 @@ All Zig unit tests live in separate `*.test.zig` files alongside the source they
When a bug filed under `issues/NNNN-slug.{md,sx}` is fixed: When a bug filed under `issues/NNNN-slug.{md,sx}` is fixed:
1. Move the repro into the feature suite as a regression test: 1. Move the repro into the feature suite as a regression test:
`git mv issues/NNNN-slug.sx examples/XXXX-<category>-<name>.sx`. `git mv issues/NNNN-slug.sx examples/<category>/XXXX-<category>-<name>.sx`.
2. Seed `examples/expected/XXXX-<category>-<name>.exit`, capture with 2. Seed `examples/<category>/expected/XXXX-<category>-<name>.exit`, capture with
`zig build test -Dupdate-goldens`, and review the diff. `zig build test -Dupdate-goldens`, and review the diff.
3. Tighten the example's comment header to describe the feature (keep a one-line 3. Tighten the example's comment header to describe the feature (keep a one-line
`Regression (issue NNNN)` note for provenance). `Regression (issue NNNN)` note for provenance).

View File

@@ -7,7 +7,7 @@ pub fn build(b: *std.Build) void {
const optimize = b.standardOptimizeOption(.{}); const optimize = b.standardOptimizeOption(.{});
const static_llvm = b.option(bool, "static-llvm", "Statically link LLVM (self-contained binary, no LLVM needed at runtime)") orelse false; const static_llvm = b.option(bool, "static-llvm", "Statically link LLVM (self-contained binary, no LLVM needed at runtime)") orelse false;
const llvm_prefix = b.option([]const u8, "llvm-prefix", "Path to LLVM installation") orelse "/opt/homebrew/opt/llvm@19"; const llvm_prefix = b.option([]const u8, "llvm-prefix", "Path to LLVM installation") orelse "/opt/homebrew/opt/llvm@22";
const include_dir = b.fmt("{s}/include", .{llvm_prefix}); const include_dir = b.fmt("{s}/include", .{llvm_prefix});
const lib_dir = b.fmt("{s}/lib", .{llvm_prefix}); const lib_dir = b.fmt("{s}/lib", .{llvm_prefix});
@@ -153,7 +153,7 @@ pub fn build(b: *std.Build) void {
mod.link_libcpp = true; mod.link_libcpp = true;
} }
} else { } else {
mod.linkSystemLibrary("LLVM-19", .{}); mod.linkSystemLibrary("LLVM-22", .{});
mod.linkSystemLibrary("clang-cpp", .{}); mod.linkSystemLibrary("clang-cpp", .{});
// clang-cpp is C++ — need libc++ on macOS // clang-cpp is C++ — need libc++ on macOS
if (target_os != .windows and target_os != .linux) { if (target_os != .windows and target_os != .linux) {
@@ -218,8 +218,47 @@ pub fn build(b: *std.Build) void {
"Regenerate example/issue snapshots instead of verifying them (use with `zig build test`)", "Regenerate example/issue snapshots instead of verifying them (use with `zig build test`)",
) orelse false; ) orelse false;
corpus_opts.addOption(bool, "update_goldens", update_goldens); corpus_opts.addOption(bool, "update_goldens", update_goldens);
// `zig build test -Dname=examples/0213-foo.sx[,examples/0214-bar.sx]` restricts
// the corpus runner to ONLY the named example(s) — full repo-relative `.sx`
// paths, comma-separated. Empty = run every example. Use it to verify or
// regenerate (-Dupdate-goldens) a specific example without re-running (or
// clobbering the snapshots of) the rest of the corpus. Because the value is
// baked into the corpus options module, changing it also busts the cached
// test-run result (the runner enumerates .sx/expected files at RUNTIME, so a
// bare snapshot edit alone would otherwise be served from cache).
const name_filter = b.option(
[]const u8,
"name",
"Run only the named example(s): comma-separated repo-relative .sx paths (e.g. examples/0213-foo.sx)",
) orelse "";
corpus_opts.addOption([]const u8, "name", name_filter);
mod.addOptions("corpus_paths", corpus_opts); mod.addOptions("corpus_paths", corpus_opts);
// `zig build [test] -Dcomptime-flat` defaults comptime evaluation to the
// flat-memory VM (`src/ir/comptime_vm.zig`), with the legacy tagged interpreter
// as the per-eval fallback — the "swap behind a build flag" step of
// `current/PLAN-COMPILER-VM.md`. Default OFF (legacy). The `SX_COMPTIME_FLAT`
// env var enables it too (either turns it on); read in `emit_llvm.zig::init`.
const comptime_flat = b.option(
bool,
"comptime-flat",
"Default comptime evaluation to the flat-memory VM (legacy interp as fallback)",
) orelse false;
// `-Dcomptime-flat-strict` (or env `SX_COMPTIME_FLAT_STRICT`): run EVERY comptime
// eval on the VM with NO legacy fallback — a VM bail becomes a build-gating error
// naming the reason. The enumeration gate for retiring `interp.zig`: when the
// corpus is green under strict mode, the VM handles everything and legacy can be
// deleted. Implies `comptime_flat`.
const comptime_flat_strict = b.option(
bool,
"comptime-flat-strict",
"Run all comptime eval on the VM with NO fallback; a bail is a hard error (interp-retirement gate)",
) orelse false;
const build_opts = b.addOptions();
build_opts.addOption(bool, "comptime_flat", comptime_flat);
build_opts.addOption(bool, "comptime_flat_strict", comptime_flat_strict);
mod.addOptions("build_opts", build_opts);
const mod_tests = b.addTest(.{ const mod_tests = b.addTest(.{
.root_module = mod, .root_module = mod,
}); });

View File

@@ -113,7 +113,10 @@ buildCompilerInstance(const char *filename,
const llvm::SmallVectorImpl<const char *> &extra_flags, const llvm::SmallVectorImpl<const char *> &extra_flags,
char **out_error) char **out_error)
{ {
auto diagOpts = new clang::DiagnosticOptions(); // LLVM 21+: DiagnosticOptions is a plain value passed by reference (no
// longer an IntrusiveRefCntPtr). It must outlive `diags` — both are locals
// in this scope, declared opts-before-engine, so destruction order is safe.
clang::DiagnosticOptions diagOpts;
auto diagIDs = new clang::DiagnosticIDs(); auto diagIDs = new clang::DiagnosticIDs();
clang::DiagnosticsEngine diags(diagIDs, diagOpts, clang::DiagnosticsEngine diags(diagIDs, diagOpts,
new clang::IgnoringDiagConsumer()); new clang::IgnoringDiagConsumer());
@@ -128,7 +131,7 @@ buildCompilerInstance(const char *filename,
driver_args.push_back("-w"); driver_args.push_back("-w");
#ifdef SX_LLVM_PREFIX #ifdef SX_LLVM_PREFIX
static std::string resource_dir = std::string(SX_LLVM_PREFIX) + "/lib/clang/19"; static std::string resource_dir = std::string(SX_LLVM_PREFIX) + "/lib/clang/22";
driver_args.push_back("-resource-dir"); driver_args.push_back("-resource-dir");
driver_args.push_back(resource_dir.c_str()); driver_args.push_back(resource_dir.c_str());
@@ -164,8 +167,10 @@ buildCompilerInstance(const char *filename,
return nullptr; return nullptr;
} }
auto CI = std::make_unique<clang::CompilerInstance>(); // LLVM 21+: setInvocation() was removed — the invocation is constructor-
CI->setInvocation(std::move(invocation)); // injected instead. createDiagnostics(DiagnosticConsumer*) still exists as
// the convenience overload (it builds a default VFS internally).
auto CI = std::make_unique<clang::CompilerInstance>(std::move(invocation));
CI->createDiagnostics(new clang::IgnoringDiagConsumer()); CI->createDiagnostics(new clang::IgnoringDiagConsumer());
return CI; return CI;
} }
@@ -283,8 +288,9 @@ extern "C" LLVMMemoryBufferRef sx_clang_compile_to_object(
return nullptr; return nullptr;
} }
// Compile LLVM module to native object code // Compile LLVM module to native object code.
std::string triple = mod->getTargetTriple(); // LLVM 21+: getTargetTriple() returns a const Triple& (was std::string).
const llvm::Triple &triple = mod->getTargetTriple();
std::string err_str; std::string err_str;
const llvm::Target *target = llvm::TargetRegistry::lookupTarget(triple, err_str); const llvm::Target *target = llvm::TargetRegistry::lookupTarget(triple, err_str);
if (!target) { if (!target) {

View File

@@ -1,24 +1,394 @@
# sx Inline Assembly — Checkpoint (ASM stream) # sx Inline Assembly — Checkpoint (ASM stream)
Companion to `current/PLAN-ASM.md`; design in Companion to `current/PLAN-ASM.md`; design in
[docs/inline-asm-design.md](../docs/inline-asm-design.md). Update after every [design/inline-asm-design.md](../design/inline-asm-design.md). Update after every
commit, one step at a time per the cadence rule (no commit may both add a test commit, one step at a time per the cadence rule (no commit may both add a test
and make it pass). and make it pass).
## Last completed step ## Last completed step
None — plan authored, not yet started. **G (indirect-memory `=*m` place outputs)** — the LAST substantive asm feature.
Unlike a write-through `=` output (which returns a value then stored), an
indirect output passes the place ADDRESS to the asm and the asm writes through
it — no return slot. `emitInlineAsm` (`src/backend/llvm/ops.zig`): indirect
outputs are excluded from the LLVM return type; their pointer is an opaque `ptr`
call arg placed **first** (arg-consuming constraint order = output-section
indirect pointers → inputs → read-write tied seeds); each gets an
`elementtype(T)` call-site attribute (required in the opaque-pointer era) via
`LLVMCreateTypeAttribute`/`LLVMAddCallSiteAttribute`; the store-back loop skips
them. New `asmIsIndirect(e, op)` helper. Lowering (`lowerAsmExpr`) stops
rejecting `*` (constraint kept verbatim, `=*m` reaches the constraint string
as-is). `asmOperandIndex` unchanged — indirect outputs still count as operands,
so `%[name]``${N}` holds. Verified by **running** on aarch64: store-through-
pointer (`str x9, %[out]` → 42, IR `"=*m,~{x9}"(ptr elementtype(i64) …)`) and a
mixed case (indirect + value output + input → `"=*m,=r,r"`, indirect ptr arg
first, `${0}/${1}/${2}` correct). Two commits per cadence: (1)
`examples/1652-platform-asm-indirect-mem.sx` locked the rejection; (2) implemented
+ flipped 1652 to a runnable aarch64-pinned example (`{ "target": "macos" }`,
ir-only elsewhere). `zig build test` green (661 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1652-*`.
Prior: **G (read-write `+` place outputs)** — a `+r` / `+{reg}` `-> @place` output is now
implemented. LLVM has no `+` constraint, so a
read-write place lowers to: an output **`=`** constraint (return slot, stored back
through the place after the call; the leading `+` rewritten to `=` in
`appendAsmConstraints`), **plus** a **tied input** (the decimal index of that
output) appended **after** the regular inputs, seeded with the place's loaded
value passed as a call arg. Tied inputs come **last** so existing operand indices
(`%[name]``${N}`) are undisturbed — `asmOperandIndex` unchanged. Lowering
(`lowerAsmExpr`) no longer rejects `+` (indirect `*` still rejected loudly).
`emitInlineAsm` (`src/backend/llvm/ops.zig`): grows arg/param arrays by the rw
count (`n_args = n_inputs + n_rw`), loads each seed (`asm.rw.seed`), emits the
tied constraint, and the existing store-back path writes the modified output back.
New `asmIsReadWrite(e, op)` helper. Verified by **running**: increment-in-place
(41→42, IR `"=r,0"`) and a mixed case (rw place + regular input + value output) →
textbook `"=r,=r,r,0"` with correct `${N}` indices and args `(input, seed)`. Two
commits per cadence: (1) `examples/1650-platform-asm-rw-place.sx` locked the
rejection; (2) implemented + flipped 1650 to a runnable aarch64-pinned example
(`{ "target": "macos" }`, ir-only elsewhere). `zig build test` green (658 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`examples/1650-*`.
Prior: **2**`-> @place` write-through outputs. An asm result can be **stored through
a place** (local / struct field) instead of returned; the place output does NOT
join the result tuple. Parser: `-> @place` parses the `@place` as an ordinary
address-of expression → an `out_place` operand (`src/parser.zig`). Lowering
(`lowerAsmExpr`): out_place operand = the lowered `@place` address, `out_ty` =
the pointee; read-write (`+`) and indirect-memory (`*`) constraints rejected
loudly (not yet implemented). Added `out_ty: TypeId` to the IR `AsmOperand`
(`src/ir/inst.zig`) so emit builds the **combined** return struct (ALL outputs).
`emitInlineAsm` rewrite (`src/backend/llvm/ops.zig`): the LLVM return type is now
built from every output's `out_ty`; after the call, out_place slots are
`store`d through their address and out_value slots rebuild the sx result — with a
**fast path** (no place outputs → the asm's struct return IS the result, so
pure-value asm IR is unchanged). Verified: write-to-local (`get42`→42), struct
field (`@p.b`), mixed value+place (`v=10 b=20`), `+` rejected. Locked with
`examples/1649-platform-asm-place-output.sx` (mixed, runs on aarch64). `zig build
test` green (657 corpus, 446 unit). Files: `src/parser.zig`, `src/ir/inst.zig`,
`src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`, `examples/1649-*`.
Prior: **F** — global (module-scope) asm. A top-level `asm { "tmpl", };` block (template
only) lowers to LLVM `module asm`, and a lib-less `extern` calls into the symbols
it defines. New `asm_global` AST node (`src/ast.zig`) + `parseAsmGlobal`
(`src/parser.zig`, dispatched from `parseTopLevel` on `kw_asm`) — rejects
`volatile` and any operands/clobbers. The node forced (and got) arms in the same
three `Node.Data` switches as `asm_expr` (`sema.zig` ×2, `semantic_diagnostics.zig`).
`Module` gains a `global_asm: ArrayList([]const u8)` (`src/ir/module.zig`);
`lowerMainAndComptime` captures each template (the dead `lowerDecls` is NOT the
top-level pass — `lowerRoot` Pass 2 uses `lowerMainAndComptime`); `emit_llvm.zig`'s
`emit()` appends each via `LLVMAppendModuleInlineAsm` (source order). Verified
end-to-end: an aarch64 `_my_add` global routine called via `extern` returns 42.
Locked with `examples/1648-platform-asm-global.sx`
(`.build { "aot": true, "target": "macos" }` → AOT build+run on aarch64, ir-only
elsewhere). `zig build test` green (656 corpus, 446 unit). **(Correction, later:
module asm ALSO runs under the JIT — `sx run` compiles to an in-memory object,
the integrated assembler assembles the `module asm` into it, ORC relocates and
runs it, so the symbol is resolvable at JIT main execution. The original "AOT
only" note was wrong; see 1653 for the JIT sibling. The genuine boundary is a
COMPILE-TIME `#run` call into a module-asm symbol, which fails loud via host
dlsym-miss — see 1654.)** Files: `src/ast.zig`, `src/parser.zig`, `src/sema.zig`,
`src/ir/semantic_diagnostics.zig`, `src/ir/module.zig`, `src/ir/lower/decl.zig`,
`src/ir/emit_llvm.zig`, `examples/1648-*`.
Prior: **E** — multi-output tuples. **Inline asm now returns tuples.** Replaced the
N>1 bail with a shared `asmResultType` helper (`src/ir/lower/expr.zig`, mixed
into `Lowering`) that derives the result type from the `out_value` operands
(0→void, 1→T, N→named tuple, named via the §II.5 effective-name rule). The key
realization: `toLLVMType(tuple)` already produces a literal struct `{T1,…,Tn}`
exactly LLVM's multi-output asm return — so **emit needed NO change**; building
the op with a tuple result type makes the asm call return the struct, which IS
sx's tuple value (destructured by the normal `tuple_get` path). `inferType`'s
`.asm_expr` arm now also delegates to `asmResultType` (single owner), so
`return asm`, `x := asm`, and `q, r := asm` all agree on the type. Verified
end-to-end on aarch64: `split(0x1234)``(lo=52, hi=18)`, a udiv/msub divmod→
`(3, 2)`. IR is textbook: `call { i64, i64 } asm "divq ${4}",
"={rax},={rdx},{rax},{rdx},r,~{cc}"(…)` → extractvalue → tuple. Converted 1640 to
the x86_64 multi-output IR lock (ir-only) + added `1647-platform-asm-aarch64-multi`
(runs on aarch64). `zig build test` green (655 corpus, 446 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `src/ir/expr_typer.zig`,
`examples/164{0,7}-*`.
Prior: **C.1 + D** — inline asm CODEGEN (lowering builds the op + LLVM emit). **Inline
assembly now runs end-to-end.** `lowerAsmExpr` (`src/ir/lower/expr.zig`) stops
bailing: it resolves each operand's effective name (§II.5 auto-naming), interns
template/constraints/clobbers, lowers input `Ref`s, derives the result `TypeId`
(0→void, 1→T), and builds the `inline_asm` op. Added a `%[name]`-references-a-
real-operand check (the last deferred validation). Multi-output (N>1) still bails
loudly ("Phase E"). `emitInlineAsm` (`src/backend/llvm/ops.zig`, port of Zig's
`airAssembly`): assembles the LLVM constraint string (outputs→inputs→`~{clobber}`,
`,``|`), rewrites the template (`%[name]``${N}`, `%%``%`, `$``$$`, `%=`
`${:uid}`), then `LLVMGetInlineAsm` + `LLVMBuildCall2` (AT&T). Dispatch wired
(`emit_llvm.zig`, replacing the C.0 `@panic`). **`llvm_shim.c`**: added
`LLVMInitializeNativeAsmParser()` — the JIT must assemble inline asm at run time.
Verified end-to-end: aarch64 `add`/`mov` run on the host (exit 42), `nop volatile`
runs (1642 now exit 0), IR is textbook (`call i64 asm "add ${0},${1},${2}",
"=r,r,r"(…)`). Locked with `examples/1645-platform-asm-aarch64-add.sx` (runs on
aarch64, ir-only elsewhere via `.build` + `.ir`). Also added the `inferType`
`.asm_expr` arm (`src/ir/expr_typer.zig`, 0→void / 1→T) — without it a bare
`x := asm {…-> T}` binding inferred `.unresolved` and silently produced 0;
regression-locked with `examples/1646-platform-asm-value-binding.sx`. Updated
1640 (now Phase-E bail) + 1642 (now runs). `zig build test` green (654 corpus,
446 unit). Files: `src/ir/lower/expr.zig`, `src/backend/llvm/ops.zig`,
`src/ir/emit_llvm.zig`, `src/ir/expr_typer.zig`, `llvm_shim.c`,
`examples/164{0,2,5,6}-*`.
Prior: **C.0** — IR op `inline_asm` (lock; no behavior change). Added `inline_asm:
InlineAsm` to the IR `Op` union + the `InlineAsm` struct (`template: StringId`,
`operands: []const AsmOperand` {role/name/constraint/operand}, `clobbers:
[]const StringId`, `has_side_effects`) in `src/ir/inst.zig` — all strings
interned, operands in source order, result on `Inst.ty`. The new variant forced
(and got) arms in two exhaustive `Op` switches: `src/ir/interp.zig` (loud
`bailDetail` — inline asm is never comptime-evaluable) and `src/ir/print.zig`
(IR dump). `src/ir/emit_llvm.zig` gets a `@panic` **tripwire** — emit lands in
Phase D, and until then `lowerAsmExpr` still bails so no `inline_asm` op is ever
created (reaching emit would be a lowering-switched-over-too-early bug). Unit
test `inline_asm op shape` in `src/ir/inst.test.zig`. `zig build test` green
(652 corpus, 446 unit). Files: `src/ir/inst.zig`, `src/ir/interp.zig`,
`src/ir/print.zig`, `src/ir/emit_llvm.zig`, `src/ir/inst.test.zig`.
Prior: **B.1** — operand-name validation (design §II.5 auto-naming rule). Extended
`lowerAsmExpr` with a `pinnedRegister(constraint)` helper (`"={eax}"``eax`,
`"+{rax}"``rax`, `"=r"`→null) and two checks: (1) **reject the echo form**
`[eax] "={eax}"` — a label identical to its own pinned register is redundant
(the operand is already auto-named after the register); (2) **reject duplicate
operand names** (ambiguous `%[name]` / result field). Locked with
`examples/1643-platform-asm-echo-name.sx` + `1644-platform-asm-duplicate-name.sx`.
`zig build test` green (652 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`.
Prior: **B.0** — asm shape validation (compile-path diagnostics). Restructured the
`.asm_expr` lowering arm into `lowerAsmExpr` (`src/ir/lower/expr.zig`, mixed into
`Lowering` in `src/ir/lower.zig`): it validates BEFORE the not-yet-implemented
codegen bail, so the user sees the real problem first. Two checklist items now
enforced with named diagnostics: (1) **template must be a compile-time-known
string** (`"..."` / `#string`); (2) **no value outputs ⇒ must be `volatile`**
(mirrors Zig — a result-less asm could be deleted). Valid shapes still bail with
the "codegen not yet implemented" message. Result-type derivation + auto-naming
stay deferred to a later step (observable only once Phase C produces a real IR
op). Locked with `examples/1641-platform-asm-missing-volatile.sx` (volatile
error) + `1642-platform-asm-nop-volatile.sx` (volatile no-output accepted →
codegen bail). `zig build test` green (650 corpus, 0 failed; 445 unit). Files:
`src/ir/lower/expr.zig`, `src/ir/lower.zig`, `examples/164{1,2}-*`.
Prior: **A.1** — parse `asm { … }` + loud lowering bail (folded A.1+A.2 into one honest
lock commit, since the loud bail IS current correct behavior — cadence option
(a)). Added `AsmExpr`/`AsmOperand` to `src/ast.zig` + the `asm_expr` `Node.Data`
arm; `parseAsmExpr` in `src/parser.zig` (`parsePrimary` `.kw_asm` dispatch) —
parses the template, flat operand list (`[name]? "constraint" -> Type` value
output / `= expr` input), and `clobbers(.…)`; `volatile`/`clobbers` recognized
contextually via `isContextualWord`. The new `asm_expr` tag forced (and got)
arms in three exhaustive `Node.Data` switches: `src/sema.zig` `analyzeNode` +
`findNodeAtOffset`, `src/ir/semantic_diagnostics.zig` `checkBindingNames` (all
recurse into template + operand payloads). Lowering bails LOUD + named in
`src/ir/lower/expr.zig` ("inline assembly codegen is not yet implemented…") via
an explicit `.asm_expr` arm (not the generic `unknown_expr` else) returning
`emitPlaceholder`. `-> @place` write-through is rejected with a clear "Phase 2"
parse error. Locked with `examples/1640-platform-asm-parse.sx` (multi-output
`divmod`, named operands, register pins, clobbers — parses then bails; called
from `main`). `zig build test` green (648 corpus, 0 failed; 445 unit). Files:
`src/ast.zig`, `src/parser.zig`, `src/sema.zig`, `src/ir/semantic_diagnostics.zig`,
`src/ir/lower/expr.zig`, `examples/1640-*`.
Prior: **A.0**`kw_asm` keyword (first compiler code). Added the `kw_asm` `Token.Tag`
variant + `.{ "asm", .kw_asm }` keyword-map entry in `src/token.zig`; `volatile` /
`clobbers` deliberately stay OUT of the global table (contextual). New exhaustive
`Tag` switch in `src/lsp/server.zig` `classifyToken` flagged the missing arm (the
intended coverage tripwire) — added `.kw_asm` to the keyword group. Lock test in
new `src/lexer.test.zig` (`asm``kw_asm`, `volatile`/`clobbers``identifier`),
wired into the `src/root.zig` barrel as `lexer_tests`. `zig build test` green (648
corpus, 0 failed; 445 unit, 0 failed — +1). Files: `src/token.zig`,
`src/lexer.test.zig`, `src/root.zig`, `src/lsp/server.zig`.
Prior: **0.2** — CLAUDE.md docs for `<name>.build`; **Phase 0 COMPLETE**.
**0.1** — corpus runner **ir-only branch** for cross-target examples. Replaced
0.0's loud placeholder bail: when `cfg.target` doesn't match the host (`ir_only`),
`sweepRoot` skips run/build/exec and verifies via `sx ir --target` only —
asserting `.exit` (ir cmd) + `.ir` (normalized stdout) + `.stderr`, never
`.stdout` (write skipped in update mode, assertion skipped in verify mode). An
`.ir` snapshot is **required** in ir-only mode — its absence is a loud failure
("needs an .ir snapshot for ir-only mode"). Locked with
`examples/1639-platform-target-cross.sx` (asm-free `main :: () -> i64 { return 0;
}`), `.build` `{ "target": "x86_64-linux" }`, + checked-in `.ir`. Verified both
guards fire: corrupting the `.ir` → IR mismatch; deleting it → the require-failure.
`zig build test` green (647 corpus, 0 failed; 444 unit). Files:
`src/corpus_run.test.zig`, `examples/1639-*`.
## Current state ## Current state
Design fully converged (`docs/inline-asm-design.md`). Feasibility confirmed: **Inline assembly works end-to-end: 0, 1, and N value outputs (tuples).** Full
`llvm_api.c.*` exposes `LLVMGetInlineAsm` / `LLVMBuildCall2` / pipeline: lex (A.0) → parse (A.1) → validate (B.0/B.1 + `%[name]` check) → IR op
`LLVMAppendModuleInlineAsm` (LLVM@19). No code written. (C.0) → lower-builds-op + LLVM emit + JIT asm-parser init (C.1/D) → multi-output
tuples (E). Register-class + register-pinned operands, inputs, **symbol operands
(`"s"` → direct `bl`/`call` to a function/global by mangled name)**, clobbers,
`#string` multi-instruction templates, `%[name]`/`%%` rewriting, and the §II.5
auto-naming rule all work and execute on the host JIT. Global `asm { … }` (Phase F) works via
lib-less `extern` under BOTH the JIT (`sx run` → 1653) and AOT (1648) — `sx run`
compiles to an object, so the integrated assembler bakes the `module asm` symbol
in and ORC resolves it. All three `-> @place` output forms now work and execute
on aarch64: **write-through** `=` (Phase 2), **read-write** `+` (tied input), and
**indirect-memory** `=*m` (pointer arg + `elementtype`, asm writes through it).
**Inline assembly is now feature-complete — no substantive features remain.** The
x86_64 syscall-write ir-only example is DONE (1651). Global asm runs under both
JIT (1653) and AOT (1648). `readme.md` now has an "Inline Assembly" section.
Known orthogonal bug: **issue 0137**`sx run` on a program with no `main`
segfaults (`src/target.zig:256-273`, unguarded JIT entry lookup). Pre-existing,
asm-independent; does NOT block the ASM stream (every example has a `main`).
Phase EF feasibility already confirmed against the live tree
(`LLVMGetInlineAsm` / `LLVMBuildCall2` / `LLVMAppendModuleInlineAsm` in LLVM@19
`Core.h`; ERR-stream `extractvalue`→tuple in `emit_llvm.zig:726-927`; lib-less
`extern`, 60 sites; `--target` a global CLI flag).
## Next step ## Next step
**A.0** — add the `kw_asm` keyword (`src/token.zig` Tag + `StaticStringMap`) and a **Inline assembly is feature-complete.** All substantive features are done:
unit lex test. Then A.1 (parse `asm { … }``AsmExpr`, lowering bails loudly). 0/1/N value outputs (tuples), register-class + pinned operands, inputs, clobbers,
`#string` templates, `%[name]`/`%%`/`$`/`%=` rewriting, §II.5 auto-naming, global
`asm { … }` (AOT), and all three `-> @place` output forms — write-through (`=`),
read-write (`+`), and indirect-memory (`=*m`). The x86_64 syscall-write ir-only
example (1651) and the output-to-`const` rejection (issue 0138) are also done.
Global asm runs under BOTH the JIT (`sx run` → object → ORC; 1653) and AOT (1648)
— the earlier "AOT only / `sx run` mishandles module-asm" note was stale and has
been corrected. The one genuine boundary is a COMPILE-TIME `#run` into a
module-asm symbol: the interpreter resolves externs via host dlsym, the symbol
isn't linked yet, so it already fails loud (`comptime extern call: symbol not
found via dlsym`) — pinned by 1654.
Remaining work, all **polish** (optional):
- None substantive. Possible niceties: tighten the `#run`-into-module-asm error
text to name module-asm specifically; broaden clobber validation to a checked
per-arch enum (design doc Phase 4).
Orthogonal: **issue 0137** (no-`main` JIT segfault).
Done since last: output-to-`const` rejection (issue 0138), x86_64 syscall-write
ir-only example (1651).
Orthogonal: **issue 0137** (no-`main` segfault).
## Log ## Log
- (init) Plan + design doc written; ASM stream opened. - (init) Plan + design doc written; ASM stream opened.
- (0.0) Corpus runner target-gating: `<name>.build` JSON config (replaces `.aot`
marker), `--target` threading, `hostMatchesTarget` execute-gate, loud
cross-target placeholder bail. Migrated 1226/1227 `.aot``.build`; locked with
1638 fixture + unit tests. `zig build test` green.
- (0.1) ir-only branch: cross-target examples verify via `sx ir --target` only
(exit+ir+stderr, no stdout; `.ir` required). Locked with 1639 fixture; verified
corrupt-.ir → mismatch and missing-.ir → loud failure. `zig build test` green.
- (0.2) docs: CLAUDE.md documents `<name>.build` JSON sidecar (aot + target +
ir-only gating), replacing stale `.aot` marker prose. **Phase 0 COMPLETE.**
- (A.0) `kw_asm` keyword in token.zig (+ map entry); LSP `classifyToken` switch
coverage; lock test in new `lexer.test.zig` (wired via root.zig). `volatile` /
`clobbers` stay contextual identifiers. `zig build test` green (445 unit, +1).
- (A.1) parse `asm { … }``AsmExpr` + loud lowering bail; `asm_expr` arms in 3
exhaustive `Node.Data` switches; `-> @place` rejected (Phase 2). Adopted operand
auto-naming rule (design §II.5). Locked with 1640 fixture. Filed orthogonal
issue 0137 (no-`main` JIT segfault). `zig build test` green (648 corpus, 445 unit).
- (B.0) asm shape validation in `lowerAsmExpr`: comptime-string template +
no-output⇒volatile, with named diagnostics before the codegen bail. Locked with
1641 (volatile error) + 1642 (volatile accepted). `zig build test` green (650
corpus, 445 unit).
- (B.1) operand-name validation: `pinnedRegister` helper + reject echo form
(`[eax] "={eax}"`) and duplicate names. Locked with 1643 + 1644. `zig build
test` green (652 corpus, 445 unit).
- (C.0) IR op `inline_asm: InlineAsm` + interp `bailDetail` + print arm + emit
`@panic` tripwire (Phase D). No behavior change (lowering still bails). Unit
test `inline_asm op shape`. `zig build test` green (652 corpus, 446 unit).
- (C.1+D) CODEGEN — `lowerAsmExpr` builds the op (effective names, interned
strings, input Refs, 0/1 result type) + `%[name]` validation; `emitInlineAsm`
(constraint string + template rewrite + `LLVMGetInlineAsm`/`BuildCall2`, AT&T);
`inferType` arm; `LLVMInitializeNativeAsmParser` for the JIT. **Inline asm runs
end-to-end.** N>1 bails (Phase E). Locked with 1645 (aarch64 add, runs) + 1646
(`:=` binding); updated 1640/1642. `zig build test` green (654 corpus, 446 unit).
- (E) multi-output tuples — `asmResultType` helper (0→void/1→T/N→named tuple),
shared by lowering + `inferType`. `toLLVMType(tuple)` == LLVM multi-output
struct, so emit unchanged; the asm struct return IS the sx tuple. Runs on
aarch64 (1647: `split``(lo,hi)`); 1640 → x86 multi-output IR lock (ir-only).
`zig build test` green (655 corpus, 446 unit).
- (F) global asm — `asm_global` AST node + `parseAsmGlobal` (top-level, rejects
volatile/operands); `Module.global_asm` captured in `lowerMainAndComptime`;
`emit()` appends via `LLVMAppendModuleInlineAsm`; call-into via lib-less
`extern`. AOT-verified (1648, `_my_add`→42). `zig build test` green (656 corpus).
- (docs) readme.md "Inline Assembly" section (b8800a2).
- (2) `-> @place` write-through — `out_place` operand; `out_ty` on the IR
AsmOperand; `emitInlineAsm` builds the combined output struct + splits
(out_place → store-through, out_value → result), fast-path when no places.
`+`/`*` rejected. Locked with 1649 (mixed, runs). `zig build test` green (657
corpus, 446 unit).
- (G) read-write `+` place outputs — `+` lowers to an output `=` + a tied input
(output-index constraint) seeded with the place's loaded value, tied inputs
appended last (operand indices undisturbed). `appendAsmConstraints` rewrites
`+``=`; `emitInlineAsm` grows args by the rw count + loads seeds;
`asmIsReadWrite` helper. Lowering stops rejecting `+` (`*` still rejected). Two
commits (cadence): 1650 locked the rejection, then flipped to a runnable
aarch64 example (`"=r,0"` IR). `zig build test` green (658 corpus, 446 unit).
- (0138) output-to-`const` rejection — fixed the underlying general bug: scalar
`@const` (address-of a folded `::` constant) reinterpreted the value as a
pointer (`inttoptr`). `src/ir/lower/expr.zig` `.address_of` now diagnoses a
scalar const (local + module) instead of falling through; array/struct consts
keep storage. asm `-> @const` gets the clean diagnostic for free (same path).
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`. Issue 0138
RESOLVED. `zig build test` green (659 corpus, 446 unit).
- (x86 syscall) x86_64 Linux `write(2)` via raw `syscall` — locks the constraint
string `={rax},{rax},{rdi},{rsi},{rdx},~{rcx},~{r11},~{memory}` (register-pinned
inputs + pinned value output + pointer input + clobbers). ir-only on aarch64
(`.ir` asserted), runs on x86_64-linux (hand-authored `"ok\n"` stdout).
`examples/1651-platform-asm-x86-syscall-write.sx`. Pure additive lock, no
compiler change. `zig build test` green (660 corpus, 446 unit).
- (G indirect) indirect-memory `=*m` place outputs — the place address is passed
as an opaque `ptr` arg (with an `elementtype(T)` call-site attr), placed before
inputs; asm writes through it; no return slot; store-back skips it.
`asmIsIndirect` helper; lowering stops rejecting `*`. Verified by running on
aarch64 (store-through → 42; mixed indirect+value+input → `"=*m,=r,r"`). Two
commits (cadence): 1652 locked the rejection, then flipped to a runnable aarch64
example. **Inline asm now feature-complete.** `zig build test` green (661 corpus,
446 unit).
- (jit) explored "asm in JIT": found it ALREADY works — `sx run` emits an
in-memory object (integrated assembler bakes in both in-function inline asm and
`module asm`), then ORC relocates+runs it. The stale "AOT only / `sx run`
mishandles module-asm" checkpoint prose was corrected. Locked global-asm-under-
JIT with `examples/1653-platform-asm-global-jit.sx` (`{ "target": "macos" }`, no
aot, → 42). `zig build test` green (662 corpus, 446 unit).
- (comptime guard) pinned the one genuine module-asm boundary:
`examples/1654-platform-asm-global-comptime-call.sx``#run` into a module-asm
symbol fails loud (`comptime extern call: symbol not found via dlsym`) because
the interpreter resolves externs via host dlsym before link. Arch-independent
(no `.build`). `zig build test` green (663 corpus, 446 unit).
- (round trip) `examples/1655-platform-asm-callback-into-sx.sx` — global-asm
trampoline that `bl _cb` back into an `export`ed sx function (sx→asm→sx, → 42).
Documented that `export` (external linkage + C symbol + C ABI) is what makes
the callback resolvable; `callconv(.c)` alone leaves it `internal` (DCE'd).
`zig build test` green (664 corpus, 446 unit).
- (symbol ops) symbol operands (`"s"`) — feed a function/global symbol; the
template emits its platform-mangled name so `bl %[fn]` is a DIRECT branch (one
fewer indirection than register-indirect `blr`, portable — no hardcoded `_`).
Emit passes the operand with its own llvm type (LLVMTypeOf), no coercion
(`asmIsSymbol` helper); lowering lowers the function RHS to `ptr @fn`. Decided
AGAINST mirroring Zig (which has no symbol operand — 483 std asm sites, none
call a function) because the direct `bl` matters. Two commits (cadence): 1656
locked the rejection (replacing an LLVM-verifier crash), then implemented +
flipped to a runnable aarch64 example (objdump-confirmed direct `bl <_cb>`).
`zig build test` green (665 corpus, 446 unit).
- (x86 cross-arch) ir-only x86_64 siblings so each emit path is locked on BOTH
arches: 1657 read-write (`"incq ${0}","=r,0"`), 1658 indirect (`"movq $$42,
${0}","=*m"`(ptr elementtype)), 1659 symbol (`"call ${2:P}"`, direct call). x86
templates validated by cross-emitting an object (integrated assembler accepts;
objdump confirms 1659's direct `call` reloc). Pure additive locks. `zig build
test` green (668 corpus, 446 unit).
- (symbol portability) made `%[fn]` portable across arches — `renderAsmTemplate`
auto-injects LLVM's `:c` modifier (`${N}``${N:c}`) for symbol (`"s"`) operands
lacking an explicit modifier (`asmNamedIsSymbol` helper). Without it x86 renders
`$cb` (a bad `call` target needing a hand-written `:P`); aarch64 unaffected.
Verified `:c``:P` for x86-64 calls (both → `R_X86_64_PLT32`). Explicit
`%[fn:X]` still wins (escape hatch). 1659 dropped its `:P` → same plain `%[fn]`
as aarch64 1656; both IRs regen to `${N:c}`. `zig build test` green (668 corpus,
446 unit).
## Known issues ## Known issues
None yet. - **0138** — RESOLVED. `@const` (address-of a `::` comptime constant) yielded a
wild pointer (`inttoptr (i64 <value> to ptr)`). Fixed by diagnosing scalar
`@const` in `src/ir/lower/expr.zig` `.address_of` (no storage; array/struct
consts unaffected). Delivered the ASM "output-to-`const` rejection" for free.
Regression `examples/1177-diagnostics-addr-of-const-rejected.sx`.
- **0137** — `sx run` on a program with no `main` segfaults (unguarded JIT entry
lookup, `src/target.zig:256-273`). Pre-existing, asm-independent. Filed
`issues/0137-jit-run-no-main-segfault.md`. Does not block A.1.

View File

@@ -0,0 +1,132 @@
# CHECKPOINT-ATOMICS — Stream A (atomics lowering)
Companion to [PLAN-ATOMICS.md](PLAN-ATOMICS.md). Update after every step (one step at a
time, per the cadence rule). New corpus category: `17xx`.
## Last completed step
**A.2 (CAS) — DONE** (A.2a lock + A.2b green). `compare_exchange`/`_weak` → LLVM `cmpxchg`
(result **`?T`, null = SUCCESS**; failure carries the actual value for retry). New IR op
`atomic_cmpxchg` + `AtomicCmpxchg{ptr, cmp, new, val_ty, success_ordering, failure_ordering,
weak}`. `emitAtomicCmpxchg`: `LLVMBuildAtomicCmpXchg` (success/failure orderings, singleThread=0)
`{T, i1}` pair; `LLVMSetWeak` for weak; `?T` result = `{ extractvalue 0 (actual),
xor(extractvalue 1, true) }` (has_value = NOT success). comptime_vm arm does real single-thread
CAS (read/compare/store-on-equal, build `?T`; weak == strong at comptime). Recognizer
(`atomic_cmpxchg`/`_weak`, 6 args) — CAS restricted to INTEGER T; BOTH orderings via
`atomicOrderingFromNode`; dual-ordering validation (failure may not be release/acq_rel nor
stronger than success, `atomicOrderingRank`). Methods `compare_exchange`/`_weak` on `Atomic($T)`
with comptime `$success`/`$failure: Ordering`. `examples/1702` green (CAS ok→20 / fail actual=20 /
weak retry loop 100→105); `examples/1186` locks a rejected ordering pair; unit test `emit: atomic
cmpxchg (strong + weak)` asserts `cmpxchg` + `cmpxchg weak`. Suite green (718/0).
### A.1 (RMW) — DONE (A.1a lock + A.1b green)
`fetch_add/sub/and/or/xor` + `fetch_min/max` → LLVM `atomicrmw` (returns OLD value). New IR op
`atomic_rmw` + `RmwKind` (no `nand`); `LLVMBuildAtomicRMW` with binop from kind, signed/unsigned
`Min/Max` from `val_ty`. RMW restricted to INTEGER T (float fadd / pointer RMW out of scope,
rejected loudly); all five orderings valid for RMW. comptime_vm does real single-thread RMW.
`examples/1701` green; unit test locks `atomicrmw add` + signed `min` vs unsigned `umin`.
## Next step
**A.3 — fence** (`atomic_fence($o: Ordering)` → LLVM `fence`), per PLAN-ATOMICS. No value
result; ordering must be acquire/release/acq_rel/seq_cst (relaxed is meaningless for a fence —
reject loudly). New IR op `atomic_fence` + dispatch/print/comptime_vm (no-op single-thread) +
`LLVMBuildFence`. Lock-then-green cadence as before.
### Earlier — A.0c (guard hardening)
Adversarial review of A.0 found two CRITICAL silent-wrong defects (raw LLVM verifier errors
via the public intrinsics) + a latent align fallback; all fixed: scalar-kind allowlist +
per-op ordering validity (call.zig), `val_ty` align bail (ops.zig). Locked by examples
1130/1131. Suite green (713/0).
### Earlier — A.0b (green)
Real atomic load/store emission: `LLVMBuildLoad2`/`LLVMBuildStore`
+ `LLVMSetOrdering` + mandatory `LLVMSetAlignment`, ordering via an explicit
sx-tag→`LLVMAtomicOrdering` switch (`llvmOrdering`). `examples/1700` green (7/42/43); IR
shows `load atomic i64, ptr … seq_cst, align 8` + `store atomic …`. Added unit test
`emit: atomic load/store (seq_cst, aligned)` in `emit_llvm.test.zig` (asserts `load
atomic`/`store atomic`/`seq_cst`/`align 8`). No fragile full-module `.ir` snapshot for 1700
(it uses `print`); the unit test is the emission-shape gate. Suite green (710 + units).
### Earlier — A.0a (lock commit)
Full atomic load/store plumbing with LLVM emission deliberately bailing loudly;
`examples/1700` locked to the bail diagnostic.
- `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (`init`/`load`/
`store`, **seq_cst-only** — see capability gap below), `atomic_load`/`atomic_store`
`#builtin` decls. **Opt-in import**, NOT in the universal `std.sx` facade (mirrors
`trace`) — putting `Ordering` in the prelude grew every program's type table 378→380 and
churned 37 `.ir` snapshots; reverted.
- IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` (all 5) + structs (inst.zig);
print arms (print.zig); comptime_vm arms reuse load/store (single-thread correct);
recognizer `tryLowerAtomicIntrinsic` (call.zig) — const-ordering-literal guard +
scalar-size guard, both loud; emit dispatch arms (emit_llvm.zig) → `emitAtomicLoad`/
`emitAtomicStore` (ops.zig) currently BAIL via `comptime_failed`.
## A.0.5 — full ordering surface (DONE)
`Atomic($T).load($o: Ordering)` / `store(v, $o)` — ordering is a COMPTIME value param,
explicit (Rust-style, no default; design §4.6). `a.load(.acquire)` emits `load atomic …
acquire`; `a.store(v, .release)` emits `store atomic … release`; `a.load(.release)` is a
compile error (per-op validity guard fires through the method path). Recognizer
`atomicOrderingFromNode` now resolves a comptime-bound ordering identifier via
`comptimeIntNamed` (+ `atomicOrderingFromTag`, with the sx-Ordering ↔ IR-AtomicOrdering
declaration-order invariant documented). 1700 migrated to explicit orderings (output
unchanged 7/42/43). Suite green (715/0).
**Unblocked by three comptime-value-param commits (workers):** enum (3c4305f), tagged_union
(d7a6857), generic-struct methods (d95ba0a). NOTE: default VALUES for comptime params on
generic-struct methods are NOT bound (orthogonal gap — free-fn defaults work); atomics
sidesteps it cleanly by requiring explicit ordering (matches the design). Candidate
follow-up, not an atomics blocker.
## Known issues / capability gaps
- **RESOLVED:** comptime-constant ordering propagation — landed via comptime value params
(3c4305f / d7a6857 / d95ba0a); A.0.5 migrated the methods, no seq_cst-only legacy.
- **Orthogonal gap (not an atomics blocker):** default VALUES for comptime params don't bind
on generic-struct methods (free-fn defaults DO work). Atomics requires explicit ordering
(design-aligned), so it's unaffected. Candidate future fix.
- **Cosmetic:** an invalid ordering passed through a method (`a.load(.release)`) reports the
diagnostic at the lib forward site (`atomic.sx`), not the user's call. Loud + correct, but
the span could be improved by threading the call-site span. Polish.
- **Latent (observed, not yet filed):** calling an *unrecognized* bodiless `#builtin`
silently returns 0 / no-ops with exit 0 (that's how 1700 behaved before recognition
landed) — a silent-fallback footgun in the generic builtin-call path, independent of
atomics. Flag to user; candidate `issues/` entry.
## Decisions (Stream A specifics; surface locked in design §4.6)
- `Atomic($T)` = pure-sx transparent 1-field struct (NO new IR type); ops = `#builtin`
intrinsics emitted as new IR ops. Minimal compiler surface.
- Ordering is compile-time-only (const enum literal), baked into the op as a Zig enum;
non-literal = loud diagnostic. sx tag → LLVM ordering via explicit switch (LLVM enum is
non-contiguous: 2/4/5/6/7).
- Atomic load/store REQUIRE explicit alignment (`LLVMSetAlignment`) — verifier mandate.
- Comptime VM treats atomics as ordinary load/store (single-thread ⇒ correct), not a bail.
- **Snapshot scope corrected:** `.ir` (LLVM IR) is arch-invariant for atomics → ONE host
`.ir` per op, not arch-gated x86/aarch64 pairs (they'd be byte-identical). Asm-level arch
divergence + weak-memory semantics are OUT of corpus scope (stress harness, Stream C).
## Log
- **carve** — wrote PLAN-ATOMICS.md + CHECKPOINT-ATOMICS.md; grounded the intrinsic path,
switch sites, LLVM-C API (no `LLVMBuildAtomicLoad`; use `LLVMBuildLoad2`+`SetOrdering`+
`SetAlignment`), and corrected the arch-`.ir` misconception (`sx ir` emits arch-invariant
LLVM IR). Stream ready; A.0a is the first implementation step.
- **A.0a** — landed lib (atomic.sx, opt-in import) + IR ops (atomic_load/atomic_store +
AtomicOrdering) + recognizer + print/vm arms + emit BAIL; locked `examples/1700` to the
bail diagnostic. Reverted a universal-facade wiring that churned 37 `.ir` snapshots
(Ordering would bloat every program's type table). Suite green (710/0).
- **A.0b** — real atomic load/store emission (LLVMBuildLoad2/Store + SetOrdering +
SetAlignment; explicit sx→LLVM ordering switch). 1700 green (7/42/43, `load atomic …
seq_cst, align 8`). Unit test added. Suite green (710 + units).
- **A.0c** — guard hardening from the adversarial review: scalar-kind allowlist + per-op
ordering validity (call.zig), val_ty align bail (ops.zig), + diagnostic examples
1130/1131. Suite green (713/0). (comptime enum value params landed via worker 3c4305f.)
- **A.0.5** — full ordering surface: `Atomic($T).load/store($o: Ordering)` comptime ordering
(explicit). Recognizer resolves comptime-bound ordering via `comptimeIntNamed`. 1700
migrated to explicit orderings (acquire/release/relaxed/seq_cst). Unblocked by
comptime-value-param workers (3c4305f/d7a6857/d95ba0a). Suite green (715/0).
- **A.1** — RMW: atomic_rmw op + RmwKind + recognizer (rmwKindFromName, integer-only) +
7 fetch_* methods/intrinsics. A.1a bail-lock → A.1b real LLVMBuildAtomicRMW (signed|unsigned
min/max). comptime_vm real RMW. 1701 + unit test. Suite green (716/0).
- **A.2** — CAS: atomic_cmpxchg op + recognizer (dual-ordering validation) + emit (?T from
{actual,!success}) + comptime VM. compare_exchange/_weak methods. examples 1702 + 1186.
Review agent died; self-verified comptime↔runtime agreement, sub-8, ordering edges.
(Commits dca396e/79895be; A.2 has_value fix folded into A.3a.)
- **A.3** — swap (atomicrmw xchg) + fence (new atomic_fence op). A.3a bail-lock → A.3b real.
examples 1703/1704/1187 + unit test. Stream A feature-complete. Suite green (721/0).

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,478 @@
# CHECKPOINT-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step at a time,
per the cadence rule). New corpus category: `18xx` concurrency.
## Last completed step
**B1.3b-1 — the x86_64 / Win64 `swap_context` sibling — VALIDATED on real hardware.** The
context switch is now proven on a SECOND architecture + ABI. A Win64 `swap_context` saves the
COMPLETE Win64 callee-saved set — 8 GP (rbx, rbp, rdi, rsi, r12-r15) + rsp **and xmm6-xmm15**
(10 XMM, 128-bit via `movups` — Win64 has callee-saved XMM, unlike SysV/aarch64) — plus a Win64
`scribble_verify` (32-byte shadow + 16-align at each `call`, COFF symbols, rsp-carried return
addr). Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only here): the 2-fiber mutual scribble printed **`0 0 P`** when built `--target
x86_64-windows-gnu --self-contained` and **run on a Windows 7 x64 VM (UTM)** — every GP + XMM
callee-saved survived. **Adversarially reviewed before the VM run** (worker emitted the real `.s`
and verified every `call` alignment, the 264-byte frame offsets, the rsp/return-addr round-trip,
swap ordering, and COFF naming against the Win64 ABI — no critical/minor bugs). The build→VM→run
loop was set up this session (cross-build needs `--self-contained`; output via the Win32
`WriteFile` boundary, the 1660 pattern). Suite green. Note: this is the GOOD-swap-only mutual
scribble (self-validating by construction; the in-process negative control was dropped to avoid an
sx fn-ptr-convention rabbit hole — the detection of this exact logic was negative-controlled on
aarch64 in 1808). The SysV/Linux x86_64 sibling (different reg set: no callee-saved XMM, args
rdi/rsi) remains for a Linux x86_64 host.
### Earlier — B1.3b-2 — mmap guard-page stacks (commit `dd532ab`)
Fiber stacks are `mmap`'d with a `PROT_NONE` GUARD PAGE at the low end (§8.1.1: a
fixed stack without a guard silently corrupts neighbors on overflow). `mmap` the `[guard |
usable]` region, `mprotect` the low 16KB page `PROT_NONE`; SP descends into the guard and faults
loudly at the boundary instead of corrupting a neighbor. Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
- **Guard FIRING validated** (manually, not corpus-pinned — a deliberate overflow crash is
host-fragile): a fiber recursing past its 128KB stack faults with `Bus error` at the guard page
(`region+GUARD`); the sx crash handler turns it into exit 134. Documented in the example header.
- **x86_64 sibling:** was deferred here (couldn't run x86_64 on this arm64 host), then DONE as
Win64 once a Windows 7 x64 VM became available — see B1.3b-1 above (`examples/1810`, `0 0 P`).
### Earlier — B1.3a-2 — the context-switch STRESS GATE (design §10.7) — DONE + adversarially reviewed
The explicit every-callee-saved-register scribble that B1.3a-1 owed. `swap_context` now saves the
COMPLETE AAPCS64 callee-saved set — integer x19-x28 + fp/lr + sp AND FP **d8-d15** (per §6.1.2
only the low 64 bits of v8-v15 are callee-saved, so `d8-d15` is exactly sufficient; x18 is Apple's
reserved platform reg, untouched). A naked `scribble_verify(self_ctx, peer, base)` loads a unique
sentinel into all 18 callee-saved regs, yields, and on resume counts the ones that didn't survive
(honoring its own caller ABI via a 176-byte frame that saves+restores the caller's callee-saved;
base reloaded from the frame post-swap; the original lr round-trips through the swap). The gate is
a **2-fiber MUTUAL scribble** (A and B scribble DISTINCT sentinels into the same physical regs, so
each survives only if `swap_context` saved+restored it — a lone fiber yielding to an idle peer
would NOT exercise preservation). Locked by `examples/1808-concurrency-fiber-switch-stress.sx`
(aarch64-pinned): `A mismatches: 0` / `B mismatches: 0`.
- **Validity proven by NEGATIVE controls:** dropping the d8-d15 save/restore → 8/8 mismatches
(exactly the FP regs); dropping x27/x28 → 2/2. The gate genuinely catches a broken switch.
- **Adversarial review (worker, per the plan): no CRITICAL bugs.** Verified the callee-saved set
is complete + correct, all frame offsets/16-alignment, the lr/sp dance, and swap read-ordering
against AAPCS64. Applied its one recommendation: `boot` now zeroes the FP ctx slots [13..20] so a
first switch-to loads 0 (not garbage) into d8-d15. Residual gaps it flagged (all spec-correct
for a call-boundary swap, documented in the example header): NZCV/FPSR not swapped; **FPCR**
(rounding mode — thread-global, bleeds across fibers if changed) and **TPIDR_EL0/TLS** (errno,
allocator thread-caches — shared by same-thread fibers) not swapped; fp=0 bootstrap blocks
unwind/signal walking past a fiber trampoline. These bite at the N×M:1 / signals stages, not the
single-thread switch.
- Suite green **734/0**, master clean. WIP probes: `.sx-tmp/scribble2.sx` (+ `_broken`/`_gp`).
### Earlier — B1.3a-1 — the foundational stackful context switch (commit `b234b7d`)
Pure sx over `abi(.naked)`: naked `swap_context` (GP-only 13-slot save) + by-hand fiber bootstrap
(SP = `alloc_bytes` stack top, LR = global-asm trampoline, x19 = `*Fiber`). Locked by
`examples/1807-concurrency-fiber-context-switch.sx`: 2-fiber ping-pong (`rounds: 6` / `canary
fails: 0`) + 64-frame deep recursion (`frames verified: 64` / `depth fails: 0`). Indirect
register/stack survival; 1808 supersedes its switch with the complete GP+FP save area + the
explicit gate.
### Earlier — B1.2 COMPLETE — the async surface works end-to-end
All three surface blockers (0151, 0152, 0153) FIXED + committed; async examples landed + green.
- **0151 fixed** (`362674f`): generic `$T` infers through generic-struct / pointer / UFCS-pack
params. Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` load/store byte-promoted to `i8` in the codegen
emitters. Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` now pins return-type resolution to the
fn's DEFINING module (mirroring `monomorphizeFunction`), so a re-exported value-failable's
`!E` resolves to the real `.error_set` TypeId — the failable channel survives the re-export
alias. Regression `1058-errors-reexport-value-failable-channel.sx`.
- **Async examples landed:** `examples/1805-concurrency-io-blocking-async.sx`
(`context.io.async((a,b)->i64 => a+b, 40, 2).await() or {…}``sum: 42` / `double: 42` /
`clock ok`) + `examples/1806-concurrency-io-cancel.sx` (`f.cancel()``await` raises
`.Canceled``or` default; `ok: 7` / `canceled: -99`). Both green, snapshots captured.
### Earlier — the three B1.2 surface fixes (committed)
Generic `$T` inference, `Atomic(bool)` byte-promotion, and re-export failable-channel pin —
details below.
- **0151 fix (committed):** four gaps closed on the inference + UFCS-dispatch path —
(1) `extractTypeParam`/`matchTypeParam(Static)` got a `parameterized_type_expr` arm
(recover the arg instance's recorded per-param bindings via `struct_instance_bindings` +
the template's ordered `type_params`, recurse positionally; this also fixes `*Box($T)`
it recurses into its `Box($T)` pointee); (2) the `pointer_type_expr` arm now falls through
to match the pointee against a non-pointer arg (auto-address-of: a `*Box($T)` param accepts
a by-value `Box($T)`, e.g. a UFCS receiver `b.m()`); (3) `ExprTyper.inferType` got a
`.lambda` arm building the closure type from the lambda's annotations (the UFCS binder types
args from the raw AST before they're lowered, so it can now bind `Closure(..) -> $R` from
the worker's declared return type); (4) a pack UFCS target routes through the SAME
`lowerPackFnCall` the direct call uses, with the receiver spliced in as `args[0]`.
- Regression tests: `examples/0214-generics-ufcs-closure-return-pack.sx` (direct + UFCS
closure-return pack) + `examples/0215-generics-infer-through-pointer.sx` (by-value /
pointer / multi-param / nested / UFCS-auto-ref struct-head inference). Issue 0151 marked
RESOLVED; repro moved into the suite.
### Earlier — B1.2 (Io capability) — LANDED + adversarially reviewed
Commits `a1b14f0` (lock) + `45d869d` (Io capability) + `3eeb965` (issue 0151 lock).
- **LANDED + review-confirmed correct** (commit 45d869d): `Io :: protocol #inline`
(spawn_raw/suspend_raw/ready/poll/now_ms/arm_timer) + `io` field on `Context`
(`{allocator; data; io}`, io LAST); BOTH `__sx_default_context` materializers
(protocol.zig + comptime_vm.zig) build an identical CBlockingIo→Io vtable (review verified
byte-for-byte agreement; `context.io.now_ms()` dispatches at runtime AND comptime); the
`push Context.{…}` omitted-field-**inherits-ambient** fix (review: correct, right fix, no
bad blast radius); `library/modules/std/io.sx` (`Future($R)`, `CBlockingIo`,
`async`/`await`/`cancel`); the `!`-protocol-impl-lint suppression; 37 `.ir` regens
(review: pure layout/type-table, no error text, zero .exit/.stdout/.stderr change).
- **BLOCKED — async surface non-functional:** `await`/`cancel` take `*Future($R)` and are
**uncallable in EVERY form** (not just UFCS) — sx can't infer a generic `$T` from a
pointer-wrapped arg (`*Future($R)`). `async(...)` (create) works via explicit call and
produces a correct `.ready` Future, but you can't `await` it. Root bug = **issue 0151
(WIDENED)**: infer `$T` from `*T`-wrapped params + closure-return-via-pack + UFCS dispatch.
Minimal repro: `unbox :: (b: *Box($T)) -> $T` fails to infer `T`.
- **No async example in the corpus** (1805 was removed because it needs the blocked surface)
→ the green suite does NOT cover async. Restore `1805` (async/await) + add `1806` (cancel)
once 0151 is fixed.
### Earlier — B1.1 (per-fiber `context` root) — DONE. Zero compiler change (confirmed by probe).
The fiber-spawn context convention works end-to-end with ordinary language features:
- `snap := context` captures the spawner's `Context` as a value;
- the snapshot is stored in a struct (the stand-in `Fiber`);
- a trampoline running under a *different* ambient context installs the fiber's stored root
with `push f.root { … }`, and the body reads the snapshot — not the trampoline's ambient
context — because `context` is an implicit slot-0 `*Context` param (call-carried, rides the
callee's own stack) and `push` allocates on the caller frame (no global, no TLS).
- Locked by `examples/1804-concurrency-context-snapshot.sx`: prints `fiber root: 42` (the
installed snapshot wins over ambient 99) + `ambient after: 99` (the `push` scope restores
the ambient context on exit). No fiber runtime yet (that's B1.3) — this proves the plumbing
it will build on. No `.build` pin (pure sx, host-independent).
- **Probe result:** the design doc's "lower as swappable indirection, never raw TLS" guarded
a non-problem — context was already param-carried, never TLS. No path re-reads
`__sx_default_context` mid-stack, so there is **no compiler obligation** here.
- `zig build && zig build test` green: **726 ran, 0 failed**.
### Earlier — B1.0 (`abi(.naked)` codegen) — complete
Replaced the emit bail with real LLVM `naked` emission:
- `emit_llvm` declaration pass: for `func.is_naked`, add the LLVM `naked` + `noinline` +
`nounwind` attributes and **skip** the `frame-pointer=all` attribute (incompatible with a
frameless function). Pass 2 now emits the `.naked` body normally — `naked` makes the
backend emit it verbatim (the inline asm + its own `ret`) with no prologue/epilogue.
- IR shape (verified): `; Function Attrs: naked noinline nounwind` / `define internal i64
@answer() #0 { entry: call void asm sideeffect "…ret…", ""() unreachable }` /
`attributes #0 = { naked noinline nounwind }`. The caller invokes it as an ordinary
`() -> i64` call (`.naked` is `call_conv == .default`).
- `examples/1800-concurrency-naked-asm.sx` — now GREEN, aarch64-pinned (`.build {"target":
"macos"}`): runs end-to-end → **exit 42** on this host, ir-only on a mismatch; `.ir`
snapshot captured.
- `examples/1801-concurrency-naked-generic.sx` (renamed from `-bail`) — the generic `.naked`
now emits a correct naked `answer__i64` (exit 42), proving generic.zig produces a naked
body, not a framed one. aarch64-pinned.
- `examples/1802-concurrency-naked-asm-x86.sx` — x86_64 cross sibling (`.build {"target":
"x86_64-linux"}`, ir-only here): `.ir` locks `naked` + `movl $42, %eax` / `ret`.
- Unit test `emit: abi(.naked) function gets the naked attribute (no frame-pointer)` in
`emit_llvm.test.zig` (asserts `naked` present, `frame-pointer` absent).
- **B1.0c (review-hardening):** a param-bearing `.naked` fn emitted invalid LLVM (loud
verifier error "cannot use argument of naked function") because the param-alloca loop
wasn't gated. Fixed forward (this *enables* the B1.3 context-switch use case rather than
rejecting it): gated the param-alloca loop on `fd.abi != .naked` in decl.zig (both paths) +
generic.zig; a naked fn's args stay in registers (read by asm), declared-but-unused in
LLVM. Locked by `examples/1803-concurrency-naked-asm-param.sx` (`add(a,b)` → x0+x1 → 42).
- `zig build && zig build test` green: **725 ran, 0 failed** + unit tests.
### Earlier — B1.0a (lock + review hardening)
Plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites + generic.zig +
pack.zig); `funcWantsImplicitCtx` skips `.naked` (no synthetic ctx, like `.c`); all
body-lowering paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx
return); `emit_llvm` Pass 2 bailed loudly (since flipped to real emission). Adversarial
review caught the generic/pack `is_naked` gap (a generic `.naked` silently shipped a framed
body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positive
(unparseable — `isLambda` breaks on the `abi` keyword).
## Current state
**B1.2 COMPLETE.** The full async surface (Io capability on Context + `async`/`await`/`cancel` +
blocking `CBlockingIo`) works end-to-end. Master GREEN (732/0), installed `sx` clean. All four
B1.2 surface bugs resolved or deferred:
- **0151 fixed** (`362674f`): generic `$T` through generic-struct / pointer / UFCS-pack params.
Regression `0214` + `0215`.
- **0152 fixed** (`e5586f6`): `Atomic(bool)` byte-promoted to `i8` in the load/store emitters.
Regression `1705`.
- **0153 fixed** (`68c1991`): `inferGenericReturnType` pins return-type resolution to the fn's
defining module, so a re-exported value-failable keeps its `!` channel. Regression `1058`.
- Issue **0150** (`void` struct field → SIGTRAP) DEFERRED — only `Future(void)` / `timeout`,
which are B1.4.
The async examples are landed + green: `1805` (`async`/`await` + `now_ms` → `sum: 42` /
`double: 42` / `clock ok`) + `1806` (`cancel` → `await` raises `.Canceled` → `or` default).
The `18xx` concurrency category now covers naked-asm (1800-1803), context-snapshot (1804), and
the async surface (1805-1806).
### B1.2 Io capability — what is LANDED + verified (commit 45d869d)
- `Io :: protocol #inline { spawn_raw; suspend_raw -> !; ready; poll; now_ms; arm_timer; }`
in `core.sx` next to `Allocator`, with `SpawnOpts{ pin: PinTarget }` + `ParkToken{ handle }`.
Six methods, each justified by a downstream consumer (B1.3-B1.5).
- `Context :: struct { allocator; data; io: Io; }` — `io` appended LAST so `allocator` stays
index 0 (the `call.zig:1229` hardcode) and `data` keeps index 1 (minimal VM-fallback churn).
- Both `__sx_default_context` materializers updated in lockstep + verified: `protocol.zig`
`emitDefaultContextGlobal` (extended `ctx_fields` 2→3, built the `CBlockingIo→Io` inline
7-word vtable `{null-ctx, fn0..fn5}` via `getOrCreateThunks("Io","CBlockingIo")`) and
`comptime_vm.zig` `materializeDefaultContext` fallback (wrote the 6 thunk func-refs at
`io_base = addr + 4*ps`, offset `+ (i+1)*ps`). The global path auto-followed the 3-field
Context type. **`context.io.now_ms()` printed `clock ok` live — the capability threads + the
vtable dispatches correctly.**
- Stateless `CBlockingIo :: struct {}` + `impl Io for CBlockingIo` (mirror of `CAllocator`):
blocking semantics — `spawn_raw`/`ready`/`poll`/`arm_timer` no-op/0, `now_ms` → `time.mono_ms()`.
- **push-inherit-omitted fix** (`stmt.zig` `lowerPush`): a `push Context.{...}` now SEEDS the
new slot from the ambient context (load+store), then overwrites ONLY the literal's named
fields — so omitted fields (now incl. `io`) are INHERITED, never zero-inited to a null
vtable. Eliminates the omitted-field footgun globally (zero per-site churn across the 17
partial-literal sites). This is the correct capability-bag semantics; it compiled clean.
- **`!`-protocol-method warning fix** (`error_analysis.zig` + a new `Lowering.impl_method_names`
set populated in `protocols.zig` `registerImplBlock`): a protocol impl method may be declared
`!` by contract (e.g. `Io.suspend_raw`) yet never raise; the "declared `!` but never errors —
drop the `!`" hint is a false positive for impl methods, now suppressed for them.
Status of the blockers that originally stopped B1.2:
- **issue 0151 — FIXED this session** (generic `$T` through generic-struct / pointer /
UFCS-pack params). `async`/`await`/`cancel` are callable. See "Last completed step".
- **issue 0152 — NEW, the current blocker** (`Atomic(bool)` → sub-byte i1 atomic; LLVM reject).
Blocks the async examples via `Future.canceled: Atomic(bool)`. Filed; codegen-level fix.
- **issue 0150** — `void` struct field SIGTRAP; only `Future(void)`/`timeout` (B1.4). DEFERRED.
Per the IMPASSABLE STOP rule: 0151 fix shipped (suite green 728/0), 0152 filed, STOPPED.
Resume B1.2's async examples once 0152 lands.
### Earlier — B1.0 + B1.1 complete
Stream A (atomics) is feature-complete (✅). Stream B1: **B1.0 + B1.1 complete.** The two
compiler-floor preconditions for the fiber runtime are in place: (1) `abi(.naked)` emits a
real LLVM `naked` function end-to-end (decl, generic, pack paths) — the context-switch
substrate; (2) per-fiber `context` root needs **no compiler change** — the spawn convention
(snapshot `context`, store, `push` it from the trampoline) is pure library sx. No
fibers/Io/scheduler code yet. Grounded floor facts:
- `context` is an implicit slot-0 `*Context` param + `push Context` is a stack `alloca` ⇒
**fiber-local for free** (confirmed by the B1.1 probe — never TLS, never re-read from the
`__sx_default_context` global mid-stack). A spawn passes the snapshot as the fiber-entry
fn's slot-0 ctx via `push f.root { entry(args) }`. Locked by `1804-...-context-snapshot`.
- Inline asm works end-to-end (lower→emit→JIT, aarch64 + x86_64) — the `.naked` body reuses it.
- **`.naked` with PARAMS works** (B1.0c, the B1.3 substrate): the param-alloca loop is gated
on `fd.abi != .naked` in decl.zig (both paths) + generic.zig — a naked fn's args stay in
ABI registers (read by the asm body), declared-but-unused in LLVM (verifier-legal).
Example `1803-concurrency-naked-asm-param.sx` (`add(a,b)` reads x0/x1). **Unsupported (loud,
not silent):** a `.naked` *variadic-pack* fn (pack.zig's param loop is intertwined with
comptime-param/`#insert` handling, and a naked fn can't read a runtime-sized pack from
registers anyway) → loud LLVM-verifier error for that nonsensical construct. Acceptable
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
## Next step
**→ B1.4 — `Io` impls / the scheduler.** The switch substrate is proven on TWO arch/ABI pairs
(aarch64 native + x86_64/Win64 on the VM), with the §10.7 stress gate, guarded mmap stacks, and
adversarial review. That's enough to build the scheduler on. B1.4 builds the deterministic-sim
`Io` (calibrated against blocking `Io` before trusting it — §8.1.3), then **B1.5** (M:1 scheduler)
replaces the hand-bootstrapped ping-pong with real `spawn`/`yield`/`resume` over the switch. The
§10.7 gate (1808) + guarded-stack path (1809) + the Win64 sibling (1810) must keep passing as the
switch is wrapped into the scheduler.
**Side thread (optional, low priority): the SysV/Linux x86_64 sibling.** A THIRD switch variant
for `x86_64-linux`: SysV callee-saved = rbx, rbp, r12-r15 + rsp (6 GP + sp; **no** callee-saved
XMM, unlike Win64) — a 7-slot ctx, args rdi/rsi/rdx, the rsp-carried return addr. Needs a Linux
x86_64 host (or a working cross-run) to RUN + the mutual-scribble gate. Not blocking — the switch
is already validated on two arch/ABI pairs.
**Deferred (do NOT block on these):** issue **0150** (`void` struct field SIGTRAP) — only
`Future(void)`/`timeout` (B1.4). The **`::` callable-parameter feature** (named-fn async workers
`async(read_a, conn)`) — WIP at `.sx-tmp/wip-callable-params/patch.diff` (parser done, inference
incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
`Context` layout settled: `{ allocator; data; io; }` (allocator index 0 fixed by
`call.zig:1229`, io last). Io protocol + materializers + push-inherit are LANDED + reviewed.
## Known issues / capability gaps
- **✅ issue 0153 — FIXED** (re-exported generic value-failable `($R, !E)` kept its `!` channel:
`inferGenericReturnType` now pins return-type resolution to the fn's defining module).
Regression: `examples/1058`. Was the LAST B1.2 surface blocker.
- **✅ issue 0152 — FIXED** (`Atomic(bool)` sub-byte i1 atomic → byte-promoted to i8 in the
load/store emitters). Regression: `examples/1705`. Unblocked `Future.canceled`.
- **✅ issue 0151 — FIXED** (generic `$T` through generic-struct / pointer / UFCS-pack params).
Regression: `examples/0214` + `0215`. Was the original B1.2 surface blocker.
- **issue 0150** (deferred) — a `void` struct field crashes the compiler (unsized-type SIGTRAP
in LLVM `getTypeSizeInBits`). Blocks `Future(void)` → `timeout` (B1.4). Repro: `issues/0150-...`.
- (Note: **issue 0149**, filed by another session against an earlier dirty binary, was a
manifestation of the pre-fix 0151 — now moot.)
- **Orthogonal (not a B1 blocker):** default VALUES for comptime params don't bind on
generic-struct methods (free-fn defaults DO work) — inherited from Stream A. Only matters
if a B2 lib type wants a defaulted comptime param; atomics/fibers require explicit, so
unaffected.
- **Issue 0144 (open, independent):** calling an unrecognized bodiless `#builtin` silently
returns 0 / exit 0 — a silent-fallback footgun in the generic builtin-call path. Filed;
leave for its own fix session unless prioritized. Not a B1 blocker.
- **Deferred design gap (documented):** the B1.4 event-loop `Io` does not yet cooperate with
a platform UI run loop (CFRunLoop/NSRunLoop/ALooper); pinning gives thread-affinity, not
run-loop integration — a §6 app-target concern, out of B1 scope.
## Decisions (Stream B1 specifics; surface locked in design §4 / §4.6)
- **The async runtime is sx LIBRARY code.** The compiler provides only: the general
primitives (inline asm ✅, `abi(.naked)` naked [B1.0], atomics ✅) + fiber-safe codegen
(`context` already fiber-local — B1.1). Schedulers, fibers, channels, futures, `Io`
vtables, `mmap` stacks are all sx.
- **`abi(.naked)` is the real spelling of the design's `callconv(.naked)`** — postfix slot,
`name :: (sig) -> Ret abi(.naked) { asm { … }; }`. B1.0 = carry it into IR + emit LLVM
`naked` + skip prologue/ctx (mirror the existing `.c` skip), NOT extend the enum (it's
already there, just inert).
- **`.naked` ≠ `.c`:** a `.c` epilogue would restore SP from the wrong stack across a context
switch (SP-in ≠ SP-out by design). `.naked` = no prologue/epilogue/frame; the asm emits its
own `ret`. This is why the switch must be `.naked`.
- **Naming:** sx-facing name is **`naked`** (keyword `abi(.naked)`, field `is_naked`, the
diagnostic), matching LLVM's `naked` attribute and the industry term (Zig/Rust/GCC/Clang).
The ABI variant was renamed `.pure → .naked` (user direction): "pure" universally means
*side-effect-free*, the opposite of a register-clobbering context switch.
- **B1.0 snapshot scope:** a `.naked` body is raw per-arch asm; LLVM's `naked` attr text is
arch-invariant. **B1.0a** = one host example locked to the emit bail (host-independent —
fires before instruction selection; no `.build` pin). **B1.0b** = pin aarch64 + add an
x86_64 cross sibling (`.build` target-gated, ir-only on mismatch), like the asm corpus
split. The `.ir` proves the `naked` attr + asm emitted, NOT register-save correctness
(that's B1.3's stress harness).
- **B1.1 — per-fiber context is library-only (CONFIRMED by probe):** push frames are
stack-`alloca`'d and the implicit ctx rides slot 0, so the spawn convention — snapshot
`context`, store it, `push f.root { entry(args) }` from the trampoline — installs the
fiber's root with no compiler change. Verified: the body reads the snapshot over a different
ambient context, and `push` restores ambient on exit (`1804-...-context-snapshot`). The
design doc's "never raw TLS" guarded a non-problem (context was never TLS).
- **Test keystones (design §10):** the **B1.3 switch-stress harness** gates the
context-switch (the one piece the deterministic `Io` can't test — §8.1.1, §10.7); the
**B1.4 deterministic-sim `Io`** (calibrated against blocking `Io` — §8.1.3) gates all
scheduling tests. Both must exist + be calibrated before the async tests they gate are
trusted. `18xx` asserts program-emitted ordering contracts, not raw interleaving.
## Log
- **carve** — wrote PLAN-FIBERS.md + CHECKPOINT-FIBERS.md. Grounded the B1 compiler floor:
`ABI.naked` inert (type_resolver.zig:237), IR `Function` has no naked flag (inst.zig:605),
attribute API pattern (emit_llvm.zig:1339 nounwind), `.c` ctx-skip precedent
(decl.zig:515), `push Context` stack-alloca + slot-0 implicit ctx (stmt.zig:1263,
lower.zig:259), `__sx_default_context` root (decl.zig:2667/2815), inline-asm corpus
(1645/1651). Corrected the design's `callconv(.naked)` → real `abi(.naked)` spelling and
the B1.0 snapshot story. B1.1 grounded as likely library-only. Baseline green (721/0).
- **B1.0a** — plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites);
`funcWantsImplicitCtx` skips `.naked` (no implicit ctx, like `.c`); both body-lowering
paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx return);
`emit_llvm` Pass 2 bails loudly on `func.is_naked`. `examples/1800-concurrency-naked-asm.sx`
locked to the bail (exit 1 + diagnostic). Suite green (722/0). (ABI variant later renamed
`.pure → .naked` — see the Naming decision above — so all `is_*`/`abi(.*)`/example names
here read `naked`.)
- **B1.0a review-hardening** — adversarial review found generic/pack Function-creation paths
left `is_naked` false (silent framed body for a generic `.naked` instance — returned 42 but
corrupted the stack). Fixed generic.zig + pack.zig (set `is_naked` + asm-only `unreachable`
cap); locked by `examples/1801-concurrency-naked-generic-bail.sx`. The review's `.naked`-
lambda CRITICAL was a false positive (unparseable — `isLambda` breaks on `abi`). Suite
green (723/0).
- **B1.0b** — real `naked` emission: emit_llvm declaration pass adds LLVM `naked`/`noinline`/
`nounwind` + skips `frame-pointer` for `func.is_naked`; Pass 2 emits the body verbatim (no
prologue). `1800` green aarch64-pinned (exit 42 + `.ir`); renamed `1801` → `-generic`
(generic `.naked` emits a naked body, exit 42); added x86_64 sibling `1802` (ir-only, `.ir`
locks `naked` + `movl $42, %eax`). Unit test asserts `naked` present + `frame-pointer`
absent. Suite green (724/0).
- **B1.0c** — review-hardening: param-bearing `.naked` emitted invalid LLVM (loud verifier
error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both paths + generic.zig)
— naked args stay in registers, read by the asm body (the B1.3 context-switch shape).
Locked by `examples/1803-concurrency-naked-asm-param.sx`. Pack `.naked` left unsupported
(loud, nonsensical). **B1.0 complete.** Suite green (725/0).
- **rename** — ABI variant `.pure → .naked` (keyword, `Function.is_naked`, diagnostics,
examples 1800-1803 `*-pure-* → *-naked-*`, docs). "pure" universally means side-effect-free
— wrong for a register-clobbering switch; "naked" matches LLVM/Zig/Rust/GCC/Clang. Pure
cosmetics, no semantic change. Suite green (725/0).
- **B1.1** — per-fiber `context` root: **zero compiler change** (probe-confirmed). The spawn
convention (snapshot `context` → store in a struct → `push f.root { entry() }` from the
trampoline) installs the fiber's root via the implicit slot-0 `*Context` param; the body
reads the snapshot, not the trampoline's ambient ctx, and the `push` scope restores ambient
on exit. Locked by `examples/1804-concurrency-context-snapshot.sx` (prints `fiber root: 42`
/ `ambient after: 99`). Suite green (726/0). **Next: B1.2 (Io interface + context.io).**
- **B1.2 (BLOCKED)** — built the full `Io` capability (protocol on `Context`, stateless
`CBlockingIo` blocking default, both `__sx_default_context` materializers, push-inherit-omitted
fix, `!`-impl-method warning fix) and VERIFIED the core works live (`context.io.now_ms()` →
`clock ok`). Two independent compiler bugs blocked the `async`/`await`/`timeout` layer:
**0150** (`void` struct field → unsized SIGTRAP, blocks `Future(void)`) and **0151** (type-var
from a fn-ptr param's return type not bound in the body, blocks `async`'s `Future(R)`). Both
filed with standalone repros + investigation prompts. Per the STOP rule: reverted ALL B1.2
working changes (master green again, 726/0; the dirty binary had broken the photo project —
see the now-moot 0149), saved WIP to `.sx-tmp/b12-wip/`, STOPPED. Resume after 0150 + 0151.
- **0151 FIXED** — generic inference now binds `$T` through a generic-struct param head, a
pointer (`*Box($T)`, incl. UFCS auto-ref), and a closure-return-via-pack on the UFCS path.
Four gaps closed: `parameterized_type_expr` arm in `extractTypeParam`/`matchTypeParam(Static)`
(recovers the arg instance's recorded per-param bindings, recurses positionally); pointer arm
falls through to match a value arg (auto-address-of); `ExprTyper.inferType` `.lambda` arm
(closure type from annotations — UFCS types args from raw AST pre-lowering); pack UFCS target
routes through `lowerPackFnCall` with the receiver spliced in as `args[0]`. Issue 0151 marked
RESOLVED; repro → `examples/0214-generics-ufcs-closure-return-pack.sx`; widened cases →
`examples/0215-generics-infer-through-pointer.sx`. Suite green 728/0. The now-callable async
surface immediately exposed a SEPARATE codegen bug — **issue 0152** (`Atomic(bool)` → sub-byte
i1 atomic, LLVM reject; `Future.canceled` hits it). Filed with standalone repro + fix prompt.
Per the STOP rule: shipped the 0151 fix, filed 0152, STOPPED. Resume the async examples
(1805/1806) after 0152.
- **0152 FIXED** — the atomic load/store emitters (`src/backend/llvm/ops.zig`) byte-promote a
sub-byte (`bool`→`i1`) access to its `i8` storage type and `trunc`/`zext` the value at the
boundary (new `atomicByteType` helper). rmw/cmpxchg left as-is (a `bool` rmw/CAS is rejected
at the sx level — integer-only — so a sub-byte element never reaches them; comments record
this). Regression `examples/1705-atomics-bool-byte-promoted.sx` (load/store round-trip). Issue
0152 marked RESOLVED. Suite green 729/0. With `Atomic(bool)` working, the async surface
exposed the TRUE remaining blocker — **issue 0153**: a re-exported generic value-failable
`($R, !E)` loses its `!` channel at the call site (the earlier "secondary `or` PHI" symptom
was this, NOT an `Atomic` cascade — confirmed it persists after 0152). Narrowed to the
generic+re-export co-requirement (non-generic re-export OK; direct generic import OK; only the
combination drops `!`). Root cause: the monomorphized return-type's error-set, reached via the
re-export alias, resolves to a non-`.error_set` TypeId, so `errorChannelOf`
(`lower/error.zig:148`) misses the channel. Filed `issues/0153-...` with a minimal co-located
2-file repro + a single-file stdlib-`await` repro + investigation prompt. Per the STOP rule:
shipped the 0152 fix, filed 0153, STOPPED. Resume the async examples after 0153.
- **0153 FIXED → B1.2 COMPLETE** — `inferGenericReturnType` (`src/ir/generics.zig`) resolved the
return-type AST in the CALL-SITE module, so a re-exported error set (`LE :: lib.LE`) resolved
to a non-`.error_set` alias and the planned call-result was a plain tuple (channel lost). Fix:
pin the source to `fd.body.source_file` around the return-type resolution, exactly as
`monomorphizeFunction` does — the `!E` now resolves to the real `.error_set`. One-function
change; full suite green (732/0), no regression. Issue 0153 RESOLVED; repro →
`examples/1058-errors-reexport-value-failable-channel.sx` (+ companion `lib.sx`). With the
channel preserved, landed the async examples: **`1805`** (`async`/`await` + `now_ms` → `sum:
42` / `double: 42` / `clock ok`) + **`1806`** (`cancel` → `await` raises `.Canceled` → `or`
default; `ok: 7` / `canceled: -99`). **B1.2 (Io capability + M:1 async surface) is COMPLETE.**
Next: B1.3 (fiber runtime) on the `.naked` context-switch substrate.
- **B1.3a-1 — context switch works.** Implemented the stackful switch in pure sx over
`abi(.naked)`: `swap_context(from, to)` (save callee-saved x19-x28 + fp/lr + sp into `*from`,
load from `*to`, `ret` onto `to`'s stack) + by-hand fiber bootstrap (SP = top of an
`alloc_bytes` stack, LR = a `.global _fib_tramp` global-asm trampoline that does `mov x0, x19;
bl _fib_body`, x19 = `*Fiber`). Proven via a probe (main↔fiber), then locked by
`examples/1807-concurrency-fiber-context-switch.sx` (aarch64-pinned): a 2-fiber ping-pong
(`rounds: 6`, `canary fails: 0` — a per-fiber stack canary survives every switch) + a 64-frame
deep recursive chain suspended at the bottom and resumed (`frames verified: 64` / `depth fails:
0`). The `bl _fib_body` reaches the sx body via `export "fib_body"` (the 1655 asm→sx pattern);
runs under JIT, ir-only on a non-arm host (`.ir` captured — `swap_context` shows `naked noinline
nounwind`). Suite green 733/0. **Honest scope:** indirect register/stack survival only; the
EXPLICIT every-callee-saved + FP scribble (§10.7) is B1.3a-2, still owed. Next: B1.3a-2.
- **B1.3a-2 — the §10.7 stress gate, adversarially reviewed.** Extended `swap_context` to the
COMPLETE AAPCS64 callee-saved set (added FP d8-d15 → 21-slot ctx) and wrote a naked
`scribble_verify` that loads a unique sentinel into all 18 callee-saved regs, yields, and counts
non-survivors on resume (176-byte frame saves/restores the caller's callee-saved + base; lr
round-trips the swap). The gate is a 2-fiber MUTUAL scribble (each clobbers the other's regs, so
survival ⇒ the switch saved+restored them). Locked by
`examples/1808-concurrency-fiber-switch-stress.sx` (`A/B mismatches: 0`). Validity proven by
negative controls (drop d8-d15 → 8/8; drop x27/x28 → 2/2). **Spawned an adversarial-review
worker (per the plan + user request): NO critical bugs** — callee-saved set complete (x18 rightly
excluded; d8-d15 suffices per §6.1.2), offsets/alignment/lr-sp dance all verified. Applied its
one rec: `boot` zeroes FP ctx slots so first-entry loads 0, not garbage. Honest residual gaps
(spec-correct for a call-boundary swap; in the example header): FPCR/FPSR/NZCV + TPIDR/TLS not
swapped, fp=0 blocks unwind — relevant at N×M:1 / signals, not here. Suite green 734/0.
Next: B1.3b (x86_64 sibling + mmap guard-page stacks).
- **B1.3b — mmap guard-page stacks (x86_64 sibling deferred).** Fiber stacks now `mmap` a
`[guard | usable]` region and `mprotect` the low 16KB page `PROT_NONE`, so a stack overflow
faults at the guard boundary instead of silently corrupting a neighbor (§8.1.1). Locked by
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
Guard FIRING validated manually (overflow → `Bus error` at `region+GUARD`, exit 134 via the sx
crash handler) — not corpus-pinned because a deliberate-overflow crash is host-fragile (and a
mere "child faulted" fork test wouldn't prove the BOUNDARY catch). The x86_64 `swap_context`
sibling was DEFERRED: `--target x86_64-macos` mislinks on this arm64 host and `x86_64-linux`
can't run here, so it could only ship un-run/un-negative-controlled — which §10.7 forbids for the
highest-risk asm. SysV target notes (rbx/rbp/r12-r15/rsp, no callee-saved XMM, rsp-carried return
addr) recorded in Next step. Suite green **735/0**. Next: x86_64 sibling (needs an x86_64 host)
OR B1.4 (`Io` impls / scheduler) on the proven aarch64 substrate.
- **B1.3b-1 — x86_64 / Win64 switch sibling VALIDATED on real hardware.** The user provided a
Windows 7 x64 VM (UTM), so the x86_64 switch became RUNNABLE (as Win64). Validated the
cross-build→VM→run loop (`--target x86_64-windows-gnu --self-contained` → PE32+; output via the
Win32 `WriteFile` boundary, the 1660 pattern). Wrote a Win64 `swap_context` (8 GP rbx/rbp/rdi/
rsi/r12-r15 + rsp + **xmm6-xmm15** via `movups` — Win64 has callee-saved XMM) + a Win64
`scribble_verify` (264-byte frame, 32-byte shadow + 16-align at each `call`, COFF symbols,
rsp-carried return addr) driving the 2-fiber mutual scribble. **Adversarially reviewed (worker
emitted the real `.s`, verified every alignment/offset/round-trip against the Win64 ABI — no
critical/minor bugs), THEN run on the VM → `0 0 P`** (all 8 GP + 10 XMM callee-saved survived).
Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
ir-only on this host; the VM run is the runtime-correctness provenance). Good-swap-only (the
in-process negative control was dropped to avoid an sx fn-ptr-convention rabbit hole; the
detection of this exact logic was negative-controlled on aarch64 in 1808). Suite green **736/0**.
The B1.3 context switch is now proven on TWO arch/ABI pairs. Next: **B1.4** (Io impls / M:1
scheduler) on the proven substrate. (Side thread: the SysV/Linux x86_64 sibling, when a Linux
x86_64 host is available.)

View File

@@ -0,0 +1,276 @@
# CHECKPOINT-METATYPE — comptime type metaprogramming (`declare` / `define`)
Companion to [PLAN-METATYPE.md](PLAN-METATYPE.md). Update after every step (one
step at a time, per the cadence rule).
## Last completed step
**`type_info` / `define` widened to TUPLE types — reflect/construct triad
complete.** `TypeInfo` gained a `` `tuple(TupleInfo) `` variant (`TupleInfo{
elements: []Type }`, positional/unnamed). `reflectTypeInfo` builds `.tuple`
(tag 2) as bare `type_tag` elements; `defineTuple` decodes `[]Type` and completes
the declare slot as a structural `.tuple` via `replaceKeyedInfo` (tuples are
structural, so the declared name is vestigial, but the slot is completed in place
so `define` returns the handle like enum/struct). `call.zig`'s `type_info` guard
admits `.tuple`. `examples/0623` (programmatic `Pair` + source-tuple round-trip).
Suite green (684). All three TypeInfo shapes now reflect + construct + round-trip
(`0619` enum, `0622` struct, `0623` tuple).
## Earlier — struct widening
**`type_info` / `define` widened to STRUCT types.** `TypeInfo` gained a
`` `struct(StructInfo) `` variant (`StructField{ name, type }`); the metatype
system now reflects AND constructs structs, not only enums.
- `meta.sx`: `StructField` / `StructInfo` / `` `struct `` TypeInfo variant.
- `interp.zig`: `reflectTypeInfo` builds `.struct` (tag 1) for a source
`@"struct"`; `define` dispatches on the TypeInfo tag (`defineType` →
`defineEnum` (0) / `defineStruct` (1)). `defineStruct` mirrors `defineEnum`
(duplicate-field-name check included) but completes the declare slot AS a
struct via `replaceKeyedInfo` — a KIND change re-keys the intern map, whereas
`updatePreservingKey` (the enum path) asserts the key is unchanged.
- `lower/call.zig`: the lower-time `type_info` guard now admits `@"struct"`.
- `examples/0622`: programmatic `Vec2` via `.struct(.{ fields = … })` + a
source-struct round-trip `define(declare("RowCopy"), type_info(Row))`. Enum
path (`0619`) unchanged. Suite green (683). Tuple is the last shape (Next step).
## Earlier — make_enum
**`make_enum(name, variants: []EnumVariant) -> Type`** — the general enum
constructor over `declare`/`define`, minting a nominal enum from a variant list
passed as a VALUE. Pure sx in `meta.sx`. `examples/0620` assembles the list in a
local then mints, exercising `define`'s value-arg SLICE decode.
## Prior step
**`type_info($T)` reflection — enum round-trip.** Reflect a type INTO a `TypeInfo`
value (the inverse of `define`'s decode), so `define(declare(n), type_info(T))`
mints a byte-identical copy with NO literal variant list.
- `inst.zig`: new `BuiltinId.type_info` (comptime-only, alongside `declare`/`define`).
- `lower/call.zig:tryLowerReflectionCall`: the old "not yet implemented" bail is
gone. Resolve `$T` at lower time, reject a non-`enum`/non-`tagged_union` arg
loudly (good span: `"type_info: 'X' is not an enum …"`), else emit
`callBuiltin(.type_info, [const_type], TypeInfo)`.
- `interp.zig:reflectTypeInfo`: builds the exact nested-aggregate Value
`defineEnum` decodes — variant `{name, payload}`, slice `{data, len}`, EnumInfo
`{variants}`, TypeInfo `{tag0, EnumInfo}`. A `tagged_union` reflects each
`field.ty` (tagless variants already carry `void`); a payloadless `` `enum ``
reflects `void` per variant. Round-trips both source enums AND constructed
(declare/define) enums.
- emit unchanged — `type_info` is always comptime-evaluated; the existing
comptime-only `else` arm in `emitCallBuiltin` (shared with declare/define)
never fires.
- Scope: **enum-only** (the symmetric inverse of `define`'s current capability).
Struct/tuple `TypeInfo` widening is a separate later step.
`examples/0619` locks it (source enum `circle:f64 / rect:i64 / empty` reflected →
reconstructed → constructs + matches). Full suite green (676 examples + units).
## Earlier step
**Self-reference — recursive enums via `declare("Name")` + `*Name`.** The
`declare`/`define` floor now supports self-referential types.
- `declare(name) -> Type` mints an empty (undefined) nominal slot NAMED `name`;
`define(handle, info) -> Type` decodes the `TypeInfo` value (variant names +
payload Type-tags), fills the slot byte-identical to a source enum, and returns
the handle (one-shot form chains: `T :: define(declare("T"), info)`). Interp
executes both against a `mint` TypeTable handle; `defineEnum` +
`decodeVariantElements` in `interp.zig`.
- **Self-reference:** `evalComptimeType`'s `preregisterForwardTypes` scans the
comptime expression (and a called ctor fn's body) for `declare("Name")` calls
and, before the body lowers, registers each as an empty forward nominal type AND
binds it as a type alias. The alias is essential — a `Name :: ctor()` decl makes
`Name` a const_decl author, so a `*Name` self-reference resolves through the
forward-ALIAS path (`type_aliases_by_source`), which a bare `findByName`
registration alone does NOT satisfy (it returns a pending empty-struct stub). The
interp's `declare` returns that same slot; `define` fills it.
- A `::` binding or type-fn body calling a `Type`-returning fn is
**comptime-evaluated** (`evalComptimeType`) — no constructor-name knowledge.
`decl.zig` trigger = `fnReturnsTypeValue`; type-fn trigger = `returnExprMintsType`.
- Nominal identity rides the type-fn instantiation cache (`renameNominalType`).
- The type NAME is on `declare(name)` (compile-time string), not `EnumInfo`.
Examples green: `0614` (one-shot), `0615` (type-fn identity), `0617` (channel
results), **`0618` (recursive `*List`: construct, match through pointer, recursive
traversal)**; `field_type` reflection `0616`. Full suite green (674 examples).
## Current state
- `modules/std/meta.sx`: `EnumVariant` / `EnumInfo{ name, variants }` / `TypeInfo`
data types; `declare` / `define` / `type_info` / `field_type` `#builtin`s;
`RecvResult($T)` / `TryResult($T)` + the general `make_enum(name, variants)` sx
constructors over `define(declare(), …)`.
- Compiler primitives only: `declare`/`define` (construction), `field_type`
(reflection). No constructor-name knowledge anywhere in the compiler — every
named constructor is sx. `declare(name)` carries the type name (compile-time
string) for forward-type registration.
- `type_info($T)` reflects an `enum`/`tagged_union`/`struct`/`tuple` INTO a
`TypeInfo` value (`call.zig` emits `callBuiltin(.type_info)`;
`interp.zig:reflectTypeInfo` builds the Value). `define` decodes `.enum` →
tagged_union, `.struct` → struct, `.tuple` → tuple (the last via
`replaceKeyedInfo`). `examples/0619` (enum) / `0622` (struct) / `0623` (tuple)
round-trip. All three TypeInfo shapes ship.
## Decision (kept)
**Meta lives in `modules/std/meta.sx`, not the prelude.** Declaring its data types
in the always-loaded prelude interns them into every module's type table and
shifts every `.ir` snapshot. On-demand import keeps the prelude clean.
## Next step
The reflect/construct triad is COMPLETE — `` `enum `` (`0619`), `` `struct ``
(`0622`), `` `tuple `` (`0623`) all reflect AND construct + round-trip. Remaining
METATYPE work is ONE deferred enhancement, a clean diagnostic rather than a crash
— filed as **issue 0141** (repro `issues/0141-*.sx` + full two-layer writeup +
investigation prompt):
- **Comptime `List` growth** — `List(T).append` at comptime bails ("struct_get:
base has no fields"). Doesn't block anything: array-literal locals already build
variant lists (`examples/0620`/`0624`). Probe `.sx-tmp/probe_makeenum.sx` /
`probe_li64.sx`. **Investigated — it's TWO layers** (both reproduce with plain
`List(i64)`, not metatype-specific; List works via `#run` because that evaluates
at EMIT time, after everything is lowered, while a metatype `::` const evaluates
at `scanDecls` time):
1. **Null comptime allocator.** `interp.zig:defaultContextValue` builds the
comptime `context.allocator` by looking up `__thunk_CAllocator_Allocator_alloc_bytes`
by name in the module's functions — but at `scanDecls` time those protocol
thunks aren't lowered yet, so `alloc_fn`/`dealloc_fn` are `.null_val` and any
comptime allocation fails. FIX (tried, works for this layer): call
`self.getOrCreateThunks("Allocator", "CAllocator")` (guarded by the same
Context/Allocator/CAllocator-registered check `emitDefaultContextGlobal` uses)
before the interp runs in `comptime.zig:runComptimeTypeFunc`.
`createProtocolThunk` saves/restores builder state, so calling it mid-lowering
is safe. After this, `alloc_fn=func_ref` — but layer 2 still bails.
2. **`struct_get` through a `*T` slot_ptr chain.** A `*List` struct receiver
(`vs.append(…)` → `append(self: *List, …)`) lands in the interp as a slot
whose contents are a slot_ptr to the actual value — `self.field` does
`struct_get` on `base=slot_ptr field_index=1` and bails. The auto-deref in
`interp.zig:.struct_get` does a single `loadSlot`; a chain-resolve loop did
NOT fix it (the final loaded value is a field-pointer aggregate that
`resolveFieldLoad` turns back into a slot_ptr — List's comptime representation
uses field-pointers + slot_ptrs the struct_get path doesn't fully resolve).
This is the deep part: comptime pointer/struct/slot resolution for `*T`
receivers, its own focused effort. Both speculative fixes were REVERTED (no
end-to-end testable win without layer 2).
The metatype surface (declare/define/type_info/field_type + make_enum) is
feature-complete for the locked design; generic type-fn body locals now work too.
- ~~**Validation + loud diagnostics**~~ — COMPLETE. duplicate variant names
(`examples/1180`); `declare()` never `define()`d (`examples/1181`, was a
`verifySizes` panic); by-value self-reference for both source (`1178`) and
CONSTRUCTED (`1182`) types via `checkInfiniteSize`. **use-before-define needs no
new check** — it's subsumed by the existing guards: a by-value cycle →
`checkInfiniteSize` ("infinitely sized"); an unfinished slot → declare-never-
defined; a bad/non-Type payload → a 0140 clean bail; a forward reference resolves
correctly via in-place slot mutation (`updatePreservingKey`); a `*Name` pointer
needs no layout. Probes `.sx-tmp/probe_ubd{1..4}.sx` confirmed: no remaining
crash or silent-corruption, only clean diagnostics / correct results.
### make_enum follow-ups (deferred capability gaps — NOT crashes; clean diagnostics)
`make_enum` itself is DONE (see Last completed step). Remaining adjacent
capabilities would let the variant list be built more freely; both error cleanly
(post-0140) rather than crash, so they're enhancements, not blockers:
- ~~Comptime slice over a non-string aggregate~~ — DONE. `arr[lo..hi]` over a
`[]EnumVariant` array now yields a real slice value at comptime (was: bailed,
string-only). Fix threaded `base_ty` onto the `Subslice` op so the interp tells
an array from a `{ptr,len}` slice, folded open-ended `hi` to a fixed array's
static length at lower time (no runtime/.ir change), and added
`interp.zig:subsliceElements`. `examples/0621` locks it.
- **Comptime `List` growth** (issue 0141). `List(T).append` at comptime bails
("struct_get: base has no fields"). Investigated — two layers (null comptime
allocator at scanDecls + `struct_get` through a `*T` slot_ptr chain); see the
detailed writeup under "Next step" and `issues/0141-*.md`. Layer 1 has a known
fix; layer 2 is deep. Probe `.sx-tmp/probe_makeenum.sx`.
- ~~Generic type-fn body locals~~ — DONE. A generic `($T) -> Type` now
comptime-evaluates its FULL body (prelude statements + return), so a local
before the return resolves. `createComptimeFunctionWithPrelude` +
`evalComptimeTypeBody`; no-prelude bodies stay on the old path. `examples/0624`.
## Known issues
- issue 0141 (OPEN, deferred enhancement — not a blocker) — `List(T).append` at
comptime bails in a type-construction `::` (two layers: null comptime allocator
+ `*T` slot_ptr `struct_get`). Workaround: array-literal locals
(`examples/0620`/`0624`). Full writeup + investigation prompt in
`issues/0141-*.md`.
- issue 0140 — comptime type-construction bail panicked instead of diagnosing —
RESOLVED. `evalComptimeType` now clears `last_bail_detail` before the interp
call and, on the `catch`, emits a build-gating `.err` at the construction span
("comptime type construction failed: {detail}") before returning the
`.unresolved` poison — so the reason is shown and no unresolved type reaches
emission unannounced. `examples/1179` locks it.
- issue 0139 — by-value self-reference segfault — RESOLVED (`checkInfiniteSize`
Pass 1g emits a loud "infinitely sized" diagnostic + breaks the cycle;
`examples/1178` locks it).
## Log
- **Generic type-fn body locals.** A generic `($T) -> Type` comptime-evaluated
only its return EXPRESSION, so a local before the return was unresolved. Now a
body with a prelude (statements before the return) has its FULL body evaluated:
`createComptimeFunctionWithPrelude` lowers the pre-return statements into the
comptime function's scope, then the return expr. No-prelude bodies (RecvResult
etc.) stay on the old path → zero regression. `examples/0624`. Suite green (685).
- **Tuple widening done — reflect/construct triad complete.** `TypeInfo` gained
`` `tuple(TupleInfo) `` (positional `[]Type`); `reflectTypeInfo` reflects a
`.tuple` (bare type_tags, tag 2), `defineType` dispatches tag 2 → `defineTuple`
(completes the slot as a structural tuple via `replaceKeyedInfo`), and the
lower-time `type_info` guard admits `.tuple`. `examples/0623`. Suite green (684).
enum/struct/tuple all reflect + construct + round-trip.
- **Struct widening done.** `TypeInfo` gained `` `struct(StructInfo) ``; `define`
dispatches on the tag (`defineType` → `defineEnum`/`defineStruct`), `reflectTypeInfo`
reflects a `@"struct"`, and the lower-time `type_info` guard admits structs.
`defineStruct` uses `replaceKeyedInfo` (kind change: tagged_union declare slot →
struct). `examples/0622` (programmatic build + source round-trip). Suite green
(683). Tuple is the last remaining shape.
- **Validation story COMPLETE.** use-before-define needs no new check — subsumed
by `checkInfiniteSize` (by-value cycles), declare-never-defined (unfinished
slots), 0140 bails (bad payloads), and in-place slot mutation (forward refs);
`*Name` pointer use needs no layout. Probed `.sx-tmp/probe_ubd{1..4}.sx`: all
clean diagnostics / correct results, no crash. `examples/1182` locks the
by-value self-ref rejection for CONSTRUCTED enums (companion to source `1178`).
- **declare()-never-defined validation.** A bare `declare("X")` with no `define`
left a zero-field nominal slot that panicked at codegen (`verifySizes`).
`evalComptimeType` now detects a zero-variant `tagged_union` result and emits a
clean diagnostic naming the type. Self-reference (declared slot completed by
`define`) is unaffected. `examples/1181` locks it. Suite green (681).
- **Duplicate variant-name validation.** Two same-named variants in a constructed
enum used to silently succeed (ambiguous construction/match). `defineEnum` now
bails naming the duplicate; `evalComptimeType` renders it (post-0140).
`examples/1180` locks it. Suite green (680).
- **Comptime subslice over non-string aggregates.** `arr[lo..hi]` at comptime
used to bail (interp `.subslice` was string-only) and the open-ended `hi` came
from a `.length` op that misread a 2-elem array as a `{ptr,len}` fat pointer.
Fix (interp-only; runtime already correct via `LLVMTypeOf`): thread `base_ty`
onto the `Subslice` op, fold open-ended `hi` to a fixed array's static length at
lower time (no IR/.ir change), add `subsliceElements`. `examples/0621` mints an
enum from `dirs[0..2]`. Suite green (679).
- **`make_enum` done.** General enum constructor `make_enum(name, variants:
[]EnumVariant) -> Type` in `meta.sx` (pure sx over declare/define). A non-generic
builder assembles the variant list in a local, then mints from it —
`examples/0620` exercises `define`'s value-arg SLICE decode. No compiler change.
Suite green (678). Deferred free-form gaps (subslice/List at comptime,
generic-type-fn locals) noted under Next step — all clean diagnostics now, not
crashes (post-0140), so enhancements rather than blockers.
- **issue 0140 fixed.** A comptime type-construction bail (`declare`/`define`/
reflection) used to panic at LLVM emission ("unresolved type reached LLVM
emission") or hide behind a cascade — `evalComptimeType` swallowed the interp's
`last_bail_detail`. Now it clears the detail before the call and renders a
build-gating `.err` at the construction span on the `catch`. `examples/1179`
locks the empty-variants case. Suite green (677). Unblocks make_enum (its
computed-slice decode failures now surface cleanly).
- **`type_info($T)` reflection done (enum round-trip).** New `BuiltinId.type_info`;
`lower/call.zig` resolves `$T`, rejects non-enum loudly, emits the builtin;
`interp.zig:reflectTypeInfo` constructs the exact nested-aggregate Value
`defineEnum` decodes (variant `{name,payload}` / slice `{data,len}` / EnumInfo /
TypeInfo `.enum`). `tagged_union` reflects `field.ty`; payloadless `` `enum ``
reflects `void`. Round-trips source AND constructed enums. Enum-only;
struct/tuple widening deferred. `examples/0619` locks it. Suite green (676).
- **By-value self-reference rejected (issue 0139, F5 partial).** New
`checkInfiniteSize` pass (Pass 1g) detects by-VALUE containment cycles (source +
comptime types, direct + mutual), emits a loud "infinitely sized" diagnostic,
and breaks the cycle (was a `typeSizeBytes` stack-overflow segfault). `*Self`
(pointer) stays valid. `examples/1178` locks the message. Suite green (675).
- **Self-reference done.** `declare(name)` + `preregisterForwardTypes` (forward
type + alias before body lowers) → `*Name` resolves; recursive `*List` enum
constructs, matches through the pointer, and traverses recursively. `0618` locks
it. `declare` gained its `name` arg; `EnumInfo.name` dropped. Suite green (674).
- **declare/define floor established.** The comptime type-construction surface is
two primitives (`declare`/`define`); all named constructors are sx. A `::` binding
or type-fn body that calls a `Type`-returning fn is comptime-evaluated (the
builtins mint the type) — no syntactic constructor recognition in the compiler.
Examples 0614 (one-shot) / 0615 (type-fn identity) / 0617 (channel results) on the
floor; `field_type` reflection (0616) unchanged.
- **Stream carved (earlier).** Selected as the first async-first foundation: gates
channel result types (`RecvResult($T)`) and `race`'s synthesized union, fully
validated, self-contained, testable in isolation (`06xx` comptime).

View File

@@ -1,6 +1,6 @@
# sx Inline Assembly — Implementation Plan (ASM stream) # sx Inline Assembly — Implementation Plan (ASM stream)
**Design source of truth:** [docs/inline-asm-design.md](../docs/inline-asm-design.md). **Design source of truth:** [design/inline-asm-design.md](../design/inline-asm-design.md).
This plan turns that doc's §II.7 stage-map + §II.8 phasing into ordered, This plan turns that doc's §II.7 stage-map + §II.8 phasing into ordered,
commit-sized, testable steps. Read the design doc first — this file is the commit-sized, testable steps. Read the design doc first — this file is the
*how/when*, not the *what/why*. *how/when*, not the *what/why*.
@@ -22,8 +22,93 @@ outputs return a tuple; templates are pure AT&T (via LLVM).
## Cadence (IMPASSIBLE) ## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass. Each feature step is either a No commit may both add a test AND make it pass. Each feature step is either a
behavior-locking PASSING test, or an xfail test the *next* commit turns green. behavior-locking PASSING test, or an xfail test the *next* commit turns green.
Arch-pinned tests live in `examples/16xx-platform-asm-*` (must declare `target=`). Arch-pinned tests live in `examples/16xx-platform-asm-*` and declare their target
Never regenerate snapshots while red. via the `expected/<name>.target` sidecar marker (Phase 0). Never regenerate
snapshots while red.
## Phase 0 — corpus target-gating (test-infra prerequisite; no compiler code)
**Why first.** The flagship v1 examples are `x86_64` (syscall-write, divmod,
cpuid) but the dev host is `aarch64`-Darwin, and the corpus runner
([src/corpus_run.test.zig](../src/corpus_run.test.zig)) currently (a) never threads
a per-example `--target` and (b) has no host-arch gate — its only skip is "marker
has no `.sx`". So D.0's `…-syscall-write` markers asserting exit/stdout describe
output the harness *cannot* produce on this host, which would violate the cadence
rule (the "next commit turns it green" can never happen). Phase 0 closes that gap.
It touches **only the runner + two fixtures** — zero compiler code, zero risk to
AE, and unblocks every arch-pinned asm example.
**Marker taxonomy (the cleanup).** The runner currently spreads per-example
*directives* across standalone boolean/value sidecars (`.aot` now, `.target`
proposed, more later). Replace that sprawl with **one optional config file,
`expected/<name>.build`**, holding all build/run directives; the output snapshots
(`.exit` / `.stdout` / `.stderr` / `.ir`) stay separate — they are
machine-regenerated data, not config. `.exit` remains the **test-discovery key**
(every test has one; `.build` is optional).
**`.build` format** — JSON, parsed with `std.json`:
```json
{ "aot": true, "target": "x86_64-linux" }
```
Parse via `std.json.parseFromSlice(BuildConfig, …)` into
`struct { aot: bool = false, target: ?[]const u8 = null }`. Field defaults cover
omitted keys; `std.json`'s default `ignore_unknown_fields = false` makes an
**unknown key a loud `error.UnknownField`** (surfaced as a runner failure, never a
silent ignore — CLAUDE.md no-silent-default rule). Extensible: future `"cpu"`,
`"link"`, `"cwd"` are just new optional struct fields, no new sidecar file and no
custom parser.
**What the directives do:**
1. **`target = <triple|shorthand>`** threads `--target <value>` into every `sx`
invocation for that example (`run` / `build` / `ir``--target` is a global
flag, confirmed [main.zig:39](../src/main.zig#L39)), AND **host-match selects
the mode.** The runner parses the leading `arch` + `os` tokens of the resolved
triple and compares them to `@import("builtin").target` (normalizing
`arm64``aarch64`):
- **match** → *execute* exactly as today (`sx run`, or `aot` build+exec) with
the target threaded, plus the `.ir` diff if an `.ir` snapshot exists. ⇒ an
x86_64 example gives **real end-to-end coverage on an x86_64 CI runner**.
- **mismatch** → **ir-only**: run *only* `sx ir <file> --target <t>`; assert
`.exit` (the ir command's exit), `.ir` (normalized stdout), and `.stderr`
(diagnostics, normally empty). Do **not** run/build/exec; do **not** assert
`.stdout`. An `.ir` snapshot is **required** in ir-only mode — its absence is
a loud runner failure ("arch-pinned <name>: ir-only mode requires an .ir
snapshot"), never a silent pass. Robust even if `sx ir` treats `--target` as
a partial no-op: the `inline_asm` op carries the template + constraint string
verbatim, so the IR snapshot still locks the exact thing §II.11 flags as
silently-miscompiling (the constraint assembler + template rewrite).
2. **`aot`** is the existing JIT-vs-build+exec switch, just relocated from the
standalone `.aot` marker into `.build`.
**Negative compile-error examples need NO `.build`.** `…-missing-volatile`
(no-output-without-`volatile`) is a Sema diagnostic raised before codegen/JIT, so
plain `sx run` reports it identically on any host — it stays a normal example with
no config file.
**update-goldens interaction:** in ir-only mode, `-Dupdate-goldens` writes `.exit`
(ir exit) + `.ir` (+ `.stderr` if non-empty) and skips `.stdout`. Execute mode
(incl. `aot`) is unchanged. `.build` is hand-authored — update-goldens never
writes it.
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | Add `BuildConfig` + `std.json` parse of `expected/<name>.build` (unknown-key ⇒ `error.UnknownField`); **migrate** the 2 existing `.aot` markers → `.build` (content `{ "aot": true }`) and delete them; thread `target`'s `--target` into the spawned argv; add `hostMatchesTarget(value) bool` (arch+os token parse, `arm64``aarch64`) gating the **execute** path. Lock with `examples/16xx-platform-target-host.sx` (trivial `main`) + a `.build` `{ "target": "<host arch triple>" }` (still runs+passes) and unit `test`s for the JSON parse + `hostMatchesTarget`. | `src/corpus_run.test.zig`, `examples/expected/1226-*.{aot→build}`, `…/1227-*`, + fixture |
| 0.1 | lock | Implement the **mismatch ⇒ ir-only** branch (skip run/build/exec; assert `.exit`+`.ir`+`.stderr` from `sx ir --target`; require `.ir`). Lock with `examples/16xx-platform-target-cross.sx` (asm-free `() -> i64 { return 0; }`) + `.build` `{ "target": "x86_64-linux" }` + a checked-in `.ir` snapshot — exercises ir-only on the arm64 host. | `src/corpus_run.test.zig` + fixture |
| 0.2 | docs | Update CLAUDE.md §"Test layout"/§"Testing" to document `.build` (format + `aot`/`target` keys) replacing the standalone `.aot` marker prose (lines ~435, ~492). | `CLAUDE.md` |
Both 0.0 and 0.1 are **lock** commits: the runner change and the fixture that
exercises it land together and pass the moment they land (the mechanism works
immediately — nothing is left red), which is the cadence rule's "lock in current
behavior" flavor, not a feature red→green. No asm lowering is gated on either.
**Phase 0 verification:** `zig build test` green; deliberately corrupt the
cross-target `.ir` fixture and confirm the runner reports an IR mismatch (proves
ir-only actually asserts, isn't a no-op); delete it and confirm the
"requires an .ir snapshot" failure fires.
**Estimated runner delta:** ~7090 lines (sidecar read + `--target` argv threading
+ `hostMatchesTarget` + the ir-only branch + update-mode tweak). Within the
"no step > ~500 new lines" rule; well under the read budget.
## Phase A — keyword + AST + parser (parses; no codegen) ## Phase A — keyword + AST + parser (parses; no codegen)
| Step | Commit | What | Files | | Step | Commit | What | Files |

225
current/PLAN-ATOMICS.md Normal file
View File

@@ -0,0 +1,225 @@
# PLAN-ATOMICS — Stream A (atomics lowering)
> **STATUS: ✅ COMPLETE (feature-complete).** All phases A.0 → A.3 landed + green.
> Surface shipped: `Atomic($T)` `load`/`store`/`fetch_add`/`sub`/`and`/`or`/`xor`/`min`/`max`/
> `swap`/`compare_exchange`/`compare_exchange_weak` (all comptime `$o: Ordering`) + free
> `fence(.ordering)`. IR ops `atomic_load`/`store`/`rmw`/`cmpxchg`/`fence`. Both LLVM emit
> AND the comptime VM implemented (verified to agree). Enabled by net-new comptime value
> params (enum/tagged_union/generic-struct methods — 3 commits). Corpus `17xx` (1700-1704) +
> `11xx` diagnostics (1130/1131/1186/1187). Commits: 22af404, 64c7db5, 8144a88, acf3183,
> 718f27e, 0531164, 68ed732, dca396e, 79895be, fca4304, b65544a (+ comptime-param 3c4305f,
> d7a6857, d95ba0a). **Unblocks Stream B2 channels + Stream C parallel schedulers.**
>
> Deferred (documented, NOT legacy — intentional scope): RMW/CAS/swap are integer-only
> (float fadd / pointer atomics out of scope); fence/orderings explicit (no defaults — the
> comptime-default-on-generic-method gap is orthogonal). Asm-level arch divergence +
> weak-memory *semantics* remain OUT of corpus scope (Stream-C stress harness).
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream A + the design-of-record
[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §3 (N1)
+ §4.6 (locked surface). Progress in [CHECKPOINT-ATOMICS.md](CHECKPOINT-ATOMICS.md).
**Goal:** net-new LLVM atomic codegen. Surface = a pure-sx `Atomic($T)` generic struct +
an `Ordering` enum (ordinary sx), with the actual atomic operations recognized as
`#builtin` intrinsics at lower-time and emitted as new IR ops. This is **100% net-new**
no atomics scaffolding exists (the only `lower.zig` "ordering" is *comparison* ordering
`< <= >=`, unrelated to memory ordering — do not mistake it for groundwork).
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `17xx` atomics.
---
## Design (grounded against the tree)
### Representation — minimal compiler surface
- **`Ordering`** is an ordinary sx enum, zero compiler coupling:
```sx
Ordering :: enum { relaxed; acquire; release; acq_rel; seq_cst; } // tags 0..4
```
- **`Atomic($T)`** is an ordinary sx **generic struct** (mirrors `List :: struct ($T: Type)`
at [list.sx:5](../library/modules/std/list.sx#L5)), a transparent 1-field wrapper —
atomicity is a property of the *operation*, not the storage, so `Atomic(i64)` has the
exact layout/size/align of `i64`. NO new IR *type*, NO type-system coupling:
```sx
Atomic :: struct ($T: Type) {
value: T;
init :: (v: T) -> Atomic(T) { return .{ value = v }; }
load :: (self: *Atomic(T), o: Ordering) -> T { return atomic_load(T, @self.value, o); }
store :: (self: *Atomic(T), v: T, o: Ordering) { atomic_store(T, @self.value, v, o); }
}
```
- The **operations** are `#builtin` intrinsic free functions, recognized by name at
lower-time (the established pattern — `size_of`/`type_info` in
[`tryLowerReflectionCall`](../src/ir/lower/call.zig#L1672), recognized BEFORE arg lowering):
```sx
atomic_load :: ($T: Type, ptr: *T, o: Ordering) -> T #builtin;
atomic_store :: ($T: Type, ptr: *T, v: T, o: Ordering) #builtin;
```
Explicit `$T` first arg follows the `size_of($T)` / `field_name($T, idx)` mixed
type+value precedent (lowest-risk; the reflection path already resolves type args).
### Ordering is compile-time-only by construction — and that forces a capability gap
LLVM atomic ordering is an **instruction attribute**, not a runtime operand, so the
ordering MUST be known at emit time. The lower-time handler reads the ordering arg's
variant name statically (it must be a **constant enum literal** `.seq_cst`) and bakes it
into the IR op as a Zig enum field (`AtomicOrdering`). A non-literal ordering is a **loud
diagnostic**, never a silent default (REJECTED-PATTERNS).
**Discovered gap (grounded):** a generic `Atomic(T)` method `load(self, o: Ordering)` would
forward `o` — a *runtime parameter* — to the intrinsic, where it is NOT a literal. And
**comptime enum value params don't exist** (`$o: Ordering` → `o` is "unresolved" in the
body; `resolveValueParamArg` folds integer constraints only). A runtime dispatch hack
(`if o == { case .acquire: atomic_load(…, .acquire) … }`) also fails: `load` with a
`release`/`acq_rel` ordering is *invalid LLVM*, so the arms can't be uniform. Therefore the
**full ordering surface is blocked on a net-new capability** (comptime-constant ordering
propagation — either comptime enum value params, or compiler-recognized `Atomic` method
calls). That capability is its **own step (A.0.5)**, sequenced before ordering-bearing ops.
### sx tag → LLVM ordering is EXPLICIT (non-contiguous!)
LLVM's `LLVMAtomicOrdering` is **not** 0..4: `Monotonic=2, Acquire=4, Release=5,
AcquireRelease=6, SequentiallyConsistent=7` ([Core.h:338-354]). The sx `Ordering` tags
(relaxed=0…seq_cst=4) map via an explicit `switch`, never an identity cast:
`relaxed→Monotonic, acquire→Acquire, release→Release, acq_rel→AcquireRelease,
seq_cst→SequentiallyConsistent`.
### LLVM-C API (verified present in `llvm-c/Core.h`, no new extern decls needed)
- Atomic load = `LLVMBuildLoad2` + `LLVMSetOrdering(v, ord)` + `LLVMSetAlignment(v, size)`
(**alignment is mandatory** on atomic load/store — LLVM verifier rejects atomics without
it). There is **no** `LLVMBuildAtomicLoad`/`Store` (the Explore agent was wrong).
- Atomic store = `LLVMBuildStore` + `LLVMSetOrdering` + `LLVMSetAlignment`.
- (Later) `LLVMBuildAtomicRMW(B, op, ptr, val, ord, singleThread)`,
`LLVMBuildAtomicCmpXchg(B, ptr, cmp, new, succOrd, failOrd, singleThread)`,
`LLVMBuildFence(B, ord, singleThread, name)`, `LLVMSetWeak`.
- `singleThread = 0` (multi-thread / cross-thread ordering). Atomic-eligible `T` =
integer / pointer / float of size 1·2·4·8(·16). **Reject non-scalar / bad-size `T`
loudly** (diagnostic), do not silently emit.
### Comptime VM treats atomics as ordinary load/store
Comptime is single-threaded, so seq_cst is trivially satisfied — the
[`comptime_vm`](../src/ir/comptime_vm.zig#L659) arms for `atomic_load`/`atomic_store`
reuse the ordinary `load`/`store` paths (correct, NOT a bail). `sx run` JITs via LLVM so
runtime atomics execute the real ops; the VM arm only matters for `#run`/const-init.
### Files the new IR op variants force (exhaustive switches)
`atomic_load` / `atomic_store` variants must be handled in every `Op` switch or the Zig
build fails (this is the desired tripwire):
- [inst.zig:159](../src/ir/inst.zig#L159) — add `atomic_load: AtomicLoad`, `atomic_store: AtomicStore` + the structs (mirror `Store` at [inst.zig:286](../src/ir/inst.zig#L286)).
- [lower/call.zig:1672](../src/ir/lower/call.zig#L1672) — recognize the intrinsics, emit the ops (new `tryLowerAtomicIntrinsic`, called alongside `tryLowerReflectionCall` at [call.zig:80](../src/ir/lower/call.zig#L80)).
- [print.zig:231](../src/ir/print.zig#L231) — print arms (sx-IR / `ir-dump`).
- [emit_llvm.zig:1566](../src/ir/emit_llvm.zig#L1566) — dispatch arms → ops.zig.
- [backend/llvm/ops.zig:325](../src/backend/llvm/ops.zig#L325) — `emitAtomicLoad`/`emitAtomicStore` (mirror `emitLoad`/`emitStore`).
- [comptime_vm.zig:659](../src/ir/comptime_vm.zig#L659) — arms reusing load/store.
- Any other `.op` switch the Zig compiler flags (module.zig / program_index.zig) — let the build tell you.
### Test snapshots — the arch-`.ir` requirement is a MISCONCEPTION for atomics
`sx ir` = [`emitIR`](../src/main.zig#L210), which emits **LLVM IR** (respects `--target`);
`sx ir-dump` is the sx-IR printer. At the **LLVM-IR level**, `load atomic i64, ptr %x
seq_cst, align 8` is **arch-invariant** — identical text for x86_64 and aarch64. The
x86-`lock`/MOV vs aarch64-`ldar`/`stlr` divergence happens only at *instruction selection*
(`sx asm`), which the corpus does **not** snapshot. So:
- **A single host `.ir` snapshot** proves the achievable gate (the `load atomic <ordering>`
keyword + correct ordering + alignment emitted). PLAN-POST §A / design §10.3's
"arch-gated x86_64 + aarch64 `.ir`" would capture **byte-identical** files — drop it.
- Optionally add ONE cross-arch ir-only example (`.build {"target":"x86_64-linux"}` on an
aarch64 host) purely as a **cross-target-emission-doesn't-crash** smoke — note in its
header that the IR body is identical to host.
- **State loudly (out of snapshot scope, parallel to the ordering-semantics caveat):**
asm-level arch lowering AND weak-memory ordering *semantics* are NOT proven by `.ir`;
those need the Stream-C stress harness, not the corpus.
---
## Phases
### A.0 — `Atomic($T)` + `Ordering` + **`seq_cst`-only** `load`/`store` ← START HERE
**Scope (descoped per the discovered gap above):** ship the net-new atomic load/store
codegen with a **`seq_cst` literal baked in the method bodies** — `load(self) -> T` /
`store(self, v)` (NO ordering param yet). The intrinsic still carries the full
`AtomicOrdering` field (always `.seq_cst` here); the recognizer + emit handle all five
orderings already, so A.0.5 only has to plumb the *constant* through. Explicit orderings
(`a.load(.acquire)`) land in A.0.5. seq_cst-only is correct (conservative-strongest), not a
silent fallback.
Two-commit cadence (lock-to-bail → green):
- **A.0a (lock)** — land the lib + IR plumbing with emit deliberately bailing:
1. New `library/modules/std/atomic.sx`: `Ordering` enum, `Atomic($T)` struct (value +
`init`/`load`/`store`), `atomic_load`/`atomic_store` `#builtin` decls. **Opt-in import
(`#import "modules/std/atomic.sx"`), NOT carried by the universal `std.sx` facade** —
mirrors `trace`. Rationale (grounded): adding the concrete `Ordering` enum to the
universal prelude registers it into EVERY program's global type table, growing
`@__sx_type_is_unsigned` (378→380) and shifting all string-global numbering → churned
37 unrelated `.ir` snapshots + bloats every binary. Atomics is a deliberate concurrency
capability, so consumers import it explicitly.
2. Add IR ops `atomic_load`/`atomic_store` + `AtomicOrdering` + the two op structs
(inst.zig); print arms; comptime_vm arms (reuse load/store); lower recognition
(`tryLowerAtomicIntrinsic`) incl. the const-ordering-literal guard + non-scalar-`T`
reject.
3. emit_llvm/ops.zig arms **bail loudly** for now: `emitAtomicLoad`/`Store` call the
emitter's bail-with-diagnostic path ("atomic load/store LLVM emission not yet
implemented") so the Zig build is exhaustive but the example is red-by-diagnostic.
4. Add `examples/1700-atomics-load-store.sx` (construct `Atomic(i64).init`, `store`,
`load`, `print`). Seed marker; capture snapshot = the emit-bail diagnostic (nonzero
exit). `zig build && zig build test` green (matches the locked bail snapshot). Commit.
- **A.0b (green)** — replace the emit bail with real emission:
`LLVMBuildLoad2`+`LLVMSetOrdering`+`LLVMSetAlignment` / `LLVMBuildStore`+`LLVMSetOrdering`
+`LLVMSetAlignment`, ordering via the explicit sx-tag→LLVM `switch`. Regen `1700` to
success output + capture its host `.ir` (asserts `load atomic`/`store atomic` + ordering).
Add a unit test in `emit_llvm.test.zig` (correct op + ordering + alignment emission).
Review the diff (no stray error text). Commit.
### A.0.5 — comptime-constant ordering propagation (the capability gap)
Enable `a.load(.acquire)` etc. — i.e. an `Ordering` that reaches the intrinsic as a
compile-time constant through a method. Two candidate designs (pick at pickup):
- **(a) comptime enum value params** — make `$o: Ordering` resolve in the body to its
variant tag (extend `comptime_value_bindings`/the typer beyond integers). General,
reusable; larger typer change.
- **(b) compiler-recognized `Atomic` methods** — special-case `Atomic(T).load/store/…`
calls (read the literal ordering arg at the method call site), bounded coupling to the
std `Atomic` type (cf. how `Vector` is special-cased). Smaller; less general.
Also enforce per-op ordering validity (load: relaxed/acquire/seq_cst; store:
relaxed/release/seq_cst; CAS's dual orderings) as **compile errors**, which is exactly what
the constant-ordering path buys. Retrofit the ordering param onto `load`/`store` here.
### A.1 — RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`)
One IR op `atomic_rmw` carrying an `RmwKind` (maps to `LLVMAtomicRMWBinOp*`). Signed vs
unsigned min/max picks `Max/Min` vs `UMax/UMin` from `T`'s signedness. Same lock→green
cadence; `17xx` examples.
### A.2 — `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**)
`atomic_cmpxchg` op (ptr, cmp, new, success_ord, failure_ord, weak). LLVM `cmpxchg`
returns `{T, i1}`; lower to `?T` where **null = success** (extract the i1, invert).
**Validate the two orderings in the compiler** (design §4.6): failure ordering may not be
`release`/`acq_rel` nor stronger than success — loud diagnostic. `_weak` sets `LLVMSetWeak`.
### A.3 — `swap` + `fence(.ordering)`
`swap` = `atomic_rmw` with `Xchg` kind (folds into A.1's op). `fence` = a new `atomic_fence`
op (ordering only) → `LLVMBuildFence`. `17xx` examples.
---
## Gates (per the corrected snapshot story)
- **unit** `emit_llvm.test.zig`: each op emits the right LLVM builder + ordering + alignment.
- **corpus** `17xx` single-thread deterministic (`sx run`, JIT executes real atomics).
- **host `.ir`** snapshot per op proves the keyword/ordering/alignment lowered.
- **OUT of snapshot scope, stated loudly:** asm-level arch divergence (`sx asm`) and
weak-memory ordering *semantics* — Stream-C stress harness territory, not the corpus.
## Kickoff prompt (A.0a — paste into a fresh session)
> Implement Stream A step A.0a (atomics lock commit) per `current/PLAN-ATOMICS.md`. Verify
> `zig build && zig build test` is green first. Then: (1) create
> `library/modules/std/atomic.sx` with the `Ordering` enum, `Atomic($T)` struct, and
> `atomic_load`/`atomic_store` `#builtin` decls; wire into `library/modules/std.sx`'s tail.
> (2) Add the `atomic_load`/`atomic_store` IR ops + `AtomicOrdering` + op structs in
> `src/ir/inst.zig`; handle them in every exhaustive `Op` switch the Zig build flags
> (print.zig, comptime_vm.zig reuse load/store, emit_llvm dispatch). (3) Add
> `tryLowerAtomicIntrinsic` in `src/ir/lower/call.zig` (recognize the two builtins, bake the
> const ordering literal into the op, loud-reject non-literal ordering AND non-scalar/bad-size
> `T`). (4) Make `emitAtomicLoad`/`emitAtomicStore` in `src/backend/llvm/ops.zig` BAIL loudly
> ("not yet implemented") this commit. (5) Add `examples/1700-atomics-load-store.sx`, seed the
> marker, capture the bail diagnostic as the locked snapshot, confirm `zig build test` green,
> commit. STOP — A.0b (real emission) is the next step. Do NOT implement emission in the same
> commit that adds the example.

741
current/PLAN-COMPILER-VM.md Normal file
View File

@@ -0,0 +1,741 @@
# PLAN — Comptime Bytecode VM + comptime memory (then re-home the compiler-API on it)
> **Direction change (2026-06-17).** The comptime compiler-API stream pivots off the
> **byte-weld**. The weld (sx structs whose layout is validated to mirror the
> compiler's Zig types) + the **serialization / marshaling** bridge at the call
> boundary is the wrong direction — it bolts a parallel layout regime and hand-built
> byte-copies onto a comptime value model that fundamentally isn't bytes. We strip it
> and build the right foundation: a **bytecode VM over byte-addressable
> memory**, where comptime values ARE native bytes (like runtime). On that base the
> compiler-API needs no weld, no validation, no marshaling — the compiler's own types
> are read/built directly as memory and its functions take/return real pointers.
>
> Supersedes the build order in `design/comptime-compiler-api.md` (kept for history).
> This is the active plan for the stream. Branch: `reify`.
## Why
`src/ir/interp.zig` is a tree-walking interpreter over the SSA IR that represents
every value as a tagged `Value` union (`int`, `float`, `aggregate: []const Value`,
`type_tag`, `heap_ptr`, …). Two consequences:
1. **Slow.** Per-value boxing in a tagged union; per-op `switch` over `Inst`; an
aggregate is a heap `[]const Value`, walked element-by-element.
2. **Not native memory.** A struct value is `[]const Value` (tagged unions), NOT the
struct's bytes. So a comptime `@ptrCast(*StructInfo)` reads the `Value` union's
memory, not a `StructInfo` — which forced the whole weld+marshal detour.
Make comptime values **native bytes in byte-addressable memory** and both problems dissolve:
structs/arrays/slices are their bytes at natural layout (no weld), the compiler's own
records are directly addressable (no marshal), and a bytecode loop over comptime memory is
fast.
## End state
- Comptime execution = a **bytecode VM** over a **byte-addressable memory** (real
host-allocated bytes; layout is **target-aware** via the type table's sizes). Values
are bytes at addresses plus a scalar register file. No tagged `Value` union.
- The comptime compiler-API: the compiler **exposes its real types + functions** to
comptime sx. sx reads/builds them as native memory and calls compiler functions by
pointer. No `abi(.zig)` weld, no `validateStructLayout`, no `register_struct`
field-by-field marshaling — gone.
- `declare`/`define`/`type_info` and `#compiler`/`BuildOptions` ride this one
mechanism; the bespoke interp arms are deleted.
- **ONE evaluator at the end — non-negotiable.** The legacy tagged-`Value` interpreter
(`interp.zig`) is **DELETED**. We do NOT ship both permanently. "Dual-path"
(a compiler-API fn with both a legacy `compiler_lib` handler AND a VM-native impl) and
the emit-time legacy fallback are **transitional only** — scaffolding while the VM
reaches parity at BOTH comptime sites (emit time AND lowering time). The flag
`-Dcomptime-flat` is the swap mechanism; once the VM runs everywhere with parity, the
flag, the fallback, and `interp.zig` all go. Any "VM-only at emit, legacy at lowering"
split is a waypoint, never the destination.
## Principles (hold at every step)
- **Green at every step.** `zig build && zig build test` pass after each sub-step. The
existing tagged-`Value` interpreter stays the live evaluator until the VM reaches
corpus parity; swap behind a build flag, then delete the old path.
- **Target-aware, not host-baked.** Flat-memory layout uses the type table's target
sizes (`pointer_size`, `typeSizeBytes`/offsets), NEVER host `@sizeOf`. This is what
keeps cross-compilation correct (the JIT-comptime alternative could not).
- **Sandboxed.** Flat-memory accesses are bounds-checked; step/call-depth budgets
remain; an OOB / bad access traps to a build-gating diagnostic with a source span —
never a compiler-process crash.
- **No silent fallbacks** (per CLAUDE.md): an unhandled op / shape bails loudly with a
named reason, never a zero/default that looks like success.
## Phases
### Phase 0 — Strip the weld / serialize / marshal machinery
Delete the wrong-direction code so the VM builds on a clean base. Pure removal +
corpus rebaseline; suite green.
- `src/ir/compiler_lib.zig`: the reflection (`weldStruct` / `bound_types` /
`FieldLayout` / `BoundType`), the layout validation (`validateStructLayout` /
`LayoutMismatch` / `SxField`). Decide the fate of the `bound_fns` host-call registry
(`intern`/`text_of` handlers) — it is likely subsumed by the VM's compiler-call path
in Phase 3, but `intern`/`text_of` may survive as the first such calls.
- `src/ir/lower/nominal.zig`: `validateWeldedStruct` + `weldedFieldOrderStr` + the
`sd.abi == .zig` validation call in `registerStructDecl`.
- `src/ir/interp.zig`: the `compiler_welded` dispatch branch.
- `src/backend/llvm/ops.zig`: the `emitCall` comptime-only gate keyed on
`compiler_welded` (re-derive the comptime-only guard from a non-weld signal if still
needed).
- Corpus: retire / convert the weld examples + diagnostics — `0625`, `0627` (welded
struct), `1183`, `1186` (weld-layout diagnostics), `1184`/`1185` (welded-fn). Keep
`0626` (`intern`/`text_of` round-trip) only if it survives the new call path.
- **Keep (re-evaluate in Phase 3), independent of the weld semantics:** the
`#library "compiler"` decl, the `abi(.x)` annotation + `extern <lib>` syntax, and the
`callconv → abi` unification. These are surface syntax that may still serve the
compiler-API; only the *weld semantics* are stripped here.
**Verification:** `zig build test` green with the weld machinery gone; the surviving
syntax still parses (parser unit tests).
### Phase 1 — Flat-memory value model (still IR-walking, no bytecode yet)
Introduce comptime memory and move comptime values onto it, **decoupled from bytecode** so
the value-model change is isolated. Each sub-step ports one op group and keeps the
corpus green; the OLD tagged path stays behind a build flag (`-Dcomptime-flat`) until
all groups land, then the shim is deleted.
1. **Machine + scalars.** A comptime memory region (host `[]u8`) with a stack (frames) +
bump-allocated heap, and a scalar register file. Port `int`/`float`/`bool`/`undef`
and arithmetic/compare/branch. Aggregates still go through a compat shim to the old
representation.
2. **Aggregates.** Structs/arrays/tuples laid out in comptime memory at **target** layout;
port `struct_init` / `struct_get` / `array` / `index_gep` to read/write bytes at
computed offsets.
3. **Slices / strings.** `{ptr, len}` fat pointers in comptime memory.
4. **Optionals / enums / tagged unions.** Tag + payload bytes.
5. **Pointers.** `alloca` / `store` / `load` / GEP unified onto comptime addresses; retire
`slot_ptr` / `heap_ptr` / `byte_ptr` in favor of comptime addresses.
6. **Closures.** Fn id + captured env materialized in comptime memory.
7. **Extern / host calls.** A struct arg is already bytes → pass its address; this
removes most of `marshalExternArg`.
8. **Reflection / minting.** `declare` / `define` / `type_info` read comptime
values; type-table mutation copies escaping data into compiler-owned memory at the
boundary (lifetime), as today.
**Verification:** with `-Dcomptime-flat` the full corpus (currently 692) is byte-for-
byte identical to the tagged path; then make the VM the default and delete the shim.
### Phase 2 — Bytecode
Compile a comptime function's IR → a compact bytecode and execute the bytecode instead
of walking `Inst`. Pure encoding/speed; semantics identical to Phase 1. Land at least a
minimal register-bytecode loop (the stream's stated goal is a *bytecode* VM); a
fragment cache is optional follow-up.
**Verification:** corpus identical to Phase 1; comptime throughput measurably improved
on a heavy-comptime micro-benchmark.
### Phase 1.final — host wiring (the remaining integration)
The wiring ENTRY POINT exists: `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a
comptime function entirely on the VM and returns a legacy `Value`, or `null` to fall
back. Unit-tested (pure `6*7` → 42; unsupported → null). Remaining to actually route the
host through it:
1. **Panic→error hardening (prerequisite).** `Machine.readWord`/`writeWord`/`bytes`
currently `assert` (debug panic) on null/OOB. For arbitrary host functions to be
safe, make them return `error.OutOfBounds` so a malformed run BAILS (→ null → legacy)
instead of crashing the compiler. Ripples through `readField`/`writeField`/slice
helpers (add `try`).
2. **Implicit context.** Host comptime functions may have `has_implicit_ctx` (param 0 =
`*Context`); the legacy `run` materializes a default ctx. The VM `run` does not — so
either materialize it too, or only route `tryEval` at funcs without implicit ctx.
3. **Wire one site** behind a flag/env (`SX_COMPTIME_FLAT`, → `-Dcomptime-flat` later):
the const-init fold in `emit_llvm.zig` `emitGlobals` (`result = tryEval(...) orelse
interp.call(...)`). Default off → corpus unaffected.
4. **Parity + coverage.** Run the corpus with the flag ON; results must be byte-identical
to legacy. Measure how many comptime evals the VM already handles; the bail `detail`s
name what to port next (tagged-union payload / any / closures / builtins).
5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge)
until the VM is the default and the legacy path is deleted.
**Status (2026-06-17): steps 14 DONE; step 5 = the next session.**
- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return
`error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of
asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through
`readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and
every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a
null-deref function → `tryEval` returns `null` (legacy fallback), not a panic.
- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a
conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with
`has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the
right size/align allocated in comptime memory, its address passed as arg 0. The common
const body never reads the ctx; a body that USES the allocator loads a fn from it and
`call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed:
handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed)
empirically confirms no divergence. (A body that read+branched on a null allocator fn
could in principle diverge; none does — parity is the guard.)
- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat)
tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT`
(a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`).
Default OFF → corpus unaffected (688 green).
- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed);
manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace
facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`,
printing HANDLED / fallback+reason per init).
- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.**
Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity
stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx
materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer
optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the
destination = none/empty; unit-test regression); curated libc MEMORY builtins on comptime
memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped,
`free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed, target-aware,
result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32
storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single —
`readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had
written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added).
- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage
**27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF.
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context (not a zeroed one): the implicit-ctx param is an
opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying
the CAllocator thunk func-refs) into comptime memory via a new recursive `layoutConst`.
With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays
reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect`
(decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body
that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` →
`context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → the VM's native comptime `malloc`. Unlocked `0606` (string global via
the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func`
(memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/
`struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by
`alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to
`.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s
rejection diagnostic byte-identical). Unit tests added (global_get, func_ref +
call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk
bottoms out at libc `malloc`, and the VM can't use a host pointer with comptime
load/store, so comptime `malloc` must allocate from comptime memory. The default context
lets the allocator PROTOCOL run; native `malloc` is its final step.
- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage
**31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF.
- **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`.
- **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set)
→ word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)`
shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 |
span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push`
/ `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions —
linked into the compiler — so the return-trace buffer the host reads is populated
identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now.
- **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):**
`readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a
stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently
hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded;
gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests
pass via 1037/1038 in the corpus).
- **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):**
`intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm
global call (`1654`, never comptime-evaluable). Every other measured corpus const-init
is handled on the VM.
At this point the comptime VM handles essentially the entire real comptime corpus
(scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls +
recursion, the implicit context + allocator protocol, globals, failables + return
traces). Phase 2 (bytecode) and Phase 3 (compiler-API on comptime memory) are the forward
work; flipping the VM to default + deleting the legacy path awaits those.
- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second
comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run <expr>;`) now
routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields
`.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed
(`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy
re-run double-push them — both sites now `sx_trace_clear()` right before the legacy
fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All
comptime evaluation now routes through the VM-with-fallback (uniform).
- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM
gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init`
reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test
-Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity
gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var;
default-built does not). This is the prerequisite to eventually making the VM default +
deleting the legacy path (which still awaits Phase 2/3 + broader confidence).
- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.**
`invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is
the safety boundary) via `Vm.callCompilerFn` — natively on comptime memory, NO legacy
`Interpreter`: `intern(s: string) -> StringId` reads the string bytes from comptime memory and
`internString`s into the (const-cast) table (pool-only, never touches type layout, so the
VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back
into comptime memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback
is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688**
both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next
compiler functions (`find_type`, `register_struct`, the reflection readers) are added the
same way — comptime pointer in, handle/pointer out, no marshaling.
**Phase 3 progress (2026-06-18):**
- **(P3.1) First read-only reflection readers — `find_type` + `type_field_count` (DONE).**
Two more `compiler`-library fns bound the same way as the `intern`/`text_of` seed
(added to `compiler_lib.bound_fns` AND `Vm.callCompilerFn`, native on comptime memory, no
marshaling). A **type handle is a plain `u32` `TypeId`** (exactly like `StringId`), so
both calls keep the seed's clean scalar shape — handle in, scalar out:
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`) and
`type_field_count(t: TypeId) -> i64` (a new `TypeTable.memberCount` query — struct/union/
tagged-union fields, enum variants, array/vector length — that BOTH the legacy handler
and the VM call, so the two paths can't drift). Example `0628` chains
`intern → find_type → type_field_count` and a not-found lookup, both folded at `#run`,
both VM-HANDLED natively (no fallback). Parity **689/689** (gate ON and OFF); VM unit test
added.
- **Decision (resolves the plan's `find_type → ?Type` sketch):** `find_type` returns a
NON-optional `TypeId`, using the codebase's dedicated `unresolved` (0) sentinel for
not-found — NOT an `?Type`. Rationale: a `Type` value resolves to `.any`
(`type_resolver.zig`), which the comptime VM does not represent; and an optional
return can't cross the legacy↔VM eval boundary (`regToValue` bridges only
word/string/struct/tuple). `unresolved` is the project-blessed unmistakable "no type"
marker (see CLAUDE.md REJECTED PATTERNS — a dedicated sentinel is the required shape),
so the caller checks the handle against 0. This keeps the reader a clean scalar mirror
of `intern`/`text_of` and defers `.any`/optional plumbing to when it's actually needed.
- **(P3.2) Field-level reflection readers — `type_nominal_name` + `type_field_name` +
`type_field_type` (DONE).** Three more readers on the same `TypeId`-handle shape (each
backed by a new `TypeTable` query that BOTH the legacy handler and the VM call, so no
drift): `type_nominal_name(t: TypeId) -> StringId` (`nominalName` — a named type's own
name; loud-bail for unnamed types), `type_field_name(t: TypeId, idx: i64) -> StringId`
(`memberName` — struct/union/tagged-union field, enum variant, named-tuple element), and
`type_field_type(t: TypeId, idx: i64) -> TypeId` (`memberType` — struct/tuple/array/vector
member type). All loud-bail on out-of-range idx / no-member (no silent default). These are
the first MULTI-ARG compiler fns (the VM's `callCompilerFn` now reads arg 1 = idx); added
`Vm.argHandle`/`argTypeId` helpers (range-checked u32/TypeId arg reads). Naming uses the
`type_*` family so nothing collides with the std metatype builtins (`field_name`/`type_name`
exist in `core.sx`). Example `0629` reflects `Pair { lo: Point; hi: Point }` — reads each
field name and the nominal name of a field's type, all folded at `#run`, all VM-HANDLED
natively. Parity **690/690** (gate ON and OFF); VM unit test added.
- **(P3.2b) Kind + enum-value readers — `type_kind` + `type_field_value` (DONE).** The last
two read-only readers the metatype's `type_info(T)` needs, completing the READ side: a
comptime sx fn can now fully reflect a struct/enum/tagged-union/tuple into data with no
`#builtin`. `type_kind(t: TypeId) -> i64` (`TypeTable.kindCode` — a stable, compiler-owned
discriminant: 0 other · 1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array ·
7 vector · 8 error_set; TOTAL — never bails, an unnamed/non-aggregate type reads `other`)
and `type_field_value(t: TypeId, idx: i64) -> i64` (`TypeTable.memberValue` — an enum
variant's explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for
a non-enum / out-of-range idx). Example `0630` reflects `Color`/`WindowFlags`(flags)/`Point`.
Parity **691/691** (gate ON and OFF); VM unit test added.
- **READ side now complete:** `find_type` + `type_kind` + `type_field_count` +
`type_field_name` + `type_field_type` + `type_nominal_name` + `type_field_value` cover
everything `reflectTypeInfo` reads.
- **(P3.3) WRITE side — `declare_type` + `pointer_to` + ONE kind-branching `register_type` (DONE).**
The mutating side is a SINGLE `register_type(handle, kind, members)` that branches on `kind`
IN THE COMPILER (subsuming `define`'s `defineStruct`/`defineEnum`/`defineTuple`), plus
`declare_type(name) -> Type` (forward handle) and `pointer_to(t) -> Type` (build `*T`
references). They take/return real `Type` values (matching meta.sx's declare/define).
- **Timing decision (per the user):** mint LAZILY at LOWERING time (single pass, NOT a
pre-emit phase, NOT two-pass) — the existing `runComptimeTypeFunc` path. So the write
side is **legacy-only** (`compiler_lib` handlers); the VM isn't wired at lowering time, so
no VM mirror is needed (the read-side readers stay dual-path for emit-time reflection). A
non-generic `-> Type` builder is now flagged `is_comptime` (`decl.zig`) so its dead body
permits the welded calls (the comptime-only gate).
- **Graph support:** forward `declare_type` handles + `pointer_to` express a
mutually-recursive A↔B graph (`*A`, `*B`, B-by-value) before bodies are filled.
`register_type` is **idempotent** — re-filling a nominal slot (same module reached via two
import edges) re-mints identically instead of erroring (`nominalIdent` reads identity from
any nominal kind). `kind` codes match `type_kind`: 1 struct · 2 enum (actual `.@"enum"`) ·
3 tagged_union · 4 tuple.
- **Two bugs fixed en route** (issue 0142): (a) a fully payloadless comptime-minted enum
was minted as an all-void `tagged_union` → `verifySizes` panic; now mints a real
`.@"enum"` (both `register_type` kind 2 AND the metatype `defineEnum`). (b) bare
`EnumType.variant` qualified construction of a payloadless variant wasn't supported (failed
for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
- Examples: `0631` (graph + actual-enum + reflection), `0632` (make_enum all-void),
`0633`/`0634`/`0635` (namespaced / bare / multi-edge import of a minted type), `0187`
(qualified variant construction). Parity 697/697 (gate ON and OFF); unit tests added.
- **Next (P3.4):** re-express `declare`/`define`/`type_info` as sx over the read+write
compiler-API and DELETE the bespoke interp arms — needs the VM hardened against malformed
lowering-time IR first (the metatype runs at lowering time), so either harden + wire the VM
there, or migrate the metatype onto the legacy compiler-API calls first. Decide when reached.
Phase 2 (bytecode) is the orthogonal speed work.
### Phase 3 — Compiler-API on comptime memory (resume the stream — no weld)
With native-byte comptime values, re-home the compiler-API:
- **Expose the compiler's real types.** Register the actual `types.zig` records
(`StructInfo`, `EnumInfo`, `Field`, …) into the comptime type table under sx-visible
names, with their **real (host) layout** — the type IS the compiler's, so there is
nothing to validate or keep in sync. (This is the projection that *replaces* the
weld's reflection — owned by the compiler, not declared in sx.)
- **Expose the compiler's functions.** `register_struct`, `find_type`, `intern`,
`text_of`, and the reflection readers operate on comptime pointers / handles
directly (no marshaling — the bytes already ARE the record).
- **Re-express** `declare` / `define` / `type_info` as sx over these; delete the
bespoke interp arms (`defineStruct` / `defineEnum` / `defineTuple` / `reflectTypeInfo`);
migrate `examples/0622` (struct), `0619`/`0620`/`0623` (enum/tuple).
- **Migrate `BuildOptions`** off `#compiler` onto this mechanism; **delete `#compiler`**.
**Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over
the exposed compiler-API; full corpus green.
### Phase 4 — Retire the legacy interp (the ONE-evaluator end state)
The metatype CONSTRUCTION + REFLECTION surface is VM-native (steps 7/8 — `0614``0624`,
`0632` all HANDLED). This phase moves EVERYTHING ELSE off `interp.zig` and deletes it.
**What the legacy interp is still used for (audited 2026-06-18) — five roles:**
| Role | Wired to VM? | Site |
|------|--------------|------|
| **A. Comptime folds** (type-fn / `::` const-init / `#run`) | ✅ VM + legacy fallback | `comptime.zig:530`, `emit_llvm.zig:871`/`971` |
| **B. `#insert` string eval** | ❌ legacy-only (VM wiring reverted — 0737 malformed-IR crash) | `comptime.zig:634` |
| **C. Post-link bundler** (`platform.bundle` — Info.plist/codesign/process/fs) | ❌ legacy-only | `core.zig:invokeByFuncId` ← `main.zig:769` |
| **D. `#compiler` hooks** (`compiler_call` — BuildOptions/bundling) | ❌ legacy-only; `Value`-based ABI | `compiler_hooks.zig`, `interp.zig:1130` |
| **E. Bail diagnostics** (`Interpreter.last_bail_*` statics) | n/a | `main.zig:464` |
Shared substrate everything traffics in: the **`Value`** tagged union (the
`regToValue`/`valueToReg` bridge + the hooks + `core.zig`) and the **host-FFI bridge**
(`host_ffi.zig` + `interp.callExtern` — dlsym + cdecl trampolines for real libc).
**DECISION (2026-06-18, user): UNIFY.** The VM gains a host-FFI escape + real-pointer
translation and runs BOTH sandboxed comptime folds AND the unsandboxed post-link bundler.
`interp.zig` is fully deleted — true ONE evaluator, two modes (sandboxed / host-effects).
**Remaining comptime-fold gaps** (full corpus fallback inventory — 15 examples; 1179/1180
are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan artifact):
`box_any`/`unbox_any` (6), `out`/print (2), `global_addr` (1), trace frames (1),
`compiler_call` (2 — role D).
**Sub-phases (dependency order; each its own session, both gates 697/0 after each):**
- **4A — finish comptime ops (small, parity-guarded).** Drive the fold fallback list to
empty except `compiler_call`:
- **4A.1** `box_any`/`unbox_any`. Word case = alloc 16B `{tag@0, value@8}`, tag =
`source_type.index()` (matches legacy comptime; note runtime `anyTag` normalizes
arbitrary-width ints), value via `writeField(source_type)` (so f32 etc. round-trip);
unbox = `readField(addr+8, target)`. Aggregate-Any payload needs the runtime
pointer-in-value-slot shape (`coerceToI64` alloca+ptrtoint) — implement or bail loudly.
- **4A.2** `out`/print → add a VM output buffer; flush through the same path as
`core.flushInterpOutput`.
- **4A.3** `global_addr` (address-of a global in comptime memory).
- **4A.4** trace frames (`sx_trace_*` / `interp_print_frames`).
- **4B — VM-native diagnostics (role E). MUST land before deleting legacy.** Today a VM
bail silently falls back; with legacy gone the VM bail IS the user-facing build-gating
diagnostic. Surface the VM's `detail`/span/file into what `main.zig` renders; turn
1179/1180-style bails into proper diagnostics. No diagnostic may regress.
- **4C — `#insert` on the VM (role B).** Re-wire `evalComptimeString` through `tryEval`;
the lowering-time-IR hardening that forced the 0737 revert is already in place. Verify
the `#insert` corpus parity.
- **4D — host FFI on the VM (role D substrate). DONE.** Solved by a better allocator, not a
pin/tag scheme: the comptime memory is now an **arena** of stable host allocations and `Addr`
IS a real host pointer (`4D.0`, `625ba0f`), so a comptime pointer and an FFI-returned host
pointer are the same value — no translation, no realloc hazard. `Vm.callHostExtern`
(`4D.1`, `e7a8708`) dispatches ANY extern via `host_ffi` dlsym + trampolines (args/returns pass
untouched); `4D.2` (`6a7f690`) adds slice/string args (→ NUL-term `char*`) + float guards.
Examples 0636/0637. **(Superseded sub-note:** the earlier "pin the buffer / comptime↔host translate"
hazard is moot — the arena never moves an allocation.)
- **`#compiler` / `compiler_call` — DELETED, replaced by the `abi(.compiler)` ABI (decision 2026-06-18,
REVISED from the earlier `abi(.zig) extern compiler` shape).** A function is *compiler-domain* — it runs in
the comptime evaluator (VM/interp), NEVER in the shipped binary — because its **ABI says so**: `abi(.compiler)`.
No `extern <lib>`, no fake `#library "compiler"`. One annotation covers BOTH roles: (a) the **compiler-API
surface** (`intern`/`find_type`/`build_options`/`set_post_link_callback`/… — bodiless decls whose Zig/VM
handler is the impl, on `compiler_lib`'s export list, dispatched by `Vm.callCompilerFn`); (b) **user
compiler-domain functions** like post-link callbacks (`bundle_main` — BODIED `abi(.compiler)`, lowered for VM
eval but emit-skipped). The `#compiler` struct attribute + the `compiler_call` IR op + the `Value`-based hook
`Registry` (`compiler_hooks.zig`) all **go away**. **Why this is cleaner than the welded-fn approach:** the
former runtime-call enforcement blocker (a `build_options()` call inside an LLVM-emitted callback body) is
MOOT — a compiler-domain function is never emitted, so its compiler-API calls never reach `emitCall`.
**Staged build (each its own step, both gates green):**
- **S1+S2 — DONE (2026-06-18):** introduced `abi(.compiler)`, REMOVED the `.zig` ABI + `abi(.zig) extern
compiler` + `#library "compiler"` (clean cutover, no legacy); migrated all compiler-API examples. The
binding now keys off `fd.abi == .compiler` (`decl.zig` `weldedCompilerFn`); a bodiless `abi(.compiler)`
decl lowers extern-like (declared-not-defined) with no implicit ctx. **700/0 both gates.**
- **S3 — DONE (2026-06-18):** emit_llvm skips BODIED `abi(.compiler)` function bodies. Added an
`is_compiler_domain` flag to the IR `Function`; a bodied `abi(.compiler)` function LOWERS its body (for VM
eval) + is flagged `is_comptime` but is NOT emitted (Pass 2 skip; declared external-linkage so the empty
decl verifies). KEY fix: a call to a comptime-only callee (compiler-API `compiler_welded` OR
`is_compiler_domain`) inside a dead comptime body now emits `undef` instead of a real `call` (`ops.zig`
`emitCall`) — the old `compiler_call` did this; without it an AOT link leaves an undefined `_double`/`_intern`
reference (this also fixed a pre-existing untested AOT breakage of the bodiless compiler-API examples).
`fnIsBodilessCompiler` distinguishes the API surface (declare-only) from a compiler-domain callback (lowered,
emit-skipped). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` const,
absent from the binary, JIT+AOT). **701/0 both gates.**
- **S4 — callback-param propagation: OPTIONAL / DEFERRED (ergonomics only).** Verified 2026-06-18: an
`abi(.compiler)` function is TYPE-compatible with a plain `() -> R` param (the ABI marks the *function* —
`is_compiler_domain` — not its *type*, which stays `() -> R` CC-default). So a callback that needs to be
compiler-domain just declares itself `abi(.compiler)` (S3) and passes to a plain param fine; auto-propagation
from an `abi(.compiler)` PARAM type is a nicety, not a prerequisite for S5. Skipped for now.
- **S5a — DONE (2026-06-18):** the corpus-covered slice. `build_options` + `set_post_link_callback` →
free `abi(.compiler)` functions (VM `callCompilerFn` arms + legacy `compiler_lib` handlers); **`BuildConfig`
threaded into the VM** via a `tryEval` param (the same one `main.zig` forwards — shared with 4E). `build.sx`
extracts `set_post_link_callback` from the `struct #compiler` as a free `ufcs` fn; `bundle_main` + the
platform registrars (`configure`) are `abi(.compiler)`. 37 examples' `.ir` snapshots regen'd (benign:
declaration renumber + `@str` suffix shift — every example imports build.sx via the prelude). Strict
`compiler_call` bails 6→2; 0602/0603/1604/1611 HANDLED. **701/0 both gates.**
- **S5b/S5c (port the ~37 hooks) — SUPERSEDED 2026-06-18 by the sx-driven build pipeline (below).**
Porting each `BuildOptions` accessor to an `abi(.compiler)` function that delegates to a `compiler_hooks`
hook just re-encodes sx-level logic (string setters/getters, `is_macos` triple-matching, list appends) as
compiler hooks. The hooks need NOTHING from the compiler except the `BuildConfig` state. So instead of 37
hooks, **drive the whole build pipeline from sx** (the logical end of "bundling lives in sx"). S5a stays as
a green intermediate; the sx-build-pipeline replaces `build_options`/`set_post_link_callback`/the whole
`#compiler` surface wholesale.
### Phase 5 — sx-driven build pipeline (replaces the BuildOptions hooks; decision 2026-06-18, user)
**The build pipeline becomes an sx program.** `BuildConfig` is plain sx data (an ordinary struct, sx-owned
end-to-end — no `#compiler`, no hooks, no shared Zig state, no weld/offset access). The compiler shrinks to
a few `abi(.compiler)` PRIMITIVES that take **explicit args** (so nothing is shared by memory), and an sx
`build()` driver orchestrates configure → emit → link → bundle. **Chosen boundary: Option B** — the compiler
keeps the proven Zig linker as a primitive; sx owns config + orchestration + bundle. (Option A — sx shells
`cc`/`ld` itself — is a later refinement once the per-target link-line logic is ported to sx.)
**File split (user decision 2026-06-19):** the low-level compiler-API PRIMITIVES live in
`library/modules/compiler.sx` (the comptime `compiler` library — renamed from the interim `std/build.sx`); the
default `build` IMPLEMENTATION (`default_build` + the `on_build` slot + the sx `BuildConfig`) lives in
`library/modules/build.sx` alongside the existing `BuildOptions` DSL. So `compiler.sx` = primitives, `build.sx` =
orchestration/default impl. **Build-callback fallibility was DROPPED (user 2026-06-19):** the primitives + the
build callback are NOT `-> !` — a failed action (e.g. `link`) BAILS on the VM (hard build error). So the shapes
below shed their `-> !`.
Shape (build-callback fallibility dropped 2026-06-19):
```sx
// library/modules/compiler.sx (the comptime `compiler` library — PRIMITIVES)
emit_object :: () -> string abi(.compiler); // emitted .o path (query)
link :: (objects: List(string), output: string, libraries: List(string),
frameworks: List(string), flags: List(string), target: string) abi(.compiler); // void; bails on failure
c_object_paths :: () -> List(string) abi(.compiler); // metadata queries
link_libraries :: () -> List(string) abi(.compiler);
// library/modules/build.sx (the build DSL — DEFAULT IMPLEMENTATION + slot)
BuildConfig :: struct { output: string; target: string; flags: List(string);
frameworks: List(string); bundle_path: string; bundle_id: string; ... }
default_build :: (config: BuildConfig) abi(.compiler) { // the default pipeline (void)
obj := emit_object(); objs := c_object_paths(); objs.append(obj);
link(objs, config.output, link_libraries(), config.frameworks, config.flags, config.target);
if config.bundle_path.len > 0 { bundle_app(config); } } // bundle_app = today's sx bundler
on_build : (BuildConfig) abi(.compiler) = default_build; // the override slot
// user overrides: build :: (config: BuildConfig) abi(.compiler) { ... } #run on_build = build;
```
The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfig` → read `on_build` → invoke
`on_build(config)` on the VM; a `raise` fails the build. Plain `sx run` fires none of it.
**Steps (each its own green step; depends on 4E first):**
- **P5.1 — 4E prereq — DONE (2026-06-19).** `core.invokeByFuncId` routes the post-link callback through the
**VM** (`comptime_vm.tryEval`), NO fallback (a side-effecting callback can't double-execute): a bail is a hard
build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). `BuildConfig` +
`import_sources` threaded in; `flushInterpOutput` deleted (VM `out` writes direct via host-FFI). Smoke test
`examples/1661-platform-post-link-vm-list` (AOT): a post-link callback GROWS a `List` (0141 — works on the VM,
bails on legacy with `struct_get`), so the build succeeds (exit 0) only via the VM. Non-empty callback `args`
rejected loudly (the `on_build(config)` arg-marshaling entry is P5.3). **702/0 both gates.**
- **P5.2 — primitives.** Split: the read-only **metadata queries are DONE (2026-06-19)** — `c_object_paths() ->
List(string)` + `link_libraries() -> List(string)` as `abi(.compiler)` fns (stdlib `library/modules/compiler.sx`),
serviced by `comptime_vm.callCompilerFn` over `BuildConfig` fields `main.zig` forwards; new VM `makeStringList`
builds the `List(string)` in comptime memory from the call's result type (`ins.ty` now threaded through
`invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both
gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits
the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all
three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER
DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard
build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks`
vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM
readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test
`1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output —
the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0.
- **P5.3 — `on_build` registrar — DONE (2026-06-19).** `on_build(cb)` registers the build callback
(`cb: (opt: BuildOptions) -> bool abi(.compiler)`); the compiler force-lowers + auto-invokes the well-known
`default_pipeline` when no override. (Implemented as a registrar, not an assignable slot — the opaque
`BuildOptions` handle is one word, so arg-passing needs no struct marshaling.)
- **P5.4 core — DONE (2026-06-19).** `default_pipeline` in `build.sx` drives the whole build; NO Zig
auto-emit/auto-link; `emit_object`/`link` are sx-called actions via the `BuildHooks` vtable;
`set_post_link_callback` deleted (all callers on `on_build`). Build-path auto-imports `modules/build.sx`.
703/0 both gates.
### THE FINAL DIRECTION (user, 2026-06-19): FULL MIGRATION — NO LEGACY LEFT.
**Decision: DROP gate-OFF entirely.** The VM becomes the SOLE comptime evaluator; `-Dcomptime-flat` is made
permanent then removed; `interp.zig` (the legacy tagged-`Value` `Interpreter`) is DELETED. There is no
dual-path, no legacy `compiler_lib` handler, no `regToValue`/`valueToReg` bridge, no VM→legacy fallback. We
migrate the BuildOptions surface DIRECTLY to VM-native `abi(.compiler)` arms (no legacy handler — there is no
legacy to handle). **All bundling + code signing for EVERY target lives in the sx `default_pipeline`.**
- **P5.5 — DONE (2026-06-19).** The 35 `BuildOptions :: struct #compiler` methods migrated to VM-native
`abi(.compiler)`: `BuildOptions :: struct { }` (opaque null-sentinel handle) + 35 free
`ufcs (self: BuildOptions, …) abi(.compiler)` decls in `build.sx`, serviced by a new
`comptime_vm.callBuildOptionFn` arm off `callCompilerFn` — **NO legacy `compiler_lib` handler** (names
registered in `bound_fns` with a single bailing stub only so `weldedCompilerFn` accepts them). Setters dupe the
arg string into the PERSISTENT `Vm.gpa` (the Compilation allocator — threaded into both `tryEval` and
`runBuildCallback` — NOT the per-eval VM arena) and write/append to the threaded `BuildConfig`; string getters
return the field (or `""`); bool getters compute from the triple (`predIsMacOS`/…); count/index getters read the
`BuildConfig` slices. **Dispatch routing (Option B):** a `#run`/const-init entry that directly calls a
compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with NO legacy fallback regardless of the
`-Dcomptime-flat` gate → gate-OFF stays green without a legacy BuildOptions handler. 5 `platform/bundle.sx`
getter-calling helpers marked `abi(.compiler)` (comptime-only bundler code). 37 `.ir` regenerated (string-pool
churn; behavior-identical, verified `.ir`-only). **703/0 BOTH gates.** BuildOptions `compiler_call` bails GONE
(1609/1614/1615 strict-clean); 1616 now bails on `shr` — a SEPARATE unported bitwise/shift VM gap
(`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`), to port FIRST in P5.6 (1616 is unpinned + can't JIT-run on
macOS regardless). Also swept the outdated "flat memory" terminology → "comptime/byte-addressable" (the VM is
arena-backed, `Addr` = real host pointer; flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
- **P5.6 — ALL bundling + code signing in `default_pipeline` (every target).** `default_pipeline` (or a
`bundle()` it calls, in `platform/bundle.sx`) performs, after `link`, the full per-target bundle when
`bundle_path()` is set — branching on `is_macos`/`is_ios_device`/`is_ios_simulator`/`is_android`:
- **macOS `.app`** — `Contents/{MacOS,Resources,Frameworks}`, `Info.plist`, embed `-framework` dylibs +
`install_name_tool` fixups, `codesign` (ad-hoc or with `codesign_identity`).
- **iOS device `.app`** — device slice, embedded `.mobileprovision` (`provisioning_profile`), entitlements,
`codesign` with the real identity; **iOS simulator `.app`** — sim slice, no provisioning, ad-hoc sign.
- **Android `.apk`** — `AndroidManifest.xml` (or `manifest_path` override), asset tree (`add_asset_dir`),
`#jni_main` Java → `javac` → `d8` → `classes.dex`, `aapt2` package, `zipalign`, `apksigner` with the
debug/`keystore_path` keystore.
All of it runs on the VM via the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI (the existing
`platform/bundle.sx` logic, now reading the VM-native accessors instead of `#compiler` hooks). The compiler
keeps ONLY the linker as a primitive (Option B). Remove the `--bundle`/`post_link_module` Zig shim — bundling
is `default_pipeline`'s job; CLI flags feed `BuildConfig` and `default_pipeline` branches on it.
- **P5.7 — DELETE all legacy.** Remove the `#compiler` attribute (parse + lower), the `compiler_call` IR op
(`inst.zig` + every switch arm + the `interp.zig:1130` dispatch), `compiler_hooks.zig`
(`HookFn`/`Registry`/all hooks). Make `-Dcomptime-flat` permanent (VM always) and **delete `interp.zig`**
(`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`/`last_bail_*`); drop the
`regToValue`/`valueToReg` bridge and the VM→legacy fallback in `emit_llvm` (`#run`/const-init) and
`comptime.zig` (type-fn / `#insert`) — a VM bail is now ALWAYS a build-gating diagnostic (4B wiring), never a
fallback. `core.invokeByFuncId` is already VM-only. Re-express `define`/`make_enum` as sx over the
compiler-API. Land the 0141 repro as a corpus test. Reconcile 1654 (asm-global at comptime) to the VM wording.
- **P5.8 — real-project validation (integration).** Build `~/projects/m3te` and `~/projects/distribution` with
the new pipeline end-to-end (their real bundle/codesign/target configs) — these are the acceptance test that
`default_pipeline` covers all targets. Fix gaps surfaced there. Add dedicated bundle smoke tests (min `.app` +
`.apk`) to the corpus (the bundler still has no `zig build test` coverage — the stream's top risk).
**End state:** ONE evaluator (the VM); ZERO legacy; the entire build — emit, link, and all bundling + code
signing for macOS/iOS-device/iOS-sim/Android — is sx in `default_pipeline`, overridable via `#run on_build(...)`.
The compiler is: parse → IR → codegen → invoke `on_build`/`default_pipeline` on the VM (which calls back into
the linker primitive). `m3te` + `distribution` build clean.
**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ;
FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F.
**Top risks:** (1) the bundler has no corpus guard (4E needs dedicated tests); (2) deleting
`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API (`abi(.compiler)`) touches the
whole build/bundle path — stage it behind real bundle builds; (3) S3's emit-skip relies on DCE dropping the
unreferenced compiler-domain declaration — verify no stray runtime reference keeps it alive (link error).
## Open questions (resolve as reached, record decisions here)
- **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed
records are host-laid-out; user comptime types are target-laid-out. The comptime
model must carry both regimes (a per-type ABI tag on layout queries). Confirm the
boundary where a comptime pointer to a compiler record is handed to host Zig code
uses host layout.
- **Exposing compiler types to sx.** Mechanism for projecting `types.zig` records into
the comptime type table with real offsets (the non-weld replacement) — a registry the
compiler owns, keyed by sx-visible name → real Zig type's layout + a host-call ABI.
- **Bytecode shape.** IR-derived vs a fresh ISA; register vs stack; fragment caching.
- **Pointer escape / lifetime.** Flat-memory pointers stored into the persistent type
table must be copied into compiler-owned memory at the boundary (as today).
- **Old-path retirement.** Keep the tagged interpreter until Phase 1 parity, then
delete — confirm no non-comptime consumer depends on `Value`.
## File map (current → touched)
| Area | File | Phase |
|------|------|-------|
| Comptime evaluator | `src/ir/interp.zig` | 0 (strip weld dispatch), 12 (rebuild) |
| Weld registry | `src/ir/compiler_lib.zig` | 0 (strip), 3 (replace with type/fn exposure) |
| Weld validation | `src/ir/lower/nominal.zig` | 0 (strip `validateWeldedStruct`) |
| Comptime-only gate | `src/backend/llvm/ops.zig` | 0 (re-derive without weld signal) |
| Host-FFI marshalling | `src/ir/host_ffi.zig` | 1 (struct-by-pointer trims it) |
| Metatype arms | `src/ir/interp.zig` (`defineStruct`/…/`reflectTypeInfo`) | 3 (delete, re-express in sx) |
| `#compiler` / BuildOptions | `library/modules/build.sx`, `src/ir/compiler_hooks.zig` | 3 (migrate, delete `#compiler`) |
| Surface syntax | `src/parser.zig`, `src/ast.zig` (`abi`/`extern`/`#library`) | kept; revisited Phase 3 |
## Status
- **Phase 0 — DONE (2026-06-17).** The struct-weld machinery is stripped:
`compiler_lib.zig` lost the type registry (`weldStruct`/`bound_types`/`BoundType`/
`FieldLayout`/`findType`/`SxField`/`LayoutMismatch`/`validateStructLayout`);
`nominal.zig` lost `validateWeldedStruct`/`weldedFieldOrderStr` + the
`sd.abi == .zig` call; the struct-weld unit tests + examples `0625`/`0627`/`1183`/
`1186` are removed. **Decision (recorded):** the `intern`/`text_of` function
host-call bridge is KEPT — it is a clean scalar dispatch (string→handle), not
weld/serialize/marshal, and is the seed Phase 3 grows the compiler-call path from.
So the `compiler_welded` dispatch (`interp.callExtern` is unchanged at HEAD — the
pre-branch in `call()`), `weldedCompilerFn` (decl.zig), the `emitCall` comptime-only
gate (ops.zig), and examples `0626`/`1184`/`1185` stay. The `#library`/`abi`/`extern`
SYNTAX stays. `zig build test` green (688 corpus, 0 failed; unit tests pass).
- **Phase 1 — in progress.**
- **Sub-step 1 — DONE.** `src/ir/comptime_vm.zig`: the comptime `Machine`
(linear byte memory + bump/stack allocator with `mark`/`reset` reclamation +
scalar `readWord`/`writeWord` (1/2/4/8, little-endian) + `bytes` views; addr 0
reserved as `null_addr`) and `Frame` (register file indexed by Ref + stack
reclamation on `deinit`). A register `Reg` is a raw u64 — immediate scalar OR
`Addr`. Standalone + unit-tested (`comptime_vm.test.zig`, in the barrel); does
NOT touch the live interpreter, so the corpus stays green (688). No op execution
yet.
- **Sub-step 2 — DONE.** The executor (`Vm` in `comptime_vm.zig`): walks the SAME
IR `Inst` over comptime frames, mirroring the legacy interp's scalar semantics
(i64 wrapping/signed + f64 register words, keyed off the result/operand `TypeId`).
Ported: constants (`const_int`/`float`/`bool`/`null`/`undef`), arithmetic
(`add`/`sub`/`mul`/`div`/`mod`/`neg`), comparison (`cmp_*`), logical
(`bool_and`/`or`/`not`), conversions (`widen`/`narrow`/`bitcast` passthrough,
`int_to_float`/`float_to_int`), terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param` (branch args passed as Refs — the same frame persists, SSA-safe).
Any other op bails loudly (`error.Unsupported` + `detail = @tagName(op)`).
Unit-tested on hand-built IR (`Fb` builder): integer add, f64 arithmetic, cond_br
branch selection, a block-param loop summing i..1, div-by-zero + unsupported-op
bails. Corpus untouched (688 green) — the executor is exercised by unit tests only,
not yet wired to real comptime eval.
- **Sub-step 3 — DONE.** Memory + structs on comptime memory. `Vm` gained an optional
`table: *const TypeTable` (target-aware layout). Ported `alloca`/`load`/`store`
(over comptime addresses, `Store.val_ty` drives width) and `struct_init`/`struct_get`/
`struct_gep` (structs laid out at the table's natural offsets). The value model: a
`Kind.word` (scalar/pointer ≤8B) sits in a register; a `Kind.aggregate` (struct)
lives in comptime memory and its "value" IS its address (read returns the address,
write memcpys), so nested structs compose and `struct_gep` is just base+offset (no
field-pointer dance). `kindOf` bails loudly on the not-yet-ported types
(slice/string/any/optional/enum/array/tuple/…). The Addr-based value model survives
allocator realloc (offsets are stable; slices are only materialized transiently).
Unit-tested: struct_init+get round-trip, alloca+gep+store+load, nested-struct
aggregate copy + nested read. Corpus untouched (688 green).
- **Sub-step 4a — DONE.** Tuples + arrays. `kindOf` widened (`tuple`/`array` →
aggregate). Ported `tuple_init`/`tuple_get` (positional, `tupleFieldOffset`),
`index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over array/pointer/
many_pointer bases; slice/string bases bail), and `length` on an array value
(static `ArrayInfo.length`). Unit-tested: mixed tuple round-trip, `[3]i64`
gep/store + index_get sum (42), array `length` (3). 688 corpus green.
- **Sub-step 4b — DONE.** Slices + strings as `{ptr@0 (pointer_size), len@8 (i64)}`
fat pointers (`kindOf`: string/slice → aggregate). Ported `const_string` (materializes
text+NUL in comptime memory + a fat pointer), `length`/`data_ptr` (read len/ptr fields),
`array_to_slice`, `subslice`, indexing *through* a slice/string (`elemAddr` loads
`.ptr` first), and `str_eq`/`str_ne` (len+memcmp). Helpers `makeSlice`/`sliceLen`/
`sliceData`. Unit-tested: string length + str_eq/ne, array→slice + slice index +
slice length (23), array subslice (43). 688 corpus green.
- **Sub-step 4c — DONE (optionals + payloadless enums).** `kindOf`: `enum` → word;
`?T` → word if pointer-child (null==0) else `{T@0, i1@sizeof(T)}` aggregate. Ported
`optional_wrap`/`unwrap`/`has_value`/`coalesce` (with `optChildIsPtr`/`optHas`
helpers; `const_null` → `null_addr` reads as none), `enum_init` (payloadless: tag is
the value), `enum_tag` (payloadless/word). Unit-tested: non-pointer `?i64`
wrap/unwrap/coalesce (91), pointer `?*i64` null==0 (99), payloadless enum tag (11).
688 corpus green.
- **Sub-step 4d — partial (`addr_of`/`deref` DONE).** `addr_of` passes through (an
aggregate value already IS its address; a pointer is already an address — mirrors
the legacy); `deref` = `readField` through the pointer (`ins.ty` is the pointee).
Unit-tested (deref a `*i64` → 77; addr_of a struct value + field read → 80).
**Deferred to the wiring phase (intentionally, not ported blind):** tagged-union
payload (`enum_init` w/ payload, `enum_payload` — the legacy stores *untyped* Values
and `field_index` indexes payload sub-fields, not variants, so a byte model's
payload type is ambiguous without a real call site), `any` boxing, closures, and the
bitwise ops. These have subtleties best resolved against actual corpus cases — the
VM's loud `error.Unsupported` + `detail` will name exactly what each real eval needs.
- **Sub-step 1.5 — direct `call` DONE.** `Vm` gained `module: *const Module`
(resolves a callee `FuncId`) + a `depth`/`max_depth` recursion guard. `call`
marshals arg Refs → Reg words and recursively `run`s the callee; aggregate args/
results pass as their `Addr` over the SHARED comptime memory (no copy). **Stack-lifetime
change:** `Frame` no longer reclaims the machine on exit (a returned aggregate's
Addr would dangle) — a comptime eval's allocations live to `Vm.deinit`;
`Machine.mark`/`reset` stay for explicit use. Extern/builtin callees (no blocks)
bail loudly (1.5b). Unit-tested: direct call (`add(20,22)+100` → 142) and recursion
(`sum(0..n)` → 15/55). 688 corpus green.
- **Sub-step 1.5b — `Reg`↔`Value` boundary bridge DONE.** The builtin/`compiler_call`/
extern handlers are all coupled to the legacy `Interpreter` (e.g. `compiler_lib`
handlers take `*Interpreter`), so the VM can't call them directly — the wiring uses
WHOLE-FUNCTION fallback instead (VM runs pure functions; a bail re-runs the whole
eval in the legacy). That needs the boundary bridge: `valueToReg` (host `Value` arg →
VM `Reg`, materializing aggregates into comptime memory) + `regToValue` (VM result →
`Value`, deep-copied out). Covers scalars + strings + structs (other aggregate shapes
bail loudly; added as wiring surfaces them). Transitional — deleted once the VM owns
comptime end-to-end. Unit-tested with round-trips. 688 corpus green.
- **Then the wiring step** (below) — now unblocked.
### Decision (2026-06-17): pivot from blind op-porting to CALLS + hybrid wiring
The common leaf ops are ported (scalars, control flow, structs, tuples, arrays, slices,
strings, optionals, payloadless enums, deref/addr_of) and unit-tested. Continuing to
port the rarer ops (tagged-union payload, any, closures) in isolation risks subtle
bugs and has low signal. The higher-value path:
1. **Calls (sub-step 1.5)** — `call` (direct), then `call_builtin`/`compiler_call`. The
shared comptime memory makes aggregate args/results pass naturally (they're Addrs). The
one design point: **aggregate-return lifetime** — a callee's stack-reclaim would
dangle a returned struct Addr, so for comptime (bounded) the VM should stop
reclaiming per-frame and let the whole eval's allocations live until `Vm.deinit`
(keep `Machine.mark/reset` for explicit use; drop it from `Frame.deinit`).
2. **Hybrid wiring** — `-Dcomptime-flat` routes a comptime eval through the VM, falling
back to the legacy interp on `error.Unsupported`. This makes the VM run the REAL
corpus, proving parity incrementally and surfacing exactly which ops each real eval
needs — far better signal than more isolated unit tests.

124
current/PLAN-DIST.md Normal file
View File

@@ -0,0 +1,124 @@
# PLAN-DIST — bundle `zig` as sx's hermetic link/libc backend
## Goal
`sx build` produces a native binary by driving a **bundled `zig`**
(`zig cc`) as the linker, so a distributed sx on Linux needs no system
`cc`/lld/libc/CRT. `sx run` (JIT) is unaffected — it never links.
This is the "be like Zig" move: reuse Zig's hermetic toolchain (lld +
crt objects + musl/glibc, all bundled in the `zig` distribution) instead
of building our own lld-in-process + libc-from-source pipeline.
> **Configuration surface** (env vars, flags, resolution order,
> activation truth table, target→ABI map, distribution layout) is
> specified in [../design/bundled-zig-link-backend-design.md](../design/bundled-zig-link-backend-design.md) — the design-of-record
> for how the backend is configured. Keep the two files in sync.
## Locked decisions
1. **Default Linux output ABI = static musl** (`x86_64-linux-musl`,
`-static`). Output runs on ANY Linux with zero deps — the property
that makes Zig binaries portable. glibc/dynamic only via explicit
`--target x86_64-linux-gnu`.
2. **Activation = auto** when a bundled/resolvable `zig` exists AND the
user passed no `--linker`. Falls back to system `cc` otherwise.
3. **Dev uses PATH `zig`** (0.16.0 already installed). Defer copying a
vendored toolchain into `libexec/` until Phase 3 packaging.
## Why `zig cc`, not raw `ld.lld`
`zig cc` is a clang-compatible driver, so it slots into the **existing**
cc-style argv branch in `src/target.zig` almost unchanged, and supplies
lld + crt objects + musl/glibc automatically per `-target`. Driving
`ld.lld` directly would force us to locate/pass crt1.o/crti.o/libc
ourselves — exactly the work we're avoiding.
## Key code anchors (verified)
- Linker selection hook: `TargetConfig.getLinker()``src/target.zig:194-196`
(`self.linker orelse "cc"`).
- Unix `cc`-style link branch: `src/target.zig:524-564` (this is where
the zig backend hooks in; `-o`/`-L`/`-l`/extra objects already pass
through clang-compatibly).
- Exe-relative resolution pattern to mirror for finding zig:
`src/imports.zig:204-227` (`discoverStdlibPaths`, `$SX_STDLIB_PATH`
override + `<exe>/..` candidates).
- `--linker` CLI flag parsing: `src/main.zig:87-90`.
- Emit triple (must agree with link target): `src/ir/emit_llvm.zig`
(`LLVMSetTarget`, ~L246-284).
## Phases
### Phase 0 — Resolve a bundled/host zig
- New `src/zig_backend.zig`: `discoverZig(alloc) -> ?[]const u8`.
Resolution order:
1. `$SX_ZIG` env override.
2. `<exe>/../libexec/zig/zig` (install layout, Phase 3).
3. `<exe>/../../zig-bundle/zig` (dev vendored layout, Phase 3).
4. `zig` on `PATH` (dev fallback — active now).
- Add `SX_DEBUG_ZIG` trace, matching existing `SX_DEBUG_*` hooks.
- No behavior change yet; just resolution + a debug/print hook to confirm.
### Phase 1 — `zig cc` link backend (core change)
- `src/target.zig`: generalize the linker from a single token to a
**driver argv**. Today `getLinker()` returns one string at `argv[0]`;
introduce a `LinkBackend` so the internal backend contributes
`{zigPath, "cc"}` as leading entries.
- In the Unix branch (L524-564), when backend = zig:
- prepend `zig cc`,
- append `-target <mapped triple>`,
- add `-static` for musl,
- everything else (`-o`, `-L`, `-l`, extra objects, extra link flags)
passes through unchanged.
- Add `sxTripleToZig()` mapping (sx shorthand/triple → zig `-target`);
unspecified-on-Linux → `x86_64-linux-musl`.
- Align emit triple: when the zig backend is selected, set the LLVM
module triple in `emit_llvm.zig` to match the link target
(x86_64-linux), so the `.o` links cleanly against musl crt.
### Phase 2 — Activation
- Auto-enable: if `discoverZig()` succeeds and no `--linker` override,
use the zig backend for `sx build`. System `cc` remains the fallback.
- Optional explicit `--self-contained` / `--no-self-contained` to force.
- Confirm `sx run`/JIT path is untouched (no link step).
### Phase 3 — Distribution packaging
- `build.zig`: a `dist` step assembling
- `bin/sx` (built with `-Dstatic-llvm`),
- `libexec/zig/` (vendored zig binary **and its `lib/`**, copied from a
pinned ziglang.org release per host arch),
- `library/` (stdlib),
into a relocatable tarball.
- Pin the zig version (currently 0.16.0).
### Phase 4 — Verify & lock
- Manual first: `sx build hello.sx` (auto zig backend) then `file`/`ldd`
the output → expect "statically linked".
- Honor snapshot-integrity + FFI-cadence rules before adding a corpus
test (host/arch-gated, likely a `.build` sidecar).
## Risks / watch
- **Bundle size**: zig + its `lib/` ≈ 5060 MB.
- **gnu vs musl ABI**: pure codegen objects link fine against musl;
TLS/stack-protector are the only realistic friction. Aligning the emit
triple (Phase 1) covers the common path.
- **macOS/Windows cross** via the same `zig cc -target` is nearly free
after Phase 1, but Apple-SDK linking has caveats — scope to Linux
target first; treat the rest as follow-up.
- **c_import.zig** also shells `cc` for C imports (JIT). Out of scope
here; same backend can absorb it later.
## Status
- [x] Phase 0 — resolve zig (`src/zig_backend.zig`)
- [x] Phase 1 — zig cc link backend (`target.zig` + `emit_llvm` triple normalize)
- [x] Phase 2 — activation (`--self-contained`/`--no-self-contained`; auto on bundled zig)
- [ ] Phase 3 — dist packaging (vendor `zig` into `libexec/`)
- [ ] Phase 4 — verify & lock (manual ✓ macOS/Linux/Windows; corpus test pending runner `--self-contained` support)
Scope landed as **macOS + Linux + Windows** (not Linux-first). See the
"Implementation status" section in
[../design/bundled-zig-link-backend-design.md](../design/bundled-zig-link-backend-design.md)
for what refined the original locked decisions.

View File

@@ -5,7 +5,7 @@
They are *one* plan: Part B can't start until Part A is a behavior-equivalent They are *one* plan: Part B can't start until Part A is a behavior-equivalent
superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant. superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant.
**Design rationale:** [docs/inline-asm-design.md](../docs/inline-asm-design.md) §II.2 **Design rationale:** [design/inline-asm-design.md](../design/inline-asm-design.md) §II.2
(Deviation 6) + §II.10 #4 + the syntax evaluation. (Deviation 6) + §II.10 #4 + the syntax evaluation.
**Decided syntax** **Decided syntax**
@@ -173,7 +173,7 @@ gate only the live tree (recommended) vs purge everything. Confirm 6 before Phas
> Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint > Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint
> `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided > `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided
> syntax, Naming constraint, Key finding) and Part A; rationale is in > syntax, Naming constraint, Key finding) and Part A; rationale is in
> `docs/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4. > `design/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4.
> >
> **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix > **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix
> keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays > keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays

254
current/PLAN-FIBERS.md Normal file
View File

@@ -0,0 +1,254 @@
# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
> **STATUS: 🚧 in progress.** B1.0 (`abi(.naked)`) ✅ + B1.1 (per-fiber `context`) ✅. **B1.2**
> (`Io` interface) is **UNBLOCKED** — the earlier "blockers" were artifacts of non-idiomatic
> syntax + a worker's dirty binary. Issue **0151 was INVALID** (the `($A)->$R` bare-fn-ptr
> form is not idiomatic sx) and is **removed**. The correct `async` idiom **works today, no
> compiler change**: `async :: (io, worker: Closure(..$args) -> $R, ..$args) -> Future($R)`
> with a **lambda worker** + the `result : Future($R) = ---; result.v = worker(..args);` build
> form (mirrors the canonical `examples/0543-packs-canonical-map.sx`). Caveats: lambda params
> must be annotated; passing a bare *named* fn as the worker is non-idiomatic (use a lambda).
> Issue **0150** (`void` struct field SIGTRAP, exit 133) is a **real** bug but only hit by
> `Future(void)`/`timeout` — **deferred** (avoid void Futures in B1.2; revisit in B1.4). Resume
> B1.2 with the corrected idiom (the WIP at `.sx-tmp/b12-wip/` has the Io-protocol/Context/
> materializer parts that WORK; rewrite the async layer to the pack-lambda form above).
Carved from [PLAN-POST-METATYPE.md](PLAN-POST-METATYPE.md) Stream B (§B1) + the
design-of-record [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md)
§4 (async), §7 steps 49, §8.1 (risks), §10 (testing). Progress in
[CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). Stream B2 (channels/cancel/stdlib) is a
separate carve ([PLAN-CHANNELS.md], when reached) and depends on this + atomics (✅).
**Goal:** the colorblind, stackful, **pure-sx** async runtime — fibers behind an `Io`
interface, an M:1 scheduler, blocking + deterministic-sim + event-loop `Io` impls. The
**compiler floor is small and net-new**: make `abi(.naked)` actually emit an LLVM `naked`
function (B1.0), and confirm/close the per-fiber `context` root (B1.1). **Everything
else — the context-switch asm, fiber bootstrap, `mmap` stacks, the scheduler, futures,
the `Io` vtables — is ordinary sx library code** (design §4, §4.4). The irreducible FFI
floor: the per-arch asm context-switch (in `.sx`), syscall `extern`s, and `mmap`.
**Cadence (IMPASSIBLE):** no commit both adds a test AND makes it pass (lock-to-bail, then
flip to green); `zig build && zig build test` green after every step; never regen snapshots
while red; scope regens with `-Dname=examples/NNNN-…sx -Dupdate-goldens` + review the diff.
New corpus category: `18xx` concurrency. On an **unrelated** compiler bug → file
`issues/NNNN`, mark this checkpoint BLOCKED, STOP (CLAUDE.md). The in-session
worker-fix override (delegate a blocker to a worker) applies only with explicit user
authorization.
---
## Design (grounded against the tree)
### B1.0 — `abi(.naked)` codegen (the one genuinely net-new compiler piece in B1)
The design doc spells this `callconv(.naked)`; the **real sx surface is `abi(.naked)`**
written in the postfix slot, `name :: (sig) -> Ret abi(.naked) { asm { … }; }` (cf.
`build_options :: () -> BuildOptions abi(.compiler);` in [build.sx:28](../library/modules/build.sx#L28)).
The sx-facing name is **`naked`** throughout (keyword, field `is_naked`, diagnostics) —
matching LLVM's `naked` attribute (the lowering mechanism) and the industry term
(Zig/Rust/GCC/Clang). The ABI variant was renamed `.pure → .naked`: "pure" universally
means *side-effect-free*, the opposite of a register-clobbering context switch.
**Grounding (verified — do not re-derive):**
- The `ABI` enum **already carries `.naked`**`ABI = enum { default, c, compiler, naked }`
([ast.zig:142](../src/ast.zig#L142)), documented "naked function (inline asm
body), no calling-convention prologue/epilogue." So B1.0 is **NOT** "extend the enum."
- `.naked` is **inert today**: [type_resolver.zig:237](../src/ir/type_resolver.zig#L237)
maps `.compiler, .naked → .default` CC, and `emit_llvm` emits **no LLVM `naked`
attribute**. So the net-new work is exactly: **carry `abi == .naked` into the IR
`Function`, emit LLVM's `naked` attr, and skip the implicit-`Context` / prologue
lowering** so the body is just the asm block + its own `ret`.
- The IR `Function` struct ([inst.zig:605](../src/ir/inst.zig#L605)) carries `call_conv`
(default/c) + `is_compiler_domain`, but **no naked flag** — add one (`is_naked: bool`).
- Attribute API is in-tree: `nounwind` is set at
[emit_llvm.zig:1339](../src/ir/emit_llvm.zig#L1339) via
`LLVMGetEnumAttributeKindForName("nounwind", 8)``LLVMCreateEnumAttribute(ctx, id, 0)`
`LLVMAddAttributeAtIndex(func, func_idx_attr /* -1 */, attr)`. The LLVM `naked` attr
is the same shape: `LLVMGetEnumAttributeKindForName("naked", 5)`.
- The `.c` ABI **already skips the implicit ctx** at lowering — `lam.abi == .c` /
`fd.abi == .c` gates (closure.zig:171, [decl.zig:515](../src/ir/lower/decl.zig#L515)).
`.naked` must skip it **too** (a `.naked` fn gets no synthetic `__sx_ctx`, no stack frame,
no prologue — args arrive in ABI registers and are read directly from asm). The
implicit-return machinery (`lowerValueBody`) must also be bypassed: a `.naked` body has no
sx return (the asm rets itself), so lower its statements and cap the block with
`unreachable`.
- **Inline asm already works end-to-end** (lower→emit→JIT): aarch64
([examples/1645](../examples/1645-platform-asm-aarch64-add.sx)), x86_64
([examples/1651](../examples/1651-platform-asm-x86-syscall-write.sx)), global asm, JIT
([1653](../examples/1653-platform-asm-global-jit.sx)). `emitInlineAsm` /
`LLVMGetInlineAsm` at [ops.zig:915](../src/backend/llvm/ops.zig#L915). The `.naked` body
is a single asm block reusing this path.
**`.naked``.c` (design §4.6 context-switch note):** a `.c` epilogue restores SP from the
frame; a context switch deliberately makes SP-in ≠ SP-out, so the `.c` epilogue would
restore from the *wrong* stack. `.naked` = no prologue/epilogue/frame — the asm emits its
own `ret`. This is *why* the switch must be `.naked`, not `.c`.
**Snapshot story (per the atomics precedent):** a `.naked` fn's *body is raw per-arch asm*
(it can't be portable — that's the point), while LLVM's `naked` attribute text is
arch-invariant. **B1.0a** (lock) needs only **one host example** locked to the emit bail —
the bail fires at the function level *before* any asm/instruction selection, so it is
host-independent (no `.build` target pin). **B1.0b** (green) adds emission, pins that
example aarch64 (`.build {"target": "aarch64-macos"}`, end-to-end on a matching host,
ir-only on a mismatch), and adds an x86_64 cross sibling — mirroring the existing asm
corpus split (1645 aarch64 / 1651 x86). The ir-only `.ir` (only producible once emission
lands in B1.0b) asserts the `naked` attribute + the asm body. State loudly: **the `.ir`
proves the `naked` keyword + asm emitted, NOT that any hand-written register save/restore
is correct** — that is the B1.3 switch-stress harness's job, never the corpus's.
### B1.1 — per-fiber `context` root (grounding says this is SMALL, likely library-only)
**Grounding (verified — closes the design doc's open sizing question):**
- `context` is an **implicit `*Context` parameter** (`__sx_ctx`, slot 0), threaded through
every default-conv sx call ([lower.zig:259](../src/ir/lower.zig#L259)) — **not raw TLS**.
Inside a function `current_ctx_ref = Ref.fromIndex(0)` (the param) → it **rides the fiber
stack frame for free**.
- `push Context.{…}` allocates the new `Context` with a **stack `alloca`** and rebinds
`current_ctx_ref` to that slot ([stmt.zig:1263](../src/ir/lower/stmt.zig#L1263)) — "No
global, no walk." So **push frames are fiber-local for free**.
- The **only shared root** is the `__sx_default_context` **global**, bound at
entry-points / `abi(.c)` fns *before any user code runs*
([decl.zig:2667](../src/ir/lower/decl.zig#L2667), :2815).
⇒ The design doc's "lower as swappable indirection, never raw TLS" guards a **non-problem**
(confirmed). The **real, now-sized** B1.1 work is purely a **library convention**: a
freshly-`spawn`ed fiber must take its root `Context` from the **spawner's snapshot** (passed
as the fiber-entry fn's `__sx_ctx` slot-0 arg by the spawn trampoline), **not** the
`__sx_default_context` global. That is sx-side (the trampoline already controls slot 0) —
**expected to be ZERO compiler change.** B1.1's first action is a probe confirming this; if
a fiber genuinely re-reads the global root mid-stack (it should not — entry binds once),
*then* and only then is there a compiler obligation. **Ground the probe before sizing any
compiler work.** Prerequisite of B1.3 (a fiber needs a valid root before it switches).
### B1.2B1.5 — pure sx over the primitives (design §4)
- **B1.2 (A1):** `Io` interface + `context.io` + `Future` + `cancel()` — a protocol/vtable
threaded exactly like `Allocator` (which already lives at `Context` field 0; see
`allocViaContext` [call.zig:1214](../src/ir/lower/call.zig#L1214)). `Io` becomes another
`Context` field. No compiler change — protocols + context already carry it.
- **B1.3 (A2):** the fiber runtime — naked context-switch asm (per-arch), bootstrap, `mmap`
stacks **with mandatory guard pages**. All sx. **Highest corruption risk in the stream**
(§8.1.1) and **untestable by the deterministic `Io`** (which tests *scheduling*, not the
*switch*). Its **first deliverable, before the scheduler AND the deterministic `Io`**: a
standalone **2-fiber ping-pong switch-stress harness** (§10.7) — scribble every
callee-saved register + a stack canary before each suspend, deep/recursive chains, verify
all survive post-resume. This harness — not B1.4 — is A2's correctness gate.
- **B1.4 (A3):** `Io` impls in order **blocking → deterministic-sim (KEYSTONE) → event-loop**
(kqueue/epoll/io_uring). Build the deterministic `Io` right after blocking; **calibrate it
against blocking `Io`** before trusting it to gate everything async (§8.1.3, §10.7) — a
deterministic-but-wrong scheduler snapshots garbage. (Open, deferred: the event loop does
**not** yet cooperate with a platform UI run loop — CFRunLoop/ALooper; that's a §6
app-target gap, out of B1.)
- **B1.5 (A5·M:1):** the single-thread scheduler — validates the whole colorblind stack
end-to-end. `18xx` corpus runs under the deterministic `Io`, asserting a **program-emitted
ordering contract** (sequence markers), not raw interleaving, so scheduler-policy tweaks
don't churn every snapshot.
### Files the compiler floor touches (B1.0 only; B1.1B1.5 are library + tests)
B1.0 (`.naked`) forces these plumbing sites:
- [ast.zig:142](../src/ast.zig#L142) — `ABI.naked` (exists; reference only).
- [inst.zig:605](../src/ir/inst.zig#L605) — add `is_naked: bool = false` to `Function`.
- [decl.zig](../src/ir/lower/decl.zig) — set `is_naked` from `fd.abi == .naked`; gate the
implicit-ctx off for `.naked` in `funcWantsImplicitCtx` (mirror the `.c` skip at
decl.zig:515) and bypass `lowerValueBody` for `.naked` bodies (lower statements + cap with
`unreachable`, in both body-lowering paths) — a `.naked` fn binds no ctx and has no sx
return.
- [type_resolver.zig:237](../src/ir/type_resolver.zig#L237) — leave CC `.default` (a `.naked`
fn-pointer type has no CC of its own; nakedness is a decl-level emit attribute).
- [emit_llvm.zig:402](../src/ir/emit_llvm.zig#L402) Pass 2 — **B1.0a:** bail loudly when
`func.is_naked` (build-gating). **B1.0b:** instead emit LLVM's `naked` attr (shape per
`nounwind` at emit_llvm.zig:1339) + the asm-only body (no prologue).
- Any `.op`/`Function`-field switch the Zig build flags — let the build tell you.
---
## Phases (xfail→green steps)
### B1.0 — `abi(.naked)` codegen — ✅ COMPLETE
- **B1.0a (lock) — ✅ DONE.** Carried `abi == .naked` into IR `Function.is_naked`; threaded
through `decl.zig` (`funcWantsImplicitCtx` skips `.naked` like `.c`; all body-lowering paths
bypass `lowerValueBody` for `.naked`, lowering the asm body + capping with `unreachable`) +
generic.zig + pack.zig; `emit_llvm` Pass 2 bailed loudly on `func.is_naked`. Locked by
`examples/1800-concurrency-naked-asm.sx` + the generic regression (review-found gap).
- **B1.0b (green) — ✅ DONE.** `emit_llvm` declaration pass adds LLVM `naked` + `noinline` +
`nounwind` for `func.is_naked` and skips `frame-pointer=all` (incompatible with a frameless
function); Pass 2 emits the body normally (`naked` ⇒ verbatim asm + own `ret`, no
prologue). `1800` pinned aarch64 → exit 42 + `.ir`; `1801-concurrency-naked-generic.sx`
(renamed from `-bail`) proves the generic path emits a naked body (exit 42);
`1802-concurrency-naked-asm-x86.sx` x86_64 cross sibling (ir-only here, `.ir` locks `naked`
+ `movl $42, %eax`). Unit test `emit: abi(.naked) function gets the naked attribute` asserts
`naked` present + `frame-pointer` absent. Suite green (724/0).
- **B1.0c (review-hardening) — ✅ DONE.** A param-bearing `.naked` fn emitted invalid LLVM
(loud verifier error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both
paths + generic.zig) so a naked fn's args stay in registers (read by the asm body) — this
*enables* B1.3's `swap_context(from, to)`. Locked by `1803-concurrency-naked-asm-param.sx`.
Pack `.naked` (variadic + naked, nonsensical) left unsupported → loud verifier error.
### B1.1 — per-fiber `context` root — ✅ COMPLETE (zero compiler change)
Probe confirmed the spawn convention works with ordinary language features: snapshot
`context` (`snap := context`), store it in a struct, and `push f.root { entry(args) }` from a
trampoline running under a different ambient context — the body reads the snapshot (via the
implicit slot-0 `*Context` param), not the ambient ctx, and `push` restores ambient on exit.
No path re-reads `__sx_default_context` mid-stack ⇒ **no compiler obligation**; this is a pure
library convention. Locked by `examples/1804-concurrency-context-snapshot.sx` (`fiber root:
42` / `ambient after: 99`). The design doc's "never raw TLS" guarded a non-problem.
### B1.2 — A1: `Io` interface + `context.io` + `Future` + `cancel()` API
Library-only. `Io` as a protocol added to `Context` (mirror `Allocator`). `Future`/`cancel`
API surface. xfail→green via an `18xx` example exercising the blocking `Io` default (real
suspend lands in B1.3). No compiler change expected; if a protocol-in-context gap appears,
file it.
### B1.3 — A2: fiber runtime (naked switch + bootstrap + guarded `mmap` stacks)
- **B1.3a (switch-stress harness FIRST)** — the standalone 2-fiber ping-pong harness
(register + canary survival, deep chains) per §10.7. This is A2's gate and predates the
scheduler + deterministic `Io`. Arch-gated run test (matching-host run; ir-only elsewhere).
- **B1.3b** — fiber bootstrap + `mmap` stacks **with guard pages** (mandatory — §8.1.1).
- (Cadence inside B1.3 follows lock→green per sub-piece; the asm switch is the highest-risk
artifact — review adversarially, with a worker if authorized.)
### B1.4 — A3: `Io` impls (blocking → deterministic-sim KEYSTONE → event-loop)
Blocking first; then the deterministic-sim `Io`, **calibrated against blocking** before any
`18xx` test trusts it; then the event loop. The deterministic `Io` is the test harness for
*all* of B1.5 + Stream B2.
### B1.5 — A5: M:1 scheduler
End-to-end validation of the colorblind stack. `18xx` corpus under the deterministic `Io`,
asserting program-emitted ordering contracts.
---
## Gates
- **B1.0:** unit `emit_llvm.test.zig` (the `naked` attr present on a `.naked` fn); two
arch-gated examples (aarch64 + x86_64) run end-to-end on a matching host, ir-only on a
mismatch (assert `naked` + asm in `.ir`). **OUT of corpus scope, stated loudly:** the
*correctness* of any hand-written register save/restore — that's the B1.3 stress harness.
- **B1.1:** an `18xx` example locking context-carried-by-slot-0 behavior + a checkpoint note
on the spawn-trampoline convention.
- **B1.3:** the **switch-stress harness is A2's gate** (register/canary survival — §10.7),
NOT a run/snapshot test; plus arch-gated run tests.
- **B1.4:** deterministic `Io` **calibrated** against blocking `Io` (§8.1.3) before trusting
it; `18xx` under the deterministic `Io`.
- **B1.5:** `18xx` ordering-contract snapshots under the deterministic `Io`.
## Kickoff prompt (B1.0b — paste into a fresh session)
> Implement Stream B1 step **B1.0b** (`abi(.naked)` real emission) per
> `current/PLAN-FIBERS.md`. Verify `zig build && zig build test` is green first (B1.0a is
> already landed: `Function.is_naked` plumbed, `decl.zig` skips ctx + bypasses implicit-return
> for `.naked`, `emit_llvm` Pass 2 bails loudly, `examples/1800-concurrency-naked-asm.sx`
> locked to the bail). Then: (1) in `src/ir/emit_llvm.zig` Pass 2 (~line 402), REPLACE the
> `func.is_naked` bail with real emission — set LLVM's `naked` attribute on the function
> (`LLVMGetEnumAttributeKindForName("naked", 5)` → `LLVMCreateEnumAttribute(ctx, id, 0)` →
> `LLVMAddAttributeAtIndex(llvm_func, -1, attr)`; shape per the `nounwind` set at
> emit_llvm.zig:1339) and emit the `.naked` body as its asm block only, no prologue/epilogue
> (the body already lowers to the inline-asm op + an `unreachable` terminator). (2) Pin
> `examples/1800-concurrency-naked-asm.sx` aarch64 with a `.build` sidecar
> `{"target":"aarch64-macos"}`; on this aarch64 host it runs end-to-end (exit 42), capture
> `.ir` + regen (`-Dname=examples/1800-concurrency-naked-asm.sx -Dupdate-goldens`), review the
> diff (assert the `.ir` shows the `naked` attr + `mov x0, #42` / `ret`, NO stray error
> text). (3) Add `examples/1802-concurrency-naked-asm-x86.sx` (x86_64 body, `.build
> {"target":"x86_64-linux"}`, ir-only on this host — requires its `.ir`, now producible).
> (4) Add a unit test in `src/ir/emit_llvm.test.zig` asserting the `naked` attribute is
> present on an `abi(.naked)` function. Confirm `zig build test` green, commit. NOTE: the
> `.ir` proves the keyword + asm emitted, NOT register-save correctness (that's the B1.3
> switch-stress harness). If you hit an UNRELATED compiler bug, file `issues/NNNN`, mark
> `CHECKPOINT-FIBERS.md` BLOCKED, and STOP.

142
current/PLAN-METATYPE.md Normal file
View File

@@ -0,0 +1,142 @@
# PLAN-METATYPE — comptime type metaprogramming (`declare` / `define` + reflection)
## Goal
Comptime type metaprogramming with the smallest possible compiler surface:
- **`declare(name) -> Type`** — mint a NEW empty (undefined) nominal type NAMED
`name`, returned as a first-class `Type` handle. The compiler registers the
forward type at compile time, so the body can reference it (`*Name`).
- **`define(handle, info) -> Type`** — fill a declared handle's body from a
`TypeInfo` *value*, and return the handle (so the one-shot form chains).
- **`type_info($T) -> TypeInfo`** — reflect a type INTO data (the inverse of
`define`'s decode). *Done for enums* (`interp.zig:reflectTypeInfo`,
`examples/0619`); struct/tuple widening pending.
- **`field_type($T, i) -> Type`** — the i-th field / variant-payload / element
type of `$T`. *Done.*
These four `#builtin`s in `library/modules/std/meta.sx` are the **entire**
compiler surface. Every higher-level constructor is **plain sx built over
`declare`/`define`** — the compiler knows none of them by name:
```sx
// one-shot (non-recursive): declare + define chained, define returns the handle
T :: define(declare("T"), .enum(.{ variants = .[ … ] }));
// recursive: a ctor fn names the forward type via declare, references it as *Name
List :: make_list();
make_list :: () -> Type {
h := declare("List");
return define(h, .enum(.{ variants = .[
EnumVariant.{ name = "cons", payload = *List }, // self-reference
EnumVariant.{ name = "nil", payload = void } ] }));
}
// type-fns are ordinary sx (channel result types, etc.)
RecvResult :: ($T: Type) -> Type {
return define(declare("RecvResult"), .enum(.{ variants = .[
EnumVariant.{ name = "value", payload = T },
EnumVariant.{ name = "closed", payload = void } ] }));
}
```
This gates channel result types (`RecvResult($T)`) and `race`'s synthesized
tagged-union (design [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §7 step 3), and replaces a would-be `enum($T)` language feature.
## How it works (the locked design)
1. **Two comptime interp builtins.** `declare` mints an empty `tagged_union` slot
in the type table; `define` decodes the `TypeInfo` value (variant-name strings +
payload `Type`-tags) and completes the slot byte-identical to a source enum's
`buildEnumInfo` output, so it flows through enum codegen unmodified. The interp
mutates the type table via a `mint` handle the host sets (`setMintTable`).
2. **No syntactic constructor recognition.** A `::` binding or type-fn body that
calls a `Type`-returning fn is **comptime-evaluated** (`evalComptimeType`): the
expression runs through the interpreter, the `declare`/`define` builtins mint the
type, and the result `type_tag` is bound. `decl.zig` triggers on a non-generic
`-> Type` fn call; `instantiateTypeFunction` triggers on a type-fn body that
returns a `define(…)` call (or a bodied `-> Type` helper) — see
`generic.zig:returnExprMintsType`.
3. **Name on `declare`.** `declare("Name")` carries the name as a compile-time
string so `preregisterForwardTypes` (in `evalComptimeType`) can register the
forward type — and bind it as a type alias — BEFORE the body lowers. That's
what makes a `*Name` self-reference resolve (a `Name :: ctor()` decl makes
`Name` a const_decl author, so `*Name` resolves through the forward-ALIAS path;
the alias binding, not just the table registration, is what satisfies it). The
interp's `declare` returns the same slot by name; `define` fills it in place.
4. **Nominal identity** rides the existing type-fn mangled-name instantiation cache:
`RecvResult(i64)` at two sites memoizes to ONE `TypeId` (the body runs once;
`renameNominalType` re-keys the minted type to the mangled name).
5. **Comptime-only, JIT-free.** `declare`/`define` are interp ops; reaching them at
runtime / emit is a hard error.
6. **Undefined-until-defined.** `declare()` mints an undefined slot; *using* it
(construct / match / size) before its `define` is a loud diagnostic. A *pointer*
to an undefined slot (`*Self`) is fine — that's what self-reference needs.
## Key code anchors
- Builtins: `BuiltinId.declare` / `.define` (`src/ir/inst.zig`); lowering to
`callBuiltin` (`src/ir/lower/call.zig:tryLowerReflectionCall`); interp exec +
`defineEnum` + `decodeVariantElements` (`src/ir/interp.zig`); `mint` field +
`setMintTable`.
- Comptime evaluation: `evalComptimeType` / `renameNominalType`
(`src/ir/lower/comptime.zig`); decl trigger `fnReturnsTypeValue`
(`src/ir/lower/decl.zig`); type-fn trigger `returnExprMintsType` +
`instantiateTypeFunction` (`src/ir/lower/generic.zig`).
- Reflection: `field_type``fieldTypeOf` (`src/ir/lower/generic.zig`).
- Surface: `library/modules/std/meta.sx` (on-demand import — NOT the prelude, to
avoid shifting every `.ir` snapshot).
## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior
lock). `zig build && zig build test` after every step. Never regenerate snapshots
while red. Examples: `06xx` (comptime), `11xx` (diagnostics).
## Status
- [x] `declare` / `define` comptime builtins + the `mint` plumbing.
- [x] Comptime evaluation of a `Type`-returning `::` RHS and type-fn body
(the only triggers; no constructor-name knowledge in the compiler).
- [x] Name-in-`TypeInfo`; nominal identity via the instantiation cache.
- [x] `field_type` reflection (`examples/0616`).
- [x] Examples green on the floor: `0614` (one-shot), `0615` (type-fn identity),
`0617` (channel result types).
- [x] **Self-reference** — recursive enums via `declare("Name")` + `*Name` in a
constructor fn (`preregisterForwardTypes` registers the forward type + alias
before the body lowers). `examples/0618` (recursive `*List`: construct, match
through the pointer, recursive traversal). Mutual recursion / by-value-self-ref
rejection fall out of the same mechanism (F5 adds the loud by-value check).
- [x] **`make_enum(name, variants: []EnumVariant)`** — the general enum constructor
over a COMPUTED (value, non-literal) variant list. Pure sx in `meta.sx`;
exercises `define` decoding a value-arg slice. `examples/0620` (array-literal
local) / `0624` (generic builder).
- [x] **Comptime slice over a non-string aggregate**`arr[lo..hi]` over an array
yields a real slice value at comptime (`base_ty` threaded onto `Subslice`;
open-ended `hi` folded to the array's static length; `subsliceElements`).
`examples/0621`.
- [x] **`type_info($T) -> TypeInfo`** — reflect `enum`/`tagged_union`/`struct`/`tuple`
INTO a value (inverse of `define`'s decode); `define` decodes all three back
(`defineEnum`/`defineStruct`/`defineTuple`, dispatched on the TypeInfo tag).
Round-trips: `examples/0619` (enum) / `0622` (struct) / `0623` (tuple). The
reflect/construct triad is complete.
- [x] **Generic type-fn body locals** — a generic `($T) -> Type` comptime-evaluates
its FULL body (prelude statements + return), so a local before the return
resolves (`createComptimeFunctionWithPrelude` / `evalComptimeTypeBody`).
`examples/0624`.
- [x] **Validation + loud diagnostics** — by-value self-reference (`checkInfiniteSize`,
source `1178` + constructed `1182`; issue 0139), duplicate variant/field names
(`1180`), `declare()` never `define()`d (`1181`, was a `verifySizes` panic),
and the 0140 bail-surfacing (`1179`). use-before-define is subsumed by these
(no new check needed). Validation story COMPLETE.
- [ ] **Comptime `List` growth** (issue 0141, DEFERRED) — `List(T).append` at
comptime bails (two layers: null comptime allocator at scanDecls + `*T`
slot_ptr `struct_get`). Non-blocking; array-literal locals cover the use case.
## Risks / watch
- **Self-ref timing** — `define` for the two-statement form must complete before any
code uses the type's layout; a use-before-define must be a loud diagnostic, not a
silent empty enum.
- Keep `declare`/`define` **comptime-only**: reaching them at runtime is a hard error
(emit should bail loudly if one ever leaks into codegen).

View File

@@ -0,0 +1,207 @@
# PLAN-POST-METATYPE — program plan for the async-first roadmap (everything after metatype)
Sequences every remaining stream after [PLAN-METATYPE.md](PLAN-METATYPE.md). This is the
**program-level** plan; each stream below is carved into its own
`PLAN-<STREAM>.md` + `CHECKPOINT-<STREAM>.md` (full step detail + kickoff prompt)
**when reached**, exactly as metatype was. Rationale, the comptime type-construction
design, risk ranking (§8.1), and the testing strategy (§10) all live in the design-of-record:
[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md).
**Cadence (IMPASSIBLE), every stream:** no commit both adds a test AND makes it pass
(lock, or xfail→green); `zig build && zig build test` green after every step; never
regenerate snapshots while red. On an unrelated compiler bug → file `issues/NNNN`,
mark the stream checkpoint BLOCKED, stop (CLAUDE.md rule).
**Ordering = async-first** (design §7): the async story needs no JIT spine, so the
JIT/FFI cluster comes after. New corpus categories: `17xx` atomics, `18xx` concurrency.
## Stream order (post-metatype)
| # | Stream | Roadmap steps | Depends on | Notes |
|---|--------|---------------|-----------|-------|
| **A** | Atomics | N1 (1) | — | ✅ **DONE**`PLAN-ATOMICS.md`. load/store/RMW/CAS/swap/fence; comptime value params landed alongside. Gates B2-channels + C-parallel |
| **B** | Async runtime | 412 | metatype, A (for channels) | the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved |
| **C** | Parallel schedulers | 1314 | A, B | N×(M:1) → M:N |
| **D** | Comptime JIT/FFI | 1518 | — (independent of async) | S1 → C1 → C2 → C3 |
| **E** | Hot-reload (deferred) | 1922 | D (S1/S2) | S2 → R1 → R2 → R3 |
A and D are independent of each other and of B's core; B is the spine of the async
story. **Recommended execution order: A → B → C → D → E** (async-first; D can slot
earlier if FFI/`#compiler`-collapse becomes a priority).
---
## Stream A — ATOMICS (N1) · ✅ **COMPLETE** — see [PLAN-ATOMICS.md](PLAN-ATOMICS.md)
**Goal:** LLVM atomic codegen — the net-new emit primitive. Surface = `Atomic($T)`
wrapper + `Ordering` enum (locked, design §4.6). **Grounding correction: this is 100%
net-new — there is NO atomics scaffolding.** `Atomic`/`Ordering` exist nowhere in
`library/` (the only `thread.sx` hit is the word "Atomically" in a comment), and the
only "ordering" in `lower.zig:1400-1418` is **comparison** ordering (`< <= > >=`),
entirely unrelated to memory ordering — do not mistake it for groundwork. A.0 must
build the type, the IR op, inference, AND lowering from zero.
**Phases:**
- A.0 `Atomic($T)` + `Ordering` lib types + `load`/`store` → LLVM `load atomic`/`store
atomic` with orderings.
- A.1 RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`).
- A.2 `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**).
- A.3 `swap` + `fence(.ordering)`.
**Gates:** unit `emit_llvm.test.zig` (correct op + ordering emission); corpus `17xx`
single-thread (deterministic); **arch-gated x86_64 + aarch64 `.ir`** (orderings lower
differently — x86 vs LL/SC). **Out of snapshot scope, state loudly:** ordering
*semantics* under weak memory (`.ir` proves the keyword emitted, not correctness).
---
## Stream B — ASYNC RUNTIME (steps 412) · splits into `PLAN-FIBERS.md` + `PLAN-CHANNELS.md`
The colorblind, stackful, pure-sx async runtime (design §4). Compiler floor is small;
the runtime is sx lib. Likely carved as two PLANs:
### B1 — Fibers + Io + M:1 (the runtime; `PLAN-FIBERS.md`) · 🚧 **CARVED** (not started; first step B1.0a)
- B1.0 **`abi(.naked)` — make the EXISTING `.naked` ABI actually naked.** The enum
already carries `.naked` (ast.zig:142, documented "naked, no prologue/epilogue"),
but it is an **inert label today**: `type_resolver.zig:237` maps `.naked → .default`
CC and there is **zero naked-attribute emission in emit_llvm**. So B1.0 is NOT
"extend the enum" (done) — it is "emit the LLVM `naked` attr + skip prologue/epilogue
lowering for `.naked`," genuinely net-new. (Roadmap §7-step-4's "extend
`CallConv {default, c}`" is stale — CallConv was renamed ABI and already gained
`compiler`/`naked` in the compiler-API stream.) Gates the context-switch.
- B1.1 **Per-fiber `context` root + `push Context`-stack storage.** Grounding correction:
`context` is **already an implicit `*Context` parameter** (comptime_vm.zig:392,
lower.zig:257 "Implicit Context parameter machinery"), **not raw TLS** — so it already
rides the fiber stack and the design doc's "lower as swappable indirection, never raw
TLS" guards a non-problem. The **real, currently-unsized** scope is: (a) where a
freshly-spawned fiber's *root* `Context` comes from, and (b) where the `push Context`
stack frames live (if on the caller stack, fiber-local for free; if a global root,
that root must become per-fiber). **Ground the current mechanism FIRST** — B1.1's size
is unknown until then, and it may be much smaller than the prior "M" estimate.
**Prerequisite of B1.3, not a successor.**
- B1.2 **A1 — `Io` interface + `context.io` + `Future` + `cancel()` API** (protocol/
vtable threaded like `Allocator`).
- B1.3 **A2 — fiber runtime**: `abi(.naked)` context-switch asm (per-arch), bootstrap,
`mmap` stacks **with mandatory guard pages** (NOT optional — a fixed-stack fiber that
overflows without a guard corrupts adjacent fiber memory silently; §8.1.1). **sx lib,
not a compiler builtin** (design §4 A2). **First deliverable of B1.3, before the
scheduler AND before the deterministic `Io`: a standalone 2-fiber ping-pong
switch-stress harness** (scribble every callee-saved reg + a stack canary before each
suspend, deep/recursive fiber chains, verify all survive post-resume — §10.7). It
needs no scheduler and is the *only* gate that catches a one-register slip; A2 is
untestable by the deterministic-`Io` harness (which tests *scheduling*, not the
*switch*), so this harness — not B1.4 — is A2's correctness gate.
- B1.4 **A3 — `Io` impls: blocking → deterministic-sim (KEYSTONE) → event-loop**
(kqueue/epoll/io_uring). Build the deterministic `Io` *before* the event loop — it
is the test harness for *scheduling* (§10.1). (Note: the **event loop does not yet
cooperate with a platform UI run loop** — CFRunLoop/NSRunLoop/ALooper; pinning gives
thread-affinity, not run-loop integration. Tracked as an open design gap for the §6
app targets, deferred out of B1.)
- B1.5 **A5·M:1 scheduler** — validates the whole colorblind stack end-to-end.
**Gates:** the **B1.3 switch-stress harness is A2's gate** (register/canary survival,
not run/snapshot — §8.1.1, §10.7) + arch-gated run tests; deterministic-`Io`
**calibrated** against blocking `Io` (don't trust an uncalibrated oracle — §8.1.3);
corpus `18xx` under deterministic `Io` asserts a program-emitted **ordering contract**
(sequence markers), not raw interleaving, so scheduler-internal policy changes don't
churn every snapshot.
### B2 — Channels + cancellation + stdlib (`PLAN-CHANNELS.md`)
- B2.0 **N3 — channels** (`Channel($T)`; `recv → RecvResult($T)` tagged union built via
**metatype** type-fn) + fiber-aware `Mutex`/`WaitGroup` (atomic fast-path from A).
- B2.1 **A6 — cancellation** = `.canceled` in the existing `!` channel (model a); per-
fiber atomic flag (A); every `io.*` a cancellation point; structured cancel-and-join;
**masked during cleanup**. Rides ERR (`try`/`onfail`/`defer`).
- B2.2 **A4 — stdlib I/O rework** — fs/socket/process onto `context.io`.
**Gates:** `18xx` under deterministic `Io`; cancellation cleanup asserted via stdout
ordering; `RecvResult` exercises the metatype primitives.
---
## Stream C — PARALLEL SCHEDULERS (steps 1314) · `PLAN-PARALLEL.md`
- C.0 **N×(M:1)** — per-thread M:1 loops + `std/thread.sx` spawn; shared state uses A
atomics; **errno-capture discipline + `context`-fiber-local** become mandatory.
- C.1 **M:N** — work-stealing (thread-safe steal queues + migration); **pinning** API
(`pin = .main | .any | .on(thread)`). M:N is **committed, not deferred** — just last.
**Gates:** data races aren't snapshottable, but "out of corpus scope" is **not** "no
plan" — Stream C is **blocked on a concrete, named stress harness landing FIRST** (a
gating artifact carved into `PLAN-PARALLEL.md`, not a footnote):
1. **Sanitizer build** — a `zig build`-integrated TSan (and ASan) variant of the
concurrency corpus; CI runs `18xx`/parallel examples under it.
2. **Run-N driver** — each parallel example executed N times (configurable, default
≥100) with interleaving perturbation (randomized ready-queue / yield injection); any
nondeterministic divergence or sanitizer report fails the build.
3. **Coverage-bound `log()`** — the harness emits, loudly, exactly which guarantees it
does and does NOT cover (per the REJECTED-PATTERNS rule against silent gaps).
This harness is the **only** correctness story for N×(M:1)/M:N; C.0/C.1 do not start
until it exists and is calibrated. Plus the **named `context`-fiber-local + errno
migration test** (M:1 can't exercise migration — §10.7).
---
## Stream D — COMPTIME JIT / FFI (steps 1518) · `PLAN-JIT.md`
Independent of async; can move earlier if `#compiler`→`extern` / bundler cleanup is
prioritized.
- D.0 **S1 — persistent JIT executor** (long-lived ORC LLJIT + host-triple emitter +
fragment cache, plumbed into the interp). Foundational for C1/C3.
- D.1 **C1 — real comptime FFI = LLVM single ABI authority** (per-signature JIT
calling-thunks via S1 + trampoline fast-path). Adversarial **layout cases** (over-
aligned/empty structs, aarch64 small-struct split, `bool` — §8.1.6).
- D.2 **C2 — `#compiler`→`extern` collapse** (hooks → exported C symbols via C1; delete
`compiler_call`/Registry). Gate: bundler corpus byte-identical pre/post.
- D.3 **C3 — comptime asm via host-JIT** (un-bail `inline_asm`; lift→JIT→cache).
`06xx` host-arch `#run` asm + `11xx` cross-arch loud-bail diagnostic.
- (S2 only if a path hits TLS/constructors — see Stream E.)
**Gates:** S1 lifecycle + cache unit tests; C1 behavior-lock trampoline cases →
xfail/green `12xx` float/struct/aggregate returns.
---
## Stream E — HOT-RELOAD (deferred) (steps 1922) · `PLAN-HOTRELOAD.md`
Deferred; R1-vs-R2 chosen at pickup. Design constraint (not optional): runtime +
long-lived fibers stay **persistent**, only **leaf logic** reloads (can't hot-swap code
with live suspended fibers).
- E.0 **S2 — ORC C++ shim** (`MachOPlatform` + redirectable symbols). **Highest risk
(§8.1.5):** only C++ in the tree, prior spike failed on `_Thread_local`, macOS-
specific — **Linux/Windows + non-Mac TLS/ctor JIT have no named plan yet.**
- E.1 **R1 — dylib hot-reload** (only needs shipped `export`; sidesteps S2).
- E.2 **R2 — JIT-resident hot-reload** (S1 + S2; ORC indirection stubs).
- E.3 **R3 — incremental compilation** (perf enabler; coarse per-file v1 first).
**Gates (when picked up):** state-survival test; the live-suspended-fiber-into-stale-
module hazard; S2 TLS + C-constructor JIT test per host OS (the exact prior-spike case).
---
## Cross-cutting (applies across streams)
- **Testing keystone:** the deterministic-sim `Io` (B1.4) gates *scheduling* tests
(§10.1); the **B1.3 switch-stress harness gates the context-switch** (the one piece
the deterministic `Io` can't test). Both must exist + be calibrated before the async
tests they gate are trusted.
- **Top risks to watch (§8.1):** A2 context-switch correctness (B1.3 — gated by its own
stress harness, not the deterministic `Io`), minted-enum → match codegen (de-risked,
metatype stream), deterministic-`Io` oracle calibration, `context`-fiber-local/errno
(C — gated by the named stress harness), S2 (E), C1 args-buffer layout (D).
- **The compiler floor stays small, but deep — net-new pieces, grounded:** atomics
(100% net-new, no scaffolding), making `abi(.naked)` actually naked (the enum variant
exists but is inert today), per-fiber `context` root + push-stack storage (`context`
is already an implicit param, NOT TLS — so this is smaller/different than "repointable
codegen" implied), `declare`/`define`/`type_info` (metatype stream — **done**), the
S1 JIT spine. Everything else — schedulers, fibers, channels, the bundler — is sx lib.
## Carving protocol
When a stream is reached: copy this section into `current/PLAN-<STREAM>.md`, expand the
phases to xfail→green steps with file anchors (from the design doc's anchor list), add
a `CHECKPOINT-<STREAM>.md`, and write a Phase-0-scoped kickoff prompt (mirror
PLAN-METATYPE's). Update [CHECKPOINT-METATYPE.md](CHECKPOINT-METATYPE.md)/this file's status as
streams complete.

View File

@@ -0,0 +1,384 @@
# Bundled `zig` Link Backend for sx — Design Doc & Proposal
> Status: **core landed (macOS / Linux / Windows).** This is the
> design-of-record for how a distributed sx links native binaries
> hermetically. The phased plan lives in
> [../current/PLAN-DIST.md](../current/PLAN-DIST.md); keep the two in sync.
> User-facing surface is documented in `readme.md` (Cross-Compilation §).
---
## Implementation status (landed)
The core backend is implemented and verified on a macOS host:
| Target | Result | Notes |
|--------|--------|-------|
| `--target linux-musl` | static ELF | `zig cc -target x86_64-linux-musl -static` |
| `--target windows-gnu` | PE32+ | `zig cc -target x86_64-windows-gnu` |
| `--target macos` | Mach-O (runs) | `zig cc -target <arch>-macos`, no `-static` |
What shipped, and where it **refined** the original locked decisions:
- **Scope = macOS + Linux + Windows** (not Linux-first). iOS/Android/wasm keep
their specialized toolchains. (`TargetConfig.zigBackendInScope`.)
- **Auto-activation = a *bundled* zig is found** (a real distribution, or a
pinned `$SX_ZIG`). A `PATH`-only zig is the dev fallback and engages **only**
under `--self-contained` — so native dev/CI builds are never silently
rerouted, across all three OSes. This is the precise meaning of the §5.5
"zig found (B)" column: **B = bundled**. *(Refinement of "auto when zig
found": PATH-zig does not auto-engage; the musl-only auto gating considered
mid-design was dropped in favor of bundled-vs-PATH, which is OS-agnostic.)*
- **No translation table** (per the triple-scheme decision): sx triples are
passed straight to `zig cc`, and `emit_llvm` runs them through
`LLVMNormalizeTargetTriple` so vendor-less zig triples (e.g.
`x86_64-windows-gnu`) land their OS/env in LLVM's canonical positions —
otherwise "windows" sits in the vendor slot and the object silently falls
back to ELF. The one unavoidable exception is **macOS**: the object must be
emitted from Apple's `apple-darwin` triple (LLVM needs it for Mach-O), but
zig's `-target` parser rejects that scheme, so the *linker* triple alone is
the vendor-less `<arch>-macos`. One OS-specific line, not a table.
- **New shorthands:** `linux-musl`, `linux-musl-arm`, `windows-gnu` (zig
scheme). The existing `linux`/`linux-arm` shorthands were also de-vendored
(`x86_64-linux-gnu`, matching the corpus runner's own expander).
Files: `src/zig_backend.zig` (discovery), `src/target.zig`
(`selectZigLinker` / `emitZigLinkArgv` / `zigTargetTriple` / dispatch in
`link`), `src/ir/emit_llvm.zig` (triple normalization), `src/main.zig`
(`--self-contained` / `--no-self-contained` + shorthands).
Not yet done: distribution packaging (Phase 3 — vendoring `zig` into
`libexec/`), and a corpus regression test (needs the runner to thread
`--self-contained`; manual verification only so far).
The sections below are the original proposal; where they say "Linux-first" or
"follow-up" for macOS/Windows, the table above supersedes them.
---
## 0. TL;DR + feasibility
**Problem.** A distributed `sx` compiler can run on a Linux box (static-LLVM
binary + relocatable `library/`), but it cannot *finish a build*: the final
link step shells out to the host's `cc`, and relies on the host's libc + CRT
objects. No `cc`/glibc/SDK on the box → no binary. That is the gap between
"sx runs here" and "sx is a toolchain here."
**Proposal.** Bundle a pinned `zig` binary inside the sx distribution and use
`zig cc` as the link backend for `sx build`. `zig cc` brings its own lld,
CRT objects, and libc (musl or glibc) for the chosen target. Default Linux
output is **statically-linked musl**, which runs on any Linux with zero
dependencies — the property that makes Zig's own output portable.
**Feasibility: high.** The change is contained:
- The linker is selected through a single hook —
`TargetConfig.getLinker()` at `src/target.zig:194-196` — and the final
link argv is built in one place, the Unix `cc`-style branch at
`src/target.zig:524-564`.
- `zig cc` is a clang-compatible driver, so `-o` / `-L` / `-l` / extra
objects pass through that branch unchanged. The backend only has to
prepend `zig cc` and add `-target …` / `-static`.
- Exe-relative resolution (for finding the bundled zig) is already solved
for the stdlib in `src/imports.zig:204-227` and can be mirrored.
- `sx run` is JIT and never links, so it is wholly unaffected.
The cost is a ~5060 MB vendored `zig` (binary + its `lib/`) in the
distribution, and version-pinning discipline.
---
## 1. Motivation & background
### 1.1 Current state
| Concern | Today | File |
|---------|-------|------|
| Compiler binary | Self-containable via `-Dstatic-llvm` (no system LLVM) | `build.zig:9-10,156-162` |
| Stdlib | Relocatable, found relative to the exe | `src/imports.zig:204-227` |
| **Linking** | **Shells to system `cc`** | `src/target.zig:524-564` |
| **libc / CRT** | **Provided by the host `cc` driver implicitly** | (no `-lc`/crt passed) |
So two of three legs of a portable toolchain already stand. The third — the
linker and the libc/CRT it pulls in — is the host dependency this design
removes.
### 1.2 Why this matters for distribution
The goal is to hand someone a tarball and have `sx build app.sx` produce a
working binary on a stock Linux machine — a fresh container, a minimal CI
image, a box without `build-essential`. Today that fails at the link step.
Zig solved exactly this problem for its own users; since sx is *built with*
Zig, the cleanest fix is to stand on Zig's hermetic toolchain rather than
re-implement it.
---
## 2. Goals & non-goals
### Goals
- `sx build` produces a native Linux binary with **no host `cc`/ld/libc/SDK**.
- Default Linux output is **portable** (static musl): runs on any Linux.
- **Zero-config in the common case**: a bundled or PATH `zig` is detected and
used automatically; the operator sets nothing.
- A fully-specified, documented configuration surface (this document) for the
cases that *do* need tuning.
- No regression for existing users: system `cc` remains a fallback, and any
explicit `--linker` still wins.
### Non-goals (this iteration)
- Reimplementing lld in-process or building libc from source (see §7 —
Zig already does both; we reuse it).
- First-class Windows/macOS cross-compilation (nearly free as a follow-up,
but unverified — §11).
- Routing C-import compilation (`src/c_import.zig`, which also shells `cc`)
through the backend.
- Glibc-floor version pinning (`…-gnu.2.28`); exposed only if needed.
---
## 3. How Zig achieves hermetic builds (the model we're borrowing)
Zig's turnkey cross-compilation rests on bundling the two things sx borrows
from the host:
1. **In-process lld.** Zig embeds LLVM's lld (ELF/COFF/Mach-O/wasm) and links
without spawning an external linker.
2. **libc as data.** Zig ships musl *source* (builds `libc.a` + `crt*.o` on
demand, cached → static, no dynamic linker → portable output) and glibc
stubs generated from `.abilist` per version. For Windows it ships mingw
`.def` files and synthesizes import libraries.
`zig cc` exposes all of this behind a clang-compatible driver: `zig cc
-target x86_64-linux-musl -static foo.o -o foo` yields a portable binary on
any host, with nothing installed. **This design consumes that driver rather
than rebuilding its internals** — the whole second column above arrives for
free by vendoring the `zig` binary.
---
## 4. Design overview
`sx build` gains a **link backend** abstraction with two implementations:
- `system_cc` — today's behavior (shell `cc`, host libc).
- `bundled_zig` — shell `<zig> cc -target <triple> [-static] …`.
Selection is automatic (§5.5): if a usable `zig` is discovered and the user
gave no explicit `--linker`, `bundled_zig` is used; otherwise `system_cc`.
The backend plugs into the existing Unix link branch — it contributes the
leading `zig cc` tokens and the `-target`/`-static` flags; the rest of the
argv assembly is unchanged because `zig cc` is clang-compatible.
One supporting change: when `bundled_zig` is active, the triple handed to
LLVM in `src/ir/emit_llvm.zig` is aligned to the link target (`x86_64-linux`)
so the emitted object links cleanly against the selected musl CRT.
---
## 5. Detailed design (the configuration surface)
### 5.1 zig discovery — resolution order
`discoverZig()` (new `src/zig_backend.zig`) returns the first hit:
1. `$SX_ZIG` — explicit override.
2. `<exe_dir>/../libexec/zig/zig`**install layout** (§6).
3. `<exe_dir>/../../zig-bundle/zig`**dev vendored layout** (§6).
4. `zig` on `PATH`**dev fallback** (the only one active today).
`<exe_dir>` is resolved exactly as `src/imports.zig` resolves the stdlib.
If none resolve, behavior depends on activation (§5.5): auto-mode silently
falls back to `system_cc`; `--self-contained` errors.
### 5.2 Environment variables
| Var | Effect | Default |
|-----|--------|---------|
| `SX_ZIG` | Absolute path to the `zig` used as the link backend. Highest-priority discovery source. | unset |
| `ZIG_LIB_DIR` | Path to the bundled zig's `lib/`. Needed **only** if `zig` was relocated away from its `lib/`. In the supported layout (§6) they ship together and zig self-locates — leave unset. | unset |
| `SX_DEBUG_ZIG` | Trace discovery: each candidate path and the chosen one (or "none → cc"). Mirrors `SX_DEBUG_STDLIB`. | unset |
| `SX_DEBUG_LINK` | **Existing.** Prints the full link argv — shows the exact `zig cc …` invocation. | unset |
| `SX_STDLIB_PATH` | **Existing.** Stdlib override; unrelated to linking but noted because a full distribution sets neither and relies on exe-relative discovery for both. | unset |
### 5.3 CLI flags (`sx build`)
| Flag | Effect |
|------|--------|
| `--self-contained` | Force `bundled_zig` ON. If no usable zig is found, **error** — do not silently fall back. |
| `--no-self-contained` | Force `system_cc`. |
| `--linker <cmd>` | **Existing.** Explicit linker; supplying it **disables** auto-activation (user's choice wins). To pin a specific zig, prefer `SX_ZIG` + `--self-contained`. |
| `--target <triple\|shorthand>` | **Existing.** Selects target + ABI (§5.4). With `bundled_zig` active and target unspecified on a Linux host → `x86_64-linux-musl` static. |
| `--sysroot <path>` | **Existing.** Forwarded to the linker; rarely needed with `bundled_zig` (zig brings its own sysroot). |
### 5.4 Target → ABI mapping
The default (no `--target`) deliberately differs from the legacy `linux`
shorthand, because portable static output is the entire point.
| `sx` invocation | zig `-target` | Link mode | Portable? |
|-----------------|---------------|-----------|-----------|
| *(no `--target`, Linux host)* | `x86_64-linux-musl` | `-static` | ✅ any Linux |
| `--target linux-musl` *(new)* | `x86_64-linux-musl` | `-static` | ✅ |
| `--target linux` / `linux-x86` | `x86_64-linux-gnu` | dynamic | ❌ host glibc, versioned |
| `--target linux-arm` | `aarch64-linux-musl` | `-static` | ✅ |
| `--target windows` | `x86_64-windows-gnu` | per zig | follow-up (§11) |
| `--target macos` / `macos-arm` | `aarch64-macos` | per zig | follow-up (§11) |
- A **new** `linux-musl` shorthand is added; the existing `linux` shorthand
keeps its current gnu/dynamic meaning for back-compat.
- The LLVM emit triple is aligned to the link target so the `.o` links
cleanly against the selected libc/CRT (§4).
### 5.5 Activation truth table
`B` = a usable zig was discovered (§5.1). Subcommand = `sx build`.
| `--self-contained` | `--no-self-contained` | `--linker` | zig found (B) | Result |
|:---:|:---:|:---:|:---:|--------|
| — | — | no | yes | **bundled_zig** (auto) |
| — | — | no | no | system `cc` (silent fallback) |
| — | — | yes | * | user's `--linker` |
| yes | — | * | yes | **bundled_zig** (forced) |
| yes | — | * | no | **error**: `--self-contained` but no zig |
| — | yes | * | * | system `cc` (forced off) |
- `--self-contained` + `--linker` together: backend choice goes to
`--self-contained`; treat the literal combination as a usage error
(document, don't guess).
- `sx run` / `sx ir` / `sx asm` never link → backend not consulted.
### 5.6 Emit-triple alignment
`src/ir/emit_llvm.zig` (`LLVMSetTarget`, ~L246-284) currently uses the host
default triple when `--target` is unspecified (on Linux,
`x86_64-unknown-linux-gnu`). When `bundled_zig` is active, set the module
triple to match the link target (`x86_64-linux`) so codegen and the musl CRT
agree. Pure codegen objects are ABI-compatible across gnu/musl; aligning the
triple removes the edge-case risk (TLS model, stack protector) up front.
---
## 6. Distribution layout (packaging)
A relocatable tree; everything resolves relative to `bin/sx`, so the whole
directory moves/untars anywhere with no env vars set:
```
sx-<os>-<arch>/
├── bin/
│ └── sx # built -Dstatic-llvm (no system LLVM dep)
├── libexec/
│ └── zig/
│ ├── zig # pinned zig binary
│ └── lib/ # zig's lib/ (musl/glibc sources, lld data, …)
└── library/ # sx stdlib (existing discovery)
└── modules/…
```
Rules:
- `zig` and its `lib/` **must** ship together under `libexec/zig/` so zig
self-locates `lib/`; splitting them forces `ZIG_LIB_DIR`.
- Pinned zig version: **0.16.0** (matches the build toolchain). Record the
exact version in the release manifest — a mismatched `zig cc` CLI is the
likeliest future breakage.
- Vendor the matching zig release per host os/arch from ziglang.org at
package time.
---
## 7. Alternatives considered
| Alternative | Why not (now) |
|-------------|---------------|
| **In-process lld + bundled musl sysroot** (sx owns the pipeline; no zig) | Requires a custom LLVM build *with* lld — the Homebrew `llvm@22` here ships none (`liblld*.a`, headers, `ld.lld` all absent) — plus a C++ lld shim and per-arch prebuilt musl. Strictly more work for the same user-visible result. The right *eventual* target if we want zero foreign binaries; tracked as a follow-up. |
| **Full Zig-style: build libc from source on demand** | Most flexible (any arch/libc version, no prebuilt blobs) but the most work; only worth it after the in-process-lld path exists. |
| **Document a hard dependency on system `cc`** | Zero engineering, but defeats the goal — the box still needs `build-essential`. Acceptable only as the current fallback, not the distribution story. |
| **Bundle just `ld.lld` + a musl sysroot (no full zig)** | Smaller than a whole zig, but we'd hand-manage crt object selection, dynamic-linker paths, and import libs — i.e. re-derive what `zig cc` already encapsulates. Bundle-size saving doesn't justify the fragility. |
Vendoring `zig` wins on effort-to-result because sx already builds with Zig:
it's a first-party dependency, not a foreign toolchain, and it unlocks
Windows/macOS targets later for nearly free.
---
## 8. Phasing
Detail in [../current/PLAN-DIST.md](../current/PLAN-DIST.md). Summary:
0. **Resolve zig**`discoverZig()` + `SX_DEBUG_ZIG`; PATH fallback only.
1. **Link backend** — generalize the linker to a driver argv; emit
`zig cc -target … -static`; align the emit triple.
2. **Auto activation** — wire the §5.5 truth table; `cc` fallback intact.
3. **Packaging**`build.zig` `dist` step assembling the §6 tree.
4. **Verify & lock**`file`/`ldd` shows "statically linked"; host/arch-gated
corpus test honoring the snapshot-integrity + FFI-cadence rules.
The minimum end-to-end proof is Phases 0+1 against PATH zig.
---
## 9. Open decisions
**Locked:**
- Default Linux ABI = **static musl** (portable output).
- Activation = **auto** when a usable zig is found and no `--linker`.
- Dev uses **PATH zig**; vendoring deferred to Phase 3.
**Still open:**
- Exact spelling of the force flags (`--self-contained` vs e.g.
`--bundled-linker`); name chosen here pending review.
- Whether auto-mode should *warn* on silent `cc` fallback or stay quiet
(leaning quiet, with `SX_DEBUG_ZIG` for diagnosis).
- Whether to gate the Phase-4 corpus test behind a `.build` `target`
sidecar or keep it manual until a Linux CI runner exists.
---
## 10. Risks
- **Bundle size** ≈ 5060 MB (zig + `lib/`). Acceptable for a toolchain;
call it out in release notes.
- **zig CLI drift** across versions — pin hard, record in the manifest;
the most likely future breakage.
- **gnu vs musl ABI** for the emitted object — covered by the emit-triple
alignment (§5.6); TLS/stack-protector are the only realistic friction.
- **Operator confusion**: default-no-target (musl) diverging from the
`linux` shorthand (gnu). Mitigated by the new `linux-musl` shorthand and
explicit documentation (§5.4).
---
## 11. Out of scope / follow-ups
- **Windows / macOS targets** via the same `zig cc -target`: nearly free
after the Linux path, but Apple-SDK and Windows specifics need their own
verification — not documented as supported until tested.
- **`src/c_import.zig`** still shells system `cc` for C imports in JIT mode;
route through the backend later.
- **In-process lld** (alternative in §7) as the eventual zero-foreign-binary
endgame.
---
## Appendix — quick recipes (once implemented)
```sh
# Portable static Linux binary (default when a bundled zig is present):
sx build app.sx -o app
file app # → "ELF 64-bit … statically linked"
# Force the backend; fail loudly if no zig is bundled:
sx build app.sx --self-contained
# Use a specific zig:
SX_ZIG=/opt/zig-0.16.0/zig sx build app.sx --self-contained
# Opt out, use the system toolchain:
sx build app.sx --no-self-contained
# Dynamic glibc instead of static musl:
sx build app.sx --target linux
# Debug discovery + the exact link invocation:
SX_DEBUG_ZIG=1 SX_DEBUG_LINK=1 sx build app.sx
```

View File

@@ -0,0 +1,253 @@
# Comptime Compiler API — `#library "compiler"` + `abi(.zig) extern`
> **⚠ SUPERSEDED (2026-06-17) — direction changed. See
> [`../current/PLAN-COMPILER-VM.md`](../current/PLAN-COMPILER-VM.md).**
> The **byte-weld** approach below (sx structs whose layout is validated to mirror
> the compiler's Zig types, plus serialization / marshaling at the call boundary) is
> the **wrong direction** and is being stripped. The comptime value model
> fundamentally isn't bytes, so the weld bolts a parallel layout regime + hand-built
> byte-copies onto it. The new foundation: a **bytecode VM over flat, byte-addressable
> memory**, where comptime values ARE native bytes — so the compiler-API needs no
> weld, no validation, no marshaling (the compiler exposes its real types/functions
> and sx reads/builds them directly as memory). The goal below (unify
> `declare`/`define`/`type_info` + `#compiler` onto one mechanism, delete the bespoke
> arms) is unchanged; only the *mechanism* is. This doc is retained for history and to
> scope the Phase 0 strip — do NOT implement the weld machinery from here.
>
> **Original status:** design-of-record. Captured a unified mechanism for
> sx↔compiler binding that subsumes the metatype `declare`/`define` primitives AND the
> `#compiler` struct attribute, and exposes the compiler's own type-table API to
> comptime sx. Design locked 2026-06-17; weld mechanism pivoted same day.
## Motivation
Today the compiler↔sx boundary is **two ad-hoc mechanisms**:
- `#compiler` structs (`BuildOptions`) — sx struct whose methods are compiler hooks
(registered in `compiler_hooks.zig`). A handle to compiler state, method-bound.
- The metatype `declare`/`define`/`type_info` `#builtin`s — comptime sx reaching
into the type table through a narrow, fixed keyhole, with a *separate, translated*
`TypeInfo` data model in `meta.sx` (marshalled by hand in `interp.zig`).
Both are the SAME idea — comptime sx interacting with the compiler — implemented
twice, differently. And the metatype path carries real costs: a projected data
model that drifts from `types.zig`, hand-written marshaling, and the staging
fragility of issue 0141 (constructor bodies lowered at `scanDecls` in a half-built
world → wrong IR).
**This unifies them.** One mechanism: a named `compiler` library that exposes a
curated set of the compiler's real types (welded by layout) and functions
(host-call bridged), reachable from comptime sx. `declare`/`define`/`type_info`
become sx library code over the real API; `#compiler` is deleted; `BuildOptions`
migrates onto it.
## The mechanism
### `#library "compiler"`
```sx
compiler :: #library "compiler";
```
A named binding target that resolves NOT to a `.dylib` but to the compiler's own
internal surface (Zig types + functions). Two defining properties:
- **It IS the safety boundary.** The `compiler` library exports exactly the
curated set of types + functions the compiler chooses to expose. Anything not on
that export list is unreachable from user comptime code — the boundary is the
lib's symbol table, not a convention.
- **It is comptime-only.** The compiler isn't present at runtime, so every function
from `compiler` resolves only under the comptime interpreter; calling one at
runtime is a clean "comptime-only symbol" error, falling out of the existing
`is_comptime` boundary. (Welded *types* are still usable as plain runtime data;
only the *functions* are comptime-gated.)
### `abi(.zig)` + `extern <lib>` — the binding surface
> **Syntax decision (2026-06-17, supersedes the original `extern(.zig) <lib>`
> single-qualifier form).** The ABI/layout selector and the linkage keyword are
> two orthogonal things, so they are two annotations, not one fused qualifier:
> - `abi(.x)` — the ABI / calling-convention annotation, in the postfix slot
> **before** `extern`/`export`. It is the unified replacement for the old
> `callconv(...)` (which is removed): `ABI = { default, c, zig, pure }` —
> `.c` (C ABI / cdecl), `.zig` (Zig-layout weld → the `compiler` library),
> `.naked` (naked asm). `.default` = unannotated (ordinary sx convention).
> - `extern <lib>` — the linkage keyword + binding source (the named library).
`abi(...)` sits where `callconv(...)` went (after the return type for fns); the
`extern`/`export` keyword and the library handle follow. For welded types, the
same `abi(.zig)` + `extern <lib>` pair sits after `struct`:
```sx
// functions:
text_of :: (id: StringId) -> string abi(.zig) extern compiler;
intern :: (s: string) -> StringId abi(.zig) extern compiler;
register_type :: (info: StructInfo) -> Type abi(.zig) extern compiler;
find_type :: (name: StringId) -> ?Type abi(.zig) extern compiler;
// types (layout-welded to the lib's real Zig type):
Field :: struct abi(.zig) extern compiler { name: StringId; ty: Type; };
StructInfo :: struct abi(.zig) extern compiler {
name: StringId; fields: []Field; is_protocol: bool; nominal_id: u32;
};
```
`abi(.zig)` = "Zig ABI / Zig layout"; `extern compiler` = the linkage + binding
source.
### Layout welding — why it's exact, not brittle
The sx compiler is itself a Zig program; `types.zig` is part of it. So at
**compiler-build time** the real record's layout is available via
`@offsetOf` / `@sizeOf` / `@alignOf`. An `abi(.zig) extern compiler` struct is laid out
to the bound Zig type's EXACT offsets (queried, not guessed), and the compiler
ASSERTS the sx declaration matches the Zig type byte-for-byte (a mismatch is a
build error — the sx side is a header checked against the implementation). Because
the same compiler builds both, they're guaranteed identical, and a `types.zig`
change re-bakes the offsets on the next build — both sides move together.
> **Implementation note (how it's exact, concretely).** No layout-override engine
> is needed. The sx header DECLARES its fields in the compiler type's **memory
> order** (Zig may reorder a struct from source order). The compiler REFLECTS the
> bound Zig type — field names from `@typeInfo`, offsets from `@offsetOf`, size
> from `@sizeOf`, nothing hand-maintained — and VALIDATES the header matches that
> memory order, with loud diagnostics on drift (*field not found*, *wrong field
> order* + the expected order, *type/layout size mismatch*). On pass the sx
> struct's NATURAL layout already equals the Zig layout, so it is an ordinary
> struct — no reorder, no padding tricks, no index/remap tables, no special LLVM
> path — and `@ptrCast`ing it to the compiler's own type and dereferencing is
> byte-identical. When `types.zig` shifts, the header stops matching and the
> developer gets a specific message to fix it.
This is what C-ABI `extern` can't do: it copies Zig's REAL layout, so Zig slices
(`{ptr,len}`), field reordering, and `union(enum)` tag placement all "just work" —
no slice→ptr+len surgery on `types.zig`, no version fragility.
### Host-call bridge (functions)
`compiler` functions dispatch, under the comptime interp, to the registered
internal Zig function — the generalization of the path that already exists
(`host_ffi.zig` resolves comptime `extern "c"` via dlsym; `compiler_hooks.zig`
registers `#compiler` method hooks). The `compiler` lib's registry maps each
exported sx name → its Zig function + welded signature.
## The exposed surface (curated)
Types (welded): `StringId` (u32 handle), `Type` (≡ `TypeId`, u32), `Field`,
`StructInfo`, `EnumInfo`, `TaggedUnionInfo`, `TupleInfo`, and a kind-tagged
`TypeInfo` view (see Risks — the `union(enum)` is the one harder shape).
Functions (comptime-only): `intern(string)->StringId`, `text_of(StringId)->string`,
`find_type(StringId)->?Type`, guarded mutators
`register_struct/register_enum/register_tuple(info)->Type`, and the reflection
readers (`type_of`, field/variant iteration) over the welded records.
`declare`/`define`/`type_info` collapse into thin sx over `register_*`/`find_type`
— or disappear. The bespoke interp arms (`.declare`/`.define`/`.type_info`,
`defineEnum`/`defineStruct`/`defineTuple`/`reflectTypeInfo`) are deleted.
## What it buys (and the one honest limit)
Dissolves: the bespoke `declare`/`define` surface, the projected `TypeInfo` model,
the hand-marshaling, the `#compiler` duplication, and the **0141 class of bugs**
registration becomes a direct, guarded API call, not "evaluate an sx stdlib body
(List/append) at `scanDecls`," so there's no body to mis-lower at a half-built
stage.
Does NOT repeal: the **ordering law** — a type's layout must exist before code
that uses it is lowered. That's inherent to the compiler, not machinery. The win
is that it stops leaking as "weird exposed stages" and becomes an encapsulated
contract inside the compiler API (the API decides how a registration slots in),
instead of the user threading `declare`→forward-slot→`define`→eval-timing by hand.
## Safety boundary
- Only the `compiler` export list is reachable — no raw `*TypeTable`.
- Mutators are **guarded** (`register_*` validate: dup field/variant names, kind
changes, well-formedness) — the same checks `define` does today, now at the API.
- Comptime-only enforcement on functions; runtime use is a clean error.
- Mirrors Zig's own discipline: comptime builds types through sanctioned doors
(`@Type`), it doesn't let user code scribble on the compiler's tables.
## BuildOptions migration
`BuildOptions :: struct #compiler { ... }` + `build_options() #compiler`
`abi(.zig) extern compiler`: the setter/getter hook-methods become `abi(.zig)
extern compiler` functions (or methods on a welded/handle `BuildOptions`), backed by the
same `BuildConfig` state. The `compiler_hooks.zig` registry becomes the `compiler`
lib's function/type registry. Net: the build DSL and the metatype API ride one
mechanism.
## `#compiler` removal
After both consumers are migrated, delete the `#compiler` attribute and its
special paths: lexer/parser token + sema handling (`src/lexer.zig`, `src/parser.zig`,
`src/sema.zig`, `src/token.zig`, `src/ast.zig`), and the `#compiler`-specific
registration in `compiler_hooks.zig` (the registry stays, re-homed under `compiler`).
sx footprint is tiny (2 lines in `library/modules/build.sx`).
## Code anchors (confirmed 2026-06-17)
Foundation that ALREADY exists:
- `#library "name"` lexes (`hash_library`, `src/lexer.zig:91`) and parses into a
`library_decl { lib_name, name }` AST node (`src/parser.zig:210`). So
`compiler :: #library "compiler";` works today (used for FFI libs like raylib).
- `extern` / `export` are keywords (`src/token.zig:46`, `kw_extern`/`kw_export`).
New work for Phase 1:
- **Lexer/parser**: the `abi(.zig)` annotation (a new `abi` keyword replacing
`callconv`; `ABI = { default, c, zig, pure }`) in the slot before `extern`,
followed by the `<lib>` handle — `… abi(.zig) extern <lib>` postfix on FN decls
(after the return type, before `extern`) and STRUCT decls (beside
`struct #compiler`). **DONE (parse-only)**`parseOptionalAbi`
(`src/parser.zig`) wired on fn decls AND struct decls, `ast.ABI`, parser unit
tests; the `callconv``abi` rename migrated 52 sx files + the compiler's
CC-mismatch diagnostic.
- **AST**: the `abi: ABI` field lives on `FnDecl` / `Lambda` / `FunctionTypeExpr`
(carries `.zig` for a welded fn); `StructDecl` gained `abi: ABI` +
`extern_lib: ?[]const u8`. **DONE.**
- **Binding registry**: re-home / generalize `src/ir/compiler_hooks.zig` (today's
`#compiler` registry) into the `compiler` lib's type+function registry, keyed by
exported sx name → Zig type (`@offsetOf` layout) / Zig fn (host-call).
- **Layout + emit**: sx struct layout (`src/ir/types.zig` / lowering) honors the
bound type's offsets; LLVM emission (`src/backend/llvm/types.zig`) hits them.
- **Host-call bridge**: extend the comptime path (`src/ir/host_ffi.zig` +
`interp.zig`) to dispatch `compiler` functions to their registered Zig fns,
comptime-only.
## Build order (each phase keeps `zig build test` green)
1. **`abi(.zig) extern <lib>` + `#library` foundation** — parse the postfix
annotation (the `#library` decl already exists); a binding registry (sx name →
Zig type/fn); the layout engine honoring the bound type's `@offsetOf` offsets +
LLVM emission that hits them; **build-time layout-equality assertion**. Prove
with `Field` (two u32s). First testable sub-step **DONE**: `abi(.zig) extern
<lib>` PARSES on a fn decl (parser unit test), AST carries the binding (`abi ==
.zig`, `extern_lib`) — no semantics yet.
2. **Weld `StructInfo`** + `StringId` accessors (`intern`/`text_of`) over the
host-call bridge.
3. **Re-express `type_info`/`define` (struct)** as sx over `register_struct`/
`find_type`; migrate `examples/0622`; delete the struct interp arms; suite green.
4. **Widen to enum/tuple** — weld `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
(optional fields → sentinels: `backing_type` `.unresolved`, `explicit_values`
len-0); migrate `examples/0619`/`0623`; delete the enum/tuple interp arms.
5. **Migrate `BuildOptions`** to `abi(.zig) extern compiler`.
6. **Delete `#compiler`**; suite green.
## Risks / open questions
- **`union(enum)` welding.** `TypeInfo` is a Zig tagged union; mirroring its tag
placement is the one shape harder than plain structs. Start with a `kind`-tagged
*view* (weld the payload structs, drive the discriminant via a `kind` accessor),
defer full-union welding. `type_info`/`define` mostly traffic in the payload
records anyway.
- **Optional fields in welded records** (`?[]const i64`, `?TypeId`) — represent via
sentinels on the sx side, or expose through accessor functions rather than raw
fields.
- **LLVM layout emission** for arbitrary external offsets (padding / byte-offset
GEPs) is the meatiest part of phase 1.
- **Mutation safety** — the guarded-mutator surface must cover every invariant the
type table relies on (interning, nominal ids, forward slots).
- **`@offsetOf` binding for nested/parameterized types** — the registry must map
each exported sx type to a concrete Zig type; generic Zig types need a concrete
instantiation to bind.

View File

@@ -0,0 +1,638 @@
# Execution-Model Evolution — Roadmap (comptime JIT · async · concurrency · hot-reload)
> Status: **exploratory design-of-record.** Captures the forward plan for sx's
> execution model across five interlocking threads. Not yet an active
> `PLAN-*`/`CHECKPOINT-*` stream — this is the shared design the streams would be
> carved from. Cross-platform shipping (the bundled-zig backend + the sx bundler)
> is **already landed**; see [bundled-zig-link-backend-design.md](bundled-zig-link-backend-design.md)
> and [../current/PLAN-DIST.md](../current/PLAN-DIST.md).
---
## 0. The thesis
sx's compiler stays small by pushing capability into **library sx + three general
primitives** (`inline asm`, `extern`/`export`, `atomics`) rather than baking
features into codegen. Concretely:
- **Async is a library, not a language feature** — colorblind, stackful fibers
behind an `Io` interface (Zig-inspired). No function coloring, no
async→state-machine transform. The implementation is pure sx down to a per-arch
inline-asm context switch.
- **Comptime gains a JIT escape hatch** — the interpreter stays the default
(debuggable, portable), but drops to a host-JIT for the one thing it can't
walk (inline asm) and, later, for whole fragments (the bundler).
- **One shared substrate** — a persistent ORC LLJIT + host-target emitter — serves
comptime-asm, the bundler, and JIT-resident hot-reload.
The honest trade is **small *surface*, but each primitive is *deep*** — not "small
compiler." The net-new **compiler** obligations this plan adds (all verified absent
today): **atomics lowering** (N1), **generic enums** `enum($T)`, **`declare` +
`define` + `type_info` + `field_type`** (comptime type metaprogramming), **`callconv(.naked)`**,
**repointable-`context` codegen** (+ per-fiber stack-limit), the **S1 persistent JIT
spine**, **C1 thunk synthesis**, **comptime-asm lifting** (C3), and (later) the **S2
ORC C++ shim**. Async itself is genuinely a library; the *enabling primitives* are a
major codegen/runtime investment. Already landed: `inline asm` (in flight),
`extern`/`export`, the `!`/`try`/`catch`/`onfail`/`raise` ERR stream, value-level
reflection, the `sx run` ORC LLJIT, and the host-FFI trampolines.
---
## 1. The spine (shared substrate)
| ID | Piece | What | Size |
|----|-------|------|------|
| **S1** | Persistent JIT executor | A long-lived ORC LLJIT + a host-triple `LLVMEmitter` + a compiled-fragment cache, plumbed into the interpreter. Today the LLJIT exists only for `sx run`'s `main` ([target.zig:319](../src/target.zig#L319)); the emitter carries one target machine ([emit_llvm.zig:274](../src/ir/emit_llvm.zig#L274)). | L |
| **S2** | ORC C++ shim | `MachOPlatform::Create` + redirectable/lazy-reexport symbols. The bare `LLVMOrcCreateLLJIT` can't do thread-locals, C constructors, or symbol redefinition — the wall the C-with-sx JIT spike hit (`_Thread_local` SIGABRT; `errors-*` examples crashed). Required by any non-trivial JIT or symbol repoint. | M |
S1/S2 are the spine: built once, consumed by **C1** (the FFI thunks — the main
near-term consumer), **C3**, and (later) **R2**. S1 alone suffices for C1/C3 (bare
calling/asm thunks — no TLS/ctors); S2 is only needed for R2 and JIT-ing C-with-sx.
---
## 2. Comptime / build layer
| ID | Piece | Unblocks | Depends | Size |
|----|-------|----------|---------|------|
| **C1** | **Real comptime FFI — JIT calling-thunks (LLVM = single ABI authority).** Trivial calls (scalar/ptr/string args, single-reg return) keep the existing `host_ffi.zig` trampoline fast-path; everything else (floats, structs-by-value, aggregate returns, >8 args, varargs) synthesizes a per-signature thunk, JIT-compiles it via **S1**, and calls it with an args buffer the interpreter fills by known layout (`type_info`). **LLVM emits the ABI-correct call — the same lowering as runtime codegen — so comptime and runtime FFI share ONE ABI implementation.** Rejected: libffi (foreign 2nd ABI impl), hand-rolled sx+asm (3rd impl + drift risk + needs C3 to run its own asm leaf anyway). | struct/string/slice/float signatures at comptime; full C interop in `#run`; lifts the bundler's API straightjacket; unifies comptime+runtime FFI | S1 (fast-path: none) | L |
| **C2** | **`#compiler``extern` collapse** — BuildOptions hooks become real exported C symbols resolved through C1; `*BuildConfig` threaded via global/handle; delete `.compiler_expr`/`compiler_call`/Registry. | one FFI mechanism, not two | C1 (`extern`/`export` already shipped) | M |
| **C3** | **Comptime asm via host-JIT** — stop bailing on `inline_asm` ([interp.zig:1019](../src/ir/interp.zig#L1019)); lift the block (operand model at [inst.zig:354](../src/ir/inst.zig#L354): inputs/`out_value`/`out_place`/`out_ty`/clobbers) to a host-arch thunk via `LLVMGetInlineAsm`, JIT, call through C1, cache by template+sig. | running asm-containing code at comptime | S1, C1 (+S2 non-trivial) | M |
| **C4** *(DROPPED)* | **JIT-the-bundler****not built** (Decision 6). Interp+C1 is the shipping bundler (I/O-bound, so native speed is moot; C1 closes the only capability gap). Remains an always-available S1 optimization if profiling ever shows the bundler's *own logic* is a hotspot. | — | — | — |
**Residue:** cross-arch comptime asm (C3) can't run on the host — narrows the bail
to the cross-compile case; needs a sharp diagnostic ("asm targets `<arch>`, host
is `<host>`").
---
## 3. Concurrency primitives (atomics + threads)
> **Why this is its own section:** we are doing **multiple OS threads**, so the
> async runtime and any lock-free structure need real atomics. OS threads already
> exist; atomics do not.
| ID | Piece | State | Size |
|----|-------|-------|------|
| **N1** | **Atomics — NET-NEW compiler feature.** Atomic load/store/RMW (`add/sub/and/or/xor/swap` + `fetch_min`/`fetch_max`; no `nand`), `compare_exchange`/`_weak` (→ `?T`, **null = success**), and fences, with orderings (relaxed/acquire/release/acq_rel/seq_cst). LLVM provides all — an **emit** feature, not a runtime library. **Surface LOCKED = `Atomic($T)` wrapper + `Ordering` enum** (not `@atomic_*``@` is address-of in sx). | **fully net-new** — zero LLVM `atomicrmw`/`cmpxchg`/`fence` emission **and no atomics scaffolding**: `Atomic`/`Ordering` exist nowhere in `library/`, and the only "ordering" in `lower.zig:1400` is *comparison* ordering (`< <= >=`), unrelated to memory ordering | M |
| **N2** | **OS threads + pthread Mutex/Cond + worker Pool** | **landed** — [std/thread.sx](../library/modules/std/thread.sx) (`pthread_create`/`join`/`detach`, in-place `Mutex`/`Cond`, bounded `Pool`). NOTE: pthread mutex **blocks the OS thread** — it is *not* fiber-aware (it would park every fiber on that thread); fiber-aware sync is N3, built on N1. | — |
| **N3** | **Fiber-aware sync** — mutex / channel / waitgroup that **suspend the fiber**, not the OS thread. Hybrid: atomic fast-path (N1) + fiber-suspend slow-path (A2/A5). Distinct from the pthread primitives in N2. | new library | M |
**Compiler obligation for N1:** the emit must map sx orderings to LLVM's and **not
reorder across atomics/fences**. Comptime is single-threaded, so the interpreter
can treat atomic ops as ordinary ops (seq_cst is trivially satisfied with one
thread) — no interp atomics machinery needed.
**N1 is a prerequisite for M:N scheduling (A5) and N3, and is broadly useful**
(lock-free queues, refcounts, the allocator). It is the load-bearing new primitive
this revision adds.
---
## 4. Async — colorblind, stackful, pure-sx
**Commitment:** no function coloring, no async→state-machine transform. Async is a
capability carried in `context` (like `context.allocator`), not a property of a
function's signature. A function does I/O through `context.io`; whether the call
suspends is decided by the `Io` *implementation*, transparently.
| ID | Piece | Notes | Size |
|----|-------|-------|------|
| **A1** | **`Io` interface + `context.io`** — a protocol/vtable threaded like `Allocator`. `io.async(fn,args) → Future`, `future.await`, cancellation. | leverages protocols + context | M |
| **A2** | **Stackful coroutine runtime — in sx lib, NOT a compiler builtin.** The context-switch is a `callconv(.naked)` sx fn with an inline-asm body (save callee-saved + SP/LR into `*from`, load from `*to`, `ret`); fiber bootstrap + stack alloc (`mmap`+guard via `extern`) also sx. The **compiler's** job is only (a) the general primitives — inline asm, `abi(.naked)`, atomics — and (b) **fiber-safe codegen**: `context` is **already an implicit `*Context` param** (not TLS — see §7 step 5), so the switch repoints it for free by swapping the per-fiber root; the open work is the per-fiber root + push-stack storage, and stack-limit guards (**mandatory, not optional** — fixed mmap stacks without a guard corrupt neighbors silently) reading from a swappable per-fiber location. Most arch-delicate sx in the tree (must match the platform callee-saved set + the compiler ABI), but it's inspectable sx, not a black box. | per-arch, arch-gated; co-validate vs codegen | M |
| **A3** | **Event-loop `Io` impls** — kqueue / epoll / io_uring drive readiness, then the (now-ready) syscall via C1. Plus a trivial **blocking `Io`**. | pure sx around syscall `extern`s | L |
| **A4** | **Stdlib I/O rework** — fs/socket/process take/use `context.io` instead of raw blocking syscalls, so existing calls participate in async. | mirrors the allocator-threading rule | M |
| **A5** | **Schedulers — M:1 → N×(M:1) → M:N, all sx std-lib `Io` vtables (committed; M:N last, not deferred).** M:1 first (minimal vehicle to validate the colorblind stack; covers I/O-bound). N×(M:1) = first parallel step (per-thread M:1 loops + `std/thread.sx` spawn; shared state uses N1 atomics — expected under parallelism, not a wart). M:N work-stealing last (most machinery: thread-safe steal queues + migration + errno/TLS discipline). All over N1 atomics + the A2 asm context-switch + `extern` syscalls. **pinning** API for thread-affine work (UI main thread, GL context). | see §4.3 | M (M:1) / M (N×M:1) / L (M:N) |
### 4.1 How control enters sx (the colorblind model)
- **sx→sx is ordinary.** The whole call chain lives on the fiber stack; a suspend
at a leaf `io.*` freezes the native stack verbatim. No frame knows it suspended.
**Zero special handling at call boundaries** — that's the point.
- **Three inbound boundaries** where the runtime enters sx:
1. **Task entry** (`io.async(fn)`) — a trampoline starts `fn` on a fresh fiber
stack via the normal calling convention.
2. **Resumption** — a context-switch (asm), *not* a call; sx continues mid-stack.
3. **C callback → sx** — must be `export`/`callconv(.c)`; runs on the event-loop
stack (not a fiber) so it **cannot itself suspend** — it may resume/enqueue a
fiber or run a non-suspending sx fn to completion (leaf-only).
### 4.2 `context` is fiber-local (the key obligation)
`context.io`/`context.allocator`/the `push Context` stack are dynamically scoped.
Fibers time-share OS threads (and **migrate** under M:N), so `context` must travel
**with the fiber** — saved/restored on every context-switch — **never a raw TLS
read.** A spawned task snapshots the spawner's context, then evolves its own
`push Context` stack. This is the CLAUDE.md "capture your owning allocator" rule one
level up: ambient state that outlives a suspension point must be carried by the
fiber.
### 4.3 Threads & the two hazard classes (why atomics)
| Model | Parallelism | Migration | Hazards |
|-------|-------------|-----------|---------|
| **M:1** (1 OS thread) | none | none | cooperative, race-free — simplest |
| **N×(M:1)** (per-thread schedulers, no migration) | yes | none | **data races** on shared state → atomics/locks |
| **M:N** (work-stealing) | yes | yes | data races **+** TLS-migration hazards |
- **Parallelism hazard** (any N>1): shared mutable state races → needs **N1
atomics** + N3 fiber-aware sync. The M:1 "no locks" simplicity is gone.
- **Migration hazard** (M:N only): a fiber that moves threads across a suspend
reads the *wrong* thread's TLS. **`errno` must be captured immediately** after
each syscall; **`context` must be fiber-local** (§4.2) — non-negotiable under M:N.
- **Pinning** (`io.pinToThread()`): some work must stay put — the **UI main
thread** (UIKit/macOS/Android — directly the app targets in §6), OpenGL
current-context, TLS-using FFI. M:N needs a "don't migrate / main-thread-only"
fiber attribute (Go's `LockOSThread`).
### 4.4 Pure-sx boundary
Everything is sx except the irreducible FFI floor: the **asm context-switch**
(per-arch, in `.sx`), **syscall `extern`s** (kernel-implemented, like any libc
binding), and **raw stack memory** (`mmap`). The schedulers, event loops, futures,
cancellation, and sync primitives are ordinary sx. Payoff: **swappable `Io`
vtables** — blocking, io_uring, kqueue, a **mock `Io`** for tests, a
**deterministic-simulation `Io`** (fake clock, scripted readiness) for reproducible
concurrency tests — all libraries.
### 4.5 Comptime async = blocking `Io`
At comptime install the **blocking `Io`**: `io.*` just blocks; no fibers, no
scheduler, no suspend. Same source, different vtable. The interpreter never needs
suspend/resume, and the FFI (C1) needs no async awareness. This is *why* the
colorblind model resolves comptime async for free.
### 4.6 Syntax surface (grounded against the grammar)
All of the concurrency/atomics surface lands on **existing** sx grammar — `enum`
tagged unions + `if x == { case … }` match ([specs.md:364,408](../specs.md#L408)),
first-class **tuples** with named fields ([specs.md:815-852](../specs.md#L815)),
`=>` closures, `struct($T)` generics, `callconv(...)`, and the ERR keywords
(`try`/`catch`/`onfail`/`raise`/`error`). `race`/`async`/`await`/`atomic` are **not
reserved words** ([specs.md:168](../specs.md#L168)), so they stay library
types/methods — no keyword additions. One genuinely-new compiler capability is
required (see end).
**Atomics (N1) — generic wrapper type.**
```sx
Ordering :: enum { relaxed; acquire; release; acq_rel; seq_cst; }
Atomic :: ($T: Type) -> Type #builtin; // atomicity carried by the type
counter : Atomic(i64) = .init(0);
counter.store(0, .relaxed);
n := counter.load(.acquire);
prev := counter.fetch_add(1, .seq_cst); // + fetch_sub/and/or/xor (min/max: open)
old := counter.swap(42, .acq_rel);
got := counter.compare_exchange(old, new, .acq_rel, .acquire); // strong → ?T (null = success)
got2 := counter.compare_exchange_weak(old, new, .acq_rel, .acquire); // may fail spuriously; for retry loops
fence(.seq_cst);
```
- CAS takes **two orderings** (success, failure); failure ordering may not be
`release`/`acq_rel` nor stronger than success — enforce in the compiler.
- Weak vs strong matters on **aarch64** (LL/SC) — weak in a loop is the idiom;
both compile identically on x86.
**Channels (N3) — methods only (no `<-`); `recv` returns a tagged union (not `(v, ok)`).**
```sx
RecvResult :: enum($T: Type) { value: T; closed; } // ordinary generic enum (not the race-synthesized union)
TryResult :: enum($T: Type) { value: T; empty; closed; } // non-blocking: 3 states a bool can't express
ch := Channel(i64).make(16); // capacity; .make() unbuffered
ch.send(v);
if ch.recv() == { case .value: (v) { use(v); } case .closed: { /* drained */ } }
ch.close();
// ergonomic layer: `for ch (v) { … }` consumes until closed, hiding RecvResult
```
**Fiber-aware locks (N3) — explicit lock + `defer` (no guard sugar).**
```sx
m : Mutex;
m.lock(); defer m.unlock();
```
**Futures & spawn (A1).**
```sx
f := context.io.async(worker, arg); // Future(R)
r := f.await(); // suspends this fiber
f.cancel();
d := context.io.timeout(5000); // a Future too — raceable like any other
```
**Pinning (A5) — spawn attribute, accepts a thread handle.**
```sx
PinTarget :: enum { any; main; on: Thread; } // default = .any (may migrate)
f := context.io.async(render, pin = .main);
f := context.io.async(worker, pin = .on(some_thread));
```
**`race` (Zig model — over futures, named tuple in → synthesized tagged-union out).**
The input is a **named tuple** (positional also allowed → `.0`/`.1` tags); the
result is an anonymous tagged union whose variants mirror the tuple's labels, each
payload = that field's `Future(T)` projected to `T`. Losers are **cancelled and
joined** before `race` returns (structured).
```sx
fa := context.io.async(read_a, conn); // Future(A)
fb := context.io.async(read_b, conn); // Future(B)
winner := context.io.race((a: fa, b: fb)); // RaceResult = enum { a: A; b: B }
if winner == {
case .a: (v) { handle_a(v); } // v : A
case .b: (v) { handle_b(v); } // v : B
}
// positional form: race((fa, fb)) → tags .0 / .1
```
The Go-style handler-map and the map literal that propped it up are **dropped**
`race` over futures subsumes select, and cancellation handles the losers.
**Cancellation rides ERR.** A cancelled `io.*` **raises**; the fiber unwinds
through `defer`/`onfail` (`try`/`catch`/`raise` are real keywords). Cancellation is
**cooperative** (observed only at suspend points — every `io.*` is a cancellation
point) and **structured** (`race` joins losers' teardown before returning). No
parallel unwind path — it reuses the error channel.
**Context switch (A2).**
```sx
swap_context :: (from: *Fiber, to: *Fiber) callconv(.naked) {
asm { /* save callee-saved + SP into *from; load from *to; ret */ };
}
```
`callconv(.naked)``callconv(.c)`: **no prologue/epilogue/frame** — required
because a context switch deliberately makes SP-in ≠ SP-out (a `.c` epilogue would
restore from the wrong stack). Body is a single `asm` block; you emit your own
`ret`. Args arrive in ABI registers, read directly from asm.
**One new compiler capability (gates `race`):** *comptime tuple→tagged-union
synthesis.* Reflection today only **reads** types (`field_count`/`field_name`/
`type_of`); `RaceResult(T)` must **construct** an anonymous `enum` from a tuple's
`(label, payload-type)` pairs. Supporting pieces: a `field_type($T, i) -> Type`
reflection accessor (we have value-level `field_value` + `type_of`, but type-only
field projection is missing) and `Future(T) → T` projection (falls out of
generics). This is the generic "derive a sum from a product" — useful beyond
`race`.
---
## 5. Dev loop / hot-reload
| ID | Piece | Notes | Depends | Size |
|----|-------|-------|---------|------|
| **R1** | **Hot-reload (dylib swap)** — host owns `State`+allocator; reloadable module is a `.dylib` with a fixed `export` interface; watch→rebuild→`dlopen`→rebind→`dlclose`. State survives (host-owned). | leans on `export` (shipped); sidesteps S2; native | — | M |
| **R2** | **Hot-reload (JIT-resident)** — program runs under S1's LLJIT; reloadable calls route through ORC indirection stubs, repointed on change. Finer granularity; same spine. | | S1, S2 | L |
| **R3** | **Incremental compilation** — dependency tracking + recompile-only-changed. Perf enabler; coarse per-file v1 suffices first. | | — | L |
**Core rule:** the data that must survive a reload cannot be owned by the code that
reloads. Code/state separation — the CLAUDE.md owning-allocator discipline, one
level up.
**Residue — state migration on layout change:** body-only changes hot-swap;
layout/signature/global-type changes are **detected** (compare new vs running
`State` layout via `types.zig`) and trigger **rebuild+restart**. Migration hooks
(`on_reload(old)→new`) are a hard later item. Design against *silent* corruption.
---
## 6. Cross-platform (mostly landed) — from a macOS laptop
### 6.1 Landed
| Capability | State | Reach from a mac |
|---|---|---|
| `extern`/`export` C linkage | done (replaced `#foreign`) | all targets |
| Bundled-`zig cc` cross-link backend | Phases 02 done; packaging pending | **macOS, Linux(-musl/static), Windows(-gnu)** verified |
| sx-side bundler (`.app`/`.apk`) | done | macOS, iOS sim/device, Android |
| JIT `sx run` (ORC LLJIT) | done | host |
| Target shorthands | done | `macos[-arm]`, `linux[-musl[-arm]]`, `windows[-gnu]`, `ios[-arm]`, `ios-sim[-arm/-x86]`, `android[-arm64/-x86_64]`, `wasm` |
### 6.2 Workflows
```sh
# macOS (native): inner loop is JIT; ship is Mach-O / .app
sx run app.sx
sx build app.sx -o app
sx build app.sx --bundle MyApp.app
# Linux (cross, landed killer feature): static, zero-dep ELF
sx build app.sx --target linux-musl -o app # scp anywhere, runs
# Windows (cross, landed, MinGW path): PE32+
sx build app.sx --target windows-gnu -o app.exe # cf. example 1660 (win32)
# iOS simulator (mac-only host)
sx build app.sx --target ios-sim --bundle App.app
# iOS device — signing threaded via the build program (BuildOptions setters)
# #run { o := build_options(); o.set_bundle_id(...); o.set_codesign_identity(...);
# o.set_provisioning_profile(...); }
sx build build.sx --target ios --bundle App.app
# Android (cross + bundle): javac → d8 → aapt2 → zipalign → apksigner, then adb
sx build app.sx --target android --apk app.apk
```
### 6.3 Where the roadmap lights up cross-platform
- **C1 + C4** → the iOS/Android **bundlers** (orchestrate ~a dozen host tools at
comptime; biggest win; always host-arch so no cross-arch risk).
- **R1/R2 + A1A5** → the **inner dev loop for non-host targets**: push-a-dylib +
remote-trigger-reload over an async laptop↔device channel — a capability that
*doesn't exist today* short of full rebuild+reinstall.
- **A1/A2 colorblind `Io`** → the dev tooling is itself async, and the **same
networking code runs blocking inside the bundler** (`adb push`) and async in the
live session — no coloring.
- **Pinning (A5)** → the UI render fiber pins to the main OS thread on every app
target.
**The single hard constraint the matrix exposes:** cross builds mean target arch ≠
host arch, so **C3's residue bites** — comptime/`#run` code reaching *target-arch*
inline asm can't execute on the mac. Native macOS dev never hits it; every cross
target must gate comptime asm to host-arch (`when host_arch == …`) or get a loud
diagnostic.
---
## 7. Linear build sequence (async-first — no parallel streams)
Single ordered list; deps satisfied at every step. **Async-first** (user-chosen): the
async story needs no JIT spine (syscalls use the existing trampoline FFI; comptime
async = blocking `Io`), so the FFI/JIT cluster comes *after*. C4 is omitted (dropped —
an S1 optimization if ever profiled). Net-new compiler prereqs (per the codebase
grounding) are explicit steps, not buried.
**Foundations — compiler primitives the async story needs (all net-new):**
1. **N1 — Atomics lowering.** IR/inference scaffolding exists; add LLVM
`atomicrmw`/`cmpxchg`/`fence` emission + orderings. Surface = `Atomic($T)` wrapper.
Gates channels/N3 + parallel schedulers.
2. ~~**Generic enums** `enum($T)`~~ **DROPPED.** `RecvResult($T)`/`TryResult($T)` are
**type-fns over `declare`/`define`** (step 3), not a new `enum($T)` language
feature — and type-fns (user `($T)->Type` in type position) **already work** (e.g.
[`Make`](../examples/0208-generics-value-param-type-function.sx),
[`Complex`](../examples/0201-generics-generic-struct.sx)). A declarative `enum($T)`
surface, if ever wanted, is later *sugar* desugaring to a type-fn over the primitives.
3. **`declare`/`define` (construction) + `type_info`/`field_type` (reflection)** —
comptime metaprogramming floor. Gates `race` synthesis **and** channel
`RecvResult`/`TryResult` (all sx type-fns over `declare`/`define`; **generic-enum
syntax dropped**). **Validated against the codebase (3 reviewers): a small
extension reusing existing machinery throughout — not net-new architecture.**
Contracts:
1. **Nominal identity via type-fn memoization** — type-fns dedup by mangled
`(fn,args)` name (generic.zig) + `findByName`, so `RecvResult(i64)` is one
`TypeId` and the body runs once. (NOT structural dedup — enums are nominal via
`nominal_id`, types.zig.)
2. **Functional through codegen** — layout / construct / match+exhaustiveness /
`toLLVMType` / `type_name`+format are **all type-table-driven, zero AST
coupling**, so a backing-decl-less minted enum flows through unmodified.
3. **Validate loudly** at the single `intern`/`internNominal` choke point
(types.zig): reject dup variants / bad backing / unresolved payloads.
4. **Comptime-only, JIT-free** — a type-table op in the interp; no S1 dependency
(keeps construction, hence channels + `race`, off the JIT critical path).
5. **Reference-based self-reference**`*Self`/`[]Self` payloads via the
explicit `declare()``define(handle, …)` split (the handle predates its
body, so it can be referenced inside it); **by-value recursion rejected**
(loud, infinite size). Reuses the reserve-placeholder→complete path recursive
*source* types already use (nominal.zig, types.zig).
- **Type-minting precedents (7):** monomorphization, protocol vtables, tuples,
vector/array, ptr/slice ctors, FFI stubs, **type-fn instantiation** — all
construct `TypeInfo` programmatically + `intern()`. **Residual = plumbing, not
capability:** name minted results by the instantiation's mangled name + input
validation.
4. **`abi(.naked)`** — *correction:* `CallConv` was renamed `ABI` and **already carries
`.naked`** (ast.zig:142, "naked, no prologue/epilogue") during the compiler-API
stream — so this is NOT "extend the enum." `.naked` is an **inert label today**:
`type_resolver.zig:237` maps it to `.default` CC and emit_llvm emits **no** naked
attribute. The net-new work is making `.naked` actually emit LLVM `naked` + skip
prologue/epilogue lowering. Gates A2.
5. **Per-fiber `context` root + push-stack storage***correction:* `context` is
**already an implicit `*Context` parameter** (comptime_vm.zig:392, lower.zig:257
"Implicit Context parameter machinery"), **not raw TLS** — so the "lower as swappable
indirection, never raw TLS" framing guards a non-problem; it already rides the fiber
stack. The real, **currently-unsized** obligation is (a) where a freshly-spawned
fiber's *root* `Context` comes from and (b) where `push Context` frames live (caller
stack ⇒ fiber-local for free; a global root ⇒ must become per-fiber) + per-fiber
stack-limit. **Ground the current mechanism before sizing this.** Prerequisite of
A2, not a successor.
**Async runtime — sx lib over the primitives:**
6. **A1 — `Io` interface + `context.io` + `Future` + `cancel()` API.**
7. **A2 — fiber runtime** (naked context-switch asm, bootstrap, `mmap` stacks).
8. **A3 — blocking `Io` → deterministic-sim `Io` (keystone, calibrated) → event-loop `Io`.**
9. **A5·M:1 — single-thread scheduler.**
10. **N3 — fiber-aware sync** (channels/mutex/waitgroup; `recv → RecvResult`).
11. **A6 — Cancellation.** `.canceled` in the `!` channel (model a); per-fiber atomic
flag (N1); every `io.*` a cancellation point; structured cancel-and-join; **masked
during cleanup**.
12. **A4 — stdlib I/O rework** (fs/socket/process onto `context.io`).
13. **A5·N×(M:1)** — first parallel (errno-capture + `context`-fiber-local discipline).
14. **A5·M:N** — work-stealing (steal queues + migration + pinning).
**Then comptime / FFI / JIT cluster:**
15. **S1 — persistent JIT spine** → 16. **C1 — real FFI (LLVM = ABI authority, on S1)**
→ 17. **C2 — `#compiler`→`extern`** → 18. **C3 — comptime asm** (S1 + C1; +S2 if
TLS/ctors).
**Deferred tail:**
19. **S2 — ORC C++ shim** (highest-risk — see §8; macOS `MachOPlatform`; ELF/COFF
unplanned) → 20. **R1 — dylib reload** (shipped `export`) → 21. **R2 —
JIT-resident reload** (S1 + S2; **↔ async live-fiber coupling**, §8) → 22. **R3 —
incremental compilation**.
Hard edges to remember: **C1 depends on S1** (the non-trivial FFI cases); **C3 depends
on C1** (calls through its thunk path); **R1/R2 couple to the async runtime** (can't
hot-swap code with live suspended fibers — runtime + long-lived fibers stay
persistent, only leaf logic reloads).
---
## 8. Irreducible hard problems (detect-and-degrade, don't pretend)
1. **State migration across layout change** (R1/R2) → v1 detects + rebuild/restart;
migration hooks later.
2. **Cross-arch comptime asm** (C3) → can't run on host; narrows the bail + loud
diagnostic; gate to host-arch.
3. **M:N migration hazards** (A5) → errno-capture discipline + fiber-local context
(mandatory), pinning for thread-affine work.
### 8.1 Highest technical risks (from review — ranked, async-first lens)
1. **A2 context-switch correctness** (in the async critical path). Silent stack
corruption, per-arch, **untestable by the deterministic-`Io` harness** (it tests
*scheduling*, not the *switch*); a one-register slip is invisible until it crashes
on the right arch. Couples *library asm* to the *compiler ABI* — ABI drift breaks
it silently later. → needs a dedicated **switch-stress test** (§10).
2. **`define` → tagged-union → match-codegen** (gates `race` + channels).
**DE-RISKED by review** (§7 step 3): all enum stages are type-table-driven with
zero AST coupling, identity is handled by existing type-fn mangled-name memoization,
and forward-declaration for self-ref already exists. Residual is *plumbing*
(name minted results by mangled name + input validation), not new architecture.
3. **Deterministic-`Io` is the test keystone yet itself uncalibrated** — a buggy
deterministic scheduler yields deterministic-*wrong* stdout that snapshots lock in.
→ calibrate against the blocking `Io` / property-test fixed order (§10).
4. **`context`-fiber-local + errno discipline** (A5 M:N). "Non-negotiable" but
enforced by manual rule, not the compiler; M:1 can't even exercise migration.
5. **S2 ORC shim** (deferred, but highest-risk when reached): only C++ in the tree,
**already failed a spike** (`_Thread_local` SIGABRT), `MachOPlatform` is
macOS-specific — **Linux/Windows JIT-resident reload + non-Mac TLS/ctor JIT have no
named plan**. One "M" box hides a per-OS effort.
6. **C1 args-buffer layout-vs-ABI** — "LLVM emits the call" covers the *call*, not the
interpreter's *buffer pack* from `type_info`. Disagreement on edge layouts
(over-aligned/empty structs, aarch64 small-struct register splitting, `bool`) =
silent comptime corruption. → adversarial layout cases (§10).
---
## 9. Decisions log (all resolved)
**Sequencing — locked:** **async-first** (§7). The async cluster (steps 114)
precedes the FFI/JIT cluster (1518) because async needs no JIT spine. **Cancellation
(A6) = model (a)** — a `.canceled` variant in the **existing `!` error channel** that
`io.*` already returns (I/O is inherently fallible, so `io.*` is already `!`-typed —
the "keep calls clean" argument for the non-local-`raise` model is moot). Reuses
`!`/`try`/`catch`/`onfail`; no new unwind primitive. **Net-new prereq surfaced by
grounding:** `callconv(.naked)` (only `.default`/`.c` today). **Generic enums dropped**
`RecvResult($T)`/`TryResult($T)` are **type-fns over `declare`/`define`** (type-fns
already work in type position, e.g. `Make`/`Complex`), so no `enum($T)` feature is
needed; construction carries two contracts (deterministic identity + functional-enum
output, §7 step 3).
**Locked (see §4.6 for the grounded surface):**
- **N1 atomics surface = generic wrapper `Atomic($T)`** + `Ordering` enum, `.init`,
`compare_exchange`/`_weak` returning `?T` (**null = success** — pinned, opposite of
most priors). (Not `@atomic_*` builtins — `@` is address-of in sx.) **RMW set** =
`add/sub/and/or/xor/swap` + `fetch_min`/`fetch_max` (free from LLVM); **no `nand`**.
- **`race` = over futures** (Zig model), **single named-tuple in** (`race((a: fa, b:
fb))`) → synthesized tagged-union out; Go-style handler-map + map literal
**dropped**. **No `async` spawn-sugar** — always `context.io.async(...)`.
- **Channels** = `send`/`recv` methods (no `<-`); **`recv` returns a tagged union**
`RecvResult($T){ value; closed }` (not `(v, ok)`), `try_recv` → `{ value; empty;
closed }`; optional `for ch (v) {…}` iteration sugar. **locks** = `lock()` + `defer
unlock()` (no guard sugar). `race`/`async`/`await` stay library, not keywords.
- **Comptime type metaprogramming = `declare`/`define` (construct) + `type_info`
(reflect) builtins only** (Zig `@Type`/`@typeInfo` model). **Everything else is sx
lib** — `make_enum`, the channel result types, `field_type`, `RaceResult`.
Construction coverage starts at **enum**, grows to struct/tuple later. `Future($T)`
exposes `Value :: T` so `Future(X)→X` is plain member access
(no `type_arg` builtin).
- **C1 FFI engine = LLVM as single ABI authority** — per-signature JIT calling-thunks
via S1 (LLVM emits the ABI-correct call, same as runtime codegen); trampoline
fast-path for trivial calls. **libffi/dyncall + hand-rolled-sx rejected** (2nd/3rd
ABI impl; hand-rolled needs C3 for its own asm leaf anyway). Promotes **S1 to
foundational** (shared by C1, C3).
**Scheduler (Decision 5) — locked:** **M:1 → N×(M:1) → M:N**, all **sx std-lib `Io`
vtables** (compiler only provides N1 atomics + the A2 asm context-switch + `extern`
syscalls). M:1 ships first (validates the colorblind stack, covers I/O-bound);
N×(M:1) is the first parallel step; **M:N is last in sequence but committed — not
deferred.** Data races under parallelism are expected and handled with atomics +
fiber-aware sync — that *is* parallelism, not a wart; M:1's lock-freedom is just a
property of the single-threaded case.
**Deferred, orthogonal additions (Decisions 67) — both addable later without
revisiting anything locked:**
- **C4 (Decision 6) — fully orthogonal; not built now.** Pure deferred optimization
riding S1 (already present for C1/C3): JIT the bundler subgraph instead of
interpreting it. Zero coupling — same bundler sx, same C1 FFI. Apply only if
profiling ever shows the bundler's *own logic* is a hotspot (it's I/O-bound, so
unlikely). Interp+C1 is the shipping bundler.
- **Hot-reload (Decision 7) — deferred; mechanism additive.** Substrate ready: R1
(dylib-swap) needs only shipped `export`; R2 (JIT-resident) needs S1 + the S2 ORC
shim. **R1-vs-R2 chosen at pickup.** One coupling (a design constraint, not a
decision change): you can't hot-swap code with **live suspended fibers** pointing
into the old module — so the async runtime + long-lived fibers stay on the
*persistent* side, only transient **leaf logic** is reloadable (or quiesce fibers
before swap).
---
## 10. Testing & gates
Inherits the project cadence (CLAUDE.md): `zig build && zig build test` after every
step; **xfail-then-green or behavior-lock — no commit both adds a test AND makes it
pass**; never regenerate snapshots while red; corpus = `examples/` + `issues/` with
`.exit`/`.stdout`/`.stderr`/`.ir` snapshots. Per-*step* gates live in the eventual
`PLAN-*` streams; this section is the design-level verification strategy that those
streams must implement.
### 10.1 The async test harness = the deterministic-simulation `Io` (the keystone)
Concurrency is nondeterministic (scheduling/readiness order), which **breaks snapshot
testing** outright. So the **deterministic-sim `Io`** (fixed clock, scripted
readiness, deterministic single-stepping scheduler) is not merely a feature — it is
**the test harness for everything async**. Every concurrency example runs under it →
reproducible stdout → snapshottable. Consequence for sequencing: **build the
deterministic `Io` right after the blocking `Io`** (it's the simplest scheduler after
blocking and it *gates the ability to test* fibers/channels/race/schedulers at all).
The 10 patterns in §4.6-adjacent examples become corpus tests only because they run
under it.
### 10.2 What is NOT snapshot-testable
True parallel **data races** (N×M:1 / M:N) are nondeterministic by construction. They
run under the deterministic `Io` for *correctness* repro, but race-detection needs a
separate **stress harness** (run-N-times / TSan-style), **not** the corpus. Any such
coverage bound must be stated loudly (a `log()`-style note in the harness), never
silently skipped — per the REJECTED-PATTERNS rule against silent gaps.
### 10.3 Arch-sensitive lowering — atomics + context-switch
Atomic orderings lower differently per arch (x86 `lock`-prefix / plain MOV vs aarch64
LL/SC / `ldar`/`stlr`), and the A2 context-switch is per-arch asm. Lock both with the
**existing inline-asm cross-arch sibling pattern**: a `.build` `{"target": "…"}`
sidecar runs **ir-only** on a non-matching host (asserts `.ir` + `.exit` + `.stderr`
from `sx ir --target`) and **end-to-end** on a matching CI runner. So `Atomic`
lowering carries **x86_64 + aarch64 `.ir`** snapshots; the context-switch gets
per-arch run tests on matching runners.
### 10.4 New corpus categories
`17xx` atomics · `18xx` concurrency (fibers/channels/race/async, all under the
deterministic `Io`). Comptime metaprogramming (`declare`/`define`/`type_info`) +
comptime-asm extend `06xx`; C1 FFI extends `12xx`; the cross-arch comptime-asm **loud bail** and
the cancellation diagnostics are `11xx`.
### 10.5 Per-piece gates (design level)
| Piece | Locks via |
|---|---|
| **N1 atomics** | unit `emit_llvm.test.zig` (LLVM `atomicrmw`/`cmpxchg`/`fence` + ordering emission); corpus `17xx` single-thread (deterministic); arch-gated `.ir` (x86_64 + aarch64) |
| **declare / define / type_info** | unit (reflect round-trips; a minted enum has correct layout/match codegen); corpus `06xx` comptime (deterministic) |
| **C1 FFI** | **behavior-lock** existing trampoline cases first; then xfail→green `12xx` comptime extern with floats / structs-by-value / aggregate (`{ptr,len}`) returns; unit for thunk-synth + args-buffer marshal |
| **S1 spine** | infra — exercised transitively via C1/C3 examples; unit for LLJIT lifecycle + thunk cache |
| **C3 comptime asm** | corpus `06xx` host-arch `#run` asm computes a value; `11xx` diagnostic asserts the cross-arch loud bail |
| **A1/A2 fibers** | unit (scheduler step, fiber bootstrap); context-switch arch-gated run tests; corpus `18xx` under deterministic `Io` |
| **A3/A5 schedulers, channels, race, cancel** | corpus `18xx` (the 10 patterns) under deterministic `Io` → deterministic snapshots; cancellation cleanup (`onfail`/`defer`) asserted via stdout ordering |
### 10.6 Cadence example (atomics, N1)
1. **xfail** — add `examples/17xx-atomics-fetch-add.sx` using `Atomic(i64).fetch_add`; seed the `.exit` marker → **red** (codegen missing). *(test added, not yet passing)*
2. **green** — emit LLVM `atomicrmw add` + ordering; example passes; capture `.stdout` + x86_64/aarch64 `.ir` snapshots; review the diff. *(makes it pass, no new test)*
This satisfies "no commit both adds a test and makes it pass," and every other piece
follows the same xfail→green (or behavior-lock→extend) shape.
### 10.7 Review-surfaced gaps (the high-corruption-risk pieces need *correctness*, not existence, tests)
The §10.5 gates prove things *run*; the §8.1 risks are silent-corruption modes a
run/snapshot test won't catch. Each needs an explicit adversarial gate:
- **A2 context-switch — switch-stress test.** Scribble *every* callee-saved register
+ a stack-canary before suspend; deep/recursive fiber chains; verify all survive
post-resume. Run/snapshot tests don't prove register preservation. (The single
highest-corruption-risk piece, §8.1.1.)
- **Deterministic-`Io` — calibrate the oracle.** Cross-check a handful of cases
against the blocking `Io` and property-test that scheduling order is actually fixed,
*before* trusting it to gate everything async (a deterministic-but-wrong scheduler
snapshots garbage).
- **`context`-fiber-local invariant — named test at the N×M:1/M:N step.** M:1 can't
exercise migration; add a test that forces a fiber to migrate and asserts it reads
*its* `context`/`errno`, not the new thread's.
- **N1 ordering *semantics* are out of snapshot scope — state it loudly.** `.ir`
snapshots prove the *keyword emitted*, not weak-memory correctness (e.g. `relaxed`
where `acquire` was needed ships green). Declare this out-of-scope parallel to
§10.2's race carve-out; lock-free structures need the stress harness.
- **C1 args-buffer — adversarial layout cases.** Over-aligned structs, empty structs,
aarch64 small-struct register splitting, `bool` — a wrong layout that happens to
print right passes a stdout test. Call these out explicitly, not just
"structs-by-value."
- **S2 — has no gate today despite a prior spike failure.** When reached, add a TLS +
C-constructor JIT test (the exact `_Thread_local` SIGABRT case), per host OS.
- **Hot-reload — no row today.** When picked up: state-survival test + the
live-suspended-fiber-into-stale-module hazard (R1/R2).

View File

@@ -14,10 +14,10 @@
## 0. TL;DR + feasibility ## 0. TL;DR + feasibility
* **Feasible today, no new infrastructure.** sx already links LLVM (`build.zig:10` * **Feasible today, no new infrastructure.** sx already links LLVM (`build.zig:10`
`/opt/homebrew/opt/llvm@19`) and `@cImport`s `llvm-c/Core.h` `/opt/homebrew/opt/llvm@22`) and `@cImport`s `llvm-c/Core.h`
(`src/llvm_api.zig:1-17`). That header exposes everything inline asm needs, (`src/llvm_api.zig:1-17`). That header exposes everything inline asm needs,
reachable right now through `llvm_api.c.*`: reachable right now through `llvm_api.c.*`:
* `LLVMGetInlineAsm(Ty, AsmString, AsmStringSize, Constraints, ConstraintsSize, HasSideEffects, IsAlignStack, Dialect, CanThrow)` — builds the asm callee (LLVM 19/21 share this 9-arg signature). * `LLVMGetInlineAsm(Ty, AsmString, AsmStringSize, Constraints, ConstraintsSize, HasSideEffects, IsAlignStack, Dialect, CanThrow)` — builds the asm callee (LLVM 1922 share this 9-arg signature).
* `LLVMInlineAsmDialectATT` / `LLVMInlineAsmDialectIntel`. * `LLVMInlineAsmDialectATT` / `LLVMInlineAsmDialectIntel`.
* `LLVMBuildCall2(...)` — already used pervasively in `src/ir/emit_llvm.zig` (e.g. the Obj-C msgSend path) — calls the asm value like a function. * `LLVMBuildCall2(...)` — already used pervasively in `src/ir/emit_llvm.zig` (e.g. the Obj-C msgSend path) — calls the asm value like a function.
* `LLVMAppendModuleInlineAsm(M, Asm, Len)` — module-level (global) asm. * `LLVMAppendModuleInlineAsm(M, Asm, Len)` — module-level (global) asm.
@@ -549,6 +549,40 @@ Lexer/token: add `kw_asm` to the `Token.Tag` enum + keyword `StaticStringMap` in
* Every `%[name]` referenced in the template must name an operand (best surfaced as * Every `%[name]` referenced in the template must name an operand (best surfaced as
a Sema diagnostic; also caught at codegen during the rewrite — §II.6). a Sema diagnostic; also caught at codegen during the rewrite — §II.6).
### Operand naming rule (auto-name from a `{reg}` pin) — DECIDED
The `[name]` label on an operand is purely an sx-surface convenience: it provides
the `%[name]` template alias and (for `out_value`) the result tuple's field name.
LLVM never sees it (it sees positional `${N}` + the constraint). To kill the
common redundancy where a label just echoes its pinned register
(`[eax] "={eax}"`), the **operand name is derived as follows**, uniformly across
every operand kind (`out_value` / `out_place` / read-write / `input`):
1. **Explicit `[name]` wins** — use it verbatim (the `%[name]` alias / field name).
2. **Else, if the constraint pins a single register**`"={eax}"`, `"{rdi}"`,
`"+{rax}"`, i.e. a `{reg}` body (optionally with a `=`/`+` prefix) — the operand
is **auto-named after that register** (`eax`, `rdi`, `rax`). Usable as
`%[eax]` and as the tuple field name.
3. **Else (register-class `=r`/`+r`/`r`, or memory `=m`, …)** — the operand has
**no implicit name**. A `[name]` is then **required** if the template
references it (`%[name]`) or, for `out_value`, if a named result field is
wanted; otherwise it is anonymous (positional tuple field).
Corollaries:
* **Reject the echo form.** An explicit `[name]` that is identical to the
register its own constraint pins (`[eax] "={eax}"`) carries no information —
emit a diagnostic ("redundant operand name `eax` — it already names the pinned
register; drop the `[eax]`"). The useful form is a label that *differs* from the
register (`[quot] "={rax}"` → field `quot` over register `rax`).
* **Result field names** (the §II.5 result-type rule above) come from each
`out_value`'s *effective* name — explicit `[name]`, else the auto-derived
register name; positional only when neither exists (a class-constrained output
with no `[name]`).
* This is a **typing-stage** rule: the parser still stores `name: ?[]const u8`
(null when no `[name]` was written); Sema computes the effective name. No
parser change.
Note: there is **no** "≤1 output" rule (that was Zig's limit; sx's tuples lift it). Note: there is **no** "≤1 output" rule (that was Zig's limit; sx's tuples lift it).
## II.6 sx IR + LLVM codegen (the part that must match Zig bit-for-bit) ## II.6 sx IR + LLVM codegen (the part that must match Zig bit-for-bit)

437
docs/inline-assembly.md Normal file
View File

@@ -0,0 +1,437 @@
# Inline Assembly in sx
A guide to writing inline assembly in sx — emitting raw target
instructions, wiring values in and out, writing through memory, and
defining whole routines in assembly.
> Looking for the *why* behind the design (how it maps to LLVM, the
> Zig comparison, the emit algorithm)? That lives in
> [inline-asm-design.md](../design/inline-asm-design.md). This page is the
> user-facing how-to.
---
## The mental model
`asm` is an **expression**. It drops to the machine: you write a
template of real instructions, declare which sx values feed registers
going in and which come back out, and the block evaluates to the
output value (or a tuple of them).
```sx
add :: (a: i64, b: i64) -> i64 {
return asm { "add %[out], %[a], %[b]", [out] "=r" -> i64, [a] "r" = a, [b] "r" = b };
}
```
Three things to know up front:
1. **The body is a brace block of comma-separated parts:** the template
string first, then operands, then an optional `clobbers(.…)` clause.
2. **Each operand is tagged by role**, not by position: `-> Type` is a
value output, `= expr` is an input, `-> @place` writes through to
existing storage. The list is flat and order-independent — there are
no positional `:` sections.
3. **The outputs decide the result.** Zero outputs → `void` (and the
block must be `volatile`); one → that type; many → a tuple.
Templates are **AT&T syntax** (lowered through LLVM), **target-specific**,
and **never run at compile time** — see [When it runs](#when-it-runs).
---
## Operands
An operand is `[name]? "constraint" <role>`. The constraint string is
the LLVM/GCC-style constraint; the role marker says what the operand
does.
### Inputs — `= expr`
`= expr` feeds a value in. The constraint picks where it lands:
```sx
[a] "r" = a // any general register
"{rdi}" = fd // pinned to a specific register (x86_64 rdi)
```
### Symbol inputs — `"s" = fn`
A `"s"` input feeds a **function or global symbol** (not a runtime value).
In the template, `%[name]` expands to the symbol's **platform-mangled
name**, so you can branch or call straight to it:
```sx
cb :: (n: i64) -> i64 export "cb" { return n + 1; }
trampoline :: (n: i64) -> i64 {
return asm volatile {
#string ASM
mov x0, %[arg]
bl %[fn] // DIRECT call — `bl _cb` on macOS, `bl cb` on Linux
mov %[res], x0
ASM,
[res] "=r" -> i64,
[arg] "r" = n,
[fn] "s" = cb, // symbol operand
clobbers(.x0, .x30, .memory),
};
}
```
The same `%[fn]` works on **x86_64** — just the branch mnemonic differs:
```sx
return asm volatile {
"call %[fn]", // x86_64 — same portable %[fn]
[ret] "={rax}" -> i64,
"{rdi}" = n,
[fn] "s" = cb,
clobbers(.rcx, .rdx, .rsi, .r8, .r9, .r10, .r11, .memory),
};
```
Two reasons to prefer this over passing a function *pointer* in a plain
`"r"` register and using an indirect `blr`/`call *`:
- **One fewer indirection** — a direct PC-relative branch, no pointer
load into a register, and a predictable (non-indirect) branch.
- **Portable** — `%[fn]` is the same on every target; the backend emits
the correctly-mangled name, so you never hardcode the macOS leading
underscore *or* a per-arch operand modifier.
**How the portability works.** A bare `%[fn]` would render differently
per target — on x86 the symbol prints as `$cb` (an immediate `$`-prefix
that `call` rejects), while aarch64 prints it bare. So for a symbol (`"s"`)
operand the compiler **auto-injects LLVM's `:c` operand modifier** (`%[fn]`
`${N:c}`, "print the constant with no punctuation"). `:c` prints the
plain symbol on every target — equivalent to the GCC `:P`/`%P0` call-target
idiom on x86 (both emit the same `R_X86_64_PLT32` relocation) and a no-op
on aarch64. You can still override it with an explicit `%[fn:X]` if you
ever need a different rendering, but for a call/branch you never should.
The callee needs a stable, externally-linked symbol — i.e. `export`
(which also gives it the C ABI). A plain or `callconv(.c)`-only function
is `internal` and gets dead-code-eliminated, so the symbol won't link.
(A global-scope `asm { … }` routine has no operand list, so it can't use
a symbol operand — it references the literal symbol in its text.)
### Value outputs — `-> Type`
`-> Type` produces a value that becomes (part of) the block's result:
```sx
[out] "=r" -> i64 // result in any register
"={rax}" -> i64 // result pinned to rax
```
### Naming and `%[name]`
Inside the template, `%[name]` refers to an operand by its **effective
name**. An operand pinned to a register is **auto-named after that
register** — `"{rdi}"` is reachable as `%[rdi]`, `"={rax}"` as `%[rax]`
— so an explicit `[name]` is only needed:
- for a register-**class** operand (`"=r"`, `"r"`), which has no register
to name it; or
- to give a pinned operand a name *different* from its register.
Two labels are rejected so names stay unambiguous:
- the **echo form** `[rax] "={rax}"` — the label just repeats the pin, so
drop it (the operand is already `%[rax]`); and
- **duplicate** operand names.
In the template, `%%` is a literal `%`, and `%=` expands to a unique id
(handy for a local label that must differ across inlinings).
### The result type
The number of **value** outputs (`-> Type`) decides the block's type:
| `-> Type` outputs | result | example |
|---|---|---|
| 0 | `void` — must be `volatile` | `asm volatile { "dmb ish" }` |
| 1 | that type `T` | `x := asm { …, "=r" -> i64 }` |
| N | a **tuple**, fields named by each operand's name | `lo, hi := asm { … }` |
With multiple outputs you get real multiple return values — a named
operand becomes a named tuple field:
```sx
// aarch64 — split a value into low/high bytes
split :: (x: u64) -> (lo: u64, hi: u64) {
return asm {
#string ASM
and %[l], %[x], #0xff
lsr %[h], %[x], #8
ASM,
[l] "=r" -> u64, // → .lo (operand 0)
[h] "=r" -> u64, // → .hi (operand 1)
[x] "r" = x,
};
}
lo, hi := split(0x1234); // (0x34, 0x12) = (52, 18)
```
---
## `volatile`
`asm volatile { … }` marks the block as having side effects, so the
optimizer won't move or delete it. It is **required whenever there are
no value outputs** — a result-less, non-volatile asm would be dead code.
```sx
barrier :: () { asm volatile { "dmb ish" }; } // aarch64 full barrier
```
A block with outputs may still be `volatile` when its effects matter
beyond the returned value (e.g. a syscall).
---
## `clobbers(.…)`
`clobbers(.…)` is a dot-name list of registers and flags the asm trashes
that aren't already operands — so the register allocator keeps clear of
them:
```sx
clobbers(.rcx, .r11, .memory) // x86_64 syscall trashes rcx, r11, and memory
clobbers(.cc) // condition flags
```
`.memory` means "this asm reads or writes memory the compiler can't see,"
and `.cc` means "the condition flags are modified."
---
## Writing through memory — `-> @place`
Sometimes the asm should write into existing storage (a local, a struct
field) rather than *return* a value. `-> @place` does that: the place
output does **not** join the result tuple. There are three forms,
distinguished by the constraint.
### Write-through — `= …` constraint
The asm computes a value into a register; sx stores it through the
place's address afterward.
```sx
compute :: () -> i64 {
other : i64 = 0;
main_val := asm volatile {
#string ASM
mov %[m], #5
mov %[o], #37
ASM,
[m] "=r" -> i64, // value output → returned into main_val
[o] "=r" -> @other, // place output → stored through @other
};
return main_val + other; // 5 + 37 = 42
}
```
A value output and one or more place outputs can mix freely; only the
value outputs build the returned tuple.
### Read-write — `+` constraint
A `+` operand is read **and** written: the place's current value is fed
in, the asm updates it in place, and the result is stored back.
```sx
// increment-in-place: x is loaded, the asm adds 1, the result is stored back
bump :: () -> i64 {
x : i64 = 41;
asm volatile { "add %[v], %[v], #1", [v] "+r" -> @x };
return x; // 42
}
```
### Indirect memory — `=*m` constraint
An `=*m` operand passes the place's **address** to the asm, which writes
through it directly (no register round-trip, no return slot):
```sx
// store 42 straight into x's storage
poke :: () -> i64 {
x : i64 = 0;
asm volatile {
#string ASM
mov x9, #42
str x9, %[out]
ASM,
[out] "=*m" -> @x,
clobbers(.x9),
};
return x; // 42
}
```
**The place must be mutable storage.** Taking the address of a scalar
`::` constant has no meaning — a scalar constant folds to its value and
has no storage — so `-> @SOME_CONST` is a compile error:
```
cannot take the address of constant 'SOME_CONST' — a scalar '::'
constant has no storage (use a '=' variable or a local copy)
```
---
## Multi-instruction templates
A single `"…"` string is one fragment. For several instructions, use a
multi-line string literal or sx's **`#string` heredoc**, which is
delivered **verbatim** — no escape processing — so you write assembly
exactly as it should appear:
```sx
serialize :: () {
asm volatile {
#string ASM
mfence
lfence
ASM,
};
}
```
---
## Global (module-scope) assembly
A top-level `asm { … }` block is **global assembly** — template only
(no operands, no `volatile`), emitted as module-level assembly. It is
the place to define a whole routine in assembly. Symbols it defines are
reached from sx with a **lib-less `extern`** declaration:
```sx
asm {
#string ASM
.global _my_add
_my_add:
add x0, x0, x1
ret
ASM,
};
my_add :: (a: i64, b: i64) -> i64 extern;
main :: () -> i64 {
return my_add(40, 2); // 42 — computed by the global-asm routine
}
```
Multiple global blocks concatenate in source order. (Symbol naming
follows the platform convention — a leading underscore on macOS, none
on Linux.)
---
## When it runs
Inline assembly is emitted into the program and runs at **runtime**,
under both execution paths:
- **`sx run` (JIT)** — the module is compiled to an in-memory object
(the integrated assembler assembles your asm, including global blocks),
then run. Both inline and global asm work.
- **`sx build` (AOT)** — same, into a native binary.
It does **not** run at **compile time**. A `#run` (comptime) call into a
global-asm symbol fails loudly:
```sx
COMPUTED :: #run my_add(40, 2); // error: the symbol isn't linked yet at comptime
```
```
comptime extern call: symbol not found via dlsym
```
The comptime interpreter resolves `extern` calls against the host
process; a module-asm symbol only exists once the program is
assembled and linked, so call it at runtime, not in a `#run`.
---
## Cookbook
**Read a register** (no inputs):
```sx
stack_ptr :: () -> u64 {
return asm { "mov %[out], sp", [out] "=r" -> u64 }; // aarch64
}
```
**x86_64 syscall**`write(2)`, with pinned registers and clobbers:
```sx
sys_write :: (fd: i64, buf: *u8, count: i64) -> i64 {
return asm volatile {
"syscall",
[ret] "={rax}" -> i64, // bytes written, in rax
"{rax}" = 1, // SYS_write
"{rdi}" = fd,
"{rsi}" = buf,
"{rdx}" = count,
clobbers(.rcx, .r11, .memory),
};
}
```
**x86_64 divmod** — one instruction, two outputs, returned as a tuple:
```sx
divmod :: (n: u64, d: u64) -> (quot: u64, rem: u64) {
return asm {
"divq %[d]",
[quot] "={rax}" -> u64,
[rem] "={rdx}" -> u64,
"{rax}" = n, "{rdx}" = 0, [d] "r" = d,
clobbers(.cc),
};
}
q, r := divmod(17, 5); // (3, 2)
```
---
## Rules of thumb
- **`asm` yields a value.** Bind it (`x := asm { … }`), `return` it, or
destructure a multi-output tuple (`a, b := asm { … }`). A block with no
value outputs must be `volatile`.
- **Pinned operands name themselves.** `"{rdi}"` is `%[rdi]`; only add
`[name]` for register-class operands or to rename. Don't echo a pin
(`[rax] "={rax}"`).
- **`%%` for a literal percent; `%[name]` for an operand.** Templates are
AT&T.
- **List everything you trash** in `clobbers(.…)` — scratch registers,
`.cc`, and `.memory` if the asm touches memory the compiler can't see.
- **`-> @place` writes storage; pick the form:** `=` (compute then
store), `+` (read-modify-write), `=*m` (write through the address).
The place must be mutable — not a scalar `::` constant.
- **Global `asm { … }`** defines symbols; import them with a lib-less
`extern`. They run under JIT and AOT, but **not** in a `#run`.
- **It's target-specific.** Gate or pick instructions per architecture;
there is no portable instruction set.
---
## See also
- [inline-asm-design.md](../design/inline-asm-design.md) — the design rationale and
LLVM mapping.
- `examples/16xx-platform-asm-*` — the full, runnable example matrix
(basic in/out, tuples, the three `-> @place` forms, global asm, the
x86_64 syscall, and the comptime-boundary guard).
- The "Inline Assembly" section of [readme.md](../readme.md) for a
one-screen overview.
```

View File

@@ -1,43 +0,0 @@
#import "modules/std.sx";
SPECIAL_VALUE :u8: 42;
resolve :: (x: u8) -> i32 {
return 12 + x;
}
Foo :: struct {
a : u2; // this will have 0 as default
b : u8 = SPECIAL_VALUE;
c : u8 = ---; // default for c is undefined
d : u8 = #run xx resolve(5); // converts i32 to u8
}
main :: () {
a : Foo; // default value of 0
print("a 0 : {}\n", a);
a.a = 1;
// a.c is still undefined at this point
a.c = 8;
print("a 1 : {}\n", a);
large: f64 = 5989.5;
b : Foo = ---; // undefined
b.a = 1;
b.c = xx large; // converts f64 to u8
// expect stdout : "b: Foo{a:1, b: 42, c: 7, d: 12}"
print("b: {}", b);
print("\n");
f := Pack.{1,0,3,5,9,100,3.5};
print("{}\n", f);
}
Pack :: struct {
a: u1;
b: u2;
c: u8;
d: u32;
f: u64;
v: i32;
x: f32;
}

View File

@@ -0,0 +1,17 @@
// Atomic($T) load/store with explicit memory orderings, single-thread.
// Stream A (atomics) A.0 + A.0.5 — the ordering is a comptime `$o: Ordering`
// param (explicit, Rust-style): a.load(.acquire) emits `load atomic … acquire`.
// An invalid combination (a.load(.release)) is a compile error (see 1131).
#import "modules/std.sx";
#import "modules/std/atomic.sx";
main :: () {
a := Atomic(i64).init(7);
print("init: {}\n", a.load(.seq_cst));
a.store(42, .release);
print("after store: {}\n", a.load(.acquire));
a.store(a.load(.relaxed) + 1, .seq_cst);
print("incremented: {}\n", a.load(.seq_cst));
}

View File

@@ -0,0 +1,39 @@
// Atomic($T) read-modify-write: fetch_add/sub/and/or/xor/min/max → LLVM atomicrmw.
// Each returns the OLD value. Stream A (atomics) A.1. Single-thread.
// Also covers signed min/max with NEGATIVES at BOTH comptime (#run) and runtime —
// they must agree (regression: comptime once did an unsigned compare).
#import "modules/std.sx";
#import "modules/std/atomic.sx";
// comptime (#run) signed min/max with a negative — must match runtime.
c_max :: () -> i64 { a := Atomic(i64).init(-5); _ := a.fetch_max(3, .seq_cst); return a.load(.seq_cst); }
c_min :: () -> i64 { a := Atomic(i64).init(-5); _ := a.fetch_min(3, .seq_cst); return a.load(.seq_cst); }
G_MAX :: #run c_max();
G_MIN :: #run c_min();
main :: () {
a := Atomic(i64).init(10);
print("old add: {}\n", a.fetch_add(5, .seq_cst)); // returns 10, now 15
print("old sub: {}\n", a.fetch_sub(3, .acq_rel)); // returns 15, now 12
print("now: {}\n", a.load(.acquire)); // 12
b := Atomic(i64).init(0xF0);
print("old and: {}\n", b.fetch_and(0x3C, .relaxed));// returns 0xF0(240), now 0x30(48)
print("old or: {}\n", b.fetch_or(0x03, .relaxed)); // returns 0x30(48), now 0x33(51)
print("old xor: {}\n", b.fetch_xor(0x0F, .relaxed));// returns 0x33(51), now 0x3C(60)
print("now: {}\n", b.load(.relaxed)); // 60
m := Atomic(i64).init(20);
print("old min: {}\n", m.fetch_min(8, .seq_cst)); // returns 20, now 8
print("old max: {}\n", m.fetch_max(15, .seq_cst)); // returns 8, now 15
print("now: {}\n", m.load(.seq_cst)); // 15
// signed min/max with a negative — comptime (#run) and runtime must agree.
s := Atomic(i64).init(-5);
_ := s.fetch_max(3, .seq_cst);
print("runtime signed max(-5,3): {}\n", s.load(.seq_cst)); // 3
s.store(-5, .seq_cst);
_ := s.fetch_min(3, .seq_cst);
print("runtime signed min(-5,3): {}\n", s.load(.seq_cst)); // -5
print("comptime signed max(-5,3)={} min(-5,3)={}\n", G_MAX, G_MIN); // 3 / -5
}

View File

@@ -0,0 +1,34 @@
// Atomic($T) compare-exchange: compare_exchange / compare_exchange_weak → LLVM
// cmpxchg. Result is `?T` — null = SUCCESS; a present value is the ACTUAL current
// value on failure (for a retry loop). Stream A (atomics) A.2. Single-thread.
#import "modules/std.sx";
#import "modules/std/atomic.sx";
main :: () {
// Successful CAS: 10 == 10 → store 20, returns null.
a := Atomic(i64).init(10);
got := a.compare_exchange(10, 20, .acq_rel, .acquire);
if got == null {
print("cas ok, now: {}\n", a.load(.acquire)); // 20
} else {
print("cas unexpected fail: {}\n", got!);
}
// Failing CAS: 99 != 20 → no store, returns the actual value (20), unchanged.
got2 := a.compare_exchange(99, 0, .seq_cst, .seq_cst);
if got2 == null {
print("cas unexpected ok\n");
} else {
print("cas failed, actual: {}, still: {}\n", got2!, a.load(.seq_cst)); // 20, 20
}
// Retry loop with the weak variant: increment a counter by 5.
counter := Atomic(i64).init(100);
cur := counter.load(.relaxed);
while true {
r := counter.compare_exchange_weak(cur, cur + 5, .acq_rel, .acquire);
if r == null { break; }
cur = r!; // retry with the observed value
}
print("after loop: {}\n", counter.load(.seq_cst)); // 105
}

View File

@@ -0,0 +1,16 @@
// Atomic($T).swap — atomic exchange (LLVM atomicrmw xchg): store the new value,
// return the OLD one. Stream A (atomics) A.3. Single-thread.
// Covers swap at BOTH comptime (#run) and runtime — they must agree.
#import "modules/std.sx";
#import "modules/std/atomic.sx";
c_swap :: () -> i64 { a := Atomic(i64).init(7); old := a.swap(42, .seq_cst); return old * 100 + a.load(.seq_cst); }
G_SWAP :: #run c_swap(); // 742 (old 7, now 42)
main :: () {
a := Atomic(i64).init(7);
old := a.swap(42, .acq_rel);
print("swap old: {}\n", old); // 7
print("swap now: {}\n", a.load(.acquire)); // 42
print("comptime swap: {}\n", G_SWAP); // 742 (matches runtime)
}

View File

@@ -0,0 +1,15 @@
// Standalone memory fence — fence(.ordering) → LLVM fence. Stream A (atomics) A.3.
// (.relaxed is rejected; see 1187.) Single-thread: a fence is observable only as
// "compiled + ran without error" here.
#import "modules/std.sx";
#import "modules/std/atomic.sx";
main :: () {
a := Atomic(i64).init(1);
a.store(2, .relaxed);
fence(.release);
a.store(3, .relaxed);
fence(.acquire);
fence(.seq_cst);
print("after fences: {}\n", a.load(.relaxed)); // 3
}

View File

@@ -0,0 +1,20 @@
// Atomic(bool) — a sub-byte (i1) element atomically loaded/stored. LLVM
// rejects a sub-byte atomic ("atomic memory access' size must be byte-
// sized"), so codegen performs the access in the byte storage type (i8)
// and trunc/zext's the value at the boundary. (rmw/cmpxchg on a bool is
// rejected at the sx level — integer-only — so only load/store apply.)
// Regression (issue 0152): Atomic(bool) emitted an i1 atomic that failed
// LLVM verification; Future.canceled: Atomic(bool) in the async layer hit it.
#import "modules/std.sx";
#import "modules/std/atomic.sx";
main :: () {
a := Atomic(bool).init(false);
print("init: {}\n", a.load(.acquire)); // false
a.store(true, .release);
print("after store: {}\n", a.load(.acquire)); // true
a.store(false, .seq_cst);
print("after reset: {}\n", a.load(.seq_cst)); // false
}

View File

@@ -0,0 +1,3 @@
init: 7
after store: 42
incremented: 43

View File

@@ -0,0 +1,13 @@
old add: 10
old sub: 15
now: 12
old and: 240
old or: 48
old xor: 51
now: 60
old min: 20
old max: 8
now: 15
runtime signed max(-5,3): 3
runtime signed min(-5,3): -5
comptime signed max(-5,3)=3 min(-5,3)=-5

View File

@@ -0,0 +1,3 @@
cas ok, now: 20
cas failed, actual: 20, still: 20
after loop: 105

View File

@@ -0,0 +1,3 @@
swap old: 7
swap now: 42
comptime swap: 742

View File

@@ -0,0 +1 @@
after fences: 3

View File

@@ -0,0 +1,3 @@
init: false
after store: true
after reset: false

View File

@@ -7,7 +7,7 @@
#import "modules/std.sx"; #import "modules/std.sx";
Show :: protocol { Show :: protocol {
show :: () -> string; show :: (self: *Self) -> string;
} }
A :: struct { x: i64; } A :: struct { x: i64; }
B :: struct { s: string; } B :: struct { s: string; }

View File

@@ -10,11 +10,11 @@ mul :: (a: i32, b: i32) -> i32 { a * b }
// P4 edge: Chained default→default calls // P4 edge: Chained default→default calls
Chained :: protocol { Chained :: protocol {
base :: (msg: string) -> i32; base :: (self: *Self, msg: string) -> i32;
wrap :: (msg: string) -> i32 { wrap :: (self: *Self, msg: string) -> i32 {
self.base(msg) + 1 self.base(msg) + 1
} }
double_wrap :: (msg: string) -> i32 { double_wrap :: (self: *Self, msg: string) -> i32 {
self.wrap(msg) + self.wrap(msg) self.wrap(msg) + self.wrap(msg)
} }
} }

View File

@@ -0,0 +1,27 @@
// Interning a large (~64KB) array type and using `{}` formatting elsewhere must
// NOT scalarize into an O(N) SelectionDAG (which crashed `sx build` / made
// `sx run` take ~12s). The array Any-unbox formats via a SLICE VIEW of its
// storage — no whole-array load.
//
// Regression (issue 0125): `any_to_string`'s `case array:` arm used to do
// `array_to_string(cast(type) val)`, loading the whole [65536]u8 by value and
// reading each element off the loaded aggregate. Now the dispatcher builds a
// `{ptr,len}` slice view of the payload pointer and formats that — output is
// identical (`[a, b, c]`), and a large unrelated array type costs nothing.
#import "modules/std.sx";
f :: () {
buf : [65536]u8 = ---;
buf[0] = 65; // 'A'
out(string.{ ptr = @buf[0], len = 1 });
out("\n");
}
main :: () -> i32 {
f();
print("{}\n", 5); // an int format — unaffected by the big array
small : [3]i64 = .[7, 8, 9];
print("{}\n", small); // array format still renders the element list
return 0;
}

Some files were not shown because too many files have changed in this diff Show More