ba28488d99
`BuildOptions :: struct #compiler { ...35 methods... }` becomes
`BuildOptions :: struct { }` (an opaque null-sentinel handle) plus 35 free
`ufcs (self: BuildOptions, …) abi(.compiler)` decls in build.sx, each serviced
by a new `comptime_vm.callBuildOptionFn` arm (off `callCompilerFn`). No legacy
`compiler_lib` handler: the names are registered in `bound_fns` with a single
bailing stub only so `weldedCompilerFn` accepts them.
- String lifetime: setters dupe the arg into the persistent `Vm.gpa` (the
Compilation allocator, threaded into both `tryEval` and `runBuildCallback` —
not the per-eval VM arena) and write/append to the threaded `BuildConfig`.
Getters read the field/slice or compute the target predicate from the triple.
- Dispatch routing (Option B): a `#run`/const-init entry that directly calls a
compiler-domain/welded fn (`emit_llvm.entryNeedsVm`) runs on the VM with no
legacy fallback regardless of the `-Dcomptime-flat` gate, so gate-OFF stays
green without a legacy BuildOptions handler (P5.7 retires the legacy interp).
- Mark the 5 `platform/bundle.sx` getter-calling helpers `abi(.compiler)` (they
are comptime-only bundler code; otherwise their now-welded getter calls trip
the runtime-call gate).
- 37 `.ir` snapshots regenerated (std transitively imports build.sx → string-
pool/type-table indices shift); verified `.ir`-only, zero behavior-stream diffs.
BuildOptions `compiler_call` strict bails gone (1609/1614/1615 strict-clean);
1616 now bails on a separate, pre-existing unported bitwise/shift VM gap (`shr`),
to port first in P5.6. 703/0 both gates.
Also sweep the outdated "flat memory" terminology to "comptime/byte-addressable"
across comptime_vm + the plan/checkpoint/CLAUDE docs: the comptime VM is
arena-backed, byte-addressable memory where `Addr` is a real host pointer, not a
flat contiguous address space (flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
1471 lines
130 KiB
Markdown
1471 lines
130 KiB
Markdown
# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`)
|
||
|
||
Companion to the design-of-record
|
||
[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan
|
||
+ phased build order live there). This stream supersedes the metatype
|
||
`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute
|
||
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
|
||
|
||
## ⏯ Resume (fresh session)
|
||
|
||
> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now
|
||
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.**
|
||
> The **byte-weld + serialization/marshaling** approach is the wrong direction and is
|
||
> being **stripped**. New foundation: a **bytecode VM over byte-addressable
|
||
> memory** so comptime values are native bytes; then the compiler-API rides on it with
|
||
> direct memory access (no weld, no validation, no marshaling). Everything below this
|
||
> banner describes the now-superseded weld state (committed on `reify` through
|
||
> `40d075c`) and is kept only to scope the Phase 0 strip. Read
|
||
> `PLAN-COMPILER-VM.md` first.
|
||
>
|
||
> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as
|
||
> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)`
|
||
> reads the `Value` union's memory, not a struct. The weld tried to bridge that with
|
||
> hand-marshaling — exactly what the design set out to kill. Comptime memory makes comptime
|
||
> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it
|
||
> breaks cross-compilation — host vs target layout — and loses the sandbox. A
|
||
> comptime VM keeps both while getting native bytes + speed.)
|
||
>
|
||
> **Next action (2026-06-18) — the WHOLE metatype surface is VM-native (steps 7+8, committed through
|
||
> `d0ebc55`; step 8 uncommitted).** `declare`/`define`/`type_info` + tagged-union `enum_init` all run
|
||
> NATIVELY on the VM (`.call_builtin` exec arm → `callBuiltinVm`; `defineFromInfo` decodes a
|
||
> `TypeInfo` from comptime memory, `buildTypeInfo` reflects one INTO comptime memory — faithful ports of
|
||
> legacy `defineEnum`/`Struct`/`Tuple`/`reflectTypeInfo`). The ENTIRE metatype range `0614`–`0624` +
|
||
> `0632` runs **HANDLED with ZERO fallback** (incl. the `define(declare, type_info(T))` round-trips
|
||
> `0619`/`0622`/`0623`); VM output byte-matches legacy. `enum_init`/`define`/`type_info` bail loudly
|
||
> on a `backing_type` tagged union rather than silent-clobber. **697/0 BOTH gates + all unit tests.**
|
||
> **THE NEXT STEP — Phase 4D.3 (`compiler_call` / #compiler hooks on the VM).** Phase 4 (legacy-interp
|
||
> retirement) is PLANNED in `PLAN-COMPILER-VM.md`; **user decision: UNIFY** (the VM runs the post-link
|
||
> bundler too, `interp.zig` fully deleted). DONE this arc (all green): **4A.1** box_any/unbox_any +
|
||
> `.any` as a 16-byte aggregate (`1526d19`); **4D.0** comptime memory → an ARENA, `Addr` = real host
|
||
> pointer, no buffer/cap/move (`625ba0f`); **4D.1** general host-FFI escape — `Vm.callHostExtern`
|
||
> dlsym + host_ffi, any extern, args/returns pass untouched since Addr IS a host pointer (`e7a8708`,
|
||
> example 0636 `toupper`); **4D.2** slice/string args → NUL-term `char*` + float-arg/return guards
|
||
> (`6a7f690`, example 0637 `strlen([:0]u8)`). **699/0 BOTH gates.**
|
||
>
|
||
> **DIRECTION CORRECTION (2026-06-18, user): `#compiler` / `compiler_call` is DELETED, not bridged
|
||
> on the VM.** `BuildOptions` is RE-EXPRESSED as **`abi(.zig) extern compiler`** functions (the
|
||
> compiler-API surface the VM already dispatches via `callCompilerFn`); the `#compiler` attribute, the
|
||
> `compiler_call` IR op, and the `Value`-based hook `Registry` (`compiler_hooks.zig`) all go away.
|
||
> So there is **NO transitional `compiler_call`→hooks shim** on the VM (I started one — threading the
|
||
> legacy interp into `tryEval` for the hook registry — and reverted it; tree clean at `b05c74f`).
|
||
> `0602`/`0603` stay on legacy fallback until the BuildOptions migration lands. **Migration shape**
|
||
> (end-state, shares the `BuildConfig`-on-the-VM prerequisite with the bundler 4E): (1) each
|
||
> `BuildOptions` setter/getter becomes a `compiler` fn in `compiler_lib.bound_fns` + `Vm.callCompilerFn`,
|
||
> reading comptime args + a `*BuildConfig` threaded into the `Vm` (the same `BuildConfig`
|
||
> `main.zig` forwards); (2) `library/modules/build.sx` declares them `abi(.zig) extern compiler`
|
||
> instead of `struct #compiler`; (3) delete the `compiler_call` op + `compiler_hooks.zig`
|
||
> `HookFn`/`Registry` + the `#compiler` parse/lower path. See `PLAN-COMPILER-VM.md` Phase 4.
|
||
>
|
||
> **Corpus-driven remainder (independent of the BuildOptions migration):** ALL PURE ops are DONE:
|
||
> `switch_br`, `type_name`, `error_tag_name_get`, `global_addr`, `type_is_unsigned`. **`out` DONE (2026-06-19,
|
||
> newest Log entry):** removed the `out` builtin — it's a plain sx fn calling libc `write`, so the VM handles it
|
||
> via host-FFI (no buffer, no special arm; no double-print because there's no `out` op to bail-then-fallback on).
|
||
> `trace_resolve` PORTED (1035). 0613/1035/0522/1038 run VM-HANDLED. Remaining side-effect op: `interp_print_frames`
|
||
> (1034 — writes the comptime frame chain; could likewise become a plain sx fn over the trace runtime).
|
||
> · **4B VM diagnostics (1179/1180) — DONE** (strict renders the proper `comptime type construction failed:`
|
||
> diagnostic; VM-gap strict bails are now ONLY the 4 `compiler_call`) · **4C** `#insert`. **BuildOptions migration — design settled +
|
||
> foundation underway (2026-06-18, see the two newest Log entries):** `#compiler`/`compiler_call` is replaced
|
||
> by `abi(.compiler)` (a compiler-domain ABI — runs in the comptime evaluator, never in the binary). **S1+S2
|
||
> DONE:** `abi(.compiler)` introduced, the old `abi(.zig) extern compiler` + `#library "compiler"` fully removed,
|
||
> all compiler-API examples migrated. **S3 DONE:** emit_llvm skips BODIED `abi(.compiler)` (compiler-domain)
|
||
> functions; a comptime-only call from a dead body emits `undef` (regression `0638`; 701/0 both gates). The
|
||
> earlier "runtime-reachability gating" blocker is MOOT — a compiler-domain callback isn't LLVM-emitted, so its
|
||
> `build_options()` calls never reach the `emitCall` gate. **S4 SKIPPED (optional ergonomics):** an
|
||
> `abi(.compiler)` function is type-compatible with a plain `() -> R` param (the ABI marks the *function*, not
|
||
> its *type*), so callbacks/registrars just declare themselves `abi(.compiler)` (S3) — no param-propagation
|
||
> needed. **S5a DONE:** `build_options` + `set_post_link_callback` → `abi(.compiler)`, `BuildConfig` threaded
|
||
> into the VM; `bundle_main` + the platform registrars marked `abi(.compiler)`; strict `compiler_call` bails
|
||
> 6→2 (0602/0603/1604/1611 HANDLED). **S5a is a GREEN INTERMEDIATE — do NOT extend it.** **DESIGN PIVOT
|
||
> (2026-06-18, user): the 37-hook BuildOptions port is DEAD — DRIVE THE BUILD PIPELINE FROM SX** (newest Log
|
||
> entry + `PLAN-COMPILER-VM.md` → Phase 5). `BuildConfig` becomes plain sx data; the compiler shrinks to a few
|
||
> `abi(.compiler)` primitives (`emit_object`/`link`/queries, explicit args, `-> !` not bool) + an `on_build`
|
||
> slot (stdlib `default_build`, user override `#run on_build = build;`). **P5.1 (= 4E) DONE (2026-06-19, newest
|
||
> Log entry):** `core.invokeByFuncId` (the post-link build-driver invocation) now runs the callback on the VM
|
||
> with **NO fallback** (a side-effecting callback can't double-execute); `BuildConfig` + `import_sources` threaded
|
||
> in; VM bail → hard build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). Smoke
|
||
> test `1661-platform-post-link-vm-list` (AOT) — a post-link callback that GROWS a `List` (0141: works on VM,
|
||
> bails on legacy with `struct_get`); build succeeds (exit 0) only via the VM. `flushInterpOutput` deleted (VM
|
||
> writes `out` direct via host-FFI). **702/0 both gates.** **P5.2 metadata queries DONE (2026-06-19, newest Log
|
||
> entry):** `c_object_paths() -> List(string)` + `link_libraries() -> List(string)` are `abi(.compiler)` primitives
|
||
> (new stdlib home `library/modules/compiler.sx`), serviced by `comptime_vm.callCompilerFn` reading `BuildConfig`
|
||
> fields `main.zig` forwards (`c_object_paths`/`link_libraries`). New reusable VM helper `makeStringList` builds a
|
||
> `List(string)` in comptime memory (target-aware via the result type's offsets); `invoke`/`callCompilerFn` now thread
|
||
> the call's result type (`ins.ty`). Legacy handlers bail loudly (VM-only by nature — post-link). Smoke test
|
||
> `1662-platform-build-pipeline-queries` (AOT, C companion → 1 object): a post-link callback checks the VM-built
|
||
> list is well-formed; build exit 0 ONLY if so (negative-probe verified: wrong count → "post-link callback
|
||
> returned false", exit 1). **`emit_object() -> string` ALSO landed** (a QUERY — the Zig driver emits eagerly, the
|
||
> primitive returns `BuildConfig.object_path`; NO vtable). So all three QUERY primitives are done. **703/0 both
|
||
> gates.** **P5.2b (`link` ACTION) DONE (2026-06-19, newest Log entry):** `link(objects, output, libraries,
|
||
> frameworks, flags, target)` dispatches through a host-installed `compiler_hooks.BuildHooks` vtable (`main.zig`
|
||
> `LinkHooksCtx` → `target.link`); **USER DECISION: the build callback is NOT fallible** — `link` is plain VOID,
|
||
> a failure BAILS (hard build error), no `-> !`/failable-tuple needed. New VM readers `readStringList`/
|
||
> `readStringArg`. Smoke test `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's
|
||
> objects to a temp output via sx `link` — the relinked binary RUNS; negative-probe verified (bad path → bail →
|
||
> build exit 1). **P5.3 (`on_build` registrar) + P5.4 CORE DONE (2026-06-19, newest Log entries):** the whole
|
||
> build is sx-driven via `default_pipeline` (force-lowered + auto-invoked; NO Zig auto-emit/auto-link);
|
||
> `on_build(cb)` is the sole callback mechanism; `set_post_link_callback` deleted. **703/0 both gates.**
|
||
> **NEXT — the FULL MIGRATION (no legacy left), spec'd as Phase 5 steps P5.5–P5.8 in `PLAN-COMPILER-VM.md`:**
|
||
> **P5.5 DONE (2026-06-19, newest Log entry):** the 35 `BuildOptions` `#compiler` methods → VM-native
|
||
> `abi(.compiler)` arms (`comptime_vm.callBuildOptionFn`, NO legacy handler); setter strings duped into the
|
||
> persistent `Vm.gpa`; `#run`/const-init compiler-domain entries routed to the VM (`entryNeedsVm`, no fallback)
|
||
> so gate-OFF stays green; 5 bundle.sx helpers marked `abi(.compiler)`. BuildOptions `compiler_call` bails GONE
|
||
> (1609/1614/1615 strict-clean; 1616 now bails on `shr` — a SEPARATE unported bitwise/shift VM gap, do FIRST in
|
||
> P5.6). 37 `.ir` regenerated (string-pool churn, behavior-identical). 703/0 BOTH gates. · P5.6 ALL bundling +
|
||
> code signing for EVERY target (macOS/iOS-device/iOS-sim/Android) in
|
||
> the sx `default_pipeline` · P5.7 DELETE `#compiler`/`compiler_call`/`compiler_hooks`/`interp.zig` + the
|
||
> `regToValue` bridge + VM→legacy fallback (drop gate-OFF; VM is the SOLE evaluator) · P5.8 build
|
||
> `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test + add `.app`/`.apk` smoke tests.
|
||
> **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` +
|
||
> flip strict-to-default (remove the fallback) + delete `interp.zig`/`Value` + re-express `define`/`make_enum`.
|
||
> See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + top risks (bundler test coverage).
|
||
> Earlier landed: dedicated `Type` builtin TypeId (`6844fb9`/`94f60c5`/`554871b`); WRITE side
|
||
> declare_type/register_type/pointer_to VM-native (`66005af`); real lowering-time Context (`eb68d9e`);
|
||
> metatype construction declare/define/enum_init (`d0ebc55`).
|
||
>
|
||
> Done so far in Phase 3:
|
||
> - **READ side (7 readers, dual-path):** `find_type`/`type_kind`/`type_field_count`/
|
||
> `type_nominal_name`/`type_field_name`/`type_field_type`/`type_field_value`, each backed by a
|
||
> `TypeTable` query both the legacy handler and the VM call (no drift). Examples 0628–0630.
|
||
> - **WRITE side (P3.3, legacy-only at lowering time):** `declare_type` + `pointer_to` + ONE
|
||
> kind-branching `register_type` (subsumes `define`'s per-kind dispatch; codes match
|
||
> `type_kind`: 1 struct · 2 actual `.@"enum"` · 3 tagged_union · 4 tuple). Idempotent re-fill
|
||
> (two-edge import). Plus two fixes (issue 0142): all-void enum → real `.@"enum"` (was a
|
||
> verifySizes panic); bare `EnumType.variant` qualified construction. Examples 0631–0635, 0187.
|
||
> - **Lowering-time VM (P3.4):** hardened the VM against malformed lowering-time IR (`refTy`,
|
||
> bailing `aggType`, bounds-checked branch targets — bails, never panics); wired `tryEval`
|
||
> into `runComptimeTypeFunc` behind the flag with legacy fallback; materialized a zeroed
|
||
> lowering-time `Context` (the global isn't built yet at lowering). All measured green.
|
||
>
|
||
> **THE WALL (next step):** a `Type` *value* is an 8-byte tid, but `.any` (the boxed-any) is a
|
||
> 16-byte `{tag,value}` — and they share one TypeId (`.any`). So a `Type` in an aggregate
|
||
> (`Member.ty`/`EnumVariant.payload`) is sized 16B while the value is 8B → every lowering-time
|
||
> type-fn bails at `const_type` / the Member-array build. Can't make `kindOf(.any)` a word:
|
||
> at EMIT time `.any` really is a 16B box (variadic any, 0603), so that would silently corrupt
|
||
> it. The correct fix is a **dedicated `Type` builtin TypeId (8B), distinct from `.any`** —
|
||
> measured at **~123 `.any` references across ~25 files** (pack.zig has 30), a ~100-touch-point
|
||
> cross-cutting change → its own focused session (USER CHOSE to pause rather than rush it).
|
||
> Rejected alternatives: a scoped "lowering-mode treats `.any` as a word" flag (silent-wrong on
|
||
> a real Any box in a reflection type-fn); scalar-only Type-fns (safe but no real corpus type-fn
|
||
> is scalar-only — they all build a Member/variant aggregate).
|
||
>
|
||
> **Decisions recorded:** `find_type` returns a non-optional `TypeId` using `unresolved`(0), NOT
|
||
> `?Type`; reader names use the `type_*` family (avoid colliding with std `field_name`/`type_name`);
|
||
> the write side is a single kind-branching `register_type`; the write side stays LEGACY-only
|
||
> until the VM runs at lowering time (needs the `Type` TypeId). End-state guarantee: ONE
|
||
> evaluator — `interp.zig` deleted; dual-path + fallback are transitional (see PLAN end state).
|
||
> Build/verify: `zig build && zig build test` (**697**, gate OFF). Run the corpus ON the VM:
|
||
> `zig build test -Dcomptime-flat` OR env `SX_COMPTIME_FLAT=1`. Coverage trace:
|
||
> `SX_COMPTIME_FLAT_TRACE=1` (now also prints lowering-time `type-fn` HANDLED/fallback lines).
|
||
|
||
### (superseded) prior weld resume
|
||
Phase 1 done; Phase 2 welded structs were working via reflection + memory-order
|
||
validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even
|
||
earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in
|
||
the compiler type's MEMORY order; the compiler reflected the bound Zig type and
|
||
VALIDATED the header. **This whole mechanism is now being stripped — see the banner.**
|
||
|
||
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
|
||
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
|
||
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
|
||
> See CLAUDE.md → Snapshot integrity.
|
||
|
||
## Last completed step
|
||
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
|
||
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
|
||
end-to-end as a byte-identical mirror of the bound Zig type.
|
||
|
||
Design (locked, supersedes the byte-layout-override plan):
|
||
- The sx header declares fields in the compiler type's MEMORY order. The compiler
|
||
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
|
||
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
|
||
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
|
||
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
|
||
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
|
||
byte-blob, no index/remap tables, no reorder, no special LLVM path.
|
||
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
|
||
*wrong field order at position N* (+ expected memory order), *type layout
|
||
mismatch* (field size), *layout mismatch* (total size / count).
|
||
|
||
What changed from the dropped plan:
|
||
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
|
||
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
|
||
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
|
||
the sx header against the memory-ordered registry.
|
||
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
|
||
(+ `weldedFieldOrderStr`).
|
||
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
|
||
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
|
||
refreshed.
|
||
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
|
||
|
||
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
|
||
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
|
||
byte-layout-override engine; superseded by the reflection+validation design above.
|
||
|
||
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
|
||
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
|
||
padding included), so it passes to a Zig handler as raw memory with zero
|
||
marshalling. (The alternative — handlers reading interp `Value` aggregates
|
||
logically, no layout override — was rejected; welded types must also be usable as
|
||
runtime data, and the design wants the literal byte weld.)
|
||
|
||
- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20,
|
||
`is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override
|
||
is genuinely required (`Field`'s two-u32 natural layout was the easy case).
|
||
- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second
|
||
`bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc,
|
||
fields, total)` — pure: orders fields by ascending byte offset, inserts padding
|
||
elements for gaps + the alignment tail, and builds the sx-field → LLVM-element
|
||
remap. This is what the LLVM type builder + struct-GEP sites will consume.
|
||
- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad);
|
||
`StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24,
|
||
pad@25..32]`, remap `[1,0,3,2]`.
|
||
- `zig build` + `zig build test` green.
|
||
|
||
### Earlier — Phase 1 polish (comptime-only enforcement)
|
||
**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error
|
||
instead of an undefined-symbol link failure.**
|
||
- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded`
|
||
AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a
|
||
`#run`/`::` initializer wrapper whose LLVM body is dead), print a clear
|
||
"comptime-only … cannot be called at runtime" error and set
|
||
`comptime_failed` (the driver halts before object/JIT emission). The enclosing
|
||
`is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green.
|
||
- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)`
|
||
→ clean error, exit 1, no link failure).
|
||
- `zig build` + `zig build test` green (458 unit + 690 corpus).
|
||
|
||
### Earlier — fifth sub-step (host-call bridge)
|
||
**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to
|
||
its registered Zig handler instead of dlsym.**
|
||
- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`,
|
||
`bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the
|
||
string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value ->
|
||
Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)`
|
||
(the same mutable-table access the metatype mint path uses); `text_of` reads the
|
||
const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern).
|
||
- IR `Function` gained `compiler_welded: bool`. `declareFunction`
|
||
(`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES:
|
||
the bound lib must be `compiler` and the name must be on the function-export
|
||
list — else a build-gating `.err` (no silent fall-through to dlsym).
|
||
- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function
|
||
routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the
|
||
export list).
|
||
- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run
|
||
text_of(intern("hello, compiler"))` folds to a string constant → prints it);
|
||
`examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name →
|
||
build error). `findFn` lookup unit-tested.
|
||
- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a
|
||
RUNTIME call would emit a reference to a non-existent extern symbol → a loud
|
||
LINK error (not silent, but not a tidy diagnostic). The examples call welded fns
|
||
only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the
|
||
immediate follow-up.
|
||
- `zig build` + `zig build test` green (458 unit tests + 689 corpus).
|
||
|
||
### Earlier — fourth sub-step (welded-struct layout validation)
|
||
**A `struct abi(.zig) extern compiler { … }` is validated against the binding
|
||
registry as a *header checked against the implementation*.**
|
||
- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns
|
||
the first `LayoutMismatch` (field count / name / size / total) or null. Plus
|
||
`lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes;
|
||
each drift flagged as the right variant).
|
||
- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`,
|
||
`validateWeldedStruct` checks the bound lib is `compiler`, the name is on the
|
||
export list (`findType`), and the sx layout (field names + `typeSizeBytes` +
|
||
total) matches the welded type — emitting a build-gating `.err` (good span into
|
||
the struct body) on any failure. No silent reinterpretation.
|
||
- `#library "compiler"` is the comptime-only internal surface, NOT a dylib —
|
||
`src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so`
|
||
load warning).
|
||
- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds,
|
||
validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld-
|
||
struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic).
|
||
- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it
|
||
isn't exercised by `Field` (two u32s = natural layout coincides with the weld).
|
||
It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound
|
||
offsets actually differ from the sx-natural ones. The validation already checks
|
||
per-field size + total, so a layout drift is caught even before the override
|
||
engine exists.
|
||
- `zig build` + `zig build test` green (456 unit tests + 687 corpus).
|
||
|
||
### Earlier — third sub-step (binding registry)
|
||
**The binding registry (welded-type lookup, layout baked from the real Zig
|
||
type).**
|
||
- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the
|
||
curated safety boundary. `BoundType { sx_name, size, alignment, fields:
|
||
[]FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real
|
||
Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a
|
||
sx-field-count mismatch is a `@compileError`, never a silent truncation).
|
||
`bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` —
|
||
two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl
|
||
resolution path will consult (returns null off the export list — clean boundary,
|
||
no silent default).
|
||
- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`.
|
||
- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real
|
||
`StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4);
|
||
an unexported name returns null. Break-verified (a wrong size → suite red,
|
||
named `ir.compiler_lib.test...`).
|
||
- `zig build` + `zig build test` green (454 unit tests).
|
||
|
||
### Earlier — second sub-step (struct-decl parse)
|
||
**`abi(.zig) extern <lib>` PARSES on a STRUCT decl (parse-only, no semantics).**
|
||
- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields.
|
||
- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler`
|
||
check), parse an optional `abi(...)` then optional `extern <lib>` — same slot
|
||
order as fn decls — and thread them onto the node. Ordinary structs are
|
||
unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent).
|
||
- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern
|
||
compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib
|
||
== "compiler"`, field list intact; a plain struct leaves `abi == .default` /
|
||
`extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite
|
||
red, confirming the test runs).
|
||
- `zig build` + `zig build test` green.
|
||
|
||
### Earlier — first sub-step (fn decls) + the syntax pivot
|
||
**`abi(.zig) extern <lib>` PARSES on a fn decl (parse-only).** Plus the syntax
|
||
pivot it required.
|
||
|
||
Syntax decision (locked 2026-06-17, supersedes the doc's original
|
||
`extern(.zig) <lib>` single-qualifier form): the ABI/layout selector and the
|
||
linkage keyword are two orthogonal annotations.
|
||
- `abi(.x)` — ABI / calling-convention annotation in the slot **before**
|
||
`extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.**
|
||
`ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the
|
||
`compiler` library), `.pure` (naked asm), `.default` (unannotated). Can appear
|
||
standalone (no extern) on any fn / fn-type / lambda.
|
||
- `extern <lib>` — linkage keyword + binding source (named library).
|
||
|
||
So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`.
|
||
|
||
What landed:
|
||
- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`;
|
||
the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`.
|
||
- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`,
|
||
keyword string `"callconv"` → `"abi"`.
|
||
- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi`
|
||
(parses `abi(.c|.zig|.pure)`); wired in the fn-decl postfix slot (before
|
||
`extern`/`export`), the function-type-expr slot, and the lambda slot;
|
||
`isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`.
|
||
- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`,
|
||
`closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the
|
||
function-type resolver maps `.zig`/`.pure` → IR `.default` (no fn-pointer-type
|
||
CC for those decl-level ABIs; neither occurs in a function-TYPE position yet).
|
||
- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the
|
||
user-facing text `callconv(.c)` → `abi(.c)`.
|
||
- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type
|
||
callback annotations — none in the fn-decl postfix slot, so no reordering).
|
||
- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 /
|
||
1200) regenerated for the rename.
|
||
- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern <lib>`
|
||
on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib ==
|
||
"compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` /
|
||
`abi(.pure)`. lexer/sema tests updated.
|
||
|
||
`zig build` + `zig build test` green (450/450 unit + 685 corpus).
|
||
|
||
## Current state
|
||
|
||
> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld
|
||
> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for
|
||
> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the
|
||
> weld *semantics* (layout reflection/validation, marshaling dispatch) go.
|
||
|
||
- `compiler :: #library "compiler";` parses + is recognised as the comptime-only
|
||
internal surface (never dlopen'd).
|
||
- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry
|
||
(faithful → ok; drift → build-gating diagnostic). `Field` welds + usable.
|
||
- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to
|
||
their registered Zig handler (`intern`/`text_of` round-trip works); unexported
|
||
names rejected at declaration. Comptime-only.
|
||
- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only
|
||
enforcement at `emitCall`); the legitimate `#run`/`::` use stays green.
|
||
- The whole Phase 1 foundation (parse → registry → struct-layout validation →
|
||
function host-call bridge → comptime-only enforcement) is in place for the
|
||
two-u32 `Field` case + the two string readers.
|
||
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
|
||
(needed by `StructInfo`'s slice field, Phase 2).
|
||
|
||
## Next step — execute `PLAN-COMPILER-VM.md`
|
||
|
||
> The weld is being stripped. The next step is **Phase 0 of
|
||
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize /
|
||
> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig`
|
||
> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics
|
||
> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then
|
||
> Phase 1 (byte-addressable value model). The weld-era "next step" below is **obsolete** —
|
||
> kept only as a record of what the weld surface was about to do.
|
||
|
||
### (obsolete) weld-era next step
|
||
Welded structs were byte-identical mirrors, so the API surface was set to grow:
|
||
|
||
- **Bind `register_struct` / `find_type`** over the host-call bridge
|
||
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
|
||
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
|
||
kind well-formedness — the checks `define` does today). Because the welded
|
||
`StructInfo` is byte-identical, the handler can read it as the real Zig
|
||
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
|
||
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
|
||
Prove: build a struct programmatically + round-trip a source one.
|
||
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
|
||
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
|
||
(`defineStruct` / the `reflectTypeInfo` struct path).
|
||
|
||
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
|
||
(optional fields → sentinels) — each just needs an sx header in the compiler
|
||
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
|
||
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
|
||
`#compiler`.
|
||
|
||
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
|
||
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
|
||
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
|
||
when reached (sentinels or accessor fns; see the design doc Risks).
|
||
|
||
## Known issues
|
||
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
|
||
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
|
||
|
||
## Log
|
||
- **P5.5 — the 35 `BuildOptions` accessors migrated off `struct #compiler` onto VM-native `abi(.compiler)` (2026-06-19).**
|
||
`BuildOptions :: struct #compiler { ...35 methods... }` → `BuildOptions :: struct { }` (an opaque
|
||
null-sentinel handle) + 35 free `ufcs (self: BuildOptions, …) abi(.compiler)` decls in
|
||
`library/modules/build.sx`, each serviced by a new `comptime_vm.callBuildOptionFn` arm (dispatched from
|
||
`callCompilerFn`). **NO legacy `compiler_lib` handler** (per the full-migration direction): the 35 names are
|
||
registered in `compiler_lib.bound_fns` only so `weldedCompilerFn` accepts them, with a single bailing stub
|
||
`handleBuildOptionsAccessor` (never reached). **String lifetime:** setters dupe the arg string into the
|
||
PERSISTENT `Vm.gpa` (the Compilation allocator threaded into both `tryEval` and `runBuildCallback` — NOT the
|
||
per-eval VM arena, whose bytes die at `Vm.deinit`), so a `#run`-set path survives to post-link. Setters
|
||
write/append the duped string to the threaded `BuildConfig` (`output_path`/`bundle_path`/…, the `link_flags`/
|
||
`frameworks`/`asset_dirs` ArrayLists); string getters return the field (or `""`); bool getters compute from the
|
||
triple (`predIsMacOS`/`predIsIOS`/…, mirroring the legacy hooks); count/indexed getters read the `BuildConfig`
|
||
slices. **Dispatch routing (Option B, chosen at start):** a `#run` / const-init entry that directly calls a
|
||
compiler-domain / compiler-welded fn (`emit_llvm.entryNeedsVm`) is routed through the VM with NO legacy fallback
|
||
— regardless of the `-Dcomptime-flat` gate — so gate-OFF stays green without a legacy BuildOptions handler
|
||
(P5.7 retires the legacy interp entirely). The 5 `platform/bundle.sx` helpers that call getters
|
||
(`build_info_plist`/`embed_framework`/`android_bundle_main`/`build_android_manifest`/`compile_jni_main_sources`)
|
||
are marked `abi(.compiler)` too (they're comptime-only bundler code; without it their now-welded getter calls
|
||
trip the runtime-call gate). **Snapshots:** 37 `.ir` churned (std transitively imports build.sx → string-pool/
|
||
type-table indices shift) — regen scoped via `-Dname`; verified ONLY `.ir` changed (zero behavior-stream diffs).
|
||
**703/0 BOTH gates.** Strict sweep: the BuildOptions `compiler_call` bails are GONE (1609/1614/1615 strict-clean);
|
||
1616 now bails on `shr` (a pre-existing, separate VM gap — bitwise/shift ops `shl`/`shr`/`bit_and`/`bit_or`/
|
||
`bit_xor`/`bit_not` are unported in `comptime_vm`, surfaced now that the iOS-device bundler runs further; 1616 is
|
||
unpinned + can't JIT-run on macOS anyway). **Also (per user): swept the outdated "flat memory" terminology** —
|
||
the comptime VM is byte-addressable, ARENA-backed memory where `Addr` is a REAL host pointer, NOT a flat
|
||
contiguous address space; "flat memory"/"flat-memory" → "comptime memory" / "byte-addressable" across
|
||
`comptime_vm.zig` + the plan/checkpoint/CLAUDE docs (flag names `-Dcomptime-flat`/`SX_COMPTIME_FLAT` kept).
|
||
> **NEXT — P5.6 (ALL bundling + code signing in `default_pipeline`).** First likely sub-task: port the
|
||
> bitwise/shift ops (`shl`/`shr`/`bit_and`/`bit_or`/`bit_xor`/`bit_not`) into `comptime_vm` so the real bundler
|
||
> path runs on the VM (the 1616 `shr` gap). Then move `platform/bundle.sx`'s per-target logic to read the
|
||
> migrated `abi(.compiler)` getters + `fs`/`process` host-FFI, call `bundle()` from `default_pipeline` after
|
||
> `link` when `bundle_path()` is set, and remove the `--bundle`/`post_link_module` Zig shim.
|
||
- **P5.4 CORE — the whole build is sx-driven via `default_pipeline`; no Zig auto-emit/auto-link (2026-06-19).**
|
||
The compiler's post-IR role is now: codegen → invoke the build callback. **There is NO auto-emit / auto-link.**
|
||
Commits (all green): (1) **core** (`d178454`) — `emit_object()` is an ACTION (verify+emit via a host
|
||
`BuildHooks` vtable; `main.BuildHooksCtx`); new query primitives `build_output`/`build_target`/
|
||
`build_frameworks`/`build_flags` (read the merged `BuildConfig`); `library/modules/build.sx` imports
|
||
`compiler.sx` + defines `default_pipeline` (emit → gather c_objs → link); the compiler **force-lowers**
|
||
`default_pipeline` (well-known name, `decl.isDefaultBuildPipeline`, force-lowered after Pass 2) and
|
||
**auto-invokes** it post-codegen when no `on_build` override (`main` final fallback `invokeByName
|
||
"default_pipeline"`); the BUILD path **auto-imports `modules/build.sx`** (prepends a synthetic import node in
|
||
`compileWithTimer`) so prelude-less programs (asm tests) still get `default_pipeline`; removed the build cache
|
||
short-circuits (a future cache can live in `default_pipeline`). (2) **on_build-only** (`65ac370`) — migrated
|
||
all 9 `set_post_link_callback` callers to `on_build(cb)` (callback gains `opt: BuildOptions`); DELETED
|
||
`set_post_link_callback`. **Override semantics changed:** an `on_build` callback REPLACES the build (must
|
||
emit+link or `return default_pipeline(opt)` — delegation verified), unlike the old post-link callback that ran
|
||
AFTER the auto-link. Reworked tests: 1662 (queries) + 1664 (override+List-grow) DELEGATE to `default_pipeline`;
|
||
deleted 1661/1663 (primitives now exercised by EVERY AOT build). `sx run` (JIT) is UNTOUCHED (emits in-process,
|
||
never invokes `default_pipeline`). Benign `.ir` churn each step; **703/0 both gates.**
|
||
> **REMAINING P5.4 (the BuildOptions-surface migration — large, mechanical, dual-path, string-lifetime-sensitive;
|
||
> NOT YET DONE):** **FINAL DIRECTION (user 2026-06-19): FULL MIGRATION — NO LEGACY. Drop gate-OFF entirely;
|
||
> the VM is the SOLE evaluator; delete `interp.zig`. Migrate DIRECTLY to VM-native `abi(.compiler)` arms — NO
|
||
> legacy `compiler_lib` handlers, NO dual-path.** See `PLAN-COMPILER-VM.md` → Phase 5 steps **P5.5–P5.8** for
|
||
> the full spec. In brief:
|
||
> - **P5.5** — migrate all 36 `BuildOptions :: struct #compiler` methods → free `ufcs … abi(.compiler)` decls +
|
||
> `comptime_vm.callCompilerFn` arms (NO legacy handler). Setters dupe strings into a PERSISTENT allocator
|
||
> (thread `emit_llvm.alloc` via e.g. `BuildConfig.string_alloc`). Kills the 4 strict `compiler_call` bails.
|
||
> - **P5.6** — ALL bundling + code signing for EVERY target (macOS `.app`, iOS device/sim, Android `.apk`:
|
||
> Info.plist/codesign/provisioning/entitlements/framework-embed/AndroidManifest/javac/d8/aapt2/zipalign/
|
||
> apksigner) runs in the sx `default_pipeline` (via the migrated getters + `fs`/`process` host-FFI). Remove the
|
||
> `--bundle`/`post_link_module` Zig shim. Compiler keeps ONLY the linker primitive (Option B).
|
||
> - **P5.7** — DELETE `#compiler` + `compiler_call` op + `compiler_hooks` (Registry/HookFn) + `interp.zig`
|
||
> (Interpreter/Value/reflectTypeInfo/callExtern) + `regToValue`/`valueToReg` + the VM→legacy fallback; make
|
||
> `-Dcomptime-flat` permanent. A VM bail is ALWAYS a build diagnostic now. Re-express `define`/`make_enum` as
|
||
> sx. Land the 0141 repro; reconcile 1654.
|
||
> - **P5.8** — build `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test that
|
||
> `default_pipeline` covers all targets; add `.app` + `.apk` bundle smoke tests (no corpus coverage today).
|
||
- **P5.3 (`on_build` registrar) — the build-callback registration mechanism; callback takes `BuildOptions` (2026-06-19).**
|
||
Per the user's design: `on_build(cb)` is the build-callback registrar (a FREE fn), generalizing
|
||
`set_post_link_callback` — the callback is `(opt: BuildOptions) -> bool abi(.compiler)` and the compiler invokes
|
||
it post-codegen WITH the opaque `BuildOptions` handle. **Key simplification:** the handle is a single
|
||
null-sentinel word, so passing it sidesteps the feared fat-`BuildConfig` marshaling. Changes: VM
|
||
`callCompilerFn` `on_build` arm + legacy `handleOnBuild` (both set `post_link_callback_fn` + a new
|
||
`BuildConfig.post_link_takes_options` flag); `comptime_vm` `runEntry`→`runEntryArgs(extra)` (implicit ctx +
|
||
explicit args) + a public `runBuildCallback(..., pass_options)`; `core.invokeByFuncId`/`invokeByName` now take
|
||
`pass_options` (was an always-empty args slice); `main.zig` passes `getPostLinkTakesOptions()`; `build.sx`
|
||
`on_build` decl. Smoke test `1664-platform-on-build-callback` (AOT). Benign 37-`.ir` churn (type table +1 for
|
||
the `on_build` fn type; behavior identical — verified only `.ir` streams changed). **705/0 both gates.**
|
||
> **CONSOLIDATED REMAINING PLAN (P5.4 — from the user's 2026-06-19 direction; large + coupled + re-churns
|
||
> snapshots; the bundler has NO corpus coverage = the stream's top risk):**
|
||
> 1. **Migrate to `on_build` ONLY** — convert every `set_post_link_callback(cb)` caller (`platform/bundle.sx`
|
||
> `bundle_main`, examples 1611/1614/1615/1616, 0602/0603) to `#run on_build(cb)` with `cb: (opt:
|
||
> BuildOptions) -> bool`; DELETE `set_post_link_callback` (build.sx + compiler_lib + VM arm).
|
||
> 2. **Bundle/Android config → sx data in the default script.** The `#compiler` accessors the user flagged —
|
||
> `set_bundle_path`/`bundle_path`/`bundle_id`/`codesign_identity`/`provisioning_profile`,
|
||
> `set_manifest_path`/`keystore_path`, `jni_main_count`/`jni_main_runtime_path_at`/`jni_main_java_source_at`
|
||
> — move into the sx `BuildConfig`/default script (sx-owned data), not compiler hooks.
|
||
> 3. **`default_pipeline` + override model.** `library/modules/build.sx` ships `#run on_build(default_pipeline)`
|
||
> (the stdlib default); a user's `#run on_build(custom)` in main.sx OVERRIDES it (LAST-WINS — already the
|
||
> behavior, since registration just overwrites `post_link_callback_fn`). `default_pipeline` calls
|
||
> `emit_object`/`c_object_paths`/`link_libraries`/`link` + the sx bundler.
|
||
> 4. **REMOVE the Zig driver's auto-emit/auto-link** (`main.compileWithTimer`) — COUPLED with (3): once
|
||
> `default_pipeline` drives emit+link, the driver must stop doing them or it double-links. Riskiest piece
|
||
> (whole build/bundle path; no corpus guard → needs dedicated bundle smoke tests).
|
||
> 5. **Delete `#compiler`/`compiler_call`/`compiler_hooks`** + the S5a `build_options` once config is sx data →
|
||
> kills the 4 strict `compiler_call` bails (1609/1614/1615/1616) → strict sweep green → `interp.zig` deletable.
|
||
- **P5.2b (`link` ACTION) — the sx `link` primitive links on the VM via a host-installed vtable; build callback de-failable'd (2026-06-19).**
|
||
Phase 5's one genuine ACTION primitive: `link(objects, output, libraries, frameworks, flags, target)` (in
|
||
`library/modules/compiler.sx`). **USER DECISION this step: drop fallibility from the build callback** — so
|
||
`link` is a plain VOID primitive (no `-> !`), and a link failure BAILS on the VM → hard build error (sidesteps
|
||
the failable-tuple-return construction entirely). **The vtable:** `comptime_vm.zig` can't depend on the driver
|
||
(`core`/`main`/`target`), so `link` dispatches through a new `compiler_hooks.BuildHooks { ctx, link_fn }` that
|
||
`main.zig` installs into `BuildConfig.build_hooks` before the post-link callback. The driver side is
|
||
`main.LinkHooksCtx` (holds allocator/io/base_config/has_jni_main; its `link` adapter unions the explicit
|
||
`flags` with the CLI ones and calls `target.link(objects[0], objects[1..], …)` — the linker treats first-vs-rest
|
||
as equal inputs). **New VM readers** (inverse of `makeStringList`): `readStringList` (a `List(string)` arg →
|
||
`[][]const u8`, element bytes are views into stable comptime arena) + `readStringArg` (a `string` arg).
|
||
Registered `link` on `bound_fns` (legacy stub bails — VM-only). **Smoke test**
|
||
`examples/1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's own objects (via
|
||
`c_object_paths` + `emit_object`) into a temp output through the sx `link` primitive — and the **relinked binary
|
||
is a FUNCTIONAL executable that runs** (verified manually). Build exit 0 only if the VM-driven link succeeds;
|
||
**negative-probe verified** (bad output path → `ld` fails → `ComptimeVmBail: comptime link: linking failed`,
|
||
build exit 1 — the P5.1 VM-reason diagnostic path). **The driver still auto-links too** (P5.2b does NOT remove
|
||
the Zig driver's `target.link`; the test links to a SEPARATE temp output) — removing the auto-link + having
|
||
`on_build` drive everything is P5.3/P5.4. **704/0 both gates.**
|
||
- **P5.2 (metadata queries) — `c_object_paths` / `link_libraries` compiler primitives + the VM `List(string)` builder (2026-06-19).**
|
||
Phase 5 step 2 (the read-only slice): two `abi(.compiler)` primitives that the sx build driver will pass to
|
||
`link` — `c_object_paths() -> List(string)` (the `#import c` companion `.o`s) and `link_libraries() -> List(string)`
|
||
(the `#library` names). They live in a NEW stdlib file `library/modules/compiler.sx` (the Phase 5 home the sx
|
||
`default_build` grows into) and are serviced by `comptime_vm.callCompilerFn` reading two new `BuildConfig`
|
||
fields (`c_object_paths`/`link_libraries`) that `main.zig` forwards before the post-link callback (alongside
|
||
`binary_path`/`target_triple`/…). **Reusable new piece:** `Vm.makeStringList(table, list_ty, items)` builds a
|
||
`List(string)` in comptime memory — backing array of `string` fat pointers + the `{items,len,cap}` struct, all laid
|
||
out from the RESULT type's field offsets/types (target-aware, no hardcoded layout). To get the result type,
|
||
`invoke`/`callCompilerFn` now thread the call instruction's `ins.ty` (the only call-result-type need so far).
|
||
Legacy (`compiler_lib`) handlers for these bail loudly (`handleBuildPipelineQuery`) — they're VM-only by nature
|
||
(the post-link callback always runs on the VM since P5.1), and a `List(string)` isn't faithfully buildable in
|
||
the legacy `Value` model (0141). Registered on `bound_fns` so `weldedCompilerFn` recognizes them. **Smoke test**
|
||
`examples/1662-platform-build-pipeline-queries` (AOT + a 1-line C `#source` → exactly one C object): a post-link
|
||
callback asserts `c_object_paths().len == 1`, `items[0].len > 0`, and iterates `link_libraries()` (liveness
|
||
touch) — build exit 0 only if the VM-built list is well-formed. **Negative-probe verified** a real guard (forcing
|
||
`len != 2` → "post-link callback returned false", build exit 1). **`emit_object() -> string` ALSO landed (same
|
||
step):** a QUERY, not an action — the compiler emits the object eagerly (the Zig driver, before the callback),
|
||
so the primitive just returns the path from a new `BuildConfig.object_path` field `main.zig` forwards (no
|
||
driver vtable needed). 1662's callback now also asserts `emit_object().len > 0`. So ALL THREE query primitives
|
||
(`emit_object`/`c_object_paths`/`link_libraries`) are done; only `link` (the genuine ACTION) remains. **No unit
|
||
test for `makeStringList`** —
|
||
constructing a `List(string)` `TypeId` in the test harness needs generic instantiation; the corpus test
|
||
exercises the real stdlib type end-to-end with a non-empty list + a negative guard instead. **`emit_object` +
|
||
`link` (the ACTIONS) deferred to P5.2b** — they must replace the Zig driver's auto-emit/auto-link (not duplicate
|
||
it), so they need the driver-restructuring + a host-installed callback vtable (the VM can't depend on
|
||
`core`/`main`/`target`). **703/0 both gates** + strict JIT run clean (no `compiler_call` bail).
|
||
- **P5.1 (= 4E) — the post-link build driver runs on the VM (NO fallback); smoke test 1661 (2026-06-19).**
|
||
Phase 5 step 1: `core.invokeByFuncId` — the post-codegen / post-link callback invocation `main.zig` fires after
|
||
`target.link` — now routes the callback through the **comptime VM** (`comptime_vm.tryEval`) instead of the
|
||
legacy `Interpreter`. **REQUIRED** because the sx build driver allocates/grows `List`s, which the legacy interp
|
||
can't do at comptime (issue 0141: `struct_get: base has no fields`); the VM can. **NO fallback** (user
|
||
directive): a side-effecting post-link callback can't safely re-run on a second evaluator (double execution),
|
||
so a VM bail is a HARD build error — `error.ComptimeVmBail`, with the reason in `comptime_vm.last_bail_reason`
|
||
(now surfaced by `main.printInterpBailDiag`, which previously only read the legacy interp's `last_bail_*`
|
||
statics). `BuildConfig` (`&emitter.build_config`) + `import_sources` are threaded into the VM call. Deleted the
|
||
now-dead `flushInterpOutput` (the VM writes `out` directly via host-FFI — no buffer to flush). Non-empty `args`
|
||
rejected loudly (`error.ComptimeVmArgsUnsupported`) — the `on_build(config)` arg-passing entry arrives in P5.3.
|
||
**Verification:** a probe with a List-growing post-link callback FAILS on the pre-change legacy path
|
||
(`sx build` exit 1, `OutOfBounds (op=struct_get)`) and SUCCEEDS after the change (exit 0). Formalized as
|
||
`examples/1661-platform-post-link-vm-list` (`{ "aot": true }`): the callback grows a `List` to 3 + returns
|
||
`len == 3`; the build links cleanly (exit 0) and the binary prints `runtime main`. AOT snapshots the binary's
|
||
streams (build stdout discarded), so the VM-success is pinned via exit 0 + `runtime main` — a legacy regression
|
||
would flip the build to exit 1 and mismatch. **No corpus example fires post-link** (none had AOT sidecars; the
|
||
platform examples register a callback at `#run` time but run JIT) — so `invokeByFuncId` was previously untested
|
||
by the corpus; 1661 is the first coverage. The 4 strict `compiler_call` bails (1609/1614/1615/1616) are
|
||
UNAFFECTED — they bail at `#run configure()` on still-`#compiler` accessors (`set_bundle_path` etc.), killed by
|
||
P5.4, not here. **702/0 both gates.**
|
||
- **4B (VM-native diagnostics) — the metatype negative tests (1179/1180) render proper diagnostics under strict; strict gap-bails now ONLY `compiler_call` (2026-06-19).**
|
||
The legacy and VM both BAIL on a `define()` validation failure with an identical detail string; only the
|
||
host's STRICT rendering differed (generic "bailed on the VM (strict)" vs the proper "comptime type
|
||
construction failed: <detail>" + span the non-strict legacy path emits). Fixed: (1) aligned the VM's `define`
|
||
messages with the legacy's exact text — `comptime define():` (was `comptime define:`), and the duplicate
|
||
variant/field cases now NAME the offender via a new `failFmt` helper (`'...' duplicate variant name 'value'`).
|
||
(2) The strict type-fn path (`lower/comptime.zig`) now emits `d.addFmt(.err, span, "comptime type construction
|
||
failed: {s}", .{vm_reason})` — the SAME diagnostic as the legacy fallback, so **1179/1180 produce their exact
|
||
expected `.stderr` under strict with NO legacy interp involved**. Left the const-init/`#run` strict paths on
|
||
the "bailed on the VM" wrapper ON PURPOSE — they still carry genuine VM-gap bails (`compiler_call`), so the
|
||
burndown sweep must keep distinguishing those. **701/0 both gates.** **STRICT GAP-BAILS NOW: only the 4
|
||
`compiler_call` (1609/1614/1615/1616 → Phase 5 sx-build-pipeline)** + 1654 (a legitimate unresolvable-symbol
|
||
diagnostic — an asm global called at comptime; the legacy can't resolve it either; reconciles to VM wording
|
||
at the 4F flip). So: BuildOptions/Phase 5 is the ONLY thing between the VM and a green strict sweep.
|
||
- **`out` is now a PLAIN SX FUNCTION (libc `write`), NOT a builtin — VM handles it via host-FFI; `trace_resolve` ported; 0522 fixed (2026-06-19).**
|
||
Per user: removed the `out` `#builtin` entirely. `library/modules/std/core.sx` now defines
|
||
`libc_write :: (fd, [*]u8, usize) -> isize extern libc "write"` + `out :: (str: string) { libc_write(1,
|
||
str.ptr, xx str.len); }`. Deleted `BuiltinId.out` (`inst.zig`), the `resolveBuiltin` "out" mapping
|
||
(`call.zig`), the sema builtins-list entry (`sema.zig`), and BOTH `.out` arms (`interp.zig` buffered-append,
|
||
`ops.zig` LLVM `write` lowering). **At comptime `out` runs through the evaluator's host-FFI** (the VM's
|
||
dlsym `write` path / the interp's extern call) — so the VM HANDLES `out` with NO special arm. Benign prelude
|
||
`.ir` churn (`[*]u8` interned earlier + `@out`→`@write` + the `out` fn body) → regen'd 54 `.ir` snapshots
|
||
(verified: only string-table renumber + the intended decl/fn-body change; zero stdout/exit changes).
|
||
**This UNMASKED two latent VM gaps the `out`-bail was hiding** (the VM now runs past `out`):
|
||
(1) **`trace_resolve`** (1035) — PORTED to the VM (`comptime_vm.zig`): unpack the `(func_id<<32|offset)`
|
||
comptime frame, resolve func name + `file:line:col` + source line via a **`source_map` now threaded into the
|
||
VM** (new `tryEval` param, `&import_sources` from emit_llvm), build the `{file,line,col,func,line_text}`
|
||
`Frame` struct in comptime memory (`makeStringValue`/`writeField`/`fieldOffset`). (2) **0522** (bare-pack
|
||
`[]Any`) — was a CRASH (`reflectArgTypeId` `@intCast` of a garbage word) → hardened to a loud bail
|
||
(`typeIdxOf` checked cast; the VM must never panic). ROOT CAUSE: after the 0143 fix `$args` materializes as
|
||
`[]type_value` (8-byte), but the example declared `describe(args: []Any)` (16-byte) → every element past the
|
||
first read at the wrong stride; the legacy's loose Value model tolerated it, the byte-accurate VM didn't. The
|
||
bare-pack elements ARE `Type`s, so the fix is the honest type — `describe(args: []Type)` (output identical).
|
||
**Result: `out`/`trace_resolve`/the 0522 pack-reflection all run VM-HANDLED under strict** (0613/1035/0522/1038
|
||
no longer bail). **701/0 BOTH gates + full suite.** (Build-pipeline relevance: the sx `default_build` driver
|
||
uses `out` for diagnostics — now VM-native; no compiler `out` builtin to special-case.)
|
||
**THEN `interp_print_frames` ported to the VM too** (1034): unlike `out` it needs the live evaluator
|
||
call-chain, so it's a VM arm (mirrors legacy `printInterpFrames`) — walks `call_stack` (skips the last frame),
|
||
writes ` at <name>` lines straight to fd 1 (consistent with `out`'s direct `write`). 1034 matches; 701/0.
|
||
**STRICT DELETION-GATE NOW DOWN TO 7 (all known categories):** `compiler_call` (4 — 1609/1614/1615/1616, the
|
||
still-`#compiler` BuildOptions accessors → Phase 5 sx-build-pipeline) · VM-diagnostic negatives (2 —
|
||
1179/1180, the `define` bail IS the expected outcome → **4B**: surface as a proper build diagnostic) ·
|
||
target-specific dlsym (1 — 1654, an asm global called at comptime; legacy can't resolve it either → a clean
|
||
diagnostic, not a bug). EVERY pure + side-effect op bail is cleared.
|
||
- **DESIGN PIVOT (2026-06-18, user) — DRIVE THE BUILD PIPELINE FROM SX; the 37-hook BuildOptions port is dead.**
|
||
Trigger: porting each `BuildOptions` accessor to an `abi(.compiler)` fn that delegates to a `compiler_hooks`
|
||
hook just re-encodes sx-level logic (setters/getters, `is_macos` triple-matching, list appends) as compiler
|
||
hooks — they need NOTHING from the compiler but the `BuildConfig` state. So instead: **`BuildConfig` becomes
|
||
plain sx data** (ordinary struct, sx-owned, no `#compiler`/hooks/shared-state/weld), and the **build pipeline
|
||
is an sx program** — the logical end of "bundling lives in sx". The compiler shrinks to a few `abi(.compiler)`
|
||
PRIMITIVES taking EXPLICIT args (`emit_object() -> !string`, `link(objects, output, libs, fws, flags, target)
|
||
-> !`, metadata queries) + an `on_build : (BuildConfig) -> ! abi(.compiler)` slot (stdlib default
|
||
`default_build`; user overrides via `#run on_build = build;`). **Chosen boundary: Option B** (compiler keeps
|
||
the Zig linker; sx owns config+orchestration+bundle); Option A (sx shells `cc`/`ld`) is a later refinement.
|
||
**NO bool** — failures are the error channel (`-> !`); VERIFIED on the current build: void `#run`, `-> !`/`-> !E`
|
||
failable `#run`, and a `raise` at `#run` fails the build with a return trace (+ suggests `#run … catch (e){…}`).
|
||
`on_build` GENERALIZES today's `post_link_callback_fn` (assignable typed global w/ default, vs a setter).
|
||
**Full design + step plan in `PLAN-COMPILER-VM.md` → Phase 5.** **S5a (below) is a green intermediate that the
|
||
sx-pipeline replaces wholesale** (don't extend it; P5.4 deletes `build_options`/`set_post_link_callback` +
|
||
all `#compiler`). **NEXT — P5.1 (= 4E):** route the post-codegen / `on_build` invocation through the VM
|
||
(`core.invokeByFuncId` → VM), REQUIRED because the sx driver allocates `List`s and the legacy interp can't
|
||
(0141, VERIFIED: comptime `List` growth works on the VM, fails on legacy with `struct_get: base has no
|
||
fields`). Add dedicated bundle smoke tests (no corpus coverage today). Both gates **701/0**.
|
||
- **S5a DONE — `build_options` + `set_post_link_callback` migrated off `#compiler` onto `abi(.compiler)`; `BuildConfig` threaded into the VM (2026-06-18).**
|
||
The corpus-covered slice of the BuildOptions migration. (1) `comptime_vm.zig` — `Vm.build_config: ?*BuildConfig`,
|
||
threaded via a new `tryEval` param (`&self.build_config` from emit_llvm's `#run`/const-init sites; `null` at
|
||
lowering-time type-fn). (2) Two `callCompilerFn` arms: `build_options` (returns the null-sentinel handle) +
|
||
`set_post_link_callback` (reads the cb `func_ref`, stores `post_link_callback_fn` on the threaded `BuildConfig`).
|
||
(3) `compiler_lib.zig` — matching legacy `handleBuildOptions`/`handleSetPostLinkCallback` (gate-OFF dual path).
|
||
(4) `build.sx` — `build_options :: () -> BuildOptions abi(.compiler);` and `set_post_link_callback` EXTRACTED
|
||
from the `struct #compiler` as a free `ufcs (…) abi(.compiler)` (so `opts.set_post_link_callback(cb)` still
|
||
resolves via UFCS); the other ~38 BuildOptions methods stay `#compiler` for now. (5) Registrars/callbacks that
|
||
call these are now compiler-domain: `platform/bundle.sx` `bundle_main :: () -> bool abi(.compiler)`, and the
|
||
six platform examples' `configure`/`configure_build` registrars marked `abi(.compiler)`; 0602/0603 reworked
|
||
the same way. **KEY learning:** every example transitively imports `build.sx` via the prelude, so the
|
||
`set_post_link_callback` method→free-function change is BENIGN `.ir` churn (declaration renumber + global
|
||
`@str`/`@tag.str` suffix shift) in all 37 examples that have `.ir` snapshots — verified line-by-line that NO
|
||
instruction/control-flow/payload changed (only auto-numbered global-name suffixes), then regen'd those 37
|
||
snapshots scoped with `-Dname`. **Strict-VM `compiler_call` bail set dropped 6→2:** 0602/0603/1604/1611 now
|
||
fully VM-HANDLED; 1609/1615 still bail on the *other* (still-`#compiler`) BuildOptions methods they use →
|
||
**S5b** (migrate the remaining ~38 setters/getters). **701/0 BOTH gates + all unit tests.**
|
||
- **S3 DONE — emit_llvm skips BODIED `abi(.compiler)` (compiler-domain) functions; comptime-only calls emit `undef` (2026-06-18).**
|
||
A BODIED `abi(.compiler)` function is a user compiler-domain function (post-link callback / compiler helper):
|
||
the comptime evaluator runs its sx body, but it NEVER runs in the binary, so the backend skips it. Changes:
|
||
(1) IR `Function` gained `is_compiler_domain: bool` (`inst.zig`). (2) `decl.zig` — new `fnIsBodilessCompiler`
|
||
splits the API surface (bodiless → declare-only, `compiler_welded`, no implicit ctx — the S1 behavior) from a
|
||
bodied `abi(.compiler)` function (lowers its body for VM eval; flagged `is_compiler_domain` + `is_comptime`;
|
||
gets normal implicit-ctx). The four S1 guards now gate on `fnIsBodilessCompiler` not `fd.abi == .compiler`.
|
||
(3) `emit_llvm.zig` — Pass 2 skips `is_compiler_domain` bodies; Pass 1 declares them EXTERNAL-linkage (an
|
||
internal empty decl fails LLVM verification). (4) **KEY** `ops.zig` `emitCall` — a call to a comptime-only
|
||
callee (`compiler_welded` OR `is_compiler_domain`) from a dead comptime body now emits `undef` instead of a
|
||
real `call`; the runtime-call gate covers both. Without the undef, an AOT `sx build` left an undefined
|
||
`_double`/`_intern` symbol — this ALSO fixed a pre-existing, untested AOT breakage of the bodiless
|
||
compiler-API examples (the corpus runs them JIT). Diagnostic reworded "compiler-library" → "compiler-domain"
|
||
(1185 snapshot regen'd). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run`
|
||
const → 84, absent from the binary via `nm`, JIT + AOT both run). **701/0 both gates + all unit tests.**
|
||
**NEXT: S4** — an `abi(.compiler)` function-TYPE param (`cb: () -> bool abi(.compiler)`) flags the bound
|
||
function compiler-domain (so a plain `bundle_main :: () -> bool { … }` becomes compiler-domain when passed to
|
||
`set_post_link_callback`). Then S5 (BuildOptions migration + delete `#compiler`/`compiler_call`/`compiler_hooks`).
|
||
- **S1+S2 DONE — `abi(.compiler)` replaces `abi(.zig) extern compiler` + `#library "compiler"` (clean cutover, no legacy path) (2026-06-18).**
|
||
Per the design pivot below, and the user's "no legacy paths": REMOVED the `.zig` ABI variant entirely (`ast.ABI`
|
||
is now `{ default, c, compiler, pure }`) and made `abi(.compiler)` the sole spelling for a compiler-domain /
|
||
compiler-API function — the ABI alone marks it, no `extern <lib>`, no fake `#library "compiler"`. Changes:
|
||
(1) `ast.zig` — `.zig` → `.compiler` (doc rewritten). (2) `parser.zig` — `parseOptionalAbi` accepts `.compiler`
|
||
(drops `.zig`); a **bodiless `abi(.compiler)` decl** (ends in `;`, no `extern`) is now accepted — synthesizes
|
||
the empty-block placeholder like an `extern` import (the Zig/VM handler is the impl). (3) `decl.zig` —
|
||
`weldedCompilerFn` keys off `fd.abi == .compiler` + export-list membership (no `extern_lib == "compiler"`
|
||
check); a bodiless `abi(.compiler)` decl lowers extern-like (`is_extern_decl`, and the two body-lowering paths
|
||
`lowerFunction`/`lazyLowerFunction` skip it) so it is declared-not-defined; `funcWantsImplicitCtx` returns
|
||
false for `abi == .compiler` (an implicit `__sx_ctx` prepend would shift args and break the handler arity —
|
||
this was the live bug surfaced + fixed). (4) `type_resolver.zig` — the function-TYPE CC switch handles
|
||
`.compiler` (sx-default CC). (5) Migrated ALL 8 compiler-API examples (0626/0628/0629/0630/0631/0633 + the
|
||
1184/1185 negatives) `… abi(.zig) extern compiler;` → `… abi(.compiler);` and deleted every `compiler ::
|
||
#library "compiler";` line; regen'd the 1184 stderr snapshot (new "not a function exported by the compiler"
|
||
wording + shifted line). (6) Updated the two parser unit tests. **All 8 examples run HANDLED on the strict VM
|
||
with byte-correct output; 1184 (unexported name) + 1185 (runtime call) still error cleanly; gate-OFF legacy
|
||
still works.** **700/0 BOTH gates + all unit tests.** NOTE: the general `#library`/`extern <lib>` PARSE paths
|
||
stay (used by `libc :: #library "c"` etc.) — only the compiler-API's USE of them is gone. `compiler_lib.lib_name`
|
||
+ the `main.zig` dlopen-skip for a "compiler" lib are now dead defensive code (harmless; a `#library "compiler"`
|
||
is just meaningless now). The struct-`abi(...)` parse slot is vestigial (weld stripped) — parse-only test kept.
|
||
**NEXT: S3** — emit_llvm skips BODIED `abi(.compiler)` functions (Pass 2, like `is_extern`); thread an
|
||
`abi(.compiler)` flag onto the IR `Function` and refine the three "today every `abi(.compiler)` fn is bodiless"
|
||
guards in `decl.zig` (marked with `S3 NOTE`) to allow a bodied callback's body to lower for VM eval while NOT
|
||
emitting it. Then S4 (callback-param propagation) + S5 (BuildOptions migration).
|
||
- **DESIGN PIVOT (2026-06-18, user) — `abi(.compiler)` is the compiler-domain ABI; DROP the fake `#library "compiler"`.**
|
||
Supersedes both the `abi(.zig) extern compiler` + `#library "compiler"` binding mechanism AND the previous
|
||
"runtime-reachability gating" idea for the BuildOptions blocker (entry below). **The unifying concept:** a
|
||
function is *compiler-domain* (runs in the comptime evaluator, NEVER in the shipped binary) because its **ABI
|
||
says so** — `abi(.compiler)` — not because it's "extern" to an imaginary library. One annotation covers
|
||
BOTH roles:
|
||
1. **Compiler-API surface** (`intern`, `text_of`, `find_type`, `declare_type`, `register_type`,
|
||
`build_options`, `set_post_link_callback`, …): bodiless `abi(.compiler)` decls (the Zig/VM handler IS the
|
||
impl). Replaces `… abi(.zig) extern compiler;` + the `compiler :: #library "compiler";` line — both GO AWAY.
|
||
2. **User compiler-domain functions** (post-link callbacks like `platform.bundle.bundle_main`): BODIED
|
||
`abi(.compiler)` functions. emit_llvm does NOT lower them (skip in Pass 2, like `is_extern`); the comptime
|
||
VM/interp evaluates them. A callback PARAM type carries it too — `set_post_link_callback(self, cb: () -> bool
|
||
abi(.compiler))` — so the bound function is flagged compiler-domain.
|
||
**Why this dissolves the BuildOptions blocker:** the welded-call enforcement (`ops.zig` `emitCall`) only fired
|
||
because comptime-only callback bodies (`bundle_main`, 0602's `configure`) were being LLVM-emitted. A bodied
|
||
`abi(.compiler)` function is never emitted → its `build_options()`/`binary_path()` calls never reach `emitCall`
|
||
as runtime code → no enforcement, no undefined-symbol risk. **1185 stays correct**: `main` is an ordinary
|
||
runtime fn (not `abi(.compiler)`) calling a compiler-domain fn → still a clean build-gating error. (The
|
||
registrar half is independently fine via the idiomatic `#run { … }` block — the welded calls sit in the
|
||
`is_comptime` `__run` wrapper; 0602/0603 only tripped via an intermediate `configure()`, a test-shape artifact.)
|
||
**Staged plan (each its own step, both gates green):**
|
||
- **S1 — introduce `abi(.compiler)`** as a new `ABI` variant that marks a function `compiler_welded` (export-list
|
||
checked) WITHOUT requiring `extern compiler`/`#library`. Add it ALONGSIDE the existing `.zig extern compiler`
|
||
path so migration is incremental; prove with one example (0626 → `abi(.compiler)`). (`.zig` is a misnomer —
|
||
"we don't really have a zig abi"; it becomes `.compiler`, ultimately replacing `.zig` once all callers move.)
|
||
- **S2 — migrate the rest of the compiler-API decls** (0628–0633, 1184/1185) to `abi(.compiler)`; drop the
|
||
`#library "compiler"` lines; regen snapshots (the 1184 unexported-name + 1185 runtime-call diagnostics must
|
||
stay red with refreshed wording). Then retire the `.zig extern compiler` parse path + `#library "compiler"`.
|
||
- **S3 — emit_llvm skips bodied `abi(.compiler)` functions** (Pass 2 `continue`, like `is_extern`); thread the
|
||
`abi(.compiler)` flag onto the IR `Function`. Prove a bodied compiler-domain function isn't emitted.
|
||
- **S4 — callback-param propagation**: an `abi(.compiler)` function-type PARAM flags the bound function
|
||
compiler-domain.
|
||
- **S5 — BuildOptions migration** (now unblocked): `build_options`/`set_post_link_callback`/… become
|
||
`abi(.compiler)` (+ VM `callCompilerFn` arms / legacy `compiler_lib` handlers; `BuildConfig` threaded into the
|
||
VM — the bundler 4E shares this); callbacks declared/typed `abi(.compiler)`; delete `#compiler`/`compiler_call`/
|
||
`compiler_hooks` Registry. Then **4E** bundler on the VM.
|
||
**Reusable facts from the reverted attempt:** only `build.sx` uses `#compiler`; VM dual-path bail-to-fallback
|
||
means the VM needs only corpus-covered fns; UFCS on a free fn needs the `ufcs` marker (composes with the ABI
|
||
annotation); the binding mechanism currently lives in `decl.zig` `weldedCompilerFn` (keys off `extern_lib ==
|
||
"compiler"` — S1 makes it key off `abi == .compiler`). Mechanism files: `ast.zig` (`ABI` enum), `parser.zig`
|
||
(`parseOptionalAbi` + the extern-compiler postfix), `decl.zig` (`weldedCompilerFn`), `compiler_lib.zig`
|
||
(export list), `comptime_vm.zig` (`callCompilerFn`), `emit_llvm.zig` (Pass-2 skip), `ops.zig` (`emitCall` gate).
|
||
- **Phase 4 — BuildOptions→`abi(.zig) extern compiler` migration ATTEMPTED, then REVERTED; BLOCKER found: the comptime-only welded-call enforcement (2026-06-18).**
|
||
Scoped an incremental slice (migrate only the corpus-covered `build_options()` + `set_post_link_callback`,
|
||
leaving the 38 bundler accessors on `#compiler` → VM bails → legacy fallback). Built it end-to-end:
|
||
threaded `BuildConfig` into the `Vm` (`tryEval` gained a `?*BuildConfig` param, passed `&self.build_config`
|
||
from emit_llvm's `#run`/const-init sites); added `callCompilerFn` arms + legacy `compiler_lib` bound-handlers
|
||
for both; rewrote `build.sx` (`build_options` → `abi(.zig) extern compiler`; extracted `set_post_link_callback`
|
||
out of the `struct #compiler` as a free `ufcs (...) abi(.zig) extern compiler` fn so `opts.set_post_link_callback(cb)`
|
||
still resolves via UFCS; added `compiler :: #library "compiler";`). All COMPILED and the welded dispatch
|
||
fired. **BLOCKED at LLVM emission, NOT a bug — a design limitation the migration surfaces:** a
|
||
`compiler_welded` call inside a NON-`is_comptime` function is a hard build-gating error (`ops.zig`
|
||
`emitCall`, the Phase-1 enforcement guarding genuine runtime misuse — example 1185). But the post-link
|
||
callback idiom calls comptime-only-API functions (`build_options()`, `binary_path()`, `bundle_path()`, …)
|
||
**inside callback bodies** (`platform/bundle.sx`'s `bundle_main :: () -> bool`, and 0602's `configure`) that
|
||
run ONLY at comptime (post-link interp/VM) yet are still LLVM-emitted as real `() -> bool` bodies. The OLD
|
||
`#compiler`/`compiler_call` path emitted those as dead `undef` (`emitCompilerCall`), so no error; the welded
|
||
enforcement instead halts the build, and it CANNOT distinguish a dead comptime-reachable body from genuine
|
||
runtime use (1185, reachable from `main`) without runtime-reachability analysis. **Reverted the whole attempt**
|
||
(kept only the green pure-ops work); both gates back to **700/0**. **THE DECISION the next session must make
|
||
FIRST (before any BuildOptions migration):** how to emit a welded call in a comptime-only-but-LLVM-emitted
|
||
function. Recommended path **A — runtime-reachability gating:** in `emit_llvm`, mark functions reachable from
|
||
runtime roots (`main` / exported runtime fns); a welded call in an UNREACHABLE function emits `undef` (dead,
|
||
like `compiler_call` did) instead of erroring, while a reachable one still errors (1185 stays red). This is
|
||
also the right foundation for eventually NOT emitting comptime-only bodies at all. Rejected: (B) marking
|
||
callbacks `is_comptime` — can't statically identify which `func_ref`s become post-link callbacks; (C) blanket
|
||
softening to `undef` — would silently swallow genuine runtime misuse (1185). **Other migration facts confirmed
|
||
this attempt (reuse next session):** only `build.sx` uses `#compiler` (the `issues/*.md` hits are doc text);
|
||
the VM dual-path bail-to-fallback means the VM needs only the corpus-covered fns, the 38 bundler accessors can
|
||
ride legacy; UFCS on a free fn requires the `ufcs` marker, which composes with `abi(.zig) extern compiler`;
|
||
`build.sx` must declare `compiler :: #library "compiler";`. Do the reachability fix as its OWN step (verify
|
||
1185 still errors + a comptime-only-body welded call now emits clean), THEN redo the BuildOptions slice on top.
|
||
- **Phase 4 burndown — three PURE comptime ops ported (`error_tag_name_get` + `global_addr` + `type_is_unsigned`); `interp_print_frames` correctly DEFERRED (2026-06-18).**
|
||
Also ported `type_is_unsigned` (a `BuiltinId` via `callBuiltinVm`): resolves the queried `TypeId` the
|
||
same way as `type_name` (a `.type_value` word, or an Any box `{tag@0,value@8}` whose tag IS the boxed
|
||
value's type) then returns `table.isUnsignedInt(tid)`. Extracted the shared resolution into a
|
||
`reflectArgTypeId` helper (VM-native `Value.reflectTypeId` mirror) so `type_name` + `type_is_unsigned`
|
||
can't drift. MATCH-verified by a new VM unit test (`type_is_unsigned(u32) - type_is_unsigned(i64) == 1`).
|
||
Strict sweep: 0600 `type_is_unsigned`→`out` (now its only remaining bail); no `type_is_unsigned` bails
|
||
remain in the corpus. **With this, all PURE comptime ops are ported** — the remaining strict bails are
|
||
side-effect (`out`/`interp_print_frames`), `compiler_call` (the BuildOptions migration), VM diagnostics
|
||
(1179/1180), and `#insert`/bundler.
|
||
Ported two side-effect-free ops onto the VM (`comptime_vm.zig` exec switch): (1) `error_tag_name_get`
|
||
— a runtime tag-id word → its name string via `table.getTagName` + `makeStringValue` (uses the table,
|
||
not the module, so it's unit-testable; `self.table == &module.types`); (2) `global_addr` — name-matches
|
||
`__sx_default_context` and returns the already-tested `materializeDefaultContext` Addr (an aggregate
|
||
value IS its address, so a downstream `load` sees the materialised Context), bailing for any other
|
||
global exactly like legacy. **MATCH verification:** `error_tag_name_get` locked in by a new VM unit
|
||
test (tag id → `"Bad"`, via `regToValue`); `global_addr` proven by the strict sweep (0600's first bail
|
||
moved past it) and reuses `materializeDefaultContext`, already exercised by every implicit-ctx comptime
|
||
call on the VM. **KEY CORRECTION to the handover's "three PURE ops" plan:** `interp_print_frames`
|
||
(1034) is NOT pure — it WRITES the comptime call-frame chain to the build output, a side effect in the
|
||
SAME bucket as `out` (the VM has no output buffer; output is direct-write, so a print-then-bail
|
||
double-prints under the legacy fallback). It must land atomically in the FINAL `out`/strict-default
|
||
step, NOT now. **Strict-sweep burndown:** 1035 `error_tag_name_get`→`out`; 0600 `global_addr`→
|
||
`type_is_unsigned` (a NEW pure-op bail surfaced — still a known pure op, next to port); 1034 stays at
|
||
`interp_print_frames` (deferred, as it should). Also fixed the stale `comptime_vm.zig` header comment
|
||
(it still said "bump/stack allocator"; the memory model is an ARENA of stable host allocations since
|
||
4D.0). **700/0 BOTH gates + all unit tests.** On `reify`.
|
||
- **Phase 4 burndown — issue 0143 FIXED (pack-as-`[]Type` stride) + regression test (2026-06-18).**
|
||
Root cause was a stale consequence of the `.type_value` migration: `buildPackSliceValue`
|
||
(`lower/pack.zig`) materialized a bare `$<pack>` `[]Type` slice as `[]Any` (16-byte elements) while
|
||
`const_type` now yields an 8-byte `.type_value` and `[]Type` resolves to `[]type_value` — so 8-byte
|
||
words sat in 16-byte slots and an 8-byte-stride reader got `[t0, pad, t1, …]`. Fixed by building the
|
||
array+slice as `.type_value` (8 bytes). Removed the stopgap `type_name` `.unresolved` guard (its
|
||
whole reason is gone; dropping it keeps any future stride bug VISIBLE as wrong output rather than a
|
||
silent fallback). Sibling `materialisePackSlice` checked — it genuinely boxes values into `[]Any`
|
||
(correct, not the same bug). Regression test `examples/0525-packs-pack-as-type-slice-arg`. **700/0
|
||
both gates.** 0114 (and 0521/0522/0524) now bail ONLY at `out` (the deferred end-state op) — the
|
||
type bug is gone. issue 0143 RESOLVED.
|
||
- **Phase 4 burndown — switch_br + type_name ported; issue 0143 filed; KEY sequencing insight: `out` is end-state-only (2026-06-18).**
|
||
Ported two PURE comptime ops (`379ed05`): `switch_br` (i64-discriminant multi-way branch — enum/error
|
||
tag or `.type_value` index) and `type_name` (Type value / Any box → `table.typeName`, with an
|
||
`.unresolved`-bail guard). Correct in isolation; 0520–0524 run GREEN under strict. **Two blockers found:**
|
||
1. **issue 0143 (FILED, OPEN) — pack-as-`[]Type` stride mismatch.** A `..$args` pack forwarded as a
|
||
`[]Type` ARGUMENT across a call is backed by a `[N x Any]` (16B) array but viewed as `[]type_value`
|
||
(8B) → half-stride reads (`[i64 <unresolved> string]` vs legacy `[i64 string bool]`). A LOWERING bug
|
||
the legacy's Value model masks; the byte-accurate VM exposes it. Blocks `examples/0114` from running
|
||
HANDLED. **Per CLAUDE.md: filed, NOT worked around** (the `type_name` `.unresolved` guard just makes
|
||
the VM decline rather than emit garbage). Repro + fix-prompt in `issues/0143-…md`.
|
||
2. **`out` (comptime print) is an END-STATE op — it cannot land while the fallback exists.** Under the
|
||
legacy fallback, an eval that prints via `out` then BAILS double-prints (the VM wrote to fd 1, then
|
||
legacy re-runs the whole eval — no rewind). 0114 demonstrated it. So a direct-write `out` is only
|
||
safe once the fallback is GONE (strict-by-default). **Revised ordering:** land the PURE ops
|
||
(switch_br/type_name/type_is_unsigned/error_tag_name_get/global_addr/interp_print_frames) + the
|
||
BuildOptions migration + #insert + bundler FIRST; then in the FINAL step flip strict-to-default
|
||
(removing the fallback) AND add `out` together — at which point every `out`-using example flips
|
||
atomically with deletion. (Most of the gap-list examples print, so they stay on fallback until that
|
||
final flip — that's expected, not a regression.) 699/0 both default gates.
|
||
- **Phase 4 — STRICT no-fallback mode (the interp-retirement enumeration gate) + full gap list (2026-06-18).**
|
||
Added `-Dcomptime-flat-strict` / env `SX_COMPTIME_FLAT_STRICT` (implies `comptime_flat`): at all
|
||
THREE comptime sites (type-fn in `lower/comptime.zig`, const-init + `#run` in `emit_llvm.zig`) a VM
|
||
bail becomes a build-gating error naming the reason INSTEAD of falling back to legacy. This forces
|
||
every comptime eval onto the VM so the complete gap set is enumerable in one sweep; when the corpus
|
||
is green under strict mode AND every example MATCHES legacy, the VM handles everything and
|
||
`interp.zig` can be deleted (4F). Default behaviour unchanged — **699/0 both default gates**.
|
||
(Fixed a wiring bug: the type-fn site's local `comptime_flat` didn't include the strict flag, so
|
||
every type-fn falsely reported `<unknown>`; now strict implies flat there too.)
|
||
**THE DELETION CHECKLIST (19 strict bails, swept via `SX_COMPTIME_FLAT_STRICT=1` over examples+issues;
|
||
0103/0800 "WRONG" were false positives — raw heap-pointer addresses the corpus normalizes):**
|
||
- `switch_br` (5): 0114, 0521, 0522, 0524, 1035 — port the type-category multi-way branch (trivial
|
||
jump). **CAUTION:** porting it (+`type_name`) UNMASKS a silent-wrong in 0114 — a `[]Type` slice
|
||
materialized when a pack (`$args`) is passed ACROSS A CALL reads its `string` element as
|
||
`<unresolved>`. Must fix that VM pack-Type-materialization bug, not just add the op.
|
||
- `compiler_call` (6): 0602, 0603, 1604, 1609, 1611, 1615 — the **BuildOptions → `abi(.zig) extern
|
||
compiler`** migration (delete `#compiler`/`compiler_call`; thread `BuildConfig` into the VM). Big.
|
||
- `out` (2): 0613, 1038 — comptime print. Direct write to fd 1, BUT only safe when the WHOLE eval is
|
||
VM-handled (a print-then-bail double-prints under the legacy re-run — 0613). Flip atomically.
|
||
- `type_name` (1): 0520 — reflection reader (`.type_value` word / Any-box tag → `table.typeName`).
|
||
- `global_addr` (1): 0600 — only `&__sx_default_context` is materialised (mirror legacy).
|
||
- `interp_print_frames` (1): 1034 — return-trace frame printing.
|
||
- VM-native diagnostics (4B) (2): 1179, 1180 — NEGATIVE tests; the VM bail (`define: enum has no
|
||
variants` / `duplicate variant name`) IS the expected outcome → must surface as the proper
|
||
build-gating diagnostic, not the generic strict error.
|
||
- dlsym not found (1): 1654 — a target-specific `extern` (asm global) called at comptime; likely a
|
||
legitimately-unresolvable case → confirm it stays a clean diagnostic.
|
||
**Sweep command:** `SX_COMPTIME_FLAT_STRICT=1 ./zig-out/bin/sx run <ex>` per example, diff vs legacy;
|
||
a strict bail prints `... bailed on the VM (strict, no fallback): <reason>`.
|
||
- **Phase 4D.2 (VM plan) — extern SLICE/string args (→ NUL-terminated `char*`) + float guards (2026-06-18).**
|
||
Extracted `marshalExternArg`: a scalar/pointer WORD passes verbatim (a `cstring` arg already works
|
||
as a pointer word via 4D.1); a `string`/slice `{ptr,len}` fat pointer is copied into a
|
||
NUL-terminated arena buffer and its `char*` passed (mirrors legacy `marshalExternArg` — what the
|
||
bundler's `popen(cmd: [:0]u8, …)` needs). Added FLOAT guards on args AND returns: floats are
|
||
`kindOf == .word` but the host_ffi trampolines have no float variant, so they bail loudly rather
|
||
than miscall through an integer register (the legacy interp doesn't support float FFI either, so
|
||
parity holds — no corpus float-FFI example exists). New example `0637-comptime-extern-slice-arg`
|
||
(`#run strlen("hello, world")` with a `[:0]u8` param → 12) runs **HANDLED on the VM**, byte-matching
|
||
legacy. **699/0 BOTH gates.** On `reify`. The FFI escape is now complete for scalar/pointer/cstring/
|
||
slice args + scalar/pointer returns — enough for the bundler's libc surface. **Next (4D.3):**
|
||
`compiler_call` (#compiler hooks — 0602/0603), the last legacy-only role besides #insert/bundler.
|
||
- **Phase 4D.1 (VM plan) — general host-FFI escape: the VM calls any extern libc fn via dlsym + host_ffi (2026-06-18).**
|
||
Replaced the "extern not ported → bail" stub in `Vm.invoke` with `callHostExtern`: resolve the
|
||
symbol via `host_ffi.lookupSymbol` (dlsym RTLD_DEFAULT) and dispatch through the `host_ffi`
|
||
trampolines, exactly like the legacy `interp.callExtern`. **Marshalling is now trivial because
|
||
`Addr` is a real host pointer (4D.0):** every WORD-kind arg passes as `usize` verbatim — a
|
||
scalar's bits OR a pointer, no translation — and a pointer return is a valid `Addr`. Picks
|
||
`callPtrRet` (void*-ABI) for pointer-ish returns, `callIntRet` (i64-ABI) otherwise; honors
|
||
variadic (`is_variadic and args > fixed`). Non-word (aggregate/string/float) args+returns bail
|
||
loudly (no silent miscall — 4D.2 adds NUL-term cstring marshalling + float). NOT per-builtin: ONE
|
||
general mechanism for all externs. New example `0636-comptime-extern-libc` (`#run toupper(97)`/
|
||
`tolower(90)` fold to 65/122) runs **HANDLED on the VM**, output byte-matching legacy. (`abs`
|
||
doesn't dlsym-resolve on macOS — a compiler builtin — and the VM fails identically to legacy,
|
||
confirming parity.) **698/0 BOTH gates** (one new example). On `reify`. **Next (4D.2):**
|
||
string/aggregate extern args (string→NUL-term cstring) + float args/returns, then `compiler_call`
|
||
(#compiler hooks, 4D.3).
|
||
- **Phase 4D.0 (VM plan) — comptime VM memory = an ARENA of stable host allocations; `Addr` = real host pointer (2026-06-18).**
|
||
Replaced the growable `ArrayList(u8)` flat buffer (which reallocs/MOVES on growth) with a
|
||
`std.heap.ArenaAllocator`: each `allocBytes` is a separate arena allocation that never moves and
|
||
is freed wholesale on `deinit` (no per-object free, no cap, no fixed buffer). **`Addr` is now the
|
||
allocation's absolute host pointer** (`@intFromPtr`), not an offset — so a comptime pointer and
|
||
an FFI-returned host pointer are the SAME kind of value, and the FFI bridge (4D.1) can pass them
|
||
to/from libc with ZERO translation and no per-call pinning (the original moving-buffer hazard is
|
||
gone by construction). `Machine.readWord/writeWord/bytes` deref the absolute pointer directly,
|
||
keeping the null-check bail (the malformed-IR / null-deref safety contract). Dropped the
|
||
offset-based upper-bounds check (can't bound an absolute pointer; the `Frame.bad_ref` guard still
|
||
catches the dominant malformed-IR vector) and the test-only `mark`/`reset` (the arena has no
|
||
cheap reset-to-mark; the VM never used them outside tests). Decision rationale (user): use a
|
||
GPA-like allocator, no artificial buffer limits. **697/0 BOTH gates + all unit tests** (rewrote
|
||
the two Machine tests: null-deref bail + arena-stability-across-grows). Pure refactor, no
|
||
comptime behavior change. **Next (4D.1):** extern-call dispatch in `Vm.invoke` — marshal args
|
||
(scalars by value, pointers as the host pointer they already are), call via `host_ffi`
|
||
trampolines, return scalars/pointers; a new `#run` libc example as the corpus guard.
|
||
- **Phase 4A.1 (VM plan) — `box_any`/`unbox_any` on the VM + `.any` as a 16-byte aggregate (2026-06-18).**
|
||
Ported the Any-boxing conversion pair: `box_any` allocates the 16-byte `{ type_tag@0, value@8 }`
|
||
box (tag = source TypeId index, matching the legacy comptime interp), writing a word source's
|
||
scalar via `writeField(source_type)` (so f32 round-trips) or an aggregate source's comptime
|
||
ADDR (the runtime pointer-in-value-slot shape); `unbox_any` reads the value slot back (word →
|
||
`readField`, aggregate → the stored ADDR). **Required making `.any` a first-class comptime
|
||
aggregate** (it was `kindOf → .unsupported`): `kindOf(.any) = .aggregate` (16B, by-address) +
|
||
`fieldOffset` special-cases `.any` to the `{@0, @8}` layout (shared with string/slice) — without
|
||
the latter, a `struct_get` on an Any panicked (`union field 'struct' while 'any' is active`),
|
||
caught + fixed (no crash; "never crash" upheld). Updated two unit tests that used `unbox_any` as
|
||
the "unported op" example → now `compiler_call`; added a box→unbox round-trip test. **697/0 BOTH
|
||
gates + all unit tests.** On `reify`. The 6 box_any examples (0114/0520–0524/1035) no longer bail
|
||
at box_any and produce VM output byte-matching legacy, but are not YET fully HANDLED — they now
|
||
fall back further at `switch_br` (comptime Any-tag type-switch), `type_name`, and `out`/print
|
||
(4A.2+/later steps). **Next (4A.2):** comptime `out`/print (VM output buffer + flush).
|
||
- **Phase 3 P3.4 step 8 (VM plan) — VM-native `type_info` REFLECTION → the whole metatype surface is HANDLED (2026-06-18).**
|
||
Ported `type_info($T)` into the VM (`callBuiltinVm` `.type_info` arm → new `buildTypeInfo`), the
|
||
inverse of step 7's `define`: reflect a type INTO a `TypeInfo` VALUE built in FLAT MEMORY (the
|
||
VM-native mirror of legacy `reflectTypeInfo`). Decodes the source type into a tag + members
|
||
(tagged-union/struct field & enum variant → `{ name, ty }`, a payloadless variant → `void`;
|
||
tuple → bare positional `Type`s), then lays out the nested value bottom-up using layouts derived
|
||
from the `TypeInfo` RESULT type (`ins.ty`, now threaded into `callBuiltinVm`): element array →
|
||
`{ptr,len}` slice → info struct (`EnumInfo`/`StructInfo`/`TupleInfo`) → `TypeInfo { tag, payload }`
|
||
tagged union (reusing step 7's tagged-union write). Variant/field names materialize via a
|
||
`makeStringValue` helper extracted from `text_of`. Same `backing_type` guard as step 7. **Result:
|
||
the ENTIRE metatype surface runs HANDLED on the VM with ZERO fallback** — `0614`–`0624` + `0632`
|
||
(0616 `field_type` folds at lower time, no comptime eval); the `define(declare, type_info(T))`
|
||
round-trips (`0619`/`0622`/`0623`) mint byte-identical copies on the VM. VM output byte-matches
|
||
legacy for all. **697/0 BOTH gates + all unit tests.** On `reify`. **Remaining VM fallbacks in the
|
||
comptime corpus are now genuinely-non-metatype** emit-time side effects: `print`/`out` (0613),
|
||
`global_addr` (0600), `compiler_call` #compiler hooks (0602/0603), and the inline-asm global
|
||
(1654). **Next:** port those (or confirm each is a legitimately-non-comptime case) to drive the
|
||
fallback list to empty, then — with user go-ahead — flip the VM to default + delete `interp.zig`.
|
||
- **Phase 3 P3.4 step 7 (VM plan) — VM-native metatype CONSTRUCTION: `declare`/`define` + tagged-union `enum_init` (2026-06-18).**
|
||
Ported the metatype type-CONSTRUCTION builtins into the VM so the construction examples run
|
||
HANDLED end-to-end (no `call_builtin` fallback). Three pieces: (1) **tagged-union `enum_init`
|
||
with payload** — the arm previously bailed; now allocates the value (zeroed), writes the tag at
|
||
offset 0 (`{ header(tag)@0, [N x i8] payload@tag_size }`, the LLVM `backend/llvm/types.zig`
|
||
layout) and copies the payload at `tag_size`. (2) A **`.call_builtin` exec arm** → new
|
||
`callBuiltinVm`, the VM-native mirror of the legacy `execBuiltinInner`: `declare(name)` mints an
|
||
empty forward nominal slot (shared `declareNominal` helper, also used by `declare_type`);
|
||
`define(handle, info)` reads the `TypeInfo` tagged-union VALUE from FLAT MEMORY (tag@0, active
|
||
payload `EnumInfo`/`StructInfo`/`TupleInfo` struct at `tag_size`, its single slice field) and
|
||
mints via `defineFromInfo`, a faithful port of legacy `defineEnum`/`defineStruct`/`defineTuple`
|
||
(all-void enum → real `.@"enum"` per issue 0142, dup-name rejection, `updatePreservingKey` vs
|
||
`replaceKeyedInfo`). (3) Refactored the `[]{name,ty}` decode out of `registerTypeVm` into a
|
||
shared `decodeMemberSlice` (+ `decodeTypeSlice` for bare-`Type` tuple elements), keyed to the
|
||
module-level `NamedMember`. Unmodeled builtins (`type_info`/`type_name`/…) return null → bail
|
||
with the builtin name → legacy fallback (dual-path parity). **Correctness guard (caught via
|
||
review):** `enum_init`/`define` assume a tag-headed layout, which is WRONG for a `backing_type`
|
||
tagged union (laid out as the backing struct) — both now bail loudly on `backing_type != null`
|
||
rather than silent-clobber. **Result:** examples `0614`/`0620`/`0621`/`0624`/`0632` run **fully
|
||
HANDLED** on the VM (define is the whole eval); `0622`/`0623` run define HANDLED then fall back
|
||
cleanly at the still-unported `type_info` reflection. VM output byte-matches legacy for all 7.
|
||
**697/0 BOTH gates + all unit tests (added: tagged-union `enum_init` payload layout).** On
|
||
`reify`. **Next:** port `type_info` (REFLECT a type → build a `TypeInfo` value in comptime memory,
|
||
the inverse — reuses the tagged-union `enum_init` write) so `0619`/`0622`/`0623` go fully HANDLED;
|
||
then the rest of the comptime corpus (drive the SX_COMPTIME_FLAT_TRACE fallback list toward the
|
||
genuinely-non-comptime cases) before the VM-default flip + legacy deletion.
|
||
- **Phase 3 P3.4 step 6 (VM plan) — REAL lowering-time Context: allocating + List-building type-fns now run HANDLED on the VM (2026-06-18).**
|
||
The VM can now evaluate a comptime type-fn that ALLOCATES at lowering time (the 0141 family) —
|
||
the legacy interp cannot. Four changes: (1) `runComptimeTypeFunc` (lower/comptime.zig) FORCES the
|
||
CAllocator→Allocator thunks to exist (`getOrCreateThunks`, idempotent, guarded by Allocator/
|
||
CAllocator registered) BEFORE eval — a type-fn const runs at scanDecls (Pass 1), before Pass 1c
|
||
builds the default-context global + thunks, so the comptime allocator was otherwise null;
|
||
(2) `materializeDefaultContext` builds a REAL context at lowering time when the global is absent —
|
||
finds the two thunks by name (`findFuncByName`) and lays their func-refs into the inline
|
||
`Allocator` value `{ctx=null, alloc_fn@+ptr, dealloc_fn@+2*ptr}` at the head of `Context`, so
|
||
`context.allocator.alloc_bytes` dispatches `call_indirect` → thunk → native VM `malloc`;
|
||
(3) `aggType` now DEREFS a pointer `base_type` (the List write path emits `struct_gep` with
|
||
`base_type = *Struct` — `fieldOffset` panicked on the pointer; now derefs to the pointee, no
|
||
panic); (4) `subslice` handles a `[*]T` many-pointer / `*T` base (a List's `items` field — the
|
||
base IS the data pointer). **Verified end-to-end (manual probe):** a compiler-API type-fn that
|
||
builds its `[]Member` in a `List(Member)` (`.append` ×3, then `register_type(handle, kind,
|
||
vs.items[0..vs.len])`) runs **HANDLED on the VM** and mints correctly (`green=7`) — the exact
|
||
0141 List-growth pattern, on the VM. **Can't be a corpus test yet** (gate-OFF/legacy still can't
|
||
allocate at lowering time — the dual-path bind), so locked in via VM unit tests instead
|
||
(many-pointer subslice; `struct_gep` with a pointer `base_type`). **697/0 BOTH gates + all unit
|
||
tests, EXIT=0.** On `reify`. **Remaining for the original 0141 repro (uses metatype `define`/
|
||
`make_enum` → `call_builtin` → legacy fallback → legacy fails):** re-express the metatype over the
|
||
compiler-API so the whole type-fn runs on the VM (no `call_builtin`). THEN the repro works on the
|
||
VM — and the dual-path bind resolves only at the VM-default-flip + legacy-deletion end-state.
|
||
- **Phase 3 P3.4 — investigation: the "real lowering-time Context" is BLOCKED by issue 0141 (2026-06-18).**
|
||
Probed whether the VM needs a REAL lowering-time `Context` (CAllocator thunk func-refs) for
|
||
allocating type-fns. **Finding: lowering-time comptime ALLOCATION fails in the LEGACY interp
|
||
too** — a type-fn that calls `context.allocator.alloc_bytes` at lowering time bails in legacy
|
||
with `comptime call_indirect: callee is not a func_ref Value (raw fn-pointers from extern calls
|
||
aren't dispatchable in interp)`, and the VM bails at parity (`call_indirect through a null
|
||
function pointer`). This is exactly issue **0141**'s root cause (its analysis already notes "the
|
||
null allocator is the same story for the CAllocator thunks") — an OPEN deferred issue. So:
|
||
(1) the VM is CORRECT (parity — both bail; no regression); (2) the real-context work is
|
||
PREMATURE — its only consumer (allocating lowering-time type-fns) can't pass gate-OFF, so no
|
||
corpus test can validate it, and even a more-capable VM can't ship a divergence during the
|
||
dual-path phase. **Consequence for the metatype re-expression:** re-expressing `define`/`make_enum`
|
||
over the compiler-API needs to BUILD `[]Member` slices dynamically (allocation) — which is
|
||
blocked by 0141 at lowering time. The viable paths are: (a) avoid allocation by passing the
|
||
caller's existing slice through (needs `EnumVariant`/`StructField` to be usable AS `Member` —
|
||
they're layout-identical `{string, Type}`, but distinct nominal types — a metatype-API decision),
|
||
or (b) wait for 0141. **No code change this step** (the VM already bails correctly). Recorded so
|
||
the next session doesn't re-derive it. 697/0 both gates unchanged.
|
||
- **Phase 3 P3.4 step 5 (VM plan) — WRITE side ported to the VM → FIRST HANDLED lowering-time type-fns (2026-06-18).**
|
||
Ported `declare_type` / `pointer_to` / `register_type` into `Vm.callCompilerFn`, mirroring the
|
||
legacy `compiler_lib` handlers (mint via `@constCast(table)` — the same mutable access the
|
||
read-side `intern` uses; the lowering-time mint target IS `&module.types`). `register_type`
|
||
reads the `[]Member` slice from FLAT MEMORY: threaded `ref_types` through `invoke` →
|
||
`callCompilerFn` so the slice's element type (`Member = {name: string, ty: Type}`) gives the
|
||
field offsets + stride; decodes each `{name, ty}` and branches on `kind` (1 struct · 2 enum ·
|
||
3 tagged_union · 4 tuple) exactly as legacy (dup-name / payload-on-enum rejections, idempotent
|
||
re-fill via `nominalIdentOf`). **Key unblock:** the synthesized comptime type-fn wrapper
|
||
(`createComptimeFunction`/`…WithPrelude`) was built with return type `.any` → `regToValue`
|
||
bailed at the VM↔legacy boundary; changed to `.type_value` (the legacy path reads via `asTypeId`
|
||
regardless, so no legacy change). **Result: the compiler-API write type-fns now run HANDLED
|
||
end-to-end on the VM at LOWERING time** — `0631` (register-graph: 2 HANDLED, A↔B cycle via
|
||
forward handles + `pointer_to`) and `0635` (multi-edge import: 2 HANDLED), parity-correct. They
|
||
run on the ZEROED lowering-time context (fixed `.[…]` member arrays, no allocation). The
|
||
metatype `make_enum`/`define` examples (`0632`) still fall back CLEANLY through
|
||
`call_builtin(define)` (the separate metatype path — re-expressing it onto the compiler-API is
|
||
the other half of P3.4). **697/0 BOTH gates + EXIT=0.** On `reify`. **Next:** (optional, deferred)
|
||
a REAL lowering-time Context (CAllocator thunk func-refs) for List-growing type-fns; and
|
||
re-express the metatype `define`/`make_enum` over the compiler-API to delete the bespoke interp
|
||
arms (the end-state: ONE evaluator).
|
||
- **Phase 3 P3.4 step 4 (VM plan) — model `.type_value` natively in the comptime VM (2026-06-18).**
|
||
The VM now HANDLES Type values instead of bailing: `kindOf(.type_value)` → `.word`; a new
|
||
`const_type` exec arm → the word `TypeId.index()`; `regToValue` maps a `.type_value` word back
|
||
to a `.type_tag` Value at the legacy boundary (`valueToReg` already mapped `.type_tag` →
|
||
index). Surfaced + fixed a VM PANIC (forbidden): `struct_init` assumed a `.@"struct"` result
|
||
type and union-access-panicked on an ARRAY literal (`EnumVariant.[ … ]`, reached now that Type
|
||
args no longer bail early) — it's the generic aggregate-literal op, so it now dispatches on the
|
||
result kind (struct / array / tuple) and BAILS loudly on anything else, never panics. **697/0
|
||
both gates** (the make_enum type-fns now run further on the VM, then bail cleanly at the
|
||
`define`/`make_enum` `call_builtin` → legacy mints — no mutation before the bail, parity holds).
|
||
VM unit test added (const_type → word → regToValue → `.type_tag`). On `reify`. **Next (the
|
||
payoff):** port the WRITE side (declare_type / register_type / pointer_to) into
|
||
`Vm.callCompilerFn` + give the lowering-time path a REAL Context (CAllocator thunk func-refs,
|
||
not zeroed) → the first HANDLED lowering-time type-fn end-to-end on the VM.
|
||
- **Phase 3 P3.4 step 3 (VM plan) — dedicated `Type` builtin TypeId: RESOLVER FLIPPED + `.any` migration (2026-06-18).**
|
||
Flipped `type_resolver:64` (`"Type"` → `.type_value`), `module.zig` `constType` (result type
|
||
→ `.type_value`), and `emitConstType` (a bare i64 carrying `tid.index()`, NOT a 16-byte Any
|
||
box). Then migrated every `.any` reference that means "a Type value", classified per CLAUDE.md
|
||
(leave the real boxed-Any refs): (a) the "Any holds a Type" **meta-marker tag** moved `.any` →
|
||
`.type_value` at all 4 consumers — `reflectArgTypeId` (LLVM), `reflectTypeId` + the
|
||
`.type_tag`-as-struct-field comptime path (interp), and `resolveTypeCategoryTags("type")`
|
||
(generic.zig); (b) reflection-builtin RETURN types `.any` → `.type_value` (`type_of`/`declare`/
|
||
`define`); the runtime `type_of(any)` now reads the tag AS a `.type_value` (no re-box); (c)
|
||
expr_typer infers a bare type-name expr as `.type_value` (with a `is_raw` backtick exemption —
|
||
`` `string `` is a value, never the reserved type); (d) `reflectionArgIsType` accepts
|
||
`.type_value` OR `.any` (a reflection arg can be a bare Type OR a boxed Any — the over-narrow
|
||
`==.type_value` was the catastrophic-regression cause, caught + fixed); (e) the comptime
|
||
`switch_br` accepts a `.type_tag` discriminant (type-category match); (f) a bare function name
|
||
in a `Type` slot now lowers to `const_type(its real function type)` instead of a func-ref
|
||
(fixed a JIT crash — was a func-ref word read as a TypeId), keeping the old string-box path only
|
||
for genuine `Any` params; (g) the field-not-found diagnostic + `formatTypeName` render
|
||
`.type_value` as "Type". Fixed 3 unit tests asserting the old `.any` Type behavior.
|
||
**697/0 BOTH gates** + all 494 unit tests (EXIT=0). Gate ON stays green because the VM's
|
||
`kindOf(.type_value)` → `.unsupported` → bails CLEANLY to legacy (no silent-wrong) — the VM
|
||
doesn't model `Type` values YET (next step), but parity holds. Regenerated 24 snapshots (22
|
||
`.ir` const_type-shape; 2 `.stderr` Any→Type — diff reviewed, only the intended changes). On
|
||
`reify`. **Next:** model `.type_value` natively in the VM (`kindOf` → word, `const_type` → word
|
||
= `TypeId.index()`, `regToValue` word → `.type_tag`) for COVERAGE, then port the WRITE side into
|
||
`callCompilerFn` + a real lowering-time Context → the first HANDLED lowering-time type-fn.
|
||
- **Phase 3 P3.4 step 2 (VM plan) — dedicated `Type` builtin TypeId: FOUNDATION landed (dead/additive) (2026-06-18).**
|
||
Added `TypeId.type_value` (slot 19) + a matching `TypeInfo.type_value` variant + the builtins
|
||
init entry — an **8-byte type handle distinct from the 16-byte boxed `.any`** (THE WALL). All
|
||
`types.zig` layout handlers wired: `sizeOf`/`typeSizeBytes` → 8, `typeAlignBytes` → 8,
|
||
`typeName` → "Type", `hashTypeInfo`/`typeInfoEql` no-payload arms. Only ONE exhaustive switch
|
||
needed a new arm (`backend/llvm/types.zig` `toLLVMTypeInfo` → `cached_i64`); every other
|
||
`switch(TypeInfo)` site has an `else` (audited when the resolver flips). **`first_user` 19 → 100**
|
||
(per the user): slots 20–99 are RESERVED builtin headroom (infos padded with the `unresolved`
|
||
tripwire), so future builtins don't renumber user TypeIds / churn `sx ir` snapshots. Cost:
|
||
~80 default entries in each binary's per-type reflection arrays (user opted in). **Still dead:**
|
||
`type_resolver.zig:64` STILL returns `.any` for "Type" — nothing produces `.type_value` yet, so
|
||
NO behavior change. Regenerated 22 IR snapshots (pure TypeId renumber to 100-base; `git diff
|
||
--name-only` confirmed ONLY `.ir` files + the 2 source files changed — no stdout/stderr/exit).
|
||
**697/0 both gates** (OFF and `-Dcomptime-flat`). **Next:** flip `type_resolver:64` →
|
||
`.type_value`, then migrate the `.any` refs that mean "a Type value" (const_type result /
|
||
reflection returns / metatype `Type` params / `.type_tag` checks) — leave the real boxed-Any
|
||
refs — file-by-file with a build after each.
|
||
- **Phase 3 P3.4 step 1 (VM plan) — lowering-time default context; first blocker cleared (2026-06-18).**
|
||
`materializeDefaultContext` now falls back to a ZEROED `Context` (found by name) when the
|
||
`__sx_default_context` global is absent — i.e. at LOWERING time, where the global isn't
|
||
emitted yet. A type-fn that never touches the allocator now runs past context setup; one
|
||
that allocates reads a null `alloc_fn` (zeroed) → `call_indirect` on the null func-ref
|
||
bails → legacy fallback (a REAL lowering-time context with the CAllocator thunk func-refs,
|
||
so allocating type-fns also run on the VM, is a follow-up). **Measurement: the bail moved
|
||
deeper** — metatype `make_enum` now bails at `const_type` (the `Type`-literal op, unported);
|
||
`register_type` type-fns bail at the welded write call (declare_type/register_type aren't in
|
||
`callCompilerFn`). No table mutation happens before either bail (the write fns bail before
|
||
minting), so parity holds: both gates **697/0**, no crashes. **Next blockers (the "model
|
||
Type" chunk):** (a) the `const_type` op → a word = `TypeId.index()`; (b) the Type-return
|
||
bridge (`regToValue` for a `Type`/`.any` word → `.type_tag`); (c) the VM-native write side
|
||
(declare_type/register_type/pointer_to in `callCompilerFn`) + a real lowering-time context.
|
||
Only once those land does a type-fn actually run end-to-end on the VM (a HANDLED case).
|
||
- **Phase 3 P3.4 (VM plan) — wire the VM at the LOWERING-time site + measure (2026-06-18).**
|
||
Routed `runComptimeTypeFunc` (the type-fn fold — the THIRD comptime call site) through
|
||
`comptime_vm.tryEval` behind `-Dcomptime-flat`/`SX_COMPTIME_FLAT` with legacy fallback,
|
||
mirroring the two emit-time folds. Extracted the shared post-check (`checkComptimeTypeResult`
|
||
— the declared-but-never-defined zero-field guard) so both paths use it. **Measurement
|
||
(SX_COMPTIME_FLAT_TRACE):** every metatype/compiler-API type-fn currently bails CLEANLY
|
||
with `no __sx_default_context global to materialize the implicit context` — at lowering
|
||
time the default-context global doesn't exist yet (it's built at emit time), so the VM bails
|
||
at context materialization, BEFORE running the body (no partial mint, no crash → legacy
|
||
mints). The hardening holds: **no crashes** across the corpus on the VM lowering-time path.
|
||
Both gates **697/0**. **So the FIRST lowering-time blocker is the implicit context, not
|
||
`Type` modeling** — the VM needs a way to materialize/skip the default context at lowering
|
||
time (most type-fns get an implicit ctx for potential `List`-growth alloc; many don't use
|
||
it). Next: materialize a lowering-time default context for the VM (or pass a null ctx +
|
||
bail only if the allocator is actually used), THEN model `Type` values + the VM-native write
|
||
side. This is near-pure fallback today — permanent scaffolding that lights up as those land.
|
||
- **Phase 3 P3.4-prep (VM plan) — harden the VM against malformed lowering-time IR (2026-06-18).**
|
||
Prerequisite for wiring the VM at the LOWERING-time comptime site (`runComptimeTypeFunc`),
|
||
where IR can be malformed (an unresolved name lowers to a dangling / `Ref.none` operand —
|
||
the 0737 crash). Closed the remaining panic vectors so the VM BAILS (→ legacy fallback)
|
||
instead of aborting: (1) a checked `Vm.refTy(ref_types, r)` replaces every raw
|
||
`ref_types[ref.index()]` in `exec` (the type-side companion to `Frame.get`'s `bad_ref`
|
||
value-side guard); (2) `aggType` is now a bailing method (`Error!TypeId`) using `refTy`;
|
||
(3) the block-dispatch loop bounds-checks the branch target before indexing
|
||
`func.blocks.items`. `global_get` was already guarded. No behavior change — gate OFF and
|
||
ON both **697/0**; unit test added (a `cmp_lt` with a `Ref.none` operand bails, not
|
||
panics). **Next:** wire `tryEval` into `runComptimeTypeFunc` behind the flag with legacy
|
||
fallback and measure (most minting type-fns will still bail at the welded-write call /
|
||
`Type`-result conversion until the VM models `Type` values + the VM-native write side land
|
||
— those are the steps that actually move lowering-time comptime onto the VM, toward
|
||
deleting legacy).
|
||
- **Phase 3 P3.3 (VM plan) — WRITE side: declare_type + pointer_to + ONE kind-branching register_type (2026-06-18).**
|
||
The mutating compiler-API: `declare_type(name) -> Type` (forward handle), `pointer_to(t) -> Type`
|
||
(build `*T`), and `register_type(handle, kind, members: []Member) -> Type` which branches on
|
||
`kind` IN THE COMPILER (subsuming define's per-kind dispatch). Take/return real `Type` values
|
||
(matching meta.sx declare/define). **Timing (per user): mint LAZILY at lowering time, single
|
||
pass** (the existing `runComptimeTypeFunc`), so the write side is **legacy-only** (`compiler_lib`
|
||
handlers) — the VM isn't wired at lowering time, no VM mirror needed; readers stay dual-path.
|
||
A non-generic `-> Type` builder is now flagged `is_comptime` (decl.zig) so its dead body permits
|
||
the welded calls. **Graph:** forward handles + `pointer_to` express mutually-recursive A↔B (`*A`,
|
||
`*B`, B-by-value); `register_type` is **idempotent** (re-fill a nominal slot reached via two
|
||
import edges — `nominalIdent`). `kind` codes match `type_kind` (1 struct · 2 actual `.@"enum"` ·
|
||
3 tagged_union · 4 tuple). **Fixed two bugs (issue 0142):** (a) a fully payloadless minted enum
|
||
was an all-void tagged_union → verifySizes panic; now a real `.@"enum"` (register_type kind 2 AND
|
||
metatype `defineEnum`); (b) bare `EnumType.variant` payloadless qualified construction wasn't
|
||
supported (failed for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
|
||
Examples 0631 (graph + actual enum + reflection), 0632 (make_enum all-void), 0633/0634/0635
|
||
(namespaced / bare / multi-edge import of a minted type), 0187 (qualified variant construction).
|
||
**Parity 697/697** (gate ON and OFF); unit tests added. **Next (P3.4):** re-express
|
||
declare/define/type_info as sx over the compiler-API + delete the bespoke interp arms (needs the
|
||
VM hardened for lowering-time IR, or the metatype migrated onto the legacy compiler-API calls).
|
||
- **Phase 3 P3.2b (VM plan) — kind + enum-value readers: `type_kind` + `type_field_value`; READ side complete (2026-06-18).**
|
||
The last two read-only readers the metatype's `type_info(T)` needs (added to
|
||
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`, each backed by a `TypeTable` query both
|
||
call): `type_kind(t) -> i64` (`kindCode` — a stable, compiler-owned discriminant: 0 other ·
|
||
1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · 7 vector · 8 error_set;
|
||
TOTAL, never bails) and `type_field_value(t, idx) -> i64` (`memberValue` — an enum variant's
|
||
explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for non-enum /
|
||
out-of-range). Example `0630-comptime-compiler-type-kind` reflects `Color` / `WindowFlags`
|
||
(flags) / `Point`. **The READ side is now COMPLETE** — `find_type` + `type_kind` +
|
||
`type_field_count` + `type_field_name`/`type_field_type`/`type_nominal_name` +
|
||
`type_field_value` cover everything `reflectTypeInfo` reads. VM unit test added. **Parity
|
||
691/691** (gate ON and OFF). **Revised forward direction (per the user):** the WRITE side is
|
||
ONE `register_type(info)` fn that branches on the kind IN THE COMPILER (subsuming `define`'s
|
||
per-kind dispatch), not a per-kind `register_struct`.
|
||
- **Phase 3 P3.2 (VM plan) — field-level reflection readers: `type_nominal_name` + `type_field_name` + `type_field_type` (2026-06-18).**
|
||
Three more `compiler`-library readers on the same `TypeId`-handle shape (added to
|
||
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`), each backed by a new `TypeTable` query
|
||
BOTH paths call (no drift): `nominalName` (a named type's own name handle; loud-bail for
|
||
unnamed types like `i64`/pointers), `memberName` (struct/union/tagged-union field, enum
|
||
variant, named-tuple element), `memberType` (struct/tuple/array/vector member type). All
|
||
loud-bail on out-of-range idx / no-member (no silent default). First MULTI-ARG compiler
|
||
fns — `callCompilerFn` reads arg 1 = idx; added `Vm.argHandle`/`argTypeId` (range-checked
|
||
u32/TypeId arg reads) and refactored `find_type`/`type_field_count` onto them. Named
|
||
`type_*` to avoid clashing with the std metatype builtins (`field_name`/`type_name` exist
|
||
in core.sx); `nominalName` (the TypeTable method) is distinct from the existing
|
||
`typeName(id) []const u8` display-string renderer. Example `0629-comptime-compiler-field-reflect`
|
||
reflects `Pair { lo: Point; hi: Point }` — each field name + the nominal name of a field's
|
||
type, all `#run`-folded, all VM-HANDLED natively. VM unit test added (type_field_name → "hi";
|
||
type_nominal_name(type_field_type(Pair,0)) → "Point"). **Parity 690/690** (gate ON and OFF).
|
||
- **Phase 3 P3.1 (VM plan) — first read-only reflection readers: `find_type` + `type_field_count` (2026-06-18).**
|
||
Two more `compiler`-library fns, bound the same way as the `intern`/`text_of` seed (added
|
||
to `compiler_lib.bound_fns` for the legacy handler + the welded-decl export check, AND to
|
||
`Vm.callCompilerFn` for the native comptime path — NO marshaling). A **type handle is a
|
||
plain `u32` `TypeId`** (like `StringId`), so both keep the seed's clean scalar shape:
|
||
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`, `unresolved`/0 if absent) and
|
||
`type_field_count(t: TypeId) -> i64` (a NEW `TypeTable.memberCount` query — struct/union/
|
||
tagged-union fields, enum variants, array/vector length — called by BOTH paths so they
|
||
can't drift; bails loudly, never a silent 0). New example `0628-comptime-compiler-find-type`
|
||
chains `intern → find_type → type_field_count` (and a not-found lookup → 0), both folded at
|
||
`#run`, both VM-HANDLED natively (trace confirms no fallback). VM unit test added
|
||
(`find_type` + `type_field_count`, struct found → 3 fields, missing → `unresolved`).
|
||
**Parity 689/689** (gate ON and OFF). **Decision (resolves the plan's `find_type → ?Type`
|
||
sketch):** return a NON-optional `TypeId` with the `unresolved` (0) sentinel for not-found,
|
||
NOT `?Type` — a `Type` value resolves to `.any` (which the comptime VM doesn't represent)
|
||
and an optional can't cross the legacy↔VM eval boundary; `unresolved` is the project-blessed
|
||
unmistakable "no type" marker. Forward (P3.2): more readers on the same handle shape
|
||
(`type_name`/`field_name`/`field_type`/kind), then `register_struct` (first mutating fn).
|
||
- **VM robustness — `Frame` bounds-check; lowering-time `#insert` wiring explored + reverted (2026-06-18).**
|
||
Explored wiring the VM at the LOWERING-time comptime site (`evalComptimeString`, the
|
||
`#insert` string fold). 12/13 `#insert` examples ran on the VM with parity, but `0737`
|
||
(an `#insert` of an unresolved `secret()`) CRASHED the VM (SIGABRT): lowering-time IR can
|
||
be malformed (a `ret Ref.none` from the unresolved name) and `Frame.get` panicked on the
|
||
out-of-range index. **Decision: reverted the lowering-time wiring** — unlike the emit-time
|
||
folds (fully lowered IR), lowering-time IR can be erroneous, and hardening the VM against
|
||
ALL malformed IR (every `ref_types[...]` / `aggType` access, not just `Frame`) is out of
|
||
scope here. The emit-time sites already give full corpus coverage. **KEPT** the defensive
|
||
fix regardless (CLAUDE.md "never crash"): `Frame.get`/`set` now bounds-check and flip a
|
||
`bad_ref` flag; the `run` loop bails (`badRef`) instead of panicking. Unit test added
|
||
(malformed `ret Ref.none` → bail, not crash). Parity **688/688** both ways.
|
||
- **Phase 3 SEED (VM plan) — compiler-call path: `intern`/`text_of` native on the VM (2026-06-18).**
|
||
`invoke` now dispatches a welded `compiler`-library fn (gated on `compiler_welded`) to
|
||
`Vm.callCompilerFn`, serviced NATIVELY on comptime memory (no legacy `Interpreter`):
|
||
`intern(string)->StringId` reads the comptime string bytes and `internString`s into the
|
||
const-cast table (pool-only — doesn't touch type layout, so cached sizes stay valid);
|
||
`text_of(StringId)->string` materializes the pooled text back into comptime memory. Unlocked
|
||
`0626`; the ONLY remaining const-init fallback is now the inline-asm global (`1654`).
|
||
Parity **688/688** (gate ON and OFF); unit test added. This is the mechanism Phase 3 grows
|
||
— the next compiler fns (`find_type`, `register_struct`, reflection readers) bind the same
|
||
way (comptime pointer in, handle/pointer out, no marshaling).
|
||
- **Phase 1.final step 9 (VM plan) — `-Dcomptime-flat` build flag (the "swap behind a build flag" step) (2026-06-18).**
|
||
Added the `-Dcomptime-flat` build option (build.zig → a `build_opts` options module on
|
||
`mod`; `emit_llvm.init` reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`). This is
|
||
the plan's "reach parity → swap behind a build flag → delete the old path" mechanism.
|
||
`zig build test -Dcomptime-flat` runs the FULL corpus on the VM (688/0). Verified the flag
|
||
toggles the binary: flag-built `sx` reports VM HANDLED with no env var; default-built does
|
||
not. Default OFF — `zig build test` unchanged (688/0). Env var still works for ad-hoc runs.
|
||
Next (forward): Phase 2 (bytecode) / Phase 3 (compiler-API on comptime memory); eventual
|
||
default-flip + legacy deletion.
|
||
- **Phase 1.final step 8 (VM plan) — wire the `#run` side-effect path + trace-clear-on-fallback (2026-06-18).**
|
||
Wired the SECOND comptime call site (`runComptimeSideEffects`, top-level `#run <expr>;`)
|
||
through `tryEval` with legacy fallback, mirroring the const-init fold. `tryEval` now
|
||
handles void/noreturn entries (→ `.void_val`) so a void side-effect doesn't bail at the
|
||
result conversion. **Fixed a trace-corruption** the new site exposed (`1035`): a
|
||
side-effect that pushes return-trace frames and then bails (e.g. on `print`) had the
|
||
legacy re-run DOUBLE-push them (`sx_trace_push` is a side effect on the shared buffer).
|
||
Both wiring sites now `sx_trace_clear()` right before the legacy fallback, discarding the
|
||
VM's partial pushes. **Parity 688/688** (gate ON and OFF). Most side-effects still bail
|
||
(print/global_addr/call_builtin) → legacy, but the path is now uniform. All comptime
|
||
evaluation routes through the VM-with-fallback.
|
||
- **Phase 1.final step 7 (VM plan) — is_comptime + failable/error cluster + signed-load fix; coverage 31→36 (2026-06-18).**
|
||
`is_comptime` → 1 (unlocked `1030`). Ported the failable/error-channel cluster (`1037`
|
||
escape, `1038` handled): `kindOf(error_set)→word`, `regToValue` bridges TUPLES (the
|
||
failable `(value…,tag)` shape `checkComptimeFailable` reads), `trace_frame` packs
|
||
`(func_id<<32|span.start)` from a new `call_stack` (pushed by invoke/runEntry), and
|
||
`sx_trace_push`/`sx_trace_clear` serviced NATIVELY (the VM calls the real sx_trace.c
|
||
functions linked into the compiler, so the return-trace buffer is populated identically
|
||
to legacy). raise/catch/or now run on the VM. **Surfaced + fixed a real GENERAL bug:**
|
||
`readField` was ZERO-extending signed sub-64-bit loads, so a stored `i32 -1` reloaded as
|
||
`0xFFFFFFFF` (+4.29e9) and `< 0` was false — silently hiding `raise error.Bad`; now
|
||
SIGN-extends `i8`/`i16`/`i32`/`isize` (gate-ON parity confirms it's a strict fix; unit
|
||
test added). VM HANDLES **36** corpus const-inits (was 31); **parity 688/688** (gate ON
|
||
and OFF). Only **2 fallbacks** remain, both principled: `intern` (`0626`, welded
|
||
compiler-API fn — Phase 3) + inline-asm global (`1654`). Forward work: Phase 2 (bytecode),
|
||
Phase 3 (compiler-API on comptime memory).
|
||
- **Phase 1.final step 6 (VM plan) — real default context + call_indirect + func_ref + global_get; coverage 27→31 (2026-06-17).**
|
||
Per the user's direction ("the VM can set up a default context"), `runEntry` now
|
||
materializes the REAL default context instead of a zeroed one. The implicit-ctx param is
|
||
an opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
|
||
and lays its initializer (`{ {null, alloc_fn, dealloc_fn}, null }`, the CAllocator thunk
|
||
func-refs) into comptime memory via a new recursive `layoutConst`. With `func_ref` (function
|
||
value encoded `FuncId.index()+1`, reserving word 0 for the null fn-ptr) and
|
||
`call_indirect` (decode word → FuncId → dispatch; 0 → bail) ported, the whole allocator
|
||
protocol runs on the VM:
|
||
`context.allocator.alloc_bytes` → call_indirect → thunk → `CAllocator.alloc_bytes` →
|
||
`libc_malloc` → native comptime malloc. Unlocked `0606` (string global). Also: `global_get`
|
||
lazily evaluates a comptime global's `comptime_func` (memoized) — unlocked `CT_CHAIN`;
|
||
field access (`fieldOffset`/`struct_get`) handles string/slice `{ptr@0,len@8}` fat
|
||
pointers (needed by `alloc_string`); `regToValue` maps function-typed words → `.func_ref`
|
||
(kept `1128`'s rejection byte-identical). Native `malloc` is still required (the thunk
|
||
bottoms out at it; a host pointer can't be used with comptime load/store). VM HANDLES
|
||
**31** corpus const-inits (was 27); **parity 688/688** (gate ON and OFF). Unit tests:
|
||
global_get, func_ref+call_indirect. Remaining fallbacks (7): `.unsupported` aggregates
|
||
(3× — `1037`/`1038`), extern/builtin `intern`+asm (2×), `trace_frame`, `is_comptime`.
|
||
- **Phase 1.final step 5 cont. (VM plan) — libc memory builtins + f32 fix; coverage 16→27 (2026-06-17).**
|
||
Identified the dominant fallback (`call to extern/builtin`) as **11× `malloc`** (0604) +
|
||
1× `intern`. Modeled a curated set of libc MEMORY builtins natively on comptime memory
|
||
(`Vm.callMemBuiltin`): `malloc`/`calloc` → `allocBytes` (16-aligned, 256-MiB cap → bail),
|
||
`free` → no-op, `memcpy`/`memmove`/`memset` on comptime bytes — sandboxed (no host heap/dlsym),
|
||
target-aware; the computed result is byte-identical to legacy (which calls real libc).
|
||
This surfaced a **real latent f32 bug**: float registers hold f64 bits, but f32 MEMORY is
|
||
the 4-byte single — `readField`/`writeField` were truncating the f64 bits (writing zeros
|
||
for `1.0`); now they `@floatCast` on f32 load/store (mirrors legacy `storeAtRawPtr`).
|
||
Result: VM HANDLES **27** corpus const-inits (was 16); **parity 688/688** (gate ON and
|
||
OFF). Unit tests added (f32 round-trip; malloc → usable comptime memory). Next: the `kindOf`
|
||
`.unsupported` aggregates (3×), `global_get` (2×), the rest.
|
||
- **Phase 1.final step 5 (VM plan) — implicit-context materialization; coverage 0→16 (2026-06-17).**
|
||
`tryEval` now MATERIALIZES the implicit ctx instead of skipping it: a `has_implicit_ctx`
|
||
comptime entry (sole param `*Context`) gets a zeroed `Context` of the right size/align
|
||
in comptime memory, its address passed as arg 0. Const bodies that ignore the ctx run; a
|
||
body that uses the allocator hits unported `call_indirect` → bails → legacy. No func-ref
|
||
materialization needed (handled bodies don't read ctx contents; parity is the guard).
|
||
Fixed a real bug surfaced by the coverage pass: storing a `null` non-pointer optional
|
||
(the `null_addr` sentinel) into an aggregate slot OOB-bailed — `writeField` now ZEROES
|
||
the destination for a `null_addr` aggregate source (= none/empty); unit-test regression
|
||
added. Result: VM HANDLES **16** corpus const-inits (was 0); **parity 688/688 both
|
||
gate ON and OFF**. Next: port the ops the trace names — `call_builtin`/`compiler_call`/
|
||
extern (13×, via the bridge), `kindOf` `.unsupported` aggregates (3×), `global_get` (2×),
|
||
func_ref / call_indirect / trace_frame / is_comptime.
|
||
- **Phase 1.final steps 1–4 (VM plan) — host wiring landed; coverage measured (2026-06-17).**
|
||
(1) **Hardening:** `Machine.readWord`/`writeWord`/`bytes` now return `error.OutOfBounds`
|
||
(null / out-of-range / oversized / overflow-safe) instead of `assert`-panicking;
|
||
`OutOfBounds` added to `Vm.Error`; `try` threaded through every helper + exec arm + the
|
||
bridge. New unit tests (accessor OOB returns; null-deref → `tryEval` null, not a crash).
|
||
(2) **Implicit context:** `tryEval` returns null for `has_implicit_ctx` funcs (legacy
|
||
fallback) — conservative; full ctx materialization deferred to step 5. (3) **Wiring:**
|
||
const-init fold in `emit_llvm.zig` `emitGlobals` is `(if comptime_flat) tryEval else
|
||
null) orelse interp.call(...)`, gated by env `SX_COMPTIME_FLAT` (read once into
|
||
`LLVMEmitter.comptime_flat`). Default OFF. (4) **Parity + coverage:** gate ON → full
|
||
corpus byte-identical (688, 0 failed) + manual 0605/0606/0607 byte-identical.
|
||
**Finding: 0 of 37 measured corpus const-inits are VM-handled — ALL are
|
||
`has_implicit_ctx`-gated.** Added a coverage-trace facility (`comptime_vm.last_bail_reason`
|
||
+ env `SX_COMPTIME_FLAT_TRACE`). **Next: step 5 = implicit-context materialization** (the
|
||
unblocker), then port the deferred ops. 688 corpus green (gate OFF).
|
||
- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).**
|
||
`comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on
|
||
the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back.
|
||
Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs
|
||
(1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of
|
||
crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind
|
||
`SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final.
|
||
688 corpus green.
|
||
- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).**
|
||
Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the
|
||
wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs
|
||
the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg →
|
||
Reg, aggregates into comptime memory) + `regToValue` (VM result → Value, deep-copied).
|
||
Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip
|
||
unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through
|
||
the VM with legacy fallback).
|
||
- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).**
|
||
`Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals
|
||
arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared
|
||
comptime memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate
|
||
Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b).
|
||
Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next:
|
||
1.5b (call_builtin/compiler_call/extern), then hybrid wiring.
|
||
- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).**
|
||
Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested
|
||
(deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops
|
||
(tagged-union payload/any/closures) blind — their byte semantics are ambiguous without
|
||
real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) +
|
||
HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so
|
||
the VM runs the real corpus and surfaces exactly what's needed. Key design point for
|
||
calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's
|
||
allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision
|
||
block.
|
||
- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).**
|
||
`kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}`
|
||
aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`;
|
||
const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91,
|
||
?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any,
|
||
closures).
|
||
- **Phase 1 sub-step 4b (VM plan) — slices + strings on comptime memory (2026-06-17).**
|
||
`{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate).
|
||
Ported `const_string` (text+NUL + fat pointer in comptime memory), `length`/`data_ptr`,
|
||
`array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and
|
||
`str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23,
|
||
subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures).
|
||
- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on comptime memory (2026-06-17).**
|
||
`kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get`
|
||
(`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over
|
||
array/pointer/many_pointer; slice/string bases bail), `length` on array values.
|
||
Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next:
|
||
sub-step 4b (slices/strings, then optionals/enums/any/closures).
|
||
- **Phase 1 sub-step 3 (VM plan) — memory + structs on comptime memory (2026-06-17).**
|
||
`Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported
|
||
`alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out
|
||
at the table's natural offsets. Value model: scalar/pointer → register word;
|
||
struct → lives in comptime memory, its value IS its address (read→addr, write→memcpy), so
|
||
nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on
|
||
not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested
|
||
(struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next:
|
||
sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls).
|
||
- **Phase 1 sub-step 2 (VM plan) — comptime executor: scalars + control flow
|
||
(2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over
|
||
comptime frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy
|
||
interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic,
|
||
comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and
|
||
`block_param`; every other op bails loudly (`error.Unsupported` + op name in
|
||
`detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic,
|
||
cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus
|
||
untouched (688 green). Next: sub-step 3 (memory + aggregates on comptime memory, where
|
||
target-aware layout enters).
|
||
- **Phase 1 sub-step 1 (VM plan) — comptime machine substrate (2026-06-17).**
|
||
New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator
|
||
with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0
|
||
reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on
|
||
deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested
|
||
(`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy
|
||
interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor +
|
||
scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule
|
||
from CLAUDE.md per request.
|
||
- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept
|
||
(2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig`
|
||
(`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/
|
||
`LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr`
|
||
+ the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and
|
||
examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function
|
||
host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3
|
||
compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the
|
||
`emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples
|
||
`0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next:
|
||
Phase 1 (byte-addressable value model) per `PLAN-COMPILER-VM.md`.
|
||
- **DIRECTION CHANGE — pivot off the byte-weld to a byte-addressable bytecode VM
|
||
(2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong
|
||
direction (it hand-marshals onto a comptime value model that isn't bytes — exactly
|
||
what the design set out to kill). New foundation: a bytecode VM over comptime memory so
|
||
comptime values are native bytes; the compiler-API then rides on it via direct memory
|
||
(no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks
|
||
cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0
|
||
strip → Phase 1 byte-addressable value model → Phase 2 bytecode → Phase 3 compiler-API on
|
||
comptime memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted
|
||
the session's uncommitted `register_struct`/`find_type` marshaling experiment back to
|
||
`reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action.
|
||
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
|
||
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
|
||
byte-blob — all explored, all unnecessary). Instead: the sx header declares
|
||
fields in the compiler type's memory order; the compiler reflects the bound Zig
|
||
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
|
||
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
|
||
On pass it's an ordinary byte-identical struct — cast + deref just works.
|
||
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
|
||
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
|
||
superseded by the reflection+validation design and deleted.
|
||
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
|
||
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
|
||
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
|
||
green (458 unit, 690 corpus).
|
||
- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).**
|
||
`compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function`
|
||
`compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`);
|
||
`interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round-
|
||
trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean
|
||
rejection deferred (loud link error today). Build + suite green (458 unit, 689
|
||
corpus).
|
||
- **Phase 1.1 fourth sub-step — welded-struct layout validation.**
|
||
`validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into
|
||
`registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against
|
||
the registry (lib == compiler, name exported, layout matches) with build-gating
|
||
diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful
|
||
Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to
|
||
Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit,
|
||
687 corpus).
|
||
- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`:
|
||
the `compiler` lib's welded-type registry; `Field` welded to
|
||
`StructInfo.Field` with layout baked from the real Zig type
|
||
(`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test
|
||
(+ null off the export list). Standalone island — not yet consumed by lowering.
|
||
Build + suite green (454 unit tests). Break-verified.
|
||
- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl`
|
||
gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern <lib>`
|
||
after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified.
|
||
Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete.
|
||
- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig)
|
||
extern <lib>` on fn decls; unified `callconv` into `abi(.c|.zig|.pure)` (removed
|
||
the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs +
|
||
snapshots. Build + suite green. The original design's `extern(.zig)` single
|
||
qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern
|
||
<lib>` (linkage + source) — recorded in the design doc's syntax-decision note.
|