Files
sx/current/CHECKPOINT-COMPILER-API.md
agra d0ebc55f99 comptime VM: VM-native metatype CONSTRUCTION — declare/define + tagged-union enum_init (P3.4 step 7)
The metatype type-construction builtins now run natively on the flat-memory
VM, so the construction examples run HANDLED end-to-end (no call_builtin
fallback to the legacy interp).

- Tagged-union enum_init WITH payload: allocate zeroed, write the tag at
  offset 0, copy the payload at tag_size ({ header, [N x i8] } layout).
- New .call_builtin exec arm -> callBuiltinVm (VM-native mirror of the legacy
  execBuiltinInner): declare(name) mints an empty forward nominal slot (shared
  declareNominal, also used by declare_type); define(handle, info) reads the
  TypeInfo tagged-union VALUE from flat memory and mints via defineFromInfo,
  a faithful port of legacy defineEnum/defineStruct/defineTuple (all-void enum
  -> real .enum per issue 0142, dup-name rejection, updatePreservingKey vs
  replaceKeyedInfo). Unmodeled builtins bail -> legacy fallback (dual-path).
- Refactored the []{name,ty} decode out of registerTypeVm into a shared
  decodeMemberSlice (+ decodeTypeSlice for bare-Type tuple elements).
- Correctness guard: enum_init/define assume a tag-headed layout, wrong for a
  backing_type tagged union (laid out as the backing struct) — both now bail
  loudly on backing_type != null rather than silent-clobber.

Examples 0614/0620/0621/0624/0632 run fully HANDLED on the VM; 0622/0623 run
define HANDLED then fall back at the still-unported type_info. VM output
byte-matches legacy for all 7. 697/0 both gates + all unit tests (added:
tagged-union enum_init payload layout).
2026-06-18 15:48:48 +03:00

870 lines
68 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`)
Companion to the design-of-record
[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan
+ phased build order live there). This stream supersedes the metatype
`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
## ⏯ Resume (fresh session)
> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.**
> The **byte-weld + serialization/marshaling** approach is the wrong direction and is
> being **stripped**. New foundation: a **bytecode VM over flat, byte-addressable
> memory** so comptime values are native bytes; then the compiler-API rides on it with
> direct memory access (no weld, no validation, no marshaling). Everything below this
> banner describes the now-superseded weld state (committed on `reify` through
> `40d075c`) and is kept only to scope the Phase 0 strip. Read
> `PLAN-COMPILER-VM.md` first.
>
> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as
> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)`
> reads the `Value` union's memory, not a struct. The weld tried to bridge that with
> hand-marshaling — exactly what the design set out to kill. Flat memory makes comptime
> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it
> breaks cross-compilation — host vs target layout — and loses the sandbox. A
> flat-memory VM keeps both while getting native bytes + speed.)
>
> **Next action (2026-06-18) — VM-native metatype CONSTRUCTION landed (step 7, uncommitted).** The
> metatype `declare`/`define` builtins + tagged-union `enum_init`-with-payload now run NATIVELY on
> the VM (new `.call_builtin` exec arm → `callBuiltinVm`/`defineFromInfo`, reading the `TypeInfo`
> value from FLAT MEMORY; faithful port of legacy `defineEnum`/`Struct`/`Tuple`). So the metatype
> CONSTRUCTION examples run **fully HANDLED** on the VM (no `call_builtin` fallback): `0614`/`0620`/
> `0621`/`0624`/`0632`; `0622`/`0623` define-HANDLED then fall back at the still-unported `type_info`.
> Both `enum_init`/`define` bail loudly on a `backing_type` tagged union (wrong layout) rather than
> silent-clobber. **697/0 BOTH gates + all unit tests** (added: tagged-union `enum_init` payload).
> NOT yet committed. **THE NEXT STEP: port `type_info`** (reflect a type → build the `TypeInfo`
> value in flat memory, the inverse of `define` — reuses tagged-union `enum_init`) so `0619`/`0622`/
> `0623` go fully HANDLED; then drive the SX_COMPTIME_FLAT_TRACE fallback list toward
> genuinely-non-comptime cases. Earlier landed: dedicated `Type` builtin TypeId (`6844fb9`/`94f60c5`/
> `554871b`); WRITE side declare_type/register_type/pointer_to VM-native (`66005af`); real
> lowering-time Context for allocating type-fns (`eb68d9e`). What's LEFT toward the end-state ONE
> evaluator: (1) finish porting the comptime corpus onto the VM (type_info next); (2) THEN flip the
> VM to default + delete `interp.zig` (with user go-ahead); (3) re-express `define`/`make_enum` as
> sx over the compiler-API once legacy is gone (allocation works only on the sole VM evaluator).
>
> Done so far in Phase 3:
> - **READ side (7 readers, dual-path):** `find_type`/`type_kind`/`type_field_count`/
> `type_nominal_name`/`type_field_name`/`type_field_type`/`type_field_value`, each backed by a
> `TypeTable` query both the legacy handler and the VM call (no drift). Examples 06280630.
> - **WRITE side (P3.3, legacy-only at lowering time):** `declare_type` + `pointer_to` + ONE
> kind-branching `register_type` (subsumes `define`'s per-kind dispatch; codes match
> `type_kind`: 1 struct · 2 actual `.@"enum"` · 3 tagged_union · 4 tuple). Idempotent re-fill
> (two-edge import). Plus two fixes (issue 0142): all-void enum → real `.@"enum"` (was a
> verifySizes panic); bare `EnumType.variant` qualified construction. Examples 06310635, 0187.
> - **Lowering-time VM (P3.4):** hardened the VM against malformed lowering-time IR (`refTy`,
> bailing `aggType`, bounds-checked branch targets — bails, never panics); wired `tryEval`
> into `runComptimeTypeFunc` behind the flag with legacy fallback; materialized a zeroed
> lowering-time `Context` (the global isn't built yet at lowering). All measured green.
>
> **THE WALL (next step):** a `Type` *value* is an 8-byte tid, but `.any` (the boxed-any) is a
> 16-byte `{tag,value}` — and they share one TypeId (`.any`). So a `Type` in an aggregate
> (`Member.ty`/`EnumVariant.payload`) is sized 16B while the value is 8B → every lowering-time
> type-fn bails at `const_type` / the Member-array build. Can't make `kindOf(.any)` a word:
> at EMIT time `.any` really is a 16B box (variadic any, 0603), so that would silently corrupt
> it. The correct fix is a **dedicated `Type` builtin TypeId (8B), distinct from `.any`** —
> measured at **~123 `.any` references across ~25 files** (pack.zig has 30), a ~100-touch-point
> cross-cutting change → its own focused session (USER CHOSE to pause rather than rush it).
> Rejected alternatives: a scoped "lowering-mode treats `.any` as a word" flag (silent-wrong on
> a real Any box in a reflection type-fn); scalar-only Type-fns (safe but no real corpus type-fn
> is scalar-only — they all build a Member/variant aggregate).
>
> **Decisions recorded:** `find_type` returns a non-optional `TypeId` using `unresolved`(0), NOT
> `?Type`; reader names use the `type_*` family (avoid colliding with std `field_name`/`type_name`);
> the write side is a single kind-branching `register_type`; the write side stays LEGACY-only
> until the VM runs at lowering time (needs the `Type` TypeId). End-state guarantee: ONE
> evaluator — `interp.zig` deleted; dual-path + fallback are transitional (see PLAN end state).
> Build/verify: `zig build && zig build test` (**697**, gate OFF). Run the corpus ON the VM:
> `zig build test -Dcomptime-flat` OR env `SX_COMPTIME_FLAT=1`. Coverage trace:
> `SX_COMPTIME_FLAT_TRACE=1` (now also prints lowering-time `type-fn` HANDLED/fallback lines).
### (superseded) prior weld resume
Phase 1 done; Phase 2 welded structs were working via reflection + memory-order
validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even
earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in
the compiler type's MEMORY order; the compiler reflected the bound Zig type and
VALIDATED the header. **This whole mechanism is now being stripped — see the banner.**
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
> See CLAUDE.md → Snapshot integrity.
## Last completed step
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
end-to-end as a byte-identical mirror of the bound Zig type.
Design (locked, supersedes the byte-layout-override plan):
- The sx header declares fields in the compiler type's MEMORY order. The compiler
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
byte-blob, no index/remap tables, no reorder, no special LLVM path.
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
*wrong field order at position N* (+ expected memory order), *type layout
mismatch* (field size), *layout mismatch* (total size / count).
What changed from the dropped plan:
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
the sx header against the memory-ordered registry.
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
(+ `weldedFieldOrderStr`).
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
refreshed.
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
byte-layout-override engine; superseded by the reflection+validation design above.
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
padding included), so it passes to a Zig handler as raw memory with zero
marshalling. (The alternative — handlers reading interp `Value` aggregates
logically, no layout override — was rejected; welded types must also be usable as
runtime data, and the design wants the literal byte weld.)
- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20,
`is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override
is genuinely required (`Field`'s two-u32 natural layout was the easy case).
- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second
`bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc,
fields, total)` — pure: orders fields by ascending byte offset, inserts padding
elements for gaps + the alignment tail, and builds the sx-field → LLVM-element
remap. This is what the LLVM type builder + struct-GEP sites will consume.
- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad);
`StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24,
pad@25..32]`, remap `[1,0,3,2]`.
- `zig build` + `zig build test` green.
### Earlier — Phase 1 polish (comptime-only enforcement)
**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error
instead of an undefined-symbol link failure.**
- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded`
AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a
`#run`/`::` initializer wrapper whose LLVM body is dead), print a clear
"comptime-only … cannot be called at runtime" error and set
`comptime_failed` (the driver halts before object/JIT emission). The enclosing
`is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green.
- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)`
→ clean error, exit 1, no link failure).
- `zig build` + `zig build test` green (458 unit + 690 corpus).
### Earlier — fifth sub-step (host-call bridge)
**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to
its registered Zig handler instead of dlsym.**
- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`,
`bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the
string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value ->
Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)`
(the same mutable-table access the metatype mint path uses); `text_of` reads the
const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern).
- IR `Function` gained `compiler_welded: bool`. `declareFunction`
(`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES:
the bound lib must be `compiler` and the name must be on the function-export
list — else a build-gating `.err` (no silent fall-through to dlsym).
- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function
routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the
export list).
- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run
text_of(intern("hello, compiler"))` folds to a string constant → prints it);
`examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name →
build error). `findFn` lookup unit-tested.
- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a
RUNTIME call would emit a reference to a non-existent extern symbol → a loud
LINK error (not silent, but not a tidy diagnostic). The examples call welded fns
only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the
immediate follow-up.
- `zig build` + `zig build test` green (458 unit tests + 689 corpus).
### Earlier — fourth sub-step (welded-struct layout validation)
**A `struct abi(.zig) extern compiler { … }` is validated against the binding
registry as a *header checked against the implementation*.**
- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns
the first `LayoutMismatch` (field count / name / size / total) or null. Plus
`lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes;
each drift flagged as the right variant).
- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`,
`validateWeldedStruct` checks the bound lib is `compiler`, the name is on the
export list (`findType`), and the sx layout (field names + `typeSizeBytes` +
total) matches the welded type — emitting a build-gating `.err` (good span into
the struct body) on any failure. No silent reinterpretation.
- `#library "compiler"` is the comptime-only internal surface, NOT a dylib —
`src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so`
load warning).
- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds,
validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld-
struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic).
- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it
isn't exercised by `Field` (two u32s = natural layout coincides with the weld).
It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound
offsets actually differ from the sx-natural ones. The validation already checks
per-field size + total, so a layout drift is caught even before the override
engine exists.
- `zig build` + `zig build test` green (456 unit tests + 687 corpus).
### Earlier — third sub-step (binding registry)
**The binding registry (welded-type lookup, layout baked from the real Zig
type).**
- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the
curated safety boundary. `BoundType { sx_name, size, alignment, fields:
[]FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real
Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a
sx-field-count mismatch is a `@compileError`, never a silent truncation).
`bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` —
two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl
resolution path will consult (returns null off the export list — clean boundary,
no silent default).
- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`.
- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real
`StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4);
an unexported name returns null. Break-verified (a wrong size → suite red,
named `ir.compiler_lib.test...`).
- `zig build` + `zig build test` green (454 unit tests).
### Earlier — second sub-step (struct-decl parse)
**`abi(.zig) extern <lib>` PARSES on a STRUCT decl (parse-only, no semantics).**
- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields.
- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler`
check), parse an optional `abi(...)` then optional `extern <lib>` — same slot
order as fn decls — and thread them onto the node. Ordinary structs are
unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent).
- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern
compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib
== "compiler"`, field list intact; a plain struct leaves `abi == .default` /
`extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite
red, confirming the test runs).
- `zig build` + `zig build test` green.
### Earlier — first sub-step (fn decls) + the syntax pivot
**`abi(.zig) extern <lib>` PARSES on a fn decl (parse-only).** Plus the syntax
pivot it required.
Syntax decision (locked 2026-06-17, supersedes the doc's original
`extern(.zig) <lib>` single-qualifier form): the ABI/layout selector and the
linkage keyword are two orthogonal annotations.
- `abi(.x)` — ABI / calling-convention annotation in the slot **before**
`extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.**
`ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the
`compiler` library), `.pure` (naked asm), `.default` (unannotated). Can appear
standalone (no extern) on any fn / fn-type / lambda.
- `extern <lib>` — linkage keyword + binding source (named library).
So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`.
What landed:
- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`;
the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`.
- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`,
keyword string `"callconv"` → `"abi"`.
- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi`
(parses `abi(.c|.zig|.pure)`); wired in the fn-decl postfix slot (before
`extern`/`export`), the function-type-expr slot, and the lambda slot;
`isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`.
- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`,
`closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the
function-type resolver maps `.zig`/`.pure` → IR `.default` (no fn-pointer-type
CC for those decl-level ABIs; neither occurs in a function-TYPE position yet).
- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the
user-facing text `callconv(.c)` → `abi(.c)`.
- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type
callback annotations — none in the fn-decl postfix slot, so no reordering).
- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 /
1200) regenerated for the rename.
- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern <lib>`
on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib ==
"compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` /
`abi(.pure)`. lexer/sema tests updated.
`zig build` + `zig build test` green (450/450 unit + 685 corpus).
## Current state
> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld
> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for
> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the
> weld *semantics* (layout reflection/validation, marshaling dispatch) go.
- `compiler :: #library "compiler";` parses + is recognised as the comptime-only
internal surface (never dlopen'd).
- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry
(faithful → ok; drift → build-gating diagnostic). `Field` welds + usable.
- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to
their registered Zig handler (`intern`/`text_of` round-trip works); unexported
names rejected at declaration. Comptime-only.
- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only
enforcement at `emitCall`); the legitimate `#run`/`::` use stays green.
- The whole Phase 1 foundation (parse → registry → struct-layout validation →
function host-call bridge → comptime-only enforcement) is in place for the
two-u32 `Field` case + the two string readers.
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
(needed by `StructInfo`'s slice field, Phase 2).
## Next step — execute `PLAN-COMPILER-VM.md`
> The weld is being stripped. The next step is **Phase 0 of
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize /
> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig`
> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics
> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then
> Phase 1 (flat-memory value model). The weld-era "next step" below is **obsolete** —
> kept only as a record of what the weld surface was about to do.
### (obsolete) weld-era next step
Welded structs were byte-identical mirrors, so the API surface was set to grow:
- **Bind `register_struct` / `find_type`** over the host-call bridge
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
kind well-formedness — the checks `define` does today). Because the welded
`StructInfo` is byte-identical, the handler can read it as the real Zig
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
Prove: build a struct programmatically + round-trip a source one.
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
(`defineStruct` / the `reflectTypeInfo` struct path).
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
(optional fields → sentinels) — each just needs an sx header in the compiler
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
`#compiler`.
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
when reached (sentinels or accessor fns; see the design doc Risks).
## Known issues
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
## Log
- **Phase 3 P3.4 step 7 (VM plan) — VM-native metatype CONSTRUCTION: `declare`/`define` + tagged-union `enum_init` (2026-06-18).**
Ported the metatype type-CONSTRUCTION builtins into the VM so the construction examples run
HANDLED end-to-end (no `call_builtin` fallback). Three pieces: (1) **tagged-union `enum_init`
with payload** — the arm previously bailed; now allocates the value (zeroed), writes the tag at
offset 0 (`{ header(tag)@0, [N x i8] payload@tag_size }`, the LLVM `backend/llvm/types.zig`
layout) and copies the payload at `tag_size`. (2) A **`.call_builtin` exec arm** → new
`callBuiltinVm`, the VM-native mirror of the legacy `execBuiltinInner`: `declare(name)` mints an
empty forward nominal slot (shared `declareNominal` helper, also used by `declare_type`);
`define(handle, info)` reads the `TypeInfo` tagged-union VALUE from FLAT MEMORY (tag@0, active
payload `EnumInfo`/`StructInfo`/`TupleInfo` struct at `tag_size`, its single slice field) and
mints via `defineFromInfo`, a faithful port of legacy `defineEnum`/`defineStruct`/`defineTuple`
(all-void enum → real `.@"enum"` per issue 0142, dup-name rejection, `updatePreservingKey` vs
`replaceKeyedInfo`). (3) Refactored the `[]{name,ty}` decode out of `registerTypeVm` into a
shared `decodeMemberSlice` (+ `decodeTypeSlice` for bare-`Type` tuple elements), keyed to the
module-level `NamedMember`. Unmodeled builtins (`type_info`/`type_name`/…) return null → bail
with the builtin name → legacy fallback (dual-path parity). **Correctness guard (caught via
review):** `enum_init`/`define` assume a tag-headed layout, which is WRONG for a `backing_type`
tagged union (laid out as the backing struct) — both now bail loudly on `backing_type != null`
rather than silent-clobber. **Result:** examples `0614`/`0620`/`0621`/`0624`/`0632` run **fully
HANDLED** on the VM (define is the whole eval); `0622`/`0623` run define HANDLED then fall back
cleanly at the still-unported `type_info` reflection. VM output byte-matches legacy for all 7.
**697/0 BOTH gates + all unit tests (added: tagged-union `enum_init` payload layout).** On
`reify`. **Next:** port `type_info` (REFLECT a type → build a `TypeInfo` value in flat memory,
the inverse — reuses the tagged-union `enum_init` write) so `0619`/`0622`/`0623` go fully HANDLED;
then the rest of the comptime corpus (drive the SX_COMPTIME_FLAT_TRACE fallback list toward the
genuinely-non-comptime cases) before the VM-default flip + legacy deletion.
- **Phase 3 P3.4 step 6 (VM plan) — REAL lowering-time Context: allocating + List-building type-fns now run HANDLED on the VM (2026-06-18).**
The VM can now evaluate a comptime type-fn that ALLOCATES at lowering time (the 0141 family) —
the legacy interp cannot. Four changes: (1) `runComptimeTypeFunc` (lower/comptime.zig) FORCES the
CAllocator→Allocator thunks to exist (`getOrCreateThunks`, idempotent, guarded by Allocator/
CAllocator registered) BEFORE eval — a type-fn const runs at scanDecls (Pass 1), before Pass 1c
builds the default-context global + thunks, so the comptime allocator was otherwise null;
(2) `materializeDefaultContext` builds a REAL context at lowering time when the global is absent —
finds the two thunks by name (`findFuncByName`) and lays their func-refs into the inline
`Allocator` value `{ctx=null, alloc_fn@+ptr, dealloc_fn@+2*ptr}` at the head of `Context`, so
`context.allocator.alloc_bytes` dispatches `call_indirect` → thunk → native VM `malloc`;
(3) `aggType` now DEREFS a pointer `base_type` (the List write path emits `struct_gep` with
`base_type = *Struct` — `fieldOffset` panicked on the pointer; now derefs to the pointee, no
panic); (4) `subslice` handles a `[*]T` many-pointer / `*T` base (a List's `items` field — the
base IS the data pointer). **Verified end-to-end (manual probe):** a compiler-API type-fn that
builds its `[]Member` in a `List(Member)` (`.append` ×3, then `register_type(handle, kind,
vs.items[0..vs.len])`) runs **HANDLED on the VM** and mints correctly (`green=7`) — the exact
0141 List-growth pattern, on the VM. **Can't be a corpus test yet** (gate-OFF/legacy still can't
allocate at lowering time — the dual-path bind), so locked in via VM unit tests instead
(many-pointer subslice; `struct_gep` with a pointer `base_type`). **697/0 BOTH gates + all unit
tests, EXIT=0.** On `reify`. **Remaining for the original 0141 repro (uses metatype `define`/
`make_enum` → `call_builtin` → legacy fallback → legacy fails):** re-express the metatype over the
compiler-API so the whole type-fn runs on the VM (no `call_builtin`). THEN the repro works on the
VM — and the dual-path bind resolves only at the VM-default-flip + legacy-deletion end-state.
- **Phase 3 P3.4 — investigation: the "real lowering-time Context" is BLOCKED by issue 0141 (2026-06-18).**
Probed whether the VM needs a REAL lowering-time `Context` (CAllocator thunk func-refs) for
allocating type-fns. **Finding: lowering-time comptime ALLOCATION fails in the LEGACY interp
too** — a type-fn that calls `context.allocator.alloc_bytes` at lowering time bails in legacy
with `comptime call_indirect: callee is not a func_ref Value (raw fn-pointers from extern calls
aren't dispatchable in interp)`, and the VM bails at parity (`call_indirect through a null
function pointer`). This is exactly issue **0141**'s root cause (its analysis already notes "the
null allocator is the same story for the CAllocator thunks") — an OPEN deferred issue. So:
(1) the VM is CORRECT (parity — both bail; no regression); (2) the real-context work is
PREMATURE — its only consumer (allocating lowering-time type-fns) can't pass gate-OFF, so no
corpus test can validate it, and even a more-capable VM can't ship a divergence during the
dual-path phase. **Consequence for the metatype re-expression:** re-expressing `define`/`make_enum`
over the compiler-API needs to BUILD `[]Member` slices dynamically (allocation) — which is
blocked by 0141 at lowering time. The viable paths are: (a) avoid allocation by passing the
caller's existing slice through (needs `EnumVariant`/`StructField` to be usable AS `Member` —
they're layout-identical `{string, Type}`, but distinct nominal types — a metatype-API decision),
or (b) wait for 0141. **No code change this step** (the VM already bails correctly). Recorded so
the next session doesn't re-derive it. 697/0 both gates unchanged.
- **Phase 3 P3.4 step 5 (VM plan) — WRITE side ported to the VM → FIRST HANDLED lowering-time type-fns (2026-06-18).**
Ported `declare_type` / `pointer_to` / `register_type` into `Vm.callCompilerFn`, mirroring the
legacy `compiler_lib` handlers (mint via `@constCast(table)` — the same mutable access the
read-side `intern` uses; the lowering-time mint target IS `&module.types`). `register_type`
reads the `[]Member` slice from FLAT MEMORY: threaded `ref_types` through `invoke` →
`callCompilerFn` so the slice's element type (`Member = {name: string, ty: Type}`) gives the
field offsets + stride; decodes each `{name, ty}` and branches on `kind` (1 struct · 2 enum ·
3 tagged_union · 4 tuple) exactly as legacy (dup-name / payload-on-enum rejections, idempotent
re-fill via `nominalIdentOf`). **Key unblock:** the synthesized comptime type-fn wrapper
(`createComptimeFunction`/`…WithPrelude`) was built with return type `.any` → `regToValue`
bailed at the VM↔legacy boundary; changed to `.type_value` (the legacy path reads via `asTypeId`
regardless, so no legacy change). **Result: the compiler-API write type-fns now run HANDLED
end-to-end on the VM at LOWERING time** — `0631` (register-graph: 2 HANDLED, A↔B cycle via
forward handles + `pointer_to`) and `0635` (multi-edge import: 2 HANDLED), parity-correct. They
run on the ZEROED lowering-time context (fixed `.[…]` member arrays, no allocation). The
metatype `make_enum`/`define` examples (`0632`) still fall back CLEANLY through
`call_builtin(define)` (the separate metatype path — re-expressing it onto the compiler-API is
the other half of P3.4). **697/0 BOTH gates + EXIT=0.** On `reify`. **Next:** (optional, deferred)
a REAL lowering-time Context (CAllocator thunk func-refs) for List-growing type-fns; and
re-express the metatype `define`/`make_enum` over the compiler-API to delete the bespoke interp
arms (the end-state: ONE evaluator).
- **Phase 3 P3.4 step 4 (VM plan) — model `.type_value` natively in the comptime VM (2026-06-18).**
The VM now HANDLES Type values instead of bailing: `kindOf(.type_value)` → `.word`; a new
`const_type` exec arm → the word `TypeId.index()`; `regToValue` maps a `.type_value` word back
to a `.type_tag` Value at the legacy boundary (`valueToReg` already mapped `.type_tag` →
index). Surfaced + fixed a VM PANIC (forbidden): `struct_init` assumed a `.@"struct"` result
type and union-access-panicked on an ARRAY literal (`EnumVariant.[ … ]`, reached now that Type
args no longer bail early) — it's the generic aggregate-literal op, so it now dispatches on the
result kind (struct / array / tuple) and BAILS loudly on anything else, never panics. **697/0
both gates** (the make_enum type-fns now run further on the VM, then bail cleanly at the
`define`/`make_enum` `call_builtin` → legacy mints — no mutation before the bail, parity holds).
VM unit test added (const_type → word → regToValue → `.type_tag`). On `reify`. **Next (the
payoff):** port the WRITE side (declare_type / register_type / pointer_to) into
`Vm.callCompilerFn` + give the lowering-time path a REAL Context (CAllocator thunk func-refs,
not zeroed) → the first HANDLED lowering-time type-fn end-to-end on the VM.
- **Phase 3 P3.4 step 3 (VM plan) — dedicated `Type` builtin TypeId: RESOLVER FLIPPED + `.any` migration (2026-06-18).**
Flipped `type_resolver:64` (`"Type"` → `.type_value`), `module.zig` `constType` (result type
→ `.type_value`), and `emitConstType` (a bare i64 carrying `tid.index()`, NOT a 16-byte Any
box). Then migrated every `.any` reference that means "a Type value", classified per CLAUDE.md
(leave the real boxed-Any refs): (a) the "Any holds a Type" **meta-marker tag** moved `.any` →
`.type_value` at all 4 consumers — `reflectArgTypeId` (LLVM), `reflectTypeId` + the
`.type_tag`-as-struct-field comptime path (interp), and `resolveTypeCategoryTags("type")`
(generic.zig); (b) reflection-builtin RETURN types `.any` → `.type_value` (`type_of`/`declare`/
`define`); the runtime `type_of(any)` now reads the tag AS a `.type_value` (no re-box); (c)
expr_typer infers a bare type-name expr as `.type_value` (with a `is_raw` backtick exemption —
`` `string `` is a value, never the reserved type); (d) `reflectionArgIsType` accepts
`.type_value` OR `.any` (a reflection arg can be a bare Type OR a boxed Any — the over-narrow
`==.type_value` was the catastrophic-regression cause, caught + fixed); (e) the comptime
`switch_br` accepts a `.type_tag` discriminant (type-category match); (f) a bare function name
in a `Type` slot now lowers to `const_type(its real function type)` instead of a func-ref
(fixed a JIT crash — was a func-ref word read as a TypeId), keeping the old string-box path only
for genuine `Any` params; (g) the field-not-found diagnostic + `formatTypeName` render
`.type_value` as "Type". Fixed 3 unit tests asserting the old `.any` Type behavior.
**697/0 BOTH gates** + all 494 unit tests (EXIT=0). Gate ON stays green because the VM's
`kindOf(.type_value)` → `.unsupported` → bails CLEANLY to legacy (no silent-wrong) — the VM
doesn't model `Type` values YET (next step), but parity holds. Regenerated 24 snapshots (22
`.ir` const_type-shape; 2 `.stderr` Any→Type — diff reviewed, only the intended changes). On
`reify`. **Next:** model `.type_value` natively in the VM (`kindOf` → word, `const_type` → word
= `TypeId.index()`, `regToValue` word → `.type_tag`) for COVERAGE, then port the WRITE side into
`callCompilerFn` + a real lowering-time Context → the first HANDLED lowering-time type-fn.
- **Phase 3 P3.4 step 2 (VM plan) — dedicated `Type` builtin TypeId: FOUNDATION landed (dead/additive) (2026-06-18).**
Added `TypeId.type_value` (slot 19) + a matching `TypeInfo.type_value` variant + the builtins
init entry — an **8-byte type handle distinct from the 16-byte boxed `.any`** (THE WALL). All
`types.zig` layout handlers wired: `sizeOf`/`typeSizeBytes` → 8, `typeAlignBytes` → 8,
`typeName` → "Type", `hashTypeInfo`/`typeInfoEql` no-payload arms. Only ONE exhaustive switch
needed a new arm (`backend/llvm/types.zig` `toLLVMTypeInfo` → `cached_i64`); every other
`switch(TypeInfo)` site has an `else` (audited when the resolver flips). **`first_user` 19 → 100**
(per the user): slots 2099 are RESERVED builtin headroom (infos padded with the `unresolved`
tripwire), so future builtins don't renumber user TypeIds / churn `sx ir` snapshots. Cost:
~80 default entries in each binary's per-type reflection arrays (user opted in). **Still dead:**
`type_resolver.zig:64` STILL returns `.any` for "Type" — nothing produces `.type_value` yet, so
NO behavior change. Regenerated 22 IR snapshots (pure TypeId renumber to 100-base; `git diff
--name-only` confirmed ONLY `.ir` files + the 2 source files changed — no stdout/stderr/exit).
**697/0 both gates** (OFF and `-Dcomptime-flat`). **Next:** flip `type_resolver:64` →
`.type_value`, then migrate the `.any` refs that mean "a Type value" (const_type result /
reflection returns / metatype `Type` params / `.type_tag` checks) — leave the real boxed-Any
refs — file-by-file with a build after each.
- **Phase 3 P3.4 step 1 (VM plan) — lowering-time default context; first blocker cleared (2026-06-18).**
`materializeDefaultContext` now falls back to a ZEROED `Context` (found by name) when the
`__sx_default_context` global is absent — i.e. at LOWERING time, where the global isn't
emitted yet. A type-fn that never touches the allocator now runs past context setup; one
that allocates reads a null `alloc_fn` (zeroed) → `call_indirect` on the null func-ref
bails → legacy fallback (a REAL lowering-time context with the CAllocator thunk func-refs,
so allocating type-fns also run on the VM, is a follow-up). **Measurement: the bail moved
deeper** — metatype `make_enum` now bails at `const_type` (the `Type`-literal op, unported);
`register_type` type-fns bail at the welded write call (declare_type/register_type aren't in
`callCompilerFn`). No table mutation happens before either bail (the write fns bail before
minting), so parity holds: both gates **697/0**, no crashes. **Next blockers (the "model
Type" chunk):** (a) the `const_type` op → a word = `TypeId.index()`; (b) the Type-return
bridge (`regToValue` for a `Type`/`.any` word → `.type_tag`); (c) the VM-native write side
(declare_type/register_type/pointer_to in `callCompilerFn`) + a real lowering-time context.
Only once those land does a type-fn actually run end-to-end on the VM (a HANDLED case).
- **Phase 3 P3.4 (VM plan) — wire the VM at the LOWERING-time site + measure (2026-06-18).**
Routed `runComptimeTypeFunc` (the type-fn fold — the THIRD comptime call site) through
`comptime_vm.tryEval` behind `-Dcomptime-flat`/`SX_COMPTIME_FLAT` with legacy fallback,
mirroring the two emit-time folds. Extracted the shared post-check (`checkComptimeTypeResult`
— the declared-but-never-defined zero-field guard) so both paths use it. **Measurement
(SX_COMPTIME_FLAT_TRACE):** every metatype/compiler-API type-fn currently bails CLEANLY
with `no __sx_default_context global to materialize the implicit context` — at lowering
time the default-context global doesn't exist yet (it's built at emit time), so the VM bails
at context materialization, BEFORE running the body (no partial mint, no crash → legacy
mints). The hardening holds: **no crashes** across the corpus on the VM lowering-time path.
Both gates **697/0**. **So the FIRST lowering-time blocker is the implicit context, not
`Type` modeling** — the VM needs a way to materialize/skip the default context at lowering
time (most type-fns get an implicit ctx for potential `List`-growth alloc; many don't use
it). Next: materialize a lowering-time default context for the VM (or pass a null ctx +
bail only if the allocator is actually used), THEN model `Type` values + the VM-native write
side. This is near-pure fallback today — permanent scaffolding that lights up as those land.
- **Phase 3 P3.4-prep (VM plan) — harden the VM against malformed lowering-time IR (2026-06-18).**
Prerequisite for wiring the VM at the LOWERING-time comptime site (`runComptimeTypeFunc`),
where IR can be malformed (an unresolved name lowers to a dangling / `Ref.none` operand —
the 0737 crash). Closed the remaining panic vectors so the VM BAILS (→ legacy fallback)
instead of aborting: (1) a checked `Vm.refTy(ref_types, r)` replaces every raw
`ref_types[ref.index()]` in `exec` (the type-side companion to `Frame.get`'s `bad_ref`
value-side guard); (2) `aggType` is now a bailing method (`Error!TypeId`) using `refTy`;
(3) the block-dispatch loop bounds-checks the branch target before indexing
`func.blocks.items`. `global_get` was already guarded. No behavior change — gate OFF and
ON both **697/0**; unit test added (a `cmp_lt` with a `Ref.none` operand bails, not
panics). **Next:** wire `tryEval` into `runComptimeTypeFunc` behind the flag with legacy
fallback and measure (most minting type-fns will still bail at the welded-write call /
`Type`-result conversion until the VM models `Type` values + the VM-native write side land
— those are the steps that actually move lowering-time comptime onto the VM, toward
deleting legacy).
- **Phase 3 P3.3 (VM plan) — WRITE side: declare_type + pointer_to + ONE kind-branching register_type (2026-06-18).**
The mutating compiler-API: `declare_type(name) -> Type` (forward handle), `pointer_to(t) -> Type`
(build `*T`), and `register_type(handle, kind, members: []Member) -> Type` which branches on
`kind` IN THE COMPILER (subsuming define's per-kind dispatch). Take/return real `Type` values
(matching meta.sx declare/define). **Timing (per user): mint LAZILY at lowering time, single
pass** (the existing `runComptimeTypeFunc`), so the write side is **legacy-only** (`compiler_lib`
handlers) — the VM isn't wired at lowering time, no VM mirror needed; readers stay dual-path.
A non-generic `-> Type` builder is now flagged `is_comptime` (decl.zig) so its dead body permits
the welded calls. **Graph:** forward handles + `pointer_to` express mutually-recursive A↔B (`*A`,
`*B`, B-by-value); `register_type` is **idempotent** (re-fill a nominal slot reached via two
import edges — `nominalIdent`). `kind` codes match `type_kind` (1 struct · 2 actual `.@"enum"` ·
3 tagged_union · 4 tuple). **Fixed two bugs (issue 0142):** (a) a fully payloadless minted enum
was an all-void tagged_union → verifySizes panic; now a real `.@"enum"` (register_type kind 2 AND
metatype `defineEnum`); (b) bare `EnumType.variant` payloadless qualified construction wasn't
supported (failed for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
Examples 0631 (graph + actual enum + reflection), 0632 (make_enum all-void), 0633/0634/0635
(namespaced / bare / multi-edge import of a minted type), 0187 (qualified variant construction).
**Parity 697/697** (gate ON and OFF); unit tests added. **Next (P3.4):** re-express
declare/define/type_info as sx over the compiler-API + delete the bespoke interp arms (needs the
VM hardened for lowering-time IR, or the metatype migrated onto the legacy compiler-API calls).
- **Phase 3 P3.2b (VM plan) — kind + enum-value readers: `type_kind` + `type_field_value`; READ side complete (2026-06-18).**
The last two read-only readers the metatype's `type_info(T)` needs (added to
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`, each backed by a `TypeTable` query both
call): `type_kind(t) -> i64` (`kindCode` — a stable, compiler-owned discriminant: 0 other ·
1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · 7 vector · 8 error_set;
TOTAL, never bails) and `type_field_value(t, idx) -> i64` (`memberValue` — an enum variant's
explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for non-enum /
out-of-range). Example `0630-comptime-compiler-type-kind` reflects `Color` / `WindowFlags`
(flags) / `Point`. **The READ side is now COMPLETE** — `find_type` + `type_kind` +
`type_field_count` + `type_field_name`/`type_field_type`/`type_nominal_name` +
`type_field_value` cover everything `reflectTypeInfo` reads. VM unit test added. **Parity
691/691** (gate ON and OFF). **Revised forward direction (per the user):** the WRITE side is
ONE `register_type(info)` fn that branches on the kind IN THE COMPILER (subsuming `define`'s
per-kind dispatch), not a per-kind `register_struct`.
- **Phase 3 P3.2 (VM plan) — field-level reflection readers: `type_nominal_name` + `type_field_name` + `type_field_type` (2026-06-18).**
Three more `compiler`-library readers on the same `TypeId`-handle shape (added to
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`), each backed by a new `TypeTable` query
BOTH paths call (no drift): `nominalName` (a named type's own name handle; loud-bail for
unnamed types like `i64`/pointers), `memberName` (struct/union/tagged-union field, enum
variant, named-tuple element), `memberType` (struct/tuple/array/vector member type). All
loud-bail on out-of-range idx / no-member (no silent default). First MULTI-ARG compiler
fns — `callCompilerFn` reads arg 1 = idx; added `Vm.argHandle`/`argTypeId` (range-checked
u32/TypeId arg reads) and refactored `find_type`/`type_field_count` onto them. Named
`type_*` to avoid clashing with the std metatype builtins (`field_name`/`type_name` exist
in core.sx); `nominalName` (the TypeTable method) is distinct from the existing
`typeName(id) []const u8` display-string renderer. Example `0629-comptime-compiler-field-reflect`
reflects `Pair { lo: Point; hi: Point }` — each field name + the nominal name of a field's
type, all `#run`-folded, all VM-HANDLED natively. VM unit test added (type_field_name → "hi";
type_nominal_name(type_field_type(Pair,0)) → "Point"). **Parity 690/690** (gate ON and OFF).
- **Phase 3 P3.1 (VM plan) — first read-only reflection readers: `find_type` + `type_field_count` (2026-06-18).**
Two more `compiler`-library fns, bound the same way as the `intern`/`text_of` seed (added
to `compiler_lib.bound_fns` for the legacy handler + the welded-decl export check, AND to
`Vm.callCompilerFn` for the native flat-memory path — NO marshaling). A **type handle is a
plain `u32` `TypeId`** (like `StringId`), so both keep the seed's clean scalar shape:
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`, `unresolved`/0 if absent) and
`type_field_count(t: TypeId) -> i64` (a NEW `TypeTable.memberCount` query — struct/union/
tagged-union fields, enum variants, array/vector length — called by BOTH paths so they
can't drift; bails loudly, never a silent 0). New example `0628-comptime-compiler-find-type`
chains `intern → find_type → type_field_count` (and a not-found lookup → 0), both folded at
`#run`, both VM-HANDLED natively (trace confirms no fallback). VM unit test added
(`find_type` + `type_field_count`, struct found → 3 fields, missing → `unresolved`).
**Parity 689/689** (gate ON and OFF). **Decision (resolves the plan's `find_type → ?Type`
sketch):** return a NON-optional `TypeId` with the `unresolved` (0) sentinel for not-found,
NOT `?Type` — a `Type` value resolves to `.any` (which the flat-memory VM doesn't represent)
and an optional can't cross the legacy↔VM eval boundary; `unresolved` is the project-blessed
unmistakable "no type" marker. Forward (P3.2): more readers on the same handle shape
(`type_name`/`field_name`/`field_type`/kind), then `register_struct` (first mutating fn).
- **VM robustness — `Frame` bounds-check; lowering-time `#insert` wiring explored + reverted (2026-06-18).**
Explored wiring the VM at the LOWERING-time comptime site (`evalComptimeString`, the
`#insert` string fold). 12/13 `#insert` examples ran on the VM with parity, but `0737`
(an `#insert` of an unresolved `secret()`) CRASHED the VM (SIGABRT): lowering-time IR can
be malformed (a `ret Ref.none` from the unresolved name) and `Frame.get` panicked on the
out-of-range index. **Decision: reverted the lowering-time wiring** — unlike the emit-time
folds (fully lowered IR), lowering-time IR can be erroneous, and hardening the VM against
ALL malformed IR (every `ref_types[...]` / `aggType` access, not just `Frame`) is out of
scope here. The emit-time sites already give full corpus coverage. **KEPT** the defensive
fix regardless (CLAUDE.md "never crash"): `Frame.get`/`set` now bounds-check and flip a
`bad_ref` flag; the `run` loop bails (`badRef`) instead of panicking. Unit test added
(malformed `ret Ref.none` → bail, not crash). Parity **688/688** both ways.
- **Phase 3 SEED (VM plan) — compiler-call path: `intern`/`text_of` native on the VM (2026-06-18).**
`invoke` now dispatches a welded `compiler`-library fn (gated on `compiler_welded`) to
`Vm.callCompilerFn`, serviced NATIVELY on flat memory (no legacy `Interpreter`):
`intern(string)->StringId` reads the flat-memory string bytes and `internString`s into the
const-cast table (pool-only — doesn't touch type layout, so cached sizes stay valid);
`text_of(StringId)->string` materializes the pooled text back into flat memory. Unlocked
`0626`; the ONLY remaining const-init fallback is now the inline-asm global (`1654`).
Parity **688/688** (gate ON and OFF); unit test added. This is the mechanism Phase 3 grows
— the next compiler fns (`find_type`, `register_struct`, reflection readers) bind the same
way (flat-memory pointer in, handle/pointer out, no marshaling).
- **Phase 1.final step 9 (VM plan) — `-Dcomptime-flat` build flag (the "swap behind a build flag" step) (2026-06-18).**
Added the `-Dcomptime-flat` build option (build.zig → a `build_opts` options module on
`mod`; `emit_llvm.init` reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`). This is
the plan's "reach parity → swap behind a build flag → delete the old path" mechanism.
`zig build test -Dcomptime-flat` runs the FULL corpus on the VM (688/0). Verified the flag
toggles the binary: flag-built `sx` reports VM HANDLED with no env var; default-built does
not. Default OFF — `zig build test` unchanged (688/0). Env var still works for ad-hoc runs.
Next (forward): Phase 2 (bytecode) / Phase 3 (compiler-API on flat memory); eventual
default-flip + legacy deletion.
- **Phase 1.final step 8 (VM plan) — wire the `#run` side-effect path + trace-clear-on-fallback (2026-06-18).**
Wired the SECOND comptime call site (`runComptimeSideEffects`, top-level `#run <expr>;`)
through `tryEval` with legacy fallback, mirroring the const-init fold. `tryEval` now
handles void/noreturn entries (→ `.void_val`) so a void side-effect doesn't bail at the
result conversion. **Fixed a trace-corruption** the new site exposed (`1035`): a
side-effect that pushes return-trace frames and then bails (e.g. on `print`) had the
legacy re-run DOUBLE-push them (`sx_trace_push` is a side effect on the shared buffer).
Both wiring sites now `sx_trace_clear()` right before the legacy fallback, discarding the
VM's partial pushes. **Parity 688/688** (gate ON and OFF). Most side-effects still bail
(print/global_addr/call_builtin) → legacy, but the path is now uniform. All comptime
evaluation routes through the VM-with-fallback.
- **Phase 1.final step 7 (VM plan) — is_comptime + failable/error cluster + signed-load fix; coverage 31→36 (2026-06-18).**
`is_comptime` → 1 (unlocked `1030`). Ported the failable/error-channel cluster (`1037`
escape, `1038` handled): `kindOf(error_set)→word`, `regToValue` bridges TUPLES (the
failable `(value…,tag)` shape `checkComptimeFailable` reads), `trace_frame` packs
`(func_id<<32|span.start)` from a new `call_stack` (pushed by invoke/runEntry), and
`sx_trace_push`/`sx_trace_clear` serviced NATIVELY (the VM calls the real sx_trace.c
functions linked into the compiler, so the return-trace buffer is populated identically
to legacy). raise/catch/or now run on the VM. **Surfaced + fixed a real GENERAL bug:**
`readField` was ZERO-extending signed sub-64-bit loads, so a stored `i32 -1` reloaded as
`0xFFFFFFFF` (+4.29e9) and `< 0` was false — silently hiding `raise error.Bad`; now
SIGN-extends `i8`/`i16`/`i32`/`isize` (gate-ON parity confirms it's a strict fix; unit
test added). VM HANDLES **36** corpus const-inits (was 31); **parity 688/688** (gate ON
and OFF). Only **2 fallbacks** remain, both principled: `intern` (`0626`, welded
compiler-API fn — Phase 3) + inline-asm global (`1654`). Forward work: Phase 2 (bytecode),
Phase 3 (compiler-API on flat memory).
- **Phase 1.final step 6 (VM plan) — real default context + call_indirect + func_ref + global_get; coverage 27→31 (2026-06-17).**
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context instead of a zeroed one. The implicit-ctx param is
an opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer (`{ {null, alloc_fn, dealloc_fn}, null }`, the CAllocator thunk
func-refs) into flat memory via a new recursive `layoutConst`. With `func_ref` (function
value encoded `FuncId.index()+1`, reserving word 0 for the null fn-ptr) and
`call_indirect` (decode word → FuncId → dispatch; 0 → bail) ported, the whole allocator
protocol runs on the VM:
`context.allocator.alloc_bytes` → call_indirect → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → native flat malloc. Unlocked `0606` (string global). Also: `global_get`
lazily evaluates a comptime global's `comptime_func` (memoized) — unlocked `CT_CHAIN`;
field access (`fieldOffset`/`struct_get`) handles string/slice `{ptr@0,len@8}` fat
pointers (needed by `alloc_string`); `regToValue` maps function-typed words → `.func_ref`
(kept `1128`'s rejection byte-identical). Native `malloc` is still required (the thunk
bottoms out at it; a host pointer can't be used with flat-memory load/store). VM HANDLES
**31** corpus const-inits (was 27); **parity 688/688** (gate ON and OFF). Unit tests:
global_get, func_ref+call_indirect. Remaining fallbacks (7): `.unsupported` aggregates
(3× — `1037`/`1038`), extern/builtin `intern`+asm (2×), `trace_frame`, `is_comptime`.
- **Phase 1.final step 5 cont. (VM plan) — libc memory builtins + f32 fix; coverage 16→27 (2026-06-17).**
Identified the dominant fallback (`call to extern/builtin`) as **11× `malloc`** (0604) +
1× `intern`. Modeled a curated set of libc MEMORY builtins natively on flat memory
(`Vm.callMemBuiltin`): `malloc`/`calloc` → `allocBytes` (16-aligned, 256-MiB cap → bail),
`free` → no-op, `memcpy`/`memmove`/`memset` on flat bytes — sandboxed (no host heap/dlsym),
target-aware; the computed result is byte-identical to legacy (which calls real libc).
This surfaced a **real latent f32 bug**: float registers hold f64 bits, but f32 MEMORY is
the 4-byte single — `readField`/`writeField` were truncating the f64 bits (writing zeros
for `1.0`); now they `@floatCast` on f32 load/store (mirrors legacy `storeAtRawPtr`).
Result: VM HANDLES **27** corpus const-inits (was 16); **parity 688/688** (gate ON and
OFF). Unit tests added (f32 round-trip; malloc → usable flat memory). Next: the `kindOf`
`.unsupported` aggregates (3×), `global_get` (2×), the rest.
- **Phase 1.final step 5 (VM plan) — implicit-context materialization; coverage 0→16 (2026-06-17).**
`tryEval` now MATERIALIZES the implicit ctx instead of skipping it: a `has_implicit_ctx`
comptime entry (sole param `*Context`) gets a zeroed `Context` of the right size/align
in flat memory, its address passed as arg 0. Const bodies that ignore the ctx run; a
body that uses the allocator hits unported `call_indirect` → bails → legacy. No func-ref
materialization needed (handled bodies don't read ctx contents; parity is the guard).
Fixed a real bug surfaced by the coverage pass: storing a `null` non-pointer optional
(the `null_addr` sentinel) into an aggregate slot OOB-bailed — `writeField` now ZEROES
the destination for a `null_addr` aggregate source (= none/empty); unit-test regression
added. Result: VM HANDLES **16** corpus const-inits (was 0); **parity 688/688 both
gate ON and OFF**. Next: port the ops the trace names — `call_builtin`/`compiler_call`/
extern (13×, via the bridge), `kindOf` `.unsupported` aggregates (3×), `global_get` (2×),
func_ref / call_indirect / trace_frame / is_comptime.
- **Phase 1.final steps 14 (VM plan) — host wiring landed; coverage measured (2026-06-17).**
(1) **Hardening:** `Machine.readWord`/`writeWord`/`bytes` now return `error.OutOfBounds`
(null / out-of-range / oversized / overflow-safe) instead of `assert`-panicking;
`OutOfBounds` added to `Vm.Error`; `try` threaded through every helper + exec arm + the
bridge. New unit tests (accessor OOB returns; null-deref → `tryEval` null, not a crash).
(2) **Implicit context:** `tryEval` returns null for `has_implicit_ctx` funcs (legacy
fallback) — conservative; full ctx materialization deferred to step 5. (3) **Wiring:**
const-init fold in `emit_llvm.zig` `emitGlobals` is `(if comptime_flat) tryEval else
null) orelse interp.call(...)`, gated by env `SX_COMPTIME_FLAT` (read once into
`LLVMEmitter.comptime_flat`). Default OFF. (4) **Parity + coverage:** gate ON → full
corpus byte-identical (688, 0 failed) + manual 0605/0606/0607 byte-identical.
**Finding: 0 of 37 measured corpus const-inits are VM-handled — ALL are
`has_implicit_ctx`-gated.** Added a coverage-trace facility (`comptime_vm.last_bail_reason`
+ env `SX_COMPTIME_FLAT_TRACE`). **Next: step 5 = implicit-context materialization** (the
unblocker), then port the deferred ops. 688 corpus green (gate OFF).
- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).**
`comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on
the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back.
Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs
(1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of
crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind
`SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final.
688 corpus green.
- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).**
Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the
wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs
the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg →
Reg, aggregates into flat memory) + `regToValue` (VM result → Value, deep-copied).
Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip
unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through
the VM with legacy fallback).
- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).**
`Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals
arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared
flat memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate
Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b).
Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next:
1.5b (call_builtin/compiler_call/extern), then hybrid wiring.
- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).**
Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested
(deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops
(tagged-union payload/any/closures) blind — their byte semantics are ambiguous without
real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) +
HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so
the VM runs the real corpus and surfaces exactly what's needed. Key design point for
calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's
allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision
block.
- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).**
`kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}`
aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`;
const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91,
?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any,
closures).
- **Phase 1 sub-step 4b (VM plan) — slices + strings on flat memory (2026-06-17).**
`{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate).
Ported `const_string` (text+NUL + fat pointer in flat memory), `length`/`data_ptr`,
`array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and
`str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23,
subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures).
- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on flat memory (2026-06-17).**
`kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get`
(`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over
array/pointer/many_pointer; slice/string bases bail), `length` on array values.
Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next:
sub-step 4b (slices/strings, then optionals/enums/any/closures).
- **Phase 1 sub-step 3 (VM plan) — memory + structs on flat memory (2026-06-17).**
`Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported
`alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out
at the table's natural offsets. Value model: scalar/pointer → register word;
struct → lives in flat memory, its value IS its address (read→addr, write→memcpy), so
nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on
not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested
(struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next:
sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls).
- **Phase 1 sub-step 2 (VM plan) — flat-memory executor: scalars + control flow
(2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over
flat-memory frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy
interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic,
comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param`; every other op bails loudly (`error.Unsupported` + op name in
`detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic,
cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus
untouched (688 green). Next: sub-step 3 (memory + aggregates on flat memory, where
target-aware layout enters).
- **Phase 1 sub-step 1 (VM plan) — flat-memory machine substrate (2026-06-17).**
New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator
with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0
reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on
deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested
(`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy
interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor +
scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule
from CLAUDE.md per request.
- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept
(2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig`
(`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/
`LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr`
+ the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and
examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function
host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3
compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the
`emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples
`0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next:
Phase 1 (flat-memory value model) per `PLAN-COMPILER-VM.md`.
- **DIRECTION CHANGE — pivot off the byte-weld to a flat-memory bytecode VM
(2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong
direction (it hand-marshals onto a comptime value model that isn't bytes — exactly
what the design set out to kill). New foundation: a bytecode VM over flat memory so
comptime values are native bytes; the compiler-API then rides on it via direct memory
(no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks
cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0
strip → Phase 1 flat-memory value model → Phase 2 bytecode → Phase 3 compiler-API on
flat memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted
the session's uncommitted `register_struct`/`find_type` marshaling experiment back to
`reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action.
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
byte-blob — all explored, all unnecessary). Instead: the sx header declares
fields in the compiler type's memory order; the compiler reflects the bound Zig
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
On pass it's an ordinary byte-identical struct — cast + deref just works.
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
superseded by the reflection+validation design and deleted.
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
green (458 unit, 690 corpus).
- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).**
`compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function`
`compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`);
`interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round-
trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean
rejection deferred (loud link error today). Build + suite green (458 unit, 689
corpus).
- **Phase 1.1 fourth sub-step — welded-struct layout validation.**
`validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into
`registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against
the registry (lib == compiler, name exported, layout matches) with build-gating
diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful
Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to
Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit,
687 corpus).
- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`:
the `compiler` lib's welded-type registry; `Field` welded to
`StructInfo.Field` with layout baked from the real Zig type
(`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test
(+ null off the export list). Standalone island — not yet consumed by lowering.
Build + suite green (454 unit tests). Break-verified.
- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl`
gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern <lib>`
after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified.
Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete.
- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig)
extern <lib>` on fn decls; unified `callconv` into `abi(.c|.zig|.pure)` (removed
the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs +
snapshots. Build + suite green. The original design's `extern(.zig)` single
qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern
<lib>` (linkage + source) — recorded in the design doc's syntax-decision note.