Files
sx/current/CHECKPOINT-COMPILER-API.md
agra 7d59b5eeb6 CHECKPOINT-COMPILER-API: refresh resume banner — Phase 3 read+write done, lowering-time VM wired; next is the dedicated Type TypeId
Records the current state (read side, write side P3.3, lowering-time hardening +
wiring + zeroed context P3.4) and pins the next focused step: a dedicated Type
builtin TypeId (8B) distinct from .any (16B box) — ~123 .any refs across ~25 files,
a cross-cutting change to run as its own session. Paused here at a clean, green
boundary (697/697 both gates) per the decision to not rush it.
2026-06-18 12:30:35 +03:00

713 lines
53 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`)
Companion to the design-of-record
[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan
+ phased build order live there). This stream supersedes the metatype
`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
## ⏯ Resume (fresh session)
> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.**
> The **byte-weld + serialization/marshaling** approach is the wrong direction and is
> being **stripped**. New foundation: a **bytecode VM over flat, byte-addressable
> memory** so comptime values are native bytes; then the compiler-API rides on it with
> direct memory access (no weld, no validation, no marshaling). Everything below this
> banner describes the now-superseded weld state (committed on `reify` through
> `40d075c`) and is kept only to scope the Phase 0 strip. Read
> `PLAN-COMPILER-VM.md` first.
>
> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as
> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)`
> reads the `Value` union's memory, not a struct. The weld tried to bridge that with
> hand-marshaling — exactly what the design set out to kill. Flat memory makes comptime
> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it
> breaks cross-compilation — host vs target layout — and loses the sandbox. A
> flat-memory VM keeps both while getting native bytes + speed.)
>
> **Next action (2026-06-18) — PAUSED at a clean boundary; next step decided.** Phase 3 READ
> and WRITE sides are DONE, and the VM is now WIRED at the lowering-time comptime site
> (hardened, with legacy fallback). The first **HANDLED** lowering-time type-fn is gated on a
> **dedicated `Type` builtin TypeId** — that is the next focused effort (see "THE WALL" below).
>
> Done so far in Phase 3:
> - **READ side (7 readers, dual-path):** `find_type`/`type_kind`/`type_field_count`/
> `type_nominal_name`/`type_field_name`/`type_field_type`/`type_field_value`, each backed by a
> `TypeTable` query both the legacy handler and the VM call (no drift). Examples 06280630.
> - **WRITE side (P3.3, legacy-only at lowering time):** `declare_type` + `pointer_to` + ONE
> kind-branching `register_type` (subsumes `define`'s per-kind dispatch; codes match
> `type_kind`: 1 struct · 2 actual `.@"enum"` · 3 tagged_union · 4 tuple). Idempotent re-fill
> (two-edge import). Plus two fixes (issue 0142): all-void enum → real `.@"enum"` (was a
> verifySizes panic); bare `EnumType.variant` qualified construction. Examples 06310635, 0187.
> - **Lowering-time VM (P3.4):** hardened the VM against malformed lowering-time IR (`refTy`,
> bailing `aggType`, bounds-checked branch targets — bails, never panics); wired `tryEval`
> into `runComptimeTypeFunc` behind the flag with legacy fallback; materialized a zeroed
> lowering-time `Context` (the global isn't built yet at lowering). All measured green.
>
> **THE WALL (next step):** a `Type` *value* is an 8-byte tid, but `.any` (the boxed-any) is a
> 16-byte `{tag,value}` — and they share one TypeId (`.any`). So a `Type` in an aggregate
> (`Member.ty`/`EnumVariant.payload`) is sized 16B while the value is 8B → every lowering-time
> type-fn bails at `const_type` / the Member-array build. Can't make `kindOf(.any)` a word:
> at EMIT time `.any` really is a 16B box (variadic any, 0603), so that would silently corrupt
> it. The correct fix is a **dedicated `Type` builtin TypeId (8B), distinct from `.any`** —
> measured at **~123 `.any` references across ~25 files** (pack.zig has 30), a ~100-touch-point
> cross-cutting change → its own focused session (USER CHOSE to pause rather than rush it).
> Rejected alternatives: a scoped "lowering-mode treats `.any` as a word" flag (silent-wrong on
> a real Any box in a reflection type-fn); scalar-only Type-fns (safe but no real corpus type-fn
> is scalar-only — they all build a Member/variant aggregate).
>
> **Decisions recorded:** `find_type` returns a non-optional `TypeId` using `unresolved`(0), NOT
> `?Type`; reader names use the `type_*` family (avoid colliding with std `field_name`/`type_name`);
> the write side is a single kind-branching `register_type`; the write side stays LEGACY-only
> until the VM runs at lowering time (needs the `Type` TypeId). End-state guarantee: ONE
> evaluator — `interp.zig` deleted; dual-path + fallback are transitional (see PLAN end state).
> Build/verify: `zig build && zig build test` (**697**, gate OFF). Run the corpus ON the VM:
> `zig build test -Dcomptime-flat` OR env `SX_COMPTIME_FLAT=1`. Coverage trace:
> `SX_COMPTIME_FLAT_TRACE=1` (now also prints lowering-time `type-fn` HANDLED/fallback lines).
### (superseded) prior weld resume
Phase 1 done; Phase 2 welded structs were working via reflection + memory-order
validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even
earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in
the compiler type's MEMORY order; the compiler reflected the bound Zig type and
VALIDATED the header. **This whole mechanism is now being stripped — see the banner.**
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
> See CLAUDE.md → Snapshot integrity.
## Last completed step
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
end-to-end as a byte-identical mirror of the bound Zig type.
Design (locked, supersedes the byte-layout-override plan):
- The sx header declares fields in the compiler type's MEMORY order. The compiler
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
byte-blob, no index/remap tables, no reorder, no special LLVM path.
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
*wrong field order at position N* (+ expected memory order), *type layout
mismatch* (field size), *layout mismatch* (total size / count).
What changed from the dropped plan:
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
the sx header against the memory-ordered registry.
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
(+ `weldedFieldOrderStr`).
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
refreshed.
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
byte-layout-override engine; superseded by the reflection+validation design above.
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
padding included), so it passes to a Zig handler as raw memory with zero
marshalling. (The alternative — handlers reading interp `Value` aggregates
logically, no layout override — was rejected; welded types must also be usable as
runtime data, and the design wants the literal byte weld.)
- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20,
`is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override
is genuinely required (`Field`'s two-u32 natural layout was the easy case).
- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second
`bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc,
fields, total)` — pure: orders fields by ascending byte offset, inserts padding
elements for gaps + the alignment tail, and builds the sx-field → LLVM-element
remap. This is what the LLVM type builder + struct-GEP sites will consume.
- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad);
`StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24,
pad@25..32]`, remap `[1,0,3,2]`.
- `zig build` + `zig build test` green.
### Earlier — Phase 1 polish (comptime-only enforcement)
**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error
instead of an undefined-symbol link failure.**
- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded`
AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a
`#run`/`::` initializer wrapper whose LLVM body is dead), print a clear
"comptime-only … cannot be called at runtime" error and set
`comptime_failed` (the driver halts before object/JIT emission). The enclosing
`is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green.
- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)`
→ clean error, exit 1, no link failure).
- `zig build` + `zig build test` green (458 unit + 690 corpus).
### Earlier — fifth sub-step (host-call bridge)
**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to
its registered Zig handler instead of dlsym.**
- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`,
`bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the
string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value ->
Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)`
(the same mutable-table access the metatype mint path uses); `text_of` reads the
const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern).
- IR `Function` gained `compiler_welded: bool`. `declareFunction`
(`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES:
the bound lib must be `compiler` and the name must be on the function-export
list — else a build-gating `.err` (no silent fall-through to dlsym).
- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function
routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the
export list).
- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run
text_of(intern("hello, compiler"))` folds to a string constant → prints it);
`examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name →
build error). `findFn` lookup unit-tested.
- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a
RUNTIME call would emit a reference to a non-existent extern symbol → a loud
LINK error (not silent, but not a tidy diagnostic). The examples call welded fns
only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the
immediate follow-up.
- `zig build` + `zig build test` green (458 unit tests + 689 corpus).
### Earlier — fourth sub-step (welded-struct layout validation)
**A `struct abi(.zig) extern compiler { … }` is validated against the binding
registry as a *header checked against the implementation*.**
- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns
the first `LayoutMismatch` (field count / name / size / total) or null. Plus
`lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes;
each drift flagged as the right variant).
- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`,
`validateWeldedStruct` checks the bound lib is `compiler`, the name is on the
export list (`findType`), and the sx layout (field names + `typeSizeBytes` +
total) matches the welded type — emitting a build-gating `.err` (good span into
the struct body) on any failure. No silent reinterpretation.
- `#library "compiler"` is the comptime-only internal surface, NOT a dylib —
`src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so`
load warning).
- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds,
validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld-
struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic).
- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it
isn't exercised by `Field` (two u32s = natural layout coincides with the weld).
It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound
offsets actually differ from the sx-natural ones. The validation already checks
per-field size + total, so a layout drift is caught even before the override
engine exists.
- `zig build` + `zig build test` green (456 unit tests + 687 corpus).
### Earlier — third sub-step (binding registry)
**The binding registry (welded-type lookup, layout baked from the real Zig
type).**
- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the
curated safety boundary. `BoundType { sx_name, size, alignment, fields:
[]FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real
Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a
sx-field-count mismatch is a `@compileError`, never a silent truncation).
`bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` —
two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl
resolution path will consult (returns null off the export list — clean boundary,
no silent default).
- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`.
- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real
`StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4);
an unexported name returns null. Break-verified (a wrong size → suite red,
named `ir.compiler_lib.test...`).
- `zig build` + `zig build test` green (454 unit tests).
### Earlier — second sub-step (struct-decl parse)
**`abi(.zig) extern <lib>` PARSES on a STRUCT decl (parse-only, no semantics).**
- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields.
- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler`
check), parse an optional `abi(...)` then optional `extern <lib>` — same slot
order as fn decls — and thread them onto the node. Ordinary structs are
unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent).
- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern
compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib
== "compiler"`, field list intact; a plain struct leaves `abi == .default` /
`extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite
red, confirming the test runs).
- `zig build` + `zig build test` green.
### Earlier — first sub-step (fn decls) + the syntax pivot
**`abi(.zig) extern <lib>` PARSES on a fn decl (parse-only).** Plus the syntax
pivot it required.
Syntax decision (locked 2026-06-17, supersedes the doc's original
`extern(.zig) <lib>` single-qualifier form): the ABI/layout selector and the
linkage keyword are two orthogonal annotations.
- `abi(.x)` — ABI / calling-convention annotation in the slot **before**
`extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.**
`ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the
`compiler` library), `.pure` (naked asm), `.default` (unannotated). Can appear
standalone (no extern) on any fn / fn-type / lambda.
- `extern <lib>` — linkage keyword + binding source (named library).
So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`.
What landed:
- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`;
the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`.
- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`,
keyword string `"callconv"` → `"abi"`.
- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi`
(parses `abi(.c|.zig|.pure)`); wired in the fn-decl postfix slot (before
`extern`/`export`), the function-type-expr slot, and the lambda slot;
`isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`.
- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`,
`closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the
function-type resolver maps `.zig`/`.pure` → IR `.default` (no fn-pointer-type
CC for those decl-level ABIs; neither occurs in a function-TYPE position yet).
- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the
user-facing text `callconv(.c)` → `abi(.c)`.
- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type
callback annotations — none in the fn-decl postfix slot, so no reordering).
- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 /
1200) regenerated for the rename.
- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern <lib>`
on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib ==
"compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` /
`abi(.pure)`. lexer/sema tests updated.
`zig build` + `zig build test` green (450/450 unit + 685 corpus).
## Current state
> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld
> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for
> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the
> weld *semantics* (layout reflection/validation, marshaling dispatch) go.
- `compiler :: #library "compiler";` parses + is recognised as the comptime-only
internal surface (never dlopen'd).
- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry
(faithful → ok; drift → build-gating diagnostic). `Field` welds + usable.
- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to
their registered Zig handler (`intern`/`text_of` round-trip works); unexported
names rejected at declaration. Comptime-only.
- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only
enforcement at `emitCall`); the legitimate `#run`/`::` use stays green.
- The whole Phase 1 foundation (parse → registry → struct-layout validation →
function host-call bridge → comptime-only enforcement) is in place for the
two-u32 `Field` case + the two string readers.
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
(needed by `StructInfo`'s slice field, Phase 2).
## Next step — execute `PLAN-COMPILER-VM.md`
> The weld is being stripped. The next step is **Phase 0 of
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize /
> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig`
> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics
> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then
> Phase 1 (flat-memory value model). The weld-era "next step" below is **obsolete** —
> kept only as a record of what the weld surface was about to do.
### (obsolete) weld-era next step
Welded structs were byte-identical mirrors, so the API surface was set to grow:
- **Bind `register_struct` / `find_type`** over the host-call bridge
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
kind well-formedness — the checks `define` does today). Because the welded
`StructInfo` is byte-identical, the handler can read it as the real Zig
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
Prove: build a struct programmatically + round-trip a source one.
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
(`defineStruct` / the `reflectTypeInfo` struct path).
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
(optional fields → sentinels) — each just needs an sx header in the compiler
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
`#compiler`.
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
when reached (sentinels or accessor fns; see the design doc Risks).
## Known issues
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
## Log
- **Phase 3 P3.4 step 1 (VM plan) — lowering-time default context; first blocker cleared (2026-06-18).**
`materializeDefaultContext` now falls back to a ZEROED `Context` (found by name) when the
`__sx_default_context` global is absent — i.e. at LOWERING time, where the global isn't
emitted yet. A type-fn that never touches the allocator now runs past context setup; one
that allocates reads a null `alloc_fn` (zeroed) → `call_indirect` on the null func-ref
bails → legacy fallback (a REAL lowering-time context with the CAllocator thunk func-refs,
so allocating type-fns also run on the VM, is a follow-up). **Measurement: the bail moved
deeper** — metatype `make_enum` now bails at `const_type` (the `Type`-literal op, unported);
`register_type` type-fns bail at the welded write call (declare_type/register_type aren't in
`callCompilerFn`). No table mutation happens before either bail (the write fns bail before
minting), so parity holds: both gates **697/0**, no crashes. **Next blockers (the "model
Type" chunk):** (a) the `const_type` op → a word = `TypeId.index()`; (b) the Type-return
bridge (`regToValue` for a `Type`/`.any` word → `.type_tag`); (c) the VM-native write side
(declare_type/register_type/pointer_to in `callCompilerFn`) + a real lowering-time context.
Only once those land does a type-fn actually run end-to-end on the VM (a HANDLED case).
- **Phase 3 P3.4 (VM plan) — wire the VM at the LOWERING-time site + measure (2026-06-18).**
Routed `runComptimeTypeFunc` (the type-fn fold — the THIRD comptime call site) through
`comptime_vm.tryEval` behind `-Dcomptime-flat`/`SX_COMPTIME_FLAT` with legacy fallback,
mirroring the two emit-time folds. Extracted the shared post-check (`checkComptimeTypeResult`
— the declared-but-never-defined zero-field guard) so both paths use it. **Measurement
(SX_COMPTIME_FLAT_TRACE):** every metatype/compiler-API type-fn currently bails CLEANLY
with `no __sx_default_context global to materialize the implicit context` — at lowering
time the default-context global doesn't exist yet (it's built at emit time), so the VM bails
at context materialization, BEFORE running the body (no partial mint, no crash → legacy
mints). The hardening holds: **no crashes** across the corpus on the VM lowering-time path.
Both gates **697/0**. **So the FIRST lowering-time blocker is the implicit context, not
`Type` modeling** — the VM needs a way to materialize/skip the default context at lowering
time (most type-fns get an implicit ctx for potential `List`-growth alloc; many don't use
it). Next: materialize a lowering-time default context for the VM (or pass a null ctx +
bail only if the allocator is actually used), THEN model `Type` values + the VM-native write
side. This is near-pure fallback today — permanent scaffolding that lights up as those land.
- **Phase 3 P3.4-prep (VM plan) — harden the VM against malformed lowering-time IR (2026-06-18).**
Prerequisite for wiring the VM at the LOWERING-time comptime site (`runComptimeTypeFunc`),
where IR can be malformed (an unresolved name lowers to a dangling / `Ref.none` operand —
the 0737 crash). Closed the remaining panic vectors so the VM BAILS (→ legacy fallback)
instead of aborting: (1) a checked `Vm.refTy(ref_types, r)` replaces every raw
`ref_types[ref.index()]` in `exec` (the type-side companion to `Frame.get`'s `bad_ref`
value-side guard); (2) `aggType` is now a bailing method (`Error!TypeId`) using `refTy`;
(3) the block-dispatch loop bounds-checks the branch target before indexing
`func.blocks.items`. `global_get` was already guarded. No behavior change — gate OFF and
ON both **697/0**; unit test added (a `cmp_lt` with a `Ref.none` operand bails, not
panics). **Next:** wire `tryEval` into `runComptimeTypeFunc` behind the flag with legacy
fallback and measure (most minting type-fns will still bail at the welded-write call /
`Type`-result conversion until the VM models `Type` values + the VM-native write side land
— those are the steps that actually move lowering-time comptime onto the VM, toward
deleting legacy).
- **Phase 3 P3.3 (VM plan) — WRITE side: declare_type + pointer_to + ONE kind-branching register_type (2026-06-18).**
The mutating compiler-API: `declare_type(name) -> Type` (forward handle), `pointer_to(t) -> Type`
(build `*T`), and `register_type(handle, kind, members: []Member) -> Type` which branches on
`kind` IN THE COMPILER (subsuming define's per-kind dispatch). Take/return real `Type` values
(matching meta.sx declare/define). **Timing (per user): mint LAZILY at lowering time, single
pass** (the existing `runComptimeTypeFunc`), so the write side is **legacy-only** (`compiler_lib`
handlers) — the VM isn't wired at lowering time, no VM mirror needed; readers stay dual-path.
A non-generic `-> Type` builder is now flagged `is_comptime` (decl.zig) so its dead body permits
the welded calls. **Graph:** forward handles + `pointer_to` express mutually-recursive A↔B (`*A`,
`*B`, B-by-value); `register_type` is **idempotent** (re-fill a nominal slot reached via two
import edges — `nominalIdent`). `kind` codes match `type_kind` (1 struct · 2 actual `.@"enum"` ·
3 tagged_union · 4 tuple). **Fixed two bugs (issue 0142):** (a) a fully payloadless minted enum
was an all-void tagged_union → verifySizes panic; now a real `.@"enum"` (register_type kind 2 AND
metatype `defineEnum`); (b) bare `EnumType.variant` payloadless qualified construction wasn't
supported (failed for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
Examples 0631 (graph + actual enum + reflection), 0632 (make_enum all-void), 0633/0634/0635
(namespaced / bare / multi-edge import of a minted type), 0187 (qualified variant construction).
**Parity 697/697** (gate ON and OFF); unit tests added. **Next (P3.4):** re-express
declare/define/type_info as sx over the compiler-API + delete the bespoke interp arms (needs the
VM hardened for lowering-time IR, or the metatype migrated onto the legacy compiler-API calls).
- **Phase 3 P3.2b (VM plan) — kind + enum-value readers: `type_kind` + `type_field_value`; READ side complete (2026-06-18).**
The last two read-only readers the metatype's `type_info(T)` needs (added to
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`, each backed by a `TypeTable` query both
call): `type_kind(t) -> i64` (`kindCode` — a stable, compiler-owned discriminant: 0 other ·
1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · 7 vector · 8 error_set;
TOTAL, never bails) and `type_field_value(t, idx) -> i64` (`memberValue` — an enum variant's
explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for non-enum /
out-of-range). Example `0630-comptime-compiler-type-kind` reflects `Color` / `WindowFlags`
(flags) / `Point`. **The READ side is now COMPLETE** — `find_type` + `type_kind` +
`type_field_count` + `type_field_name`/`type_field_type`/`type_nominal_name` +
`type_field_value` cover everything `reflectTypeInfo` reads. VM unit test added. **Parity
691/691** (gate ON and OFF). **Revised forward direction (per the user):** the WRITE side is
ONE `register_type(info)` fn that branches on the kind IN THE COMPILER (subsuming `define`'s
per-kind dispatch), not a per-kind `register_struct`.
- **Phase 3 P3.2 (VM plan) — field-level reflection readers: `type_nominal_name` + `type_field_name` + `type_field_type` (2026-06-18).**
Three more `compiler`-library readers on the same `TypeId`-handle shape (added to
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`), each backed by a new `TypeTable` query
BOTH paths call (no drift): `nominalName` (a named type's own name handle; loud-bail for
unnamed types like `i64`/pointers), `memberName` (struct/union/tagged-union field, enum
variant, named-tuple element), `memberType` (struct/tuple/array/vector member type). All
loud-bail on out-of-range idx / no-member (no silent default). First MULTI-ARG compiler
fns — `callCompilerFn` reads arg 1 = idx; added `Vm.argHandle`/`argTypeId` (range-checked
u32/TypeId arg reads) and refactored `find_type`/`type_field_count` onto them. Named
`type_*` to avoid clashing with the std metatype builtins (`field_name`/`type_name` exist
in core.sx); `nominalName` (the TypeTable method) is distinct from the existing
`typeName(id) []const u8` display-string renderer. Example `0629-comptime-compiler-field-reflect`
reflects `Pair { lo: Point; hi: Point }` — each field name + the nominal name of a field's
type, all `#run`-folded, all VM-HANDLED natively. VM unit test added (type_field_name → "hi";
type_nominal_name(type_field_type(Pair,0)) → "Point"). **Parity 690/690** (gate ON and OFF).
- **Phase 3 P3.1 (VM plan) — first read-only reflection readers: `find_type` + `type_field_count` (2026-06-18).**
Two more `compiler`-library fns, bound the same way as the `intern`/`text_of` seed (added
to `compiler_lib.bound_fns` for the legacy handler + the welded-decl export check, AND to
`Vm.callCompilerFn` for the native flat-memory path — NO marshaling). A **type handle is a
plain `u32` `TypeId`** (like `StringId`), so both keep the seed's clean scalar shape:
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`, `unresolved`/0 if absent) and
`type_field_count(t: TypeId) -> i64` (a NEW `TypeTable.memberCount` query — struct/union/
tagged-union fields, enum variants, array/vector length — called by BOTH paths so they
can't drift; bails loudly, never a silent 0). New example `0628-comptime-compiler-find-type`
chains `intern → find_type → type_field_count` (and a not-found lookup → 0), both folded at
`#run`, both VM-HANDLED natively (trace confirms no fallback). VM unit test added
(`find_type` + `type_field_count`, struct found → 3 fields, missing → `unresolved`).
**Parity 689/689** (gate ON and OFF). **Decision (resolves the plan's `find_type → ?Type`
sketch):** return a NON-optional `TypeId` with the `unresolved` (0) sentinel for not-found,
NOT `?Type` — a `Type` value resolves to `.any` (which the flat-memory VM doesn't represent)
and an optional can't cross the legacy↔VM eval boundary; `unresolved` is the project-blessed
unmistakable "no type" marker. Forward (P3.2): more readers on the same handle shape
(`type_name`/`field_name`/`field_type`/kind), then `register_struct` (first mutating fn).
- **VM robustness — `Frame` bounds-check; lowering-time `#insert` wiring explored + reverted (2026-06-18).**
Explored wiring the VM at the LOWERING-time comptime site (`evalComptimeString`, the
`#insert` string fold). 12/13 `#insert` examples ran on the VM with parity, but `0737`
(an `#insert` of an unresolved `secret()`) CRASHED the VM (SIGABRT): lowering-time IR can
be malformed (a `ret Ref.none` from the unresolved name) and `Frame.get` panicked on the
out-of-range index. **Decision: reverted the lowering-time wiring** — unlike the emit-time
folds (fully lowered IR), lowering-time IR can be erroneous, and hardening the VM against
ALL malformed IR (every `ref_types[...]` / `aggType` access, not just `Frame`) is out of
scope here. The emit-time sites already give full corpus coverage. **KEPT** the defensive
fix regardless (CLAUDE.md "never crash"): `Frame.get`/`set` now bounds-check and flip a
`bad_ref` flag; the `run` loop bails (`badRef`) instead of panicking. Unit test added
(malformed `ret Ref.none` → bail, not crash). Parity **688/688** both ways.
- **Phase 3 SEED (VM plan) — compiler-call path: `intern`/`text_of` native on the VM (2026-06-18).**
`invoke` now dispatches a welded `compiler`-library fn (gated on `compiler_welded`) to
`Vm.callCompilerFn`, serviced NATIVELY on flat memory (no legacy `Interpreter`):
`intern(string)->StringId` reads the flat-memory string bytes and `internString`s into the
const-cast table (pool-only — doesn't touch type layout, so cached sizes stay valid);
`text_of(StringId)->string` materializes the pooled text back into flat memory. Unlocked
`0626`; the ONLY remaining const-init fallback is now the inline-asm global (`1654`).
Parity **688/688** (gate ON and OFF); unit test added. This is the mechanism Phase 3 grows
— the next compiler fns (`find_type`, `register_struct`, reflection readers) bind the same
way (flat-memory pointer in, handle/pointer out, no marshaling).
- **Phase 1.final step 9 (VM plan) — `-Dcomptime-flat` build flag (the "swap behind a build flag" step) (2026-06-18).**
Added the `-Dcomptime-flat` build option (build.zig → a `build_opts` options module on
`mod`; `emit_llvm.init` reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`). This is
the plan's "reach parity → swap behind a build flag → delete the old path" mechanism.
`zig build test -Dcomptime-flat` runs the FULL corpus on the VM (688/0). Verified the flag
toggles the binary: flag-built `sx` reports VM HANDLED with no env var; default-built does
not. Default OFF — `zig build test` unchanged (688/0). Env var still works for ad-hoc runs.
Next (forward): Phase 2 (bytecode) / Phase 3 (compiler-API on flat memory); eventual
default-flip + legacy deletion.
- **Phase 1.final step 8 (VM plan) — wire the `#run` side-effect path + trace-clear-on-fallback (2026-06-18).**
Wired the SECOND comptime call site (`runComptimeSideEffects`, top-level `#run <expr>;`)
through `tryEval` with legacy fallback, mirroring the const-init fold. `tryEval` now
handles void/noreturn entries (→ `.void_val`) so a void side-effect doesn't bail at the
result conversion. **Fixed a trace-corruption** the new site exposed (`1035`): a
side-effect that pushes return-trace frames and then bails (e.g. on `print`) had the
legacy re-run DOUBLE-push them (`sx_trace_push` is a side effect on the shared buffer).
Both wiring sites now `sx_trace_clear()` right before the legacy fallback, discarding the
VM's partial pushes. **Parity 688/688** (gate ON and OFF). Most side-effects still bail
(print/global_addr/call_builtin) → legacy, but the path is now uniform. All comptime
evaluation routes through the VM-with-fallback.
- **Phase 1.final step 7 (VM plan) — is_comptime + failable/error cluster + signed-load fix; coverage 31→36 (2026-06-18).**
`is_comptime` → 1 (unlocked `1030`). Ported the failable/error-channel cluster (`1037`
escape, `1038` handled): `kindOf(error_set)→word`, `regToValue` bridges TUPLES (the
failable `(value…,tag)` shape `checkComptimeFailable` reads), `trace_frame` packs
`(func_id<<32|span.start)` from a new `call_stack` (pushed by invoke/runEntry), and
`sx_trace_push`/`sx_trace_clear` serviced NATIVELY (the VM calls the real sx_trace.c
functions linked into the compiler, so the return-trace buffer is populated identically
to legacy). raise/catch/or now run on the VM. **Surfaced + fixed a real GENERAL bug:**
`readField` was ZERO-extending signed sub-64-bit loads, so a stored `i32 -1` reloaded as
`0xFFFFFFFF` (+4.29e9) and `< 0` was false — silently hiding `raise error.Bad`; now
SIGN-extends `i8`/`i16`/`i32`/`isize` (gate-ON parity confirms it's a strict fix; unit
test added). VM HANDLES **36** corpus const-inits (was 31); **parity 688/688** (gate ON
and OFF). Only **2 fallbacks** remain, both principled: `intern` (`0626`, welded
compiler-API fn — Phase 3) + inline-asm global (`1654`). Forward work: Phase 2 (bytecode),
Phase 3 (compiler-API on flat memory).
- **Phase 1.final step 6 (VM plan) — real default context + call_indirect + func_ref + global_get; coverage 27→31 (2026-06-17).**
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context instead of a zeroed one. The implicit-ctx param is
an opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer (`{ {null, alloc_fn, dealloc_fn}, null }`, the CAllocator thunk
func-refs) into flat memory via a new recursive `layoutConst`. With `func_ref` (function
value encoded `FuncId.index()+1`, reserving word 0 for the null fn-ptr) and
`call_indirect` (decode word → FuncId → dispatch; 0 → bail) ported, the whole allocator
protocol runs on the VM:
`context.allocator.alloc_bytes` → call_indirect → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → native flat malloc. Unlocked `0606` (string global). Also: `global_get`
lazily evaluates a comptime global's `comptime_func` (memoized) — unlocked `CT_CHAIN`;
field access (`fieldOffset`/`struct_get`) handles string/slice `{ptr@0,len@8}` fat
pointers (needed by `alloc_string`); `regToValue` maps function-typed words → `.func_ref`
(kept `1128`'s rejection byte-identical). Native `malloc` is still required (the thunk
bottoms out at it; a host pointer can't be used with flat-memory load/store). VM HANDLES
**31** corpus const-inits (was 27); **parity 688/688** (gate ON and OFF). Unit tests:
global_get, func_ref+call_indirect. Remaining fallbacks (7): `.unsupported` aggregates
(3× — `1037`/`1038`), extern/builtin `intern`+asm (2×), `trace_frame`, `is_comptime`.
- **Phase 1.final step 5 cont. (VM plan) — libc memory builtins + f32 fix; coverage 16→27 (2026-06-17).**
Identified the dominant fallback (`call to extern/builtin`) as **11× `malloc`** (0604) +
1× `intern`. Modeled a curated set of libc MEMORY builtins natively on flat memory
(`Vm.callMemBuiltin`): `malloc`/`calloc` → `allocBytes` (16-aligned, 256-MiB cap → bail),
`free` → no-op, `memcpy`/`memmove`/`memset` on flat bytes — sandboxed (no host heap/dlsym),
target-aware; the computed result is byte-identical to legacy (which calls real libc).
This surfaced a **real latent f32 bug**: float registers hold f64 bits, but f32 MEMORY is
the 4-byte single — `readField`/`writeField` were truncating the f64 bits (writing zeros
for `1.0`); now they `@floatCast` on f32 load/store (mirrors legacy `storeAtRawPtr`).
Result: VM HANDLES **27** corpus const-inits (was 16); **parity 688/688** (gate ON and
OFF). Unit tests added (f32 round-trip; malloc → usable flat memory). Next: the `kindOf`
`.unsupported` aggregates (3×), `global_get` (2×), the rest.
- **Phase 1.final step 5 (VM plan) — implicit-context materialization; coverage 0→16 (2026-06-17).**
`tryEval` now MATERIALIZES the implicit ctx instead of skipping it: a `has_implicit_ctx`
comptime entry (sole param `*Context`) gets a zeroed `Context` of the right size/align
in flat memory, its address passed as arg 0. Const bodies that ignore the ctx run; a
body that uses the allocator hits unported `call_indirect` → bails → legacy. No func-ref
materialization needed (handled bodies don't read ctx contents; parity is the guard).
Fixed a real bug surfaced by the coverage pass: storing a `null` non-pointer optional
(the `null_addr` sentinel) into an aggregate slot OOB-bailed — `writeField` now ZEROES
the destination for a `null_addr` aggregate source (= none/empty); unit-test regression
added. Result: VM HANDLES **16** corpus const-inits (was 0); **parity 688/688 both
gate ON and OFF**. Next: port the ops the trace names — `call_builtin`/`compiler_call`/
extern (13×, via the bridge), `kindOf` `.unsupported` aggregates (3×), `global_get` (2×),
func_ref / call_indirect / trace_frame / is_comptime.
- **Phase 1.final steps 14 (VM plan) — host wiring landed; coverage measured (2026-06-17).**
(1) **Hardening:** `Machine.readWord`/`writeWord`/`bytes` now return `error.OutOfBounds`
(null / out-of-range / oversized / overflow-safe) instead of `assert`-panicking;
`OutOfBounds` added to `Vm.Error`; `try` threaded through every helper + exec arm + the
bridge. New unit tests (accessor OOB returns; null-deref → `tryEval` null, not a crash).
(2) **Implicit context:** `tryEval` returns null for `has_implicit_ctx` funcs (legacy
fallback) — conservative; full ctx materialization deferred to step 5. (3) **Wiring:**
const-init fold in `emit_llvm.zig` `emitGlobals` is `(if comptime_flat) tryEval else
null) orelse interp.call(...)`, gated by env `SX_COMPTIME_FLAT` (read once into
`LLVMEmitter.comptime_flat`). Default OFF. (4) **Parity + coverage:** gate ON → full
corpus byte-identical (688, 0 failed) + manual 0605/0606/0607 byte-identical.
**Finding: 0 of 37 measured corpus const-inits are VM-handled — ALL are
`has_implicit_ctx`-gated.** Added a coverage-trace facility (`comptime_vm.last_bail_reason`
+ env `SX_COMPTIME_FLAT_TRACE`). **Next: step 5 = implicit-context materialization** (the
unblocker), then port the deferred ops. 688 corpus green (gate OFF).
- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).**
`comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on
the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back.
Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs
(1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of
crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind
`SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final.
688 corpus green.
- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).**
Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the
wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs
the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg →
Reg, aggregates into flat memory) + `regToValue` (VM result → Value, deep-copied).
Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip
unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through
the VM with legacy fallback).
- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).**
`Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals
arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared
flat memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate
Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b).
Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next:
1.5b (call_builtin/compiler_call/extern), then hybrid wiring.
- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).**
Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested
(deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops
(tagged-union payload/any/closures) blind — their byte semantics are ambiguous without
real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) +
HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so
the VM runs the real corpus and surfaces exactly what's needed. Key design point for
calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's
allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision
block.
- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).**
`kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}`
aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`;
const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91,
?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any,
closures).
- **Phase 1 sub-step 4b (VM plan) — slices + strings on flat memory (2026-06-17).**
`{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate).
Ported `const_string` (text+NUL + fat pointer in flat memory), `length`/`data_ptr`,
`array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and
`str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23,
subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures).
- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on flat memory (2026-06-17).**
`kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get`
(`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over
array/pointer/many_pointer; slice/string bases bail), `length` on array values.
Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next:
sub-step 4b (slices/strings, then optionals/enums/any/closures).
- **Phase 1 sub-step 3 (VM plan) — memory + structs on flat memory (2026-06-17).**
`Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported
`alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out
at the table's natural offsets. Value model: scalar/pointer → register word;
struct → lives in flat memory, its value IS its address (read→addr, write→memcpy), so
nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on
not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested
(struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next:
sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls).
- **Phase 1 sub-step 2 (VM plan) — flat-memory executor: scalars + control flow
(2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over
flat-memory frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy
interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic,
comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param`; every other op bails loudly (`error.Unsupported` + op name in
`detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic,
cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus
untouched (688 green). Next: sub-step 3 (memory + aggregates on flat memory, where
target-aware layout enters).
- **Phase 1 sub-step 1 (VM plan) — flat-memory machine substrate (2026-06-17).**
New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator
with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0
reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on
deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested
(`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy
interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor +
scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule
from CLAUDE.md per request.
- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept
(2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig`
(`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/
`LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr`
+ the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and
examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function
host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3
compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the
`emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples
`0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next:
Phase 1 (flat-memory value model) per `PLAN-COMPILER-VM.md`.
- **DIRECTION CHANGE — pivot off the byte-weld to a flat-memory bytecode VM
(2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong
direction (it hand-marshals onto a comptime value model that isn't bytes — exactly
what the design set out to kill). New foundation: a bytecode VM over flat memory so
comptime values are native bytes; the compiler-API then rides on it via direct memory
(no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks
cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0
strip → Phase 1 flat-memory value model → Phase 2 bytecode → Phase 3 compiler-API on
flat memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted
the session's uncommitted `register_struct`/`find_type` marshaling experiment back to
`reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action.
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
byte-blob — all explored, all unnecessary). Instead: the sx header declares
fields in the compiler type's memory order; the compiler reflects the bound Zig
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
On pass it's an ordinary byte-identical struct — cast + deref just works.
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
superseded by the reflection+validation design and deleted.
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
green (458 unit, 690 corpus).
- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).**
`compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function`
`compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`);
`interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round-
trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean
rejection deferred (loud link error today). Build + suite green (458 unit, 689
corpus).
- **Phase 1.1 fourth sub-step — welded-struct layout validation.**
`validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into
`registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against
the registry (lib == compiler, name exported, layout matches) with build-gating
diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful
Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to
Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit,
687 corpus).
- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`:
the `compiler` lib's welded-type registry; `Field` welded to
`StructInfo.Field` with layout baked from the real Zig type
(`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test
(+ null off the export list). Standalone island — not yet consumed by lowering.
Build + suite green (454 unit tests). Break-verified.
- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl`
gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern <lib>`
after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified.
Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete.
- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig)
extern <lib>` on fn decls; unified `callconv` into `abi(.c|.zig|.pure)` (removed
the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs +
snapshots. Build + suite green. The original design's `extern(.zig)` single
qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern
<lib>` (linkage + source) — recorded in the design doc's syntax-decision note.