Files
sx/current/PLAN-COMPILER-VM.md
agra 0367d96d9b comptime VM: host wiring, full corpus parity, build flag, Phase 3 seed
Phase 1.final of the flat-memory comptime VM — wire the host through it,
reach corpus parity, and gate it behind a build flag — plus the first
Phase 3 (compiler-API) step. Default OFF; legacy interpreter unchanged.

Host wiring + hardening:
- Machine accessors return error.OutOfBounds (no debug panic) on bad
  addresses; Frame.get/set bounds-check and bail (no panic) on a malformed
  operand ref (e.g. a ret Ref.none from an unresolved name).
- tryEval routed at both comptime call sites in emit_llvm — the const-init
  fold and the #run side-effect path — with per-eval legacy fallback;
  yields .void_val for void/noreturn entries. Both sites sx_trace_clear()
  before the legacy fallback so a partial VM run that pushed trace frames
  doesn't double-push on re-run.

VM coverage (all corpus const-inits except the inline-asm global):
- Implicit context materialized from the __sx_default_context global; the
  full allocator protocol runs on the VM (context.allocator.alloc ->
  call_indirect -> CAllocator thunk -> libc_malloc -> native flat malloc).
- Native libc memory builtins (malloc/calloc/free/memcpy/memmove/memset)
  on flat memory; f32 stored/loaded as the 4-byte single; signed sub-64-bit
  loads sign-extended; global_get (lazy + memoized); func_ref/call_indirect
  (func-ref encoded fid+1, 0 reserved for null); string/slice fat-pointer
  field access; is_comptime; the failable/error cluster (error_set tuples,
  trace_frame + native sx_trace_push/clear -> raise/catch/or + return traces).

Build flag + Phase 3 seed:
- -Dcomptime-flat (build_opts module) OR SX_COMPTIME_FLAT env enables the VM;
  zig build test -Dcomptime-flat runs the full corpus on the VM (688/0).
- intern/text_of serviced natively on flat memory via Vm.callCompilerFn
  (compiler_welded boundary) — the seed the rest of the compiler-API grows on.

Parity 688/688 gate ON and OFF. Unit tests added throughout. The
lowering-time #insert wiring was explored and reverted (lowering-time IR can
be malformed; full malformed-IR hardening is a prerequisite, deferred).
2026-06-18 08:27:58 +03:00

418 lines
30 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PLAN — Comptime Bytecode VM + flat memory (then re-home the compiler-API on it)
> **Direction change (2026-06-17).** The comptime compiler-API stream pivots off the
> **byte-weld**. The weld (sx structs whose layout is validated to mirror the
> compiler's Zig types) + the **serialization / marshaling** bridge at the call
> boundary is the wrong direction — it bolts a parallel layout regime and hand-built
> byte-copies onto a comptime value model that fundamentally isn't bytes. We strip it
> and build the right foundation: a **bytecode VM over flat, byte-addressable
> memory**, where comptime values ARE native bytes (like runtime). On that base the
> compiler-API needs no weld, no validation, no marshaling — the compiler's own types
> are read/built directly as memory and its functions take/return real pointers.
>
> Supersedes the build order in `design/comptime-compiler-api.md` (kept for history).
> This is the active plan for the stream. Branch: `reify`.
## Why
`src/ir/interp.zig` is a tree-walking interpreter over the SSA IR that represents
every value as a tagged `Value` union (`int`, `float`, `aggregate: []const Value`,
`type_tag`, `heap_ptr`, …). Two consequences:
1. **Slow.** Per-value boxing in a tagged union; per-op `switch` over `Inst`; an
aggregate is a heap `[]const Value`, walked element-by-element.
2. **Not native memory.** A struct value is `[]const Value` (tagged unions), NOT the
struct's bytes. So a comptime `@ptrCast(*StructInfo)` reads the `Value` union's
memory, not a `StructInfo` — which forced the whole weld+marshal detour.
Make comptime values **native bytes in a flat memory** and both problems dissolve:
structs/arrays/slices are their bytes at natural layout (no weld), the compiler's own
records are directly addressable (no marshal), and a bytecode loop over flat memory is
fast.
## End state
- Comptime execution = a **bytecode VM** over a **flat linear memory** (real
host-allocated bytes; layout is **target-aware** via the type table's sizes). Values
are bytes at addresses plus a scalar register file. No tagged `Value` union.
- The comptime compiler-API: the compiler **exposes its real types + functions** to
comptime sx. sx reads/builds them as native memory and calls compiler functions by
pointer. No `abi(.zig)` weld, no `validateStructLayout`, no `register_struct`
field-by-field marshaling — gone.
- `declare`/`define`/`type_info` and `#compiler`/`BuildOptions` ride this one
mechanism; the bespoke interp arms are deleted.
## Principles (hold at every step)
- **Green at every step.** `zig build && zig build test` pass after each sub-step. The
existing tagged-`Value` interpreter stays the live evaluator until the VM reaches
corpus parity; swap behind a build flag, then delete the old path.
- **Target-aware, not host-baked.** Flat-memory layout uses the type table's target
sizes (`pointer_size`, `typeSizeBytes`/offsets), NEVER host `@sizeOf`. This is what
keeps cross-compilation correct (the JIT-comptime alternative could not).
- **Sandboxed.** Flat-memory accesses are bounds-checked; step/call-depth budgets
remain; an OOB / bad access traps to a build-gating diagnostic with a source span —
never a compiler-process crash.
- **No silent fallbacks** (per CLAUDE.md): an unhandled op / shape bails loudly with a
named reason, never a zero/default that looks like success.
## Phases
### Phase 0 — Strip the weld / serialize / marshal machinery
Delete the wrong-direction code so the VM builds on a clean base. Pure removal +
corpus rebaseline; suite green.
- `src/ir/compiler_lib.zig`: the reflection (`weldStruct` / `bound_types` /
`FieldLayout` / `BoundType`), the layout validation (`validateStructLayout` /
`LayoutMismatch` / `SxField`). Decide the fate of the `bound_fns` host-call registry
(`intern`/`text_of` handlers) — it is likely subsumed by the VM's compiler-call path
in Phase 3, but `intern`/`text_of` may survive as the first such calls.
- `src/ir/lower/nominal.zig`: `validateWeldedStruct` + `weldedFieldOrderStr` + the
`sd.abi == .zig` validation call in `registerStructDecl`.
- `src/ir/interp.zig`: the `compiler_welded` dispatch branch.
- `src/backend/llvm/ops.zig`: the `emitCall` comptime-only gate keyed on
`compiler_welded` (re-derive the comptime-only guard from a non-weld signal if still
needed).
- Corpus: retire / convert the weld examples + diagnostics — `0625`, `0627` (welded
struct), `1183`, `1186` (weld-layout diagnostics), `1184`/`1185` (welded-fn). Keep
`0626` (`intern`/`text_of` round-trip) only if it survives the new call path.
- **Keep (re-evaluate in Phase 3), independent of the weld semantics:** the
`#library "compiler"` decl, the `abi(.x)` annotation + `extern <lib>` syntax, and the
`callconv → abi` unification. These are surface syntax that may still serve the
compiler-API; only the *weld semantics* are stripped here.
**Verification:** `zig build test` green with the weld machinery gone; the surviving
syntax still parses (parser unit tests).
### Phase 1 — Flat-memory value model (still IR-walking, no bytecode yet)
Introduce flat memory and move comptime values onto it, **decoupled from bytecode** so
the value-model change is isolated. Each sub-step ports one op group and keeps the
corpus green; the OLD tagged path stays behind a build flag (`-Dcomptime-flat`) until
all groups land, then the shim is deleted.
1. **Machine + scalars.** A flat memory region (host `[]u8`) with a stack (frames) +
bump-allocated heap, and a scalar register file. Port `int`/`float`/`bool`/`undef`
and arithmetic/compare/branch. Aggregates still go through a compat shim to the old
representation.
2. **Aggregates.** Structs/arrays/tuples laid out in flat memory at **target** layout;
port `struct_init` / `struct_get` / `array` / `index_gep` to read/write bytes at
computed offsets.
3. **Slices / strings.** `{ptr, len}` fat pointers in flat memory.
4. **Optionals / enums / tagged unions.** Tag + payload bytes.
5. **Pointers.** `alloca` / `store` / `load` / GEP unified onto flat addresses; retire
`slot_ptr` / `heap_ptr` / `byte_ptr` in favor of flat-memory addresses.
6. **Closures.** Fn id + captured env materialized in flat memory.
7. **Extern / host calls.** A struct arg is already bytes → pass its address; this
removes most of `marshalExternArg`.
8. **Reflection / minting.** `declare` / `define` / `type_info` read flat-memory
values; type-table mutation copies escaping data into compiler-owned memory at the
boundary (lifetime), as today.
**Verification:** with `-Dcomptime-flat` the full corpus (currently 692) is byte-for-
byte identical to the tagged path; then make flat the default and delete the shim.
### Phase 2 — Bytecode
Compile a comptime function's IR → a compact bytecode and execute the bytecode instead
of walking `Inst`. Pure encoding/speed; semantics identical to Phase 1. Land at least a
minimal register-bytecode loop (the stream's stated goal is a *bytecode* VM); a
fragment cache is optional follow-up.
**Verification:** corpus identical to Phase 1; comptime throughput measurably improved
on a heavy-comptime micro-benchmark.
### Phase 1.final — host wiring (the remaining integration)
The wiring ENTRY POINT exists: `comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a
comptime function entirely on the VM and returns a legacy `Value`, or `null` to fall
back. Unit-tested (pure `6*7` → 42; unsupported → null). Remaining to actually route the
host through it:
1. **Panic→error hardening (prerequisite).** `Machine.readWord`/`writeWord`/`bytes`
currently `assert` (debug panic) on null/OOB. For arbitrary host functions to be
safe, make them return `error.OutOfBounds` so a malformed run BAILS (→ null → legacy)
instead of crashing the compiler. Ripples through `readField`/`writeField`/slice
helpers (add `try`).
2. **Implicit context.** Host comptime functions may have `has_implicit_ctx` (param 0 =
`*Context`); the legacy `run` materializes a default ctx. The VM `run` does not — so
either materialize it too, or only route `tryEval` at funcs without implicit ctx.
3. **Wire one site** behind a flag/env (`SX_COMPTIME_FLAT`, → `-Dcomptime-flat` later):
the const-init fold in `emit_llvm.zig` `emitGlobals` (`result = tryEval(...) orelse
interp.call(...)`). Default off → corpus unaffected.
4. **Parity + coverage.** Run the corpus with the flag ON; results must be byte-identical
to legacy. Measure how many comptime evals the VM already handles; the bail `detail`s
name what to port next (tagged-union payload / any / closures / builtins).
5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge)
until the VM is the default and the legacy path is deleted.
**Status (2026-06-17): steps 14 DONE; step 5 = the next session.**
- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return
`error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of
asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through
`readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and
every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a
null-deref function → `tryEval` returns `null` (legacy fallback), not a panic.
- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a
conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with
`has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the
right size/align allocated in flat memory, its address passed as arg 0. The common
const body never reads the ctx; a body that USES the allocator loads a fn from it and
`call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed:
handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed)
empirically confirms no divergence. (A body that read+branched on a null allocator fn
could in principle diverge; none does — parity is the guard.)
- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat)
tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT`
(a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`).
Default OFF → corpus unaffected (688 green).
- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed);
manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace
facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`,
printing HANDLED / fallback+reason per init).
- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.**
Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity
stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx
materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer
optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the
destination = none/empty; unit-test regression); curated libc MEMORY builtins on flat
memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped,
`free` → no-op, `memcpy`/`memmove`/`memset` on flat bytes — sandboxed, target-aware,
result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32
storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single —
`readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had
written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added).
- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage
**27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF.
Per the user's direction ("the VM can set up a default context"), `runEntry` now
materializes the REAL default context (not a zeroed one): the implicit-ctx param is an
opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying
the CAllocator thunk func-refs) into flat memory via a new recursive `layoutConst`.
With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays
reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect`
(decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body
that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` →
`context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` →
`libc_malloc` → the VM's native flat-memory `malloc`. Unlocked `0606` (string global via
the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func`
(memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/
`struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by
`alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to
`.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s
rejection diagnostic byte-identical). Unit tests added (global_get, func_ref +
call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk
bottoms out at libc `malloc`, and the VM can't use a host pointer with flat-memory
load/store, so comptime `malloc` must allocate from flat memory. The default context
lets the allocator PROTOCOL run; native `malloc` is its final step.
- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage
**31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF.
- **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`.
- **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set)
→ word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)`
shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 |
span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push`
/ `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions —
linked into the compiler — so the return-trace buffer the host reads is populated
identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now.
- **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):**
`readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a
stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently
hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded;
gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests
pass via 1037/1038 in the corpus).
- **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):**
`intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm
global call (`1654`, never comptime-evaluable). Every other measured corpus const-init
is handled on the VM.
At this point the flat-memory VM handles essentially the entire real comptime corpus
(scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls +
recursion, the implicit context + allocator protocol, globals, failables + return
traces). Phase 2 (bytecode) and Phase 3 (compiler-API on flat memory) are the forward
work; flipping the VM to default + deleting the legacy path awaits those.
- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second
comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run <expr>;`) now
routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields
`.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed
(`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy
re-run double-push them — both sites now `sx_trace_clear()` right before the legacy
fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All
comptime evaluation now routes through the VM-with-fallback (uniform).
- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM
gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init`
reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test
-Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity
gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var;
default-built does not). This is the prerequisite to eventually making the VM default +
deleting the legacy path (which still awaits Phase 2/3 + broader confidence).
- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.**
`invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is
the safety boundary) via `Vm.callCompilerFn` — natively on flat memory, NO legacy
`Interpreter`: `intern(s: string) -> StringId` reads the string bytes from flat memory and
`internString`s into the (const-cast) table (pool-only, never touches type layout, so the
VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back
into flat memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback
is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688**
both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next
compiler functions (`find_type`, `register_struct`, the reflection readers) are added the
same way — flat-memory pointer in, handle/pointer out, no marshaling.
### Phase 3 — Compiler-API on flat memory (resume the stream — no weld)
With native-byte comptime values, re-home the compiler-API:
- **Expose the compiler's real types.** Register the actual `types.zig` records
(`StructInfo`, `EnumInfo`, `Field`, …) into the comptime type table under sx-visible
names, with their **real (host) layout** — the type IS the compiler's, so there is
nothing to validate or keep in sync. (This is the projection that *replaces* the
weld's reflection — owned by the compiler, not declared in sx.)
- **Expose the compiler's functions.** `register_struct`, `find_type`, `intern`,
`text_of`, and the reflection readers operate on flat-memory pointers / handles
directly (no marshaling — the bytes already ARE the record).
- **Re-express** `declare` / `define` / `type_info` as sx over these; delete the
bespoke interp arms (`defineStruct` / `defineEnum` / `defineTuple` / `reflectTypeInfo`);
migrate `examples/0622` (struct), `0619`/`0620`/`0623` (enum/tuple).
- **Migrate `BuildOptions`** off `#compiler` onto this mechanism; **delete `#compiler`**.
**Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over
the exposed compiler-API; full corpus green.
## Open questions (resolve as reached, record decisions here)
- **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed
records are host-laid-out; user comptime types are target-laid-out. The flat-memory
model must carry both regimes (a per-type ABI tag on layout queries). Confirm the
boundary where a flat-memory pointer to a compiler record is handed to host Zig code
uses host layout.
- **Exposing compiler types to sx.** Mechanism for projecting `types.zig` records into
the comptime type table with real offsets (the non-weld replacement) — a registry the
compiler owns, keyed by sx-visible name → real Zig type's layout + a host-call ABI.
- **Bytecode shape.** IR-derived vs a fresh ISA; register vs stack; fragment caching.
- **Pointer escape / lifetime.** Flat-memory pointers stored into the persistent type
table must be copied into compiler-owned memory at the boundary (as today).
- **Old-path retirement.** Keep the tagged interpreter until Phase 1 parity, then
delete — confirm no non-comptime consumer depends on `Value`.
## File map (current → touched)
| Area | File | Phase |
|------|------|-------|
| Comptime evaluator | `src/ir/interp.zig` | 0 (strip weld dispatch), 12 (rebuild) |
| Weld registry | `src/ir/compiler_lib.zig` | 0 (strip), 3 (replace with type/fn exposure) |
| Weld validation | `src/ir/lower/nominal.zig` | 0 (strip `validateWeldedStruct`) |
| Comptime-only gate | `src/backend/llvm/ops.zig` | 0 (re-derive without weld signal) |
| Host-FFI marshalling | `src/ir/host_ffi.zig` | 1 (struct-by-pointer trims it) |
| Metatype arms | `src/ir/interp.zig` (`defineStruct`/…/`reflectTypeInfo`) | 3 (delete, re-express in sx) |
| `#compiler` / BuildOptions | `library/modules/build.sx`, `src/ir/compiler_hooks.zig` | 3 (migrate, delete `#compiler`) |
| Surface syntax | `src/parser.zig`, `src/ast.zig` (`abi`/`extern`/`#library`) | kept; revisited Phase 3 |
## Status
- **Phase 0 — DONE (2026-06-17).** The struct-weld machinery is stripped:
`compiler_lib.zig` lost the type registry (`weldStruct`/`bound_types`/`BoundType`/
`FieldLayout`/`findType`/`SxField`/`LayoutMismatch`/`validateStructLayout`);
`nominal.zig` lost `validateWeldedStruct`/`weldedFieldOrderStr` + the
`sd.abi == .zig` call; the struct-weld unit tests + examples `0625`/`0627`/`1183`/
`1186` are removed. **Decision (recorded):** the `intern`/`text_of` function
host-call bridge is KEPT — it is a clean scalar dispatch (string→handle), not
weld/serialize/marshal, and is the seed Phase 3 grows the compiler-call path from.
So the `compiler_welded` dispatch (`interp.callExtern` is unchanged at HEAD — the
pre-branch in `call()`), `weldedCompilerFn` (decl.zig), the `emitCall` comptime-only
gate (ops.zig), and examples `0626`/`1184`/`1185` stay. The `#library`/`abi`/`extern`
SYNTAX stays. `zig build test` green (688 corpus, 0 failed; unit tests pass).
- **Phase 1 — in progress.**
- **Sub-step 1 — DONE.** `src/ir/comptime_vm.zig`: the flat-memory `Machine`
(linear byte memory + bump/stack allocator with `mark`/`reset` reclamation +
scalar `readWord`/`writeWord` (1/2/4/8, little-endian) + `bytes` views; addr 0
reserved as `null_addr`) and `Frame` (register file indexed by Ref + stack
reclamation on `deinit`). A register `Reg` is a raw u64 — immediate scalar OR
`Addr`. Standalone + unit-tested (`comptime_vm.test.zig`, in the barrel); does
NOT touch the live interpreter, so the corpus stays green (688). No op execution
yet.
- **Sub-step 2 — DONE.** The executor (`Vm` in `comptime_vm.zig`): walks the SAME
IR `Inst` over flat-memory frames, mirroring the legacy interp's scalar semantics
(i64 wrapping/signed + f64 register words, keyed off the result/operand `TypeId`).
Ported: constants (`const_int`/`float`/`bool`/`null`/`undef`), arithmetic
(`add`/`sub`/`mul`/`div`/`mod`/`neg`), comparison (`cmp_*`), logical
(`bool_and`/`or`/`not`), conversions (`widen`/`narrow`/`bitcast` passthrough,
`int_to_float`/`float_to_int`), terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param` (branch args passed as Refs — the same frame persists, SSA-safe).
Any other op bails loudly (`error.Unsupported` + `detail = @tagName(op)`).
Unit-tested on hand-built IR (`Fb` builder): integer add, f64 arithmetic, cond_br
branch selection, a block-param loop summing i..1, div-by-zero + unsupported-op
bails. Corpus untouched (688 green) — the executor is exercised by unit tests only,
not yet wired to real comptime eval.
- **Sub-step 3 — DONE.** Memory + structs on flat memory. `Vm` gained an optional
`table: *const TypeTable` (target-aware layout). Ported `alloca`/`load`/`store`
(over flat addresses, `Store.val_ty` drives width) and `struct_init`/`struct_get`/
`struct_gep` (structs laid out at the table's natural offsets). The value model: a
`Kind.word` (scalar/pointer ≤8B) sits in a register; a `Kind.aggregate` (struct)
lives in flat memory and its "value" IS its address (read returns the address,
write memcpys), so nested structs compose and `struct_gep` is just base+offset (no
field-pointer dance). `kindOf` bails loudly on the not-yet-ported types
(slice/string/any/optional/enum/array/tuple/…). The Addr-based value model survives
allocator realloc (offsets are stable; slices are only materialized transiently).
Unit-tested: struct_init+get round-trip, alloca+gep+store+load, nested-struct
aggregate copy + nested read. Corpus untouched (688 green).
- **Sub-step 4a — DONE.** Tuples + arrays. `kindOf` widened (`tuple`/`array` →
aggregate). Ported `tuple_init`/`tuple_get` (positional, `tupleFieldOffset`),
`index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over array/pointer/
many_pointer bases; slice/string bases bail), and `length` on an array value
(static `ArrayInfo.length`). Unit-tested: mixed tuple round-trip, `[3]i64`
gep/store + index_get sum (42), array `length` (3). 688 corpus green.
- **Sub-step 4b — DONE.** Slices + strings as `{ptr@0 (pointer_size), len@8 (i64)}`
fat pointers (`kindOf`: string/slice → aggregate). Ported `const_string` (materializes
text+NUL in flat memory + a fat pointer), `length`/`data_ptr` (read len/ptr fields),
`array_to_slice`, `subslice`, indexing *through* a slice/string (`elemAddr` loads
`.ptr` first), and `str_eq`/`str_ne` (len+memcmp). Helpers `makeSlice`/`sliceLen`/
`sliceData`. Unit-tested: string length + str_eq/ne, array→slice + slice index +
slice length (23), array subslice (43). 688 corpus green.
- **Sub-step 4c — DONE (optionals + payloadless enums).** `kindOf`: `enum` → word;
`?T` → word if pointer-child (null==0) else `{T@0, i1@sizeof(T)}` aggregate. Ported
`optional_wrap`/`unwrap`/`has_value`/`coalesce` (with `optChildIsPtr`/`optHas`
helpers; `const_null` → `null_addr` reads as none), `enum_init` (payloadless: tag is
the value), `enum_tag` (payloadless/word). Unit-tested: non-pointer `?i64`
wrap/unwrap/coalesce (91), pointer `?*i64` null==0 (99), payloadless enum tag (11).
688 corpus green.
- **Sub-step 4d — partial (`addr_of`/`deref` DONE).** `addr_of` passes through (an
aggregate value already IS its address; a pointer is already an address — mirrors
the legacy); `deref` = `readField` through the pointer (`ins.ty` is the pointee).
Unit-tested (deref a `*i64` → 77; addr_of a struct value + field read → 80).
**Deferred to the wiring phase (intentionally, not ported blind):** tagged-union
payload (`enum_init` w/ payload, `enum_payload` — the legacy stores *untyped* Values
and `field_index` indexes payload sub-fields, not variants, so a byte model's
payload type is ambiguous without a real call site), `any` boxing, closures, and the
bitwise ops. These have subtleties best resolved against actual corpus cases — the
VM's loud `error.Unsupported` + `detail` will name exactly what each real eval needs.
- **Sub-step 1.5 — direct `call` DONE.** `Vm` gained `module: *const Module`
(resolves a callee `FuncId`) + a `depth`/`max_depth` recursion guard. `call`
marshals arg Refs → Reg words and recursively `run`s the callee; aggregate args/
results pass as their `Addr` over the SHARED flat memory (no copy). **Stack-lifetime
change:** `Frame` no longer reclaims the machine on exit (a returned aggregate's
Addr would dangle) — a comptime eval's allocations live to `Vm.deinit`;
`Machine.mark`/`reset` stay for explicit use. Extern/builtin callees (no blocks)
bail loudly (1.5b). Unit-tested: direct call (`add(20,22)+100` → 142) and recursion
(`sum(0..n)` → 15/55). 688 corpus green.
- **Sub-step 1.5b — `Reg`↔`Value` boundary bridge DONE.** The builtin/`compiler_call`/
extern handlers are all coupled to the legacy `Interpreter` (e.g. `compiler_lib`
handlers take `*Interpreter`), so the VM can't call them directly — the wiring uses
WHOLE-FUNCTION fallback instead (VM runs pure functions; a bail re-runs the whole
eval in the legacy). That needs the boundary bridge: `valueToReg` (host `Value` arg →
VM `Reg`, materializing aggregates into flat memory) + `regToValue` (VM result →
`Value`, deep-copied out). Covers scalars + strings + structs (other aggregate shapes
bail loudly; added as wiring surfaces them). Transitional — deleted once the VM owns
comptime end-to-end. Unit-tested with round-trips. 688 corpus green.
- **Then the wiring step** (below) — now unblocked.
### Decision (2026-06-17): pivot from blind op-porting to CALLS + hybrid wiring
The common leaf ops are ported (scalars, control flow, structs, tuples, arrays, slices,
strings, optionals, payloadless enums, deref/addr_of) and unit-tested. Continuing to
port the rarer ops (tagged-union payload, any, closures) in isolation risks subtle
bugs and has low signal. The higher-value path:
1. **Calls (sub-step 1.5)** — `call` (direct), then `call_builtin`/`compiler_call`. The
shared flat memory makes aggregate args/results pass naturally (they're Addrs). The
one design point: **aggregate-return lifetime** — a callee's stack-reclaim would
dangle a returned struct Addr, so for comptime (bounded) the VM should stop
reclaiming per-frame and let the whole eval's allocations live until `Vm.deinit`
(keep `Machine.mark/reset` for explicit use; drop it from `Frame.deinit`).
2. **Hybrid wiring** — `-Dcomptime-flat` routes a comptime eval through the VM, falling
back to the legacy interp on `error.Unsupported`. This makes the VM run the REAL
corpus, proving parity incrementally and surfacing exactly which ops each real eval
needs — far better signal than more isolated unit tests.