comptime VM: host wiring, full corpus parity, build flag, Phase 3 seed
Phase 1.final of the flat-memory comptime VM — wire the host through it, reach corpus parity, and gate it behind a build flag — plus the first Phase 3 (compiler-API) step. Default OFF; legacy interpreter unchanged. Host wiring + hardening: - Machine accessors return error.OutOfBounds (no debug panic) on bad addresses; Frame.get/set bounds-check and bail (no panic) on a malformed operand ref (e.g. a ret Ref.none from an unresolved name). - tryEval routed at both comptime call sites in emit_llvm — the const-init fold and the #run side-effect path — with per-eval legacy fallback; yields .void_val for void/noreturn entries. Both sites sx_trace_clear() before the legacy fallback so a partial VM run that pushed trace frames doesn't double-push on re-run. VM coverage (all corpus const-inits except the inline-asm global): - Implicit context materialized from the __sx_default_context global; the full allocator protocol runs on the VM (context.allocator.alloc -> call_indirect -> CAllocator thunk -> libc_malloc -> native flat malloc). - Native libc memory builtins (malloc/calloc/free/memcpy/memmove/memset) on flat memory; f32 stored/loaded as the 4-byte single; signed sub-64-bit loads sign-extended; global_get (lazy + memoized); func_ref/call_indirect (func-ref encoded fid+1, 0 reserved for null); string/slice fat-pointer field access; is_comptime; the failable/error cluster (error_set tuples, trace_frame + native sx_trace_push/clear -> raise/catch/or + return traces). Build flag + Phase 3 seed: - -Dcomptime-flat (build_opts module) OR SX_COMPTIME_FLAT env enables the VM; zig build test -Dcomptime-flat runs the full corpus on the VM (688/0). - intern/text_of serviced natively on flat memory via Vm.callCompilerFn (compiler_welded boundary) — the seed the rest of the compiler-API grows on. Parity 688/688 gate ON and OFF. Unit tests added throughout. The lowering-time #insert wiring was explored and reverted (lowering-time IR can be malformed; full malformed-IR hardening is a prerequisite, deferred).
This commit is contained in:
@@ -142,6 +142,117 @@ host through it:
|
||||
5. Grow coverage (port the deferred ops + `call_builtin`/`compiler_call` via the bridge)
|
||||
until the VM is the default and the legacy path is deleted.
|
||||
|
||||
**Status (2026-06-17): steps 1–4 DONE; step 5 = the next session.**
|
||||
- **(1) Hardening — DONE.** `Machine.readWord`/`writeWord`/`bytes` return
|
||||
`error.OutOfBounds` (null / out-of-range / oversized / overflow-safe) instead of
|
||||
asserting. `OutOfBounds` added to `Vm.Error`; `try` threaded through
|
||||
`readField`/`writeField`/`optHas`/`makeSlice`/`sliceLen`/`sliceData`/`elemAddr` and
|
||||
every exec arm + the bridge. New unit tests: hardened-accessor OOB returns, and a
|
||||
null-deref function → `tryEval` returns `null` (legacy fallback), not a panic.
|
||||
- **(2) Implicit context — DONE (materialized, 2026-06-17 step 5).** Initially a
|
||||
conservative skip; now `tryEval` MATERIALIZES the implicit ctx: a comptime entry with
|
||||
`has_implicit_ctx` (whose sole param is the `*Context`) gets a zeroed `Context` of the
|
||||
right size/align allocated in flat memory, its address passed as arg 0. The common
|
||||
const body never reads the ctx; a body that USES the allocator loads a fn from it and
|
||||
`call_indirect`s (unported) → bails → legacy. No func-ref materialization was needed:
|
||||
handled bodies don't read the ctx contents, and gate-ON corpus parity (688, 0 failed)
|
||||
empirically confirms no divergence. (A body that read+branched on a null allocator fn
|
||||
could in principle diverge; none does — parity is the guard.)
|
||||
- **(3) Wire one site — DONE.** Const-init fold in `emitGlobals` is `(if comptime_flat)
|
||||
tryEval(...) else null) orelse interp.call(...)`. Gated by env `SX_COMPTIME_FLAT`
|
||||
(a `LLVMEmitter.comptime_flat` field read once from `std.c.getenv` in `init`).
|
||||
Default OFF → corpus unaffected (688 green).
|
||||
- **(4) Parity + coverage — DONE.** Gate ON: full corpus byte-identical (688, 0 failed);
|
||||
manual `sx run` of 0605/0606/0607/0608 byte-identical to gate-OFF. Coverage-trace
|
||||
facility in place (`comptime_vm.last_bail_reason` + env `SX_COMPTIME_FLAT_TRACE`,
|
||||
printing HANDLED / fallback+reason per init).
|
||||
- **(5) Implicit-context materialization + memory builtins + f32 — DONE; op-porting CONTINUES.**
|
||||
Coverage climbed **0 → 16 → 27** handled corpus const-inits (fallbacks 22 → 11); parity
|
||||
stays **688/688** (gate ON and OFF) at every step. Landed, in order: implicit ctx
|
||||
materialized (→16); `writeField` null-aggregate fix (storing a `null` non-pointer
|
||||
optional `null_addr` sentinel into an aggregate slot OOB-bailed → now ZEROES the
|
||||
destination = none/empty; unit-test regression); curated libc MEMORY builtins on flat
|
||||
memory (`Vm.callMemBuiltin`: `malloc`/`calloc` → `allocBytes` 16-aligned & 256-MiB-capped,
|
||||
`free` → no-op, `memcpy`/`memmove`/`memset` on flat bytes — sandboxed, target-aware,
|
||||
result byte-identical to legacy; unlocked `0604`'s 11 comptime mallocs); and an **f32
|
||||
storage fix** (float registers hold f64 bits, but f32 memory is the 4-byte single —
|
||||
`readField`/`writeField` now `@floatCast` instead of truncating the f64 bits, which had
|
||||
written zeros for `1.0`; a real latent bug `0604` surfaced; unit tests added).
|
||||
- **(6) Real default context + call_indirect + func_ref + global_get — DONE.** Coverage
|
||||
**27 → 31** handled (fallbacks 11 → 7); parity stays **688/688** both gate ON and OFF.
|
||||
Per the user's direction ("the VM can set up a default context"), `runEntry` now
|
||||
materializes the REAL default context (not a zeroed one): the implicit-ctx param is an
|
||||
opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
|
||||
and lays its initializer constant (`{ {null, alloc_fn, dealloc_fn}, null }`, carrying
|
||||
the CAllocator thunk func-refs) into flat memory via a new recursive `layoutConst`.
|
||||
With `func_ref` (a function value encoded as `FuncId.index() + 1` so word 0 stays
|
||||
reserved for the NULL function pointer — `funcRefWord`/`funcRefToId`) and `call_indirect`
|
||||
(decode the callee word → `FuncId` → dispatch; 0 → bail) ported, a comptime body
|
||||
that allocates via `context.allocator` now runs ENTIRELY on the VM: `alloc_string` →
|
||||
`context.allocator.alloc_bytes` → `call_indirect` → thunk → `CAllocator.alloc_bytes` →
|
||||
`libc_malloc` → the VM's native flat-memory `malloc`. Unlocked `0606` (string global via
|
||||
the allocator). Also: `global_get` lazily evaluates a comptime global's `comptime_func`
|
||||
(memoized in `global_cache`) — unlocked `CT_CHAIN`; struct field access (`fieldOffset`/
|
||||
`struct_get`) now handles string/slice `{ptr@0,len@8}` fat pointers (needed by
|
||||
`alloc_string`'s `s.ptr`/`s.len`); and `regToValue` maps a function-typed word back to
|
||||
`.func_ref` so a func-ref result serializes identically to legacy (kept `1128`'s
|
||||
rejection diagnostic byte-identical). Unit tests added (global_get, func_ref +
|
||||
call_indirect). **Note: native `malloc` is still REQUIRED** — the CAllocator thunk
|
||||
bottoms out at libc `malloc`, and the VM can't use a host pointer with flat-memory
|
||||
load/store, so comptime `malloc` must allocate from flat memory. The default context
|
||||
lets the allocator PROTOCOL run; native `malloc` is its final step.
|
||||
- **(7) `is_comptime` + failable/error cluster + the signed-load fix — DONE.** Coverage
|
||||
**31 → 36** handled (fallbacks 7 → 2); parity stays **688/688** both gate ON and OFF.
|
||||
- **`is_comptime`** → always 1 on the VM (folds to false in compiled code). Unlocked `1030`.
|
||||
- **Failable / error-channel cluster** (`1037` escape, `1038` handled): `kindOf(error_set)
|
||||
→ word` (a u32 tag id); `regToValue` now bridges TUPLES (the failable `(value…, tag)`
|
||||
shape the host's `checkComptimeFailable` reads); `trace_frame` packs `(func_id<<32 |
|
||||
span.start)` from a new `call_stack` (pushed by `invoke`/`runEntry`); and `sx_trace_push`
|
||||
/ `sx_trace_clear` are serviced NATIVELY (the VM calls the real sx_trace.c functions —
|
||||
linked into the compiler — so the return-trace buffer the host reads is populated
|
||||
identically to the legacy dlsym path). `raise`/`catch`/`or` all run on the VM now.
|
||||
- **Signed sub-64-bit load fix (a real GENERAL bug the failable case surfaced):**
|
||||
`readField` now SIGN-extends `i8`/`i16`/`i32`/`isize` loads (was zero-extending, so a
|
||||
stored `i32 -1` reloaded as `0xFFFFFFFF` = +4.29e9 and `< 0` was false — which silently
|
||||
hid `raise error.Bad`). Affects any negative signed sub-64-bit value stored & reloaded;
|
||||
gate-ON corpus parity confirms it's a strict fix. Unit test added (+ failable tests
|
||||
pass via 1037/1038 in the corpus).
|
||||
- **Remaining fallbacks (2, both principled — the VM correctly stays on legacy):**
|
||||
`intern` (`0626`, the welded compiler-API fn — Phase 3 re-homes it) and the inline-asm
|
||||
global call (`1654`, never comptime-evaluable). Every other measured corpus const-init
|
||||
is handled on the VM.
|
||||
At this point the flat-memory VM handles essentially the entire real comptime corpus
|
||||
(scalars, control flow, structs/tuples/arrays/slices/strings/optionals/enums, calls +
|
||||
recursion, the implicit context + allocator protocol, globals, failables + return
|
||||
traces). Phase 2 (bytecode) and Phase 3 (compiler-API on flat memory) are the forward
|
||||
work; flipping the VM to default + deleting the legacy path awaits those.
|
||||
- **(8) Wire the `#run` side-effect path; trace-clear-on-fallback — DONE.** The second
|
||||
comptime call site (`emit_llvm.runComptimeSideEffects`, top-level `#run <expr>;`) now
|
||||
routes through `tryEval` with legacy fallback, like the const-init fold; `tryEval` yields
|
||||
`.void_val` for a void/noreturn entry. Fixed a trace-corruption the new site exposed
|
||||
(`1035`): a side-effect that pushes trace frames then bails (on `print`) had the legacy
|
||||
re-run double-push them — both sites now `sx_trace_clear()` right before the legacy
|
||||
fallback to discard the VM's partial pushes. Parity **688/688** both gate ON and OFF. All
|
||||
comptime evaluation now routes through the VM-with-fallback (uniform).
|
||||
- **(9) `-Dcomptime-flat` build flag — DONE (the "swap behind a build flag" step).** The VM
|
||||
gate is now a build option (`build.zig` → a `build_opts` module on `mod`; `emit_llvm.init`
|
||||
reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`), default OFF. `zig build test
|
||||
-Dcomptime-flat` runs the FULL corpus on the VM (688/0) — the build-integrated parity
|
||||
gate. Verified the flag toggles the binary (flag-built `sx` uses the VM with no env var;
|
||||
default-built does not). This is the prerequisite to eventually making the VM default +
|
||||
deleting the legacy path (which still awaits Phase 2/3 + broader confidence).
|
||||
- **(10) Compiler-call path on the VM — `intern`/`text_of` native (Phase 3 SEED) — DONE.**
|
||||
`invoke` now services a welded `compiler`-library function (the `compiler_welded` flag is
|
||||
the safety boundary) via `Vm.callCompilerFn` — natively on flat memory, NO legacy
|
||||
`Interpreter`: `intern(s: string) -> StringId` reads the string bytes from flat memory and
|
||||
`internString`s into the (const-cast) table (pool-only, never touches type layout, so the
|
||||
VM's cached sizes stay valid); `text_of(id) -> string` materializes the pooled text back
|
||||
into flat memory as a fat pointer. Unlocked `0626` — the ONLY remaining const-init fallback
|
||||
is now the inline-asm global (`1654`, genuinely not comptime-evaluable). Parity **688/688**
|
||||
both gate ON and OFF; unit test added. This is the mechanism Phase 3 grows: the next
|
||||
compiler functions (`find_type`, `register_struct`, the reflection readers) are added the
|
||||
same way — flat-memory pointer in, handle/pointer out, no marshaling.
|
||||
|
||||
### Phase 3 — Compiler-API on flat memory (resume the stream — no weld)
|
||||
With native-byte comptime values, re-home the compiler-API:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user