From 3283effa9746978a9acc932727a6ba20f09621ac Mon Sep 17 00:00:00 2001 From: agra Date: Thu, 18 Jun 2026 16:45:25 +0300 Subject: [PATCH] =?UTF-8?q?plan:=20Phase=204=20=E2=80=94=20retire=20the=20?= =?UTF-8?q?legacy=20interp=20(ONE-evaluator=20end=20state)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audited the 5 roles interp.zig still serves (A comptime folds, B #insert, C post-link bundler, D #compiler hooks, E bail diagnostics) and the shared substrate (Value tagged union + host_ffi bridge). User decision: UNIFY — the VM gains a host-FFI escape + real-pointer translation and runs the post-link bundler too; interp.zig fully deleted. Dependency-ordered sub-phases recorded in PLAN-COMPILER-VM.md: 4A finish comptime ops (box_any/unbox_any, out/print, global_addr, trace) -> 4B VM-native diagnostics -> 4C #insert -> 4D host FFI + #compiler hooks -> 4E post-link bundler (+ dedicated bundle tests) -> 4F flip default + delete interp.zig/Value + re-express define/make_enum over the compiler-API. No code change — planning + checkpoint only. --- current/CHECKPOINT-COMPILER-API.md | 15 ++++-- current/PLAN-COMPILER-VM.md | 77 ++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 5 deletions(-) diff --git a/current/CHECKPOINT-COMPILER-API.md b/current/CHECKPOINT-COMPILER-API.md index c636e770..43115da9 100644 --- a/current/CHECKPOINT-COMPILER-API.md +++ b/current/CHECKPOINT-COMPILER-API.md @@ -34,11 +34,16 @@ with ONE welded mechanism. Branch: `reify` (off `master`). Update after every st > `0632` runs **HANDLED with ZERO fallback** (incl. the `define(declare, type_info(T))` round-trips > `0619`/`0622`/`0623`); VM output byte-matches legacy. `enum_init`/`define`/`type_info` bail loudly > on a `backing_type` tagged union rather than silent-clobber. **697/0 BOTH gates + all unit tests.** -> **THE NEXT STEP:** drive the remaining VM fallbacks (now ALL genuinely-non-metatype emit-time side -> effects — `print`/`out` 0613, `global_addr` 0600, `compiler_call` #compiler hooks 0602/0603, -> inline-asm global 1654) to empty (port or confirm each is legitimately non-comptime), THEN — with -> explicit user go-ahead — flip the VM to default + delete `interp.zig` (the end-state ONE evaluator), -> and re-express `define`/`make_enum` as sx over the compiler-API (allocation works on the sole VM). +> **THE NEXT STEP — Phase 4 (legacy-interp retirement) is now PLANNED in `PLAN-COMPILER-VM.md`.** +> Audited the 5 roles the legacy interp still serves (A comptime folds · B `#insert` · C post-link +> bundler · D `#compiler` hooks · E bail diagnostics). **User decision (2026-06-18): UNIFY** — the VM +> gains a host-FFI escape + real-pointer translation and runs the post-link bundler too; `interp.zig` +> fully deleted (true ONE evaluator). Dependency-ordered sub-phases: **4A** finish comptime ops +> (box_any/unbox_any · out/print · global_addr · trace) → **4B** VM-native diagnostics → **4C** +> `#insert` → **4D** host FFI + `#compiler` hooks → **4E** bundler (+ dedicated bundle tests, no +> corpus guard) → **4F** flip default + delete `interp.zig`/`Value` + re-express `define`/`make_enum`. +> Starting at **4A.1 (box_any/unbox_any)**. See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + +> top risks (flat-pointer escape on buffer realloc; bundler test coverage). > Earlier landed: dedicated `Type` builtin TypeId (`6844fb9`/`94f60c5`/`554871b`); WRITE side > declare_type/register_type/pointer_to VM-native (`66005af`); real lowering-time Context (`eb68d9e`); > metatype construction declare/define/enum_init (`d0ebc55`). diff --git a/current/PLAN-COMPILER-VM.md b/current/PLAN-COMPILER-VM.md index 5a8cb4b1..22106f04 100644 --- a/current/PLAN-COMPILER-VM.md +++ b/current/PLAN-COMPILER-VM.md @@ -360,6 +360,83 @@ With native-byte comptime values, re-home the compiler-API: **Verification:** the metatype + `#compiler` surfaces are gone, re-expressed as sx over the exposed compiler-API; full corpus green. +### Phase 4 — Retire the legacy interp (the ONE-evaluator end state) + +The metatype CONSTRUCTION + REFLECTION surface is VM-native (steps 7/8 — `0614`–`0624`, +`0632` all HANDLED). This phase moves EVERYTHING ELSE off `interp.zig` and deletes it. + +**What the legacy interp is still used for (audited 2026-06-18) — five roles:** + +| Role | Wired to VM? | Site | +|------|--------------|------| +| **A. Comptime folds** (type-fn / `::` const-init / `#run`) | ✅ VM + legacy fallback | `comptime.zig:530`, `emit_llvm.zig:871`/`971` | +| **B. `#insert` string eval** | ❌ legacy-only (VM wiring reverted — 0737 malformed-IR crash) | `comptime.zig:634` | +| **C. Post-link bundler** (`platform.bundle` — Info.plist/codesign/process/fs) | ❌ legacy-only | `core.zig:invokeByFuncId` ← `main.zig:769` | +| **D. `#compiler` hooks** (`compiler_call` — BuildOptions/bundling) | ❌ legacy-only; `Value`-based ABI | `compiler_hooks.zig`, `interp.zig:1130` | +| **E. Bail diagnostics** (`Interpreter.last_bail_*` statics) | n/a | `main.zig:464` | + +Shared substrate everything traffics in: the **`Value`** tagged union (the +`regToValue`/`valueToReg` bridge + the hooks + `core.zig`) and the **host-FFI bridge** +(`host_ffi.zig` + `interp.callExtern` — dlsym + cdecl trampolines for real libc). + +**DECISION (2026-06-18, user): UNIFY.** The VM gains a host-FFI escape + real-pointer +translation and runs BOTH sandboxed comptime folds AND the unsandboxed post-link bundler. +`interp.zig` is fully deleted — true ONE evaluator, two modes (sandboxed / host-effects). + +**Remaining comptime-fold gaps** (full corpus fallback inventory — 15 examples; 1179/1180 +are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan artifact): +`box_any`/`unbox_any` (6), `out`/print (2), `global_addr` (1), trace frames (1), +`compiler_call` (2 — role D). + +**Sub-phases (dependency order; each its own session, both gates 697/0 after each):** + +- **4A — finish comptime ops (small, parity-guarded).** Drive the fold fallback list to + empty except `compiler_call`: + - **4A.1** `box_any`/`unbox_any`. Word case = alloc 16B `{tag@0, value@8}`, tag = + `source_type.index()` (matches legacy comptime; note runtime `anyTag` normalizes + arbitrary-width ints), value via `writeField(source_type)` (so f32 etc. round-trip); + unbox = `readField(addr+8, target)`. Aggregate-Any payload needs the runtime + pointer-in-value-slot shape (`coerceToI64` alloca+ptrtoint) — implement or bail loudly. + - **4A.2** `out`/print → add a VM output buffer; flush through the same path as + `core.flushInterpOutput`. + - **4A.3** `global_addr` (address-of a global in flat memory). + - **4A.4** trace frames (`sx_trace_*` / `interp_print_frames`). +- **4B — VM-native diagnostics (role E). MUST land before deleting legacy.** Today a VM + bail silently falls back; with legacy gone the VM bail IS the user-facing build-gating + diagnostic. Surface the VM's `detail`/span/file into what `main.zig` renders; turn + 1179/1180-style bails into proper diagnostics. No diagnostic may regress. +- **4C — `#insert` on the VM (role B).** Re-wire `evalComptimeString` through `tryEval`; + the lowering-time-IR hardening that forced the 0737 revert is already in place. Verify + the `#insert` corpus parity. +- **4D — host FFI + `#compiler` hooks on the VM (role D + substrate). The big one.** + - **4D.1** Host-FFI escape in `Vm.invoke`: for an extern/bodyless callee not modeled by + `callMemBuiltin`, route to `host_ffi` trampolines — marshal each arg (scalar word, or + flat `Addr` → real host pointer `machine.mem.items.ptr + addr`), call, map the return. + **Central hazard:** flat-memory buffer realloc-on-growth invalidates a live host + pointer — PIN the buffer (or copy) while a host pointer is outstanding. (Note: `#run` + already calls host extern via the interp today, so host effects at comptime are not a + new policy.) + - **4D.2** `compiler_call`: route to the hook registry. Transitional shim first — convert + flat↔`Value` at the boundary (`regToValue`/`valueToReg`) so hooks work unchanged; then + (in 4F) rewrite `HookFn` to a flat-memory ABI to drop `Value`. +- **4E — post-link bundler on the VM (role C).** Depends on 4D. Route + `core.invokeByFuncId` / `main.zig`'s post-link call through the VM. **No corpus + coverage** (only runs on `sx build --bundle/--apk`) — add dedicated bundle smoke tests + (min `.app` + `.apk`); gate on real bundle builds, not just `zig build test`. Riskiest + phase. +- **4F — flip + delete (the end state).** Flip the VM to default (retire `-Dcomptime-flat`); + delete `interp.zig` (`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`); + drop the `regToValue`/`valueToReg` bridge; rewrite `HookFn` to the flat-memory ABI + (drops `Value`); simplify `core.zig` (`invokeByFuncId` → VM) + `main.zig` (`last_bail_*` + → VM diagnostics); remove the dual-path `compiler_lib` handlers (keep only VM-native + `callCompilerFn`); re-express `define`/`make_enum` as sx over the compiler-API (allocation + works on the sole evaluator) and land the original 0141 repro as a corpus test. + +**Dependencies:** 4A → (4B, 4C independent) ; 4D → 4E ; all → 4F. +**Top risks:** (1) flat-pointer escape to host FFI on buffer realloc (4D.1); (2) the bundler +has no corpus guard (4E needs dedicated tests); (3) `Value` deletion is gated on the +flat-memory `HookFn` rewrite (4D.2→4F). + ## Open questions (resolve as reached, record decisions here) - **Host-ABI vs target-ABI split.** The compiler runs on the host, so its OWN exposed