diff --git a/current/CHECKPOINT-COMPILER-API.md b/current/CHECKPOINT-COMPILER-API.md index dab28dd8..1bbb73a5 100644 --- a/current/CHECKPOINT-COMPILER-API.md +++ b/current/CHECKPOINT-COMPILER-API.md @@ -346,6 +346,29 @@ when reached (sentinels or accessor fns; see the design doc Risks). `List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.) ## Log +- **Phase 3 P3.4 step 6 (VM plan) — REAL lowering-time Context: allocating + List-building type-fns now run HANDLED on the VM (2026-06-18).** + The VM can now evaluate a comptime type-fn that ALLOCATES at lowering time (the 0141 family) — + the legacy interp cannot. Four changes: (1) `runComptimeTypeFunc` (lower/comptime.zig) FORCES the + CAllocator→Allocator thunks to exist (`getOrCreateThunks`, idempotent, guarded by Allocator/ + CAllocator registered) BEFORE eval — a type-fn const runs at scanDecls (Pass 1), before Pass 1c + builds the default-context global + thunks, so the comptime allocator was otherwise null; + (2) `materializeDefaultContext` builds a REAL context at lowering time when the global is absent — + finds the two thunks by name (`findFuncByName`) and lays their func-refs into the inline + `Allocator` value `{ctx=null, alloc_fn@+ptr, dealloc_fn@+2*ptr}` at the head of `Context`, so + `context.allocator.alloc_bytes` dispatches `call_indirect` → thunk → native VM `malloc`; + (3) `aggType` now DEREFS a pointer `base_type` (the List write path emits `struct_gep` with + `base_type = *Struct` — `fieldOffset` panicked on the pointer; now derefs to the pointee, no + panic); (4) `subslice` handles a `[*]T` many-pointer / `*T` base (a List's `items` field — the + base IS the data pointer). **Verified end-to-end (manual probe):** a compiler-API type-fn that + builds its `[]Member` in a `List(Member)` (`.append` ×3, then `register_type(handle, kind, + vs.items[0..vs.len])`) runs **HANDLED on the VM** and mints correctly (`green=7`) — the exact + 0141 List-growth pattern, on the VM. **Can't be a corpus test yet** (gate-OFF/legacy still can't + allocate at lowering time — the dual-path bind), so locked in via VM unit tests instead + (many-pointer subslice; `struct_gep` with a pointer `base_type`). **697/0 BOTH gates + all unit + tests, EXIT=0.** On `reify`. **Remaining for the original 0141 repro (uses metatype `define`/ + `make_enum` → `call_builtin` → legacy fallback → legacy fails):** re-express the metatype over the + compiler-API so the whole type-fn runs on the VM (no `call_builtin`). THEN the repro works on the + VM — and the dual-path bind resolves only at the VM-default-flip + legacy-deletion end-state. - **Phase 3 P3.4 — investigation: the "real lowering-time Context" is BLOCKED by issue 0141 (2026-06-18).** Probed whether the VM needs a REAL lowering-time `Context` (CAllocator thunk func-refs) for allocating type-fns. **Finding: lowering-time comptime ALLOCATION fails in the LEGACY interp diff --git a/issues/0141-comptime-list-growth-in-type-construction.md b/issues/0141-comptime-list-growth-in-type-construction.md index 1337a77b..d8a1040d 100644 --- a/issues/0141-comptime-list-growth-in-type-construction.md +++ b/issues/0141-comptime-list-growth-in-type-construction.md @@ -5,6 +5,21 @@ > (`examples/0620`/`0624`); only the `List`-grown form fails. Filed to record the > two-layer root cause for a dedicated session. Surfaces a CLEAN diagnostic, not a > crash. +> +> **UPDATE (2026-06-18): the flat-memory VM now HANDLES this pattern via the +> compiler-API.** A type-fn that builds its members in a `List` (`.append` then +> `register_type(handle, kind, vs.items[0..vs.len])`) runs end-to-end on the VM +> (`-Dcomptime-flat`) — both 0141 blockers are gone there: lowering-time allocation +> works (the CAllocator thunks are now force-created in `runComptimeTypeFunc` and +> the VM's `materializeDefaultContext` lays their func-refs into a real context), +> and pointer field access has no slot_ptr chain (flat memory; `aggType` derefs a +> pointer `base_type`, `subslice` handles `[*]T`). What still fails is the ORIGINAL +> repro below, which uses the metatype `define`/`make_enum` (`#builtin` → +> `call_builtin` → VM falls back to legacy → legacy's slot_ptr + null-allocator +> bugs). Resolving the repro on the VM needs the metatype re-expressed over the +> compiler-API (so the type-fn hits no `call_builtin`); shipping it on BOTH gates +> needs the VM-default flip + legacy deletion. See CHECKPOINT-COMPILER-API +> (2026-06-18 "step 6"). ## Symptom diff --git a/src/ir/comptime_vm.test.zig b/src/ir/comptime_vm.test.zig index b43cfbf7..5957a1ef 100644 --- a/src/ir/comptime_vm.test.zig +++ b/src/ir/comptime_vm.test.zig @@ -450,6 +450,76 @@ test "comptime_vm exec: subslice of an array" { try std.testing.expectEqual(@as(i64, 43), toI64(try v.run(&fb.func, &.{}))); } +test "comptime_vm exec: subslice of a many-pointer ([*]T) — base IS the data pointer" { + const alloc = std.testing.allocator; + var table = types.TypeTable.init(alloc); + defer table.deinit(); + const arr = table.intern(.{ .array = .{ .element = .i64, .length = 5 } }); + const aptr = table.intern(.{ .pointer = .{ .pointee = arr } }); + const i64ptr = table.intern(.{ .pointer = .{ .pointee = .i64 } }); + const mptr = table.intern(.{ .many_pointer = .{ .element = .i64 } }); + const sl = table.intern(.{ .slice = .{ .element = .i64 } }); + + // a := {0,10,20,30,40}; s := ([*]i64 a)[1..4]; return len(s) + s[0] + s[2] → 43 + var fb = Fb.init(alloc, &.{}, .i64); + defer fb.deinit(); + const b0 = fb.block(&.{}); + const a = fb.add(b0, inst(.{ .alloca = arr }, aptr)); + inline for (0..5) |k| { + const ik = fb.add(b0, inst(.{ .const_int = @intCast(k) }, .i64)); + const g = fb.add(b0, inst(.{ .index_gep = .{ .lhs = ref(a), .rhs = ref(ik) } }, i64ptr)); + const cv = fb.add(b0, inst(.{ .const_int = @as(i64, @intCast(k)) * 10 }, .i64)); + _ = fb.add(b0, inst(.{ .store = .{ .ptr = ref(g), .val = ref(cv), .val_ty = .i64 } }, .void)); + } + // The alloca result IS the array's base address — subslice it as a `[*]i64`. + const lo = fb.add(b0, inst(.{ .const_int = 1 }, .i64)); + const hi = fb.add(b0, inst(.{ .const_int = 4 }, .i64)); + const s = fb.add(b0, inst(.{ .subslice = .{ .base = ref(a), .lo = ref(lo), .hi = ref(hi), .base_ty = mptr } }, sl)); + const slen = fb.add(b0, inst(.{ .length = .{ .operand = ref(s) } }, .i64)); + const z = fb.add(b0, inst(.{ .const_int = 0 }, .i64)); + const e0 = fb.add(b0, inst(.{ .index_get = .{ .lhs = ref(s), .rhs = ref(z) } }, .i64)); + const two = fb.add(b0, inst(.{ .const_int = 2 }, .i64)); + const e2 = fb.add(b0, inst(.{ .index_get = .{ .lhs = ref(s), .rhs = ref(two) } }, .i64)); + const t = fb.add(b0, inst(.{ .add = .{ .lhs = ref(slen), .rhs = ref(e0) } }, .i64)); + const sum = fb.add(b0, inst(.{ .add = .{ .lhs = ref(t), .rhs = ref(e2) } }, .i64)); + _ = fb.add(b0, inst(.{ .ret = .{ .operand = ref(sum) } }, .void)); + + var v = vm.Vm.init(alloc); + v.table = &table; + defer v.deinit(); + try std.testing.expectEqual(@as(i64, 43), toI64(try v.run(&fb.func, &.{}))); +} + +test "comptime_vm exec: struct_gep with an explicit pointer base_type derefs to the field (no panic)" { + const alloc = std.testing.allocator; + var table = types.TypeTable.init(alloc); + defer table.deinit(); + const s_ty = table.intern(.{ .@"struct" = .{ .name = dummy, .fields = &.{ + .{ .name = dummy, .ty = .i64 }, + .{ .name = dummy, .ty = .i64 }, + } } }); + const sptr = table.intern(.{ .pointer = .{ .pointee = s_ty } }); + const i64ptr = table.intern(.{ .pointer = .{ .pointee = .i64 } }); + + // p := alloca S (a *S); struct_gep(p, field 1) with base_type = *S → &p.y; + // store 80; load → 80. Exercises aggType derefing a POINTER base_type (the + // List write path sets base_type = *Struct; without the deref fieldOffset panics). + var fb = Fb.init(alloc, &.{}, .i64); + defer fb.deinit(); + const b0 = fb.block(&.{}); + const p = fb.add(b0, inst(.{ .alloca = s_ty }, sptr)); + const g = fb.add(b0, inst(.{ .struct_gep = .{ .base = ref(p), .field_index = 1, .base_type = sptr } }, i64ptr)); + const v80 = fb.add(b0, inst(.{ .const_int = 80 }, .i64)); + _ = fb.add(b0, inst(.{ .store = .{ .ptr = ref(g), .val = ref(v80), .val_ty = .i64 } }, .void)); + const got = fb.add(b0, inst(.{ .load = .{ .operand = ref(g) } }, .i64)); + _ = fb.add(b0, inst(.{ .ret = .{ .operand = ref(got) } }, .void)); + + var v = vm.Vm.init(alloc); + v.table = &table; + defer v.deinit(); + try std.testing.expectEqual(@as(i64, 80), toI64(try v.run(&fb.func, &.{}))); +} + test "comptime_vm exec: non-pointer optional wrap/unwrap/has_value/coalesce" { const alloc = std.testing.allocator; var table = types.TypeTable.init(alloc); diff --git a/src/ir/comptime_vm.zig b/src/ir/comptime_vm.zig index 1374f125..a793162b 100644 --- a/src/ir/comptime_vm.zig +++ b/src/ir/comptime_vm.zig @@ -348,19 +348,35 @@ pub const Vm = struct { if (g.init_val) |iv| try self.layoutConst(table, iv, g.ty, addr); return addr; } - // No `__sx_default_context` global yet — this is the LOWERING-time path - // (the global is emitted later, at codegen). Materialize a ZEROED `Context` - // of the right size instead: a type-fn that never touches the allocator - // ignores it; one that DOES allocate reads a null `alloc_fn` (zeroed) and - // `call_indirect` on the null func-ref bails → legacy fallback (which has a - // real default context). A real lowering-time context (with the CAllocator - // thunk func-refs, so allocating type-fns also run on the VM) is a follow-up. - // `internString` of an existing name is idempotent (pool-only, no layout - // change) — the same `@constCast` the reader handlers use on the table. + // No `__sx_default_context` global yet — the LOWERING-time path (a type-fn + // const runs at scanDecls, before Pass 1c emits that global). Build the + // REAL default context directly from the CAllocator→Allocator thunks + // (forced to exist by `runComptimeTypeFunc` before this runs), mirroring + // the legacy `defaultContextValue` / `emitDefaultContextGlobal`: the inline + // `Allocator` value is `{ ctx: *void = null, alloc_fn, dealloc_fn }` (three + // pointer-sized words) at the head of `Context`, so `context.allocator` + // dispatches `alloc_bytes` → `call_indirect` → the thunk → native `malloc`, + // all on the VM. If a thunk is absent (std not imported), the field stays + // null (zeroed) and an allocating body bails — same as a non-std program. const ctx_name = @constCast(table).internString("Context"); const ctx_ty = table.findByName(ctx_name) orelse return self.failMsg("comptime VM: no Context type to materialize the implicit context"); - return self.machine.allocBytes(table.typeSizeBytes(ctx_ty), table.typeAlignBytes(ctx_ty)); // zeroed + const addr = self.machine.allocBytes(table.typeSizeBytes(ctx_ty), table.typeAlignBytes(ctx_ty)); // zeroed + const ps: Addr = table.pointer_size; + if (self.findFuncByName(module, "__thunk_CAllocator_Allocator_alloc_bytes")) |fid| + try self.machine.writeWord(addr + ps, ps, funcRefWord(fid)); // allocator.alloc_fn @ +ptr_size + if (self.findFuncByName(module, "__thunk_CAllocator_Allocator_dealloc_bytes")) |fid| + try self.machine.writeWord(addr + 2 * ps, ps, funcRefWord(fid)); // allocator.dealloc_fn @ +2*ptr_size + return addr; + } + + /// Find a module function by its exact name → its `FuncId`, or null. Used to + /// resolve the CAllocator thunk func-refs for the lowering-time default context. + fn findFuncByName(_: *Vm, module: *const Module, name: []const u8) ?inst_mod.FuncId { + for (module.functions.items, 0..) |*f, i| { + if (std.mem.eql(u8, module.types.getString(f.name), name)) return inst_mod.FuncId.fromIndex(@intCast(i)); + } + return null; } /// Lay a static `ConstantValue` of type `ty` into flat memory at `addr` (the @@ -689,6 +705,10 @@ pub const Vm = struct { } else if (!bty.isBuiltin()) { switch (table.get(bty)) { .array => |a| elem = a.element, + // `[*]T` (a List's `items` field) / `*T`: the base IS the + // data pointer; subslicing yields `{ base + lo, hi - lo }`. + .many_pointer => |mp| elem = mp.element, + .pointer => |p| elem = p.pointee, .slice => |sl| { elem = sl.element; data = try self.sliceData(table, base); @@ -1496,13 +1516,15 @@ pub const Vm = struct { /// lowering set it, else the base operand's Ref type — dereferenced when the /// base is a POINTER (`struct_gep` on an `alloca` result is `*S` → `S`). fn aggType(self: *Vm, table: *const types.TypeTable, fa: inst_mod.FieldAccess, ref_types: []const TypeId) Error!TypeId { - if (fa.base_type) |bt| return bt; - const rt = try self.refTy(ref_types, fa.base); - if (!rt.isBuiltin()) { - const info = table.get(rt); - if (info == .pointer) return info.pointer.pointee; - } - return rt; + // The explicit `base_type` when lowering set it, else the base operand's + // Ref type. Either way, deref ONE pointer level when the result is a + // pointer-to-struct: a `struct_gep`/`struct_get` on a `*Struct` receiver + // (e.g. `list.field` where `list: *List`) computes the field offset on the + // POINTEE struct, with the base register already holding the pointer + // address. Lowering sets `base_type = *Struct` on the write/lvalue path. + const raw = fa.base_type orelse (try self.refTy(ref_types, fa.base)); + if (!raw.isBuiltin() and table.get(raw) == .pointer) return table.get(raw).pointer.pointee; + return raw; } /// The byte offset of tuple element `idx` — the positional analogue of diff --git a/src/ir/lower/comptime.zig b/src/ir/lower/comptime.zig index 5e6fbd8c..b4d36a44 100644 --- a/src/ir/lower/comptime.zig +++ b/src/ir/lower/comptime.zig @@ -488,6 +488,24 @@ fn preludeBeforeReturn(body: *const Node) []const *const Node { /// `declare()` never completed by `define()` (a zero-field nominal slot that /// would otherwise panic at codegen). `span` locates both diagnostics. pub fn runComptimeTypeFunc(self: *Lowering, func_id: FuncId, span: ast.Span) ?TypeId { + // Force the CAllocator→Allocator thunks to exist BEFORE the type-fn evaluates. + // A type-fn const runs at scanDecls time (Pass 1), BEFORE `emitDefaultContextGlobal` + // (Pass 1c) builds the default context + those thunks — so the comptime + // `context.allocator` is otherwise null and any allocation bails (issue 0141). + // `getOrCreateThunks` is idempotent (cached in `protocol_thunk_map`), so the + // later Pass-1c call reuses these. Guarded exactly like `emitDefaultContextGlobal` + // (skip when the std allocator types aren't registered). This lets the + // flat-memory VM materialize a REAL lowering-time context (the func-refs are + // dispatchable on the VM via `call_indirect` → thunk → native malloc). + { + const tbl = &self.module.types; + if (tbl.findByName(tbl.internString("Allocator")) != null and + tbl.findByName(tbl.internString("CAllocator")) != null) + { + _ = self.getOrCreateThunks("Allocator", "CAllocator"); + } + } + var interp = interp_mod.Interpreter.init(self.module, self.alloc); defer interp.deinit(); if (self.diagnostics) |d| if (d.import_sources) |sm| interp.setSourceMap(sm);