From e843b7769dddefbe35b9b7d41aef2e541f9b5215 Mon Sep 17 00:00:00 2001 From: agra Date: Mon, 25 May 2026 12:06:59 +0300 Subject: [PATCH] checkpoint: refresh MEM after silent-arm sweep + raw-ptr work MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CHECKPOINT-MEM.md "Next step" still pointed at Phase 1.2 from the old MEM plan — but four commits have landed since: matchContextAllocCall drop, typed raw-pointer stores, call-conv mismatch detection, and the silent-arms sweep. The "Current state" section also still listed matchContextAllocCall as preserved and tied test counts to 152. Updates: - "Last completed step" now points at the silent-arm sweep + typed Store work. - "Current state" rewritten: matchContextAllocCall is GONE, interp raw-pointer paths enumerated, val_ty threading mentioned, call-conv check called out. - "Phase 0.3 audit findings" rewritten as historical context — chess no longer touches any pattern-match bypass; protocol dispatch runs. - "Next step" recommends Phase 1.3 (closure env through context), notes Phase 1.2 was considered and skipped. - Three new log entries for the four post-Step-9 commits. --- current/CHECKPOINT-MEM.md | 160 +++++++++++++++++++++++++------------- 1 file changed, 105 insertions(+), 55 deletions(-) diff --git a/current/CHECKPOINT-MEM.md b/current/CHECKPOINT-MEM.md index 467c69f..31a08c1 100644 --- a/current/CHECKPOINT-MEM.md +++ b/current/CHECKPOINT-MEM.md @@ -5,9 +5,14 @@ Tracking checkpoint for the mem.sx Zig-aligned implementation ## Last completed step -- **Implicit-Context refactor (Steps 1-9 of - `~/.claude/plans/lets-see-options-for-merry-dijkstra.md`)** — all - shipped. See "Current state" for what landed; "Log" for commits. +- **Interp silent-arm sweep + typed raw-pointer stores.** Every + `else =>` arm in the interp now bails with a `bailDetail("...")` + reason that surfaces through the host diagnostic as + `op=X/X: `. `inst.Store` carries `val_ty: TypeId` so + comptime raw-pointer stores honour the declared destination width + (no more 8-byte-everywhere assumption). New CLAUDE.md REJECTED + PATTERN forbids silent unimplemented arms going forward. + 154/154 example tests + chess on macOS / iOS sim / Android green. ## Current state @@ -101,79 +106,124 @@ What landed: treat ref 0 as the Context aggregate; `.load` of an aggregate is passthrough; `.global_addr` of `__sx_default_context` returns the aggregate directly. -- `matchContextAllocCall` is preserved as a comptime escape hatch — - the ct_module spun up by `evalComptimeString` doesn't get the full - Allocator/CAllocator/Context type registration, so the protocol- - dispatch IR can't run in the interp. Codegen also benefits from - the direct libc malloc/free in the trivial default-context case. +- `matchContextAllocCall` is GONE (commit `d415bcc`). Comptime now + runs the full Allocator-protocol dispatch chain — the same IR + codegen emits — by reusing the parent module instead of spinning + up a separate `ct_module`. The interp gained raw-pointer paths + (`index_gep`, `index_get`, `store`, `marshalForeignArg`, + `asString`) so `context.allocator.alloc` bottoms out at host + `libc_malloc` and the returned pointer survives downstream sx ops. +- `inst.Store` carries `val_ty: TypeId` so the interp's raw-pointer + store honours the destination width — no more "assume 8 bytes" + silent clobber. Regression test at + `examples/132-comptime-typed-store-widths.sx` exercises every + primitive width (u8/u16/u32/u64, s8..s64, bool, f32, f64) via + comptime checksums compared to runtime checksums. +- Call-convention mismatch at bare-fn → fn-pointer coercion is now + a compile error (commit `f886d5f`). The chess-debug sweep that + surfaced the bug also moved `#foreign` decls to default + `callconv(.c)` and fixed every consumer-side sx callback exposed + to a C API. Regression test: + `examples/131-callconv-mismatch-diagnostic.sx`. +- Interp silent-arm sweep (commit `e9df33a`). Every `else =>` arm + has a named bail reason via `bailDetail` / `typeErrorDetail`. + `.deref` and `.unbox_any` used to silently pass through arbitrary + Value kinds — now enumerated. `#run const` errors no longer + swallow into `void_val`; emit_llvm surfaces them via std.debug. - C-side callback into sx requires `callconv(.c)` on the sx fn (and on any fn-pointer TYPE the user casts a C fn-pointer through). Tests adjusted: `examples/61-objc-roundtrip.sx`, `examples/62-objc-class.sx`, `examples/95..97-objc-block*.sx`, `examples/ffi-06-callback.sx`. -152/152 example tests pass. 11 JNI/ObjC IR snapshots regen for the -ctx-prepended thunk signatures. `examples/75-push-context-with-arena.sx` -still demonstrates push-as-stack-discipline. `08-types.txt` regen for -the undefined-init drift (the test prints `=---` fields). +154/154 example tests pass (two new regression tests added: 131 and +132). Chess green on macOS / iOS sim / Android. ISSUE-MEM-002 (the `context.allocator.alloc(size)` pattern-match -bypass) is closed in the sense that user-typed `context.allocator.X` -now flows through the real protocol vtable at codegen time. The -pattern-match remains in the lowering as a comptime escape hatch -(documented at `matchContextAllocCall`). +bypass) is FULLY CLOSED. User-typed `context.allocator.X` flows +through the real protocol vtable at codegen *and* runs the same +chain at comptime in the interp. No remaining shortcut. ## Next step -Phase 1.2 (free → `context.allocator.dealloc`) and Phase 1.3 (closure -env allocation through context) are unblocked. Phase 1.4 (codegen -serializer for all interp Value variants) is also unblocked. +Phase 1.3 (closure env allocation through context) and Phase 1.4 +(codegen serializer for all interp Value variants) are unblocked. +Phase 1.2 (free / malloc through context) was considered and +**skipped** — `context.allocator.alloc/dealloc` already works +directly; wrapper-only `malloc`/`free` would be lossy renames. -The plan at `~/.claude/plans/lets-see-options-for-merry-dijkstra.md` -is fully shipped. Next session can pick Phase 1.2 from the MEM plan -at `~/.claude/plans/tidy-doodling-cray.md`. +Suggested next move: **Phase 1.3**. Closure trampolines in +[lower.zig:lowerLambda](../src/ir/lower.zig#L5549) call +`.heap_alloc` directly for the env pointer; routing through +`context.allocator.alloc` means closures respect +`push Context.{ allocator = ... }` and get leak-tracked by +`TrackingAllocator`. Contained change. Regression test pattern: +mirror `examples/130-xx-value-routes-through-context-allocator.sx` +with a closure that captures a variable, install a tracker via +`push`, verify the tracker's counter incremented. -## Phase 0.3 audit findings — chess allocator usage +## Phase 0.3 audit findings — chess allocator usage (closed) -Sites where chess code touches an allocator API: +After Step 5 / matchContextAllocCall removal, every consumer call +to `context.allocator.X` flows through the real protocol vtable. +This section is left for history — the audit drove which sites +needed migration, but no chess code actually needed any allocator- +API change. The sites that used to bypass the protocol via the +`.heap_alloc` pattern-match now dispatch through the inline +Allocator value naturally. -- `~/projects/game/quick.sx` — standalone quicksort demo. Uses - `GPA.init()` + `Arena.init(context.allocator, …)` (already on the - new one-line `init` API). Calls `context.allocator.alloc(N*size)` - which goes through ISSUE-MEM-002's `.heap_alloc` pattern-match — - i.e. bypasses the protocol. Will be fixed by Phase 1. - `~/projects/game/main.sx` — 7 sites of `context.allocator.alloc(size_of(T))` for platform/GPU/pipeline - state. Same ISSUE-MEM-002 bypass; Phase 1 cleanup applies. + state. Now real protocol dispatch. - `~/projects/game/chess/game.sx` — `ChessGameState.init` captures - `context.allocator` into a `parent_allocator: Allocator` field, then - restores it via `push Context.{ allocator = self.parent_allocator, - data = … }` in `select_square`. This is exactly the protocol-store + - protocol-restore shape covered by Phase 0.2 sanity tests — known - working. -- `~/projects/game/chess/pieces.sx` — declares its own `free :: (ptr: - *void) #foreign` plus `read_file_bytes` and calls `free(xx bytes)` - directly on a libc-malloc'd buffer (returned by a foreign read - helper). Bypasses the allocator protocol intentionally — this is C - interop for buffers owned by C side. No change needed. - -Net: chess is well-positioned. After Phase 1 lands, the seven -`context.allocator.alloc` sites in main.sx + the one in quick.sx will -start flowing through the protocol vtable instead of `.heap_alloc`, -which means `--leak-check` (Phase 5) will start counting them for -free. No chess code needs migration on the allocator API itself. + `context.allocator` into a `parent_allocator: Allocator` field + and restores it via `push Context.{ allocator = ... }`. Worked + before, still works. +- `~/projects/game/chess/pieces.sx` — declares its own `free` + bound to libc and calls it on a libc-malloc'd buffer (from a + foreign reader). Intentional C-interop bypass — no change needed. +- `~/projects/game/quick.sx` — quicksort demo. Same flow as main.sx. ## Log -- **2026-05-25** — Implicit-Context refactor SHIPPED end-to-end. All - 9 plan steps (lets-see-options-for-merry-dijkstra.md) landed. - `context` is no longer an LLVM global; every sx function carries - `__sx_ctx` at slot 0; `context.X` reads load through - `current_ctx_ref`; `push Context.{...}` is alloca + rebind; FFI- - inbound entries install `&__sx_default_context`; interp bootstraps - the default Context on top-level call; `matchContextAllocCall` - preserved as comptime escape hatch. 152/152 + unit tests green. - Three commits: `29784c2` (Steps 1-2), `92c6b47` (Step 3), +- **2026-05-25 (late)** — Interp silent-arm sweep (`e9df33a`). + Every `else =>` arm has a `bailDetail` reason; `.deref` / + `.unbox_any` previously silently passed through arbitrary Value + kinds, now enumerated. `#run const` errors surface a real + diagnostic via emit_llvm instead of becoming `void_val`. + CLAUDE.md REJECTED PATTERNS gained the "silent unimplemented + arms" section (`4de565b`). 154/154 + chess green. +- **2026-05-25 (mid)** — Typed raw-pointer stores (`f2b3868`). + `inst.Store` carries `val_ty: TypeId`, threaded by + `builder.store` and consumed by `storeAtRawPtr` to write exactly + the declared destination width. Regression at + `examples/132-comptime-typed-store-widths.sx` exercises every + primitive width via comptime/runtime checksum comparison. + `index_get` raw-pointer arm added (was bailing). Comptime init + errors no longer swallow into zero. +- **2026-05-25 (mid)** — Drop `matchContextAllocCall` (`d415bcc`). + Comptime now runs the full Allocator-protocol dispatch chain by + reusing the parent module instead of spinning up a fresh + ct_module. Interp gained `.int` / `.byte_ptr` arms in + `index_gep`, `store`, `marshalForeignArg`, `asString`. Closes + ISSUE-MEM-002 fully. JNI stub binding extended to call + current_ctx_ref → &__sx_default_context (used to be gated on + isExportedEntryName). +- **2026-05-25 (mid)** — Reject call-conv mismatch at bare-fn → + fn-pointer coercion (`f886d5f`). `#foreign` decls now default to + `callconv(.c)`. Library audit (`619aff8`) — all C-side callbacks + already followed the rule; documented the one remaining gap + (`xx : *void` cast to opaque, ambiguous from cast alone). + Regression at `examples/131-callconv-mismatch-diagnostic.sx`. +- **2026-05-25 (early)** — Implicit-Context refactor SHIPPED + end-to-end. All 9 plan steps + (`lets-see-options-for-merry-dijkstra.md`) landed. `context` is + no longer an LLVM global; every sx function carries `__sx_ctx` + at slot 0; `context.X` reads load through `current_ctx_ref`; + `push Context.{...}` is alloca + rebind; FFI-inbound entries + install `&__sx_default_context`; interp bootstraps the default + Context on top-level call. 152/152 + unit tests green. + Commits: `29784c2` (Steps 1-2), `92c6b47` (Step 3), `4bf5908` (Steps 5-7), `b69a2ea` (Step 8). - **2026-05-24** — Phase 1.1 shipped: `buildProtocolValue` heap-copy now routes through `context.allocator.alloc` via the new