checkpoint: refresh MEM after silent-arm sweep + raw-ptr work

CHECKPOINT-MEM.md "Next step" still pointed at Phase 1.2 from the
old MEM plan — but four commits have landed since: matchContextAllocCall
drop, typed raw-pointer stores, call-conv mismatch detection, and the
silent-arms sweep. The "Current state" section also still listed
matchContextAllocCall as preserved and tied test counts to 152.

Updates:
- "Last completed step" now points at the silent-arm sweep + typed
  Store work.
- "Current state" rewritten: matchContextAllocCall is GONE, interp
  raw-pointer paths enumerated, val_ty threading mentioned,
  call-conv check called out.
- "Phase 0.3 audit findings" rewritten as historical context — chess
  no longer touches any pattern-match bypass; protocol dispatch runs.
- "Next step" recommends Phase 1.3 (closure env through context),
  notes Phase 1.2 was considered and skipped.
- Three new log entries for the four post-Step-9 commits.
This commit is contained in:
agra
2026-05-25 12:06:59 +03:00
parent e9df33a7e3
commit e843b7769d

View File

@@ -5,9 +5,14 @@ Tracking checkpoint for the mem.sx Zig-aligned implementation
## Last completed step
- **Implicit-Context refactor (Steps 1-9 of
`~/.claude/plans/lets-see-options-for-merry-dijkstra.md`)** — all
shipped. See "Current state" for what landed; "Log" for commits.
- **Interp silent-arm sweep + typed raw-pointer stores.** Every
`else =>` arm in the interp now bails with a `bailDetail("...")`
reason that surfaces through the host diagnostic as
`op=X/X: <reason>`. `inst.Store` carries `val_ty: TypeId` so
comptime raw-pointer stores honour the declared destination width
(no more 8-byte-everywhere assumption). New CLAUDE.md REJECTED
PATTERN forbids silent unimplemented arms going forward.
154/154 example tests + chess on macOS / iOS sim / Android green.
## Current state
@@ -101,79 +106,124 @@ What landed:
treat ref 0 as the Context aggregate; `.load` of an aggregate is
passthrough; `.global_addr` of `__sx_default_context` returns the
aggregate directly.
- `matchContextAllocCall` is preserved as a comptime escape hatch —
the ct_module spun up by `evalComptimeString` doesn't get the full
Allocator/CAllocator/Context type registration, so the protocol-
dispatch IR can't run in the interp. Codegen also benefits from
the direct libc malloc/free in the trivial default-context case.
- `matchContextAllocCall` is GONE (commit `d415bcc`). Comptime now
runs the full Allocator-protocol dispatch chain — the same IR
codegen emits — by reusing the parent module instead of spinning
up a separate `ct_module`. The interp gained raw-pointer paths
(`index_gep`, `index_get`, `store`, `marshalForeignArg`,
`asString`) so `context.allocator.alloc` bottoms out at host
`libc_malloc` and the returned pointer survives downstream sx ops.
- `inst.Store` carries `val_ty: TypeId` so the interp's raw-pointer
store honours the destination width — no more "assume 8 bytes"
silent clobber. Regression test at
`examples/132-comptime-typed-store-widths.sx` exercises every
primitive width (u8/u16/u32/u64, s8..s64, bool, f32, f64) via
comptime checksums compared to runtime checksums.
- Call-convention mismatch at bare-fn → fn-pointer coercion is now
a compile error (commit `f886d5f`). The chess-debug sweep that
surfaced the bug also moved `#foreign` decls to default
`callconv(.c)` and fixed every consumer-side sx callback exposed
to a C API. Regression test:
`examples/131-callconv-mismatch-diagnostic.sx`.
- Interp silent-arm sweep (commit `e9df33a`). Every `else =>` arm
has a named bail reason via `bailDetail` / `typeErrorDetail`.
`.deref` and `.unbox_any` used to silently pass through arbitrary
Value kinds — now enumerated. `#run const` errors no longer
swallow into `void_val`; emit_llvm surfaces them via std.debug.
- C-side callback into sx requires `callconv(.c)` on the sx fn (and
on any fn-pointer TYPE the user casts a C fn-pointer through).
Tests adjusted: `examples/61-objc-roundtrip.sx`,
`examples/62-objc-class.sx`, `examples/95..97-objc-block*.sx`,
`examples/ffi-06-callback.sx`.
152/152 example tests pass. 11 JNI/ObjC IR snapshots regen for the
ctx-prepended thunk signatures. `examples/75-push-context-with-arena.sx`
still demonstrates push-as-stack-discipline. `08-types.txt` regen for
the undefined-init drift (the test prints `=---` fields).
154/154 example tests pass (two new regression tests added: 131 and
132). Chess green on macOS / iOS sim / Android.
ISSUE-MEM-002 (the `context.allocator.alloc(size)` pattern-match
bypass) is closed in the sense that user-typed `context.allocator.X`
now flows through the real protocol vtable at codegen time. The
pattern-match remains in the lowering as a comptime escape hatch
(documented at `matchContextAllocCall`).
bypass) is FULLY CLOSED. User-typed `context.allocator.X` flows
through the real protocol vtable at codegen *and* runs the same
chain at comptime in the interp. No remaining shortcut.
## Next step
Phase 1.2 (free → `context.allocator.dealloc`) and Phase 1.3 (closure
env allocation through context) are unblocked. Phase 1.4 (codegen
serializer for all interp Value variants) is also unblocked.
Phase 1.3 (closure env allocation through context) and Phase 1.4
(codegen serializer for all interp Value variants) are unblocked.
Phase 1.2 (free / malloc through context) was considered and
**skipped**`context.allocator.alloc/dealloc` already works
directly; wrapper-only `malloc`/`free` would be lossy renames.
The plan at `~/.claude/plans/lets-see-options-for-merry-dijkstra.md`
is fully shipped. Next session can pick Phase 1.2 from the MEM plan
at `~/.claude/plans/tidy-doodling-cray.md`.
Suggested next move: **Phase 1.3**. Closure trampolines in
[lower.zig:lowerLambda](../src/ir/lower.zig#L5549) call
`.heap_alloc` directly for the env pointer; routing through
`context.allocator.alloc` means closures respect
`push Context.{ allocator = ... }` and get leak-tracked by
`TrackingAllocator`. Contained change. Regression test pattern:
mirror `examples/130-xx-value-routes-through-context-allocator.sx`
with a closure that captures a variable, install a tracker via
`push`, verify the tracker's counter incremented.
## Phase 0.3 audit findings — chess allocator usage
## Phase 0.3 audit findings — chess allocator usage (closed)
Sites where chess code touches an allocator API:
After Step 5 / matchContextAllocCall removal, every consumer call
to `context.allocator.X` flows through the real protocol vtable.
This section is left for history — the audit drove which sites
needed migration, but no chess code actually needed any allocator-
API change. The sites that used to bypass the protocol via the
`.heap_alloc` pattern-match now dispatch through the inline
Allocator value naturally.
- `~/projects/game/quick.sx` — standalone quicksort demo. Uses
`GPA.init()` + `Arena.init(context.allocator, …)` (already on the
new one-line `init` API). Calls `context.allocator.alloc(N*size)`
which goes through ISSUE-MEM-002's `.heap_alloc` pattern-match —
i.e. bypasses the protocol. Will be fixed by Phase 1.
- `~/projects/game/main.sx` — 7 sites of
`context.allocator.alloc(size_of(T))` for platform/GPU/pipeline
state. Same ISSUE-MEM-002 bypass; Phase 1 cleanup applies.
state. Now real protocol dispatch.
- `~/projects/game/chess/game.sx``ChessGameState.init` captures
`context.allocator` into a `parent_allocator: Allocator` field, then
restores it via `push Context.{ allocator = self.parent_allocator,
data = … }` in `select_square`. This is exactly the protocol-store +
protocol-restore shape covered by Phase 0.2 sanity tests — known
working.
- `~/projects/game/chess/pieces.sx` — declares its own `free :: (ptr:
*void) #foreign` plus `read_file_bytes` and calls `free(xx bytes)`
directly on a libc-malloc'd buffer (returned by a foreign read
helper). Bypasses the allocator protocol intentionally — this is C
interop for buffers owned by C side. No change needed.
Net: chess is well-positioned. After Phase 1 lands, the seven
`context.allocator.alloc` sites in main.sx + the one in quick.sx will
start flowing through the protocol vtable instead of `.heap_alloc`,
which means `--leak-check` (Phase 5) will start counting them for
free. No chess code needs migration on the allocator API itself.
`context.allocator` into a `parent_allocator: Allocator` field
and restores it via `push Context.{ allocator = ... }`. Worked
before, still works.
- `~/projects/game/chess/pieces.sx` — declares its own `free`
bound to libc and calls it on a libc-malloc'd buffer (from a
foreign reader). Intentional C-interop bypass — no change needed.
- `~/projects/game/quick.sx` — quicksort demo. Same flow as main.sx.
## Log
- **2026-05-25** — Implicit-Context refactor SHIPPED end-to-end. All
9 plan steps (lets-see-options-for-merry-dijkstra.md) landed.
`context` is no longer an LLVM global; every sx function carries
`__sx_ctx` at slot 0; `context.X` reads load through
`current_ctx_ref`; `push Context.{...}` is alloca + rebind; FFI-
inbound entries install `&__sx_default_context`; interp bootstraps
the default Context on top-level call; `matchContextAllocCall`
preserved as comptime escape hatch. 152/152 + unit tests green.
Three commits: `29784c2` (Steps 1-2), `92c6b47` (Step 3),
- **2026-05-25 (late)** — Interp silent-arm sweep (`e9df33a`).
Every `else =>` arm has a `bailDetail` reason; `.deref` /
`.unbox_any` previously silently passed through arbitrary Value
kinds, now enumerated. `#run const` errors surface a real
diagnostic via emit_llvm instead of becoming `void_val`.
CLAUDE.md REJECTED PATTERNS gained the "silent unimplemented
arms" section (`4de565b`). 154/154 + chess green.
- **2026-05-25 (mid)** — Typed raw-pointer stores (`f2b3868`).
`inst.Store` carries `val_ty: TypeId`, threaded by
`builder.store` and consumed by `storeAtRawPtr` to write exactly
the declared destination width. Regression at
`examples/132-comptime-typed-store-widths.sx` exercises every
primitive width via comptime/runtime checksum comparison.
`index_get` raw-pointer arm added (was bailing). Comptime init
errors no longer swallow into zero.
- **2026-05-25 (mid)** — Drop `matchContextAllocCall` (`d415bcc`).
Comptime now runs the full Allocator-protocol dispatch chain by
reusing the parent module instead of spinning up a fresh
ct_module. Interp gained `.int` / `.byte_ptr` arms in
`index_gep`, `store`, `marshalForeignArg`, `asString`. Closes
ISSUE-MEM-002 fully. JNI stub binding extended to call
current_ctx_ref → &__sx_default_context (used to be gated on
isExportedEntryName).
- **2026-05-25 (mid)** — Reject call-conv mismatch at bare-fn →
fn-pointer coercion (`f886d5f`). `#foreign` decls now default to
`callconv(.c)`. Library audit (`619aff8`) — all C-side callbacks
already followed the rule; documented the one remaining gap
(`xx <sx_fn> : *void` cast to opaque, ambiguous from cast alone).
Regression at `examples/131-callconv-mismatch-diagnostic.sx`.
- **2026-05-25 (early)** — Implicit-Context refactor SHIPPED
end-to-end. All 9 plan steps
(`lets-see-options-for-merry-dijkstra.md`) landed. `context` is
no longer an LLVM global; every sx function carries `__sx_ctx`
at slot 0; `context.X` reads load through `current_ctx_ref`;
`push Context.{...}` is alloca + rebind; FFI-inbound entries
install `&__sx_default_context`; interp bootstraps the default
Context on top-level call. 152/152 + unit tests green.
Commits: `29784c2` (Steps 1-2), `92c6b47` (Step 3),
`4bf5908` (Steps 5-7), `b69a2ea` (Step 8).
- **2026-05-24** — Phase 1.1 shipped: `buildProtocolValue` heap-copy
now routes through `context.allocator.alloc` via the new