Files
sx/current/CHECKPOINT-COMPILER-API.md
agra 18af8eb845 comptime-API: strip the byte-weld; pivot to a flat-memory comptime VM
The byte-weld (sx structs whose layout was validated to mirror the
compiler's Zig records) plus the serialization/marshaling bridge was the
wrong direction: it bolted a parallel layout regime and hand-built
byte-copies onto a comptime value model that fundamentally isn't bytes.

Strip the struct-weld machinery:
- compiler_lib.zig loses the type registry (weldStruct / bound_types /
  BoundType / FieldLayout / findType / SxField / LayoutMismatch /
  validateStructLayout); it is now just the intern/text_of function
  host-call bridge (kept as the Phase-3 compiler-call seed).
- nominal.zig loses validateWeldedStruct / weldedFieldOrderStr + the
  sd.abi == .zig validation call.
- Remove the struct-weld unit tests and examples 0625/0627 (welded
  structs) + 1183/1186 (weld-layout diagnostics).
- The #library / abi / extern syntax stays.

Record the new direction: a bytecode VM over flat, byte-addressable
memory so comptime values are native bytes (no weld/validation/marshal),
target-aware (preserves cross-compilation) and sandboxed. See
current/PLAN-COMPILER-VM.md (Phase 0 strip -> Phase 1 flat-memory value
model -> Phase 2 bytecode -> Phase 3 compiler-API on flat memory).
design/comptime-compiler-api.md gets a SUPERSEDED banner. Also drop the
"~500 lines / split the step" rule from CLAUDE.md.
2026-06-17 19:29:36 +03:00

447 lines
30 KiB
Markdown

# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`)
Companion to the design-of-record
[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan
+ phased build order live there). This stream supersedes the metatype
`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
## ⏯ Resume (fresh session)
> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.**
> The **byte-weld + serialization/marshaling** approach is the wrong direction and is
> being **stripped**. New foundation: a **bytecode VM over flat, byte-addressable
> memory** so comptime values are native bytes; then the compiler-API rides on it with
> direct memory access (no weld, no validation, no marshaling). Everything below this
> banner describes the now-superseded weld state (committed on `reify` through
> `40d075c`) and is kept only to scope the Phase 0 strip. Read
> `PLAN-COMPILER-VM.md` first.
>
> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as
> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)`
> reads the `Value` union's memory, not a struct. The weld tried to bridge that with
> hand-marshaling — exactly what the design set out to kill. Flat memory makes comptime
> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it
> breaks cross-compilation — host vs target layout — and loses the sandbox. A
> flat-memory VM keeps both while getting native bytes + speed.)
>
> **Next action:** execute Phase 0 of `PLAN-COMPILER-VM.md` (strip the weld machinery),
> then Phase 1 (flat-memory value model). Build/verify: `zig build && zig build test`.
### (superseded) prior weld resume
Phase 1 done; Phase 2 welded structs were working via reflection + memory-order
validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even
earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in
the compiler type's MEMORY order; the compiler reflected the bound Zig type and
VALIDATED the header. **This whole mechanism is now being stripped — see the banner.**
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
> See CLAUDE.md → Snapshot integrity.
## Last completed step
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
end-to-end as a byte-identical mirror of the bound Zig type.
Design (locked, supersedes the byte-layout-override plan):
- The sx header declares fields in the compiler type's MEMORY order. The compiler
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
byte-blob, no index/remap tables, no reorder, no special LLVM path.
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
*wrong field order at position N* (+ expected memory order), *type layout
mismatch* (field size), *layout mismatch* (total size / count).
What changed from the dropped plan:
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
the sx header against the memory-ordered registry.
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
(+ `weldedFieldOrderStr`).
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
refreshed.
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
byte-layout-override engine; superseded by the reflection+validation design above.
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
padding included), so it passes to a Zig handler as raw memory with zero
marshalling. (The alternative — handlers reading interp `Value` aggregates
logically, no layout override — was rejected; welded types must also be usable as
runtime data, and the design wants the literal byte weld.)
- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20,
`is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override
is genuinely required (`Field`'s two-u32 natural layout was the easy case).
- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second
`bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc,
fields, total)` — pure: orders fields by ascending byte offset, inserts padding
elements for gaps + the alignment tail, and builds the sx-field → LLVM-element
remap. This is what the LLVM type builder + struct-GEP sites will consume.
- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad);
`StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24,
pad@25..32]`, remap `[1,0,3,2]`.
- `zig build` + `zig build test` green.
### Earlier — Phase 1 polish (comptime-only enforcement)
**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error
instead of an undefined-symbol link failure.**
- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded`
AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a
`#run`/`::` initializer wrapper whose LLVM body is dead), print a clear
"comptime-only … cannot be called at runtime" error and set
`comptime_failed` (the driver halts before object/JIT emission). The enclosing
`is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green.
- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)`
→ clean error, exit 1, no link failure).
- `zig build` + `zig build test` green (458 unit + 690 corpus).
### Earlier — fifth sub-step (host-call bridge)
**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to
its registered Zig handler instead of dlsym.**
- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`,
`bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the
string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value ->
Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)`
(the same mutable-table access the metatype mint path uses); `text_of` reads the
const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern).
- IR `Function` gained `compiler_welded: bool`. `declareFunction`
(`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES:
the bound lib must be `compiler` and the name must be on the function-export
list — else a build-gating `.err` (no silent fall-through to dlsym).
- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function
routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the
export list).
- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run
text_of(intern("hello, compiler"))` folds to a string constant → prints it);
`examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name →
build error). `findFn` lookup unit-tested.
- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a
RUNTIME call would emit a reference to a non-existent extern symbol → a loud
LINK error (not silent, but not a tidy diagnostic). The examples call welded fns
only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the
immediate follow-up.
- `zig build` + `zig build test` green (458 unit tests + 689 corpus).
### Earlier — fourth sub-step (welded-struct layout validation)
**A `struct abi(.zig) extern compiler { … }` is validated against the binding
registry as a *header checked against the implementation*.**
- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns
the first `LayoutMismatch` (field count / name / size / total) or null. Plus
`lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes;
each drift flagged as the right variant).
- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`,
`validateWeldedStruct` checks the bound lib is `compiler`, the name is on the
export list (`findType`), and the sx layout (field names + `typeSizeBytes` +
total) matches the welded type — emitting a build-gating `.err` (good span into
the struct body) on any failure. No silent reinterpretation.
- `#library "compiler"` is the comptime-only internal surface, NOT a dylib —
`src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so`
load warning).
- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds,
validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld-
struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic).
- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it
isn't exercised by `Field` (two u32s = natural layout coincides with the weld).
It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound
offsets actually differ from the sx-natural ones. The validation already checks
per-field size + total, so a layout drift is caught even before the override
engine exists.
- `zig build` + `zig build test` green (456 unit tests + 687 corpus).
### Earlier — third sub-step (binding registry)
**The binding registry (welded-type lookup, layout baked from the real Zig
type).**
- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the
curated safety boundary. `BoundType { sx_name, size, alignment, fields:
[]FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real
Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a
sx-field-count mismatch is a `@compileError`, never a silent truncation).
`bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` —
two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl
resolution path will consult (returns null off the export list — clean boundary,
no silent default).
- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`.
- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real
`StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4);
an unexported name returns null. Break-verified (a wrong size → suite red,
named `ir.compiler_lib.test...`).
- `zig build` + `zig build test` green (454 unit tests).
### Earlier — second sub-step (struct-decl parse)
**`abi(.zig) extern <lib>` PARSES on a STRUCT decl (parse-only, no semantics).**
- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields.
- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler`
check), parse an optional `abi(...)` then optional `extern <lib>` — same slot
order as fn decls — and thread them onto the node. Ordinary structs are
unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent).
- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern
compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib
== "compiler"`, field list intact; a plain struct leaves `abi == .default` /
`extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite
red, confirming the test runs).
- `zig build` + `zig build test` green.
### Earlier — first sub-step (fn decls) + the syntax pivot
**`abi(.zig) extern <lib>` PARSES on a fn decl (parse-only).** Plus the syntax
pivot it required.
Syntax decision (locked 2026-06-17, supersedes the doc's original
`extern(.zig) <lib>` single-qualifier form): the ABI/layout selector and the
linkage keyword are two orthogonal annotations.
- `abi(.x)` — ABI / calling-convention annotation in the slot **before**
`extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.**
`ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the
`compiler` library), `.pure` (naked asm), `.default` (unannotated). Can appear
standalone (no extern) on any fn / fn-type / lambda.
- `extern <lib>` — linkage keyword + binding source (named library).
So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`.
What landed:
- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`;
the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`.
- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`,
keyword string `"callconv"` → `"abi"`.
- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi`
(parses `abi(.c|.zig|.pure)`); wired in the fn-decl postfix slot (before
`extern`/`export`), the function-type-expr slot, and the lambda slot;
`isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`.
- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`,
`closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the
function-type resolver maps `.zig`/`.pure` → IR `.default` (no fn-pointer-type
CC for those decl-level ABIs; neither occurs in a function-TYPE position yet).
- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the
user-facing text `callconv(.c)` → `abi(.c)`.
- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type
callback annotations — none in the fn-decl postfix slot, so no reordering).
- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 /
1200) regenerated for the rename.
- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern <lib>`
on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib ==
"compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` /
`abi(.pure)`. lexer/sema tests updated.
`zig build` + `zig build test` green (450/450 unit + 685 corpus).
## Current state
> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld
> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for
> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the
> weld *semantics* (layout reflection/validation, marshaling dispatch) go.
- `compiler :: #library "compiler";` parses + is recognised as the comptime-only
internal surface (never dlopen'd).
- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry
(faithful → ok; drift → build-gating diagnostic). `Field` welds + usable.
- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to
their registered Zig handler (`intern`/`text_of` round-trip works); unexported
names rejected at declaration. Comptime-only.
- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only
enforcement at `emitCall`); the legitimate `#run`/`::` use stays green.
- The whole Phase 1 foundation (parse → registry → struct-layout validation →
function host-call bridge → comptime-only enforcement) is in place for the
two-u32 `Field` case + the two string readers.
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
(needed by `StructInfo`'s slice field, Phase 2).
## Next step — execute `PLAN-COMPILER-VM.md`
> The weld is being stripped. The next step is **Phase 0 of
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize /
> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig`
> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics
> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then
> Phase 1 (flat-memory value model). The weld-era "next step" below is **obsolete** —
> kept only as a record of what the weld surface was about to do.
### (obsolete) weld-era next step
Welded structs were byte-identical mirrors, so the API surface was set to grow:
- **Bind `register_struct` / `find_type`** over the host-call bridge
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
kind well-formedness — the checks `define` does today). Because the welded
`StructInfo` is byte-identical, the handler can read it as the real Zig
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
Prove: build a struct programmatically + round-trip a source one.
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
(`defineStruct` / the `reflectTypeInfo` struct path).
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
(optional fields → sentinels) — each just needs an sx header in the compiler
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
`#compiler`.
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
when reached (sentinels or accessor fns; see the design doc Risks).
## Known issues
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
## Log
- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).**
`comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on
the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back.
Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs
(1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of
crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind
`SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final.
688 corpus green.
- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).**
Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the
wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs
the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg →
Reg, aggregates into flat memory) + `regToValue` (VM result → Value, deep-copied).
Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip
unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through
the VM with legacy fallback).
- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).**
`Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals
arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared
flat memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate
Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b).
Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next:
1.5b (call_builtin/compiler_call/extern), then hybrid wiring.
- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).**
Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested
(deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops
(tagged-union payload/any/closures) blind — their byte semantics are ambiguous without
real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) +
HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so
the VM runs the real corpus and surfaces exactly what's needed. Key design point for
calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's
allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision
block.
- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).**
`kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}`
aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`;
const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91,
?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any,
closures).
- **Phase 1 sub-step 4b (VM plan) — slices + strings on flat memory (2026-06-17).**
`{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate).
Ported `const_string` (text+NUL + fat pointer in flat memory), `length`/`data_ptr`,
`array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and
`str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23,
subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures).
- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on flat memory (2026-06-17).**
`kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get`
(`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over
array/pointer/many_pointer; slice/string bases bail), `length` on array values.
Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next:
sub-step 4b (slices/strings, then optionals/enums/any/closures).
- **Phase 1 sub-step 3 (VM plan) — memory + structs on flat memory (2026-06-17).**
`Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported
`alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out
at the table's natural offsets. Value model: scalar/pointer → register word;
struct → lives in flat memory, its value IS its address (read→addr, write→memcpy), so
nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on
not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested
(struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next:
sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls).
- **Phase 1 sub-step 2 (VM plan) — flat-memory executor: scalars + control flow
(2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over
flat-memory frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy
interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic,
comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and
`block_param`; every other op bails loudly (`error.Unsupported` + op name in
`detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic,
cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus
untouched (688 green). Next: sub-step 3 (memory + aggregates on flat memory, where
target-aware layout enters).
- **Phase 1 sub-step 1 (VM plan) — flat-memory machine substrate (2026-06-17).**
New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator
with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0
reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on
deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested
(`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy
interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor +
scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule
from CLAUDE.md per request.
- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept
(2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig`
(`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/
`LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr`
+ the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and
examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function
host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3
compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the
`emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples
`0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next:
Phase 1 (flat-memory value model) per `PLAN-COMPILER-VM.md`.
- **DIRECTION CHANGE — pivot off the byte-weld to a flat-memory bytecode VM
(2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong
direction (it hand-marshals onto a comptime value model that isn't bytes — exactly
what the design set out to kill). New foundation: a bytecode VM over flat memory so
comptime values are native bytes; the compiler-API then rides on it via direct memory
(no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks
cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0
strip → Phase 1 flat-memory value model → Phase 2 bytecode → Phase 3 compiler-API on
flat memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted
the session's uncommitted `register_struct`/`find_type` marshaling experiment back to
`reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action.
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
byte-blob — all explored, all unnecessary). Instead: the sx header declares
fields in the compiler type's memory order; the compiler reflects the bound Zig
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
On pass it's an ordinary byte-identical struct — cast + deref just works.
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
superseded by the reflection+validation design and deleted.
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
green (458 unit, 690 corpus).
- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).**
`compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function`
`compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`);
`interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round-
trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean
rejection deferred (loud link error today). Build + suite green (458 unit, 689
corpus).
- **Phase 1.1 fourth sub-step — welded-struct layout validation.**
`validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into
`registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against
the registry (lib == compiler, name exported, layout matches) with build-gating
diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful
Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to
Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit,
687 corpus).
- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`:
the `compiler` lib's welded-type registry; `Field` welded to
`StructInfo.Field` with layout baked from the real Zig type
(`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test
(+ null off the export list). Standalone island — not yet consumed by lowering.
Build + suite green (454 unit tests). Break-verified.
- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl`
gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern <lib>`
after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified.
Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete.
- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig)
extern <lib>` on fn decls; unified `callconv` into `abi(.c|.zig|.pure)` (removed
the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs +
snapshots. Build + suite green. The original design's `extern(.zig)` single
qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern
<lib>` (linkage + source) — recorded in the design doc's syntax-decision note.