The mutating compiler-API, minting types LAZILY at lowering time (single pass,
the existing runComptimeTypeFunc path — so the write side is legacy-only; the
VM isn't wired at lowering time, and the read-side readers stay dual-path):
declare_type(name) -> Type forward nominal handle (≈ declare)
pointer_to(t) -> Type build *T references
register_type(handle, kind, members) ONE kind-branching fill (≈ unified define)
register_type branches on kind IN THE COMPILER (subsuming define's per-kind
dispatch); codes match type_kind: 1 struct, 2 actual .@"enum", 3 tagged_union,
4 tuple. Members are {name: string, ty: Type}. A non-generic `-> Type` builder is
now flagged is_comptime (decl.zig) so its dead body permits the welded calls.
Graph support: forward declare_type handles + pointer_to express a mutually-
recursive A<->B graph (*A, *B, B-by-value) before bodies are filled. register_type
is idempotent — re-filling a nominal slot (a minting module reached via two import
edges) re-mints identically rather than erroring (nominalIdent reads identity from
any nominal kind).
Fixes (issue 0142):
- A fully payloadless comptime-minted enum was minted as an all-void tagged_union,
whose IR size disagrees with its LLVM size -> verifySizes panic. Now mints a real
.@"enum" (register_type kind 2 AND the metatype defineEnum).
- Bare `EnumType.variant` qualified construction of a payloadless variant wasn't
supported (failed for hand-written enums too — the type name lowered to a Type
value). Added in lowerFieldAccess via isPayloadlessVariant; payload-carrying
variants keep their call form.
Examples: 0631 (graph + actual enum + reflection), 0632 (make_enum all-void),
0633/0634/0635 (namespaced / bare / multi-edge import of a minted type), 0187
(qualified variant construction). Unit tests added.
Parity 697/697 (gate OFF and -Dcomptime-flat).
657 lines
48 KiB
Markdown
657 lines
48 KiB
Markdown
# CHECKPOINT-COMPILER-API — comptime `compiler` library (`#library "compiler"` + `abi(.zig) extern`)
|
||
|
||
Companion to the design-of-record
|
||
[../design/comptime-compiler-api.md](../design/comptime-compiler-api.md) (the plan
|
||
+ phased build order live there). This stream supersedes the metatype
|
||
`declare`/`define`/`type_info` `#builtin`s and the `#compiler` struct attribute
|
||
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
|
||
|
||
## ⏯ Resume (fresh session)
|
||
|
||
> **⚠ DIRECTION CHANGED (2026-06-17). The active plan is now
|
||
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md), NOT the weld.**
|
||
> The **byte-weld + serialization/marshaling** approach is the wrong direction and is
|
||
> being **stripped**. New foundation: a **bytecode VM over flat, byte-addressable
|
||
> memory** so comptime values are native bytes; then the compiler-API rides on it with
|
||
> direct memory access (no weld, no validation, no marshaling). Everything below this
|
||
> banner describes the now-superseded weld state (committed on `reify` through
|
||
> `40d075c`) and is kept only to scope the Phase 0 strip. Read
|
||
> `PLAN-COMPILER-VM.md` first.
|
||
>
|
||
> **Why the pivot:** the comptime evaluator (`src/ir/interp.zig`) represents values as
|
||
> tagged `Value` unions, NOT native bytes — so a comptime `@ptrCast(*StructInfo)`
|
||
> reads the `Value` union's memory, not a struct. The weld tried to bridge that with
|
||
> hand-marshaling — exactly what the design set out to kill. Flat memory makes comptime
|
||
> values real bytes, so the bridge disappears. (JIT-native comptime was rejected: it
|
||
> breaks cross-compilation — host vs target layout — and loses the sandbox. A
|
||
> flat-memory VM keeps both while getting native bytes + speed.)
|
||
>
|
||
> **Next action (2026-06-18):** **Phase 3 is UNDER WAY — the READ side is COMPLETE.** The VM
|
||
> hosts seven read-only reflection readers, all on the same plain-`u32`-handle shape (a
|
||
> `TypeId` is a `u32`, like `StringId`), so the calls stay clean scalar host-calls — handle
|
||
> in, scalar out, no marshaling: `find_type(name) -> TypeId`, `type_kind(t) -> i64`,
|
||
> `type_field_count(t) -> i64`, `type_nominal_name(t) -> StringId`,
|
||
> `type_field_name(t, idx) -> StringId`, `type_field_type(t, idx) -> TypeId`,
|
||
> `type_field_value(t, idx) -> i64`. Each is backed by a `TypeTable` query
|
||
> (`findByName`/`kindCode`/`memberCount`/`nominalName`/`memberName`/`memberType`/`memberValue`)
|
||
> that BOTH the legacy handler and the VM call, so the two paths can't drift. Together they
|
||
> cover everything `reflectTypeInfo` reads. Examples `0628`–`0630`, all VM-HANDLED natively.
|
||
> Parity **691/691** (gate ON and OFF), VM unit tests added. Phase 1.final op-porting was
|
||
> already complete (the VM covers scalars/control-flow/aggregates/strings/optionals/enums,
|
||
> calls+recursion, the implicit context + full allocator protocol, globals, failables +
|
||
> return traces); both comptime call sites route through the VM with legacy fallback.
|
||
> **Forward (P3.3) — revised direction:** the WRITE side is ONE `register_type(info)` fn that
|
||
> takes a type-info value and **branches on the kind in the compiler** (subsuming
|
||
> `define`'s per-kind dispatch), NOT a per-kind `register_struct`. Open design points when
|
||
> reached: the flat-memory shape of `info`, the mutable-table / host-ABI-vs-target-ABI
|
||
> boundary, pointer-escape/lifetime. Re-expressing `declare`/`define`/`type_info` as sx (the
|
||
> metatype, which runs at LOWERING time) needs the VM hardened against malformed
|
||
> lowering-time IR first — keep it on the legacy path until then. Phase 2 (bytecode) is the
|
||
> orthogonal speed work. **Decisions recorded:** `find_type` returns a non-optional `TypeId`
|
||
> using the `unresolved` (0) sentinel, NOT `?Type`; reader names use the `type_*` family to
|
||
> avoid colliding with the std metatype builtins (`field_name`/`type_name` in core.sx); the
|
||
> write side is a single kind-branching `register_type` — see `PLAN-COMPILER-VM.md` Phase 3
|
||
> progress note.
|
||
> Build/verify: `zig build && zig build test` (691, gate OFF). Run the corpus ON the VM:
|
||
> `zig build test -Dcomptime-flat` (the build flag) OR env `SX_COMPTIME_FLAT=1`. Coverage
|
||
> trace: `SX_COMPTIME_FLAT_TRACE=1`.
|
||
|
||
### (superseded) prior weld resume
|
||
Phase 1 done; Phase 2 welded structs were working via reflection + memory-order
|
||
validation (the `computeWeldPlan`/byte-blob "GEP engine" was explored + DROPPED even
|
||
earlier). A welded `Name :: struct abi(.zig) extern compiler { … }` declared fields in
|
||
the compiler type's MEMORY order; the compiler reflected the bound Zig type and
|
||
VALIDATED the header. **This whole mechanism is now being stripped — see the banner.**
|
||
|
||
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
|
||
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
|
||
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
|
||
> See CLAUDE.md → Snapshot integrity.
|
||
|
||
## Last completed step
|
||
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
|
||
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
|
||
end-to-end as a byte-identical mirror of the bound Zig type.
|
||
|
||
Design (locked, supersedes the byte-layout-override plan):
|
||
- The sx header declares fields in the compiler type's MEMORY order. The compiler
|
||
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
|
||
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
|
||
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
|
||
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
|
||
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
|
||
byte-blob, no index/remap tables, no reorder, no special LLVM path.
|
||
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
|
||
*wrong field order at position N* (+ expected memory order), *type layout
|
||
mismatch* (field size), *layout mismatch* (total size / count).
|
||
|
||
What changed from the dropped plan:
|
||
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
|
||
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
|
||
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
|
||
the sx header against the memory-ordered registry.
|
||
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
|
||
(+ `weldedFieldOrderStr`).
|
||
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
|
||
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
|
||
refreshed.
|
||
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
|
||
|
||
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
|
||
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
|
||
byte-layout-override engine; superseded by the reflection+validation design above.
|
||
|
||
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
|
||
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
|
||
padding included), so it passes to a Zig handler as raw memory with zero
|
||
marshalling. (The alternative — handlers reading interp `Value` aggregates
|
||
logically, no layout override — was rejected; welded types must also be usable as
|
||
runtime data, and the design wants the literal byte weld.)
|
||
|
||
- Measured: Zig reorders `StructInfo` to `fields`@0, `name`@16, `nominal_id`@20,
|
||
`is_protocol`@24, size 32 — vs sx-natural `name`@0, `fields`@8, … So the override
|
||
is genuinely required (`Field`'s two-u32 natural layout was the easy case).
|
||
- `compiler_lib.zig`: registered `StructInfo` (`weldStruct`, the second
|
||
`bound_types` entry). Added `WeldElement` / `WeldPlan` + `computeWeldPlan(alloc,
|
||
fields, total)` — pure: orders fields by ascending byte offset, inserts padding
|
||
elements for gaps + the alignment tail, and builds the sx-field → LLVM-element
|
||
remap. This is what the LLVM type builder + struct-GEP sites will consume.
|
||
- Unit-tested (`compiler_lib.test.zig`): `Field` → identity plan (2 elems, no pad);
|
||
`StructInfo` → 5 elems `[fields@0, name@16, nominal_id@20, is_protocol@24,
|
||
pad@25..32]`, remap `[1,0,3,2]`.
|
||
- `zig build` + `zig build test` green.
|
||
|
||
### Earlier — Phase 1 polish (comptime-only enforcement)
|
||
**A RUNTIME call to a `fn abi(.zig) extern compiler` is a clean build-gating error
|
||
instead of an undefined-symbol link failure.**
|
||
- `emitCall` (`src/backend/llvm/ops.zig`): when the callee is `compiler_welded`
|
||
AND the ENCLOSING function is not `is_comptime` (i.e. genuine runtime code, not a
|
||
`#run`/`::` initializer wrapper whose LLVM body is dead), print a clear
|
||
"comptime-only … cannot be called at runtime" error and set
|
||
`comptime_failed` (the driver halts before object/JIT emission). The enclosing
|
||
`is_comptime` guard is what keeps the legitimate `#run` use (example 0626) green.
|
||
- Corpus: `examples/1185-diagnostics-weld-fn-runtime-call.sx` (runtime `intern(…)`
|
||
→ clean error, exit 1, no link failure).
|
||
- `zig build` + `zig build test` green (458 unit + 690 corpus).
|
||
|
||
### Earlier — fifth sub-step (host-call bridge)
|
||
**A `fn abi(.zig) extern compiler` dispatches, under the comptime interpreter, to
|
||
its registered Zig handler instead of dlsym.**
|
||
- `compiler_lib.zig`: function registry — `BoundFn { sx_name, handler }`,
|
||
`bound_fns` = `intern(string)->StringId` + `text_of(StringId)->string` (the
|
||
string-pool round-trip), `findFn`, and `FnHandler` (`*Interpreter, []Value ->
|
||
Value`). `intern` mutates via `interp.mint orelse @constCast(&module.types)`
|
||
(the same mutable-table access the metatype mint path uses); `text_of` reads the
|
||
const pool. Imports `interp.zig` (the compiler_hooks↔interp cycle pattern).
|
||
- IR `Function` gained `compiler_welded: bool`. `declareFunction`
|
||
(`src/ir/lower/decl.zig`) sets it via `weldedCompilerFn`, which also VALIDATES:
|
||
the bound lib must be `compiler` and the name must be on the function-export
|
||
list — else a build-gating `.err` (no silent fall-through to dlsym).
|
||
- `interp.call()`: before the dlsym/extern path, a `compiler_welded` function
|
||
routes to `compiler_lib.findFn(name).handler(self, args)` (clean bail off the
|
||
export list).
|
||
- Corpus: `examples/0626-comptime-weld-fn-intern-text-of.sx` (`#run
|
||
text_of(intern("hello, compiler"))` folds to a string constant → prints it);
|
||
`examples/1184-diagnostics-weld-fn-unexported.sx` (unexported welded-fn name →
|
||
build error). `findFn` lookup unit-tested.
|
||
- **Runtime-call rejection is NOT yet clean** — welded fns are comptime-only; a
|
||
RUNTIME call would emit a reference to a non-existent extern symbol → a loud
|
||
LINK error (not silent, but not a tidy diagnostic). The examples call welded fns
|
||
only inside `#run`. A dedicated "comptime-only symbol" emit diagnostic is the
|
||
immediate follow-up.
|
||
- `zig build` + `zig build test` green (458 unit tests + 689 corpus).
|
||
|
||
### Earlier — fourth sub-step (welded-struct layout validation)
|
||
**A `struct abi(.zig) extern compiler { … }` is validated against the binding
|
||
registry as a *header checked against the implementation*.**
|
||
- `compiler_lib.zig`: `validateStructLayout(bt, sx_fields, total)` — pure, returns
|
||
the first `LayoutMismatch` (field count / name / size / total) or null. Plus
|
||
`lib_name = "compiler"` and `SxField`. Unit-tested (faithful `Field` passes;
|
||
each drift flagged as the right variant).
|
||
- `registerStructDecl` (`src/ir/lower/nominal.zig`): for `sd.abi == .zig`,
|
||
`validateWeldedStruct` checks the bound lib is `compiler`, the name is on the
|
||
export list (`findType`), and the sx layout (field names + `typeSizeBytes` +
|
||
total) matches the welded type — emitting a build-gating `.err` (good span into
|
||
the struct body) on any failure. No silent reinterpretation.
|
||
- `#library "compiler"` is the comptime-only internal surface, NOT a dylib —
|
||
`src/main.zig`'s dlopen walker skips it (was emitting a spurious `libcompiler.so`
|
||
load warning).
|
||
- Corpus: `examples/0625-comptime-weld-struct-field.sx` (faithful `Field` welds,
|
||
validates, usable as data → `name=7 ty=3`); `examples/1183-diagnostics-weld-
|
||
struct-field-count.sx` (one-field `Field` → build-gating field-count diagnostic).
|
||
- **Offset-override / GEP emission for non-natural Zig layouts is NOT here** — it
|
||
isn't exercised by `Field` (two u32s = natural layout coincides with the weld).
|
||
It arrives with `StructInfo` in Phase 2 (slices/reordering), where the bound
|
||
offsets actually differ from the sx-natural ones. The validation already checks
|
||
per-field size + total, so a layout drift is caught even before the override
|
||
engine exists.
|
||
- `zig build` + `zig build test` green (456 unit tests + 687 corpus).
|
||
|
||
### Earlier — third sub-step (binding registry)
|
||
**The binding registry (welded-type lookup, layout baked from the real Zig
|
||
type).**
|
||
- New `src/ir/compiler_lib.zig` — the `compiler` library's binding registry, the
|
||
curated safety boundary. `BoundType { sx_name, size, alignment, fields:
|
||
[]FieldLayout{name, offset, size} }`; `weldStruct` bakes the layout from a real
|
||
Zig struct via `@sizeOf`/`@alignOf`/`@offsetOf` at compiler-build time (a
|
||
sx-field-count mismatch is a `@compileError`, never a silent truncation).
|
||
`bound_types` exports `Field` (welded to `types.TypeInfo.StructInfo.Field` —
|
||
two `u32`s); `findType(sx_name) ?*const BoundType` is the lookup the welded-decl
|
||
resolution path will consult (returns null off the export list — clean boundary,
|
||
no silent default).
|
||
- Registered in the barrel (`src/ir/ir.zig`): `compiler_lib` + `compiler_lib_tests`.
|
||
- Tests (`src/ir/compiler_lib.test.zig`): `findType("Field")` equals the real
|
||
`StructInfo.Field` `@sizeOf`/`@alignOf`/`@offsetOf` (8 bytes, two u32s at 0/4);
|
||
an unexported name returns null. Break-verified (a wrong size → suite red,
|
||
named `ir.compiler_lib.test...`).
|
||
- `zig build` + `zig build test` green (454 unit tests).
|
||
|
||
### Earlier — second sub-step (struct-decl parse)
|
||
**`abi(.zig) extern <lib>` PARSES on a STRUCT decl (parse-only, no semantics).**
|
||
- `ast.StructDecl` gained `abi: ABI` + `extern_lib: ?[]const u8` binding fields.
|
||
- `parseStructDecl` (`src/parser.zig`): after `struct` (and the `#compiler`
|
||
check), parse an optional `abi(...)` then optional `extern <lib>` — same slot
|
||
order as fn decls — and thread them onto the node. Ordinary structs are
|
||
unperturbed (`parseOptionalAbi`/`parseOptionalExternExport` no-op when absent).
|
||
- Parser unit tests (`src/parser.test.zig`): `Field :: struct abi(.zig) extern
|
||
compiler { name: StringId; ty: Type; }` parses with `abi == .zig`, `extern_lib
|
||
== "compiler"`, field list intact; a plain struct leaves `abi == .default` /
|
||
`extern_lib == null`. Break-verified (a wrong-sentinel assert turns the suite
|
||
red, confirming the test runs).
|
||
- `zig build` + `zig build test` green.
|
||
|
||
### Earlier — first sub-step (fn decls) + the syntax pivot
|
||
**`abi(.zig) extern <lib>` PARSES on a fn decl (parse-only).** Plus the syntax
|
||
pivot it required.
|
||
|
||
Syntax decision (locked 2026-06-17, supersedes the doc's original
|
||
`extern(.zig) <lib>` single-qualifier form): the ABI/layout selector and the
|
||
linkage keyword are two orthogonal annotations.
|
||
- `abi(.x)` — ABI / calling-convention annotation in the slot **before**
|
||
`extern`/`export`. **Unified replacement for `callconv(...)`, which is removed.**
|
||
`ABI = { default, c, zig, pure }`: `.c` (C ABI), `.zig` (Zig-layout weld → the
|
||
`compiler` library), `.pure` (naked asm), `.default` (unannotated). Can appear
|
||
standalone (no extern) on any fn / fn-type / lambda.
|
||
- `extern <lib>` — linkage keyword + binding source (named library).
|
||
|
||
So a welded binding is `text_of :: (id: StringId) -> string abi(.zig) extern compiler;`.
|
||
|
||
What landed:
|
||
- **AST** (`src/ast.zig`): `CallingConvention` → `ABI { default, c, zig, pure }`;
|
||
the `call_conv` field → `abi: ABI` on `FnDecl` / `Lambda` / `FunctionTypeExpr`.
|
||
- **Lexer/token** (`src/token.zig`, `src/lexer.zig`): `kw_callconv` → `kw_abi`,
|
||
keyword string `"callconv"` → `"abi"`.
|
||
- **Parser** (`src/parser.zig`): `parseOptionalCallConv` → `parseOptionalAbi`
|
||
(parses `abi(.c|.zig|.pure)`); wired in the fn-decl postfix slot (before
|
||
`extern`/`export`), the function-type-expr slot, and the lambda slot;
|
||
`isFunctionDef`/`hasFnBodyAfterArrow` recognise `kw_abi`.
|
||
- **AST→IR map** (`src/ir/type_resolver.zig`, `src/ir/lower/decl.zig`, `sema.zig`,
|
||
`closure.zig`): the AST `.abi == .c` reads kept their C-ABI meaning; the
|
||
function-type resolver maps `.zig`/`.pure` → IR `.default` (no fn-pointer-type
|
||
CC for those decl-level ABIs; neither occurs in a function-TYPE position yet).
|
||
- **CC-mismatch diagnostic** (`src/ir/lower/expr.zig`, `src/sema.zig`): the
|
||
user-facing text `callconv(.c)` → `abi(.c)`.
|
||
- **sx migration**: 52 `.sx` files `callconv(` → `abi(` (all were function-type
|
||
callback annotations — none in the fn-decl postfix slot, so no reordering).
|
||
- **Docs**: `readme.md`, `specs.md`, the design doc, snapshots (0114 / 1104 /
|
||
1200) regenerated for the rename.
|
||
- **Tests**: parser unit tests in `src/parser.test.zig` — `abi(.zig) extern <lib>`
|
||
on a fn decl (asserts `abi == .zig`, `extern_export == .extern_`, `extern_lib ==
|
||
"compiler"`); bare `extern` leaves `abi == .default`; standalone `abi(.c)` /
|
||
`abi(.pure)`. lexer/sema tests updated.
|
||
|
||
`zig build` + `zig build test` green (450/450 unit + 685 corpus).
|
||
|
||
## Current state
|
||
|
||
> **Pivoted — see the banner + `PLAN-COMPILER-VM.md`.** The items below are the weld
|
||
> machinery as it stands on `reify` HEAD (`40d075c`); they are the **strip list** for
|
||
> Phase 0, not the forward direction. The `#library`/`abi`/`extern` *syntax* stays; the
|
||
> weld *semantics* (layout reflection/validation, marshaling dispatch) go.
|
||
|
||
- `compiler :: #library "compiler";` parses + is recognised as the comptime-only
|
||
internal surface (never dlopen'd).
|
||
- `abi(.zig) extern compiler` STRUCTS: layout-validated against the registry
|
||
(faithful → ok; drift → build-gating diagnostic). `Field` welds + usable.
|
||
- `abi(.zig) extern compiler` FUNCTIONS: dispatched under the comptime interp to
|
||
their registered Zig handler (`intern`/`text_of` round-trip works); unexported
|
||
names rejected at declaration. Comptime-only.
|
||
- A RUNTIME call to a welded fn is a clean build-gating error (comptime-only
|
||
enforcement at `emitCall`); the legitimate `#run`/`::` use stays green.
|
||
- The whole Phase 1 foundation (parse → registry → struct-layout validation →
|
||
function host-call bridge → comptime-only enforcement) is in place for the
|
||
two-u32 `Field` case + the two string readers.
|
||
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
|
||
(needed by `StructInfo`'s slice field, Phase 2).
|
||
|
||
## Next step — execute `PLAN-COMPILER-VM.md`
|
||
|
||
> The weld is being stripped. The next step is **Phase 0 of
|
||
> [`PLAN-COMPILER-VM.md`](PLAN-COMPILER-VM.md)** — remove the weld / serialize /
|
||
> marshal machinery (`compiler_lib.zig` reflection+validation, `nominal.zig`
|
||
> `validateWeldedStruct`, the `compiler_welded` dispatch, the weld examples/diagnostics
|
||
> 0625/0627/1183/1184/1185/1186), keeping the `#library`/`abi`/`extern` *syntax*. Then
|
||
> Phase 1 (flat-memory value model). The weld-era "next step" below is **obsolete** —
|
||
> kept only as a record of what the weld surface was about to do.
|
||
|
||
### (obsolete) weld-era next step
|
||
Welded structs were byte-identical mirrors, so the API surface was set to grow:
|
||
|
||
- **Bind `register_struct` / `find_type`** over the host-call bridge
|
||
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
|
||
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
|
||
kind well-formedness — the checks `define` does today). Because the welded
|
||
`StructInfo` is byte-identical, the handler can read it as the real Zig
|
||
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
|
||
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
|
||
Prove: build a struct programmatically + round-trip a source one.
|
||
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
|
||
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
|
||
(`defineStruct` / the `reflectTypeInfo` struct path).
|
||
|
||
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
|
||
(optional fields → sentinels) — each just needs an sx header in the compiler
|
||
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
|
||
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
|
||
`#compiler`.
|
||
|
||
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
|
||
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
|
||
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
|
||
when reached (sentinels or accessor fns; see the design doc Risks).
|
||
|
||
## Known issues
|
||
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
|
||
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
|
||
|
||
## Log
|
||
- **Phase 3 P3.3 (VM plan) — WRITE side: declare_type + pointer_to + ONE kind-branching register_type (2026-06-18).**
|
||
The mutating compiler-API: `declare_type(name) -> Type` (forward handle), `pointer_to(t) -> Type`
|
||
(build `*T`), and `register_type(handle, kind, members: []Member) -> Type` which branches on
|
||
`kind` IN THE COMPILER (subsuming define's per-kind dispatch). Take/return real `Type` values
|
||
(matching meta.sx declare/define). **Timing (per user): mint LAZILY at lowering time, single
|
||
pass** (the existing `runComptimeTypeFunc`), so the write side is **legacy-only** (`compiler_lib`
|
||
handlers) — the VM isn't wired at lowering time, no VM mirror needed; readers stay dual-path.
|
||
A non-generic `-> Type` builder is now flagged `is_comptime` (decl.zig) so its dead body permits
|
||
the welded calls. **Graph:** forward handles + `pointer_to` express mutually-recursive A↔B (`*A`,
|
||
`*B`, B-by-value); `register_type` is **idempotent** (re-fill a nominal slot reached via two
|
||
import edges — `nominalIdent`). `kind` codes match `type_kind` (1 struct · 2 actual `.@"enum"` ·
|
||
3 tagged_union · 4 tuple). **Fixed two bugs (issue 0142):** (a) a fully payloadless minted enum
|
||
was an all-void tagged_union → verifySizes panic; now a real `.@"enum"` (register_type kind 2 AND
|
||
metatype `defineEnum`); (b) bare `EnumType.variant` payloadless qualified construction wasn't
|
||
supported (failed for hand-written enums too) — added in `lowerFieldAccess` (`isPayloadlessVariant`).
|
||
Examples 0631 (graph + actual enum + reflection), 0632 (make_enum all-void), 0633/0634/0635
|
||
(namespaced / bare / multi-edge import of a minted type), 0187 (qualified variant construction).
|
||
**Parity 697/697** (gate ON and OFF); unit tests added. **Next (P3.4):** re-express
|
||
declare/define/type_info as sx over the compiler-API + delete the bespoke interp arms (needs the
|
||
VM hardened for lowering-time IR, or the metatype migrated onto the legacy compiler-API calls).
|
||
- **Phase 3 P3.2b (VM plan) — kind + enum-value readers: `type_kind` + `type_field_value`; READ side complete (2026-06-18).**
|
||
The last two read-only readers the metatype's `type_info(T)` needs (added to
|
||
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`, each backed by a `TypeTable` query both
|
||
call): `type_kind(t) -> i64` (`kindCode` — a stable, compiler-owned discriminant: 0 other ·
|
||
1 struct · 2 enum · 3 tagged_union · 4 tuple · 5 union · 6 array · 7 vector · 8 error_set;
|
||
TOTAL, never bails) and `type_field_value(t, idx) -> i64` (`memberValue` — an enum variant's
|
||
explicit value or ordinal; mirrors the `field_value_int` builtin; loud-bail for non-enum /
|
||
out-of-range). Example `0630-comptime-compiler-type-kind` reflects `Color` / `WindowFlags`
|
||
(flags) / `Point`. **The READ side is now COMPLETE** — `find_type` + `type_kind` +
|
||
`type_field_count` + `type_field_name`/`type_field_type`/`type_nominal_name` +
|
||
`type_field_value` cover everything `reflectTypeInfo` reads. VM unit test added. **Parity
|
||
691/691** (gate ON and OFF). **Revised forward direction (per the user):** the WRITE side is
|
||
ONE `register_type(info)` fn that branches on the kind IN THE COMPILER (subsuming `define`'s
|
||
per-kind dispatch), not a per-kind `register_struct`.
|
||
- **Phase 3 P3.2 (VM plan) — field-level reflection readers: `type_nominal_name` + `type_field_name` + `type_field_type` (2026-06-18).**
|
||
Three more `compiler`-library readers on the same `TypeId`-handle shape (added to
|
||
`compiler_lib.bound_fns` AND `Vm.callCompilerFn`), each backed by a new `TypeTable` query
|
||
BOTH paths call (no drift): `nominalName` (a named type's own name handle; loud-bail for
|
||
unnamed types like `i64`/pointers), `memberName` (struct/union/tagged-union field, enum
|
||
variant, named-tuple element), `memberType` (struct/tuple/array/vector member type). All
|
||
loud-bail on out-of-range idx / no-member (no silent default). First MULTI-ARG compiler
|
||
fns — `callCompilerFn` reads arg 1 = idx; added `Vm.argHandle`/`argTypeId` (range-checked
|
||
u32/TypeId arg reads) and refactored `find_type`/`type_field_count` onto them. Named
|
||
`type_*` to avoid clashing with the std metatype builtins (`field_name`/`type_name` exist
|
||
in core.sx); `nominalName` (the TypeTable method) is distinct from the existing
|
||
`typeName(id) []const u8` display-string renderer. Example `0629-comptime-compiler-field-reflect`
|
||
reflects `Pair { lo: Point; hi: Point }` — each field name + the nominal name of a field's
|
||
type, all `#run`-folded, all VM-HANDLED natively. VM unit test added (type_field_name → "hi";
|
||
type_nominal_name(type_field_type(Pair,0)) → "Point"). **Parity 690/690** (gate ON and OFF).
|
||
- **Phase 3 P3.1 (VM plan) — first read-only reflection readers: `find_type` + `type_field_count` (2026-06-18).**
|
||
Two more `compiler`-library fns, bound the same way as the `intern`/`text_of` seed (added
|
||
to `compiler_lib.bound_fns` for the legacy handler + the welded-decl export check, AND to
|
||
`Vm.callCompilerFn` for the native flat-memory path — NO marshaling). A **type handle is a
|
||
plain `u32` `TypeId`** (like `StringId`), so both keep the seed's clean scalar shape:
|
||
`find_type(name: StringId) -> TypeId` (`TypeTable.findByName`, `unresolved`/0 if absent) and
|
||
`type_field_count(t: TypeId) -> i64` (a NEW `TypeTable.memberCount` query — struct/union/
|
||
tagged-union fields, enum variants, array/vector length — called by BOTH paths so they
|
||
can't drift; bails loudly, never a silent 0). New example `0628-comptime-compiler-find-type`
|
||
chains `intern → find_type → type_field_count` (and a not-found lookup → 0), both folded at
|
||
`#run`, both VM-HANDLED natively (trace confirms no fallback). VM unit test added
|
||
(`find_type` + `type_field_count`, struct found → 3 fields, missing → `unresolved`).
|
||
**Parity 689/689** (gate ON and OFF). **Decision (resolves the plan's `find_type → ?Type`
|
||
sketch):** return a NON-optional `TypeId` with the `unresolved` (0) sentinel for not-found,
|
||
NOT `?Type` — a `Type` value resolves to `.any` (which the flat-memory VM doesn't represent)
|
||
and an optional can't cross the legacy↔VM eval boundary; `unresolved` is the project-blessed
|
||
unmistakable "no type" marker. Forward (P3.2): more readers on the same handle shape
|
||
(`type_name`/`field_name`/`field_type`/kind), then `register_struct` (first mutating fn).
|
||
- **VM robustness — `Frame` bounds-check; lowering-time `#insert` wiring explored + reverted (2026-06-18).**
|
||
Explored wiring the VM at the LOWERING-time comptime site (`evalComptimeString`, the
|
||
`#insert` string fold). 12/13 `#insert` examples ran on the VM with parity, but `0737`
|
||
(an `#insert` of an unresolved `secret()`) CRASHED the VM (SIGABRT): lowering-time IR can
|
||
be malformed (a `ret Ref.none` from the unresolved name) and `Frame.get` panicked on the
|
||
out-of-range index. **Decision: reverted the lowering-time wiring** — unlike the emit-time
|
||
folds (fully lowered IR), lowering-time IR can be erroneous, and hardening the VM against
|
||
ALL malformed IR (every `ref_types[...]` / `aggType` access, not just `Frame`) is out of
|
||
scope here. The emit-time sites already give full corpus coverage. **KEPT** the defensive
|
||
fix regardless (CLAUDE.md "never crash"): `Frame.get`/`set` now bounds-check and flip a
|
||
`bad_ref` flag; the `run` loop bails (`badRef`) instead of panicking. Unit test added
|
||
(malformed `ret Ref.none` → bail, not crash). Parity **688/688** both ways.
|
||
- **Phase 3 SEED (VM plan) — compiler-call path: `intern`/`text_of` native on the VM (2026-06-18).**
|
||
`invoke` now dispatches a welded `compiler`-library fn (gated on `compiler_welded`) to
|
||
`Vm.callCompilerFn`, serviced NATIVELY on flat memory (no legacy `Interpreter`):
|
||
`intern(string)->StringId` reads the flat-memory string bytes and `internString`s into the
|
||
const-cast table (pool-only — doesn't touch type layout, so cached sizes stay valid);
|
||
`text_of(StringId)->string` materializes the pooled text back into flat memory. Unlocked
|
||
`0626`; the ONLY remaining const-init fallback is now the inline-asm global (`1654`).
|
||
Parity **688/688** (gate ON and OFF); unit test added. This is the mechanism Phase 3 grows
|
||
— the next compiler fns (`find_type`, `register_struct`, reflection readers) bind the same
|
||
way (flat-memory pointer in, handle/pointer out, no marshaling).
|
||
- **Phase 1.final step 9 (VM plan) — `-Dcomptime-flat` build flag (the "swap behind a build flag" step) (2026-06-18).**
|
||
Added the `-Dcomptime-flat` build option (build.zig → a `build_opts` options module on
|
||
`mod`; `emit_llvm.init` reads `build_opts.comptime_flat or SX_COMPTIME_FLAT env`). This is
|
||
the plan's "reach parity → swap behind a build flag → delete the old path" mechanism.
|
||
`zig build test -Dcomptime-flat` runs the FULL corpus on the VM (688/0). Verified the flag
|
||
toggles the binary: flag-built `sx` reports VM HANDLED with no env var; default-built does
|
||
not. Default OFF — `zig build test` unchanged (688/0). Env var still works for ad-hoc runs.
|
||
Next (forward): Phase 2 (bytecode) / Phase 3 (compiler-API on flat memory); eventual
|
||
default-flip + legacy deletion.
|
||
- **Phase 1.final step 8 (VM plan) — wire the `#run` side-effect path + trace-clear-on-fallback (2026-06-18).**
|
||
Wired the SECOND comptime call site (`runComptimeSideEffects`, top-level `#run <expr>;`)
|
||
through `tryEval` with legacy fallback, mirroring the const-init fold. `tryEval` now
|
||
handles void/noreturn entries (→ `.void_val`) so a void side-effect doesn't bail at the
|
||
result conversion. **Fixed a trace-corruption** the new site exposed (`1035`): a
|
||
side-effect that pushes return-trace frames and then bails (e.g. on `print`) had the
|
||
legacy re-run DOUBLE-push them (`sx_trace_push` is a side effect on the shared buffer).
|
||
Both wiring sites now `sx_trace_clear()` right before the legacy fallback, discarding the
|
||
VM's partial pushes. **Parity 688/688** (gate ON and OFF). Most side-effects still bail
|
||
(print/global_addr/call_builtin) → legacy, but the path is now uniform. All comptime
|
||
evaluation routes through the VM-with-fallback.
|
||
- **Phase 1.final step 7 (VM plan) — is_comptime + failable/error cluster + signed-load fix; coverage 31→36 (2026-06-18).**
|
||
`is_comptime` → 1 (unlocked `1030`). Ported the failable/error-channel cluster (`1037`
|
||
escape, `1038` handled): `kindOf(error_set)→word`, `regToValue` bridges TUPLES (the
|
||
failable `(value…,tag)` shape `checkComptimeFailable` reads), `trace_frame` packs
|
||
`(func_id<<32|span.start)` from a new `call_stack` (pushed by invoke/runEntry), and
|
||
`sx_trace_push`/`sx_trace_clear` serviced NATIVELY (the VM calls the real sx_trace.c
|
||
functions linked into the compiler, so the return-trace buffer is populated identically
|
||
to legacy). raise/catch/or now run on the VM. **Surfaced + fixed a real GENERAL bug:**
|
||
`readField` was ZERO-extending signed sub-64-bit loads, so a stored `i32 -1` reloaded as
|
||
`0xFFFFFFFF` (+4.29e9) and `< 0` was false — silently hiding `raise error.Bad`; now
|
||
SIGN-extends `i8`/`i16`/`i32`/`isize` (gate-ON parity confirms it's a strict fix; unit
|
||
test added). VM HANDLES **36** corpus const-inits (was 31); **parity 688/688** (gate ON
|
||
and OFF). Only **2 fallbacks** remain, both principled: `intern` (`0626`, welded
|
||
compiler-API fn — Phase 3) + inline-asm global (`1654`). Forward work: Phase 2 (bytecode),
|
||
Phase 3 (compiler-API on flat memory).
|
||
- **Phase 1.final step 6 (VM plan) — real default context + call_indirect + func_ref + global_get; coverage 27→31 (2026-06-17).**
|
||
Per the user's direction ("the VM can set up a default context"), `runEntry` now
|
||
materializes the REAL default context instead of a zeroed one. The implicit-ctx param is
|
||
an opaque `*void`, so `materializeDefaultContext` finds the `__sx_default_context` global
|
||
and lays its initializer (`{ {null, alloc_fn, dealloc_fn}, null }`, the CAllocator thunk
|
||
func-refs) into flat memory via a new recursive `layoutConst`. With `func_ref` (function
|
||
value encoded `FuncId.index()+1`, reserving word 0 for the null fn-ptr) and
|
||
`call_indirect` (decode word → FuncId → dispatch; 0 → bail) ported, the whole allocator
|
||
protocol runs on the VM:
|
||
`context.allocator.alloc_bytes` → call_indirect → thunk → `CAllocator.alloc_bytes` →
|
||
`libc_malloc` → native flat malloc. Unlocked `0606` (string global). Also: `global_get`
|
||
lazily evaluates a comptime global's `comptime_func` (memoized) — unlocked `CT_CHAIN`;
|
||
field access (`fieldOffset`/`struct_get`) handles string/slice `{ptr@0,len@8}` fat
|
||
pointers (needed by `alloc_string`); `regToValue` maps function-typed words → `.func_ref`
|
||
(kept `1128`'s rejection byte-identical). Native `malloc` is still required (the thunk
|
||
bottoms out at it; a host pointer can't be used with flat-memory load/store). VM HANDLES
|
||
**31** corpus const-inits (was 27); **parity 688/688** (gate ON and OFF). Unit tests:
|
||
global_get, func_ref+call_indirect. Remaining fallbacks (7): `.unsupported` aggregates
|
||
(3× — `1037`/`1038`), extern/builtin `intern`+asm (2×), `trace_frame`, `is_comptime`.
|
||
- **Phase 1.final step 5 cont. (VM plan) — libc memory builtins + f32 fix; coverage 16→27 (2026-06-17).**
|
||
Identified the dominant fallback (`call to extern/builtin`) as **11× `malloc`** (0604) +
|
||
1× `intern`. Modeled a curated set of libc MEMORY builtins natively on flat memory
|
||
(`Vm.callMemBuiltin`): `malloc`/`calloc` → `allocBytes` (16-aligned, 256-MiB cap → bail),
|
||
`free` → no-op, `memcpy`/`memmove`/`memset` on flat bytes — sandboxed (no host heap/dlsym),
|
||
target-aware; the computed result is byte-identical to legacy (which calls real libc).
|
||
This surfaced a **real latent f32 bug**: float registers hold f64 bits, but f32 MEMORY is
|
||
the 4-byte single — `readField`/`writeField` were truncating the f64 bits (writing zeros
|
||
for `1.0`); now they `@floatCast` on f32 load/store (mirrors legacy `storeAtRawPtr`).
|
||
Result: VM HANDLES **27** corpus const-inits (was 16); **parity 688/688** (gate ON and
|
||
OFF). Unit tests added (f32 round-trip; malloc → usable flat memory). Next: the `kindOf`
|
||
`.unsupported` aggregates (3×), `global_get` (2×), the rest.
|
||
- **Phase 1.final step 5 (VM plan) — implicit-context materialization; coverage 0→16 (2026-06-17).**
|
||
`tryEval` now MATERIALIZES the implicit ctx instead of skipping it: a `has_implicit_ctx`
|
||
comptime entry (sole param `*Context`) gets a zeroed `Context` of the right size/align
|
||
in flat memory, its address passed as arg 0. Const bodies that ignore the ctx run; a
|
||
body that uses the allocator hits unported `call_indirect` → bails → legacy. No func-ref
|
||
materialization needed (handled bodies don't read ctx contents; parity is the guard).
|
||
Fixed a real bug surfaced by the coverage pass: storing a `null` non-pointer optional
|
||
(the `null_addr` sentinel) into an aggregate slot OOB-bailed — `writeField` now ZEROES
|
||
the destination for a `null_addr` aggregate source (= none/empty); unit-test regression
|
||
added. Result: VM HANDLES **16** corpus const-inits (was 0); **parity 688/688 both
|
||
gate ON and OFF**. Next: port the ops the trace names — `call_builtin`/`compiler_call`/
|
||
extern (13×, via the bridge), `kindOf` `.unsupported` aggregates (3×), `global_get` (2×),
|
||
func_ref / call_indirect / trace_frame / is_comptime.
|
||
- **Phase 1.final steps 1–4 (VM plan) — host wiring landed; coverage measured (2026-06-17).**
|
||
(1) **Hardening:** `Machine.readWord`/`writeWord`/`bytes` now return `error.OutOfBounds`
|
||
(null / out-of-range / oversized / overflow-safe) instead of `assert`-panicking;
|
||
`OutOfBounds` added to `Vm.Error`; `try` threaded through every helper + exec arm + the
|
||
bridge. New unit tests (accessor OOB returns; null-deref → `tryEval` null, not a crash).
|
||
(2) **Implicit context:** `tryEval` returns null for `has_implicit_ctx` funcs (legacy
|
||
fallback) — conservative; full ctx materialization deferred to step 5. (3) **Wiring:**
|
||
const-init fold in `emit_llvm.zig` `emitGlobals` is `(if comptime_flat) tryEval else
|
||
null) orelse interp.call(...)`, gated by env `SX_COMPTIME_FLAT` (read once into
|
||
`LLVMEmitter.comptime_flat`). Default OFF. (4) **Parity + coverage:** gate ON → full
|
||
corpus byte-identical (688, 0 failed) + manual 0605/0606/0607 byte-identical.
|
||
**Finding: 0 of 37 measured corpus const-inits are VM-handled — ALL are
|
||
`has_implicit_ctx`-gated.** Added a coverage-trace facility (`comptime_vm.last_bail_reason`
|
||
+ env `SX_COMPTIME_FLAT_TRACE`). **Next: step 5 = implicit-context materialization** (the
|
||
unblocker), then port the deferred ops. 688 corpus green (gate OFF).
|
||
- **Phase 1.final start (VM plan) — wiring entry point `tryEval` (2026-06-17).**
|
||
`comptime_vm.tryEval(gpa, module, func_id) ?Value` runs a comptime function entirely on
|
||
the VM, returns a legacy `Value` (deep-copied to `gpa`) or `null` to fall back.
|
||
Unit-tested (pure 6*7 → 42; unbox_any → null). NOT yet routed into the host: needs
|
||
(1) panic→error hardening of `Machine` accessors so arbitrary funcs bail instead of
|
||
crashing, (2) implicit-ctx handling, (3) wiring at `emit_llvm` const-init behind
|
||
`SX_COMPTIME_FLAT`, (4) corpus parity run. See `PLAN-COMPILER-VM.md` Phase 1.final.
|
||
688 corpus green.
|
||
- **Phase 1 sub-step 1.5b (VM plan) — Reg↔Value boundary bridge (2026-06-17).**
|
||
Builtin/compiler_call/extern handlers are coupled to the legacy `Interpreter`, so the
|
||
wiring will use WHOLE-FUNCTION fallback (VM runs pure functions; bail → legacy re-runs
|
||
the whole eval). Built the boundary bridge that enables it: `valueToReg` (Value arg →
|
||
Reg, aggregates into flat memory) + `regToValue` (VM result → Value, deep-copied).
|
||
Covers scalars/strings/structs; other shapes bail. Transitional. Round-trip
|
||
unit-tested. 688 corpus green. Next: the wiring (flag + route a comptime entry through
|
||
the VM with legacy fallback).
|
||
- **Phase 1 sub-step 1.5 (VM plan) — direct `call` + stack-lifetime change (2026-06-17).**
|
||
`Vm` gained `module` (callee resolution) + `depth`/`max_depth` guard. `call` marshals
|
||
arg Refs → Reg and recursively runs the callee; aggregates pass as Addrs over shared
|
||
flat memory. `Frame` no longer reclaims the machine on exit (else a returned aggregate
|
||
Addr dangles) — allocations live to `Vm.deinit`. Extern/builtin callees bail (1.5b).
|
||
Unit-tested: direct call (142), recursion sum(0..n) (15/55). 688 corpus green. Next:
|
||
1.5b (call_builtin/compiler_call/extern), then hybrid wiring.
|
||
- **Phase 1 sub-step 4d (VM plan) — deref/addr_of; pivot decision (2026-06-17).**
|
||
Ported `addr_of` (pass-through) + `deref` (readField through pointer), unit-tested
|
||
(deref *i64 → 77, addr_of struct + field → 80). DECIDED to stop porting rarer ops
|
||
(tagged-union payload/any/closures) blind — their byte semantics are ambiguous without
|
||
real call sites — and pivot to CALLS (sub-step 1.5: `call`, then builtin/compiler) +
|
||
HYBRID WIRING (`-Dcomptime-flat` → VM with legacy fallback on `error.Unsupported`), so
|
||
the VM runs the real corpus and surfaces exactly what's needed. Key design point for
|
||
calls: aggregate-return lifetime → drop per-frame stack reclaim (let a comptime eval's
|
||
allocations live to `Vm.deinit`). 688 corpus green. See `PLAN-COMPILER-VM.md` decision
|
||
block.
|
||
- **Phase 1 sub-step 4c (VM plan) — optionals + payloadless enums (2026-06-17).**
|
||
`kindOf`: enum → word; `?T` → word (pointer-child, null==0) or `{T@0,i1@sizeof(T)}`
|
||
aggregate. Ported optional_wrap/unwrap/has_value/coalesce (`optChildIsPtr`/`optHas`;
|
||
const_null reads as none) + payloadless enum_init/enum_tag. Unit-tested (?i64 → 91,
|
||
?*i64 null==0 → 99, enum tag → 11). 688 corpus green. Next: 4d (tagged unions, any,
|
||
closures).
|
||
- **Phase 1 sub-step 4b (VM plan) — slices + strings on flat memory (2026-06-17).**
|
||
`{ptr@0(pointer_size), len@8(i64)}` fat pointers (kindOf: string/slice → aggregate).
|
||
Ported `const_string` (text+NUL + fat pointer in flat memory), `length`/`data_ptr`,
|
||
`array_to_slice`, `subslice`, index-through-slice (`elemAddr` loads `.ptr`), and
|
||
`str_eq`/`str_ne` (memcmp). Unit-tested (str length+eq/ne, array→slice index sum=23,
|
||
subslice sum=43). 688 corpus green. Next: 4c (optionals/enums/any/closures).
|
||
- **Phase 1 sub-step 4a (VM plan) — tuples + arrays on flat memory (2026-06-17).**
|
||
`kindOf` widened (tuple/array → aggregate). Ported `tuple_init`/`tuple_get`
|
||
(`tupleFieldOffset`), `index_get`/`index_gep` (`elemAddr` = base + idx*elem_size over
|
||
array/pointer/many_pointer; slice/string bases bail), `length` on array values.
|
||
Unit-tested (mixed tuple, [3]i64 index sum=42, length=3). 688 corpus green. Next:
|
||
sub-step 4b (slices/strings, then optionals/enums/any/closures).
|
||
- **Phase 1 sub-step 3 (VM plan) — memory + structs on flat memory (2026-06-17).**
|
||
`Vm` gained optional `table: *const TypeTable` (target-aware layout). Ported
|
||
`alloca`/`load`/`store` + `struct_init`/`struct_get`/`struct_gep`, laying structs out
|
||
at the table's natural offsets. Value model: scalar/pointer → register word;
|
||
struct → lives in flat memory, its value IS its address (read→addr, write→memcpy), so
|
||
nested structs compose and `struct_gep` = base+offset. `kindOf` bails loudly on
|
||
not-yet-ported types. Addr-based values survive allocator realloc. Unit-tested
|
||
(struct round-trip, alloca+gep+store+load, nested struct). 688 corpus green. Next:
|
||
sub-step 4 (arrays/slices/strings/optionals/enums/tuples/any/closures, then calls).
|
||
- **Phase 1 sub-step 2 (VM plan) — flat-memory executor: scalars + control flow
|
||
(2026-06-17).** Added `Vm` to `comptime_vm.zig`: walks the same IR `Inst` over
|
||
flat-memory frames (register `Reg` = scalar bits or `Addr`), mirroring the legacy
|
||
interp's scalar semantics (i64 wrapping/signed, f64). Ported constants, arithmetic,
|
||
comparison, logical, conversions, terminators (`br`/`cond_br`/`ret`/`ret_void`) and
|
||
`block_param`; every other op bails loudly (`error.Unsupported` + op name in
|
||
`detail`). Unit-tested on hand-built tiny IR (`Fb` builder): int add, f64 arithmetic,
|
||
cond_br selection, a block-param loop, div-by-zero + unsupported-op bails. Corpus
|
||
untouched (688 green). Next: sub-step 3 (memory + aggregates on flat memory, where
|
||
target-aware layout enters).
|
||
- **Phase 1 sub-step 1 (VM plan) — flat-memory machine substrate (2026-06-17).**
|
||
New `src/ir/comptime_vm.zig`: `Machine` (linear byte memory + bump/stack allocator
|
||
with `mark`/`reset`, scalar `readWord`/`writeWord` 1/2/4/8 LE, `bytes` views, addr 0
|
||
reserved as `null_addr`) + `Frame` (Ref-indexed register file, stack reclamation on
|
||
deinit). `Reg` = raw u64 (immediate scalar OR `Addr`). Unit-tested
|
||
(`comptime_vm.test.zig`), registered in the barrel; standalone — the legacy
|
||
interpreter stays live, corpus untouched (688 green). Next: sub-step 2 (executor +
|
||
scalar/branch ops over the same IR). Also removed the "~500 lines / split step" rule
|
||
from CLAUDE.md per request.
|
||
- **Phase 0 (VM plan) — struct-weld stripped; `intern`/`text_of` bridge kept
|
||
(2026-06-17).** Removed the struct-weld registry from `compiler_lib.zig`
|
||
(`weldStruct`/`bound_types`/`BoundType`/`FieldLayout`/`findType`/`SxField`/
|
||
`LayoutMismatch`/`validateStructLayout`), `validateWeldedStruct`/`weldedFieldOrderStr`
|
||
+ the `sd.abi == .zig` call from `nominal.zig`, the struct-weld unit tests, and
|
||
examples `0625`/`0627`/`1183`/`1186`. KEPT (decision) the `intern`/`text_of` function
|
||
host-call bridge — a clean scalar dispatch, not weld/serialize/marshal, the Phase-3
|
||
compiler-call seed — so `weldedCompilerFn`, the `compiler_welded` dispatch, the
|
||
`emitCall` comptime-only gate, the `#library`/`abi`/`extern` syntax, and examples
|
||
`0626`/`1184`/`1185` remain. `zig build test` green (688 corpus, 0 failed). Next:
|
||
Phase 1 (flat-memory value model) per `PLAN-COMPILER-VM.md`.
|
||
- **DIRECTION CHANGE — pivot off the byte-weld to a flat-memory bytecode VM
|
||
(2026-06-17).** Decided the weld + serialization/marshaling bridge is the wrong
|
||
direction (it hand-marshals onto a comptime value model that isn't bytes — exactly
|
||
what the design set out to kill). New foundation: a bytecode VM over flat memory so
|
||
comptime values are native bytes; the compiler-API then rides on it via direct memory
|
||
(no weld/validation/marshaling). JIT-native comptime was weighed and rejected (breaks
|
||
cross-compilation, loses the sandbox). Wrote `current/PLAN-COMPILER-VM.md` (Phase 0
|
||
strip → Phase 1 flat-memory value model → Phase 2 bytecode → Phase 3 compiler-API on
|
||
flat memory). Banner added to `design/comptime-compiler-api.md` (superseded). Reverted
|
||
the session's uncommitted `register_struct`/`find_type` marshaling experiment back to
|
||
`reify` HEAD (40d075c). No code stripped yet — Phase 0 is the next action.
|
||
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
|
||
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
|
||
byte-blob — all explored, all unnecessary). Instead: the sx header declares
|
||
fields in the compiler type's memory order; the compiler reflects the bound Zig
|
||
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
|
||
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
|
||
On pass it's an ordinary byte-identical struct — cast + deref just works.
|
||
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
|
||
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
|
||
superseded by the reflection+validation design and deleted.
|
||
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
|
||
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
|
||
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
|
||
green (458 unit, 690 corpus).
|
||
- **Phase 1.1 fifth sub-step — host-call bridge (welded functions).**
|
||
`compiler_lib` function registry (`intern`/`text_of`) + `findFn`; IR `Function`
|
||
`compiler_welded` flag set/validated in `declareFunction` (`weldedCompilerFn`);
|
||
`interp.call()` dispatches welded calls to the Zig handler. Examples 0626 (round-
|
||
trip) + 1184 (unexported-fn diagnostic); `findFn` unit-tested. Runtime-call clean
|
||
rejection deferred (loud link error today). Build + suite green (458 unit, 689
|
||
corpus).
|
||
- **Phase 1.1 fourth sub-step — welded-struct layout validation.**
|
||
`validateStructLayout` (pure, unit-tested) + `validateWeldedStruct` wired into
|
||
`registerStructDecl`: a `struct abi(.zig) extern compiler` is validated against
|
||
the registry (lib == compiler, name exported, layout matches) with build-gating
|
||
diagnostics. `#library "compiler"` no longer dlopen'd. Examples 0625 (faithful
|
||
Field) + 1183 (field-count mismatch diagnostic). Offset-override/GEP deferred to
|
||
Phase 2 (not exercised by Field's natural layout). Build + suite green (456 unit,
|
||
687 corpus).
|
||
- **Phase 1.1 third sub-step — binding registry.** New `src/ir/compiler_lib.zig`:
|
||
the `compiler` lib's welded-type registry; `Field` welded to
|
||
`StructInfo.Field` with layout baked from the real Zig type
|
||
(`@offsetOf`/`@sizeOf`/`@alignOf`); `findType` lookup proven by unit test
|
||
(+ null off the export list). Standalone island — not yet consumed by lowering.
|
||
Build + suite green (454 unit tests). Break-verified.
|
||
- **Phase 1.1 second sub-step — struct-decl binding parses.** `ast.StructDecl`
|
||
gained `abi` + `extern_lib`; `parseStructDecl` parses `abi(.zig) extern <lib>`
|
||
after `struct`. Parser unit tests (welded `Field` + plain struct), break-verified.
|
||
Build + suite green. Parse-only sub-step (fns + structs) of Phase 1.1 complete.
|
||
- **Phase 1.1 first sub-step + `callconv`→`abi` unification.** Parsed `abi(.zig)
|
||
extern <lib>` on fn decls; unified `callconv` into `abi(.c|.zig|.pure)` (removed
|
||
the `callconv` keyword), migrated 52 sx files + compiler diagnostics + docs +
|
||
snapshots. Build + suite green. The original design's `extern(.zig)` single
|
||
qualifier was split into `abi(.zig)` (ABI/layout, before extern) + `extern
|
||
<lib>` (linkage + source) — recorded in the design doc's syntax-decision note.
|