on_build is now the sole post-build callback mechanism. Migrated the 9 callers
(0602/0603/1611/1614/1615/1616 + the platform bundle_main) from
opts.set_post_link_callback(cb) to on_build(cb), giving each callback the
(opt: BuildOptions) param. Deleted set_post_link_callback from build.sx,
compiler_lib (bound_fns + handleSetPostLinkCallback), and the VM arm.
Reworked the P5 smoke tests for the new semantics: an on_build override REPLACES
the build (must emit+link or delegate), unlike the old post-link callback which
ran after the auto-link. 1662 (queries) + 1664 (override+List-grow) now delegate
to default_pipeline for the real build; deleted 1661/1663 (the primitives are now
exercised by every AOT build). bundle_main invoked with pass_options=true.
Benign 37-.ir churn (build.sx shrank). 703/0 both gates.
The compiler's post-IR role shrinks to: codegen -> invoke the build callback.
There is NO Zig auto-emit / auto-link anymore; emit + link are sx-called actions.
- emit_object() is now an ACTION (verify + emit via a host BuildHooks vtable),
returning the object path. New query primitives build_output/build_target/
build_frameworks/build_flags (data reads from the merged BuildConfig).
- library/modules/build.sx imports compiler.sx and defines default_pipeline:
emit_object -> gather c_object_paths -> link(objs, output, libs, fws, flags,
target). The std<->build import cycle is handled by the resolver.
- The compiler FORCE-LOWERS default_pipeline (well-known name) and AUTO-INVOKES
it post-codegen when no on_build/set_post_link_callback override was
registered (driver's final fallback: invokeByName default_pipeline).
- Prelude-less programs (e.g. asm tests) don't import build.sx, so the BUILD
path auto-imports modules/build.sx (idempotent if already transitive) so
default_pipeline is always available. JIT sx run is untouched (emits in-process).
- Removed the build cache short-circuits (incompatible with the always-run sx
driver; a future cache can live in default_pipeline).
Benign 37-.ir churn (build.sx grew); zero behavior changes (verified diff is
.ir-only). 705/0 both gates.
Per user design: on_build(build) is the build-callback registrar (a free fn),
generalizing set_post_link_callback — the callback is (opt: BuildOptions) ->
bool and the compiler invokes it post-codegen WITH the BuildOptions handle.
- VM: callCompilerFn 'on_build' arm + legacy handleOnBuild, both set
post_link_callback_fn + a new BuildConfig.post_link_takes_options flag.
- comptime_vm: runEntry refactored to runEntryArgs(extra) (implicit ctx +
explicit args); new public runBuildCallback(..., pass_options) passes the
opaque BuildOptions handle (one word) after the ctx. The fat-config
marshaling fear is moot — the handle is a single null-sentinel word.
- core.invokeByFuncId/invokeByName take pass_options (was an unused args
slice); main.zig passes comp.getPostLinkTakesOptions().
- build.sx: on_build decl (set_post_link_callback kept for now).
Smoke test examples/1664-platform-on-build-callback (AOT): #run on_build(build)
with build :: (opt: BuildOptions) -> bool; the callback is invoked with the
handle arg (runEntryArgs param-count match) and runs the primitives.
Benign .ir churn (37 snapshots: type table +1 for the on_build fn type +
global renumber; behavior identical). 705/0 both gates.
Per user direction: the low-level abi(.compiler) primitive surface is the
comptime 'compiler' library, so name the file compiler.sx (a peer of build.sx)
instead of the interim std/build.sx — which also frees the 'build' name for the
default build IMPLEMENTATION (default_build + on_build slot), which will live in
modules/build.sx alongside the BuildOptions DSL.
Updated the two example imports + the plan's Phase 5 file-split note. 704/0
both gates.
The one genuine action primitive: link(objects, output, libraries, frameworks,
flags, target) in library/modules/std/build.sx. Per the user decision to drop
fallibility from the build callback, link is plain VOID — a link failure bails
on the VM (hard build error), no -> ! / failable-tuple needed.
comptime_vm.zig can't depend on the driver (core/main/target), so link
dispatches through a new compiler_hooks.BuildHooks { ctx, link } vtable that
main.zig installs into BuildConfig.build_hooks before the post-link callback.
The driver side is main.LinkHooksCtx (unions explicit + CLI link flags, calls
target.link). New VM readers readStringList / readStringArg (inverse of
makeStringList) decode the List(string)/string args from flat memory.
Smoke test examples/1663-platform-build-pipeline-link (AOT): a post-link
callback re-links the build's own objects (c_object_paths + emit_object) into a
temp output via sx link — the relinked binary is a functional executable that
runs. Negative-probe verified (bad path -> ld fails -> ComptimeVmBail -> build
exit 1). The Zig driver still auto-links; removing that is P5.4.
704/0 both gates.
The compiler emits the sx object eagerly (the Zig driver, before the post-link
callback), so emit_object is a QUERY (not an action): it returns the path from
a new BuildConfig.object_path field main.zig forwards — no driver vtable. This
completes the build-pipeline QUERY primitives (emit_object / c_object_paths /
link_libraries); only link (the genuine action) remains for the vtable step.
Extended examples/1662 to also assert emit_object().len > 0. 703/0 both gates.
Two abi(.compiler) build-pipeline primitives the sx driver will pass to link:
- c_object_paths() -> List(string) (#import c companion objects)
- link_libraries() -> List(string) (#library names)
They live in a new stdlib home library/modules/std/build.sx and are serviced
by comptime_vm.callCompilerFn reading two new BuildConfig fields that main.zig
forwards before the post-link callback. New reusable VM helper makeStringList
builds a List(string) in flat memory from the call's result type offsets
(target-aware); invoke/callCompilerFn now thread ins.ty for that. Legacy
handlers bail loudly (VM-only by nature — post-link; List(string) isn't
faithfully buildable in the legacy Value model, 0141).
Smoke test examples/1662-platform-build-pipeline-queries (AOT + a 1-line C
#source → one object): a post-link callback verifies the VM-built list is
well-formed; build exit 0 only if so (negative-probe confirmed a real guard).
emit_object + link (the actions) deferred to P5.2b — they replace the Zig
driver's auto-emit/auto-link and need a host-installed callback vtable.
703/0 both gates.
core.invokeByFuncId routes the post-link callback through comptime_vm.tryEval
instead of the legacy Interpreter. REQUIRED because the sx build driver
allocates/grows Lists, which the legacy interp can't do at comptime (issue
0141: struct_get: base has no fields); the VM can. No fallback (a
side-effecting post-link callback can't double-execute): a VM bail is a hard
build error (comptime_vm.last_bail_reason, surfaced by printInterpBailDiag).
BuildConfig + import_sources threaded in; non-empty args rejected loudly.
flushInterpOutput deleted (VM out writes direct via host-FFI).
Smoke test examples/1661-platform-post-link-vm-list (AOT): a post-link
callback grows a List to 3 + returns len==3, so the build succeeds (exit 0)
only via the VM. First corpus coverage of the post-link path.
702/0 both gates.
Lands the full VM/compiler-API arc on branch reify (701/0 both gates):
- abi(.compiler) ABI replaces abi(.zig) extern compiler + the fake
#library "compiler"; bodiless decl = compiler-API surface, bodied =
user compiler-domain fn (lowered for VM eval, emit-skipped).
- out is a plain sx fn (libc write) — the out builtin deleted; the VM
handles it via host-FFI. trace_resolve + interp_print_frames ported.
- 4B VM-native diagnostics: 1179/1180 render proper comptime type
construction failed: under strict.
- S5a: build_options/set_post_link_callback on abi(.compiler) with
BuildConfig threaded into the VM (green intermediate).
- 0522 fixed (describe(args: []Type)); regression 0638.
Strict deletion-gate down to 4 compiler_call bails (1609/1614/1615/1616)
+ 1654 (legitimate unresolvable-symbol diagnostic).
buildPackSliceValue (lower/pack.zig) materialized a bare `$<pack>` []Type slice
as []Any (16-byte elements) — a stale mapping from before the dedicated Type
builtin (.type_value, 8 bytes) replaced Type -> .any. It stored 8-byte const_type
words into 16-byte slots, so a []Type reader (8-byte stride) read [t0, pad, t1, ...]
instead of [t0, t1, ...]. The legacy interp's tagged-Value model hid it; the
byte-accurate comptime VM exposed it (a `..$args` pack forwarded as a []Type
argument across a call read its elements shifted/garbled).
Fix: build the pack-slice array + slice as .type_value (8 bytes). Removed the
stopgap type_name .unresolved guard (379ed05) now that the root cause is fixed.
Regression test examples/0525-packs-pack-as-type-slice-arg.sx (outer(42,"hi",true)
-> inner($args: []Type) -> "i64 string bool"). 700/0 both gates; issue 0143 RESOLVED.
0143: a ..$args pack forwarded as a []Type argument across a call is backed by a
[N x Any] (16B) array but viewed as []type_value (8B) -> half-stride reads. A
lowering bug the legacy Value model masks; the byte-accurate VM exposes it. Blocks
examples/0114 on the VM. Filed per CLAUDE.md (not worked around; the type_name
.unresolved guard only makes the VM decline rather than emit garbage).
Checkpoint also records the sequencing insight: comptime `out` (print) can only
land once the fallback is removed (a print-then-bail double-prints under the legacy
re-run), so side-effecting ops + fallback-removal are the FINAL step; pure ops +
migrations land first.
Port two pure (side-effect-free) comptime ops toward the no-fallback gate:
- switch_br: multi-way branch on an i64 discriminant (enum/error tag, or a
.type_value whose word is its TypeId index) — mirrors the legacy
asInt-orelse-asTypeId switch.
- type_name(x): a Type value (.type_value word) or Any box ({tag,value}; tag ==
type_value means the boxed Type's id is in the value slot) -> table.typeName.
Guards an .unresolved TypeId (bail, not "<unresolved>") to surface a bad
slice/pack read instead of emitting garbage (see issue 0143).
These are correct in isolation (0520-0524 run green under strict mode) but flip
nothing yet because their examples also print via `out`, which can only land at
the end state: under the legacy fallback a print-then-bail double-prints (no
re-run rewind), so `out` is deferred until the fallback is removed. 699/0 both
default gates.
Add -Dcomptime-flat-strict / env SX_COMPTIME_FLAT_STRICT (implies comptime_flat):
at all three comptime sites (type-fn in lower/comptime.zig, const-init + #run in
emit_llvm.zig) a VM bail becomes a build-gating error naming the reason INSTEAD of
falling back to legacy. Forces every comptime eval onto the VM so the complete gap
set is enumerable in one sweep; when the corpus is green under strict mode AND every
example matches legacy, interp.zig can be deleted.
Default behaviour unchanged (699/0 both default gates). Fixed a wiring bug: the
type-fn site's local comptime_flat didn't include the strict flag (every type-fn
falsely reported <unknown>); strict now implies flat there too.
Swept the gap list (19 strict bails): switch_br (5, + unmasks a []Type-across-call
silent-wrong in 0114), compiler_call (6, = the BuildOptions->abi(.zig) extern
compiler migration), out (2), type_name (1), global_addr (1), interp_print_frames
(1), 2 negative-test diagnostics (1179/1180), 1 dlsym (1654). Recorded as the
deletion checklist in CHECKPOINT-COMPILER-API.md.
Direction correction (user): the VM does NOT get a transitional compiler_call →
hook-registry shim. BuildOptions is re-expressed as abi(.zig) extern compiler
functions (the compiler-API the VM already dispatches via callCompilerFn), and the
#compiler attribute + compiler_call op + Value-based hook Registry are deleted.
Also record that 4D (host FFI) is DONE via the arena/absolute-pointer allocator
(the earlier pin/tag hazard is moot — the arena never moves an allocation), and
that 0602/0603 stay on legacy fallback until the BuildOptions migration lands.
No code change (reverted the speculative compiler_call bridge).
Update the resume banner: box_any/unbox_any (4A.1), arena allocator with absolute
host-pointer Addr (4D.0), general host-FFI escape (4D.1), slice/string args + float
guards (4D.2) all landed and green (699/0 both gates). Next is 4D.3 (#compiler
hooks / compiler_call) with the hook-ABI + build_config findings recorded for a fast
restart.
Extract marshalExternArg: a scalar/pointer word passes verbatim (a cstring arg
already works as a pointer word via 4D.1); a string/slice {ptr,len} fat pointer is
copied into a NUL-terminated arena buffer and its char* passed -- mirrors the legacy
marshalExternArg, and is what the bundler's popen(cmd: [:0]u8, ...) needs.
Add float guards on args AND returns: floats are kindOf == .word but the host_ffi
trampolines have no float variant, so bail loudly rather than miscall through an
integer register (the legacy interp doesn't support float FFI either -> parity).
New example 0637-comptime-extern-slice-arg (#run strlen("hello, world") with a
[:0]u8 param -> 12) runs HANDLED on the VM, byte-matching legacy. 699/0 both gates.
The FFI escape now covers scalar/pointer/cstring/slice args + scalar/pointer returns.
Replace the "extern not ported -> bail" stub in Vm.invoke with callHostExtern:
resolve the symbol via host_ffi.lookupSymbol (dlsym RTLD_DEFAULT) and dispatch
through the host_ffi trampolines, like the legacy interp.callExtern.
Marshalling is trivial now that Addr is a real host pointer (4D.0): every WORD-kind
arg passes as usize verbatim (a scalar's bits OR a pointer, no translation), and a
pointer return is a valid Addr. Picks callPtrRet (void*-ABI) for pointer-ish
returns, callIntRet (i64-ABI) otherwise; honors variadic. Non-word
(aggregate/string/float) args+returns bail loudly (4D.2 adds them). One general
mechanism for all externs, not per-builtin special cases.
New example 0636-comptime-extern-libc (#run toupper(97)/tolower(90) -> 65/122) runs
HANDLED on the VM, output byte-matching legacy. 698/0 both gates.
Replace the growable ArrayList(u8) flat buffer (reallocs/MOVES on growth) with a
std.heap.ArenaAllocator. Each allocBytes is a separate arena allocation that never
moves and is freed wholesale on deinit -- no per-object free, no cap, no fixed
buffer. Addr is now the allocation's ABSOLUTE host pointer (@intFromPtr), not an
offset, so a flat-memory pointer and an FFI-returned host pointer are the same kind
of value -- the FFI bridge (4D.1) passes them to/from libc with zero translation and
no per-call pinning (the moving-buffer hazard is gone by construction).
readWord/writeWord/bytes deref the absolute pointer with a null-check bail (the
malformed-IR / null-deref safety contract). Dropped the offset-based upper-bounds
check (can't bound an absolute pointer; Frame.bad_ref still catches the dominant
malformed-IR vector) and the test-only mark/reset (arena has no reset-to-mark; the
VM never used them outside tests).
697/0 both gates + all unit tests (rewrote the two Machine tests). Pure refactor, no
comptime behavior change.
Ported the Any-boxing conversion pair:
- box_any: alloc the 16-byte { type_tag@0, value@8 } box, tag = source TypeId
index (matches the legacy comptime interp; runtime anyTag also normalizes
arbitrary-width ints). Value slot holds a word source's scalar bytes (via
writeField(source_type) so f32 round-trips) or an aggregate source's
flat-memory ADDR (the runtime pointer-in-value-slot shape).
- unbox_any: read the value slot back (word -> readField; aggregate -> the
stored ADDR).
Required promoting .any to a first-class flat-memory aggregate (was
kindOf -> .unsupported): kindOf(.any) = .aggregate (16B, by-address) and
fieldOffset special-cases .any to the {@0, @8} layout (shared with
string/slice). Without the latter a struct_get on an Any panicked
(union field 'struct' while 'any' is active) -- caught + fixed, no crash.
Updated two unit tests that used unbox_any as the "unported op" example ->
compiler_call; added a box->unbox round-trip test. 697/0 both gates + all
unit tests. The 6 box_any examples no longer bail at box_any (output matches
legacy) but fall back further at switch_br/type_name/out (later 4A steps).
Audited the 5 roles interp.zig still serves (A comptime folds, B #insert,
C post-link bundler, D #compiler hooks, E bail diagnostics) and the shared
substrate (Value tagged union + host_ffi bridge).
User decision: UNIFY — the VM gains a host-FFI escape + real-pointer
translation and runs the post-link bundler too; interp.zig fully deleted.
Dependency-ordered sub-phases recorded in PLAN-COMPILER-VM.md:
4A finish comptime ops (box_any/unbox_any, out/print, global_addr, trace) ->
4B VM-native diagnostics -> 4C #insert -> 4D host FFI + #compiler hooks ->
4E post-link bundler (+ dedicated bundle tests) -> 4F flip default + delete
interp.zig/Value + re-express define/make_enum over the compiler-API.
No code change — planning + checkpoint only.
Ported type_info($T) into the VM (callBuiltinVm .type_info arm -> new
buildTypeInfo), the inverse of step 7's define: reflect a type INTO a TypeInfo
VALUE built in flat memory (VM-native mirror of legacy reflectTypeInfo).
- Decodes the source type into a tag + members (tagged-union/struct field &
enum variant -> {name, ty}, payloadless variant -> void; tuple -> bare
positional Types), then lays the nested value out bottom-up using layouts
derived from the TypeInfo RESULT type (ins.ty, now threaded into
callBuiltinVm): element array -> {ptr,len} slice -> info struct
(EnumInfo/StructInfo/TupleInfo) -> TypeInfo {tag, payload} tagged union
(reusing step 7's tagged-union write).
- Variant/field names materialize via makeStringValue, extracted from text_of.
- Same backing_type guard as step 7 (bail rather than mis-read the tag).
The ENTIRE metatype surface now runs HANDLED on the VM with ZERO fallback:
0614-0624 + 0632 (0616 field_type folds at lower time). The
define(declare, type_info(T)) round-trips (0619/0622/0623) mint byte-identical
copies on the VM; VM output byte-matches legacy for all. 697/0 both gates + all
unit tests. Remaining VM fallbacks in the comptime corpus are now genuinely
non-metatype emit-time side effects (print/global_addr/compiler_call/inline-asm).
The metatype type-construction builtins now run natively on the flat-memory
VM, so the construction examples run HANDLED end-to-end (no call_builtin
fallback to the legacy interp).
- Tagged-union enum_init WITH payload: allocate zeroed, write the tag at
offset 0, copy the payload at tag_size ({ header, [N x i8] } layout).
- New .call_builtin exec arm -> callBuiltinVm (VM-native mirror of the legacy
execBuiltinInner): declare(name) mints an empty forward nominal slot (shared
declareNominal, also used by declare_type); define(handle, info) reads the
TypeInfo tagged-union VALUE from flat memory and mints via defineFromInfo,
a faithful port of legacy defineEnum/defineStruct/defineTuple (all-void enum
-> real .enum per issue 0142, dup-name rejection, updatePreservingKey vs
replaceKeyedInfo). Unmodeled builtins bail -> legacy fallback (dual-path).
- Refactored the []{name,ty} decode out of registerTypeVm into a shared
decodeMemberSlice (+ decodeTypeSlice for bare-Type tuple elements).
- Correctness guard: enum_init/define assume a tag-headed layout, wrong for a
backing_type tagged union (laid out as the backing struct) — both now bail
loudly on backing_type != null rather than silent-clobber.
Examples 0614/0620/0621/0624/0632 run fully HANDLED on the VM; 0622/0623 run
define HANDLED then fall back at the still-unported type_info. VM output
byte-matches legacy for all 7. 697/0 both gates + all unit tests (added:
tagged-union enum_init payload layout).
The VM can now evaluate a comptime type-fn that allocates at lowering time (the
0141 family) — the legacy interp cannot. Four changes:
- runComptimeTypeFunc (lower/comptime.zig): force the CAllocator->Allocator thunks
to exist (getOrCreateThunks, idempotent, guarded) BEFORE eval. A type-fn const
runs at scanDecls (Pass 1), before Pass 1c builds the default-context global +
thunks, so the comptime allocator was otherwise null.
- materializeDefaultContext: build a REAL context at lowering time when the global
is absent — find the two thunks by name and lay their func-refs into the inline
Allocator value at the head of Context, so context.allocator.alloc_bytes
dispatches call_indirect -> thunk -> native VM malloc.
- aggType: deref a pointer base_type (the List write path emits struct_gep with
base_type = *Struct; fieldOffset panicked on the pointer — now derefs, no panic).
- subslice: handle a [*]T many-pointer / *T base (a List's items field — the base
IS the data pointer).
Verified end-to-end (manual probe): a compiler-API type-fn building its []Member in
a List(Member) runs HANDLED on the VM and mints (green=7) — the 0141 List-growth
pattern. Can't be a corpus test yet (gate-OFF/legacy can't allocate at lowering
time — the dual-path bind), so locked in via VM unit tests (many-pointer subslice;
struct_gep with a pointer base_type). 697/0 both gates + all unit tests.
declare_type / pointer_to / register_type are now serviced natively in
Vm.callCompilerFn, mirroring the legacy compiler_lib handlers (mint via
@constCast(table) — the lowering-time mint target is &module.types). register_type
reads the []Member slice from flat memory: ref_types is threaded through invoke ->
callCompilerFn so the slice element type (Member = {name: string, ty: Type}) gives
the field offsets + stride; each {name, ty} is decoded and minted with the same
kind branching + dup/payload rejections + idempotent re-fill as legacy.
Key unblock: the synthesized comptime type-fn wrapper was built with return type
.any, so regToValue bailed at the VM<->legacy boundary; changed to .type_value
(the legacy path reads via asTypeId regardless). The compiler-API write type-fns
(0631 register-graph, 0635 multi-edge import) now run HANDLED end-to-end on the VM
at lowering time — parity-correct, on the zeroed lowering-time context (fixed
member arrays, no allocation). The metatype make_enum/define examples still fall
back cleanly through call_builtin(define).
697/0 both gates + EXIT=0.
kindOf(.type_value) -> .word; new const_type exec arm -> word = TypeId.index();
regToValue maps a .type_value word back to a .type_tag Value at the legacy
boundary. The VM now runs comptime evals involving Type values instead of
bailing.
This reached a latent VM panic: struct_init assumed a .@"struct" result type and
union-access-panicked on an array literal (EnumVariant.[...]). It is the generic
aggregate-literal op, so it now dispatches on the result kind (struct/array/
tuple) and bails loudly on anything else — never panics (CLAUDE.md no-panic).
697/0 both gates (make_enum type-fns run further on the VM, then bail cleanly at
the define call_builtin -> legacy mints; no mutation before bail). VM unit test
added (const_type -> word -> regToValue -> .type_tag).
type_resolver "Type" -> .type_value; const_type result + emitConstType now a
bare 8-byte i64 handle (not a 16-byte Any box). Migrated every .any ref meaning
"a Type value", leaving real boxed-Any refs:
- "Any holds a Type" meta-marker tag .any -> .type_value at all 4 consumers
(reflectArgTypeId, reflectTypeId, the comptime type_tag-as-struct path,
resolveTypeCategoryTags "type").
- reflection-builtin return types (type_of/declare/define) -> .type_value;
runtime type_of(any) reads the tag as a .type_value (no re-box).
- expr_typer: a bare type-name expr is .type_value (backtick is_raw exempt).
- reflectionArgIsType accepts .type_value OR .any (a reflection arg can be a
bare Type or a boxed Any).
- comptime switch_br accepts a .type_tag discriminant (type-category match).
- a bare function name in a Type slot -> const_type(its function type), not a
func-ref (fixes a JIT crash); old string-box kept only for genuine Any params.
- field-not-found diagnostic + formatTypeName render .type_value as "Type".
Fixed 3 unit tests asserting the old .any behavior. 697/0 both gates (gate ON
bails cleanly to legacy since the VM doesn't model Type values yet) + 494 unit
tests. 24 snapshots regenerated (22 .ir const_type shape; 2 .stderr Any->Type).
Add TypeId.type_value (slot 19) + matching TypeInfo.type_value variant: an
8-byte type handle, distinct from the 16-byte boxed .any. All types.zig layout
handlers wired (size/align 8, display "Type", hash/eql); toLLVMTypeInfo -> i64.
Reserve builtin headroom: first_user 19 -> 100 (slots 20-99 padded with the
unresolved tripwire) so future builtins don't renumber user TypeIds / churn
sx ir snapshots. 22 IR snapshots regenerated (pure renumber to 100-base).
type_resolver still returns .any for "Type" — nothing produces .type_value
yet, so no behavior change. 697/0 both gates.
Records the current state (read side, write side P3.3, lowering-time hardening +
wiring + zeroed context P3.4) and pins the next focused step: a dedicated Type
builtin TypeId (8B) distinct from .any (16B box) — ~123 .any refs across ~25 files,
a cross-cutting change to run as its own session. Paused here at a clean, green
boundary (697/697 both gates) per the decision to not rush it.
materializeDefaultContext now falls back to a zeroed Context (found by name) when
the __sx_default_context global is absent — i.e. at lowering time, where the global
isn't emitted yet. A type-fn that never touches the allocator runs past context
setup; one that allocates reads a null alloc_fn (zeroed) and call_indirect on the
null func-ref bails to legacy (a real lowering-time context with the CAllocator
thunk func-refs is a follow-up).
Measurement (SX_COMPTIME_FLAT_TRACE): the bail moved deeper — make_enum now bails
at const_type (the Type-literal op, unported); register_type type-fns bail at the
welded write call. No table mutation before either bail (write fns bail before
minting), so parity holds: both gates 697/0, no crashes.
Next: model the const_type op + the Type-return bridge + the VM-native write side,
which together let a type-fn run end-to-end on the VM.
Route runComptimeTypeFunc (the type-fn fold — the third comptime call site)
through comptime_vm.tryEval behind -Dcomptime-flat/SX_COMPTIME_FLAT with legacy
fallback, mirroring the two emit-time folds. Extract the shared post-check
(checkComptimeTypeResult — the declared-but-never-defined zero-field guard) so the
VM and legacy paths share it.
Measurement (SX_COMPTIME_FLAT_TRACE): every metatype/compiler-API type-fn bails
CLEANLY at "no __sx_default_context global to materialize the implicit context" —
at lowering time the default-context global doesn't exist yet (it's built at emit
time), so the VM bails at context materialization, before running the body (no
partial mint, no crash -> legacy mints). The hardening holds: no crashes across
the corpus on the lowering-time VM path.
So the first lowering-time blocker is the implicit context, not Type modeling.
Both gates 697/0. Near-pure fallback today — permanent scaffolding that lights up
as the default-context handling + Type modeling + VM-native write side land.
Prerequisite for wiring the VM at the lowering-time comptime site
(runComptimeTypeFunc), where IR can be malformed (an unresolved name lowers to a
dangling / Ref.none operand — the 0737 crash). Close the remaining panic vectors
so the VM bails (-> legacy fallback) instead of aborting:
- Vm.refTy(ref_types, r): a bounds-checked accessor replacing every raw
ref_types[ref.index()] in exec — the type-side companion to Frame.get's
bad_ref value-side guard.
- aggType is now a bailing method (Error!TypeId) routed through refTy.
- the block-dispatch loop bounds-checks the branch target before indexing
func.blocks.items (a malformed br target). global_get was already guarded.
No behavior change: gate OFF and -Dcomptime-flat both 697/0. Unit test added
(a cmp_lt with a Ref.none operand bails, not panics).
Dual-path + emit-time legacy fallback are transitional scaffolding only; the VM
must reach parity at BOTH comptime sites (emit time AND lowering time), after
which the -Dcomptime-flat flag, the fallback, and interp.zig are all removed.
We do not ship both evaluators permanently.
The mutating compiler-API, minting types LAZILY at lowering time (single pass,
the existing runComptimeTypeFunc path — so the write side is legacy-only; the
VM isn't wired at lowering time, and the read-side readers stay dual-path):
declare_type(name) -> Type forward nominal handle (≈ declare)
pointer_to(t) -> Type build *T references
register_type(handle, kind, members) ONE kind-branching fill (≈ unified define)
register_type branches on kind IN THE COMPILER (subsuming define's per-kind
dispatch); codes match type_kind: 1 struct, 2 actual .@"enum", 3 tagged_union,
4 tuple. Members are {name: string, ty: Type}. A non-generic `-> Type` builder is
now flagged is_comptime (decl.zig) so its dead body permits the welded calls.
Graph support: forward declare_type handles + pointer_to express a mutually-
recursive A<->B graph (*A, *B, B-by-value) before bodies are filled. register_type
is idempotent — re-filling a nominal slot (a minting module reached via two import
edges) re-mints identically rather than erroring (nominalIdent reads identity from
any nominal kind).
Fixes (issue 0142):
- A fully payloadless comptime-minted enum was minted as an all-void tagged_union,
whose IR size disagrees with its LLVM size -> verifySizes panic. Now mints a real
.@"enum" (register_type kind 2 AND the metatype defineEnum).
- Bare `EnumType.variant` qualified construction of a payloadless variant wasn't
supported (failed for hand-written enums too — the type name lowered to a Type
value). Added in lowerFieldAccess via isPayloadlessVariant; payload-carrying
variants keep their call form.
Examples: 0631 (graph + actual enum + reflection), 0632 (make_enum all-void),
0633/0634/0635 (namespaced / bare / multi-edge import of a minted type), 0187
(qualified variant construction). Unit tests added.
Parity 697/697 (gate OFF and -Dcomptime-flat).
The last two read-only readers the metatype's type_info(T) needs, each backed by
a TypeTable query both the legacy handler and the VM call (no drift):
type_kind(t: TypeId) -> i64 (kindCode; stable discriminant, total — never bails)
type_field_value(t: TypeId, idx) -> i64 (memberValue; enum explicit value or ordinal)
kindCode codes (compiler-owned, stable): 0 other / 1 struct / 2 enum /
3 tagged_union / 4 tuple / 5 union / 6 array / 7 vector / 8 error_set.
With these, the READ side is complete: find_type + type_kind + type_field_count +
type_field_{name,type} + type_nominal_name + type_field_value cover everything
reflectTypeInfo reads — a comptime sx fn can fully reflect a struct/enum/tuple
into data with no #builtin.
Example 0630 reflects Color / WindowFlags(flags) / Point. VM unit test added.
Revised forward direction: the write side will be ONE register_type(info) fn that
branches on the kind in the compiler (subsuming define's per-kind dispatch), not a
per-kind register_struct.
Parity 691/691 (gate OFF and -Dcomptime-flat).
Three more read-only compiler-API readers on the TypeId-handle shape, each backed
by a new TypeTable query that both the legacy handler and the VM call (no drift):
type_nominal_name(t: TypeId) -> StringId (nominalName; loud-bail for unnamed types)
type_field_name(t: TypeId, idx: i64) -> StringId (memberName)
type_field_type(t: TypeId, idx: i64) -> TypeId (memberType)
All loud-bail on out-of-range idx / no-member — no silent default. First multi-arg
compiler fns (callCompilerFn now reads arg 1 = idx); added Vm.argHandle/argTypeId
range-checked arg readers and moved find_type/type_field_count onto them. Names use
the type_* family to avoid colliding with the std metatype builtins (field_name /
type_name in core.sx); the new TypeTable.nominalName is distinct from the existing
typeName(id) display-string renderer.
Example 0629 reflects Pair { lo: Point; hi: Point } — each field name + the nominal
name of a field's type, #run-folded, VM-HANDLED natively. VM unit test added.
Parity 690/690 (gate OFF and -Dcomptime-flat).
First read-only compiler-API reflection readers, bound the same way as the
intern/text_of seed (compiler_lib.bound_fns + Vm.callCompilerFn, native on flat
memory, no marshaling). A type handle is a plain u32 TypeId (like StringId), so
both stay clean scalar host-calls:
find_type(name: StringId) -> TypeId (TypeTable.findByName; unresolved/0 if absent)
type_field_count(t: TypeId) -> i64 (new TypeTable.memberCount; loud-bail, no silent 0)
memberCount is the single source both the legacy handler and the VM read, so the
two paths can't drift. find_type returns a non-optional TypeId using the
unresolved(0) sentinel for not-found rather than ?Type — a Type value is
.any-typed (which the flat-memory VM does not represent) and an optional can't
cross the legacy<->VM eval boundary; unresolved is the project-blessed "no type"
marker.
Example 0628 chains intern -> find_type -> type_field_count (+ a not-found
lookup), folded at #run, VM-HANDLED natively. VM unit test added.
Parity 689/689 (gate OFF and -Dcomptime-flat).
Phase 1.final of the flat-memory comptime VM — wire the host through it,
reach corpus parity, and gate it behind a build flag — plus the first
Phase 3 (compiler-API) step. Default OFF; legacy interpreter unchanged.
Host wiring + hardening:
- Machine accessors return error.OutOfBounds (no debug panic) on bad
addresses; Frame.get/set bounds-check and bail (no panic) on a malformed
operand ref (e.g. a ret Ref.none from an unresolved name).
- tryEval routed at both comptime call sites in emit_llvm — the const-init
fold and the #run side-effect path — with per-eval legacy fallback;
yields .void_val for void/noreturn entries. Both sites sx_trace_clear()
before the legacy fallback so a partial VM run that pushed trace frames
doesn't double-push on re-run.
VM coverage (all corpus const-inits except the inline-asm global):
- Implicit context materialized from the __sx_default_context global; the
full allocator protocol runs on the VM (context.allocator.alloc ->
call_indirect -> CAllocator thunk -> libc_malloc -> native flat malloc).
- Native libc memory builtins (malloc/calloc/free/memcpy/memmove/memset)
on flat memory; f32 stored/loaded as the 4-byte single; signed sub-64-bit
loads sign-extended; global_get (lazy + memoized); func_ref/call_indirect
(func-ref encoded fid+1, 0 reserved for null); string/slice fat-pointer
field access; is_comptime; the failable/error cluster (error_set tuples,
trace_frame + native sx_trace_push/clear -> raise/catch/or + return traces).
Build flag + Phase 3 seed:
- -Dcomptime-flat (build_opts module) OR SX_COMPTIME_FLAT env enables the VM;
zig build test -Dcomptime-flat runs the full corpus on the VM (688/0).
- intern/text_of serviced natively on flat memory via Vm.callCompilerFn
(compiler_welded boundary) — the seed the rest of the compiler-API grows on.
Parity 688/688 gate ON and OFF. Unit tests added throughout. The
lowering-time #insert wiring was explored and reverted (lowering-time IR can
be malformed; full malformed-IR hardening is a prerequisite, deferred).
Phase 1 of the flat-memory comptime VM (current/PLAN-COMPILER-VM.md),
built standalone + unit-tested with the legacy interpreter still live and
the corpus untouched (688 green).
src/ir/comptime_vm.zig:
- Machine: one linear byte memory (comptime stack+heap) with a bump/stack
allocator (mark/reset), scalar readWord/writeWord (1/2/4/8 LE) + byte
views; addr 0 reserved as null_addr. Frame: a Ref-indexed register file
(Reg = raw u64: immediate scalar bits OR an Addr). Target-aware layout
comes from the type table, so cross-compilation stays correct.
- Vm executor over the SAME SSA IR, mirroring the legacy interp's scalar
semantics (i64 wrapping/signed, f64). Ported: constants, arithmetic,
comparison, logical, conversions, control flow (br/cond_br/ret + block
params); structs (alloca/load/store/struct_init/get/gep at target
offsets); tuples; arrays (index_get/gep, length); slices+strings as
{ptr,len} fat pointers (const_string, data_ptr, subslice,
array_to_slice, str_eq/ne, index-through-slice); optionals (pointer and
{T,i1} shapes); payloadless enums; deref/addr_of; direct + recursive
call over the shared flat memory (depth-guarded). The value model: a
word for scalars/pointers, by-address for aggregates (a struct's value
IS its Addr). Any unported op bails loudly (error.Unsupported + detail).
- Reg<->Value boundary bridge (valueToReg / regToValue) + tryEval, the
hybrid-wiring entry point: run a comptime fn on the VM, return a legacy
Value or null to fall back. Transitional, for the legacy interop edge.
Registered in the ir.zig barrel.
The byte-weld (sx structs whose layout was validated to mirror the
compiler's Zig records) plus the serialization/marshaling bridge was the
wrong direction: it bolted a parallel layout regime and hand-built
byte-copies onto a comptime value model that fundamentally isn't bytes.
Strip the struct-weld machinery:
- compiler_lib.zig loses the type registry (weldStruct / bound_types /
BoundType / FieldLayout / findType / SxField / LayoutMismatch /
validateStructLayout); it is now just the intern/text_of function
host-call bridge (kept as the Phase-3 compiler-call seed).
- nominal.zig loses validateWeldedStruct / weldedFieldOrderStr + the
sd.abi == .zig validation call.
- Remove the struct-weld unit tests and examples 0625/0627 (welded
structs) + 1183/1186 (weld-layout diagnostics).
- The #library / abi / extern syntax stays.
Record the new direction: a bytecode VM over flat, byte-addressable
memory so comptime values are native bytes (no weld/validation/marshal),
target-aware (preserves cross-compilation) and sandboxed. See
current/PLAN-COMPILER-VM.md (Phase 0 strip -> Phase 1 flat-memory value
model -> Phase 2 bytecode -> Phase 3 compiler-API on flat memory).
design/comptime-compiler-api.md gets a SUPERSEDED banner. Also drop the
"~500 lines / split the step" rule from CLAUDE.md.
Replace the explored byte-layout-override engine (offset-ordered LLVM structs /
weld plans / byte-blobs — all unnecessary) with a much simpler design: a welded
`struct abi(.zig) extern compiler { … }` is a bodied header declaring its fields
in the bound compiler type's MEMORY order. The compiler reflects the real Zig
type (field names via @typeInfo, offsets via @offsetOf, size via @sizeOf —
nothing hand-maintained) and validates the header matches, with loud diagnostics.
On pass it is an ordinary struct whose natural layout already equals the Zig
layout — no reorder, no padding, no index/remap tables, no special LLVM path — so
@ptrCast'ing it to the compiler's own type and dereferencing is byte-identical.
When types.zig shifts, the header stops matching and the developer gets a specific
message to fix it.
- compiler_lib.zig: weldStruct reflects field names and bakes bound_types fields
in ascending-offset (memory) order; deleted computeWeldPlan/WeldPlan/WeldElement.
- nominal.zig validateWeldedStruct: precise diagnostics — field-not-found,
wrong-field-order (+ expected memory order), type-layout (size) mismatch,
total-size mismatch.
- Examples: 0627 (StructInfo in memory order, byte-identical, usable),
1186 (source-order StructInfo -> wrong-field-order diagnostic); 1183 refreshed.
- Design doc + checkpoint updated.
`zig build test -Dname=examples/0625-foo.sx[,examples/0626-bar.sx]` runs ONLY the
named example(s) — full repo-relative .sx paths, comma-separated (a leading `./`
is tolerated). Empty = run everything (unchanged default).
Why: a full `-Dupdate-goldens` re-runs and rewrites all ~690 snapshots, so one
flaky/host-divergent example (AOT links, cross-arch `target` examples) can clobber
a good snapshot. `-Dname` regenerates only the named example(s) and touches
nothing else. It also busts the cached test-run result — the corpus enumerates
.sx/expected files at runtime, so a bare snapshot edit alone is otherwise served
from cache.
- build.zig: new `name` option threaded onto corpus_paths.
- corpus_run.test.zig: `nameMatchesFilter` + a per-example skip in the run loop.
- CLAUDE.md: document the targeted-regen workflow under Snapshot integrity.
Add the COMPILER-API stream to CLAUDE.md's session-start router and a
`## ⏯ Resume` block to CHECKPOINT-COMPILER-API.md (next action = sub-step 2.2,
read order, build/verify, and the cross-arch snapshot-regen gotcha).
Introduce the welded comptime `compiler` library (`#library "compiler"` +
`abi(.zig) extern compiler`), per design/comptime-compiler-api.md, and unify
`callconv(...)` into the new `abi(...)` annotation.
abi(...) replaces callconv(...):
- New ABI enum { default, c, zig, pure }; `abi(.c|.zig|.pure)` parses in the
postfix slot before extern/export (and standalone). `kw_callconv` -> `kw_abi`.
- Migrated 52 sx files, the call-convention-mismatch diagnostic, and docs
(readme/specs) from `callconv(.c)` to `abi(.c)`.
Phase 1 — welded compiler library (parse -> registry -> validation -> bridge):
- `abi(.zig) extern compiler` parses on fn decls (carries abi/extern_lib) and
struct decls (StructDecl.abi/extern_lib).
- `#library "compiler"` is the comptime-only internal surface — never dlopen'd.
- src/ir/compiler_lib.zig: the binding registry (the safety boundary). `Field`
welded to StructInfo.Field with layout baked from the real Zig type
(@offsetOf/@sizeOf); `findType`/`findFn`. Welded structs are layout-validated
at registration (field set + total size) as a header checked against the impl.
- Host-call bridge: a `fn abi(.zig) extern compiler` dispatches under the
comptime interp to its registered Zig handler (intern/text_of round-trip),
never dlsym. IR Function.compiler_welded; validated in declareFunction.
- Comptime-only enforcement: a runtime call to a welded fn is a clean
build-gating error (emitCall), not an undefined-symbol link failure.
Phase 2.1 — byte-layout weld foundation:
- Decision: full byte-layout weld (sx struct laid out byte-identically to the
bound Zig type). Registered StructInfo (first non-natural / Zig-reordered
layout). `computeWeldPlan` — pure offset-ordered element plan + padding +
sx-field->LLVM-element remap; unit-tested. Emit/interp wiring is the next
sub-step (2.2+, see current/CHECKPOINT-COMPILER-API.md).
Examples: 0625/0626 (welded struct + fn round-trip), 1183/1184/1185
(layout-mismatch, unexported-fn, runtime-call diagnostics).
