First read-only compiler-API reflection readers, bound the same way as the
intern/text_of seed (compiler_lib.bound_fns + Vm.callCompilerFn, native on flat
memory, no marshaling). A type handle is a plain u32 TypeId (like StringId), so
both stay clean scalar host-calls:
find_type(name: StringId) -> TypeId (TypeTable.findByName; unresolved/0 if absent)
type_field_count(t: TypeId) -> i64 (new TypeTable.memberCount; loud-bail, no silent 0)
memberCount is the single source both the legacy handler and the VM read, so the
two paths can't drift. find_type returns a non-optional TypeId using the
unresolved(0) sentinel for not-found rather than ?Type — a Type value is
.any-typed (which the flat-memory VM does not represent) and an optional can't
cross the legacy<->VM eval boundary; unresolved is the project-blessed "no type"
marker.
Example 0628 chains intern -> find_type -> type_field_count (+ a not-found
lookup), folded at #run, VM-HANDLED natively. VM unit test added.
Parity 689/689 (gate OFF and -Dcomptime-flat).
Phase 1.final of the flat-memory comptime VM — wire the host through it,
reach corpus parity, and gate it behind a build flag — plus the first
Phase 3 (compiler-API) step. Default OFF; legacy interpreter unchanged.
Host wiring + hardening:
- Machine accessors return error.OutOfBounds (no debug panic) on bad
addresses; Frame.get/set bounds-check and bail (no panic) on a malformed
operand ref (e.g. a ret Ref.none from an unresolved name).
- tryEval routed at both comptime call sites in emit_llvm — the const-init
fold and the #run side-effect path — with per-eval legacy fallback;
yields .void_val for void/noreturn entries. Both sites sx_trace_clear()
before the legacy fallback so a partial VM run that pushed trace frames
doesn't double-push on re-run.
VM coverage (all corpus const-inits except the inline-asm global):
- Implicit context materialized from the __sx_default_context global; the
full allocator protocol runs on the VM (context.allocator.alloc ->
call_indirect -> CAllocator thunk -> libc_malloc -> native flat malloc).
- Native libc memory builtins (malloc/calloc/free/memcpy/memmove/memset)
on flat memory; f32 stored/loaded as the 4-byte single; signed sub-64-bit
loads sign-extended; global_get (lazy + memoized); func_ref/call_indirect
(func-ref encoded fid+1, 0 reserved for null); string/slice fat-pointer
field access; is_comptime; the failable/error cluster (error_set tuples,
trace_frame + native sx_trace_push/clear -> raise/catch/or + return traces).
Build flag + Phase 3 seed:
- -Dcomptime-flat (build_opts module) OR SX_COMPTIME_FLAT env enables the VM;
zig build test -Dcomptime-flat runs the full corpus on the VM (688/0).
- intern/text_of serviced natively on flat memory via Vm.callCompilerFn
(compiler_welded boundary) — the seed the rest of the compiler-API grows on.
Parity 688/688 gate ON and OFF. Unit tests added throughout. The
lowering-time #insert wiring was explored and reverted (lowering-time IR can
be malformed; full malformed-IR hardening is a prerequisite, deferred).
The byte-weld (sx structs whose layout was validated to mirror the
compiler's Zig records) plus the serialization/marshaling bridge was the
wrong direction: it bolted a parallel layout regime and hand-built
byte-copies onto a comptime value model that fundamentally isn't bytes.
Strip the struct-weld machinery:
- compiler_lib.zig loses the type registry (weldStruct / bound_types /
BoundType / FieldLayout / findType / SxField / LayoutMismatch /
validateStructLayout); it is now just the intern/text_of function
host-call bridge (kept as the Phase-3 compiler-call seed).
- nominal.zig loses validateWeldedStruct / weldedFieldOrderStr + the
sd.abi == .zig validation call.
- Remove the struct-weld unit tests and examples 0625/0627 (welded
structs) + 1183/1186 (weld-layout diagnostics).
- The #library / abi / extern syntax stays.
Record the new direction: a bytecode VM over flat, byte-addressable
memory so comptime values are native bytes (no weld/validation/marshal),
target-aware (preserves cross-compilation) and sandboxed. See
current/PLAN-COMPILER-VM.md (Phase 0 strip -> Phase 1 flat-memory value
model -> Phase 2 bytecode -> Phase 3 compiler-API on flat memory).
design/comptime-compiler-api.md gets a SUPERSEDED banner. Also drop the
"~500 lines / split the step" rule from CLAUDE.md.
Replace the explored byte-layout-override engine (offset-ordered LLVM structs /
weld plans / byte-blobs — all unnecessary) with a much simpler design: a welded
`struct abi(.zig) extern compiler { … }` is a bodied header declaring its fields
in the bound compiler type's MEMORY order. The compiler reflects the real Zig
type (field names via @typeInfo, offsets via @offsetOf, size via @sizeOf —
nothing hand-maintained) and validates the header matches, with loud diagnostics.
On pass it is an ordinary struct whose natural layout already equals the Zig
layout — no reorder, no padding, no index/remap tables, no special LLVM path — so
@ptrCast'ing it to the compiler's own type and dereferencing is byte-identical.
When types.zig shifts, the header stops matching and the developer gets a specific
message to fix it.
- compiler_lib.zig: weldStruct reflects field names and bakes bound_types fields
in ascending-offset (memory) order; deleted computeWeldPlan/WeldPlan/WeldElement.
- nominal.zig validateWeldedStruct: precise diagnostics — field-not-found,
wrong-field-order (+ expected memory order), type-layout (size) mismatch,
total-size mismatch.
- Examples: 0627 (StructInfo in memory order, byte-identical, usable),
1186 (source-order StructInfo -> wrong-field-order diagnostic); 1183 refreshed.
- Design doc + checkpoint updated.
Add the COMPILER-API stream to CLAUDE.md's session-start router and a
`## ⏯ Resume` block to CHECKPOINT-COMPILER-API.md (next action = sub-step 2.2,
read order, build/verify, and the cross-arch snapshot-regen gotcha).
Introduce the welded comptime `compiler` library (`#library "compiler"` +
`abi(.zig) extern compiler`), per design/comptime-compiler-api.md, and unify
`callconv(...)` into the new `abi(...)` annotation.
abi(...) replaces callconv(...):
- New ABI enum { default, c, zig, pure }; `abi(.c|.zig|.pure)` parses in the
postfix slot before extern/export (and standalone). `kw_callconv` -> `kw_abi`.
- Migrated 52 sx files, the call-convention-mismatch diagnostic, and docs
(readme/specs) from `callconv(.c)` to `abi(.c)`.
Phase 1 — welded compiler library (parse -> registry -> validation -> bridge):
- `abi(.zig) extern compiler` parses on fn decls (carries abi/extern_lib) and
struct decls (StructDecl.abi/extern_lib).
- `#library "compiler"` is the comptime-only internal surface — never dlopen'd.
- src/ir/compiler_lib.zig: the binding registry (the safety boundary). `Field`
welded to StructInfo.Field with layout baked from the real Zig type
(@offsetOf/@sizeOf); `findType`/`findFn`. Welded structs are layout-validated
at registration (field set + total size) as a header checked against the impl.
- Host-call bridge: a `fn abi(.zig) extern compiler` dispatches under the
comptime interp to its registered Zig handler (intern/text_of round-trip),
never dlsym. IR Function.compiler_welded; validated in declareFunction.
- Comptime-only enforcement: a runtime call to a welded fn is a clean
build-gating error (emitCall), not an undefined-symbol link failure.
Phase 2.1 — byte-layout weld foundation:
- Decision: full byte-layout weld (sx struct laid out byte-identically to the
bound Zig type). Registered StructInfo (first non-natural / Zig-reordered
layout). `computeWeldPlan` — pure offset-ordered element plan + padding +
sx-field->LLVM-element remap; unit-tested. Emit/interp wiring is the next
sub-step (2.2+, see current/CHECKPOINT-COMPILER-API.md).
Examples: 0625/0626 (welded struct + fn round-trip), 1183/1184/1185
(layout-mismatch, unexported-fn, runtime-call diagnostics).
File the last METATYPE deferred enhancement: List(T).append at comptime
bails ('struct_get: base has no fields') in a type-construction ::.
Standalone repro + two-layer root cause (null comptime allocator at
scanDecls; *T slot_ptr struct_get) + investigation prompt. Non-blocking:
array-literal locals already build variant lists (examples/0620/0624).
Checkpoint + Known issues reference 0141.
Investigated the last deferred enhancement. List(T).append at comptime
fails in two independent layers (both reproduce with plain List(i64);
List works via #run because that evaluates at emit time, after lowering):
1. null comptime allocator — defaultContextValue looks up the
CAllocator->Allocator thunks by name, but they aren't lowered at
scanDecls time. Fixable by forcing getOrCreateThunks before the interp
runs in runComptimeTypeFunc (tried, works for this layer).
2. struct_get through a *T slot_ptr chain (the *List receiver) — the
deep part; comptime pointer/struct/slot resolution, its own session.
Speculative fixes reverted (no end-to-end win without layer 2).
A failing declare/define (e.g. empty variant list) bails correctly in
the interp, but evalComptimeType swallows last_bail_detail via
`catch return null`; the decl poisons to .unresolved with no diagnostic
and reaches LLVM emission -> panic ("unresolved type reached LLVM
emission"), or hides behind a misleading downstream cascade.
Pre-existing (plain define path), surfaced while starting the make_enum
step. Blocks make_enum's computed (pointer-backed) []EnumVariant slice
decode. Repro + investigation prompt filed; CHECKPOINT-METATYPE marked
BLOCKED. Session paused pending fix per CLAUDE.md IMPASSABLE rule.
Discovered while testing metatype self-reference: a by-VALUE self-ref
(`payload = List`, not `*List`) infinite-loops typeSizeBytes → segfault
instead of a loud "infinite size" diagnostic. PRE-EXISTING — a hand-written
source enum `enum { node: Bad; leaf }` crashes identically, so it's a
general type-system gap (the comptime F5 by-value-rejection inherits the
fix). Filed per the IMPASSABLE rule; metatype checkpoint notes it.
The compiler concept is declare/define (comptime type construction); the
old "reify" framing is gone from the entire repo.
- Rename: PLAN-REIFY → PLAN-METATYPE, CHECKPOINT-REIFY → CHECKPOINT-METATYPE,
PLAN-POST-REIFY → PLAN-POST-METATYPE (both rewritten around declare/define);
examples 0614/0615/0617 → comptime-metatype-* (+ their expected/ triplets),
headers rewritten.
- Scrub reify from design/execution-evolution-roadmap.md (§7 step 3 contracts,
§8.1, §9 decisions, §10 gates) → declare/define / comptime type construction.
- core.sx prelude pointer + parser.test.zig surface lock updated to the
declare/define builtins (define(handle, info) -> Type; EnumInfo.name).
No behavior change; renamed examples match their renamed snapshots. Full
suite green (673), all unit tests pass. Zero `reify` tokens remain in
src/docs/sx/examples.
Record the verified pass-order / define-timing / parse / dispatch findings
from F1 investigation, and make explicit that the floor work MUST delete
reifyType + the E :: reify decl hook + findReturnReifyCall (reify lives only
in meta.sx). Removal can't precede the floor, so they land together; suite
never left red across a session boundary.
User-directed redirection. The compiler should expose ONLY declare() and
define(handle, info) as comptime type-table primitives; reify / make_enum /
RecvResult / TryResult all become plain sx in meta.sx (reify ==
{ h := declare(); define(h, info); return h; }). The AST-walking reifyType
and every syntactic reify recognition (decl.zig E :: reify hook, generic.zig
findReturnReifyCall routing) are to be DELETED, replaced by generic comptime
evaluation of a Type-returning expression.
PLAN-REIFY gains a RE-ARCHITECTURE section: the irreducible compiler floor
(declare = empty nominal slot; define = decode a TypeInfo VALUE + fill via
updatePreservingKey; comptime-eval a Type-returning ::-RHS/type-fn body),
the resolved naming/identity story (declare mints anonymous, the binding site
names it; identity via the existing instantiation cache), and an F1-F5 phase
table that re-greens 0614/0615/0617 on the floor.
No code change in this commit — the in-session Phase 3.2 attempt (make_enum +
eval-decode reader) was reverted (reset to 9306ad5) so the floor is built
first. Checkpoint records the revert + sets next step = F1.
REIFY Phase 3.1. Add RecvResult($T) and TryResult($T) to meta.sx as
type-fns over reify (value-or-closed; value-or-empty-or-closed). They
need NO new compiler machinery — reify-of-a-literal in a type-fn body is
exactly the Phase 1 path — so the channel result types are pure sx
library code. examples/0617 green (both construct + match, incl.
payload-less .closed / .empty). Suite green (673 examples, 447 unit).
make_enum(variants) (3.2) and type_info (2.2) remain — both blocked on a
generalized reify reader (reifyType currently AST-walks a literal
TypeInfo). Plan/checkpoint updated.
REIFY Phase 2.1. fieldTypeOf (lower/generic.zig, re-exported on Lowering)
returns the i-th member type of T: struct field / tagged-union + union
variant payload (.void for a tagless variant) / tuple element / array +
vector element. Out-of-range and memberless types poison to .unresolved
with a loud diagnostic (never a silent default). Wired into
resolveTypeCallWithBindings (replacing the Phase-2 bail); since it folds
to a TypeId at lower time it composes inside type_eq / type_name / any
type-arg slot.
examples/0616 green: struct fields (name via field_name + type via
field_type), type_eq fold, tagged-union payloads incl. quit -> void.
Suite green (672 examples, 447 unit).
type_info($T) -> TypeInfo (reflect into a value, inverse of reify) is
NOT done — still bails loudly; it's the larger Phase 2.2 step (widen the
TypeInfo data model + comptime value construction). Plan/checkpoint updated.
REIFY Phase 1.1 (Phase 1 complete). instantiateTypeFunction detects a
type-fn body that returns reify(...) (findReturnReifyCall) and routes it
to reifyType under the instantiation's name — mangled for inline use,
the alias name for `Foo :: Box(i64)` — with the type-arg bindings active
so reify payloads (`payload = T`) resolve against the instantiation args.
Placed before the general case, whose resolveTypeWithBindings would
route the reify call to the inline-position loud bail.
Registering under the mangled name lets the top-of-instantiation cache
return the SAME TypeId on a second instantiation, so Box(i64) resolved
at two independent sites is ONE type (Contract 1). examples/0615 green
(build()->consume() cross-site + `b : Box(i64) = .none`). Suite green
(671 examples, 447 unit).
User picked the declaration-vs-definition split over reserve/complete.
declare() returns a forward nominal Type handle (named from the :: LHS);
define(handle, info) fills its body. reify(info) stays the one-shot
sugar. Updated PLAN-REIFY Phase 4 + Contract 5 + CHECKPOINT-REIFY.
User-directed API decision: replace the reify_rec((self)=>...) closure
with an explicit reserve() -> Type handle + complete(handle, info) pair.
reserve() returns a forward nominal Type usable freely in any later
TypeInfo (*List, []List, and across types for mutual recursion the
one-self closure couldn't express); reify(info) stays as the one-shot
sugar. Maps onto existing reserve->complete machinery. Captured in
PLAN-REIFY Phase 4 + Contract 5 + CHECKPOINT-REIFY.
REIFY Phase 0.2 (Phase 0 complete). Lowering.reifyType (lower/nominal.zig)
reads the flat-enum TypeInfo literal off the AST, synthesizes an
ast.EnumDecl, and feeds it through the SAME type_bridge.buildEnumInfo
path source enums use — so the minted type is byte-identical to a
hand-written `enum { value: i64; closed; }` and flows through enum
codegen (layout / construct / match) UNMODIFIED (Contract 2).
Wired at the `E :: reify(...)` const-decl hook in lower/decl.zig
(replacing the Phase-0.0 loud bail). Unsupported argument shapes bail
loudly via reifyBail — never a silent default. The generic.zig inline
reify path now reports it's only supported in a `::` binding (Phase 0).
examples/0614 green: reify a {value: i64, closed} enum, construct
.value(3) and .closed, match both -> "value 3" / "closed". Full suite
green (670 examples, 447 unit).
REIFY Phase 0.1. Add the end-to-end Phase-0 example: reify a flat enum
(value: i64, closed) from a TypeInfo literal, construct E.value(3) /
E.closed, and match both arms. Seed an empty expected/*.exit marker.
RED by design (reify still bails -> "unparseable expected exit"); the
next commit (0.2) implements reify and turns it green. Satisfies the
no-commit-both-adds-a-test-and-passes cadence.
REIFY Phase 0.0. Add the comptime type-metaprogramming surface as the
on-demand module modules/std/meta.sx (NOT the prelude — declaring its
data types in always-loaded core.sx interns them into every module's
type table and shifts every .ir snapshot):
- EnumVariant / EnumInfo / TypeInfo data types. TypeInfo's variant uses
the backtick raw escape `enum so it reads as the keyword.
- reify / type_info / field_type as bodyless #builtin decls.
Each builtin bails LOUDLY when reached unimplemented (no silent default):
- reify(...) in a :: type-alias position -> decl.zig .call branch
(also the Phase 0.2 construction hook); poisons the alias .unresolved.
- reify / field_type in any other type position ->
generic.zig resolveTypeCallWithBindings.
- type_info(...) in expression position -> call.zig tryLowerReflectionCall.
Unit test src/parser.test.zig (registered in root.zig) locks that the
decls parse. zig build test green (447 unit, 669 examples).
Add the async-first execution-model roadmap (comptime JIT spine, colorblind
fibers/Io, atomics, hot-reload) with all seven decisions resolved and
three-way reviewed, and carve the first stream: comptime type_info/reify
(PLAN-REIFY + checkpoint) — the codebase-validated foundation for channel
result types and race's synthesized tagged union.
Drive a bundled `zig` as `zig cc` for the AOT link step, supplying lld + CRT
+ libc (musl/glibc/mingw) so `sx build` produces native binaries with no host
toolchain. Default Linux output is static musl (portable-anywhere).
- src/zig_backend.zig: discover zig ($SX_ZIG / bundled-next-to-exe / PATH);
bundled-vs-PATH provenance gates auto-activation.
- src/target.zig: selectZigLinker + emitZigLinkArgv + zigTargetTriple, dispatched
before the per-OS branches; macOS/Linux/Windows in scope.
- src/ir/emit_llvm.zig: LLVMNormalizeTargetTriple so vendor-less zig triples
(e.g. x86_64-windows-gnu) parse to the correct OS/object format (COFF not ELF).
- src/main.zig: --self-contained / --no-self-contained; linux-musl, linux-musl-arm,
windows-gnu shorthands; de-vendor linux/linux-arm to match the corpus runner.
- examples/1660: Windows Win32 print-42 + exit(0) via kernel32 (ir-only off-Windows).
Auto-activates only for a bundled zig; a PATH-only zig engages under
--self-contained, so native dev/CI builds are never silently rerouted.
Docs: readme Cross-Compilation, design/bundled-zig-link-backend-design.md, current/PLAN-DIST.md.
Updates the symbol-operand guide: x86 now uses the same plain %[fn] as
aarch64, and a 'How the portability works' note explains the mechanism
(compiler auto-injects LLVM's :c modifier for "s" operands, equivalent
to GCC :P/%P0 for x86 calls, no-op on aarch64, overridable). Drops the
stale per-arch :P guidance; checkpoint updated.
Adds a 'Symbol inputs — "s" = fn' section to docs/inline-assembly.md
(direct bl/call, portability, the export-vs-callconv linkage point) and
logs the symbol-operand + round-trip work in CHECKPOINT-ASM.
Moves docs/inline-asm-design.md -> design/inline-asm-design.md (the
internal design record now lives under design/, separate from the
user-facing docs/). Updates all links: current/CHECKPOINT-ASM.md,
current/PLAN-ASM.md, current/PLAN-EXTERN-EXPORT.md (../docs -> ../design)
and docs/inline-assembly.md (same-dir -> ../design).
sx run compiles to an object before ORC relocation, so module asm is
assembled in and its symbols resolve at JIT main execution. Corrected
the Phase F note, Current state, and Next step; the only real boundary
is a compile-time #run into a module-asm symbol (loud dlsym-miss).
Filed issues/0138: `@const` (address-of a `::` comptime constant) lowers
to `inttoptr (i64 <value> to ptr)` — segfaults on deref, invalid store for
asm `-> @const`. Root cause in src/ir/lower/expr.zig .address_of (not asm).
Marked CHECKPOINT-ASM Next step BLOCKED on 0138 for the output-to-const
rejection item.
An asm result can be STORED through a place (a local / struct field) instead of
returned; the place output does not join the result tuple.
- parser.zig: `-> @place` parses `@place` as an ordinary address-of expression
→ an out_place operand (the in-function form; reuses the existing `@` prefix).
- inst.zig: AsmOperand gains out_ty (the output slot's value type) so emit can
build the combined return struct without re-deriving from Inst.ty.
- lower/expr.zig: out_place operand = the lowered @place address, out_ty = the
pointee. Read-write (`+`) and indirect-memory (`*`) constraints rejected loudly
(not yet implemented) rather than miscompiled.
- ops.zig emitInlineAsm: the LLVM return type is built from ALL outputs
(out_value + out_place); after the call, out_place slots are stored through
their address and out_value slots rebuild the sx result. Fast path when there
are no place outputs (the struct return IS the result — pure-value asm IR
unchanged).
Verified: write-to-local (42), struct field, mixed value+place (v=10 b=20), `+`
rejected. Locked with 1649-platform-asm-place-output (mixed, runs on aarch64).
zig build test green (657 corpus, 446 unit).
A top-level `asm { "tmpl", };` block (template only) lowers to LLVM `module asm`;
a lib-less `extern` declaration calls into the symbols it defines (the import
direction reuses the existing C-FFI extern path — no new surface).
- ast.zig: asm_global node (AsmGlobal { template }).
- parser.zig: parseAsmGlobal, dispatched from parseTopLevel on kw_asm — rejects
`volatile` and any operands/clobbers (template only). The in-function asm
expression form stays in parsePrimary.
- module.zig: Module.global_asm list; lower/decl.zig captures each template in
lowerMainAndComptime (the real top-level pass — lowerDecls is dead for
top-level); emit_llvm.zig emit() appends each via LLVMAppendModuleInlineAsm in
source order.
- the new node forced asm_global arms in sema.zig (analyzeNode +
findNodeAtOffset) and semantic_diagnostics.zig (checkBindingNames).
Verified end-to-end: an aarch64 `_my_add` global routine, called via `extern`,
returns 42 — AOT only (the ORC JIT doesn't link module-asm symbols; global-asm
symbols live in the final linked binary). Locked with 1648-platform-asm-global
({ "aot": true, "target": "macos" } → AOT build+run on aarch64, ir-only else).
zig build test green (656 corpus, 446 unit).
