lab/sx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

comptime VM arc: abi(.compiler) ABI, out as sx fn, VM-native diagnostics, BuildConfig threaded

Browse Source 
Lands the full VM/compiler-API arc on branch reify (701/0 both gates):
- abi(.compiler) ABI replaces abi(.zig) extern compiler + the fake
  #library "compiler"; bodiless decl = compiler-API surface, bodied =
  user compiler-domain fn (lowered for VM eval, emit-skipped).
- out is a plain sx fn (libc write) — the out builtin deleted; the VM
  handles it via host-FFI. trace_resolve + interp_print_frames ported.
- 4B VM-native diagnostics: 1179/1180 render proper comptime type
  construction failed: under strict.
- S5a: build_options/set_post_link_callback on abi(.compiler) with
  BuildConfig threaded into the VM (green intermediate).
- 0522 fixed (describe(args: []Type)); regression 0638.

Strict deletion-gate down to 4 compiler_call bails (1609/1614/1615/1616)
+ 1654 (legitimate unresolvable-symbol diagnostic).
This commit is contained in:
agra
2026-06-19 07:04:10 +03:00
parent fdc4ee2331
commit 2060373c16
80 changed files with 12684 additions and 11922 deletions
Download Patch File Download Diff File Expand all files Collapse all files

113
current/PLAN-COMPILER-VM.md
View File

@@ -416,18 +416,98 @@ are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan ar
untouched); `4D.2` (`6a7f690`) adds slice/string args (→ NUL-term `char*`) + float guards.
Examples 0636/0637. **(Superseded sub-note:** the earlier "pin the buffer / flat↔host translate"
hazard is moot — the arena never moves an allocation.)
- **`#compiler` / `compiler_call` — NOT bridged on the VM. DELETED, not ported (decision
2026-06-18).** `BuildOptions` is RE-EXPRESSED as **`abi(.zig) extern compiler`** functions (the
compiler-API surface the VM already dispatches via `callCompilerFn`), and the `#compiler` struct
attribute + the `compiler_call` IR op + the `Value`-based hook `Registry` (`compiler_hooks.zig`)
all **go away**. So there is NO transitional `compiler_call`→hooks shim on the VM — `0602`/`0603`
stay on legacy fallback until this migration lands. Migration shape (part of the end state, with
the bundler): (1) each `BuildOptions` setter/getter becomes a `compiler` function in
`compiler_lib.bound_fns` + `Vm.callCompilerFn`, reading flat-memory args + a **`BuildConfig`
threaded into the VM** (the same `BuildConfig` `main.zig` forwards); (2) `library/modules/build.sx`
declares them `abi(.zig) extern compiler` instead of `struct #compiler`; (3) delete the
`compiler_call` op, `compiler_hooks.zig`'s `HookFn`/`Registry`, and the `#compiler` parse/lower
path. The `BuildConfig` threading is the shared prerequisite with the bundler (4E).
- **`#compiler` / `compiler_call` — DELETED, replaced by the `abi(.compiler)` ABI (decision 2026-06-18,
REVISED from the earlier `abi(.zig) extern compiler` shape).** A function is *compiler-domain* — it runs in
the comptime evaluator (VM/interp), NEVER in the shipped binary — because its **ABI says so**: `abi(.compiler)`.
No `extern <lib>`, no fake `#library "compiler"`. One annotation covers BOTH roles: (a) the **compiler-API
surface** (`intern`/`find_type`/`build_options`/`set_post_link_callback`/… — bodiless decls whose Zig/VM
handler is the impl, on `compiler_lib`'s export list, dispatched by `Vm.callCompilerFn`); (b) **user
compiler-domain functions** like post-link callbacks (`bundle_main` — BODIED `abi(.compiler)`, lowered for VM
eval but emit-skipped). The `#compiler` struct attribute + the `compiler_call` IR op + the `Value`-based hook
`Registry` (`compiler_hooks.zig`) all **go away**. **Why this is cleaner than the welded-fn approach:** the
former runtime-call enforcement blocker (a `build_options()` call inside an LLVM-emitted callback body) is
MOOT — a compiler-domain function is never emitted, so its compiler-API calls never reach `emitCall`.
**Staged build (each its own step, both gates green):**
- **S1+S2 — DONE (2026-06-18):** introduced `abi(.compiler)`, REMOVED the `.zig` ABI + `abi(.zig) extern
compiler` + `#library "compiler"` (clean cutover, no legacy); migrated all compiler-API examples. The
binding now keys off `fd.abi == .compiler` (`decl.zig` `weldedCompilerFn`); a bodiless `abi(.compiler)`
decl lowers extern-like (declared-not-defined) with no implicit ctx. **700/0 both gates.**
- **S3 — DONE (2026-06-18):** emit_llvm skips BODIED `abi(.compiler)` function bodies. Added an
`is_compiler_domain` flag to the IR `Function`; a bodied `abi(.compiler)` function LOWERS its body (for VM
eval) + is flagged `is_comptime` but is NOT emitted (Pass 2 skip; declared external-linkage so the empty
decl verifies). KEY fix: a call to a comptime-only callee (compiler-API `compiler_welded` OR
`is_compiler_domain`) inside a dead comptime body now emits `undef` instead of a real `call` (`ops.zig`
`emitCall`) — the old `compiler_call` did this; without it an AOT link leaves an undefined `_double`/`_intern`
reference (this also fixed a pre-existing untested AOT breakage of the bodiless compiler-API examples).
`fnIsBodilessCompiler` distinguishes the API surface (declare-only) from a compiler-domain callback (lowered,
emit-skipped). Regression: `examples/0638-comptime-domain-fn-not-emitted` (`double` folds a `#run` const,
absent from the binary, JIT+AOT). **701/0 both gates.**
- **S4 — callback-param propagation: OPTIONAL / DEFERRED (ergonomics only).** Verified 2026-06-18: an
`abi(.compiler)` function is TYPE-compatible with a plain `() -> R` param (the ABI marks the *function* —
`is_compiler_domain` — not its *type*, which stays `() -> R` CC-default). So a callback that needs to be
compiler-domain just declares itself `abi(.compiler)` (S3) and passes to a plain param fine; auto-propagation
from an `abi(.compiler)` PARAM type is a nicety, not a prerequisite for S5. Skipped for now.
- **S5a — DONE (2026-06-18):** the corpus-covered slice. `build_options` + `set_post_link_callback` →
free `abi(.compiler)` functions (VM `callCompilerFn` arms + legacy `compiler_lib` handlers); **`BuildConfig`
threaded into the VM** via a `tryEval` param (the same one `main.zig` forwards — shared with 4E). `build.sx`
extracts `set_post_link_callback` from the `struct #compiler` as a free `ufcs` fn; `bundle_main` + the
platform registrars (`configure`) are `abi(.compiler)`. 37 examples' `.ir` snapshots regen'd (benign:
declaration renumber + `@str` suffix shift — every example imports build.sx via the prelude). Strict
`compiler_call` bails 6→2; 0602/0603/1604/1611 HANDLED. **701/0 both gates.**
- **S5b/S5c (port the ~37 hooks) — SUPERSEDED 2026-06-18 by the sx-driven build pipeline (below).**
Porting each `BuildOptions` accessor to an `abi(.compiler)` function that delegates to a `compiler_hooks`
hook just re-encodes sx-level logic (string setters/getters, `is_macos` triple-matching, list appends) as
compiler hooks. The hooks need NOTHING from the compiler except the `BuildConfig` state. So instead of 37
hooks, **drive the whole build pipeline from sx** (the logical end of "bundling lives in sx"). S5a stays as
a green intermediate; the sx-build-pipeline replaces `build_options`/`set_post_link_callback`/the whole
`#compiler` surface wholesale.
### Phase 5 — sx-driven build pipeline (replaces the BuildOptions hooks; decision 2026-06-18, user)
**The build pipeline becomes an sx program.** `BuildConfig` is plain sx data (an ordinary struct, sx-owned
end-to-end — no `#compiler`, no hooks, no shared Zig state, no weld/offset access). The compiler shrinks to
a few `abi(.compiler)` PRIMITIVES that take **explicit args** (so nothing is shared by memory), and an sx
`build()` driver orchestrates configure → emit → link → bundle. **Chosen boundary: Option B** — the compiler
keeps the proven Zig linker as a primitive; sx owns config + orchestration + bundle. (Option A — sx shells
`cc`/`ld` itself — is a later refinement once the per-target link-line logic is ported to sx.)
Shape (all syntax verified on the current build 2026-06-18 — void `#run`, `-> !` / `-> !E` failable `#run`,
a `raise` at `#run` fails the build with a return trace):
```sx
// library/modules/std/build.sx (stdlib)
BuildErr :: error { EmitFailed, LinkFailed, BundleFailed }
BuildConfig :: struct { output: string; target: string; flags: List(string);
frameworks: List(string); bundle_path: string; bundle_id: string;
is_macos :: (self: *BuildConfig) -> bool { ... }
add_framework :: (self: *BuildConfig, n: string) { self.frameworks.append(n); } }
// compiler primitives — explicit args, failure on the error channel (NO bool):
emit_object :: () -> !string abi(.compiler); // IR → .o path
link :: (objects: List(string), output: string, libraries: List(string),
frameworks: List(string), flags: List(string), target: string) -> ! abi(.compiler);
c_object_paths :: () -> List(string) abi(.compiler); // metadata queries
link_libraries :: () -> List(string) abi(.compiler);
default_build :: (config: BuildConfig) -> ! abi(.compiler) { // the default pipeline
obj := try emit_object(); objs := c_object_paths(); objs.append(obj);
try link(objs, config.output, link_libraries(), config.frameworks, config.flags, config.target);
if config.bundle_path.len > 0 { try bundle_app(config); } } // bundle_app = today's sx bundler
on_build : (BuildConfig) -> ! abi(.compiler) = default_build; // the override slot
// user overrides: build :: (config: BuildConfig) -> ! abi(.compiler) { ... } #run on_build = build;
```
The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfig` → read `on_build` → invoke
`on_build(config)` on the VM; a `raise` fails the build. Plain `sx run` fires none of it.
**Steps (each its own green step; depends on 4E first):**
- **P5.1 — 4E prereq:** route the post-codegen / `on_build` invocation through the **VM** (`core.invokeByFuncId`
→ VM). REQUIRED because the driver allocates (`List`) and the legacy interp can't (0141 — verified: comptime
`List` growth works on the VM, fails on legacy). Add dedicated bundle smoke tests (no corpus coverage today).
- **P5.2 — primitives:** expose `emit_object` + `link` (reuse `target.zig` linker) + metadata queries
(`c_object_paths`/`link_libraries`/host-triple) as `abi(.compiler)` fns taking explicit args.
- **P5.3 — `on_build` slot:** a comptime-assignable compiler slot (GENERALIZES today's `post_link_callback_fn`:
an assignable typed global with a stdlib default, vs a setter). `#run on_build = build;` captures the
`FuncId`; the compiler invokes it post-codegen with the CLI-derived `BuildConfig`.
- **P5.4 — sx `default_build` + `BuildConfig`:** write the stdlib pipeline; move config/orchestration into sx.
**Delete** `#compiler` / `compiler_call` / `compiler_hooks` (`HookFn`/`Registry`) + the S5a
`build_options`/`set_post_link_callback` (config is now sx data passed as primitive args).
- **4E — post-link bundler on the VM (role C).** Depends on the FFI escape (done) + the
`BuildConfig`-on-the-VM threading above. Route `core.invokeByFuncId` / `main.zig`'s post-link
call through the VM. **No corpus coverage** (only runs on `sx build --bundle/--apk`) — add
@@ -440,11 +520,12 @@ are legitimate negative-test bails that BECOME VM diagnostics, 1145 is a scan ar
VM-native `callCompilerFn`); re-express `define`/`make_enum` as sx over the compiler-API
(allocation works on the sole evaluator) and land the original 0141 repro as a corpus test.
**Dependencies:** 4A → (4B, 4C independent) ; FFI(done)+`BuildConfig`-on-VM → (BuildOptions
migration, 4E) → 4F.
**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ;
FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F.
**Top risks:** (1) the bundler has no corpus guard (4E needs dedicated tests); (2) deleting
`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API touches the whole
build/bundle path — stage it behind real bundle builds.
`#compiler`/`compiler_call` + re-expressing `BuildOptions` over the compiler-API (`abi(.compiler)`) touches the
whole build/bundle path — stage it behind real bundle builds; (3) S3's emit-skip relies on DCE dropping the
unreferenced compiler-domain declaration — verify no stray runtime reference keeps it alive (link error).
## Open questions (resolve as reached, record decisions here)