P5.2 metadata queries: c_object_paths / link_libraries on the VM

Two abi(.compiler) build-pipeline primitives the sx driver will pass to link:
- c_object_paths() -> List(string)  (#import c companion objects)
- link_libraries() -> List(string)  (#library names)

They live in a new stdlib home library/modules/std/build.sx and are serviced
by comptime_vm.callCompilerFn reading two new BuildConfig fields that main.zig
forwards before the post-link callback. New reusable VM helper makeStringList
builds a List(string) in flat memory from the call's result type offsets
(target-aware); invoke/callCompilerFn now thread ins.ty for that. Legacy
handlers bail loudly (VM-only by nature — post-link; List(string) isn't
faithfully buildable in the legacy Value model, 0141).

Smoke test examples/1662-platform-build-pipeline-queries (AOT + a 1-line C
#source → one object): a post-link callback verifies the VM-built list is
well-formed; build exit 0 only if so (negative-probe confirmed a real guard).

emit_object + link (the actions) deferred to P5.2b — they replace the Zig
driver's auto-emit/auto-link and need a host-installed callback vtable.

703/0 both gates.
This commit is contained in:
agra
2026-06-19 07:42:27 +03:00
parent 7cba33ea6d
commit 44dfdcddf9
13 changed files with 191 additions and 11 deletions

View File

@@ -86,10 +86,20 @@ with ONE welded mechanism. Branch: `reify` (off `master`). Update after every st
> in; VM bail → hard build error (`comptime_vm.last_bail_reason` surfaced by `main.printInterpBailDiag`). Smoke
> test `1661-platform-post-link-vm-list` (AOT) — a post-link callback that GROWS a `List` (0141: works on VM,
> bails on legacy with `struct_get`); build succeeds (exit 0) only via the VM. `flushInterpOutput` deleted (VM
> writes `out` direct via host-FFI). **702/0 both gates.** **NEXT — P5.2:** expose `emit_object` + `link`
> (reuse `target.zig`) + metadata queries (`c_object_paths`/`link_libraries`/host-triple) as `abi(.compiler)`
> primitives taking EXPLICIT args. Then P5.3 (`on_build` slot — invoke WITH the `BuildConfig` arg; needs a VM
> entry that marshals args, the gap `invokeByFuncId` rejects today) · P5.4 (sx `default_build` + delete
> writes `out` direct via host-FFI). **702/0 both gates.** **P5.2 metadata queries DONE (2026-06-19, newest Log
> entry):** `c_object_paths() -> List(string)` + `link_libraries() -> List(string)` are `abi(.compiler)` primitives
> (new stdlib home `library/modules/std/build.sx`), serviced by `comptime_vm.callCompilerFn` reading `BuildConfig`
> fields `main.zig` forwards (`c_object_paths`/`link_libraries`). New reusable VM helper `makeStringList` builds a
> `List(string)` in flat memory (target-aware via the result type's offsets); `invoke`/`callCompilerFn` now thread
> the call's result type (`ins.ty`). Legacy handlers bail loudly (VM-only by nature — post-link). Smoke test
> `1662-platform-build-pipeline-queries` (AOT, C companion → 1 object): a post-link callback checks the VM-built
> list is well-formed; build exit 0 ONLY if so (negative-probe verified: wrong count → "post-link callback
> returned false", exit 1). **703/0 both gates.** **NEXT — P5.2b: `emit_object` + `link` (the ACTIONS).** These
> are NOT data reads — in the end state the Zig driver stops auto-emitting/auto-linking and the sx driver calls
> them, so they need the driver-restructuring (a callback vtable the host installs into the VM, since
> `comptime_vm.zig` can't depend on `core`/`main`/`target`). `link` is fallible (`-> !`) — its failable return
> shape is the new VM piece. Then P5.3 (`on_build` slot — invoke WITH the `BuildConfig` arg; needs a VM entry
> that marshals args, the gap `invokeByFuncId` rejects today) · P5.4 (sx `default_build` + delete
> `#compiler`/`compiler_call`/`compiler_hooks` + the S5a `build_options`/`set_post_link_callback`) — P5.4 kills
> the 4 strict `compiler_call` bails (1609/1614/1615/1616).
> **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` +
@@ -403,6 +413,28 @@ when reached (sentinels or accessor fns; see the design doc Risks).
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
## Log
- **P5.2 (metadata queries) — `c_object_paths` / `link_libraries` compiler primitives + the VM `List(string)` builder (2026-06-19).**
Phase 5 step 2 (the read-only slice): two `abi(.compiler)` primitives that the sx build driver will pass to
`link` — `c_object_paths() -> List(string)` (the `#import c` companion `.o`s) and `link_libraries() -> List(string)`
(the `#library` names). They live in a NEW stdlib file `library/modules/std/build.sx` (the Phase 5 home the sx
`default_build` grows into) and are serviced by `comptime_vm.callCompilerFn` reading two new `BuildConfig`
fields (`c_object_paths`/`link_libraries`) that `main.zig` forwards before the post-link callback (alongside
`binary_path`/`target_triple`/…). **Reusable new piece:** `Vm.makeStringList(table, list_ty, items)` builds a
`List(string)` in flat memory — backing array of `string` fat pointers + the `{items,len,cap}` struct, all laid
out from the RESULT type's field offsets/types (target-aware, no hardcoded layout). To get the result type,
`invoke`/`callCompilerFn` now thread the call instruction's `ins.ty` (the only call-result-type need so far).
Legacy (`compiler_lib`) handlers for these bail loudly (`handleBuildPipelineQuery`) — they're VM-only by nature
(the post-link callback always runs on the VM since P5.1), and a `List(string)` isn't faithfully buildable in
the legacy `Value` model (0141). Registered on `bound_fns` so `weldedCompilerFn` recognizes them. **Smoke test**
`examples/1662-platform-build-pipeline-queries` (AOT + a 1-line C `#source` → exactly one C object): a post-link
callback asserts `c_object_paths().len == 1`, `items[0].len > 0`, and iterates `link_libraries()` (liveness
touch) — build exit 0 only if the VM-built list is well-formed. **Negative-probe verified** a real guard (forcing
`len != 2` → "post-link callback returned false", build exit 1). **No unit test for `makeStringList`** —
constructing a `List(string)` `TypeId` in the test harness needs generic instantiation; the corpus test
exercises the real stdlib type end-to-end with a non-empty list + a negative guard instead. **`emit_object` +
`link` (the ACTIONS) deferred to P5.2b** — they must replace the Zig driver's auto-emit/auto-link (not duplicate
it), so they need the driver-restructuring + a host-installed callback vtable (the VM can't depend on
`core`/`main`/`target`). **703/0 both gates** + strict JIT run clean (no `compiler_call` bail).
- **P5.1 (= 4E) — the post-link build driver runs on the VM (NO fallback); smoke test 1661 (2026-06-19).**
Phase 5 step 1: `core.invokeByFuncId` — the post-codegen / post-link callback invocation `main.zig` fires after
`target.link` — now routes the callback through the **comptime VM** (`comptime_vm.tryEval`) instead of the

View File

@@ -504,8 +504,15 @@ The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfi
`examples/1661-platform-post-link-vm-list` (AOT): a post-link callback GROWS a `List` (0141 — works on the VM,
bails on legacy with `struct_get`), so the build succeeds (exit 0) only via the VM. Non-empty callback `args`
rejected loudly (the `on_build(config)` arg-marshaling entry is P5.3). **702/0 both gates.**
- **P5.2 — primitives:** expose `emit_object` + `link` (reuse `target.zig` linker) + metadata queries
(`c_object_paths`/`link_libraries`/host-triple) as `abi(.compiler)` fns taking explicit args.
- **P5.2 — primitives.** Split: the read-only **metadata queries are DONE (2026-06-19)** — `c_object_paths() ->
List(string)` + `link_libraries() -> List(string)` as `abi(.compiler)` fns (stdlib `library/modules/std/build.sx`),
serviced by `comptime_vm.callCompilerFn` over `BuildConfig` fields `main.zig` forwards; new VM `makeStringList`
builds the `List(string)` in flat memory from the call's result type (`ins.ty` now threaded through
`invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both
gates. **P5.2b — the ACTIONS (`emit_object -> !string`, `link(...) -> !`, reuse `target.zig`):** still TODO —
they replace the Zig driver's auto-emit/auto-link, so they need the driver restructuring + a host-installed
callback vtable (`comptime_vm.zig` can't depend on `core`/`main`/`target`); `link`'s fallible `-> !` return is
the new VM shape.
- **P5.3 — `on_build` slot:** a comptime-assignable compiler slot (GENERALIZES today's `post_link_callback_fn`:
an assignable typed global with a stdlib default, vs a setter). `#run on_build = build;` captures the
`FuncId`; the compiler invokes it post-codegen with the CLI-derived `BuildConfig`.

View File

@@ -0,0 +1 @@
long c_marker(void) { return 42; }

View File

@@ -0,0 +1,40 @@
#import "modules/std.sx";
#import "modules/build.sx";
#import "modules/std/build.sx";
// P5.2 smoke test — the build-pipeline metadata queries (`c_object_paths` /
// `link_libraries`) run on the comptime VM and return a `List(string)` the VM
// builds in flat memory. Registered as a post-link callback (which runs on the
// VM — core.invokeByFuncId), so the lists are live there.
//
// The `#import c` source compiles to one `.o`, so `c_object_paths()` must return
// a one-element list whose single string is the (non-empty) object path. AOT
// snapshots the executed binary, so success is observed via the build exit code:
// the callback returns true only if the VM-built list is well-formed.
#import c {
#source "1662-platform-build-pipeline-queries.c";
};
c_marker :: () -> i64 extern;
check :: () -> bool abi(.compiler) {
objs := c_object_paths();
libs := link_libraries();
if objs.len != 1 { return false; }
if objs.items[0].len == 0 { return false; }
// `link_libraries` must be a well-formed (possibly empty) list — touch each
// entry so a malformed fat pointer would fault rather than pass silently.
sum : i64 = 0;
i : i64 = 0;
while i < libs.len { sum += libs.items[i].len; i += 1; }
return sum >= 0;
}
configure :: () abi(.compiler) {
opts := build_options();
opts.set_post_link_callback(check);
}
#run configure();
main :: () { print("runtime main: {}\n", c_marker()); }

View File

@@ -0,0 +1 @@
{ "aot": true }

View File

@@ -0,0 +1 @@
0

View File

@@ -0,0 +1 @@

View File

@@ -0,0 +1 @@
runtime main: 42

View File

@@ -0,0 +1,17 @@
// Phase 5 — sx-driven build pipeline (compiler primitives).
//
// These run in the comptime evaluator (`abi(.compiler)`), serviced by
// `comptime_vm.callCompilerFn`. They are called from the post-link build driver
// (a callback registered via `set_post_link_callback`), which always runs on the
// comptime VM (`core.invokeByFuncId`) — the VM, unlike the legacy interpreter,
// can allocate/grow the `List`s the driver builds (issue 0141).
//
// This is the home the sx `default_build` pipeline grows into; for now it exposes
// the read-only build metadata queries.
#import "modules/std.sx";
// The C companion object files for this build (`#import c { #source ... }`,
// compiled to `.o`) and the `#library` link names. The sx driver passes them to
// the linker. Answered from the compiler's accumulated build state.
c_object_paths :: () -> List(string) abi(.compiler);
link_libraries :: () -> List(string) abi(.compiler);

View File

@@ -57,6 +57,15 @@ pub const BuildConfig = struct {
/// sx bundler can switch on iOS vs. macOS vs. simulator.
target_triple: ?[]const u8 = null,
/// C companion object files (`#import c { #source ... }`, compiled to `.o`)
/// and `#library` link names, forwarded by main.zig before the post-link
/// callback so the sx-driven build pipeline (Phase 5) can read them via the
/// `c_object_paths()` / `link_libraries()` compiler primitives and pass them
/// to `link`. Slices reference compiler-owned memory that outlives the
/// callback.
c_object_paths: []const []const u8 = &.{},
link_libraries: []const []const u8 = &.{},
/// Frameworks the binary links against (`-framework` names) and
/// the search paths to look them up in (`-F` directories), forwarded
/// from the link step so the sx bundler can embed them into

View File

@@ -61,8 +61,25 @@ pub const bound_fns = [_]BoundFn{
// ── BuildOptions (migrated off `#compiler` onto `abi(.compiler)`) ─────────
.{ .sx_name = "build_options", .handler = handleBuildOptions },
.{ .sx_name = "set_post_link_callback", .handler = handleSetPostLinkCallback },
// ── build-pipeline metadata queries (Phase 5.2) ──────────────────────────
// VM-only: the post-link callback that calls these always runs on the VM
// (`core.invokeByFuncId`), so `comptime_vm.callCompilerFn` services them and
// these legacy handlers are never reached (they bail loudly rather than
// fabricate a silent empty List). Registered here only so `weldedCompilerFn`
// recognizes the names as compiler-API functions.
.{ .sx_name = "c_object_paths", .handler = handleBuildPipelineQuery },
.{ .sx_name = "link_libraries", .handler = handleBuildPipelineQuery },
};
/// Legacy-path stub for the Phase 5 build-pipeline metadata queries — see the
/// `bound_fns` comment. These return a `List(string)` the legacy `Value` model
/// can't faithfully build (issue 0141), and the only caller (the post-link
/// callback) runs on the VM, so bail loudly here instead of guessing.
fn handleBuildPipelineQuery(_: *Interpreter, _: []const Value) InterpError!Value {
Interpreter.last_bail_detail = "build-pipeline query (c_object_paths/link_libraries) is VM-only (post-link); not available on the legacy interpreter";
return error.CannotEvalComptime;
}
// Kind codes accepted by `register_type` — mirror `TypeTable.kindCode`. An
// enum-like type is minted as a `tagged_union` (the general payload-carrying
// form, as `define` does), so both 2 (`enum`) and 3 (`tagged_union`) are taken.

View File

@@ -909,7 +909,7 @@ pub const Vm = struct {
// ── Calls ───────────────────────────────────────────
// Direct call: resolve the static callee `FuncId` and dispatch.
.call => |c| return .{ .value = try self.invoke(c.callee, c.args, frame, ref_types) },
.call => |c| return .{ .value = try self.invoke(c.callee, c.args, frame, ref_types, ins.ty) },
// Indirect call: the callee is a `func_ref` value (its `FuncId.index()`
// as a word) in a register — e.g. an allocator protocol's `alloc_fn`.
// A null (0) function pointer can't be dispatched → bail.
@@ -919,7 +919,7 @@ pub const Vm = struct {
self.detail = "comptime VM: call_indirect through a null function pointer";
return error.Unsupported;
};
return .{ .value = try self.invoke(fid, ci.args, frame, ref_types) };
return .{ .value = try self.invoke(fid, ci.args, frame, ref_types, ins.ty) };
},
// ── Globals / function values ───────────────────────
@@ -1135,7 +1135,7 @@ pub const Vm = struct {
/// extern/bodyless callee routes to the native libc memory builtins (else
/// bails); a normal callee runs on the VM. Aggregate args pass as their Addr
/// over the shared flat memory (no copy).
fn invoke(self: *Vm, fid: inst_mod.FuncId, args: []const Ref, frame: *Frame, ref_types: []const TypeId) Error!Reg {
fn invoke(self: *Vm, fid: inst_mod.FuncId, args: []const Ref, frame: *Frame, ref_types: []const TypeId, result_ty: TypeId) Error!Reg {
const module = self.module orelse return self.failMsg("comptime VM: call needs a module (not provided)");
if (fid.index() >= module.functions.items.len) return self.failMsg("comptime VM: call to an out-of-range function id");
const callee = module.getFunction(fid);
@@ -1149,7 +1149,7 @@ pub const Vm = struct {
// the comptime compiler-API, serviced natively on flat memory (Phase 3
// seed). The `compiler_welded` flag is the safety boundary.
if (callee.compiler_welded) {
if (try self.callCompilerFn(name, args, frame, ref_types)) |r| return r;
if (try self.callCompilerFn(name, args, frame, ref_types, result_ty)) |r| return r;
}
// General host-FFI escape: any other extern resolves via dlsym and is
// dispatched through the host_ffi trampolines. Because `Addr` is a real
@@ -1336,7 +1336,7 @@ pub const Vm = struct {
return @enumFromInt(try self.argHandle(args, frame, i));
}
fn callCompilerFn(self: *Vm, name: []const u8, args: []const Ref, frame: *Frame, ref_types: []const TypeId) Error!?Reg {
fn callCompilerFn(self: *Vm, name: []const u8, args: []const Ref, frame: *Frame, ref_types: []const TypeId, result_ty: TypeId) Error!?Reg {
const table = try self.requireTable();
if (std.mem.eql(u8, name, "intern")) {
if (args.len != 1) return self.failMsg("comptime intern: expected one string arg");
@@ -1452,6 +1452,23 @@ pub const Vm = struct {
bc.post_link_callback_fn = fid;
return @as(Reg, null_addr);
}
// ── build-pipeline metadata queries (Phase 5.2) ─────────────────────
// Read-only: the compiler answers them from the `BuildConfig` `main.zig`
// forwards before the post-link callback runs. Each builds a fresh
// `List(string)` in flat memory (the result type drives its layout) — no
// driver action, so they're pure data even in the sx-driven end state.
if (std.mem.eql(u8, name, "c_object_paths")) {
if (args.len != 0) return self.failMsg("comptime c_object_paths: expected no args");
const bc = self.build_config orelse
return self.failMsg("comptime c_object_paths: no build config threaded into the VM");
return try self.makeStringList(table, result_ty, bc.c_object_paths);
}
if (std.mem.eql(u8, name, "link_libraries")) {
if (args.len != 0) return self.failMsg("comptime link_libraries: expected no args");
const bc = self.build_config orelse
return self.failMsg("comptime link_libraries: no build config threaded into the VM");
return try self.makeStringList(table, result_ty, bc.link_libraries);
}
return null; // not a known compiler function → caller bails to legacy
}
@@ -2199,4 +2216,35 @@ pub const Vm = struct {
fn sliceData(self: *Vm, table: *const types.TypeTable, base: Addr) Error!Addr {
return self.machine.readWord(base, table.pointer_size);
}
/// Build a `List(string)` aggregate in flat memory from host strings and
/// return its Addr (the VM's aggregate value IS its address). `list_ty` is
/// the result type of the calling primitive (`List(string)`); its field
/// offsets/types drive the layout (target-aware via the table), so this works
/// for any `{ items: [*]string, len: i64, cap: i64 }`-shaped struct. Used by
/// the metadata-query compiler primitives (`c_object_paths`/`link_libraries`).
fn makeStringList(self: *Vm, table: *const types.TypeTable, list_ty: TypeId, items: []const []const u8) Error!Reg {
if (list_ty.isBuiltin() or table.get(list_ty) != .@"struct")
return self.failMsg("comptime List builder: result type is not a List struct");
const str_size = table.typeSizeBytes(.string);
// Backing array of `items.len` `string` fat pointers (null when empty —
// the List's `items` is then a null `[*]string`, matching `len`/`cap` 0).
const backing: Addr = if (items.len == 0) null_addr else self.machine.allocBytes(items.len * str_size, 8);
for (items, 0..) |s, i| {
try self.writeField(table, backing + i * str_size, .string, try self.makeStringValue(table, s));
}
// The List struct: field 0 = items ([*]string), 1 = len (i64), 2 = cap (i64).
const addr = self.machine.allocBytes(table.typeSizeBytes(list_ty), 8);
const items_fty = table.memberType(list_ty, 0) orelse
return self.failMsg("comptime List builder: result type has no items field");
const len_fty = table.memberType(list_ty, 1) orelse
return self.failMsg("comptime List builder: result type has no len field");
const cap_fty = table.memberType(list_ty, 2) orelse
return self.failMsg("comptime List builder: result type has no cap field");
const n: Reg = @bitCast(@as(i64, @intCast(items.len)));
try self.writeField(table, addr + fieldOffset(table, list_ty, 0), items_fty, backing);
try self.writeField(table, addr + fieldOffset(table, list_ty, 1), len_fty, n);
try self.writeField(table, addr + fieldOffset(table, list_ty, 2), cap_fty, n);
return addr;
}
};

View File

@@ -717,6 +717,11 @@ fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []cons
if (merged_config.triple) |t| e.build_config.target_triple = std.mem.span(t);
e.build_config.target_frameworks = fws;
e.build_config.target_framework_paths = merged_config.framework_paths;
// Phase 5: the sx-driven build pipeline reads these via the
// `c_object_paths()` / `link_libraries()` compiler primitives. Both
// slices reference compileWithTimer locals that outlive the callback.
e.build_config.c_object_paths = c_obj_paths;
e.build_config.link_libraries = libs;
// Android-specific bundling state.
if (e.build_config.manifest_path == null) e.build_config.manifest_path = merged_config.manifest_path;
if (e.build_config.keystore_path == null) e.build_config.keystore_path = merged_config.keystore_path;