P5.2b: link() build-pipeline action on the VM via a host vtable

The one genuine action primitive: link(objects, output, libraries, frameworks,
flags, target) in library/modules/std/build.sx. Per the user decision to drop
fallibility from the build callback, link is plain VOID — a link failure bails
on the VM (hard build error), no -> ! / failable-tuple needed.

comptime_vm.zig can't depend on the driver (core/main/target), so link
dispatches through a new compiler_hooks.BuildHooks { ctx, link } vtable that
main.zig installs into BuildConfig.build_hooks before the post-link callback.
The driver side is main.LinkHooksCtx (unions explicit + CLI link flags, calls
target.link). New VM readers readStringList / readStringArg (inverse of
makeStringList) decode the List(string)/string args from flat memory.

Smoke test examples/1663-platform-build-pipeline-link (AOT): a post-link
callback re-links the build's own objects (c_object_paths + emit_object) into a
temp output via sx link — the relinked binary is a functional executable that
runs. Negative-probe verified (bad path -> ld fails -> ComptimeVmBail -> build
exit 1). The Zig driver still auto-links; removing that is P5.4.

704/0 both gates.
This commit is contained in:
agra
2026-06-19 08:11:36 +03:00
parent 83de0fa04d
commit f7362ee013
12 changed files with 206 additions and 14 deletions

View File

@@ -96,15 +96,21 @@ with ONE welded mechanism. Branch: `reify` (off `master`). Update after every st
> list is well-formed; build exit 0 ONLY if so (negative-probe verified: wrong count → "post-link callback
> returned false", exit 1). **`emit_object() -> string` ALSO landed** (a QUERY — the Zig driver emits eagerly, the
> primitive returns `BuildConfig.object_path`; NO vtable). So all three QUERY primitives are done. **703/0 both
> gates.** **NEXT — P5.2b: `link(...) -> !` (the one genuine ACTION).** In the end state the Zig driver stops
> auto-linking and the sx driver calls `link`, so it needs the driver-restructuring (a callback vtable the host
> installs into the VM, since `comptime_vm.zig` can't depend on `core`/`main`/`target`) + a List(string)-arg
> READER (inverse of `makeStringList`) + the fallible `-> !` return shape (a `(value…, tag=0)` tuple, since `!T`
> is a tuple — `makeOkFailable`). Build it tested via a post-link callback linking to a TEMP output (avoids
> clobbering the real binary; the Zig driver still links until P5.4). Then P5.3 (`on_build` slot — invoke WITH the `BuildConfig` arg; needs a VM entry
> that marshals args, the gap `invokeByFuncId` rejects today) · P5.4 (sx `default_build` + delete
> `#compiler`/`compiler_call`/`compiler_hooks` + the S5a `build_options`/`set_post_link_callback`) — P5.4 kills
> the 4 strict `compiler_call` bails (1609/1614/1615/1616).
> gates.** **P5.2b (`link` ACTION) DONE (2026-06-19, newest Log entry):** `link(objects, output, libraries,
> frameworks, flags, target)` dispatches through a host-installed `compiler_hooks.BuildHooks` vtable (`main.zig`
> `LinkHooksCtx` → `target.link`); **USER DECISION: the build callback is NOT fallible** — `link` is plain VOID,
> a failure BAILS (hard build error), no `-> !`/failable-tuple needed. New VM readers `readStringList`/
> `readStringArg`. Smoke test `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's
> objects to a temp output via sx `link` — the relinked binary RUNS; negative-probe verified (bad path → bail →
> build exit 1). The Zig driver STILL auto-links (1663 links a separate temp output); removing the auto-link is
> P5.4. **704/0 both gates.** **ALL build primitives now exist** (queries `emit_object`/`c_object_paths`/
> `link_libraries` + action `link`). **NEXT — P5.3: the `on_build` slot.** A comptime-assignable compiler global
> (generalizes `post_link_callback_fn`: an assignable typed global w/ a stdlib default, vs a setter). `#run
> on_build = build;` captures the FuncId; the compiler invokes `on_build(config)` post-codegen WITH the
> CLI-derived `BuildConfig` as an ARG — which needs a VM entry that MARSHALS args (the gap `core.invokeByFuncId`
> rejects today with `ComptimeVmArgsUnsupported`). Then P5.4 (sx `default_build` that calls the primitives +
> RESTRUCTURE the driver to stop auto-emit/auto-link + delete `#compiler`/`compiler_call`/`compiler_hooks` + the
> S5a `build_options`/`set_post_link_callback`) — P5.4 kills the 4 strict `compiler_call` bails (1609/1614/1615/1616).
> **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` +
> flip strict-to-default (remove the fallback) + delete `interp.zig`/`Value` + re-express `define`/`make_enum`.
> See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + top risks (bundler test coverage).
@@ -416,6 +422,25 @@ when reached (sentinels or accessor fns; see the design doc Risks).
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
## Log
- **P5.2b (`link` ACTION) — the sx `link` primitive links on the VM via a host-installed vtable; build callback de-failable'd (2026-06-19).**
Phase 5's one genuine ACTION primitive: `link(objects, output, libraries, frameworks, flags, target)` (in
`library/modules/std/build.sx`). **USER DECISION this step: drop fallibility from the build callback** — so
`link` is a plain VOID primitive (no `-> !`), and a link failure BAILS on the VM → hard build error (sidesteps
the failable-tuple-return construction entirely). **The vtable:** `comptime_vm.zig` can't depend on the driver
(`core`/`main`/`target`), so `link` dispatches through a new `compiler_hooks.BuildHooks { ctx, link_fn }` that
`main.zig` installs into `BuildConfig.build_hooks` before the post-link callback. The driver side is
`main.LinkHooksCtx` (holds allocator/io/base_config/has_jni_main; its `link` adapter unions the explicit
`flags` with the CLI ones and calls `target.link(objects[0], objects[1..], …)` — the linker treats first-vs-rest
as equal inputs). **New VM readers** (inverse of `makeStringList`): `readStringList` (a `List(string)` arg →
`[][]const u8`, element bytes are views into stable flat-memory arena) + `readStringArg` (a `string` arg).
Registered `link` on `bound_fns` (legacy stub bails — VM-only). **Smoke test**
`examples/1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's own objects (via
`c_object_paths` + `emit_object`) into a temp output through the sx `link` primitive — and the **relinked binary
is a FUNCTIONAL executable that runs** (verified manually). Build exit 0 only if the VM-driven link succeeds;
**negative-probe verified** (bad output path → `ld` fails → `ComptimeVmBail: comptime link: linking failed`,
build exit 1 — the P5.1 VM-reason diagnostic path). **The driver still auto-links too** (P5.2b does NOT remove
the Zig driver's `target.link`; the test links to a SEPARATE temp output) — removing the auto-link + having
`on_build` drive everything is P5.3/P5.4. **704/0 both gates.**
- **P5.2 (metadata queries) — `c_object_paths` / `link_libraries` compiler primitives + the VM `List(string)` builder (2026-06-19).**
Phase 5 step 2 (the read-only slice): two `abi(.compiler)` primitives that the sx build driver will pass to
`link` — `c_object_paths() -> List(string)` (the `#import c` companion `.o`s) and `link_libraries() -> List(string)`

View File

@@ -511,11 +511,13 @@ The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfi
`invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both
gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits
the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all
three QUERY primitives are done. **P5.2b — `link(...) -> !` (the one genuine ACTION):** still TODO — it replaces
the Zig driver's auto-link, so it needs the driver restructuring + a host-installed callback vtable
(`comptime_vm.zig` can't depend on `core`/`main`/`target`) + a `List(string)`-arg reader (inverse of
`makeStringList`) + the fallible `-> !` return (a `(value…, tag=0)` tuple, since `!T` is a tuple). Test it via a
post-link callback linking to a TEMP output (the Zig driver still links until P5.4).
three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER
DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard
build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks`
vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM
readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test
`1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output —
the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0.
- **P5.3 — `on_build` slot:** a comptime-assignable compiler slot (GENERALIZES today's `post_link_callback_fn`:
an assignable typed global with a stdlib default, vs a setter). `#run on_build = build;` captures the
`FuncId`; the compiler invokes it post-codegen with the CLI-derived `BuildConfig`.

View File

@@ -0,0 +1,31 @@
#import "modules/std.sx";
#import "modules/build.sx";
#import "modules/std/build.sx";
// P5.2b smoke test — the `link` build-pipeline ACTION runs on the comptime VM,
// dispatching through the host-installed linker hook (the VM can't link itself).
// The post-link callback (which runs on the VM — core.invokeByFuncId) re-links
// the build's own objects into a temp output via the sx `link` primitive. A link
// failure bails → the build fails; success → the callback returns true and the
// build's binary runs ("runtime main"). AOT snapshots the binary, so the link's
// success is observed via the build exit code.
relink :: () -> bool abi(.compiler) {
objs := List(string).{};
cobjs := c_object_paths();
i : i64 = 0;
while i < cobjs.len { objs.append(cobjs.items[i]); i += 1; }
objs.append(emit_object());
fws := List(string).{};
flags := List(string).{};
link(objs, ".sx-tmp/1663-link-out", link_libraries(), fws, flags, "");
return true;
}
configure :: () abi(.compiler) {
opts := build_options();
opts.set_post_link_callback(relink);
}
#run configure();
main :: () { print("runtime main\n"); }

View File

@@ -0,0 +1 @@
{ "aot": true }

View File

@@ -0,0 +1 @@
0

View File

@@ -0,0 +1 @@

View File

@@ -0,0 +1 @@
runtime main

View File

@@ -20,3 +20,10 @@ link_libraries :: () -> List(string) abi(.compiler);
// eagerly; this returns its path (a query, not an action). The sx driver passes
// it to `link` alongside the C objects.
emit_object :: () -> string abi(.compiler);
// Link `objects` into `output`, with the given libraries / frameworks / link
// flags / target triple. The one genuine ACTION primitive — the compiler keeps
// the proven linker (Option B); the sx driver orchestrates. Not fallible (the
// build callback isn't): a link failure fails the build directly.
link :: (objects: List(string), output: string, libraries: List(string),
frameworks: List(string), flags: List(string), target: string) abi(.compiler);

View File

@@ -72,6 +72,13 @@ pub const BuildConfig = struct {
/// (the compiler emits the object eagerly; the primitive returns its path).
object_path: ?[]const u8 = null,
/// Host-installed callbacks for build-pipeline ACTIONS the comptime VM can't
/// perform itself (it can't depend on the driver — `core`/`main`/`target`).
/// main.zig installs this before the post-link callback; the VM's `link`
/// primitive dispatches through it. Null outside a post-link build (a `link`
/// call then bails loudly — it's a post-codegen-only action).
build_hooks: ?*const BuildHooks = null,
/// Frameworks the binary links against (`-framework` names) and
/// the search paths to look them up in (`-F` directories), forwarded
/// from the link step so the sx bundler can embed them into
@@ -106,6 +113,28 @@ pub const BuildConfig = struct {
}
};
/// Host-installed callbacks for build-pipeline ACTIONS the comptime VM dispatches
/// but can't perform itself (it must not depend on the driver: `core`/`main`/
/// `target`). main.zig builds the concrete `ctx` + functions and points
/// `BuildConfig.build_hooks` at it before invoking the post-link callback. The
/// build callback is NOT fallible (Phase 5 decision) — a failed action returns an
/// error here and the VM surfaces it as a hard build error.
pub const BuildHooks = struct {
ctx: *anyopaque,
/// Link `objects` → `output`, with the given `libraries` / `frameworks` /
/// link `flags` / `target` triple. (`objects` is the full object list; the
/// adapter splits it for the underlying linker.)
link: *const fn (
ctx: *anyopaque,
objects: []const []const u8,
output: []const u8,
libraries: []const []const u8,
frameworks: []const []const u8,
flags: []const []const u8,
target: []const u8,
) anyerror!void,
};
// ── Hook system ─────────────────────────────────────────────────────────
pub const HookError = error{

View File

@@ -70,6 +70,7 @@ pub const bound_fns = [_]BoundFn{
.{ .sx_name = "c_object_paths", .handler = handleBuildPipelineQuery },
.{ .sx_name = "link_libraries", .handler = handleBuildPipelineQuery },
.{ .sx_name = "emit_object", .handler = handleBuildPipelineQuery },
.{ .sx_name = "link", .handler = handleBuildPipelineQuery },
};
/// Legacy-path stub for the Phase 5 build-pipeline primitives — see the

View File

@@ -1480,6 +1480,27 @@ pub const Vm = struct {
return self.failMsg("comptime emit_object: no object was emitted (object_path unset)");
return try self.makeStringValue(table, path);
}
// `link(objects, output, libraries, frameworks, flags, target)` — the one
// genuine ACTION: dispatch to the host-installed linker (the VM can't link
// itself). Void return (the build callback isn't fallible — Phase 5
// decision); a link failure bails loudly → hard build error. `ref_types`
// gives each List(string) arg its concrete type for the flat-memory reader.
if (std.mem.eql(u8, name, "link")) {
if (args.len != 6) return self.failMsg("comptime link: expected (objects, output, libraries, frameworks, flags, target)");
const bc = self.build_config orelse
return self.failMsg("comptime link: no build config threaded into the VM");
const hooks = bc.build_hooks orelse
return self.failMsg("comptime link: no build hooks installed (link is a post-codegen-only action)");
const objects = try self.readStringList(table, ref_types[args[0].index()], frame.get(args[0].index()));
const output = try self.readStringArg(table, frame.get(args[1].index()));
const libraries = try self.readStringList(table, ref_types[args[2].index()], frame.get(args[2].index()));
const frameworks = try self.readStringList(table, ref_types[args[3].index()], frame.get(args[3].index()));
const flags = try self.readStringList(table, ref_types[args[4].index()], frame.get(args[4].index()));
const target_str = try self.readStringArg(table, frame.get(args[5].index()));
hooks.link(hooks.ctx, objects, output, libraries, frameworks, flags, target_str) catch
return self.failMsg("comptime link: linking failed");
return @as(Reg, null_addr); // void
}
return null; // not a known compiler function → caller bails to legacy
}
@@ -2258,4 +2279,31 @@ pub const Vm = struct {
try self.writeField(table, addr + fieldOffset(table, list_ty, 2), cap_fty, n);
return addr;
}
/// Read a `string` argument (a `{ptr, len}` fat pointer at `val`) as a host
/// `[]const u8`. The bytes are a VIEW into flat memory (Addr is a real host
/// pointer over a stable arena), valid for the duration of the call.
fn readStringArg(self: *Vm, table: *const types.TypeTable, val: Reg) Error![]const u8 {
const len: usize = @intCast(try self.sliceLen(val));
if (len == 0) return "";
return try self.machine.bytes(try self.sliceData(table, val), len);
}
/// Read a `List(string)` aggregate (at `addr`) into a host `[][]const u8` —
/// the inverse of `makeStringList`. Element string bytes are VIEWS into flat
/// memory (stable arena); the outer array is gpa-allocated (freed at
/// `Vm.deinit`). Used by the `link` primitive to read its List args.
fn readStringList(self: *Vm, table: *const types.TypeTable, list_ty: TypeId, addr: Addr) Error![]const []const u8 {
if (list_ty.isBuiltin() or table.get(list_ty) != .@"struct")
return self.failMsg("comptime List reader: arg type is not a List struct");
const items_ptr = try self.machine.readWord(addr + fieldOffset(table, list_ty, 0), table.pointer_size);
const len: usize = @intCast(try self.machine.readWord(addr + fieldOffset(table, list_ty, 1), 8));
const str_size = table.typeSizeBytes(.string);
const out = self.gpa.alloc([]const u8, len) catch return self.failMsg("comptime List reader: out of memory");
var i: usize = 0;
while (i < len) : (i += 1) {
out[i] = try self.readStringArg(table, items_ptr + i * str_size);
}
return out;
}
};

View File

@@ -568,6 +568,40 @@ fn compile(allocator: std.mem.Allocator, io: std.Io, input_path: []const u8, out
timer.printAll();
}
/// Driver-side adapter behind the `link` build-pipeline primitive (Phase 5). The
/// comptime VM can't link itself (it must not depend on `target`), so it
/// dispatches `link(...)` through a `BuildHooks` whose `ctx` is one of these. The
/// VM passes the full object list; `target.link` takes (first object, rest), but
/// treats both as plain inputs, so the split is immaterial.
const LinkHooksCtx = struct {
allocator: std.mem.Allocator,
io: std.Io,
base_config: sx.target.TargetConfig,
has_jni_main: bool,
fn link(
ctx_opaque: *anyopaque,
objects: []const []const u8,
output: []const u8,
libraries: []const []const u8,
frameworks: []const []const u8,
flags: []const []const u8,
target: []const u8,
) anyerror!void {
_ = target; // the triple is already encoded in base_config (CLI-derived);
// explicit-triple reconciliation is a P5.4 concern when sx owns the config.
const self: *LinkHooksCtx = @ptrCast(@alignCast(ctx_opaque));
if (objects.len == 0) return error.NoObjects;
var cfg = self.base_config;
// Union the explicit `flags` with the CLI-derived ones (don't drop either).
var all_flags: std.ArrayList([]const u8) = .empty;
for (self.base_config.extra_link_flags) |f| try all_flags.append(self.allocator, f);
for (flags) |f| try all_flags.append(self.allocator, f);
cfg.extra_link_flags = all_flags.items;
try sx.target.link(self.allocator, self.io, objects[0], objects[1..], output, libraries, frameworks, cfg, self.has_jni_main);
}
};
fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []const u8, output_path: []const u8, target_config: sx.target.TargetConfig, timer: *Timing, enable_cache: bool, stdlib_paths: []const []const u8) !void {
// Phase A: read + parse + resolveImports (fast: ~0.5ms)
timer.mark();
@@ -695,6 +729,16 @@ fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []cons
};
timer.record("link");
// Driver-side linker adapter behind the sx `link` primitive (Phase 5). Lives
// on this stack frame so it outlives the post-link callback invocation below.
var link_ctx = LinkHooksCtx{
.allocator = allocator,
.io = io,
.base_config = merged_config,
.has_jni_main = comp.getJniMainEmissions().len > 0,
};
var build_hooks = sx.ir.compiler_hooks.BuildHooks{ .ctx = &link_ctx, .link = LinkHooksCtx.link };
// Make the linked binary's path + bundling config visible to the
// post-link callback via `BuildOptions.binary_path()`,
// `BuildOptions.bundle_path()`, etc. CLI flags
@@ -702,6 +746,7 @@ fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []cons
// bundler doesn't need a separate code path.
if (comp.ir_emitter) |*e| {
e.build_config.binary_path = final_output;
e.build_config.build_hooks = &build_hooks;
// `--apk <path>` is a transitional alias for the bundle_path
// → post_link_module = "platform.bundle" auto-fallback. The
// sx Android bundler reads `bundle_path()` regardless of which