diff --git a/current/CHECKPOINT-COMPILER-API.md b/current/CHECKPOINT-COMPILER-API.md index 96a011bc..6cafc89e 100644 --- a/current/CHECKPOINT-COMPILER-API.md +++ b/current/CHECKPOINT-COMPILER-API.md @@ -96,15 +96,21 @@ with ONE welded mechanism. Branch: `reify` (off `master`). Update after every st > list is well-formed; build exit 0 ONLY if so (negative-probe verified: wrong count → "post-link callback > returned false", exit 1). **`emit_object() -> string` ALSO landed** (a QUERY — the Zig driver emits eagerly, the > primitive returns `BuildConfig.object_path`; NO vtable). So all three QUERY primitives are done. **703/0 both -> gates.** **NEXT — P5.2b: `link(...) -> !` (the one genuine ACTION).** In the end state the Zig driver stops -> auto-linking and the sx driver calls `link`, so it needs the driver-restructuring (a callback vtable the host -> installs into the VM, since `comptime_vm.zig` can't depend on `core`/`main`/`target`) + a List(string)-arg -> READER (inverse of `makeStringList`) + the fallible `-> !` return shape (a `(value…, tag=0)` tuple, since `!T` -> is a tuple — `makeOkFailable`). Build it tested via a post-link callback linking to a TEMP output (avoids -> clobbering the real binary; the Zig driver still links until P5.4). Then P5.3 (`on_build` slot — invoke WITH the `BuildConfig` arg; needs a VM entry -> that marshals args, the gap `invokeByFuncId` rejects today) · P5.4 (sx `default_build` + delete -> `#compiler`/`compiler_call`/`compiler_hooks` + the S5a `build_options`/`set_post_link_callback`) — P5.4 kills -> the 4 strict `compiler_call` bails (1609/1614/1615/1616). +> gates.** **P5.2b (`link` ACTION) DONE (2026-06-19, newest Log entry):** `link(objects, output, libraries, +> frameworks, flags, target)` dispatches through a host-installed `compiler_hooks.BuildHooks` vtable (`main.zig` +> `LinkHooksCtx` → `target.link`); **USER DECISION: the build callback is NOT fallible** — `link` is plain VOID, +> a failure BAILS (hard build error), no `-> !`/failable-tuple needed. New VM readers `readStringList`/ +> `readStringArg`. Smoke test `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's +> objects to a temp output via sx `link` — the relinked binary RUNS; negative-probe verified (bad path → bail → +> build exit 1). The Zig driver STILL auto-links (1663 links a separate temp output); removing the auto-link is +> P5.4. **704/0 both gates.** **ALL build primitives now exist** (queries `emit_object`/`c_object_paths`/ +> `link_libraries` + action `link`). **NEXT — P5.3: the `on_build` slot.** A comptime-assignable compiler global +> (generalizes `post_link_callback_fn`: an assignable typed global w/ a stdlib default, vs a setter). `#run +> on_build = build;` captures the FuncId; the compiler invokes `on_build(config)` post-codegen WITH the +> CLI-derived `BuildConfig` as an ARG — which needs a VM entry that MARSHALS args (the gap `core.invokeByFuncId` +> rejects today with `ComptimeVmArgsUnsupported`). Then P5.4 (sx `default_build` that calls the primitives + +> RESTRUCTURE the driver to stop auto-emit/auto-link + delete `#compiler`/`compiler_call`/`compiler_hooks` + the +> S5a `build_options`/`set_post_link_callback`) — P5.4 kills the 4 strict `compiler_call` bails (1609/1614/1615/1616). > **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` + > flip strict-to-default (remove the fallback) + delete `interp.zig`/`Value` + re-express `define`/`make_enum`. > See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + top risks (bundler test coverage). @@ -416,6 +422,25 @@ when reached (sentinels or accessor fns; see the design doc Risks). `List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.) ## Log +- **P5.2b (`link` ACTION) — the sx `link` primitive links on the VM via a host-installed vtable; build callback de-failable'd (2026-06-19).** + Phase 5's one genuine ACTION primitive: `link(objects, output, libraries, frameworks, flags, target)` (in + `library/modules/std/build.sx`). **USER DECISION this step: drop fallibility from the build callback** — so + `link` is a plain VOID primitive (no `-> !`), and a link failure BAILS on the VM → hard build error (sidesteps + the failable-tuple-return construction entirely). **The vtable:** `comptime_vm.zig` can't depend on the driver + (`core`/`main`/`target`), so `link` dispatches through a new `compiler_hooks.BuildHooks { ctx, link_fn }` that + `main.zig` installs into `BuildConfig.build_hooks` before the post-link callback. The driver side is + `main.LinkHooksCtx` (holds allocator/io/base_config/has_jni_main; its `link` adapter unions the explicit + `flags` with the CLI ones and calls `target.link(objects[0], objects[1..], …)` — the linker treats first-vs-rest + as equal inputs). **New VM readers** (inverse of `makeStringList`): `readStringList` (a `List(string)` arg → + `[][]const u8`, element bytes are views into stable flat-memory arena) + `readStringArg` (a `string` arg). + Registered `link` on `bound_fns` (legacy stub bails — VM-only). **Smoke test** + `examples/1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's own objects (via + `c_object_paths` + `emit_object`) into a temp output through the sx `link` primitive — and the **relinked binary + is a FUNCTIONAL executable that runs** (verified manually). Build exit 0 only if the VM-driven link succeeds; + **negative-probe verified** (bad output path → `ld` fails → `ComptimeVmBail: comptime link: linking failed`, + build exit 1 — the P5.1 VM-reason diagnostic path). **The driver still auto-links too** (P5.2b does NOT remove + the Zig driver's `target.link`; the test links to a SEPARATE temp output) — removing the auto-link + having + `on_build` drive everything is P5.3/P5.4. **704/0 both gates.** - **P5.2 (metadata queries) — `c_object_paths` / `link_libraries` compiler primitives + the VM `List(string)` builder (2026-06-19).** Phase 5 step 2 (the read-only slice): two `abi(.compiler)` primitives that the sx build driver will pass to `link` — `c_object_paths() -> List(string)` (the `#import c` companion `.o`s) and `link_libraries() -> List(string)` diff --git a/current/PLAN-COMPILER-VM.md b/current/PLAN-COMPILER-VM.md index c935ddf7..38c7faf6 100644 --- a/current/PLAN-COMPILER-VM.md +++ b/current/PLAN-COMPILER-VM.md @@ -511,11 +511,13 @@ The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfi `invoke`/`callCompilerFn`). Smoke test `1662-platform-build-pipeline-queries` (AOT + C companion). 703/0 both gates. **`emit_object() -> string` is also DONE (2026-06-19)** as a QUERY (not an action): the Zig driver emits the object eagerly, so the primitive just returns the path from `BuildConfig.object_path` (no vtable). So all - three QUERY primitives are done. **P5.2b — `link(...) -> !` (the one genuine ACTION):** still TODO — it replaces - the Zig driver's auto-link, so it needs the driver restructuring + a host-installed callback vtable - (`comptime_vm.zig` can't depend on `core`/`main`/`target`) + a `List(string)`-arg reader (inverse of - `makeStringList`) + the fallible `-> !` return (a `(value…, tag=0)` tuple, since `!T` is a tuple). Test it via a - post-link callback linking to a TEMP output (the Zig driver still links until P5.4). + three QUERY primitives are done. **P5.2b — `link(...)` (the one genuine ACTION) — DONE (2026-06-19).** USER + DECISION: the build callback is NOT fallible, so `link` is plain VOID (no `-> !`) and a failure BAILS (hard + build error) — no failable-tuple construction. It dispatches through a host-installed `compiler_hooks.BuildHooks` + vtable (`comptime_vm.zig` can't depend on the driver); `main.LinkHooksCtx.link` adapts to `target.link`. New VM + readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test + `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output — + the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0. - **P5.3 — `on_build` slot:** a comptime-assignable compiler slot (GENERALIZES today's `post_link_callback_fn`: an assignable typed global with a stdlib default, vs a setter). `#run on_build = build;` captures the `FuncId`; the compiler invokes it post-codegen with the CLI-derived `BuildConfig`. diff --git a/examples/1663-platform-build-pipeline-link.sx b/examples/1663-platform-build-pipeline-link.sx new file mode 100644 index 00000000..68a0de65 --- /dev/null +++ b/examples/1663-platform-build-pipeline-link.sx @@ -0,0 +1,31 @@ +#import "modules/std.sx"; +#import "modules/build.sx"; +#import "modules/std/build.sx"; + +// P5.2b smoke test — the `link` build-pipeline ACTION runs on the comptime VM, +// dispatching through the host-installed linker hook (the VM can't link itself). +// The post-link callback (which runs on the VM — core.invokeByFuncId) re-links +// the build's own objects into a temp output via the sx `link` primitive. A link +// failure bails → the build fails; success → the callback returns true and the +// build's binary runs ("runtime main"). AOT snapshots the binary, so the link's +// success is observed via the build exit code. + +relink :: () -> bool abi(.compiler) { + objs := List(string).{}; + cobjs := c_object_paths(); + i : i64 = 0; + while i < cobjs.len { objs.append(cobjs.items[i]); i += 1; } + objs.append(emit_object()); + fws := List(string).{}; + flags := List(string).{}; + link(objs, ".sx-tmp/1663-link-out", link_libraries(), fws, flags, ""); + return true; +} + +configure :: () abi(.compiler) { + opts := build_options(); + opts.set_post_link_callback(relink); +} +#run configure(); + +main :: () { print("runtime main\n"); } diff --git a/examples/expected/1663-platform-build-pipeline-link.build b/examples/expected/1663-platform-build-pipeline-link.build new file mode 100644 index 00000000..40462038 --- /dev/null +++ b/examples/expected/1663-platform-build-pipeline-link.build @@ -0,0 +1 @@ +{ "aot": true } diff --git a/examples/expected/1663-platform-build-pipeline-link.exit b/examples/expected/1663-platform-build-pipeline-link.exit new file mode 100644 index 00000000..573541ac --- /dev/null +++ b/examples/expected/1663-platform-build-pipeline-link.exit @@ -0,0 +1 @@ +0 diff --git a/examples/expected/1663-platform-build-pipeline-link.stderr b/examples/expected/1663-platform-build-pipeline-link.stderr new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/examples/expected/1663-platform-build-pipeline-link.stderr @@ -0,0 +1 @@ + diff --git a/examples/expected/1663-platform-build-pipeline-link.stdout b/examples/expected/1663-platform-build-pipeline-link.stdout new file mode 100644 index 00000000..6ce5eca1 --- /dev/null +++ b/examples/expected/1663-platform-build-pipeline-link.stdout @@ -0,0 +1 @@ +runtime main diff --git a/library/modules/std/build.sx b/library/modules/std/build.sx index d9832a19..bb804a1b 100644 --- a/library/modules/std/build.sx +++ b/library/modules/std/build.sx @@ -20,3 +20,10 @@ link_libraries :: () -> List(string) abi(.compiler); // eagerly; this returns its path (a query, not an action). The sx driver passes // it to `link` alongside the C objects. emit_object :: () -> string abi(.compiler); + +// Link `objects` into `output`, with the given libraries / frameworks / link +// flags / target triple. The one genuine ACTION primitive — the compiler keeps +// the proven linker (Option B); the sx driver orchestrates. Not fallible (the +// build callback isn't): a link failure fails the build directly. +link :: (objects: List(string), output: string, libraries: List(string), + frameworks: List(string), flags: List(string), target: string) abi(.compiler); diff --git a/src/ir/compiler_hooks.zig b/src/ir/compiler_hooks.zig index 43a60b76..c631adcf 100644 --- a/src/ir/compiler_hooks.zig +++ b/src/ir/compiler_hooks.zig @@ -72,6 +72,13 @@ pub const BuildConfig = struct { /// (the compiler emits the object eagerly; the primitive returns its path). object_path: ?[]const u8 = null, + /// Host-installed callbacks for build-pipeline ACTIONS the comptime VM can't + /// perform itself (it can't depend on the driver — `core`/`main`/`target`). + /// main.zig installs this before the post-link callback; the VM's `link` + /// primitive dispatches through it. Null outside a post-link build (a `link` + /// call then bails loudly — it's a post-codegen-only action). + build_hooks: ?*const BuildHooks = null, + /// Frameworks the binary links against (`-framework` names) and /// the search paths to look them up in (`-F` directories), forwarded /// from the link step so the sx bundler can embed them into @@ -106,6 +113,28 @@ pub const BuildConfig = struct { } }; +/// Host-installed callbacks for build-pipeline ACTIONS the comptime VM dispatches +/// but can't perform itself (it must not depend on the driver: `core`/`main`/ +/// `target`). main.zig builds the concrete `ctx` + functions and points +/// `BuildConfig.build_hooks` at it before invoking the post-link callback. The +/// build callback is NOT fallible (Phase 5 decision) — a failed action returns an +/// error here and the VM surfaces it as a hard build error. +pub const BuildHooks = struct { + ctx: *anyopaque, + /// Link `objects` → `output`, with the given `libraries` / `frameworks` / + /// link `flags` / `target` triple. (`objects` is the full object list; the + /// adapter splits it for the underlying linker.) + link: *const fn ( + ctx: *anyopaque, + objects: []const []const u8, + output: []const u8, + libraries: []const []const u8, + frameworks: []const []const u8, + flags: []const []const u8, + target: []const u8, + ) anyerror!void, +}; + // ── Hook system ───────────────────────────────────────────────────────── pub const HookError = error{ diff --git a/src/ir/compiler_lib.zig b/src/ir/compiler_lib.zig index 35d5e5b4..9a86c3c2 100644 --- a/src/ir/compiler_lib.zig +++ b/src/ir/compiler_lib.zig @@ -70,6 +70,7 @@ pub const bound_fns = [_]BoundFn{ .{ .sx_name = "c_object_paths", .handler = handleBuildPipelineQuery }, .{ .sx_name = "link_libraries", .handler = handleBuildPipelineQuery }, .{ .sx_name = "emit_object", .handler = handleBuildPipelineQuery }, + .{ .sx_name = "link", .handler = handleBuildPipelineQuery }, }; /// Legacy-path stub for the Phase 5 build-pipeline primitives — see the diff --git a/src/ir/comptime_vm.zig b/src/ir/comptime_vm.zig index 150cef1f..f1a600d8 100644 --- a/src/ir/comptime_vm.zig +++ b/src/ir/comptime_vm.zig @@ -1480,6 +1480,27 @@ pub const Vm = struct { return self.failMsg("comptime emit_object: no object was emitted (object_path unset)"); return try self.makeStringValue(table, path); } + // `link(objects, output, libraries, frameworks, flags, target)` — the one + // genuine ACTION: dispatch to the host-installed linker (the VM can't link + // itself). Void return (the build callback isn't fallible — Phase 5 + // decision); a link failure bails loudly → hard build error. `ref_types` + // gives each List(string) arg its concrete type for the flat-memory reader. + if (std.mem.eql(u8, name, "link")) { + if (args.len != 6) return self.failMsg("comptime link: expected (objects, output, libraries, frameworks, flags, target)"); + const bc = self.build_config orelse + return self.failMsg("comptime link: no build config threaded into the VM"); + const hooks = bc.build_hooks orelse + return self.failMsg("comptime link: no build hooks installed (link is a post-codegen-only action)"); + const objects = try self.readStringList(table, ref_types[args[0].index()], frame.get(args[0].index())); + const output = try self.readStringArg(table, frame.get(args[1].index())); + const libraries = try self.readStringList(table, ref_types[args[2].index()], frame.get(args[2].index())); + const frameworks = try self.readStringList(table, ref_types[args[3].index()], frame.get(args[3].index())); + const flags = try self.readStringList(table, ref_types[args[4].index()], frame.get(args[4].index())); + const target_str = try self.readStringArg(table, frame.get(args[5].index())); + hooks.link(hooks.ctx, objects, output, libraries, frameworks, flags, target_str) catch + return self.failMsg("comptime link: linking failed"); + return @as(Reg, null_addr); // void + } return null; // not a known compiler function → caller bails to legacy } @@ -2258,4 +2279,31 @@ pub const Vm = struct { try self.writeField(table, addr + fieldOffset(table, list_ty, 2), cap_fty, n); return addr; } + + /// Read a `string` argument (a `{ptr, len}` fat pointer at `val`) as a host + /// `[]const u8`. The bytes are a VIEW into flat memory (Addr is a real host + /// pointer over a stable arena), valid for the duration of the call. + fn readStringArg(self: *Vm, table: *const types.TypeTable, val: Reg) Error![]const u8 { + const len: usize = @intCast(try self.sliceLen(val)); + if (len == 0) return ""; + return try self.machine.bytes(try self.sliceData(table, val), len); + } + + /// Read a `List(string)` aggregate (at `addr`) into a host `[][]const u8` — + /// the inverse of `makeStringList`. Element string bytes are VIEWS into flat + /// memory (stable arena); the outer array is gpa-allocated (freed at + /// `Vm.deinit`). Used by the `link` primitive to read its List args. + fn readStringList(self: *Vm, table: *const types.TypeTable, list_ty: TypeId, addr: Addr) Error![]const []const u8 { + if (list_ty.isBuiltin() or table.get(list_ty) != .@"struct") + return self.failMsg("comptime List reader: arg type is not a List struct"); + const items_ptr = try self.machine.readWord(addr + fieldOffset(table, list_ty, 0), table.pointer_size); + const len: usize = @intCast(try self.machine.readWord(addr + fieldOffset(table, list_ty, 1), 8)); + const str_size = table.typeSizeBytes(.string); + const out = self.gpa.alloc([]const u8, len) catch return self.failMsg("comptime List reader: out of memory"); + var i: usize = 0; + while (i < len) : (i += 1) { + out[i] = try self.readStringArg(table, items_ptr + i * str_size); + } + return out; + } }; diff --git a/src/main.zig b/src/main.zig index 0a8b4c0e..7963eacf 100644 --- a/src/main.zig +++ b/src/main.zig @@ -568,6 +568,40 @@ fn compile(allocator: std.mem.Allocator, io: std.Io, input_path: []const u8, out timer.printAll(); } +/// Driver-side adapter behind the `link` build-pipeline primitive (Phase 5). The +/// comptime VM can't link itself (it must not depend on `target`), so it +/// dispatches `link(...)` through a `BuildHooks` whose `ctx` is one of these. The +/// VM passes the full object list; `target.link` takes (first object, rest), but +/// treats both as plain inputs, so the split is immaterial. +const LinkHooksCtx = struct { + allocator: std.mem.Allocator, + io: std.Io, + base_config: sx.target.TargetConfig, + has_jni_main: bool, + + fn link( + ctx_opaque: *anyopaque, + objects: []const []const u8, + output: []const u8, + libraries: []const []const u8, + frameworks: []const []const u8, + flags: []const []const u8, + target: []const u8, + ) anyerror!void { + _ = target; // the triple is already encoded in base_config (CLI-derived); + // explicit-triple reconciliation is a P5.4 concern when sx owns the config. + const self: *LinkHooksCtx = @ptrCast(@alignCast(ctx_opaque)); + if (objects.len == 0) return error.NoObjects; + var cfg = self.base_config; + // Union the explicit `flags` with the CLI-derived ones (don't drop either). + var all_flags: std.ArrayList([]const u8) = .empty; + for (self.base_config.extra_link_flags) |f| try all_flags.append(self.allocator, f); + for (flags) |f| try all_flags.append(self.allocator, f); + cfg.extra_link_flags = all_flags.items; + try sx.target.link(self.allocator, self.io, objects[0], objects[1..], output, libraries, frameworks, cfg, self.has_jni_main); + } +}; + fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []const u8, output_path: []const u8, target_config: sx.target.TargetConfig, timer: *Timing, enable_cache: bool, stdlib_paths: []const []const u8) !void { // Phase A: read + parse + resolveImports (fast: ~0.5ms) timer.mark(); @@ -695,6 +729,16 @@ fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []cons }; timer.record("link"); + // Driver-side linker adapter behind the sx `link` primitive (Phase 5). Lives + // on this stack frame so it outlives the post-link callback invocation below. + var link_ctx = LinkHooksCtx{ + .allocator = allocator, + .io = io, + .base_config = merged_config, + .has_jni_main = comp.getJniMainEmissions().len > 0, + }; + var build_hooks = sx.ir.compiler_hooks.BuildHooks{ .ctx = &link_ctx, .link = LinkHooksCtx.link }; + // Make the linked binary's path + bundling config visible to the // post-link callback via `BuildOptions.binary_path()`, // `BuildOptions.bundle_path()`, etc. CLI flags @@ -702,6 +746,7 @@ fn compileWithTimer(allocator: std.mem.Allocator, io: std.Io, input_path: []cons // bundler doesn't need a separate code path. if (comp.ir_emitter) |*e| { e.build_config.binary_path = final_output; + e.build_config.build_hooks = &build_hooks; // `--apk ` is a transitional alias for the bundle_path // → post_link_module = "platform.bundle" auto-fallback. The // sx Android bundler reads `bundle_path()` regardless of which