bundling: fs/process stdlib + post-link callback + Apple .app in sx

Campaign Weeks 3-6 of /Users/agra/.claude/plans/lets-plan-to-move-splendid-pumpkin.md
land in one push: the bundling pipeline that used to live in
src/target.zig (createBundle, embedFramework, extractEntitlements,
buildInfoPlist, codesign) now lives in
library/modules/platform/bundle.sx and runs in the IR interpreter
after target.link() returns.

New language-side surface:
- library/modules/fs.sx — POSIX libc bindings (open/read/write/close,
  mkdir/unlink/rmdir, chmod, rename, access, basename/dirname). Variadic
  open() lowers to C's varargs via the new args: ..T form. Direct libc
  calls bypass *File method dispatch so they work from the post-link
  IR interpreter.
- library/modules/process.sx — popen-based run(cmd) returning
  ProcessResult{ exit_code, stdout }, plus env() and find_executable().
- library/modules/std.sx — xml_escape(s) and variadic path_join(parts).
- library/modules/compiler.sx — BuildOptions grows
  set_post_link_callback / set_post_link_module / binary_path
  accessors; bundle_path/bundle_id/codesign_identity/provisioning_profile
  setters + accessors; per-target predicates is_macos/is_ios/
  is_ios_device/is_ios_simulator + target_triple; framework_count /
  framework_at(i) / framework_path_count / framework_path_at(i);
  add_asset_dir(src, dest) + asset_dir_count / src_at / dest_at.

Compiler-side wiring:
- src/ir/compiler_hooks.zig — BuildConfig now carries post_link_callback_fn,
  post_link_module, binary_path, bundle_*, target_triple,
  target_frameworks, target_framework_paths, asset_dirs. Hook registry
  exposes every accessor; getters return "" / 0 for unset fields so
  bundle.sx can treat absent values uniformly.
- src/ir/host_ffi.zig (new) — dlsym(RTLD_DEFAULT) + arity-switched cdecl
  trampolines so #foreign("c") declarations resolve through the host
  libc during #run / post-link interpretation.
- src/ir/interp.zig — callForeign dispatch; build_config pointer
  injection so accessor hooks see live state during re-entry.
- src/core.zig — keeps the IR module alive past generateCode; exposes
  invokeByName / invokeByFuncId so main.zig can re-enter the
  interpreter after linking.
- src/main.zig — wires bundle/codesign/provisioning CLI flags +
  target_triple + framework lists into BuildConfig; invokes the
  post-link callback (by FuncId or by <module>.bundle_main lookup) once
  target.link() returns. When --bundle is set but no callback is
  registered, auto-falls-back to post_link_module = "platform.bundle"
  so the legacy --bundle CLI keeps working for any program that imports
  modules/platform/bundle.sx.

Apple .app bundler (library/modules/platform/bundle.sx):
- Single bundle_main entry covers macOS, iOS simulator, iOS device.
  Per-target Info.plist switch keys off is_ios()/is_ios_simulator() —
  iOS emits UIDeviceFamily / LSRequiresIPhoneOS /
  UIApplicationSceneManifest / DTPlatformName (iPhoneOS or
  iPhoneSimulator); macOS emits the minimal CFBundle* set.
- iOS-only steps:
  - Provisioning embed: fs.read_file + fs.write_file to
    <bundle>/embedded.mobileprovision.
  - Framework embed: recursive cp -R per -F search path into
    <bundle>/Frameworks/<Name>.framework/ (until fs.sx grows list_dir).
  - Entitlements extraction: four process.run calls (security cms -D,
    plutil -extract Entitlements xml1, plutil -extract
    ApplicationIdentifierPrefix.0, plutil -replace application-identifier)
    resolving the wildcard <TEAM>.* -> <TEAM>.<bundle_id>.
  - Real codesign with --entitlements when present.
- Asset dirs (add_asset_dir): recursive cp -R src/. into <bundle>/dest/.
  Missing src is treated as "nothing to do" so projects can register
  add_asset_dir("assets", "assets") unconditionally.

Parser:
- parseStmt() now accepts #import \"path\"; and #framework \"Name\"; as
  statement-position tokens. Needed for top-level
  inline if OS == .android { #import \"modules/platform/android.sx\"; }
  blocks (issue-0042 flatten pass surfaces them); chess's
  inline-if-with-#import was rejected at parse time before this fix.

Removals from src/target.zig:
- createBundle, embedFramework, extractEntitlements, buildInfoPlist,
  codesign (~210 lines). main.zig no longer calls createBundle after
  link(); the sx callback is the single entry point.

Tests / regression markers (all run under sx run host JIT):
- examples/115-post-link-callback.sx — callback registration round-trip.
- examples/116-fs-roundtrip.sx — fs.write_file -> fs.read_file -> exists.
- examples/117-process-roundtrip.sx — process.run + env + find_executable.
- examples/118-macos-bundle.sx — macOS .app via bundle_main callback.
- examples/119-interp-cast-ptr-cmp.sx — cast(T) val under interpreter.
- examples/120-interp-variadic-any.sx — variadic ..Any indexing in IR
  interpreter.
- examples/121-ios-sim-bundle.sx — iOS-sim cross-compile + .app with
  iOS-shaped Info.plist (added to tests/cross_compile.sh as the
  ios-sim tuple).
- examples/122-ios-device-bundle.sx — iOS device cross-compile +
  full codesign pipeline (provisioning embed + entitlements
  extraction + --entitlements codesign). Manually verified end-to-end:
  installed via xcrun devicectl device install app + launched
  successfully on iPhone 17 Pro.
- examples/123-inline-if-import-in-body.sx — locks in the parser fix.

zig build && zig build test && bash tests/run_examples.sh => 141 passed,
0 failed; bash tests/cross_compile.sh => 7 passed, 0 failed.
This commit is contained in:
agra
2026-05-22 19:03:31 +03:00
parent 30fed66616
commit 5cc62e63c3
50 changed files with 2827 additions and 589 deletions

View File

@@ -108,6 +108,7 @@ pub const InterpError = error{
const compiler_hooks = @import("compiler_hooks.zig");
pub const BuildConfig = compiler_hooks.BuildConfig;
const host_ffi = @import("host_ffi.zig");
// ── Interpreter ─────────────────────────────────────────────────────────
@@ -130,6 +131,14 @@ pub const Interpreter = struct {
// Compiler hook registry for #compiler methods
hooks: compiler_hooks.Registry,
// First op tag that bailed with InterpError, captured the first
// time the interpreter unwinds so callers can surface "op=foo at
// <file>:<offset>" alongside the bare error name. Static so it
// survives Interpreter teardown (lifetime: program global).
pub var last_bail_op: ?[]const u8 = null;
pub var last_bail_file: ?[]const u8 = null;
pub var last_bail_offset: u32 = 0;
pub fn init(module: *const Module, alloc: Allocator) Interpreter {
var hooks = compiler_hooks.Registry.init(alloc);
hooks.registerDefaults();
@@ -217,6 +226,121 @@ pub const Interpreter = struct {
return .undef;
}
/// Marshal a single sx Value into a `usize` slot for a cdecl host call.
/// Strings are made null-terminated; pointer-like values pass their
/// underlying address. The returned `usize` is only valid for the
/// duration of this call — caller-allocated buffers are tracked in
/// `tmp` so they get freed once the call returns.
fn marshalForeignArg(self: *Interpreter, v: Value, tmp: *std.ArrayList([]u8)) !usize {
return switch (v) {
.int => |i| @bitCast(i),
.boolean => |b| @intFromBool(b),
.null_val => 0,
.heap_ptr => |hp| blk: {
const mem = self.heapSlice(hp) orelse return error.TypeError;
break :blk @intFromPtr(mem.ptr) + hp.offset;
},
.string => |s| blk: {
const buf = try self.alloc.alloc(u8, s.len + 1);
@memcpy(buf[0..s.len], s);
buf[s.len] = 0;
tmp.append(self.alloc, buf) catch return error.TypeError;
break :blk @intFromPtr(buf.ptr);
},
.aggregate => |fields| blk: {
// Fat string pointer: { ptr, len }. Pass the raw bytes
// null-terminated so libc string APIs work.
if (fields.len == 2) {
const len: usize = @intCast(fields[1].asInt() orelse return error.TypeError);
switch (fields[0]) {
.heap_ptr => |hp| {
const mem = self.heapSlice(hp) orelse return error.TypeError;
const start = hp.offset;
const slice = mem[start .. start + len];
const buf = try self.alloc.alloc(u8, len + 1);
@memcpy(buf[0..len], slice);
buf[len] = 0;
tmp.append(self.alloc, buf) catch return error.TypeError;
break :blk @intFromPtr(buf.ptr);
},
.string => |s| {
const slice = if (len <= s.len) s[0..len] else s;
const buf = try self.alloc.alloc(u8, slice.len + 1);
@memcpy(buf[0..slice.len], slice);
buf[slice.len] = 0;
tmp.append(self.alloc, buf) catch return error.TypeError;
break :blk @intFromPtr(buf.ptr);
},
else => return error.TypeError,
}
}
return error.TypeError;
},
else => error.TypeError,
};
}
fn callForeign(self: *Interpreter, func: *const inst_mod.Function, args: []const Value) InterpError!Value {
const name = self.module.types.getString(func.name);
const symbol = (host_ffi.lookupSymbol(self.alloc, name) catch return error.CannotEvalComptime) orelse {
return error.CannotEvalComptime;
};
var packed_args: [8]usize = undefined;
if (args.len > packed_args.len) return error.CannotEvalComptime;
var tmp = std.ArrayList([]u8).empty;
defer {
for (tmp.items) |buf| self.alloc.free(buf);
tmp.deinit(self.alloc);
}
for (args, 0..) |a, i| {
packed_args[i] = self.marshalForeignArg(a, &tmp) catch return error.TypeError;
}
const argv = packed_args[0..args.len];
// Variadic foreign functions (declared `args: ..T`) must be
// dispatched through C-variadic trampolines so the trailing
// args land in the right place per the target's variadic
// ABI. The fixed-arity trampolines would put them in arg
// registers, and the callee would read garbage from the
// stack.
const fixed = func.params.len;
const variadic = func.is_variadic and args.len > fixed;
const ret = func.ret;
if (ret == .void) {
if (variadic) {
host_ffi.callVoidRetVar(symbol, fixed, argv) catch return error.CannotEvalComptime;
} else {
host_ffi.callVoidRet(symbol, argv) catch return error.CannotEvalComptime;
}
return .void_val;
}
if (ret == .s8 or ret == .s16 or ret == .s32 or ret == .s64 or
ret == .u8 or ret == .u16 or ret == .u32 or ret == .u64 or
ret == .usize or ret == .isize)
{
const r = if (variadic)
host_ffi.callIntRetVar(symbol, fixed, argv) catch return error.CannotEvalComptime
else
host_ffi.callIntRet(symbol, argv) catch return error.CannotEvalComptime;
return Value{ .int = r };
}
if (ret == .bool) {
const r = if (variadic)
host_ffi.callIntRetVar(symbol, fixed, argv) catch return error.CannotEvalComptime
else
host_ffi.callIntRet(symbol, argv) catch return error.CannotEvalComptime;
return Value{ .boolean = r != 0 };
}
const r = if (variadic)
host_ffi.callPtrRetVar(symbol, fixed, argv) catch return error.CannotEvalComptime
else
host_ffi.callPtrRet(symbol, argv) catch return error.CannotEvalComptime;
return Value{ .int = @bitCast(@as(u64, r)) };
}
pub fn call(self: *Interpreter, func_id: FuncId, args: []const Value) InterpError!Value {
if (self.call_depth >= self.max_call_depth) return error.StackOverflow;
self.call_depth += 1;
@@ -224,7 +348,10 @@ pub const Interpreter = struct {
const func = self.module.getFunction(func_id);
if (func.is_extern or func.blocks.items.len == 0) {
return error.CannotEvalComptime;
// Dispatch to host libc via dlsym. Lets `#run` (and the
// post-link bundler) call ordinary foreign symbols like
// `puts`, `getenv`, `posix_spawn`, etc.
return self.callForeign(func, args);
}
// Compute total refs: params + all instructions across all blocks
@@ -269,6 +396,11 @@ pub const Interpreter = struct {
}
const result = self.execInst(instruction, &frame, &current_block, &block_args) catch |err| {
if (last_bail_op == null) {
last_bail_op = @tagName(instruction.op);
last_bail_file = func.source_file;
last_bail_offset = instruction.span.start;
}
return err;
};
switch (result) {
@@ -982,8 +1114,14 @@ pub const Interpreter = struct {
}
},
// Type-as-value sentinel emitted for the type arg of
// `cast(T) val`. Result is never read (the cast lowering
// consumes the type from the AST, not the IR Ref), so an
// undef value is sufficient — matches the LLVM emitter.
.placeholder => return .{ .value = .undef },
// ── Not yet evaluable at comptime ──────────────────
.call_closure, .protocol_call_dynamic, .protocol_erase, .closure_create, .context_load, .context_store, .context_save, .context_restore, .union_get, .union_gep, .vec_splat, .vec_extract, .vec_insert, .placeholder => {
.call_closure, .protocol_call_dynamic, .protocol_erase, .closure_create, .context_load, .context_store, .context_save, .context_restore, .union_get, .union_gep, .vec_splat, .vec_extract, .vec_insert => {
return error.CannotEvalComptime;
},
}
@@ -1187,14 +1325,13 @@ pub const Interpreter = struct {
const parent = frame.loadSlot(parent_slot);
switch (parent) {
.aggregate => |parent_fields| {
if (field_idx < parent_fields.len) {
// Clone the aggregate and update the field
const new_fields = self.alloc.alloc(Value, parent_fields.len) catch return false;
@memcpy(new_fields, parent_fields);
new_fields[field_idx] = new_val;
frame.storeSlot(parent_slot, .{ .aggregate = new_fields });
return true;
}
const new_len = @max(field_idx + 1, parent_fields.len);
const new_fields = self.alloc.alloc(Value, new_len) catch return false;
@memcpy(new_fields[0..parent_fields.len], parent_fields);
for (new_fields[parent_fields.len..]) |*f| f.* = .undef;
new_fields[field_idx] = new_val;
frame.storeSlot(parent_slot, .{ .aggregate = new_fields });
return true;
},
.undef => {
// Initialize a new aggregate from undef