bundling: Android APK pipeline moved into sx; android.sx state-on-plat
Week 7 of /Users/agra/.claude/plans/lets-plan-to-move-splendid-pumpkin.md
plus the android.sx refactor + three sx-compiler fixes hit along the way
to get chess on Pixel 7 Pro responding to touch end-to-end.
library/modules/platform/bundle.sx now covers the Android APK shape
alongside macOS / iOS-sim / iOS-device. `android_bundle_main` discovers
the SDK ($ANDROID_HOME / $ANDROID_SDK_ROOT / $HOME/Library/Android/sdk),
picks the highest-versioned build-tools + platforms via
`process.run("ls .. | sort -V | tail -1")`, stages
`<apk>.stage/lib/arm64-v8a/<libfoo.so>`, synthesizes
AndroidManifest.xml (NativeActivity vs `#jni_main` Activity branch),
writes each `#jni_main` decl's Java source under
`<stage>/java/<pkg>/<Cls>.java`, runs javac --release 11 + d8 to
produce classes.dex, aapt2-links the unaligned APK, appends lib/ +
classes.dex + each registered asset tree via zip, zipalign + ensure
debug keystore via keytool + apksigner sign.
Compiler-side accessors (src/ir/compiler_hooks.zig + library/modules/compiler.sx):
- is_android predicate.
- set_manifest_path / manifest_path + set_keystore_path / keystore_path.
- jni_main_count / jni_main_foreign_path_at(i) /
jni_main_java_source_at(i) surface the `#jni_main` emissions that
the Zig createApk previously consumed directly.
- main.zig wires manifest_path, keystore_path, and the per-decl
(foreign_path, java_source) parallel slices into BuildConfig before
invoking the post-link callback.
CLI `--apk <path>` keeps working as a transitional alias: it now feeds
bundle_path so the existing auto-`post_link_module = "platform.bundle"`
shim fires the same way as `--bundle`. main.zig no longer calls
target.createApk directly.
Deletions in src/target.zig: createApk, compileJniMainSources,
buildJniMainManifest, buildAndroidManifest, ensureDebugKeystore,
libNameFromSoBasename, plus helpers splitForeignPath / discoverJavac /
discoverAndroidSdk / findHighestSubdir / runProcess / runProcessIn
(~400 lines). git grep returns only the obituary comment.
library/modules/platform/android.sx refactor (chess Android dependency):
- Module-level globals retired (g_app_window, g_egl_*, g_viewport_*,
g_dpi_scale, g_should_stop, g_render_thread*, g_user_main_fn,
g_touch_*) → AndroidPlatform struct fields.
- All sx_android_* helpers take `plat: *AndroidPlatform` as first arg.
Render thread receives plat via pthread_create's arg.
- New `logical_w: f32 = 0.0` field. Consumers set it before init() to
define the design width in points; `recompute_scale` derives
`dpi_scale = pixel_w / logical_w` (or 1.0 if unset). Called on
init / set_viewport / egl_init. drain_touches divides incoming
physical pixel coords by dpi_scale so chess sees logical-space
positions matching its layout. Touch lands on the right squares.
Three sx-compiler bugs hit + fixed along the way:
1. Top-level `inline if OS == .X { decls }` body decls were silently
dropped because scanDecls/lowerDecls had no .if_expr arm. New
`flattenComptimeConditionals` pre-pass in src/imports.zig
(threaded via ComptimeContext from core.zig) hoists matching arms
recursively. Regression at examples/124-inline-if-hoist-toplevel.sx.
2. Parser rejected `#import` / `#framework` inside inline-if bodies
because parseStmt in src/parser.zig only had arms for `#insert`.
Added the missing arms. Regression at
examples/123-inline-if-import-in-body.sx (landed earlier).
3. JNI `Call<T>Method` switches in src/ir/emit_llvm.zig (instance /
nonvirtual / static) were missing `.f32` rows — jfloat returns
(e.g. MotionEvent.getX/getY) fell into the silent-undef else arm.
Chess's sx_android_push_touch(plat, getAction(), getX(), getY())
delivered garbage f32 coords to the touch ring, so taps landed
nowhere recognisable. Added `.f32 => Jni.Call{Static,Nonvirtual,}FloatMethod`
rows to all three switches; lifted unsupported-type detection
from emit_llvm into lowerForeignMethodCall with proper
source-spanned diagnostics (`isJniReturnTypeSupported`). Regressions
at examples/ffi-jni-call-10-jfloat-return.sx,
examples/ffi-jni-class-09-multi-float-args.sx,
examples/ffi-jni-call-11-unsupported-return-diag.sx.
Stale-snapshot drift in tests/expected/ffi-objc-call-03-selector-sharing.ir
and ffi-objc-call-06-sret-return.ir picks up the new BuildOptions
accessor extern decls (is_android, set_manifest_path,
set_keystore_path, jni_main_count, jni_main_foreign_path_at,
jni_main_java_source_at). Verified diff is dead-decl-only.
Chess on Pixel 7 Pro: tap on e2 white pawn -> yellow selection +
green dots on legal e3/e4 targets; tap on e4 -> board updates with
1. e4, "Black to move" + "1. e4" in info panel.
zig build && zig build test && bash tests/run_examples.sh -> 145/145
green. bash tests/cross_compile.sh -> 7/7 green.
This commit is contained in:
@@ -64,6 +64,26 @@ pub const BuildConfig = struct {
|
||||
target_frameworks: []const []const u8 = &.{},
|
||||
target_framework_paths: []const []const u8 = &.{},
|
||||
|
||||
/// User-supplied `AndroidManifest.xml` override (`--manifest <path>`
|
||||
/// or `BuildOptions.set_manifest_path("...")`). When null, the
|
||||
/// Android bundler synthesizes a default manifest.
|
||||
manifest_path: ?[]const u8 = null,
|
||||
/// User-supplied debug keystore path (`--keystore <path>` or
|
||||
/// `BuildOptions.set_keystore_path("...")`). When null, the Android
|
||||
/// bundler uses `$HOME/.android/debug.keystore` (auto-generated on
|
||||
/// first use via `keytool`).
|
||||
keystore_path: ?[]const u8 = null,
|
||||
|
||||
/// `#jni_main #jni_class("path") { ... }` decls discovered during
|
||||
/// lowering, paired with their pre-rendered Java source. The
|
||||
/// Android bundler writes each entry to
|
||||
/// `<stage>/java/<pkg>/<Class>.java`, compiles via `javac` + `d8`,
|
||||
/// and bundles the resulting `classes.dex` into the APK. Slices
|
||||
/// reference compiler-owned memory that outlives the post-link
|
||||
/// callback.
|
||||
jni_main_foreign_paths: []const []const u8 = &.{},
|
||||
jni_main_java_sources: []const []const u8 = &.{},
|
||||
|
||||
pub fn deinit(self: *BuildConfig, alloc: Allocator) void {
|
||||
self.link_flags.deinit(alloc);
|
||||
self.frameworks.deinit(alloc);
|
||||
@@ -126,12 +146,22 @@ pub const Registry = struct {
|
||||
self.hooks.put("BuildOptions.bundle_id", &hookGetBundleId) catch {};
|
||||
self.hooks.put("BuildOptions.codesign_identity", &hookGetCodesignIdentity) catch {};
|
||||
self.hooks.put("BuildOptions.provisioning_profile", &hookGetProvisioningProfile) catch {};
|
||||
// Target accessors — mirror TargetConfig.is{MacOS,IOS,IOSDevice,IOSSimulator}()
|
||||
// Target accessors — mirror TargetConfig.is{MacOS,IOS,IOSDevice,IOSSimulator,Android}()
|
||||
self.hooks.put("BuildOptions.target_triple", &hookGetTargetTriple) catch {};
|
||||
self.hooks.put("BuildOptions.is_macos", &hookIsMacOS) catch {};
|
||||
self.hooks.put("BuildOptions.is_ios", &hookIsIOS) catch {};
|
||||
self.hooks.put("BuildOptions.is_ios_device", &hookIsIOSDevice) catch {};
|
||||
self.hooks.put("BuildOptions.is_ios_simulator", &hookIsIOSSimulator) catch {};
|
||||
self.hooks.put("BuildOptions.is_android", &hookIsAndroid) catch {};
|
||||
// Android-specific setters + accessors
|
||||
self.hooks.put("BuildOptions.set_manifest_path", &hookSetManifestPath) catch {};
|
||||
self.hooks.put("BuildOptions.manifest_path", &hookGetManifestPath) catch {};
|
||||
self.hooks.put("BuildOptions.set_keystore_path", &hookSetKeystorePath) catch {};
|
||||
self.hooks.put("BuildOptions.keystore_path", &hookGetKeystorePath) catch {};
|
||||
// #jni_main class emissions, exposed by index so bundle.sx can iterate.
|
||||
self.hooks.put("BuildOptions.jni_main_count", &hookJniMainCount) catch {};
|
||||
self.hooks.put("BuildOptions.jni_main_foreign_path_at", &hookJniMainForeignPathAt) catch {};
|
||||
self.hooks.put("BuildOptions.jni_main_java_source_at", &hookJniMainJavaSourceAt) catch {};
|
||||
// Framework list accessors (for `.app/Frameworks/` embedding)
|
||||
self.hooks.put("BuildOptions.framework_count", &hookFrameworkCount) catch {};
|
||||
self.hooks.put("BuildOptions.framework_at", &hookFrameworkAt) catch {};
|
||||
@@ -408,6 +438,60 @@ fn hookIsIOSSimulator(_: *const Interpreter, _: []const Value, bc: *BuildConfig,
|
||||
return Value{ .boolean = ios and sim };
|
||||
}
|
||||
|
||||
fn hookIsAndroid(_: *const Interpreter, _: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
return Value{ .boolean = tripleContains(bc.target_triple, "android") };
|
||||
}
|
||||
|
||||
// ── Android-specific bundling setters + accessors ─────────────────────
|
||||
|
||||
fn hookSetManifestPath(interp: *const Interpreter, args: []const Value, bc: *BuildConfig, alloc: Allocator) HookError!Value {
|
||||
if (args.len < 2) return .void_val;
|
||||
if (args[1].asString(interp)) |s| {
|
||||
bc.manifest_path = alloc.dupe(u8, s) catch return error.CannotEvalComptime;
|
||||
}
|
||||
return .void_val;
|
||||
}
|
||||
|
||||
fn hookGetManifestPath(_: *const Interpreter, _: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
return Value{ .string = bc.manifest_path orelse "" };
|
||||
}
|
||||
|
||||
fn hookSetKeystorePath(interp: *const Interpreter, args: []const Value, bc: *BuildConfig, alloc: Allocator) HookError!Value {
|
||||
if (args.len < 2) return .void_val;
|
||||
if (args[1].asString(interp)) |s| {
|
||||
bc.keystore_path = alloc.dupe(u8, s) catch return error.CannotEvalComptime;
|
||||
}
|
||||
return .void_val;
|
||||
}
|
||||
|
||||
fn hookGetKeystorePath(_: *const Interpreter, _: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
return Value{ .string = bc.keystore_path orelse "" };
|
||||
}
|
||||
|
||||
// ── #jni_main emission accessors ──────────────────────────────────────
|
||||
// The Android bundler walks these as `0..jni_main_count()` and reads
|
||||
// each entry's `(foreign_path, java_source)` pair so it can write a
|
||||
// `.java` file per decl, compile via javac, and produce classes.dex
|
||||
// via d8 before zipping into the APK.
|
||||
|
||||
fn hookJniMainCount(_: *const Interpreter, _: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
return Value{ .int = @intCast(bc.jni_main_foreign_paths.len) };
|
||||
}
|
||||
|
||||
fn hookJniMainForeignPathAt(_: *const Interpreter, args: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
if (args.len < 2) return Value{ .string = "" };
|
||||
const idx = args[1].asInt() orelse return error.TypeError;
|
||||
if (idx < 0 or @as(usize, @intCast(idx)) >= bc.jni_main_foreign_paths.len) return Value{ .string = "" };
|
||||
return Value{ .string = bc.jni_main_foreign_paths[@intCast(idx)] };
|
||||
}
|
||||
|
||||
fn hookJniMainJavaSourceAt(_: *const Interpreter, args: []const Value, bc: *BuildConfig, _: Allocator) HookError!Value {
|
||||
if (args.len < 2) return Value{ .string = "" };
|
||||
const idx = args[1].asInt() orelse return error.TypeError;
|
||||
if (idx < 0 or @as(usize, @intCast(idx)) >= bc.jni_main_java_sources.len) return Value{ .string = "" };
|
||||
return Value{ .string = bc.jni_main_java_sources[@intCast(idx)] };
|
||||
}
|
||||
|
||||
// ── Framework list accessors ──────────────────────────────────────────
|
||||
// The Apple .app bundler in `library/modules/platform/bundle.sx` walks
|
||||
// the framework list to recursively copy each `<Name>.framework`
|
||||
|
||||
@@ -1288,6 +1288,7 @@ pub const LLVMEmitter = struct {
|
||||
.void => Jni.CallStaticVoidMethod,
|
||||
.s32 => Jni.CallStaticIntMethod,
|
||||
.s64 => Jni.CallStaticLongMethod,
|
||||
.f32 => Jni.CallStaticFloatMethod,
|
||||
.f64 => Jni.CallStaticDoubleMethod,
|
||||
.bool => Jni.CallStaticBooleanMethod,
|
||||
else => {
|
||||
@@ -1301,6 +1302,7 @@ pub const LLVMEmitter = struct {
|
||||
.void => Jni.CallNonvirtualVoidMethod,
|
||||
.s32 => Jni.CallNonvirtualIntMethod,
|
||||
.s64 => Jni.CallNonvirtualLongMethod,
|
||||
.f32 => Jni.CallNonvirtualFloatMethod,
|
||||
.f64 => Jni.CallNonvirtualDoubleMethod,
|
||||
.bool => Jni.CallNonvirtualBooleanMethod,
|
||||
else => {
|
||||
@@ -1314,6 +1316,7 @@ pub const LLVMEmitter = struct {
|
||||
.void => Jni.CallVoidMethod,
|
||||
.s32 => Jni.CallIntMethod,
|
||||
.s64 => Jni.CallLongMethod,
|
||||
.f32 => Jni.CallFloatMethod,
|
||||
.f64 => Jni.CallDoubleMethod,
|
||||
.bool => Jni.CallBooleanMethod,
|
||||
else => {
|
||||
|
||||
@@ -138,6 +138,7 @@ pub const Interpreter = struct {
|
||||
pub var last_bail_op: ?[]const u8 = null;
|
||||
pub var last_bail_file: ?[]const u8 = null;
|
||||
pub var last_bail_offset: u32 = 0;
|
||||
pub var last_bail_builtin: ?[]const u8 = null;
|
||||
|
||||
pub fn init(module: *const Module, alloc: Allocator) Interpreter {
|
||||
var hooks = compiler_hooks.Registry.init(alloc);
|
||||
@@ -656,7 +657,12 @@ pub const Interpreter = struct {
|
||||
const args = self.alloc.alloc(Value, c.args.len) catch return error.CannotEvalComptime;
|
||||
defer self.alloc.free(args);
|
||||
for (c.args, 0..) |ref, i| {
|
||||
args[i] = frame.getRef(ref);
|
||||
// Inline any slot_ptr field-refs in the caller's frame before
|
||||
// the value crosses the call boundary. slot_ptr indices are
|
||||
// frame-local; if a slice/aggregate carrying one is passed to
|
||||
// the callee, the callee would later resolve the index against
|
||||
// its own slot table and read garbage.
|
||||
args[i] = self.materializeForCall(frame, frame.getRef(ref));
|
||||
}
|
||||
const result = try self.call(c.callee, args);
|
||||
return .{ .value = result };
|
||||
@@ -1218,6 +1224,38 @@ pub const Interpreter = struct {
|
||||
|
||||
// ── Slot chain resolution ────────────────────────────────────
|
||||
|
||||
/// Walk an aggregate Value and rewrite any embedded `slot_ptr` that points
|
||||
/// to a field-ref slot in `frame` (the marker shape `{parent_slot, idx, ..}`
|
||||
/// emitted by `struct_gep` / `index_gep`) into the resolved parent value.
|
||||
/// Slot indices are frame-local; a slice passed across a call would otherwise
|
||||
/// read its data_ptr out of the callee's slot table.
|
||||
fn materializeForCall(self: *Interpreter, frame: *Frame, val: Value) Value {
|
||||
switch (val) {
|
||||
.aggregate => |fields| {
|
||||
const new_fields = self.alloc.alloc(Value, fields.len) catch return val;
|
||||
for (fields, 0..) |f, i| {
|
||||
new_fields[i] = self.materializeForCall(frame, f);
|
||||
}
|
||||
return .{ .aggregate = new_fields };
|
||||
},
|
||||
.slot_ptr => |slot| {
|
||||
const stored = frame.loadSlot(slot);
|
||||
if (stored == .aggregate) {
|
||||
const ref_fields = stored.aggregate;
|
||||
if (ref_fields.len >= 2) {
|
||||
const parent_slot_val = ref_fields[0].asInt() orelse return val;
|
||||
if (ref_fields[1].asInt() == null) return val;
|
||||
const parent_slot: u32 = @intCast(parent_slot_val);
|
||||
const parent = frame.loadSlot(parent_slot);
|
||||
return self.materializeForCall(frame, parent);
|
||||
}
|
||||
}
|
||||
return val;
|
||||
},
|
||||
else => return val,
|
||||
}
|
||||
}
|
||||
|
||||
/// Follow a slot_ptr through field-pointer / index-gep chains
|
||||
/// to get the underlying value. Handles nested dereferences.
|
||||
fn resolveSlotChain(self: *Interpreter, frame: *Frame, val: Value) Value {
|
||||
@@ -1354,6 +1392,14 @@ pub const Interpreter = struct {
|
||||
// ── Builtin call dispatch ──────────────────────────────────────
|
||||
|
||||
fn execBuiltin(self: *Interpreter, bi: inst_mod.BuiltinCall, frame: *Frame, _: TypeId) InterpError!ExecResult {
|
||||
const result = self.execBuiltinInner(bi, frame) catch |err| {
|
||||
if (last_bail_builtin == null) last_bail_builtin = @tagName(bi.builtin);
|
||||
return err;
|
||||
};
|
||||
return result;
|
||||
}
|
||||
|
||||
fn execBuiltinInner(self: *Interpreter, bi: inst_mod.BuiltinCall, frame: *Frame) InterpError!ExecResult {
|
||||
switch (bi.builtin) {
|
||||
.malloc => {
|
||||
const size_val = frame.getRef(bi.args[0]);
|
||||
@@ -1378,10 +1424,16 @@ pub const Interpreter = struct {
|
||||
.heap_ptr => |hp| hp,
|
||||
else => return error.CannotEvalComptime,
|
||||
};
|
||||
// Get source bytes
|
||||
const src_bytes: []const u8 = switch (src) {
|
||||
.heap_ptr => |hp| self.heapSlice(hp) orelse return error.CannotEvalComptime,
|
||||
.string => |s| s,
|
||||
// Raw host address (e.g. a `*u8` returned by a foreign
|
||||
// call like getenv). Read `len` bytes across the FFI
|
||||
// boundary into the sx-managed dst.
|
||||
.int => |addr| blk: {
|
||||
const raw: [*]const u8 = @ptrFromInt(@as(usize, @bitCast(addr)));
|
||||
break :blk raw[0..len];
|
||||
},
|
||||
else => return error.CannotEvalComptime,
|
||||
};
|
||||
self.heapMemcpy(dst_hp, src_bytes, len);
|
||||
|
||||
@@ -55,6 +55,30 @@ pub const Options = struct {
|
||||
lib_name: ?[]const u8 = null,
|
||||
};
|
||||
|
||||
/// Inject a `static { System.loadLibrary("<lib>"); }` block into an already-
|
||||
/// rendered Java source. Used when the output path isn't known until after
|
||||
/// `#run` blocks execute — `collectJniMainEmissions` runs during lowering,
|
||||
/// before `BuildOptions.set_output_path(...)` has populated the lib name.
|
||||
/// Returns a newly-allocated string; caller owns it.
|
||||
pub fn injectLoadLibrary(allocator: Allocator, java_source: []const u8, lib_name: []const u8) ![]u8 {
|
||||
const marker = " {\n";
|
||||
const class_pos = std.mem.indexOf(u8, java_source, "public class ") orelse return try allocator.dupe(u8, java_source);
|
||||
const brace_rel = std.mem.indexOf(u8, java_source[class_pos..], marker) orelse return try allocator.dupe(u8, java_source);
|
||||
const insert_at = class_pos + brace_rel + marker.len;
|
||||
// Already injected? Skip.
|
||||
if (std.mem.indexOf(u8, java_source, "System.loadLibrary(") != null) {
|
||||
return try allocator.dupe(u8, java_source);
|
||||
}
|
||||
var buf: std.ArrayList(u8) = .empty;
|
||||
errdefer buf.deinit(allocator);
|
||||
try buf.appendSlice(allocator, java_source[0..insert_at]);
|
||||
try buf.appendSlice(allocator, " static { System.loadLibrary(\"");
|
||||
try buf.appendSlice(allocator, lib_name);
|
||||
try buf.appendSlice(allocator, "\"); }\n");
|
||||
try buf.appendSlice(allocator, java_source[insert_at..]);
|
||||
return try buf.toOwnedSlice(allocator);
|
||||
}
|
||||
|
||||
/// Emit a `.java` source for the given foreign-class decl. Result is
|
||||
/// heap-allocated through `allocator`; caller owns it.
|
||||
pub fn emitJavaSource(
|
||||
|
||||
@@ -4150,6 +4150,19 @@ pub const Lowering = struct {
|
||||
|
||||
const ret_ty = if (method.return_type) |rt| self.resolveType(rt) else .void;
|
||||
|
||||
// Reject return types the JNI emit path can't dispatch — emit_llvm's
|
||||
// Call<T>Method switch only covers void / bool / s32 / s64 / f32 / f64
|
||||
// / pointer-returning. Anything else (s8 / s16 / u8 / u16 / aggregates)
|
||||
// would silently lower to LLVMGetUndef and produce wrong arguments at
|
||||
// the call site (chess Android touch shipped broken because s32→s32+
|
||||
// f32 returns hit the undef path before .f32 was wired up).
|
||||
if (!isJniReturnTypeSupported(&self.module.types, ret_ty)) {
|
||||
if (self.diagnostics) |d| {
|
||||
d.addFmt(.err, span, "JNI method '{s}.{s}' returns '{s}', which isn't supported by the JNI call-method lowering yet — only void/bool/s32/s64/f32/f64 and pointers are wired up", .{ fcd.name, method.name, self.module.types.typeName(ret_ty) });
|
||||
}
|
||||
return Ref.none;
|
||||
}
|
||||
|
||||
const cache_key: inst_mod.CacheKey = .{
|
||||
.name_str = method_name,
|
||||
.sig_str = desc_str,
|
||||
@@ -10295,6 +10308,23 @@ fn jniMapParamType(self: *Lowering, type_node: *ast.Node) TypeId {
|
||||
return self.resolveType(type_node);
|
||||
}
|
||||
|
||||
/// Whether emit_llvm's `jni_msg_send` lowering can dispatch a Call<T>Method
|
||||
/// for this return type. Anything outside this set falls into the `else`
|
||||
/// arm of the switches in `emit_llvm.zig` and would silently produce
|
||||
/// `LLVMGetUndef` — a footgun that previously shipped (chess Android touch
|
||||
/// went undef because `MotionEvent.getX() -> f32` wasn't in the switch).
|
||||
/// Pointer-typed returns route through `CallObjectMethod`.
|
||||
pub fn isJniReturnTypeSupported(table: *const @import("types.zig").TypeTable, ret_ty: TypeId) bool {
|
||||
return switch (ret_ty) {
|
||||
.void, .bool, .s32, .s64, .f32, .f64 => true,
|
||||
else => blk: {
|
||||
if (ret_ty.isBuiltin()) break :blk false;
|
||||
const info = table.get(ret_ty);
|
||||
break :blk info == .pointer or info == .many_pointer;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/// Encode a (foreign_path, method_name) pair as the JNI-resolved symbol
|
||||
/// `Java_<pkg-mangled>_<Class>_sx_1<method-mangled>`. JNI mangling:
|
||||
/// `/` → `_`, `_` → `_1`. The `sx_` prefix matches the Java-side
|
||||
|
||||
Reference in New Issue
Block a user