49b39ba07a8d9afe3444b66659989ecc58fdff21
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
agra
|
632e64512b |
bundling: Android APK pipeline moved into sx; android.sx state-on-plat
Week 7 of /Users/agra/.claude/plans/lets-plan-to-move-splendid-pumpkin.md
plus the android.sx refactor + three sx-compiler fixes hit along the way
to get chess on Pixel 7 Pro responding to touch end-to-end.
library/modules/platform/bundle.sx now covers the Android APK shape
alongside macOS / iOS-sim / iOS-device. `android_bundle_main` discovers
the SDK ($ANDROID_HOME / $ANDROID_SDK_ROOT / $HOME/Library/Android/sdk),
picks the highest-versioned build-tools + platforms via
`process.run("ls .. | sort -V | tail -1")`, stages
`<apk>.stage/lib/arm64-v8a/<libfoo.so>`, synthesizes
AndroidManifest.xml (NativeActivity vs `#jni_main` Activity branch),
writes each `#jni_main` decl's Java source under
`<stage>/java/<pkg>/<Cls>.java`, runs javac --release 11 + d8 to
produce classes.dex, aapt2-links the unaligned APK, appends lib/ +
classes.dex + each registered asset tree via zip, zipalign + ensure
debug keystore via keytool + apksigner sign.
Compiler-side accessors (src/ir/compiler_hooks.zig + library/modules/compiler.sx):
- is_android predicate.
- set_manifest_path / manifest_path + set_keystore_path / keystore_path.
- jni_main_count / jni_main_foreign_path_at(i) /
jni_main_java_source_at(i) surface the `#jni_main` emissions that
the Zig createApk previously consumed directly.
- main.zig wires manifest_path, keystore_path, and the per-decl
(foreign_path, java_source) parallel slices into BuildConfig before
invoking the post-link callback.
CLI `--apk <path>` keeps working as a transitional alias: it now feeds
bundle_path so the existing auto-`post_link_module = "platform.bundle"`
shim fires the same way as `--bundle`. main.zig no longer calls
target.createApk directly.
Deletions in src/target.zig: createApk, compileJniMainSources,
buildJniMainManifest, buildAndroidManifest, ensureDebugKeystore,
libNameFromSoBasename, plus helpers splitForeignPath / discoverJavac /
discoverAndroidSdk / findHighestSubdir / runProcess / runProcessIn
(~400 lines). git grep returns only the obituary comment.
library/modules/platform/android.sx refactor (chess Android dependency):
- Module-level globals retired (g_app_window, g_egl_*, g_viewport_*,
g_dpi_scale, g_should_stop, g_render_thread*, g_user_main_fn,
g_touch_*) → AndroidPlatform struct fields.
- All sx_android_* helpers take `plat: *AndroidPlatform` as first arg.
Render thread receives plat via pthread_create's arg.
- New `logical_w: f32 = 0.0` field. Consumers set it before init() to
define the design width in points; `recompute_scale` derives
`dpi_scale = pixel_w / logical_w` (or 1.0 if unset). Called on
init / set_viewport / egl_init. drain_touches divides incoming
physical pixel coords by dpi_scale so chess sees logical-space
positions matching its layout. Touch lands on the right squares.
Three sx-compiler bugs hit + fixed along the way:
1. Top-level `inline if OS == .X { decls }` body decls were silently
dropped because scanDecls/lowerDecls had no .if_expr arm. New
`flattenComptimeConditionals` pre-pass in src/imports.zig
(threaded via ComptimeContext from core.zig) hoists matching arms
recursively. Regression at examples/124-inline-if-hoist-toplevel.sx.
2. Parser rejected `#import` / `#framework` inside inline-if bodies
because parseStmt in src/parser.zig only had arms for `#insert`.
Added the missing arms. Regression at
examples/123-inline-if-import-in-body.sx (landed earlier).
3. JNI `Call<T>Method` switches in src/ir/emit_llvm.zig (instance /
nonvirtual / static) were missing `.f32` rows — jfloat returns
(e.g. MotionEvent.getX/getY) fell into the silent-undef else arm.
Chess's sx_android_push_touch(plat, getAction(), getX(), getY())
delivered garbage f32 coords to the touch ring, so taps landed
nowhere recognisable. Added `.f32 => Jni.Call{Static,Nonvirtual,}FloatMethod`
rows to all three switches; lifted unsupported-type detection
from emit_llvm into lowerForeignMethodCall with proper
source-spanned diagnostics (`isJniReturnTypeSupported`). Regressions
at examples/ffi-jni-call-10-jfloat-return.sx,
examples/ffi-jni-class-09-multi-float-args.sx,
examples/ffi-jni-call-11-unsupported-return-diag.sx.
Stale-snapshot drift in tests/expected/ffi-objc-call-03-selector-sharing.ir
and ffi-objc-call-06-sret-return.ir picks up the new BuildOptions
accessor extern decls (is_android, set_manifest_path,
set_keystore_path, jni_main_count, jni_main_foreign_path_at,
jni_main_java_source_at). Verified diff is dead-decl-only.
Chess on Pixel 7 Pro: tap on e2 white pawn -> yellow selection +
green dots on legal e3/e4 targets; tap on e4 -> board updates with
1. e4, "Black to move" + "1. e4" in info panel.
zig build && zig build test && bash tests/run_examples.sh -> 145/145
green. bash tests/cross_compile.sh -> 7/7 green.
|
||
|
agra
|
5cc62e63c3 |
bundling: fs/process stdlib + post-link callback + Apple .app in sx
Campaign Weeks 3-6 of /Users/agra/.claude/plans/lets-plan-to-move-splendid-pumpkin.md
land in one push: the bundling pipeline that used to live in
src/target.zig (createBundle, embedFramework, extractEntitlements,
buildInfoPlist, codesign) now lives in
library/modules/platform/bundle.sx and runs in the IR interpreter
after target.link() returns.
New language-side surface:
- library/modules/fs.sx — POSIX libc bindings (open/read/write/close,
mkdir/unlink/rmdir, chmod, rename, access, basename/dirname). Variadic
open() lowers to C's varargs via the new args: ..T form. Direct libc
calls bypass *File method dispatch so they work from the post-link
IR interpreter.
- library/modules/process.sx — popen-based run(cmd) returning
ProcessResult{ exit_code, stdout }, plus env() and find_executable().
- library/modules/std.sx — xml_escape(s) and variadic path_join(parts).
- library/modules/compiler.sx — BuildOptions grows
set_post_link_callback / set_post_link_module / binary_path
accessors; bundle_path/bundle_id/codesign_identity/provisioning_profile
setters + accessors; per-target predicates is_macos/is_ios/
is_ios_device/is_ios_simulator + target_triple; framework_count /
framework_at(i) / framework_path_count / framework_path_at(i);
add_asset_dir(src, dest) + asset_dir_count / src_at / dest_at.
Compiler-side wiring:
- src/ir/compiler_hooks.zig — BuildConfig now carries post_link_callback_fn,
post_link_module, binary_path, bundle_*, target_triple,
target_frameworks, target_framework_paths, asset_dirs. Hook registry
exposes every accessor; getters return "" / 0 for unset fields so
bundle.sx can treat absent values uniformly.
- src/ir/host_ffi.zig (new) — dlsym(RTLD_DEFAULT) + arity-switched cdecl
trampolines so #foreign("c") declarations resolve through the host
libc during #run / post-link interpretation.
- src/ir/interp.zig — callForeign dispatch; build_config pointer
injection so accessor hooks see live state during re-entry.
- src/core.zig — keeps the IR module alive past generateCode; exposes
invokeByName / invokeByFuncId so main.zig can re-enter the
interpreter after linking.
- src/main.zig — wires bundle/codesign/provisioning CLI flags +
target_triple + framework lists into BuildConfig; invokes the
post-link callback (by FuncId or by <module>.bundle_main lookup) once
target.link() returns. When --bundle is set but no callback is
registered, auto-falls-back to post_link_module = "platform.bundle"
so the legacy --bundle CLI keeps working for any program that imports
modules/platform/bundle.sx.
Apple .app bundler (library/modules/platform/bundle.sx):
- Single bundle_main entry covers macOS, iOS simulator, iOS device.
Per-target Info.plist switch keys off is_ios()/is_ios_simulator() —
iOS emits UIDeviceFamily / LSRequiresIPhoneOS /
UIApplicationSceneManifest / DTPlatformName (iPhoneOS or
iPhoneSimulator); macOS emits the minimal CFBundle* set.
- iOS-only steps:
- Provisioning embed: fs.read_file + fs.write_file to
<bundle>/embedded.mobileprovision.
- Framework embed: recursive cp -R per -F search path into
<bundle>/Frameworks/<Name>.framework/ (until fs.sx grows list_dir).
- Entitlements extraction: four process.run calls (security cms -D,
plutil -extract Entitlements xml1, plutil -extract
ApplicationIdentifierPrefix.0, plutil -replace application-identifier)
resolving the wildcard <TEAM>.* -> <TEAM>.<bundle_id>.
- Real codesign with --entitlements when present.
- Asset dirs (add_asset_dir): recursive cp -R src/. into <bundle>/dest/.
Missing src is treated as "nothing to do" so projects can register
add_asset_dir("assets", "assets") unconditionally.
Parser:
- parseStmt() now accepts #import \"path\"; and #framework \"Name\"; as
statement-position tokens. Needed for top-level
inline if OS == .android { #import \"modules/platform/android.sx\"; }
blocks (issue-0042 flatten pass surfaces them); chess's
inline-if-with-#import was rejected at parse time before this fix.
Removals from src/target.zig:
- createBundle, embedFramework, extractEntitlements, buildInfoPlist,
codesign (~210 lines). main.zig no longer calls createBundle after
link(); the sx callback is the single entry point.
Tests / regression markers (all run under sx run host JIT):
- examples/115-post-link-callback.sx — callback registration round-trip.
- examples/116-fs-roundtrip.sx — fs.write_file -> fs.read_file -> exists.
- examples/117-process-roundtrip.sx — process.run + env + find_executable.
- examples/118-macos-bundle.sx — macOS .app via bundle_main callback.
- examples/119-interp-cast-ptr-cmp.sx — cast(T) val under interpreter.
- examples/120-interp-variadic-any.sx — variadic ..Any indexing in IR
interpreter.
- examples/121-ios-sim-bundle.sx — iOS-sim cross-compile + .app with
iOS-shaped Info.plist (added to tests/cross_compile.sh as the
ios-sim tuple).
- examples/122-ios-device-bundle.sx — iOS device cross-compile +
full codesign pipeline (provisioning embed + entitlements
extraction + --entitlements codesign). Manually verified end-to-end:
installed via xcrun devicectl device install app + launched
successfully on iPhone 17 Pro.
- examples/123-inline-if-import-in-body.sx — locks in the parser fix.
zig build && zig build test && bash tests/run_examples.sh => 141 passed,
0 failed; bash tests/cross_compile.sh => 7 passed, 0 failed.
|
||
|
agra
|
0bb7b8cc27 |
issue-0037 fixed: ptr↔int conversion in coerceToType / bitcast emit
109/109 regression tests pass; chess Android + iOS-sim still
build clean.
Root cause: sx's `xx <ptr>` cast targeting an integer type
(common pattern: `xx u64 = xx @some_global`) lowered to a no-op
because `coerceToType` had branches for int↔float and same-kind
widen/narrow, but nothing for pointer↔integer. The cast left the
value as a pointer Ref, and `emitInst`'s `.ret` arm tried to
coerce a `ptr` value to an `i64` slot — coerceArg had no
ptr↔int branch either, fell through to undef.
Why it worked in main but failed in helpers: an
`alloca u64`+`store ptr @g, alloca`+`load i64, alloca` sequence
preserves the address bits as raw memory, so the
"store-then-load through an alloca" workaround happened to do
the right thing without a real cast. A `ret i64 <ptr>` has no
such intermediate slot and triggers an LLVM type mismatch.
Fix layered into two existing IR opcodes:
lower.zig (coerceToType):
new branch — when src and dst types are ptr↔int, emit a
`bitcast` IR opcode with the right from/to. Mirrors how
int↔float emits `.int_to_float` / `.float_to_int`.
emit_llvm.zig (.bitcast arm):
dispatch ptr→int to `LLVMBuildPtrToInt` (+ trunc/zext if the
target int width != 64), int→ptr to `LLVMBuildIntToPtr`. The
"real bitcast" path stays for same-kind type punning.
Modern LLVM's BuildBitCast rejects ptr↔int directly, hence
the dispatch.
The fix also closes a quiet behavior gap that affected non-`#foreign`
globals (any `xx @<global>` from a helper fn). Surfaced while
investigating issue-0037; verified independently with a
non-`#foreign` sx-side global of type `s64`.
File mechanics: issue-0037 promoted to a focused feature example
per CLAUDE.md's resolution flow:
examples/issue-0037.sx -> examples/102-foreign-global-from-helper.sx
tests/expected/issue-0037.{txt,exit} -> tests/expected/102-foreign-global-from-helper.{txt,exit}
ffi-objc-call-03 + ffi-objc-call-06 IR snapshots updated to
reflect the ptr→int store-via-ptrtoint shape that's now correct
at the LLVM-IR level (same bits in memory, but properly typed).
|
||
|
agra
|
d43385112c |
ffi 1.6: objc_msg_send IR opcode + per-call-site LLVM fn type
102/102 regression tests pass; chess Android + iOS-sim still build
clean. `ffi-objc-call-04-primitive-returns` flips from xfail to
passing with both nil-recv and real-recv flavors of *void / s64
returns exercised.
Key change: a new `objc_msg_send` IR opcode bundles (recv, sel,
extra args) and carries the return type via the `Inst.ty` field.
emit_llvm.zig builds a per-call-site LLVM function type from the
argument Refs' IR types (recv/sel as ptr; extra args through
abiCoerceParamType) and dispatches with LLVMBuildCall2. One
declared `@objc_msgSend` symbol is reused across every return
type — opaque pointers make the function value type-erased, so
each call site picks its own ABI.
before: one (recv, sel) -> ptr LLVM declaration, hard-coded
per call site; only void return wired in 1.3.
after: same declaration, each call site provides a fresh
LLVMBuildCall2 fn-type → s64 / *void / bool / f64
returns all dispatch correctly without separate FuncIds.
Selector init mechanism: stayed with the @llvm.global_ctors
constructor. Investigated clang's
`__DATA,__objc_selrefs` + `externally_initialized` shape — works
for fully-linked binaries (dyld substitutes the SEL at load
time) but **LLVM ORC JIT** (the engine behind `sx run`) doesn't
process Mach-O Obj-C metadata sections, so the slot keeps its
initial value (the method-name string pointer) and dispatch
crashes with "<null selector>". The portable choice: keep the
constructor AND inject a direct call to it at `main`'s entry —
idempotent under dyld (sel_registerName returns the same SEL on
re-registration), required for ORC JIT.
Files touched:
src/ir/inst.zig | new ObjcMsgSend struct + opcode
src/ir/lower.zig | drop the void-only restriction; emit the
new opcode; remove the orphaned
getObjcMsgSendFid path (objc_msgSend
declaration moved to emit_llvm)
src/ir/emit_llvm.zig | objc_msg_send arm (per-call-site
LLVMBuildCall2); lazy `@objc_msgSend`
declaration via getObjcMsgSendValue;
emitObjcSelectorInit refactored to inject
the ctor call at main's entry
src/ir/{print,interp}.zig | switch arms for the new opcode
`ffi-objc-call-03-selector-sharing.ir` snapshot updates to
reflect the new shape (the `call ... @objc_msgSend` call sites
no longer mention a typed wrapper).
|
||
|
agra
|
b8a412ddc7 |
ffi 1.5: intern Obj-C selectors — one static SEL slot per unique name
101/101 regression tests pass; the IR snapshot for the selector-
sharing test diff flips from four per-call `sel_registerName` calls
to two (one per unique selector) routed through a module-init
constructor — matching what clang emits for `@selector(...)`.
Hot-path cost collapses from a libobjc hashtable lookup per call to
a single load of a static `SEL*` slot:
Before (Phase 1.3):
%sel = call ptr @sel_registerName(<"init">)
call ptr @objc_msgSend(<recv>, %sel)
After (Phase 1.5):
%sel = load ptr, ptr @OBJC_SELECTOR_REFERENCES_init
call ptr @objc_msgSend(<recv>, %sel)
+ @OBJC_SELECTOR_REFERENCES_init = internal global ptr null
+ @OBJC_SELECTOR_REFERENCES_release = internal global ptr null
+ define internal void @__sx_objc_selector_init() {
+ %sel = call ptr @sel_registerName(ptr @OBJC_METH_VAR_NAME_)
+ store ptr %sel, ptr @OBJC_SELECTOR_REFERENCES_init
+ %sel1 = call ptr @sel_registerName(ptr @OBJC_METH_VAR_NAME_.2)
+ store ptr %sel1, ptr @OBJC_SELECTOR_REFERENCES_release
+ ret void
+ }
+ @llvm.global_ctors = appending global [1 x { i32, ptr, ptr }]
+ [{ ..., ptr @__sx_objc_selector_init, ptr null }]
Implementation:
module.zig | new `objc_selector_cache: ArrayList(ObjcSelectorEntry)`
with `lookupObjcSelector` / `appendObjcSelector`. List
(not hashmap) keeps emit order stable across builds so
the IR snapshot doesn't flicker on rehash.
lower.zig | `internObjcSelector(sel)` creates the slot on first
use, returns the same `GlobalId` on every subsequent
call to the same selector. lowerFfiIntrinsicCall now
emits `global_addr + load` for literal selectors.
Non-literal selectors keep the `sel_registerName`
fallback. Declaring `sel_registerName` lazily on
first intern so emit_llvm finds it for the
constructor body.
emit_llvm.zig | new `emitObjcSelectorInit` pass synthesizes a void
constructor that loops over the cache, calls
`sel_registerName` for each unique selector string,
stores the result in the slot. Constructor is
registered in `@llvm.global_ctors` with default
priority (65535) so dyld runs it before main.
The `@OBJC_METH_VAR_NAME_` private string globals and unnamed-addr
flag match clang's exact emission shape — picked up by the system
linker into the right Mach-O sections on macOS / iOS. Chess
Android + iOS-sim still build clean (no `#objc_call` in chess yet —
phase-3 migration will start exercising this).
|
||
|
agra
|
26a04e49d0 |
tests: IR-snapshot harness — diff sx ir output when .ir present
run_examples.sh now supports an optional `tests/expected/<name>.ir` sibling to `.txt`/`.exit`. When present, the runner also captures `sx ir <file>` output, normalizes target-/host-specific noise (module ID, target triple/datalayout, attribute groups, LLVM's auto-suffixed %temp numbering), and diffs against the snapshot. `--update` regenerates it alongside the runtime output. Catches lowering changes that don't affect what the program prints — exactly the shape Phase 1.5's selector interning will produce (same runtime output, very different IR). First snapshot: `ffi-objc-call-03-selector-sharing.ir`. Today the test emits four `call ptr @sel_registerName(ptr @str.N)` lines for its four call sites; after 1.5 we expect two static `@OBJC_SELECTOR_REFERENCES_<sel>` globals + loads at each call site. The diff between the two snapshots will be the visible artifact of the optimization. |