4de565b7da86ea46a7ca6681d73cd1c32fc5bc16
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
agra
|
d415bcceaa |
mem: drop matchContextAllocCall — interp reaches real memory through libc
Comptime now runs the full Allocator-protocol dispatch chain — the
same IR codegen emits — instead of being short-circuited at lowering
by an AST pattern-match. `context.allocator.alloc(size)` flows
through the protocol thunk into `CAllocator.alloc → libc_malloc`,
returning a real host-libc pointer. The interp picks it up as a raw
`.int` Value and treats it as memory.
The pieces:
- `evalComptimeString` now uses the parent module instead of spinning
up a fresh ct_module. The parent already has every type, protocol,
impl, and thunk registered (Allocator, CAllocator, Context, the
GPA/Tracker thunks), so the dispatch chain runs without a separate
scan pass. The comptime function is appended to the parent module;
it's `is_comptime` so codegen skips it.
- Interp gains raw-pointer paths:
- `index_gep(.aggregate{.int data_ptr, .int len}, idx)` produces a
new `.byte_ptr` (a new Value variant) — byte-granular pointer that
`store` writes 1 byte through. Mirrors the existing heap_ptr
semantics for the same op shape.
- `index_gep(.int, idx)` returns `.int = p + idx` (byte-addressed).
- `store(.int_ptr, val)` writes val's bytes via `@ptrFromInt`.
Handles int (8B), float (8B), bool (1B), null_val (8B of zeros).
- `store(.byte_ptr, val)` writes a single byte.
- `marshalForeignArg` handles `.aggregate{.int data, .int len}` and
`.byte_ptr` — both copy bytes into a null-terminated tmp buffer
for the C-side call.
- `asString` reads `len` bytes from a `.int` data field via
`@ptrFromInt`.
- `resolveFieldLoad` / `resolveFieldStore` reject field-pointer
aggregates whose first field is a wide integer (would otherwise
mis-trigger on a struct stored on the stack with an int pointer
in field 0).
- `lowerFunction` / `lazyLowerFunction` / `synthesizeJniMainStub`
bind `current_ctx_ref = &__sx_default_context` for every
callconv(.c) sx entry — not just `isExportedEntryName`. The JNI
stubs need this so `context.X` in the body resolves through
current_ctx_ref now that the pattern-match is gone.
- `matchContextAllocCall` and its dispatch site are deleted.
11 JNI/ObjC `.ir` snapshots regen — the comptime function appended to
the parent module shifts string-pool indices. 153/153 example tests
pass, chess green on macOS / iOS sim / Android.
|
||
|
agra
|
f886d5f1be |
mem: reject call-conv mismatches at bare-fn → fn-ptr coercion
Passing a default-conv sx function to a `callconv(.c)` fn-pointer slot (e.g. pthread_create's start routine) used to silently mismatch ABIs: the C-side caller didn't supply __sx_ctx, so the sx-side body read its first user param as garbage. The bug surfaced as a SIGSEGV inside ANativeWindow_setBuffersGeometry on Android during chess bringup. Now the compiler rejects the coercion outright at the bare-fn name lookup site: error: call-convention mismatch: 'sx_handler' is declared with default sx convention but the target type expects callconv(.c) Also: `#foreign` declarations without an explicit `callconv` now default to `.c` instead of `.default`. Every external C symbol is by definition C-conv; the previous default silently typed `objc_msgSend` (et al.) as default-conv, so the check would fire on the consumer side when the user typed a fn-ptr as `callconv(.c)`. With the foreign-default fix, the existing typed-msgSend casts in `std/objc.sx` and `gpu/metal.sx` keep type-checking and the rule is "C-conv on both sides or neither." Caught by the new check (fixed in the same commit): - `ios_gl_proc` in `platform/uikit.sx` lacked callconv(.c) but was passed to `load_gl` whose `get_proc` slot expects it. - `ffi_apply_callback` / `ffi_apply_callback2` in `examples/ffi-06-callback.sx` had default-conv fn-ptr params but the C bodies (in the companion .c) are unambiguously C-conv. Regression test: `examples/131-callconv-mismatch-diagnostic.sx` locks in the diagnostic shape (sx-conv fn → callconv(.c) slot). 153/153 example tests pass. Chess green on macOS / iOS sim / Android. |
||
|
agra
|
b69a2ea29c |
mem: Step 8 — delete context global from std.sx
The `context : Context = ---;` global in `library/modules/std.sx` had
no remaining readers — all `context.X` lookups in user code resolve
through `current_ctx_ref` (Step 5), `push Context.{...}` uses an alloca
slot (Step 6), and `allocViaContext` sources from the lowering's
current ref. `emitDefaultContextInit` (the only writer) was already
removed in Step 5.
`inferExprType` for the `context` identifier now returns the registered
`Context` type when implicit-ctx is enabled, mirroring the lowering's
identifier-handling fast path. Without this, `context.allocator` would
type as `s64` (the fallback) and the field access would fail.
11 JNI/ObjC IR snapshots regen — the `@context` LLVM global is gone
from each.
152/152 example tests pass.
|
||
|
agra
|
4bf5908792 |
mem: Steps 5-7 — context-identifier rebind + interp ctx bootstrap
Step 5 — `context` resolves through `current_ctx_ref`. The compile-time
emit of the default GPA into the `context` global is gone; entry points
already bind `current_ctx_ref` to `&__sx_default_context` and every
sx-to-sx call forwards it. `allocViaContext` sources from
`current_ctx_ref` too. `matchContextAllocCall` is kept as a comptime
escape hatch: the ct_module spun up by `evalComptimeString` doesn't get
the full Allocator/CAllocator/Context type registration so the protocol-
dispatch chain wouldn't run in the interp; codegen also wins from the
direct libc malloc/free.
Step 6 — `push Context.{...}` stack-discipline rewrite. Allocates a
fresh `Context` slot, binds `current_ctx_ref` to it for the body's
lexical scope, restores on exit. No global, no walk.
Step 7 — interp parity. `defaultContextValue()` builds the Context
aggregate (CAllocator thunks for alloc/dealloc, null data) on demand.
`interp.call` bootstraps slot_ptr(0) when an entry function with
implicit ctx is called sans args; `materializeCtxArg` dereferences the
caller's slot_ptr into the aggregate at every sx-to-sx call boundary so
the callee's `load(ref_0)` lands on the value; `load` of an aggregate
is a passthrough. `.global_addr` of `__sx_default_context` returns the
aggregate directly so exported entries' first-line `global_addr(...)`
runs cleanly in `#run`.
`ct_lowering` inherits `implicit_ctx_enabled` + `has_implicit_ctx` so
functions lowered into the ct module carry ctx like their main-module
twins.
152/152 example tests pass. Snapshots regen.
|
||
|
agra
|
92c6b47f12 |
mem: Step 3 — thread __sx_ctx through closure/fn-pointer/method dispatch
Continues the implicit-Context refactor. Bare-fn trampolines, lambda trampolines, and protocol thunks now carry __sx_ctx at slot 0; call sites for closures, fn-pointer variables, and method dispatch prepend the caller's current ctx. - emit_llvm.zig:1687 call_indirect treats `fp_ctx_slots` leading args as opaque ptr (the implicit ctx) when the fn-pointer is default-conv under has_implicit_ctx. - lower.zig:fnPtrTypeWantsCtx predicate gates the prepend at both scope-local and global fn-pointer call sites. - lower.zig:fixupMethodReceiver skips __sx_ctx when probing the receiver param's type. - lower.zig:lowerLambda builds closure type from user-visible params only (skip ctx + env). - lower.zig:closure(bare_fn) builds closure type from user-visible params only. - module.zig: Module.has_implicit_ctx flag mirrors Lowering's switch so emit_llvm can read it without a back-pointer. Tests updated: - 5 ObjC-block/runtime tests get `callconv(.c)` on fn-ptr types cast from `objc_msgSend` / Block.invoke (C-side calls into sx). - ffi-06-callback gets `callconv(.c)` on double_it/add_with_ctx — the registered C-side callbacks. - 08-types snapshot regen (undefined-init drift from layout shift). - 11 JNI/ObjC .ir snapshots regen for the ctx-prepended thunk signatures. 151/152 example tests pass. Remaining failure (05-run) is the comptime/interp path that requires Step 7 (callWithDefaultContext). |
||
|
agra
|
29784c22a8 |
mem: implicit-context foundation + many compiler fixes
The session-long set of changes that lay the groundwork for the
Jai-literal implicit-Context-parameter refactor. Lots of accumulated
work; the new arrival is the implicit-ctx foundation (steps 1+2 of
the plan in current/CHECKPOINT-MEM.md):
Step 1 — `CAllocator :: struct {}` stateless allocator in
library/modules/allocators.sx, delegating directly to
libc_malloc/libc_free. `ConstantValue` in src/ir/inst.zig gains a
`func_ref: FuncId` leaf so nested aggregates can carry function
pointers (the inline Allocator value's fn-ptr fields). Switch
sites updated in emit_llvm.zig, print.zig, interp.zig.
Step 2 — `emitDefaultContextGlobal` in src/ir/lower.zig synthesises
a static `__sx_default_context` global with a nested-aggregate
init_val pointing at the CAllocator → Allocator thunks. The
second-pass `initVtableGlobals` in emit_llvm.zig is generalised
to handle `.aggregate` init_vals (re-emits after func_map is
populated so func_ref leaves resolve to real symbols).
Also folded in from earlier work this session:
- Phase 1.1: `xx value` heap-copy in `buildProtocolValue` routes
through `context.allocator` via the new `allocViaContext` helper.
- interp.zig: `marshalForeignArg` double-offset bug fixed —
`heapSlice` already adds `hp.offset` to the slice ptr, so the
extra `+ hp.offset` was scribbling memcpy/memset into adjacent
heap state, corrupting `heap.items[0]`. Symptom: `build_format`
at comptime produced zero bytes, all `print` calls failed.
- Lazy lowering: `lazyLowerFunction` now declares foreign-body
functions as extern stubs in the local (comptime) module so
cross-module foreign calls resolve.
- Allocator API: all stdlib allocators on one-line `init() -> *T`
(CAllocator/GPA: libc-backed; Arena/TrackingAllocator: parent-
backed; BufAlloc: embeds state at head of user buffer).
- issues 0038 (transitive #import), 0039 (chess + stdlib migration
fallout), 0040 (generic struct method dot-dispatch), 0041
(pointer types as type-arg), 0042 (alias name resolution) — all
fixed; regression tests in examples/.
- Diagnostic: `emitError` now embeds the lowering's
`current_source_file` and enclosing function in the literal
message; SX_TRACE_UNRESOLVED=1 dumps a Zig stack trace at the
emit site so misattributed spans can't hide where the failure
is.
- tools/verify-step.sh (all-platforms gate) and tools/scratch.sh
(interp/codegen parity tester) added.
Test suite: 152 example tests pass; chess builds + screenshots on
macOS / iOS sim / Android.
|
||
|
agra
|
5cc62e63c3 |
bundling: fs/process stdlib + post-link callback + Apple .app in sx
Campaign Weeks 3-6 of /Users/agra/.claude/plans/lets-plan-to-move-splendid-pumpkin.md
land in one push: the bundling pipeline that used to live in
src/target.zig (createBundle, embedFramework, extractEntitlements,
buildInfoPlist, codesign) now lives in
library/modules/platform/bundle.sx and runs in the IR interpreter
after target.link() returns.
New language-side surface:
- library/modules/fs.sx — POSIX libc bindings (open/read/write/close,
mkdir/unlink/rmdir, chmod, rename, access, basename/dirname). Variadic
open() lowers to C's varargs via the new args: ..T form. Direct libc
calls bypass *File method dispatch so they work from the post-link
IR interpreter.
- library/modules/process.sx — popen-based run(cmd) returning
ProcessResult{ exit_code, stdout }, plus env() and find_executable().
- library/modules/std.sx — xml_escape(s) and variadic path_join(parts).
- library/modules/compiler.sx — BuildOptions grows
set_post_link_callback / set_post_link_module / binary_path
accessors; bundle_path/bundle_id/codesign_identity/provisioning_profile
setters + accessors; per-target predicates is_macos/is_ios/
is_ios_device/is_ios_simulator + target_triple; framework_count /
framework_at(i) / framework_path_count / framework_path_at(i);
add_asset_dir(src, dest) + asset_dir_count / src_at / dest_at.
Compiler-side wiring:
- src/ir/compiler_hooks.zig — BuildConfig now carries post_link_callback_fn,
post_link_module, binary_path, bundle_*, target_triple,
target_frameworks, target_framework_paths, asset_dirs. Hook registry
exposes every accessor; getters return "" / 0 for unset fields so
bundle.sx can treat absent values uniformly.
- src/ir/host_ffi.zig (new) — dlsym(RTLD_DEFAULT) + arity-switched cdecl
trampolines so #foreign("c") declarations resolve through the host
libc during #run / post-link interpretation.
- src/ir/interp.zig — callForeign dispatch; build_config pointer
injection so accessor hooks see live state during re-entry.
- src/core.zig — keeps the IR module alive past generateCode; exposes
invokeByName / invokeByFuncId so main.zig can re-enter the
interpreter after linking.
- src/main.zig — wires bundle/codesign/provisioning CLI flags +
target_triple + framework lists into BuildConfig; invokes the
post-link callback (by FuncId or by <module>.bundle_main lookup) once
target.link() returns. When --bundle is set but no callback is
registered, auto-falls-back to post_link_module = "platform.bundle"
so the legacy --bundle CLI keeps working for any program that imports
modules/platform/bundle.sx.
Apple .app bundler (library/modules/platform/bundle.sx):
- Single bundle_main entry covers macOS, iOS simulator, iOS device.
Per-target Info.plist switch keys off is_ios()/is_ios_simulator() —
iOS emits UIDeviceFamily / LSRequiresIPhoneOS /
UIApplicationSceneManifest / DTPlatformName (iPhoneOS or
iPhoneSimulator); macOS emits the minimal CFBundle* set.
- iOS-only steps:
- Provisioning embed: fs.read_file + fs.write_file to
<bundle>/embedded.mobileprovision.
- Framework embed: recursive cp -R per -F search path into
<bundle>/Frameworks/<Name>.framework/ (until fs.sx grows list_dir).
- Entitlements extraction: four process.run calls (security cms -D,
plutil -extract Entitlements xml1, plutil -extract
ApplicationIdentifierPrefix.0, plutil -replace application-identifier)
resolving the wildcard <TEAM>.* -> <TEAM>.<bundle_id>.
- Real codesign with --entitlements when present.
- Asset dirs (add_asset_dir): recursive cp -R src/. into <bundle>/dest/.
Missing src is treated as "nothing to do" so projects can register
add_asset_dir("assets", "assets") unconditionally.
Parser:
- parseStmt() now accepts #import \"path\"; and #framework \"Name\"; as
statement-position tokens. Needed for top-level
inline if OS == .android { #import \"modules/platform/android.sx\"; }
blocks (issue-0042 flatten pass surfaces them); chess's
inline-if-with-#import was rejected at parse time before this fix.
Removals from src/target.zig:
- createBundle, embedFramework, extractEntitlements, buildInfoPlist,
codesign (~210 lines). main.zig no longer calls createBundle after
link(); the sx callback is the single entry point.
Tests / regression markers (all run under sx run host JIT):
- examples/115-post-link-callback.sx — callback registration round-trip.
- examples/116-fs-roundtrip.sx — fs.write_file -> fs.read_file -> exists.
- examples/117-process-roundtrip.sx — process.run + env + find_executable.
- examples/118-macos-bundle.sx — macOS .app via bundle_main callback.
- examples/119-interp-cast-ptr-cmp.sx — cast(T) val under interpreter.
- examples/120-interp-variadic-any.sx — variadic ..Any indexing in IR
interpreter.
- examples/121-ios-sim-bundle.sx — iOS-sim cross-compile + .app with
iOS-shaped Info.plist (added to tests/cross_compile.sh as the
ios-sim tuple).
- examples/122-ios-device-bundle.sx — iOS device cross-compile +
full codesign pipeline (provisioning embed + entitlements
extraction + --entitlements codesign). Manually verified end-to-end:
installed via xcrun devicectl device install app + launched
successfully on iPhone 17 Pro.
- examples/123-inline-if-import-in-body.sx — locks in the parser fix.
zig build && zig build test && bash tests/run_examples.sh => 141 passed,
0 failed; bash tests/cross_compile.sh => 7 passed, 0 failed.
|
||
|
agra
|
6a3260ff65 |
ffi 2.16c green: TL fallback via C-helper runtime + always-omit env in #jni_call
`#jni_call` collapses to a single surface — env is *always* implicit:
either picked up from the lexically-enclosing `#jni_env(env) { ... }`
block's Ref (cheap, register-resident, no TL touch) or from the
runtime's thread-local slot via `sx_jni_env_tl_get()` (one fn call
per dispatch). The explicit-env shape is gone — chess and the
existing tests migrate cleanly by wrapping their helper-fn bodies
in `#jni_env(env) { ... }`.
The TL slot lives outside the user's IR module so the LLVM ORC JIT
can load object files cleanly without `orc_rt` for TLS support:
library/vendors/sx_jni_runtime/sx_jni_env_tl.c:
static _Thread_local void *sx_jni_env_tl_slot;
void *sx_jni_env_tl_get(void) { return sx_jni_env_tl_slot; }
void sx_jni_env_tl_set(void *env) { sx_jni_env_tl_slot = env; }
Linkage:
- sx-the-compiler links the .c file via build.zig so the JIT
process-symbol generator resolves `sx_jni_env_tl_get`/`_set`.
- AOT targets get the same .c file auto-linked via the lowering
pass: when lower touches the TL externs, it sets
`needs_jni_env_tl_runtime`, and `Compilation.lowerToIR` appends a
synthetic `CImportInfo` to `lowering_extra_c_sources` that
`collectCImportSources` merges with user-written ones.
Lowering-side changes:
- `getJniEnvTlFids` lazily declares the two externs (parallel
to `getSelRegisterNameFid`) and flips `needs_jni_env_tl_runtime`.
- `#jni_env(env) { body }` emits save→set→body→restore via three
`call` ops to the externs; the inner body sees env via the
lexical-direct stack.
- `lowerJniCall` resolves env from `jni_env_stack` (top) or the TL
fallback. The explicit-env branch is gone.
- `jni_env_stack_base` tracks per-fn lexical scope so lazy-lowering
a callee doesn't accidentally see the caller's Ref (Refs are only
valid inside one fn's instruction stream).
Test migration (mechanical):
- ffi-jni-call-{01..09}: each helper fn wraps `#jni_call(...)`
bodies in `#jni_env(env) { ... }`. Returning values pass through
the block as an expression — `#jni_env` now also lowers in
expression position.
Verified:
- zig build test + tests/run_examples.sh: 130/130 green.
- tests/cross_compile.sh: 3/3 green.
- Chess APK rebuilt + reinstalled on Pixel. Board renders with
status-bar clearance + info panel intact; no crashes in logcat.
Safe-insets dispatch through `#jni_env` + lexical-direct now
fully exercised end-to-end on real hardware.
|
||
|
agra
|
0d883b412d |
ffi 1.17: #jni_call(name, sig) literal-keyed slot interning
Two `#jni_call` sites with the same string-literal `(name, sig)` pair
now share a single `jclass` GlobalRef slot and a single `jmethodID`
slot, populated lazily on the first call to any matching site.
Non-literal sites keep the per-call `GetObjectClass` + `GetMethodID`
sequence from step 1.15.
Per-call-site lowering for literal sites:
%cached_mid = load ptr, @SX_JNI_MID_<key>
%is_cached = icmp ne ptr %cached_mid, null
br i1 %is_cached, cont, miss
miss:
%local_cls = GetObjectClass(env, target)
%global_cls = NewGlobalRef(env, local_cls) ; vtable slot 21
store ptr %global_cls, @SX_JNI_CLS_<key>
%fresh_mid = GetMethodID(env, global_cls, name, sig)
store ptr %fresh_mid, @SX_JNI_MID_<key>
br cont
cont:
%mid = phi ptr [%cached_mid, before], [%fresh_mid, miss]
call <Type>Method(env, target, %mid, args...)
Wiring:
- `JniMsgSend.cache_key: ?CacheKey` (new) carries `(name_str,
sig_str)` when both `name` and `sig` are string-literal AST nodes;
empty for non-literal call sites.
- `lower.zig` populates `cache_key` from the AST.
- `emit_llvm.zig` `getOrCreateJniSlots(name, sig)` returns the
`{cls_slot, mid_slot}` pair, creating and caching them on first
lookup. Key is `name\x00sig` so the separator can't collide with
any JNI identifier byte.
- `mangleJniKey` builds an LLVM-identifier suffix from the pair, used
in the `@SX_JNI_{CLS,MID}_<suffix>` global names.
IR snapshot at `tests/expected/ffi-jni-call-03-methodid-sharing.ir`
updated: two call sites against literal `("noop", "()V")` now share
`@SX_JNI_CLS_noop____V` and `@SX_JNI_MID_noop____V`. Pre-1.17 snapshot
had two independent `GetMethodID` calls; post-1.17 has one global
slot pair plus per-call lazy-init branches.
Note: an unrelated regression in `examples/ffi-objc-call-12-rect-u64-returns.sx`
exists in the working tree (parse error from an in-progress C-import
block) and is left untouched.
|
||
|
agra
|
13018ef3b4 |
ffi 1.16: lock in pre-caching #jni_call IR shape
Adds `examples/ffi-jni-call-03-methodid-sharing.sx` with two `#jni_call` sites against the same (class, method, sig). Today each site emits its own `GetObjectClass` + `GetMethodID` + `Call<Type>Method` sequence (8 vtable indirections total for the two-call test); 1.17 will collapse the two `GetMethodID` calls into a single cached `jmethodID` static slot populated at module init, mirroring the `OBJC_SELECTOR_REFERENCES_*` shape that 1.5 introduced for `#objc_call`. Runtime is a no-op — `unused_jni` is reachable through a runtime-readable `g_should_call` global that stays false, so the JNI dereferences never execute. A plain `if false` would get constant-folded, taking the function definition out of the IR entirely; the global keeps both the function and its body present for the IR-snapshot harness. IR snapshot at `tests/expected/ffi-jni-call-03-methodid-sharing.ir` locks the pre-caching shape. The next commit (1.17) updates it to the collapsed shape. 113/113 host tests pass. |