sx/current/CHECKPOINT-EXTERN-EXPORT.md

sx extern/export + #foreign retirement — Checkpoint (FFI-linkage stream)

Companion to current/PLAN-EXTERN-EXPORT.md — one merged plan: Part A adds extern/export, Part B migrates #foreign and purges foreign. Update after every commit, one step at a time per the cadence rule.

Last completed step

Phase 3.1 (green) — PHASE 3 COMPLETE. Postfix extern/export on #objc_class/ #jni_class aggregates fully works. parseForeignClassDecl now parses an optional extern/export modifier in the slot between the ("X") directive args and the { body (parser.zig:~1409): extern→is_foreign_eff = true (reference an existing runtime class, == legacy #foreign); export→is_foreign_eff = false (define + register a new sx class, == bare #objc_class with no #foreign). The modifier maps straight onto the same is_foreign decision the prefix #foreign already fed the node, so no objc_class.zig/ lowering change was needed — the new surface reuses the existing reference-vs-define path. Examples: 1348 (objc extern import, dispatches NSObject.alloc().init() → green via JIT), 1349 (objc export defined class, SxBar.alloc()/bump/get → counter: 2), 1426 (jni extern import, parse-only parse-only ok). Suite green (641 corpus / 443 unit, 0 fail).

Prior: Phase 2.2 (green) — PHASE 2 COMPLETE. export (define + expose) fully works:

external linkage + C ABI + no sx ctx + force-lowered root + optional "csym" rename. All four export-gap conditions filled in decl.zig: (i) .external linkage for extern_export == .export_ on both define paths (lowerFunctionBodyInto, lowerFunction); (ii) C-ABI promotion on the define paths + declareFunction stub cc; (iv) funcWantsImplicitCtx returns false for any non-.none modifier; force-lower: export fns are lowering roots in lowerMainAndComptime (else an uncalled export fn stays a bodiless declare); (iii) export … "csym" declares the stub under the C name

• lazyLowerFunction promotes the body into it via foreign_name_map. Examples 1226 (bare export, C calls sx_square → 37/82) + 1227 (export "triple_c", C calls triple_c → 22) green via the new AOT corpus mode. Suite green (638 corpus / 443 unit, 0 fail).

AOT corpus mode + run_examples.sh retired. C→sx-by-name can't link under the corpus's sx run JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's flat-namespace lookup), so an expected/<name>.aot marker switches an example to a sx build + execute flow. The standalone tests/run_examples.sh was deleted — zig build test is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).

Current state

Syntax: bare extern/export, postfix after callconv(.c), extern ⇒ callconv(.c). Decision 4 revised (user 2026-06-14): extern carries an optional LIB+"csym" axis (extern_lib/extern_name) like #foreign; the #library decl + build-flag linking stays separate. extern (PHASE 1) + export (PHASE 2) FULLY WORKING. extern: functions — bare (f :: (…) -> R extern;) AND renamed (extern [LIB] "csym"); data globals — bare + renamed. export: functions — bare (f :: (…) -> R export {…}) AND renamed (export "csym"); external linkage, C ABI, no ctx, force-lowered as a root. All behavior-equivalent to the matching #foreign form. extern_lib is parsed + stored but is a reference only — actual linking stays the #library/build-flag axis. Aggregates NOT started (Phase 3). Part B foreign footprint to purge: 643 lines / ~57 identifiers in src/ + 28 doc lines. End-state invariant: zero foreign (Phase 9.4 gate). Examples: 1223 (extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn, AOT), 1227 (export fn rename, AOT).

Next step

Phase 4 — interplay, diagnostics, docs (+ the A→B gate). (a) Diagnostics for combining the surfaces: reject extern+export together; decide/handle prefix #foreign and postfix extern/export on the same aggregate (today the postfix silently overrides is_foreign — Phase 4 should reject the redundant/contradictory combo, mirroring the fn path). (b) extern+callconv stacking/redundancy on fns. (c) docs — specs.md/readme.md document the three extern/export axes (fns, globals, aggregates); #foreign stays documented until the Part B cutover. (d) GATE A→B (hard): unit test asserting #foreign and extern/export lower to identical IR for a sample fn / global / class — lock before any Part B migration. Also pick up the two Deferred items below at this gate. (interplay/diagnostics/docs + the A→B gate: unit test that #foreign and extern/export lower to identical IR) before Part B migration.

FUTURE MILESTONE — C→sx-by-name in JIT (sx run). Investigated this session (user-requested spike, RESOLVED feasible-but-blocked). Adding the C #source objects directly into the ORC JITDylib (LLVMOrcLLJITAddObjectFile) instead of dlopen'ing a dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a ~20-line spike ran 1226 via sx run (37/82) and all 13 existing #source FFI examples still passed. BLOCKER: C objects using _Thread_local (the return-trace runtime sx_trace.c) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC MachOPlatform set up (the bare LLVMOrcCreateLLJIT default doesn't), and C constructors/__mod_init_func won't run without ORC initializer support. 42 errors-* examples crashed in the spike. A real impl needs a C++ shim in llvm_shim.c (LLJITBuilder().setObjectLinkingLayerCreator(...) + MachOPlatform::Create) — its own milestone, NOT Phase 2/3 scope. The AOT .aot-marker corpus mode is the pragmatic test path and works today. Spike fully reverted (target.zig/main.zig at HEAD).

Deferred (do in Phase 4): (a) docs — specs.md/readme.md document extern/export (the plan defers docs to Phase 4; #foreign stays documented until the Part B cutover); (b) visibility-gate equivalence — bare extern (no extern_lib) is currently unconditionally visible via the c_import_bare gate (decl.zig:~2241, fd.body.data != .foreign_expr → return true), whereas a lib-less #foreign is policed by visibleOverEdges. Single-file examples don't exercise this; verify/align it at the A→B gate (a bare-extern lib-less fn should be policed like its #foreign twin). Adding it now would be untested — needs a cross-module example.

Open decisions

Part A ratified (bare / postfix / ⇒ callconv(.c) / lib-separate). Part B (confirm before Phase 9): runtime-class rename target — Runtime*Class* (recommended); historical carve-out — keep issues/*.md provenance, gate the live tree only.

Log

• (init) Plan written; FFI-linkage stream opened.
• (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
• (0.0) Added kw_extern/kw_export tokens + keyword-map entries + LSP keyword classification + lex linkage keywords test. Suite green; no identifier collisions in the corpus. lock commit.
• (0.1) Added ast.ExternExportModifier + FnDecl.extern_export + VarDecl.is_extern/extern_name + parseOptionalExternExport() (unconsumed) + 2 parser unit tests. Suite green (443/633). lock commit.
• (1.0a) Wired fn-path extern parsing (parseFnDecl + both lookahead predicates) + added FnDecl.extern_lib/extern_name + VarDecl.extern_lib per user feedback (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering. Suite green (443/633). lock commit.
• (1.0b) Added examples/1223-ffi-extern-fn.sx + hand-authored success snapshots. RED (634 ran, 1 failed — sema body produces no value). xfail commit; 1.1 greens it.
• (1.1) Wired extern fn lowering (6 edits in decl.zig, all declare-only routing mirroring foreign_expr): funcWantsImplicitCtx + declareFunction cc + lazyLowerFunction/lowerFunction/lowerFunctionBodyInto guards. 1223 green; declare i32 @abs(i32) (C ABI, no ctx). Suite green (634/443). green commit.
• (1.2a) Added examples/1224-ffi-extern-fn-rename.sx (c_abs :: … extern "abs";) + hand-authored success snapshot (c_abs(-42) = 42). RED (635 ran, 1 failed — parse error: "abs" after extern not yet accepted). xfail; 1.2b greens it. (Also recovered a formatter-clobbered parser.zig — see Known issues.)
• (1.2b) parseFnDecl parses the optional [LIB] ["csym"] tail into extern_lib/extern_name; declareFunction unifies the rename (foreign c_name OR extern_name → declare under C name, map sx→C) and extends the dedupe guard to extern. 1224 green (c_abs→abs); 1223 unregressed. Suite green (635/443). green commit. extern_lib parsed+stored (lib linking stays the #library axis).
• (1.2c) Added examples/1225-ffi-extern-global.sx (__stdinp : *void extern;, mirrors #foreign global 1205) + success snapshot. RED (636 ran, 1 failed — parse error: var-decl extern not accepted). xfail; 1.2d greens it.
• (1.2d) Parser kw_extern branch in the var-decl path ([LIB] ["csym"] → is_extern/extern_lib/extern_name) + registerTopLevelGlobal/globalInitValue consume is_extern. 1225 green (@__stdinp = external global ptr). Suite green (636/443). green commit. PHASE 1 COMPLETE — extern fns + globals fully work.
• (JIT spike) User-requested feasibility investigation of C→sx-by-name in sx run (JIT). Verdict: feasible via LLVMOrcLLJITAddObjectFile (C objects into the ORC JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling (sx_trace.c's _Thread_local SIGABRTs without the ORC MachOPlatform). Own future milestone (see Next step). Spike reverted; no commit.
• (2.0) Added the AOT corpus mode (expected/<name>.aot → sx build + execute) to corpus_run.test.zig + retired tests/run_examples.sh (verify-step.sh/CLAUDE.md updated) + examples/1226-ffi-export-fn.{sx,c,h} (C calls sx_square back). RED (AOT link fails: _sx_square undefined — export not lowered). xfail; 2.1 greens it.
• (2.1) Filled export gaps i/ii/iv in decl.zig (.external linkage + .c cc on both define paths; funcWantsImplicitCtx false for any non-.none modifier) + force-lower export fns as roots in lowerMainAndComptime. 1226 green via AOT (37/82). Suite green (637/443). green commit.
• (2.2a) Added examples/1227-ffi-export-fn-rename.sx (export "triple_c", C calls triple_c). RED (define path emits @sx_triple, ignores extern_name → C ref undefined). xfail; 2.2b greens it.
• (2.2b) declareFunction rename branch fires for export (stub under C name + sx→C in foreign_name_map); lazyLowerFunction resolves the stub by that C name so the body promotes into the C-named function (define @triple_c). sx-side call sites resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite green (638/443). green commit. PHASE 2 COMPLETE — export fully works.
• (3.0) Added examples/1348-ffi-objc-extern-class.sx (postfix extern on #objc_class, new spelling of #foreign #objc_class). RED (parser: expected '{' after the directive). Hand-authored green snapshots. xfail commit; 3.1 greens it.
• (3.1a) Wired the postfix extern/export aggregate slot in parseForeignClassDecl (optional modifier between ("X") and {; var is_foreign_eff overrides the passed is_foreign, threaded into the foreign_class_decl node). No lowering change — reuses the existing is_foreign reference-vs-define path. 1348 green. Suite green (639/443). green commit. PHASE 3 COMPLETE.
• (3.1b) Behavior-lock: added examples/1426-ffi-jni-extern-class.sx (jni extern, parse-only) + examples/1349-ffi-objc-export-class.sx (objc export defined class, counter: 2). Both pass against the 3.1a parser change (locked in their own commit per the cadence rule). Suite green (641/443). lock commit. (Note: -Dupdate-goldens newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr 0-byte per repo convention; runner normalizes both.)

Known issues

• Workflow hazard (1.2): an editor format-on-save (or zig fmt) clobbered the working-tree src/parser.zig between commits — it reformatted one-liners AND silently dropped my hasFnBodyAfterArrow extern edit, reverting 1223 to a parse error. Recovered with git checkout src/parser.zig (HEAD had the correct, committed version). After any Edit-tool change to a file the IDE may have open, rebuild + run the affected example before trusting the edit.