sx/current/CHECKPOINT-EXTERN-EXPORT.md

# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)

Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
every commit, one step at a time per the cadence rule.

## Last completed step
**Phase 3.0** (xfail) — `examples/1348-ffi-objc-extern-class.sx` exercises the new
postfix-`extern` aggregate spelling (`NSObject :: #objc_class("NSObject") extern { … }`,
the new way to write `#foreign #objc_class("NSObject") { … }`). RED: parser rejects
`extern` after the `#objc_class(...)` directive (`expected '{'`). Green-state snapshots
hand-authored (exit 0, stdout `extern-class dispatch ok`). Suite: 639 corpus / 443 unit,
**1 fail (1348, the intended xfail)**. Step 3.1 wires parse+lowering to green it.

### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
unit, 0 fail).

**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).

## Current state
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
Aggregates NOT started (Phase 3). Part B `foreign` footprint to purge: 643 lines / ~57
identifiers in `src/` + 28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4
gate). Examples: 1223 (extern bare fn), 1224 (extern fn rename), 1225 (extern bare
global), 1226 (export bare fn, AOT), 1227 (export fn rename, AOT).

## Next step
**Phase 3.1** (green) — wire the postfix `extern`/`export` aggregate path. `parseForeignClassDecl`
already consumes the directive + `("X")` + body; add an optional postfix-modifier slot
**after** the `)` and before the `{`: `extern`→reference (== `is_foreign`), `export`→
define+register (== no `#foreign`). Map the postfix modifier onto the same downstream
`is_foreign` decision that `tryParseForeignClassPrefix` feeds, so `objc_class.zig` lowering
is unchanged. Green 1348; add a jni postfix test + an `export` (defined-class) test for the
per-runtime coverage. Then Phase 4 (interplay/diagnostics/docs + the A→B gate: unit test that
`#foreign` and `extern`/`export` lower to identical IR) before Part B migration.

**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).

**Deferred (do in Phase 4):** (a) docs — `specs.md`/`readme.md` document
`extern`/`export` (the plan defers docs to Phase 4; `#foreign` stays documented until
the Part B cutover); (b) visibility-gate equivalence — bare `extern` (no `extern_lib`)
is currently unconditionally visible via the `c_import_bare` gate
(`decl.zig:~2241`, `fd.body.data != .foreign_expr → return true`), whereas a lib-less
`#foreign` is policed by `visibleOverEdges`. Single-file examples don't exercise this;
verify/align it at the A→B gate (a bare-extern lib-less fn should be policed like its
`#foreign` twin). Adding it now would be untested — needs a cross-module example.

## Open decisions
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B (confirm
before Phase 9): runtime-class rename target — `Runtime*Class*` (recommended);
historical carve-out — keep `issues/*.md` provenance, gate the live tree only.

## Log
- (init) Plan written; FFI-linkage stream opened.
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
  classification + `lex linkage keywords` test. Suite green; no identifier collisions
  in the corpus. `lock` commit.
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
  `VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
  parser unit tests. Suite green (443/633). `lock` commit.
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
  added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
  (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
  Suite green (443/633). `lock` commit.
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
  RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
  mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
  `lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
  `declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
  hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
  error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
  recovered a formatter-clobbered `parser.zig` — see Known issues.)
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
  `extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
  extern_name → declare under C name, map sx→C) and extends the dedupe guard to
  extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
  `green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
  mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
  error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
  `is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
  consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
  (636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
  (JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
  JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
  (`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
  milestone (see Next step). Spike reverted; no commit.
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
  `corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
  updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
  link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
  define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
  export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
  (637/443). `green` commit.
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
  `triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
  undefined). `xfail`; 2.2b greens it.
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
  sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
  the body promotes into the C-named function (`define @triple_c`). sx-side call sites
  resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
  green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
  new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
  directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.

## Known issues
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
  working-tree `src/parser.zig` between commits — it reformatted one-liners AND
  silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
  error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
  committed version). **After any Edit-tool change to a file the IDE may have open,
  rebuild + run the affected example before trusting the edit.**