sx/current/CHECKPOINT-EXTERN-EXPORT.md

# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)

Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
every commit, one step at a time per the cadence rule.

## Last completed step
**Phase 4** (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces:
(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) —
asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample
fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start
migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side
fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix
`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse
error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic
— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean
error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three
`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays
documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail).
NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a
harmless dup (both `.c`), and any non-`.c` callconv already errors on its own.

### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/
`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional
`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{`
body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime
class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx
class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same
`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/
lowering change was needed** — the new surface reuses the existing reference-vs-define path.
Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via
JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`),
**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443
unit, 0 fail).

### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
unit, 0 fail).

**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).

## Current state
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class`
(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the
**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A
COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` +
~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223
(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn,
AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426
(jni extern class), 1174/1175 (interplay diagnostics).

## Next step
**PART B — Phase 5 (`#foreign` becomes an alias for `extern`).** Part A is complete and
the A→B gate is locked, so migration can begin. Phase 5.0 (`lock`): route the four
`#foreign` parser paths (`parser.zig:316,425,1305,1970`) to build the **same** extern-named
AST as `extern`/`export` — suite green, all snapshots unchanged. Phase 5.1 (`lock`): unit
test that `#foreign` and `extern` produce identical IR (the gate already covers this — extend
or reuse `lowerSrcToIr`). Then Phases 6–7 migrate stdlib + examples (empty snapshot diff per
batch), Phase 8 cutover (hard-reject `#foreign`), Phase 9 total `foreign` purge.

**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target
(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep
`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames;
the plan says confirm before Phase 9, but worth raising with the user before sinking Part B
effort. **Also pick up the two Deferred items below at the start of Part B** (the
visibility-gate equivalence in particular needs a cross-module example).

**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).

**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md`
document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) visibility-gate
equivalence — bare `extern` (no `extern_lib`)
is currently unconditionally visible via the `c_import_bare` gate
(`decl.zig:~2241`, `fd.body.data != .foreign_expr → return true`), whereas a lib-less
`#foreign` is policed by `visibleOverEdges`. Single-file examples don't exercise this;
verify/align it at the A→B gate (a bare-extern lib-less fn should be policed like its
`#foreign` twin). Adding it now would be untested — needs a cross-module example.

## Open decisions
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B (confirm
before Phase 9): runtime-class rename target — `Runtime*Class*` (recommended);
historical carve-out — keep `issues/*.md` provenance, gate the live tree only.

## Log
- (init) Plan written; FFI-linkage stream opened.
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
  classification + `lex linkage keywords` test. Suite green; no identifier collisions
  in the corpus. `lock` commit.
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
  `VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
  parser unit tests. Suite green (443/633). `lock` commit.
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
  added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
  (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
  Suite green (443/633). `lock` commit.
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
  RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
  mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
  `lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
  `declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
  hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
  error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
  recovered a formatter-clobbered `parser.zig` — see Known issues.)
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
  `extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
  extern_name → declare under C name, map sx→C) and extends the dedupe guard to
  extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
  `green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
  mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
  error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
  `is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
  consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
  (636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
  (JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
  JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
  (`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
  milestone (see Next step). Spike reverted; no commit.
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
  `corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
  updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
  link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
  define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
  export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
  (637/443). `green` commit.
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
  `triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
  undefined). `xfail`; 2.2b greens it.
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
  sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
  the body promotes into the C-named function (`define @triple_c`). sx-side call sites
  resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
  green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
  new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
  directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.
- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl`
  (optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed
  `is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses
  the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443).
  `green` commit. **PHASE 3 COMPLETE.**
- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`,
  parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class,
  `counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per
  the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens`
  newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr
  0-byte per repo convention; runner normalizes both.)
- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`:
  `#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class.
  Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the
  equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three).
  Suite green (641/444). `test` commit.
- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign`
  + postfix `export` on an aggregate previously surfaced a confusing internal
  "emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`:
  `parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green.
- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the
  `#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit.
- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on
  one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`:
  `parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite
  green (643/444). **PHASE 4 COMPLETE → PART A DONE.** (Workflow note: `-Dupdate-goldens`
  keeps newline-normalizing the 5 empty-stderr files [1226/1227/1348/1349/1426] — revert
  that churn after each regen; a flaky `0712-sha256-streaming` timeout appears only under
  concurrent `zig build` load, not a real failure.)

## Known issues
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
  working-tree `src/parser.zig` between commits — it reformatted one-liners AND
  silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
  error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
  committed version). **After any Edit-tool change to a file the IDE may have open,
  rebuild + run the affected example before trusting the edit.**