sx/current/CHECKPOINT-EXTERN-EXPORT.md

# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)

Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
every commit, one step at a time per the cadence rule.

## Last completed step
**Phase 5.0 prereq — visibility-gate equivalence** (xfail `717c35d` → fix
`7d8ba1a`) — the first of the two deferred fn-path prerequisites is now LOCKED.
The non-transitive C-import visibility gate (`isVisible(.c_import_bare)`,
`decl.zig:2249`) used to recognise only the legacy `#foreign` body shape; a bare
`extern` fn (empty-block body + `extern_export == .extern_`) escaped the gate via
the `body != foreign_expr → return true` arm and was caught only by the general
`isNameVisible` gate — yielding the generic "not visible" wording instead of the
C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings
route to `visibleOverEdges`, and a library-bound `extern LIB` (like `#foreign LIB`)
stays unconditionally visible — so a future fn-decl `#foreign`→`extern` migration
is byte-identical at this gate. New cross-module example **1228**
(`examples/1228-ffi-extern-c-non-transitive`, main → b → c) pins the equivalence:
referencing c's lib-less `#foreign` AND `extern` twins transitively both produce
the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed).
**Empirical finding** (probe, not yet acted on): the bare-extern twin was NEVER a
silent visibility hole — the general `isNameVisible` gate already denied it; only
the *diagnostic wording* diverged. The fix aligns the wording + gate ownership.

### Prior: Phase 5.0 (global path) (`refactor` lock, commit `e5ddfbe`) — **PART B STARTED.**
First of the four `#foreign` parser paths migrated onto the extern AST: the
data-global form `name : T #foreign [lib] ["csym"];` now builds the same
extern-named `VarDecl` (`is_extern`/`extern_lib`/`extern_name`) that postfix
`extern` already produces, instead of `is_foreign`/`foreign_lib`/`foreign_name`.
Behavior-preserving — lowering coalesces both forms identically
(`decl.zig:1119,1127,1141`), so zero snapshot churn. The fn-decl, const-with-type,
and runtime-class `#foreign` paths still build the legacy AST.

### Prior: Phase 4 (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces:
(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) —
asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample
fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start
migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side
fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix
`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse
error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic
— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean
error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three
`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays
documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail).
NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a
harmless dup (both `.c`), and any non-`.c` callconv already errors on its own.

### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/
`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional
`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{`
body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime
class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx
class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same
`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/
lowering change was needed** — the new surface reuses the existing reference-vs-define path.
Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via
JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`),
**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443
unit, 0 fail).

### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
unit, 0 fail).

**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).

## Current state
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class`
(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the
**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A
COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` +
~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223
(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn,
AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426
(jni extern class), 1174/1175 (interplay diagnostics).

## Next step
**PART B — Phase 5.0, fn-path variadic prerequisite** (the SECOND of the two
deferred fn-path prereqs; visibility-gate equivalence is now DONE). The
`is_variadic` drop in `declareFunction` is gated on `is_foreign` only, so a
migrated variadic `extern` (e.g. `printf`) would lose its `...` tail. Extend the
gate to cover `extern_export == .extern_`. Cadence: xfail an `extern` variadic fn
losing its `...` (locks the gap), then the next commit fixes the gate. Find the
exact site via `grep -n "is_variadic" src/ir/lower/decl.zig` (the checkpoint's old
`decl.zig:2097` line ref may have drifted). AFTER both prereqs land, the fn-decl
`#foreign` body-marker path (`parser.zig:~2062`) can migrate onto `extern`.

**Investigation findings (this session — reorder the remaining paths):**
- **const-with-type** (`parser.zig:316`, `name :: type_expr #foreign …`) is a
  **DEAD path**: it builds `const_decl{value = foreign_expr}`, but
  `registerTypedModuleConst` (`decl.zig:848-851`) bails on a `foreign_expr` value
  (`else => return`), so it registers no const and emits no symbol — a probe
  (`g_abs :: FP #foreign "abs";`) returns `unresolved 'g_abs'` at the use site, and
  the form is used NOWHERE in `library`/`examples`/`issues`. Its migration target is
  ambiguous because the `foreign_expr` value node is SHARED with the fn-decl path,
  which isn't migrated yet. **Decision (user, 2026-06-14): defer it — migrate it
  alongside the fn-decl path once `foreign_expr`'s extern shape is decided.** The
  checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the
  "confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it).
- **runtime-class prefix** (`parser.zig:~1351`, `#foreign #objc_class/#jni_class`) is
  **ALREADY coalesced**: both prefix `#foreign` and postfix `extern` feed the single
  `is_foreign_eff`→`is_foreign` field on `foreign_class_decl` (`parser.zig:1421-1432`),
  so there is NO Phase 5.0 AST change for it — only the Phase 9.2 `Runtime*Class*`
  rename remains. Drop it from the Phase 5.0 path list.

So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then
the fn-decl `#foreign` body-marker migration. const-with-type + runtime-class need
no standalone Phase 5.0 commit.

Then Phase 5.1 (`lock`): unit test that `#foreign` and `extern` produce identical IR (the
A→B gate already covers fn/global/class — extend or reuse `lowerSrcToIr`). Then Phases 6–7
migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject
`#foreign`), Phase 9 total `foreign` purge.

**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target
(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep
`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames;
the plan says confirm before Phase 9, but worth raising with the user before sinking Part B
effort. **Also pick up the two Deferred items below at the start of Part B** (the
visibility-gate equivalence in particular needs a cross-module example).

**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).

**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md`
document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) ~~visibility-gate
equivalence~~ — **DONE** (`717c35d`/`7d8ba1a`): the `c_import_bare` gate now polices a
lib-less `extern` fn identically to its lib-less `#foreign` twin (same C-specific
diagnostic); a library-bound `extern LIB` stays unconditionally visible. Locked by the
cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the
general `isNameVisible` gate already denied it; only the diagnostic wording diverged.)

## Open decisions
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B:
- **Decision 5 RATIFIED** (user, 2026-06-14): runtime-class rename target = `Runtime*Class*`
  (object-model axis, not linkage). Drives the Phase 9.2 identifier renames.
- **Decision 6 STILL OPEN**: historical carve-out — keep `issues/*.md` (+ design-doc prose)
  as provenance & gate only the live tree (recommended) vs purge everything. The user did
  NOT confirm this at the Part A milestone; confirm before Phase 9.

## Log
- (5.0 prereq vis xfail) Added cross-module example `1228-ffi-extern-c-non-transitive`
  (main → b → c). Main references c's lib-less `#foreign` + `extern` twins
  transitively; expected snapshot pins the DESIRED equivalent C-specific
  diagnostic for both. RED (extern twin gets the generic "not visible" wording —
  443/444). `test`/xfail commit `717c35d`; the fix greens it.
- (5.0 prereq vis fix) Extended `isVisible(.c_import_bare)` (`decl.zig:2249`) to
  switch on the body: a `foreign_expr` body OR an `extern_export == .extern_` decl
  with no lib both route to `visibleOverEdges`; a library-bound decl stays
  unconditionally visible. 1228 green — both twins emit "C function not visible".
  Suite green (644 corpus / 444 unit, 0 failed). `fix`/green commit `7d8ba1a`.
  **Deferred prereq (b) CLOSED.** Investigation this session also found
  const-with-type is a DEAD parser path (defer per user) and the runtime-class
  prefix is already coalesced (no Phase 5.0 change) — see Next step.
- (5.0 global) **PART B STARTED.** Routed the `#foreign` data-global parser path
  (`parser.zig:425`) onto the extern-named `VarDecl` (`is_extern`/`extern_lib`/
  `extern_name`) — the same AST postfix `extern` builds. Behavior-preserving
  (lowering coalesces both at `decl.zig:1119,1127,1141`); zero snapshot churn. Suite
  green (444/444 unit, 643 corpus). `refactor` lock, commit `e5ddfbe`. Remaining
  Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic
  prereqs), runtime-class prefix (1305).
- (init) Plan written; FFI-linkage stream opened.
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
  classification + `lex linkage keywords` test. Suite green; no identifier collisions
  in the corpus. `lock` commit.
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
  `VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
  parser unit tests. Suite green (443/633). `lock` commit.
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
  added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
  (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
  Suite green (443/633). `lock` commit.
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
  RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
  mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
  `lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
  `declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
  hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
  error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
  recovered a formatter-clobbered `parser.zig` — see Known issues.)
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
  `extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
  extern_name → declare under C name, map sx→C) and extends the dedupe guard to
  extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
  `green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
  mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
  error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
  `is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
  consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
  (636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
  (JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
  JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
  (`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
  milestone (see Next step). Spike reverted; no commit.
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
  `corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
  updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
  link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
  define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
  export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
  (637/443). `green` commit.
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
  `triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
  undefined). `xfail`; 2.2b greens it.
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
  sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
  the body promotes into the C-named function (`define @triple_c`). sx-side call sites
  resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
  green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
  new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
  directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.
- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl`
  (optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed
  `is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses
  the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443).
  `green` commit. **PHASE 3 COMPLETE.**
- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`,
  parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class,
  `counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per
  the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens`
  newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr
  0-byte per repo convention; runner normalizes both.)
- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`:
  `#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class.
  Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the
  equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three).
  Suite green (641/444). `test` commit.
- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign`
  + postfix `export` on an aggregate previously surfaced a confusing internal
  "emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`:
  `parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green.
- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the
  `#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit.
- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on
  one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`:
  `parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite
  green (643/444). **PHASE 4 COMPLETE → PART A DONE.**
- (golden-fix) **`-Dupdate-goldens` churn RESOLVED.** Root cause was NOT a code bug:
  `writeGolden` always writes `content + "\n"` (empty → canonical 1-byte `\n`, used by 484
  of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte
  *outliers* (verify trims trailing `\n` so both forms passed, but regen always rewrote them
  to 1-byte). Conformed all 5 to the 1-byte form → `-Dupdate-goldens` is now idempotent, no
  more churn. (Separately: a flaky `0712-sha256-streaming` >10s timeout appears only under
  concurrent `zig build` load — not a real failure; re-run serially.)

## Known issues
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
  working-tree `src/parser.zig` between commits — it reformatted one-liners AND
  silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
  error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
  committed version). **After any Edit-tool change to a file the IDE may have open,
  rebuild + run the affected example before trusting the edit.**