sx/current/PLAN-EXTERN-EXPORT.md

# sx `extern` / `export` + `#foreign` retirement — Plan (FFI-linkage stream)

**One stream, two parts.** **Part A** adds `extern`/`export` (the linkage surface);
**Part B** migrates every `#foreign` onto it and purges `foreign` from the tree.
They are *one* plan: Part B can't start until Part A is a behavior-equivalent
superset of `#foreign`, and Part A isn't "done" until Part B reaches the invariant.

**Design rationale:** [design/inline-asm-design.md](../design/inline-asm-design.md) §II.2
(Deviation 6) + §II.10 #4 + the syntax evaluation.

**Decided syntax**
```sx
name :: (sig) -> Ret  [callconv(.x)]  [extern | export]  [LIB] ["csym"]  [;|{…}];   // functions
Name :: #objc_class("X")  [extern | export]  { … };                                 // aggregates (mirrors `struct #compiler`)
g    :  Type  extern  [LIB] ["csym"];                                               // extern global
```
- `extern` = import (no body, external linkage, C ABI, no sx ctx) — `#foreign`'s role.
- `export` = define **and** expose (body + external linkage + C ABI + no ctx) — **new**.
- `extern`/`export` imply `callconv(.c)`; write `callconv` only to override.
- Optional `LIB` (a `#library` alias) + `"csym"` rename mirror `#foreign LIB "csym"`,
  so `extern` is a true `#foreign` **superset** (Gate A→B): carried on
  `extern_lib`/`extern_name`. The `#library` declaration + build-flag linking
  mechanism stays a separate axis — `extern` *references* a lib, it doesn't fold
  in `#library` itself. (Revises the original "library fully separate" decision 4.)

> **END-STATE INVARIANT (hard requirement).** After this stream, `foreign` appears
> **nowhere** in the live tree — not the `#foreign` surface, and **not** internal
> identifiers. The extern AST is **not** named `foreign_expr`. Enforced by the
> Phase 9.4 grep gate. Scope today: 643 `foreign` lines / ~57 identifiers in `src/`
> + 28 in live docs — most of it the objc/jni **runtime-class** machinery.

**Naming constraint (so we can actually reach the invariant):** introduce
`extern`-named representations only — do **not** reuse or extend
`ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Carry extern/export on a new
`FnDecl.extern_export` modifier with a `;`/`{…}` body (so there is **no** `*_expr`
node for it) + `FnDecl.extern_lib`/`extern_name`; add `VarDecl.is_extern`/
`extern_lib`/`extern_name`. The IR is already extern-named (`Function.is_extern`,
`Builder.declareExtern`).

**Key finding (scopes Part A):** the IR + LLVM emit **already support everything** —
`Function.linkage` (`.external/.internal/.private`), `is_extern`, `call_conv`, and a
raw un-mangled symbol name are all emitted by `declareFunction`
(`emit_llvm.zig:1225-1300`). Part A is a **parser + lowering** job, no codegen change.

## Cadence (IMPASSIBLE)
No commit may both add a test AND make it pass (xfail-then-green, or a behavior-lock).
`zig build && zig build test` after every step. Never regenerate snapshots while red.

---

# PART A — add `extern` / `export` (alongside `#foreign`)

## Phase 0 — tokens + parser plumbing
| Step | Commit | What | Files |
|---|---|---|---|
| 0.0 | lock | add `kw_extern`, `kw_export` (Tag enum + `StaticStringMap`, beside `kw_callconv` at `token.zig:45,282`); unit lex test | `src/token.zig` |
| 0.1 | lock | `parseOptionalExternExport()` (mirror `parseOptionalCallConv`, `parser.zig:3669`) + `ast.ExternExportModifier` enum + `FnDecl.extern_export` + `VarDecl.is_extern`/`extern_name` fields; **not yet consumed**; unit AST test | `src/parser.zig`, `src/ast.zig` |

## Phase 1 — `extern` (import; equivalent to lib-less `#foreign`)
| Step | Commit | What | Files |
|---|---|---|---|
| 1.0 | xfail | accept postfix `extern` after the callconv slot (`parser.zig:1950`); `examples/12xx-ffi-extern-fn.sx` extern-binds a libc symbol — red (lowering not wired) | `src/parser.zig` |
| 1.1 | green | lowering: `extern` ⇒ `is_extern`, `.external`, `callconv(.c)`, no ctx — route through `declareExtern` like a lib-less `#foreign` (anchors `decl.zig:1123,387,2110,2113`). Example green | `src/ir/lower/decl.zig` |
| 1.2 | green | optional `extern "csym"` rename + extern-global form `g : T extern;` (`parser.zig:425` path) | `src/parser.zig`, `src/ir/lower/decl.zig` |

## Phase 2 — `export` (define + expose; the NEW capability)
Fills the four export-gap conditions (all in `src/ir/lower/decl.zig`):
| Gap | Anchor | Fix |
|---|---|---|
| (i) linkage forced `.internal` | `:2382`, `:2514` | also `.external` when `extern_export == .export` |
| (ii) C ABI not promoted | `:2110` | also `.c` when `== .export` |
| (iii) no symbol-name override | `emit_llvm.zig:1226` raw name | parse optional `export "csym"`; map in the name map |
| (iv) ctx param not suppressed | `:387` `funcWantsImplicitCtx` | also suppress when `== .export` |

| Step | Commit | What | Files |
|---|---|---|---|
| 2.0 | xfail | multi-file test: an `export fn` called from a companion `.c` caller (same `XXXX-` prefix) — red (still internal) | `examples/12xx-ffi-export-fn.{sx,c}` + `expected/` |
| 2.1 | green | gaps (i),(ii),(iv): `export` ⇒ external + C-ABI + no-ctx on a **defined** fn (uses `beginFunction`, not `declareExtern`) | `src/ir/lower/decl.zig` |
| 2.2 | green | gap (iii): `export "csym"` symbol-name override | `src/parser.zig`, `src/ir/lower/decl.zig` |

## Phase 3 — aggregates (objc / jni runtime classes)
| Step | Commit | What | Files |
|---|---|---|---|
| 3.0 | xfail | `#objc_class("X") extern { … }` (import) + `… export { … }` (define) parse alongside legacy `#foreign #objc_class` | `src/parser.zig` (`tryParseForeignClassPrefix` :1305, `parseForeignClassDecl` :1369) |
| 3.1 | green | map postfix `extern`→reference, `export`→define+register; per-runtime tests (objc, jni) | `src/parser.zig`, `src/ir/lower/decl.zig`, `src/ir/lower/objc_class.zig` |

## Phase 4 — interplay, diagnostics, docs
`extern`+`callconv` stacking/redundancy; reject `extern`+`export` together;
`specs.md` documents `extern`/`export` (the three axes); `#foreign` still documented
until Part B cutover.

> **GATE A→B.** `extern`/`export` are a behavior-equivalent **superset** of
> `#foreign`. Lock with a unit test asserting `#foreign` and `extern` lower to
> identical IR for a sample fn / global / class. Do not start Part B before this.

---

# PART B — migrate `#foreign` → `extern`/`export`, then purge `foreign`

**Inventory (drives the batches):** `#foreign` = 466 uses. ~391 sx-code (308 fns
[207 lib / 196 rename], 75 classes [39 objc / 31 jni], 8 globals) + ~145 example
snapshots. 6 libs (`sqlib`98 `libc`61 `objc`22 `tlib`12 `raylib`7 `clib/pcaplib`3).
Hotspots: `vendors/sqlite`(98), `platform/{android,uikit,android_jni,sdl3}`,
`std/{socket,thread,fs,time}`, `ffi/{objc,raylib}`.

## Phase 5 — `#foreign` becomes an alias for `extern`
| Step | Commit | What | Files |
|---|---|---|---|
| 5.0 | lock | route the `#foreign` parser paths (`parser.zig:316,425,1305,1970`) to build the *same extern-named* AST as `extern`/`export`. Suite green, snapshots unchanged | `src/parser.zig` |
| 5.1 | lock | unit test: `#foreign` and `extern` produce identical IR (fn/global/class) | `src/ir/lower/decl.test.zig` |

## Phase 6 — migrate stdlib (behavior-preserving; snapshot diff must be EMPTY)
One commit per batch; rewrite `#foreign`→`extern` (fns/globals),
`#foreign #objc_class`→`#objc_class … extern`, defined classes → `… export`.
| Step | Batch | ~sites |
|---|---|---|
| 6.1 | `library/vendors/sqlite/` | 98 |
| 6.2 | `library/modules/platform/` (uikit/android/android_jni/sdl3) | ~95 |
| 6.3 | `library/modules/std/` (socket/thread/fs/time/process/…) | ~60 |
| 6.4 | `library/modules/ffi/` (objc/raylib/objc_block/…) | ~50 |
| 6.5 | remaining `library/` + `vendors/` | remainder |

## Phase 7 — migrate examples + issues (empty snapshot diff; review every diff)
| Step | Batch |
|---|---|
| 7.1 | `examples/12xx-ffi-*` (plain C) |
| 7.2 | `examples/13xx-ffi-objc-*` |
| 7.3 | `examples/14xx-ffi-jni-*` |
| 7.4 | `issues/*` repros + stragglers |
A non-empty diff ⇒ the alias wasn't behavior-equivalent — stop, fix Phase 5.

## Phase 8 — cutover
| Step | Commit | What |
|---|---|---|
| 8.0 | xfail | `examples/11xx-diagnostics-foreign-removed.sx` expects a "`#foreign` removed; use `extern`/`export`" diagnostic — still accepted (red) |
| 8.1 | green | parser hard-rejects `#foreign` (mirrors the variadic `name: ..T` cutover); `specs.md` drops `#foreign`, documents `extern`/`export` |

## Phase 9 — total `foreign` purge (the invariant)
`foreign` must not appear anywhere in the live tree, surface *or* internal. Each step
a mechanical, behavior-preserving rename commit (snapshots unchanged), small
per-file/subsystem commits — not one sweep.
| Step | What | Identifiers (count → new) |
|---|---|---|
| 9.0 | delete the surface | `hash_foreign`(11) + lexer entry + the 4 parse paths + the alias |
| 9.1 | rename **linkage** → `extern*` | `foreign_expr`(25) **eliminated** (folds into modifier) · `is_foreign`(39)→`is_extern` · `foreign_lib`/`foreign_name`→`extern_*` · `foreign_name_map`→`extern_name_map` · `callForeign`(8)→`callExtern` · `marshalForeignArg`→`marshalExternArg` · `is_foreign_c_api`(5)→`is_extern_c_api` · `dedupeForeignSymbol`→`dedupeExternSymbol` |
| 9.2 | rename **runtime-class** machinery → `runtime*` (decision 5) | `ForeignClassDecl`(65) · `ForeignMethodDecl`(31) · `ForeignClassMember`(20) · `ForeignFieldDecl`(15) · `foreign_class_map`(44) · `current_foreign_class`(34)/`_method` · `foreign_path`(62) · `ForeignRuntime` · `parse/tryParseForeignClass*` · `lowerForeign{Method,Static}Call` · `findForeign{Method,Property}InChain` · `resolveForeign*` · `register*ForeignClass*` · `foreignClass*Type` · `*ForeignRefs` |
| 9.3 | purge **live docs** (28 lines) | `specs.md`/`readme.md`/`CLAUDE.md`: drop `#foreign`, document `extern`/`export`; fix file-roles + FFI/bundling notes |
| 9.4 | **acceptance gate** | `grep -rniE 'foreign' src/ library/ examples/ specs.md readme.md CLAUDE.md` → **0** |

---

## Open decisions
*Part A (ratified — recommendations stand):* 1. bare keywords (not `#extern`).
2. aggregate position postfix (`#objc_class(…) extern`, like `struct #compiler`).
3. `extern ⇒ callconv(.c)`. 4. **REVISED** (user, 2026-06-14): `extern` carries an
optional `LIB`+`"csym"` axis (`extern_lib`/`extern_name`), mirroring `#foreign LIB
"csym"`, so it's a true `#foreign` superset (Gate A→B). The `#library` declaration +
build-flag linking mechanism stays separate — `extern` references a lib, doesn't
fold in `#library`. (Was: "library fully separate / not on `extern`".)
*Part B:* 5. runtime-class rename target — **RATIFIED `Runtime*Class*`** (user, 2026-06-14;
it's the object-model axis, not linkage). 6. historical carve-out — **STILL OPEN** (user did
not confirm at the Part A milestone): keep `issues/*.md` (+ design-doc prose) as provenance &
gate only the live tree (recommended) vs purge everything. Confirm 6 before Phase 9.

## Relationship to ASM
`PLAN-ASM.md` Phase F (global asm) consumes `extern` (import the asm symbol) and
`export` (let asm call back into sx) — do it after **Part A Phase 2**.

---

## Kickoff prompt (paste into a fresh session to start Part A)

> Work the FFI-linkage stream per `current/PLAN-EXTERN-EXPORT.md` (+ checkpoint
> `current/CHECKPOINT-EXTERN-EXPORT.md`). First read the plan's header (Decided
> syntax, Naming constraint, Key finding) and Part A; rationale is in
> `design/inline-asm-design.md` §II.2 (Deviation 6) + §II.10 #4.
>
> **This session = Part A, Phases 0 and 1 only** (`extern` works as a bare postfix
> keyword equivalent to a lib-less `#foreign` fn/global binding; `#foreign` stays
> untouched). Do NOT start Phase 2 (`export`) or Part B (migration).
>
> **Cadence (IMPASSIBLE):** no commit may both add a test and make it pass — lock
> behavior with a passing test, or land an xfail the next commit turns green.
> `zig build && zig build test` after every step.
>
> **Naming constraint (hard):** introduce only `extern`-named AST — do NOT reuse or
> extend `ForeignExpr`/`foreign_expr`/`VarDecl.is_foreign`. Use a new
> `FnDecl.extern_export` modifier (body `;` or `{…}`) and `VarDecl.is_extern`/
> `extern_name`. IR is already extern-named (`Function.is_extern`, `declareExtern`).
>
> Steps (commit after each; update the checkpoint each time):
> - 0.0 lock: `kw_extern`/`kw_export` tokens + map entries beside `kw_callconv`
>   (`src/token.zig:45,282`) + unit lex test.
> - 0.1 lock: `parseOptionalExternExport()` (mirror `parseOptionalCallConv`,
>   `parser.zig:3669`) + `ast.ExternExportModifier` + `FnDecl.extern_export` +
>   `VarDecl.is_extern`/`extern_name` (parsed, unconsumed) + unit AST test.
> - 1.0 xfail: accept postfix `extern` after the callconv slot (`parser.zig:1950`);
>   add `examples/12xx-ffi-extern-fn.sx` that extern-binds a libc symbol (red).
> - 1.1 green: in `src/ir/lower/decl.zig`, lower `extern` like a lib-less `#foreign`
>   import — `is_extern`, `.external`, `callconv(.c)`, no ctx, via `declareExtern`
>   (anchors :1123, :387, :2110, :2113). Example goes green.
> - 1.2 green: optional `extern "csym"` rename + extern-global `g : T extern;`
>   (`parser.zig:425`).
>
> Stop at end of Phase 1. Verify: suite green; the `extern` libc binding runs;
> `#foreign` still works with no snapshot diffs. If you hit an unrelated compiler
> bug, follow the CLAUDE.md IMPASSIBLE RULE (file an issue, stop).