Files
sx/current/CHECKPOINT-EXTERN-EXPORT.md

48 KiB
Raw Blame History

sx extern/export + #foreign retirement — Checkpoint (FFI-linkage stream)

Companion to current/PLAN-EXTERN-EXPORT.md — one merged plan: Part A adds extern/export, Part B migrates #foreign and purges foreign. Update after every commit, one step at a time per the cadence rule.

Last completed step

Phase 8 — CUTOVER: parser hard-rejects #foreign (feat! commit 3811311, preceded by the 8.0 xfail 8180faf + 3 pre-cutover refactors 2cce6a3/720556b/ d132aab). PHASE 8 COMPLETE. The prefix #foreign linkage directive is removed: all four parse sites (const-with-type 316, data global 425, fn body 2065, runtime-class prefix via caller 260) reject it with the migration message "#foreign has been removed; use the postfix extern (import) / export (define) linkage keyword instead"; added a span-aware failAt for the runtime-class case (the lookahead consumes the token before the reject decision). New example 1176 (diagnostics-foreign-removed) pins it. Pre-cutover migrations (all green, behavior-preserving): the 7 identity ffi-foreign-* test DECLS (2cce6a3), the two keyword-neutral diagnostic tests 1172 + 1228 with intentional snapshot regens (720556b), and the 4 multi-file example companions Phase 7 missed (0729/a+b, 1617/c, 1623/mod — d132aab). Deleted obsolete tests 1174 (#foreign+postfix conflict, now unreachable) + 1620 (#foreign nosuchunit, superseded by extern twin 1231), the GATE A→B unit test + lowerSrcToIr helper (nothing left to compare), and converted the in-source parse void function with foreign body parser test to postfix extern. specs.md + readme.md document extern/export as the sole C-linkage surface. Suite green (646 corpus / 444 unit, 0 failed).

Prior: Phase 7.4 — migrate straggler examples #foreignextern (refactor commit

1a8991a). PHASE 7 MIGRATABLE WORK COMPLETE (7.17.4 done). Migrated 16 fn/global examples across categories (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/ 1622/1628/1635/1636). Marker'd ones corpus-validated; the 3 unmarked uikit importers (1607/1608/1616) verified byte-identical via sx ir probes. Empty snapshot diff; suite green (647/444).

Phase 7 net result: every example that uses #foreign incidentally (FFI plumbing, output-preserving) is now on extern/export. The 24 files still holding #foreign are exactly the intended keep-list, all deferred to the Phase 8 cutover:

  • foreign-asserting diagnostics (migrating changes a snapshot): 1172, 1174, 1219 (stdout label), 1228 (equivalence test), 1620.
  • Identity ffi-foreign-* feature tests (real decls; rename/dedup at cutover): 1205-global, 1205-global-helper, 1207, 1218, 1219, 1306, 1318.
  • Comment-only / provenance prose (decls=0; #foreign only in comments): 0716, 0729, 1216, 1223, 1224, 1225, 1229, 1230, 1231, 1332, 1348, 1349, 1426, + issues/0030. Lesson (7.3): the robust class-prefix transform is the GENERAL form s/#foreign\s+#(\w+)\((\"[^\"]*\")\)\s*\{/#$1($2) extern {/ — 1417 also used #jni_interface/#objc_protocol/#swift_class/#swift_struct/#swift_protocol, and a #(objc|jni)_class-only regex left extern in prefix position → parse error. All such directives accept the postfix modifier (probed). Bare defined #objc_class/#jni_class examples (no #foreign) were left untouched — not a purge target (define→export is an optional consistency pass, deferrable).

Prior: Phase 7.1 — migrate incidental 12xx ffi examples (refactor commit 731fb8d).

Migrated 12 plain-C examples (1200/1206/1209-1215/1220/1221/1222); established the keep-list policy above. Phase 7.2 (a68f7c2): 18 13xx obj-c examples (prefix→postfix classes). Phase 7.3 (2888f6f): 13 14xx jni examples incl. 1417 multi-runtime.

Prior: Phase 6.5 — migrate gpu/ #foreignextern; library/ now #foreign-free

(refactor commit 32a7628). PHASE 6 COMPLETE. Final batch: gpu/gles3.sx (eglGetProcAddress + 1 comment) + gpu/metal.sx (MTLCreateSystemDefaultDevice), bare fn markers → extern. Verified byte-identical sx ir on importers 1610 (gles3) + 1606 (metal). Zero #foreign remains anywhere under library/ — verified by grep -rln '#foreign' library/ → no matches. Suite green (647 corpus / 444 unit, 0 failed).

Prior: Phase 6.4 — migrate ffi/ #foreignextern (refactor commit 666a2e2).

objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + objc.sx's 2 import runtime classes (prefix→postfix extern). objc + objc_block validated by the 50 marked 13xx corpus examples (incl. import classes 1300/1301 + defined classes 1339/1349); raylib/ffi-sdl3/wasm verified by byte-identical sx ir probes pre/post. Empty snapshot diff; suite green.

Prior: Phase 6.3 — migrate std/ #foreignextern (refactor commit 59f90d2).

Pure source rename across 11 std modules (~60 sites): cli/core/fmt/fs/log/net.kqueue/process/socket/thread/time/trace. All fn-decl markers — bare #foreign;, #foreign libc;/#foreign tlib; (LIB ref), and #foreign libc "csym"; (LIB+rename) → the same extern … tail (extern carries the identical [LIB] ["csym"] axis); plus 2 stale comment mentions (fmt/fs). No class forms in std. These modules ARE host-corpus-exercised → empty snapshot diff is direct validation. Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 batches: 6.4 ffi (~50, has runtime classes), 6.5 remainder.

Prior: Phase 6.2 — migrate platform/ #foreignextern/export (refactor commit

2cd5d7b). Pure source rename across uikit/android/android_jni/sdl3 (~64 sites): 30 fn … #foreign;… extern;; 34 import runtime classes #foreign #objc_class/#jni_class("X") {#…_class("X") extern { (prefix→postfix); 4 defined Sx* obj-c classes #objc_class("X") {… export {. Behavior-preserving; empty snapshot diff. Verification (these modules are largely uncompiled by the host corpus — bundle examples import bundle.sx, not the runtime modules; android.sx only compiles under OS==.android): byte-identical sx ir on uikit importers 1610 + 1606 (which DO compile uikit incl. the 4 defined Sx* classes on host) and an sdl3 direct-import probe; android.sx verified by an identical 4-error dedup set (host pthread clashes — the keyword-neutral "foreign symbol already bound" dedup message is unchanged, and the probe parsed all migrated extern jni classes + EGL fns cleanly before hitting them). Suite green (647 corpus / 444 unit, 0 failed). Remaining Phase 6 batches: 6.3 std (~60), 6.4 ffi (~50), 6.5 remainder.

Prior: Phase 6.1 — migrate vendors/sqlite #foreignextern (refactor commit

410a52e). PART B PHASE 6 STARTED. Pure source rename: all 97 sqlite3_* … #foreign sqlib "csym"; fn decls → extern sqlib "csym"; (+ the one stale header-comment reference, line 9). The extern_lib axis references the sqlib #import c unit identically to #foreign sqlib, so IR/output is byte-identical — empty snapshot diff (only sqlite.sx changed), and example 1624 (vendor-sqlite-module) stdout byte-unchanged. Suite green (647 corpus / 444 unit, 0 failed).

Prior: Phase 5.1 — annotate A→B gate post-flip + add fn-rename case (test commit

93e7b6f). PHASE 5 COMPLETE → PART B Phase 5 done. The A→B gate (lower.test.zig) already asserted #foreignextern/export byte-identical IR for fn / global / Obj-C class; post-Phase-5.0 the fn-decl + data-global paths build the SAME extern-named AST, so cases 1/2 are now STRUCTURALLY identical (guaranteed by construction, not coincidence). Annotated the gate header to record this and keep it as a regression tripwire (catches a future reader re-diverging the spellings, or a revert of the flip); case 3 (runtime class) stays behaviorally — not structurally — equal via the single is_foreign_eff field. Added a fn-rename case (case 2b, extern_name axis: c_abs"abs") to broaden coverage beyond bare import (verified IR-identical via sx ir probe before adding). Test-only, no snapshot churn. Suite green (647 corpus / 444 unit, 0 failed).

Prior: Phase 5.0 — fn-decl #foreign body-marker FLIP (refactor commit 6b94bb6).

PHASE 5.0 PARSER ROUTING COMPLETE. The fn-body #foreign [LIB] ["csym"] marker now builds the SAME extern AST postfix extern produces (extern_export = .extern_

  • extern_lib/extern_name + empty-block body) instead of a foreign_expr body. Behavior-preserving — all four prereqs (visibility, variadic, plain-free, lib-ref) ensure every downstream reader coalesces is_foreign with extern_export, so IR + runtime are byte-identical (full corpus + A→B gate green). Decision 7 churn realised: example 1620's lib-ref error flips "#foreign library" → "extern library" (the only snapshot moved; hand-edited, not regen). Parser unit test updated to assert the extern shape. Spot-checked 1219/1218/0729 (foreign rename / cvariadic / same-name) end-to-end. All four #foreign parser paths now resolved: global (e5ddfbe) + fn-body (6b94bb6) flipped onto extern; const-with-type is dead (deferred); runtime-class is already coalesced (is_foreign_eff). c_import.zig auto-synthesis STILL emits foreign_expr bodies (Phase 6+), so both shapes coexist — every reader stays dual. Suite green (647 corpus / 444 unit, 0 failed).

Prior: Phase 5.0 prereqs 3 & 4 — plain-free classification + extern lib-ref validation

(plain-free: xfail 2706521 → fix 3c94c14; lib-ref: xfail 38c3240 → fix ad6aed3). Two MORE extern/#foreign divergences found while de-risking the fn-path flip, both now closed. FOUR prereqs total done — the fn-decl flip fully de-risked.

  • Prereq 3 (plain-free): isPlainFreeFn/isPlainFreeFnDecl (resolver.zig:178, generic.zig:815) excluded a #foreign body but classified an empty-block extern fn as a plain free fn — so existing extern fns were wrongly counted in the bare-call ambiguity verdict (example: two same-name extern libc "abs" authors errored ambiguous, while the #foreign twin 0729 compiles). Both predicates now also exclude extern_export == .extern_; export (real body) stays plain-free. Example 1230.
  • Prereq 4 (lib-ref validation): checkForeignRefs (c_import.zig) validated only foreign_expr.library_ref, so a bogus extern nosuchunit "abs" compiled silently while #foreign nosuchunit errors (1620). Now reads the lib ref from EITHER spelling and names the surface keyword in the diagnostic (so 1620 stays byte-unchanged). Example 1231.
  • Two OTHER classifying sites probed and found BENIGN for extern (no flip prereq): namespace/qualified dispatch (registerQualifiedFn decl.zig:2208, namespace gate call.zig:729) — a namespaced extern fn resolves identically to its #foreign twin (probe: cm.c_abs(-9) → 9 both ways; the registered qualified alias resolves to the same extern symbol).

Prior: Phase 5.0 prereq — extern C-variadic tail (xfail 9a2c78d → fix 0fdc821) — the SECOND deferred fn-path prerequisite. BOTH original fn-path prereqs done. The C-variadic ... handling was keyed on the #foreign (foreign_expr)

body shape at two sites — the is_variadic drop in declareFunction (decl.zig:2097) and the call-site early-out in packVariadicCallArgs (pack.zig:302). A variadic extern therefore kept its trailing slice param and slice-packed the extras → garbage at the C ABI (probe: sum_ints(3,10,20,30) → 53316585, not 60). Both gates now also fire for extern_export == .extern_, so a variadic extern drops the ..args: []T, sets is_variadic, and passes extras through the C ... slot with default argument promotion — byte-identical to its #foreign twin. New example 1229 (1229-ffi-extern-cvariadic, JIT #source, int-sum + double-avg). Suite green (645 corpus / 444 unit, 0 failed).

Prior: Phase 5.0 prereq — visibility-gate equivalence (xfail 717c35d → fix 7d8ba1a) — the first of the two deferred fn-path prerequisites.

The non-transitive C-import visibility gate (isVisible(.c_import_bare), decl.zig:2249) used to recognise only the legacy #foreign body shape; a bare extern fn (empty-block body + extern_export == .extern_) escaped the gate via the body != foreign_expr → return true arm and was caught only by the general isNameVisible gate — yielding the generic "not visible" wording instead of the C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings route to visibleOverEdges, and a library-bound extern LIB (like #foreign LIB) stays unconditionally visible — so a future fn-decl #foreignextern migration is byte-identical at this gate. New cross-module example 1228 (examples/1228-ffi-extern-c-non-transitive, main → b → c) pins the equivalence: referencing c's lib-less #foreign AND extern twins transitively both produce the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed). Empirical finding (probe, not yet acted on): the bare-extern twin was NEVER a silent visibility hole — the general isNameVisible gate already denied it; only the diagnostic wording diverged. The fix aligns the wording + gate ownership.

Prior: Phase 5.0 (global path) (refactor lock, commit e5ddfbe) — PART B STARTED.

First of the four #foreign parser paths migrated onto the extern AST: the data-global form name : T #foreign [lib] ["csym"]; now builds the same extern-named VarDecl (is_extern/extern_lib/extern_name) that postfix extern already produces, instead of is_foreign/foreign_lib/foreign_name. Behavior-preserving — lowering coalesces both forms identically (decl.zig:1119,1127,1141), so zero snapshot churn. The fn-decl, const-with-type, and runtime-class #foreign paths still build the legacy AST.

Prior: Phase 4 (green) — PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED. Four pieces:

(1) GATE A→B unit test (lower.test.zig, lowerSrcToIr helper + "GATE A→B" test) — asserts #foreign and extern/export lower to byte-identical printed IR for a sample fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start migrating #foreign until it's green. Verified live (negative-probe: mutating one side fails the assertion). (2) Diagnostic — #foreign + postfix conflict (1174): prefix #foreign combined with postfix extern/export on an aggregate is now a clean parse error (was a confusing internal "compiler bug" during class synthesis). (3) Diagnostic — extern+export mutual exclusion (1175): both keywords on one fn decl is a clean error (was bare "expected ';'"). (4) Docs: specs.md + readme.md document the three extern/export axes (fns, globals, aggregates) alongside #foreign (which stays documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail). NOTE: extern+callconv redundancy needs no diagnostic — callconv(.c) extern is a harmless dup (both .c), and any non-.c callconv already errors on its own.

Prior: Phase 3.1 (green) — PHASE 3 COMPLETE. Postfix extern/export on #objc_class/

#jni_class aggregates fully works. parseForeignClassDecl now parses an optional extern/export modifier in the slot between the ("X") directive args and the { body (parser.zig:~1409): externis_foreign_eff = true (reference an existing runtime class, == legacy #foreign); exportis_foreign_eff = false (define + register a new sx class, == bare #objc_class with no #foreign). The modifier maps straight onto the same is_foreign decision the prefix #foreign already fed the node, so no objc_class.zig/ lowering change was needed — the new surface reuses the existing reference-vs-define path. Examples: 1348 (objc extern import, dispatches NSObject.alloc().init() → green via JIT), 1349 (objc export defined class, SxBar.alloc()/bump/getcounter: 2), 1426 (jni extern import, parse-only parse-only ok). Suite green (641 corpus / 443 unit, 0 fail).

Prior: Phase 2.2 (green) — PHASE 2 COMPLETE. export (define + expose) fully works:

external linkage + C ABI + no sx ctx + force-lowered root + optional "csym" rename. All four export-gap conditions filled in decl.zig: (i) .external linkage for extern_export == .export_ on both define paths (lowerFunctionBodyInto, lowerFunction); (ii) C-ABI promotion on the define paths + declareFunction stub cc; (iv) funcWantsImplicitCtx returns false for any non-.none modifier; force-lower: export fns are lowering roots in lowerMainAndComptime (else an uncalled export fn stays a bodiless declare); (iii) export … "csym" declares the stub under the C name

  • lazyLowerFunction promotes the body into it via foreign_name_map. Examples 1226 (bare export, C calls sx_square → 37/82) + 1227 (export "triple_c", C calls triple_c → 22) green via the new AOT corpus mode. Suite green (638 corpus / 443 unit, 0 fail).

AOT corpus mode + run_examples.sh retired. C→sx-by-name can't link under the corpus's sx run JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's flat-namespace lookup), so an expected/<name>.aot marker switches an example to a sx build + execute flow. The standalone tests/run_examples.sh was deleted — zig build test is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).

Current state

Syntax: bare extern/export, postfix after callconv(.c), extern ⇒ callconv(.c). Decision 4 revised (user 2026-06-14): extern carries an optional LIB+"csym" axis (extern_lib/extern_name) like #foreign; the #library decl + build-flag linking stays separate. extern (PHASE 1) + export (PHASE 2) FULLY WORKING. extern: functions — bare (f :: (…) -> R extern;) AND renamed (extern [LIB] "csym"); data globals — bare + renamed. export: functions — bare (f :: (…) -> R export {…}) AND renamed (export "csym"); external linkage, C ABI, no ctx, force-lowered as a root. All behavior-equivalent to the matching #foreign form. extern_lib is parsed + stored but is a reference only — actual linking stays the #library/build-flag axis. Aggregates DONE (Phase 3): postfix extern/export on #objc_class/#jni_class (reference vs define+register). Interplay/diagnostics/docs DONE (Phase 4) + the A→B GATE IS LOCKED (#foreignextern/export IR for fn/global/class). PART A COMPLETE. Part B foreign footprint to purge: 643 lines / ~57 identifiers in src/ + ~28 doc lines. End-state invariant: zero foreign (Phase 9.4 gate). Examples: 1223 (extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn, AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426 (jni extern class), 1174/1175 (interplay diagnostics).

Next step

PART B — Phase 9 (total foreign purge — the zero-foreign invariant). Phases 58 COMPLETE: extern/export is the sole C-linkage SURFACE; #foreign is hard-rejected. Decision 6 RATIFIED (user, 2026-06-15): PURGE EVERYTHING — the Phase 9.4 gate is absolute (grep -rniE 'foreign' src/ library/ examples/ issues/ specs.md readme.md CLAUDE.md → 0), including issues/*.md writeups.

Remaining foreign footprint (run grep -rniIE 'foreign' per area to scope each step):

  • Comment-only #foreign in examples/ — the deferred provenance comments (0716, 0729, 1216, 1223/1224/1225/1229/1230/1231, 1332, 1348, 1349, 1426 + the migrated feature tests' leftover comments + issues/0030.sx). ⚠ Many CONTRAST #foreign vs extern ("no #foreign, no #library") — a blind s/#foreign/extern/ yields nonsense; rewrite each comment to stay coherent. Mechanical-ish but needs reading.
  • issues/*.md prose (~12 files) — bug writeups referencing #foreign. Rewrite to extern/export per Decision 6 (purge everything). Provenance is preserved in the git history + the (Regression issue NNNN) note, not the keyword spelling.
  • Internal src/ identifiers (Phase 9.1/9.2 — the big mechanical rename):
    • 9.1 linkage: foreign_expr(still BUILT by c_import.zig auto-synthesis — migrate that path first, then the node folds away) · is_foreignis_extern · foreign_lib/ foreign_nameextern_* · foreign_name_mapextern_name_map · callForeigncallExtern · marshalForeignArg · is_foreign_c_api · dedupeForeignSymbol · the "foreign symbol already bound" diagnostic text → "extern symbol" (surfaces in 1172).
    • 9.2 runtime-class → Runtime*Class* (Decision 5 ratified): ForeignClassDecl(65) · ForeignMethodDecl · ForeignClassMember · ForeignFieldDecl · foreign_class_map · current_foreign_class/_method · foreign_path · parse/tryParseForeignClass* · lowerForeign{Method,Static}Call · findForeign*InChain · resolveForeign* · register*ForeignClass* · *ForeignRefs · ForeignRuntime.
    • 9.0 surface: the hash_foreign token + lexer entry + the 4 parse-path REJECTION stubs + the lex hash_foreign test. ⚠ DESIGN CHOICE: keep the token so #foreign keeps its nice migration message, or delete it (→ generic "unknown directive")? Recommend KEEP the token + rejection through at least one release for a good deprecation, deferring the token deletion. Decide before 9.0.
    • 9.3 docs: CLAUDE.md still references #foreign in several places (file-roles, bundling, the rejected-patterns examples) — purge those too for the gate.
  • 9.4 acceptance gate: the absolute grep → 0 across all gated areas.

Suggested order: 9.3-examples (comment rewrite) + issues/*.md first (low-risk text), then 9.1/9.2 internal renames (mechanical, snapshot-neutral — but 9.1's diagnostic-text rename intentionally regens 1172), then 9.0 surface decision, then the 9.4 gate.

  • 6.2 verification note (carry forward): the platform/ runtime modules (uikit/android/android_jni) are NOT compiled by any marker'd host corpus test — verify future platform-adjacent migrations via direct sx ir on importers (1610/1606 compile uikit on host) or import probes, not the corpus alone.
  • Phases 67 (refactor batches, empty snapshot diff per batch): migrate the stdlib + examples from #foreign spelling to extern. Because the AST is already unified, this is a pure SOURCE rename (… #foreign LIB "sym";… extern LIB "sym"; for fns; the global/const forms similarly), and IR/output must be byte-identical per batch. NOTE: c_import.zig auto-synthesis (#import c {#include}) still BUILDS foreign_expr bodies internally — that's a compiler-internal path, migrated separately (likely Phase 8/9 area), not a source-spelling change.
  • Then Phase 8 (cutover: hard-reject the #foreign keyword) and Phase 9 (purge all foreign identifiers — needs Decision 5 [done, Runtime*Class*] + Decision 6 [open, historical carve-out]).

Watch items carried forward:

  • c_import.zig:262 auto-synthesis still emits foreign_expr — both shapes coexist until that path is migrated; keep every body.data == .foreign_expr reader dual (checked exhaustively this stream).
  • const-with-type #foreign parser path (parser.zig:316) is still on foreign_expr but DEAD (registers no const); migrate or delete it at the Phase 8 cutover.
  • The decl.zig:2055 "foreign symbol … already bound" dedupe message is keyword-neutral and fires for both forms — no churn, but reword to "extern" at cutover for consistency. Route the fn-decl #foreign path so a #foreign fn builds the SAME extern AST that postfix extern already produces, instead of a foreign_expr body. This is the highest-value path (the bulk of #foreign usage). Key sub-questions to resolve before/while routing:
  • The foreign_expr node carries library_ref + c_name; the extern fn carries extern_export = .extern_ + extern_lib + extern_name on the FnDecl with an empty-block body. Migration = the parser's fn-body #foreign arm (parser.zig:~2062) builds the extern shape (set extern_export, map library_ref→extern_lib, c_name→extern_name) rather than a foreign_expr.
  • Lowering ALREADY coalesces the two at every fn site checked this stream (decl.zig 2088/2124/2132/2156/2324/2531 read is_foreign OR extern_export), and the two prereq gates (visibility decl.zig:2249, variadic decl.zig:2097 + pack.zig:302) now do too — so the migration should be behavior-preserving with ZERO snapshot churn. VERIFY with the A→B gate test (lower.test.zig) + a full zig build test after routing; any churn means a site still reads foreign_expr structurally and must be coalesced first.
  • ⚠ This ALSO migrates the const-with-type path implicitly IF it shares the same foreign_expr→extern reshape (it builds const_decl{value=foreign_expr}). Decide: reshape the const path's value node alongside, or leave the dead const path on foreign_expr until Phase 8 cutover. The const path is dead (see findings below), so leaving it is acceptable; but the parser arm is shared-ish — check whether the fn-body arm change touches it.
  • Cadence: because the migration is behavior-preserving (no churn), it's a single refactor/lock commit (like the 5.0 global-path commit e5ddfbe), NOT an xfail→fix pair.

Investigation findings (this session — reorder the remaining paths):

  • const-with-type (parser.zig:316, name :: type_expr #foreign …) is a DEAD path: it builds const_decl{value = foreign_expr}, but registerTypedModuleConst (decl.zig:848-851) bails on a foreign_expr value (else => return), so it registers no const and emits no symbol — a probe (g_abs :: FP #foreign "abs";) returns unresolved 'g_abs' at the use site, and the form is used NOWHERE in library/examples/issues. Its migration target is ambiguous because the foreign_expr value node is SHARED with the fn-decl path, which isn't migrated yet. Decision (user, 2026-06-14): defer it — migrate it alongside the fn-decl path once foreign_expr's extern shape is decided. The checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the "confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it).
  • runtime-class prefix (parser.zig:~1351, #foreign #objc_class/#jni_class) is ALREADY coalesced: both prefix #foreign and postfix extern feed the single is_foreign_effis_foreign field on foreign_class_decl (parser.zig:1421-1432), so there is NO Phase 5.0 AST change for it — only the Phase 9.2 Runtime*Class* rename remains. Drop it from the Phase 5.0 path list.

So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then the fn-decl #foreign body-marker migration. const-with-type + runtime-class need no standalone Phase 5.0 commit.

Then Phase 5.1 (lock): unit test that #foreign and extern produce identical IR (the A→B gate already covers fn/global/class — extend or reuse lowerSrcToIr). Then Phases 67 migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject #foreign), Phase 9 total foreign purge.

⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6): runtime-class rename target (Runtime*Class* recommended vs Extern*Class*) and the historical carve-out (keep issues/*.md provenance, gate live tree only — recommended). These decide Phase 9 renames; the plan says confirm before Phase 9, but worth raising with the user before sinking Part B effort. Also pick up the two Deferred items below at the start of Part B (the visibility-gate equivalence in particular needs a cross-module example).

FUTURE MILESTONE — C→sx-by-name in JIT (sx run). Investigated this session (user-requested spike, RESOLVED feasible-but-blocked). Adding the C #source objects directly into the ORC JITDylib (LLVMOrcLLJITAddObjectFile) instead of dlopen'ing a dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a ~20-line spike ran 1226 via sx run (37/82) and all 13 existing #source FFI examples still passed. BLOCKER: C objects using _Thread_local (the return-trace runtime sx_trace.c) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC MachOPlatform set up (the bare LLVMOrcCreateLLJIT default doesn't), and C constructors/__mod_init_func won't run without ORC initializer support. 42 errors-* examples crashed in the spike. A real impl needs a C++ shim in llvm_shim.c (LLJITBuilder().setObjectLinkingLayerCreator(...) + MachOPlatform::Create) — its own milestone, NOT Phase 2/3 scope. The AOT .aot-marker corpus mode is the pragmatic test path and works today. Spike fully reverted (target.zig/main.zig at HEAD).

Deferred (carry into Part B): (a) docs — DONE in Phase 4 (specs.md/readme.md document extern/export; #foreign stays until the Part B cutover); (b) visibility-gate equivalenceDONE (717c35d/7d8ba1a): the c_import_bare gate now polices a lib-less extern fn identically to its lib-less #foreign twin (same C-specific diagnostic); a library-bound extern LIB stays unconditionally visible. Locked by the cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the general isNameVisible gate already denied it; only the diagnostic wording diverged.)

Open decisions

Part A ratified (bare / postfix / ⇒ callconv(.c) / lib-separate). Part B:

  • Decision 5 RATIFIED (user, 2026-06-14): runtime-class rename target = Runtime*Class* (object-model axis, not linkage). Drives the Phase 9.2 identifier renames.
  • Decision 6 RATIFIED (user, 2026-06-15): PURGE EVERYTHING — the Phase 9.4 gate is absolute, including issues/*.md writeups (NOT the recommended keep-provenance default). Every #foreign/foreign reference in the gated tree (src/ library/ examples/ issues/ specs.md readme.md CLAUDE.md) is rewritten to extern/export; provenance lives in git history + (Regression issue NNNN) notes, not the keyword spelling.
  • Decision 7 RATIFIED (user, 2026-06-15): accept the churn#foreign-spelled decls produce extern-worded diagnostics; example 1620 regenerated (only snapshot moved). Aligns with Part B's extern-only end state; the interim oddity is cosmetic and removed at the Phase 8 cutover. Landed in the fn-body flip 6b94bb6. (Original framing below.) — interim diagnostic wording for #foreign-spelled decls (gated the fn-body flip). Once the flip lands, a #foreign-spelled fn builds the extern AST, so any diagnostic that reads the unified AST can no longer tell the user wrote #foreign vs extern. Concretely, example 1620's lib-ref error flips "#foreign library…" → "extern library…". Options: (A, recommended) accept the narrow churn — regen 1620 as intentional; it aligns with Part B's extern-only end state and the interim oddity (#foreign source → "extern" message) is cosmetic and short-lived (Phase 8 cutover removes #foreign). (B) retain a one-bit surface marker on FnDecl (wrote_foreign) so interim diagnostics stay keyword-accurate (zero churn, small extra plumbing, marker deleted at cutover). Affects only diagnostic wording — IR/behavior identical either way.

Log

  • (8.1 cutover) PHASE 8 COMPLETE. Parser hard-rejects #foreign at all 4 sites (const/global/fn-body via self.fail; runtime-class via self.failAt at the caller, new helper); greens xfail 1176. Deleted obsolete 1174 + 1620, the GATE A→B test + lowerSrcToIr helper; converted the in-source parser test to postfix extern; extern_exportconst. specs.md + readme.md drop #foreign. Suite green (646/444). feat! 3811311.
  • (8.0 xfail) Added 1176-diagnostics-foreign-removed.sx pinning the desired rejection. RED (still accepted). test/xfail 8180faf.
  • (8 pre-cutover) Migrated the 4 multi-file example companions Phase 7 missed (0729/a+b, 1617/c, 1623/mod). refactor d132aab.
  • (8 pre-cutover) Migrated keyword-neutral diagnostics 1172 (decl→extern, message stays internal "foreign symbol") + 1228 (→ two foreign-free extern symbols c_abs_one/_two), intentional snapshot regens reviewed. refactor 720556b.
  • (8 pre-cutover) Migrated the 7 identity ffi-foreign-* test decls to extern/export (decls only; comments left for Phase 9.3). refactor 2cce6a3.
  • (7.4 stragglers) PHASE 7 MIGRATABLE WORK COMPLETE. Migrated 16 fn/global examples (0415/0602/0603/1024/1025/1605/1607-1609/1611/1616/1619/1622/1628/1635/1636) #foreignextern; 1607/1608/1616 (unmarked) verified by sx ir probes. 24-file keep-list remains by design (deferred to Phase 8). Suite green (647/444). refactor 1a8991a.
  • (7.3 14xx) Migrated 13 jni examples (1410-1419/1423/1424/1425). 1417 (all-runtimes) hit a parse-error trap: a #(objc|jni)_class-only regex left extern in PREFIX position on #jni_interface/#objc_protocol/#swift_* lines → fixed with the GENERAL #foreign #(\w+)("X") {#$1("X") extern { rewrite (all such directives accept the postfix modifier, probed). Kept 1426 (comment-only). Suite green. refactor 2888f6f.
  • (7.2 13xx) Migrated 18 obj-c examples (1308/1311-1321/1341-1347): prefix→postfix import classes + fn markers. Kept identity 1306/1318, comment-only 1332/1348/1349. No 13xx snapshot asserts on foreign. Suite green. refactor a68f7c2.
  • (7.1 12xx) PHASE 7 STARTED. Migrated 12 incidental plain-C examples (1200/1206/1209-1215/1220/1221/1222) #foreignextern; output byte-identical, empty snapshot diff, corpus-validated. Established the keep-list policy (see Last completed step): kept 1172/1174/1620/1228 + ffi-foreign-* (1205/1207/1216/1218/1219)
    • comment-only 1223/1229/1230/1231 for Phase 8. Suite green (647/444). refactor 731fb8d.
  • (6.5 gpu) PHASE 6 COMPLETE. Migrated gpu/gles3.sx + gpu/metal.sx (3 sites); library/ now #foreign-free (grep -rln '#foreign' library/ → 0). Verified byte-identical sx ir on importers 1610/1606. Suite green (647/444). refactor 32a7628.
  • (6.4 ffi) Migrated ffi/ objc/objc_block/raylib/sdl3/wasm (~51 sites): fn markers + objc.sx's 2 import classes (prefix→postfix extern). objc/objc_block validated by 50 marked 13xx examples; raylib/ffi-sdl3/wasm by sx ir probes pre/post. Empty snapshot diff; suite green (647/444). refactor 666a2e2.
  • (6.3 std) Migrated 11 std/ modules (~60 sites): cli/core/fmt/fs/log/net.kqueue/ process/socket/thread/time/trace. All fn-decl markers (bare / libc|tlib LIB ref / libc "csym" rename) → extern … + 2 comment mentions; no class forms. Host-corpus- exercised → empty snapshot diff validates. Suite green (647/444). refactor 59f90d2.
  • (6.2 platform) Migrated platform/ (uikit/android/android_jni/sdl3, ~64 sites): 30 fn #foreign;extern;, 34 import classes prefix #foreign #objc/jni_class→ postfix … extern {, 4 defined Sx* objc classes → … export {. Behavior- preserving, empty snapshot diff. Verified byte-identical sx ir on uikit importers 1610/1606 + sdl3 probe; android via identical 4-error dedup set (host-only module). Suite green (647/444). refactor 2cd5d7b. NOTE: these runtime modules aren't in the marker'd host corpus — verified out-of-band.
  • (6.1 sqlite) PHASE 6 STARTED. Migrated vendors/sqlite/sqlite.sx: 97 … #foreign sqlib "csym"; fn decls → extern sqlib "csym"; (+ line-9 comment). extern_lib references the sqlib #import c unit like #foreign sqlib; IR byte-identical, empty snapshot diff, example 1624 stdout unchanged. Suite green (647/444). refactor 410a52e.
  • (5.1 gate annotate) PHASE 5 COMPLETE. Annotated the A→B gate header (lower.test.zig) to record that post-Phase-5.0 the fn/global #foreign paths build the same extern-named AST → cases 1/2 are structurally (not coincidentally) identical; the gate stays as a regression tripwire. Added fn-rename case 2b (c_abs"abs", extern_name axis), IR-identical per a sx ir probe. Test-only, no snapshot churn. Suite green (647/444). test 93e7b6f.
  • (5.0 fn-body flip) PHASE 5.0 PARSER ROUTING COMPLETE. Flipped the fn-body #foreign parser arm (parser.zig:~2062) onto the extern AST (empty-block body + extern_export = .extern_ + extern_lib/extern_name); extern_export made var so the body arm can route onto it. Updated the parser unit test to assert the extern shape. Behavior-preserving via the four prereqs; only example 1620's lib-ref message churned ("#foreign library"→"extern library", Decision 7, hand-edited). Suite green (647 corpus / 444 unit). refactor 6b94bb6.
  • (5.0 prereq plain-free xfail) Added 1230-ffi-extern-same-name-authors (two flat authors of absval via extern libc "abs"; the extern twin of #foreign 0729). RED — extern authors wrongly counted as ambiguous (646/1 fail). test/xfail 2706521.
  • (5.0 prereq plain-free fix) isPlainFreeFn/isPlainFreeFnDecl now also exclude extern_export == .extern_ (external C symbol, no sx body; name-keyed first-wins like #foreign); export stays plain-free. 1230 green (absval = 7). Suite green (646/444). fix/green 3c94c14.
  • (5.0 prereq lib-ref xfail) Added 1231-ffi-extern-undeclared-lib (extern nosuchunit "abs" — bogus lib ref). RED — compiles silently (extern lib ref unvalidated). test/xfail 38c3240.
  • (5.0 prereq lib-ref fix) checkForeignRefs (c_import.zig) now reads the lib ref from either spelling (foreign_expr.library_ref OR extern_lib) and names the surface keyword, so 1620 (#foreign) is byte-unchanged and 1231 (extern) gets "extern library … not declared". 1231 green. Suite green (647/444). fix/green ad6aed3. ALL FOUR fn-path prereqs DONE → fn-body flip de-risked; awaiting Decision 7 (interim wording).
  • (5.0 prereq variadic xfail) Added 1229-ffi-extern-cvariadic (JIT #source, int-sum + double-avg, extern C-variadic). Expected snapshot pins the DESIRED correct output. RED (variadic extern slice-packs extras → garbage: sum_ints(3,10,20,30) → 53316585; doubles → 0.0). test/xfail 9a2c78d.
  • (5.0 prereq variadic fix) Extended the two C-variadic gates — the is_variadic drop in declareFunction (decl.zig:2097) and the early-out in packVariadicCallArgs (pack.zig:302) — to fire for extern_export == .extern_ as well as a foreign_expr body. 1229 green (60 / 2.000000). Suite green (645 corpus / 444 unit, 0 failed). fix/green 0fdc821. BOTH fn-path prereqs DONE → fn-decl #foreign body-marker migration unblocked.
  • (5.0 prereq vis xfail) Added cross-module example 1228-ffi-extern-c-non-transitive (main → b → c). Main references c's lib-less #foreign + extern twins transitively; expected snapshot pins the DESIRED equivalent C-specific diagnostic for both. RED (extern twin gets the generic "not visible" wording — 443/444). test/xfail commit 717c35d; the fix greens it.
  • (5.0 prereq vis fix) Extended isVisible(.c_import_bare) (decl.zig:2249) to switch on the body: a foreign_expr body OR an extern_export == .extern_ decl with no lib both route to visibleOverEdges; a library-bound decl stays unconditionally visible. 1228 green — both twins emit "C function not visible". Suite green (644 corpus / 444 unit, 0 failed). fix/green commit 7d8ba1a. Deferred prereq (b) CLOSED. Investigation this session also found const-with-type is a DEAD parser path (defer per user) and the runtime-class prefix is already coalesced (no Phase 5.0 change) — see Next step.
  • (5.0 global) PART B STARTED. Routed the #foreign data-global parser path (parser.zig:425) onto the extern-named VarDecl (is_extern/extern_lib/ extern_name) — the same AST postfix extern builds. Behavior-preserving (lowering coalesces both at decl.zig:1119,1127,1141); zero snapshot churn. Suite green (444/444 unit, 643 corpus). refactor lock, commit e5ddfbe. Remaining Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic prereqs), runtime-class prefix (1305).
  • (init) Plan written; FFI-linkage stream opened.
  • (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
  • (0.0) Added kw_extern/kw_export tokens + keyword-map entries + LSP keyword classification + lex linkage keywords test. Suite green; no identifier collisions in the corpus. lock commit.
  • (0.1) Added ast.ExternExportModifier + FnDecl.extern_export + VarDecl.is_extern/extern_name + parseOptionalExternExport() (unconsumed) + 2 parser unit tests. Suite green (443/633). lock commit.
  • (1.0a) Wired fn-path extern parsing (parseFnDecl + both lookahead predicates) + added FnDecl.extern_lib/extern_name + VarDecl.extern_lib per user feedback (decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering. Suite green (443/633). lock commit.
  • (1.0b) Added examples/1223-ffi-extern-fn.sx + hand-authored success snapshots. RED (634 ran, 1 failed — sema body produces no value). xfail commit; 1.1 greens it.
  • (1.1) Wired extern fn lowering (6 edits in decl.zig, all declare-only routing mirroring foreign_expr): funcWantsImplicitCtx + declareFunction cc + lazyLowerFunction/lowerFunction/lowerFunctionBodyInto guards. 1223 green; declare i32 @abs(i32) (C ABI, no ctx). Suite green (634/443). green commit.
  • (1.2a) Added examples/1224-ffi-extern-fn-rename.sx (c_abs :: … extern "abs";) + hand-authored success snapshot (c_abs(-42) = 42). RED (635 ran, 1 failed — parse error: "abs" after extern not yet accepted). xfail; 1.2b greens it. (Also recovered a formatter-clobbered parser.zig — see Known issues.)
  • (1.2b) parseFnDecl parses the optional [LIB] ["csym"] tail into extern_lib/extern_name; declareFunction unifies the rename (foreign c_name OR extern_name → declare under C name, map sx→C) and extends the dedupe guard to extern. 1224 green (c_absabs); 1223 unregressed. Suite green (635/443). green commit. extern_lib parsed+stored (lib linking stays the #library axis).
  • (1.2c) Added examples/1225-ffi-extern-global.sx (__stdinp : *void extern;, mirrors #foreign global 1205) + success snapshot. RED (636 ran, 1 failed — parse error: var-decl extern not accepted). xfail; 1.2d greens it.
  • (1.2d) Parser kw_extern branch in the var-decl path ([LIB] ["csym"]is_extern/extern_lib/extern_name) + registerTopLevelGlobal/globalInitValue consume is_extern. 1225 green (@__stdinp = external global ptr). Suite green (636/443). green commit. PHASE 1 COMPLETEextern fns + globals fully work.
  • (JIT spike) User-requested feasibility investigation of C→sx-by-name in sx run (JIT). Verdict: feasible via LLVMOrcLLJITAddObjectFile (C objects into the ORC JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling (sx_trace.c's _Thread_local SIGABRTs without the ORC MachOPlatform). Own future milestone (see Next step). Spike reverted; no commit.
  • (2.0) Added the AOT corpus mode (expected/<name>.aotsx build + execute) to corpus_run.test.zig + retired tests/run_examples.sh (verify-step.sh/CLAUDE.md updated) + examples/1226-ffi-export-fn.{sx,c,h} (C calls sx_square back). RED (AOT link fails: _sx_square undefined — export not lowered). xfail; 2.1 greens it.
  • (2.1) Filled export gaps i/ii/iv in decl.zig (.external linkage + .c cc on both define paths; funcWantsImplicitCtx false for any non-.none modifier) + force-lower export fns as roots in lowerMainAndComptime. 1226 green via AOT (37/82). Suite green (637/443). green commit.
  • (2.2a) Added examples/1227-ffi-export-fn-rename.sx (export "triple_c", C calls triple_c). RED (define path emits @sx_triple, ignores extern_name → C ref undefined). xfail; 2.2b greens it.
  • (2.2b) declareFunction rename branch fires for export (stub under C name + sx→C in foreign_name_map); lazyLowerFunction resolves the stub by that C name so the body promotes into the C-named function (define @triple_c). sx-side call sites resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite green (638/443). green commit. PHASE 2 COMPLETEexport fully works.
  • (3.0) Added examples/1348-ffi-objc-extern-class.sx (postfix extern on #objc_class, new spelling of #foreign #objc_class). RED (parser: expected '{' after the directive). Hand-authored green snapshots. xfail commit; 3.1 greens it.
  • (3.1a) Wired the postfix extern/export aggregate slot in parseForeignClassDecl (optional modifier between ("X") and {; var is_foreign_eff overrides the passed is_foreign, threaded into the foreign_class_decl node). No lowering change — reuses the existing is_foreign reference-vs-define path. 1348 green. Suite green (639/443). green commit. PHASE 3 COMPLETE.
  • (3.1b) Behavior-lock: added examples/1426-ffi-jni-extern-class.sx (jni extern, parse-only) + examples/1349-ffi-objc-export-class.sx (objc export defined class, counter: 2). Both pass against the 3.1a parser change (locked in their own commit per the cadence rule). Suite green (641/443). lock commit. (Note: -Dupdate-goldens newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr 0-byte per repo convention; runner normalizes both.)
  • (4.gate) GATE A→B — added lowerSrcToIr helper + "GATE A→B" test to lower.test.zig: #foreignextern/export byte-identical printed IR for fn / global / Obj-C class. Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the equivalence was prototyped first with sx ir (LLVM IR byte-identical for all three). Suite green (641/444). test commit.
  • (4.diag1) Added examples/1174-diagnostics-foreign-postfix-conflict.sx — prefix #foreign
    • postfix export on an aggregate previously surfaced a confusing internal "emitObjcDefinedClassAllocImp … compiler bug". xfail (golden = clean message) → green: parseForeignClassDecl rejects the combo at the postfix keyword (failFmt). Suite green.
  • (4.docs) specs.md (new "extern/export linkage keywords" subsection after the #foreign FFI docs) + readme.md (C Interop section) document the three axes. docs commit.
  • (4.diag2) Added examples/1175-diagnostics-extern-export-conflict.sxextern export on one fn decl previously gave bare "expected ';'". xfail (golden = clean message) → green: parseFnDecl rejects a second linkage keyword after parseOptionalExternExport. Suite green (643/444). PHASE 4 COMPLETE → PART A DONE.
  • (golden-fix) -Dupdate-goldens churn RESOLVED. Root cause was NOT a code bug: writeGolden always writes content + "\n" (empty → canonical 1-byte \n, used by 484 of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte outliers (verify trims trailing \n so both forms passed, but regen always rewrote them to 1-byte). Conformed all 5 to the 1-byte form → -Dupdate-goldens is now idempotent, no more churn. (Separately: a flaky 0712-sha256-streaming >10s timeout appears only under concurrent zig build load — not a real failure; re-run serially.)

Known issues

  • Workflow hazard (1.2): an editor format-on-save (or zig fmt) clobbered the working-tree src/parser.zig between commits — it reformatted one-liners AND silently dropped my hasFnBodyAfterArrow extern edit, reverting 1223 to a parse error. Recovered with git checkout src/parser.zig (HEAD had the correct, committed version). After any Edit-tool change to a file the IDE may have open, rebuild + run the affected example before trusting the edit.