fn-decl #foreign body marker now builds the unified extern AST. All four #foreign parser paths resolved (global + fn-body flipped; const-with-type dead; runtime-class already coalesced). Decision 7 ratified (accept churn). Next: Phase 5.1 (confirm A->B gate covers post-flip) then Phases 6-7 (source #foreign->extern rename in stdlib + examples).
432 lines
32 KiB
Markdown
432 lines
32 KiB
Markdown
# sx `extern`/`export` + `#foreign` retirement — Checkpoint (FFI-linkage stream)
|
||
|
||
Companion to `current/PLAN-EXTERN-EXPORT.md` — one merged plan: **Part A** adds
|
||
`extern`/`export`, **Part B** migrates `#foreign` and purges `foreign`. Update after
|
||
every commit, one step at a time per the cadence rule.
|
||
|
||
## Last completed step
|
||
**Phase 5.0 — fn-decl `#foreign` body-marker FLIP** (`refactor` commit `6b94bb6`).
|
||
**PHASE 5.0 PARSER ROUTING COMPLETE.** The fn-body `#foreign [LIB] ["csym"]` marker
|
||
now builds the SAME extern AST postfix `extern` produces (`extern_export = .extern_`
|
||
+ `extern_lib`/`extern_name` + empty-block body) instead of a `foreign_expr` body.
|
||
Behavior-preserving — all four prereqs (visibility, variadic, plain-free, lib-ref)
|
||
ensure every downstream reader coalesces `is_foreign` with `extern_export`, so IR +
|
||
runtime are byte-identical (full corpus + A→B gate green). Decision 7 churn realised:
|
||
example 1620's lib-ref error flips "#foreign library" → "extern library" (the only
|
||
snapshot moved; hand-edited, not regen). Parser unit test updated to assert the extern
|
||
shape. Spot-checked 1219/1218/0729 (foreign rename / cvariadic / same-name) end-to-end.
|
||
**All four `#foreign` parser paths now resolved:** global (`e5ddfbe`) + fn-body
|
||
(`6b94bb6`) flipped onto extern; const-with-type is dead (deferred); runtime-class is
|
||
already coalesced (`is_foreign_eff`). `c_import.zig` auto-synthesis STILL emits
|
||
`foreign_expr` bodies (Phase 6+), so both shapes coexist — every reader stays dual.
|
||
Suite green (647 corpus / 444 unit, 0 failed).
|
||
|
||
### Prior: Phase 5.0 prereqs 3 & 4 — plain-free classification + extern lib-ref validation
|
||
(plain-free: xfail `2706521` → fix `3c94c14`; lib-ref: xfail `38c3240` → fix
|
||
`ad6aed3`). Two MORE extern/#foreign divergences found while de-risking the fn-path
|
||
flip, both now closed. **FOUR prereqs total done — the fn-decl flip fully de-risked.**
|
||
- **Prereq 3 (plain-free):** `isPlainFreeFn`/`isPlainFreeFnDecl` (resolver.zig:178,
|
||
generic.zig:815) excluded a `#foreign` body but classified an empty-block `extern`
|
||
fn as a plain free fn — so existing extern fns were wrongly counted in the bare-call
|
||
ambiguity verdict (example: two same-name `extern libc "abs"` authors errored
|
||
ambiguous, while the `#foreign` twin 0729 compiles). Both predicates now also
|
||
exclude `extern_export == .extern_`; `export` (real body) stays plain-free. Example
|
||
**1230**.
|
||
- **Prereq 4 (lib-ref validation):** `checkForeignRefs` (c_import.zig) validated only
|
||
`foreign_expr.library_ref`, so a bogus `extern nosuchunit "abs"` compiled silently
|
||
while `#foreign nosuchunit` errors (1620). Now reads the lib ref from EITHER spelling
|
||
and names the surface keyword in the diagnostic (so 1620 stays byte-unchanged).
|
||
Example **1231**.
|
||
- Two OTHER classifying sites probed and found BENIGN for extern (no flip prereq):
|
||
namespace/qualified dispatch (`registerQualifiedFn` decl.zig:2208, namespace gate
|
||
call.zig:729) — a namespaced `extern` fn resolves identically to its `#foreign` twin
|
||
(probe: `cm.c_abs(-9)` → 9 both ways; the registered qualified alias resolves to the
|
||
same extern symbol).
|
||
|
||
### Prior: Phase 5.0 prereq — extern C-variadic tail (xfail `9a2c78d` → fix `0fdc821`) — the SECOND deferred fn-path prerequisite. **BOTH original fn-path prereqs done.** The C-variadic `...` handling was keyed on the `#foreign` (`foreign_expr`)
|
||
body shape at two sites — the `is_variadic` drop in `declareFunction`
|
||
(`decl.zig:2097`) and the call-site early-out in `packVariadicCallArgs`
|
||
(`pack.zig:302`). A variadic `extern` therefore kept its trailing slice param and
|
||
slice-packed the extras → garbage at the C ABI (probe: `sum_ints(3,10,20,30)` →
|
||
53316585, not 60). Both gates now also fire for `extern_export == .extern_`, so a
|
||
variadic `extern` drops the `..args: []T`, sets `is_variadic`, and passes extras
|
||
through the C `...` slot with default argument promotion — byte-identical to its
|
||
`#foreign` twin. New example **1229** (`1229-ffi-extern-cvariadic`, JIT `#source`,
|
||
int-sum + double-avg). Suite green (645 corpus / 444 unit, 0 failed).
|
||
|
||
### Prior: Phase 5.0 prereq — visibility-gate equivalence (xfail `717c35d` → fix `7d8ba1a`) — the first of the two deferred fn-path prerequisites.
|
||
The non-transitive C-import visibility gate (`isVisible(.c_import_bare)`,
|
||
`decl.zig:2249`) used to recognise only the legacy `#foreign` body shape; a bare
|
||
`extern` fn (empty-block body + `extern_export == .extern_`) escaped the gate via
|
||
the `body != foreign_expr → return true` arm and was caught only by the general
|
||
`isNameVisible` gate — yielding the generic "not visible" wording instead of the
|
||
C-specific "C function not visible; add #import" one. Now BOTH lib-less spellings
|
||
route to `visibleOverEdges`, and a library-bound `extern LIB` (like `#foreign LIB`)
|
||
stays unconditionally visible — so a future fn-decl `#foreign`→`extern` migration
|
||
is byte-identical at this gate. New cross-module example **1228**
|
||
(`examples/1228-ffi-extern-c-non-transitive`, main → b → c) pins the equivalence:
|
||
referencing c's lib-less `#foreign` AND `extern` twins transitively both produce
|
||
the identical C-specific diagnostic. Suite green (644 corpus / 444 unit, 0 failed).
|
||
**Empirical finding** (probe, not yet acted on): the bare-extern twin was NEVER a
|
||
silent visibility hole — the general `isNameVisible` gate already denied it; only
|
||
the *diagnostic wording* diverged. The fix aligns the wording + gate ownership.
|
||
|
||
### Prior: Phase 5.0 (global path) (`refactor` lock, commit `e5ddfbe`) — **PART B STARTED.**
|
||
First of the four `#foreign` parser paths migrated onto the extern AST: the
|
||
data-global form `name : T #foreign [lib] ["csym"];` now builds the same
|
||
extern-named `VarDecl` (`is_extern`/`extern_lib`/`extern_name`) that postfix
|
||
`extern` already produces, instead of `is_foreign`/`foreign_lib`/`foreign_name`.
|
||
Behavior-preserving — lowering coalesces both forms identically
|
||
(`decl.zig:1119,1127,1141`), so zero snapshot churn. The fn-decl, const-with-type,
|
||
and runtime-class `#foreign` paths still build the legacy AST.
|
||
|
||
### Prior: Phase 4 (green) — **PHASE 4 COMPLETE → PART A DONE; GATE A→B LOCKED.** Four pieces:
|
||
(1) **GATE A→B unit test** (`lower.test.zig`, `lowerSrcToIr` helper + "GATE A→B" test) —
|
||
asserts `#foreign` and `extern`/`export` lower to byte-identical printed IR for a sample
|
||
fn, data global, and Obj-C runtime class. This is the hard gate: Part B may not start
|
||
migrating `#foreign` until it's green. Verified live (negative-probe: mutating one side
|
||
fails the assertion). (2) **Diagnostic — `#foreign` + postfix conflict** (1174): prefix
|
||
`#foreign` combined with postfix `extern`/`export` on an aggregate is now a clean parse
|
||
error (was a confusing internal "compiler bug" during class synthesis). (3) **Diagnostic
|
||
— `extern`+`export` mutual exclusion** (1175): both keywords on one fn decl is a clean
|
||
error (was bare "expected ';'"). (4) **Docs**: `specs.md` + `readme.md` document the three
|
||
`extern`/`export` axes (fns, globals, aggregates) alongside `#foreign` (which stays
|
||
documented until the Part B cutover). Suite green (643 corpus / 444 unit, 0 fail).
|
||
NOTE: `extern`+`callconv` redundancy needs no diagnostic — `callconv(.c) extern` is a
|
||
harmless dup (both `.c`), and any non-`.c` callconv already errors on its own.
|
||
|
||
### Prior: Phase 3.1 (green) — **PHASE 3 COMPLETE.** Postfix `extern`/`export` on `#objc_class`/
|
||
`#jni_class` aggregates fully works. `parseForeignClassDecl` now parses an optional
|
||
`extern`/`export` modifier in the slot **between** the `("X")` directive args and the `{`
|
||
body (`parser.zig:~1409`): `extern`→`is_foreign_eff = true` (reference an existing runtime
|
||
class, == legacy `#foreign`); `export`→`is_foreign_eff = false` (define + register a new sx
|
||
class, == bare `#objc_class` with no `#foreign`). The modifier maps straight onto the same
|
||
`is_foreign` decision the prefix `#foreign` already fed the node, so **no `objc_class.zig`/
|
||
lowering change was needed** — the new surface reuses the existing reference-vs-define path.
|
||
Examples: **1348** (objc `extern` import, dispatches `NSObject.alloc().init()` → green via
|
||
JIT), **1349** (objc `export` defined class, `SxBar.alloc()`/`bump`/`get` → `counter: 2`),
|
||
**1426** (jni `extern` import, parse-only `parse-only ok`). Suite green (641 corpus / 443
|
||
unit, 0 fail).
|
||
|
||
### Prior: Phase 2.2 (green) — **PHASE 2 COMPLETE.** `export` (define + expose) fully works:
|
||
external linkage + C ABI + no sx ctx + force-lowered root + optional `"csym"` rename.
|
||
All four export-gap conditions filled in `decl.zig`: (i) `.external` linkage for
|
||
`extern_export == .export_` on both define paths (`lowerFunctionBodyInto`,
|
||
`lowerFunction`); (ii) C-ABI promotion on the define paths + `declareFunction` stub cc;
|
||
(iv) `funcWantsImplicitCtx` returns false for any non-`.none` modifier; **force-lower**:
|
||
`export` fns are lowering roots in `lowerMainAndComptime` (else an uncalled export fn
|
||
stays a bodiless `declare`); (iii) `export … "csym"` declares the stub under the C name
|
||
+ `lazyLowerFunction` promotes the body into it via `foreign_name_map`. Examples **1226**
|
||
(bare export, C calls `sx_square` → 37/82) + **1227** (`export "triple_c"`, C calls
|
||
`triple_c` → 22) green via the new **AOT corpus mode**. Suite green (638 corpus / 443
|
||
unit, 0 fail).
|
||
|
||
**AOT corpus mode + run_examples.sh retired.** C→sx-by-name can't link under the
|
||
corpus's `sx run` JIT mode (a JIT-resident symbol is invisible to a dlopen'd C dylib's
|
||
flat-namespace lookup), so an `expected/<name>.aot` marker switches an example to a
|
||
`sx build` + execute flow. The standalone `tests/run_examples.sh` was deleted —
|
||
`zig build test` is now the sole corpus runner (verify-step.sh + CLAUDE.md updated).
|
||
|
||
## Current state
|
||
Syntax: bare `extern`/`export`, postfix after `callconv(.c)`, `extern ⇒ callconv(.c)`.
|
||
**Decision 4 revised** (user 2026-06-14): `extern` carries an optional `LIB`+`"csym"`
|
||
axis (`extern_lib`/`extern_name`) like `#foreign`; the `#library` decl + build-flag
|
||
linking stays separate. **`extern` (PHASE 1) + `export` (PHASE 2) FULLY WORKING.**
|
||
extern: functions — bare (`f :: (…) -> R extern;`) AND renamed (`extern [LIB] "csym"`);
|
||
data globals — bare + renamed. export: functions — bare (`f :: (…) -> R export {…}`)
|
||
AND renamed (`export "csym"`); external linkage, C ABI, no ctx, force-lowered as a root.
|
||
All behavior-equivalent to the matching `#foreign` form. `extern_lib` is parsed + stored
|
||
but is a *reference* only — actual linking stays the `#library`/build-flag axis.
|
||
**Aggregates DONE (Phase 3)**: postfix `extern`/`export` on `#objc_class`/`#jni_class`
|
||
(reference vs define+register). **Interplay/diagnostics/docs DONE (Phase 4)** + the
|
||
**A→B GATE IS LOCKED** (`#foreign` ≡ `extern`/`export` IR for fn/global/class). **PART A
|
||
COMPLETE.** Part B `foreign` footprint to purge: 643 lines / ~57 identifiers in `src/` +
|
||
~28 doc lines. End-state invariant: **zero `foreign`** (Phase 9.4 gate). Examples: 1223
|
||
(extern bare fn), 1224 (extern fn rename), 1225 (extern bare global), 1226 (export bare fn,
|
||
AOT), 1227 (export fn rename, AOT), 1348 (objc extern class), 1349 (objc export class), 1426
|
||
(jni extern class), 1174/1175 (interplay diagnostics).
|
||
|
||
## Next step
|
||
**PART B — Phase 5.1: lock `#foreign` ≡ `extern` IR, then Phases 6–7 (migrate
|
||
stdlib + examples).** Phase 5.0 parser routing is COMPLETE.
|
||
|
||
- **Phase 5.1** (`lock`/`test`): the A→B gate (`lower.test.zig`) already asserts
|
||
`#foreign` ≡ `extern`/`export` byte-identical IR for fn / global / Obj-C class, and
|
||
it's now TRIVIALLY true for the fn path (both build the same AST). Decide whether 5.1
|
||
needs anything beyond the existing gate — likely just confirm/annotate it covers the
|
||
post-flip world (the fn path no longer has two distinct shapes to compare). Could fold
|
||
into a doc note rather than a new test.
|
||
- **Phases 6–7** (`refactor` batches, empty snapshot diff per batch): migrate the
|
||
stdlib + examples from `#foreign` spelling to `extern`. Because the AST is already
|
||
unified, this is a pure SOURCE rename (`… #foreign LIB "sym";` → `… extern LIB "sym";`
|
||
for fns; the global/const forms similarly), and IR/output must be byte-identical per
|
||
batch. NOTE: `c_import.zig` auto-synthesis (`#import c {#include}`) still BUILDS
|
||
`foreign_expr` bodies internally — that's a compiler-internal path, migrated separately
|
||
(likely Phase 8/9 area), not a source-spelling change.
|
||
- **Then Phase 8** (cutover: hard-reject the `#foreign` keyword) and **Phase 9** (purge
|
||
all `foreign` identifiers — needs Decision 5 [done, `Runtime*Class*`] + Decision 6
|
||
[open, historical carve-out]).
|
||
|
||
**Watch items carried forward:**
|
||
- `c_import.zig:262` auto-synthesis still emits `foreign_expr` — both shapes coexist
|
||
until that path is migrated; keep every `body.data == .foreign_expr` reader dual
|
||
(checked exhaustively this stream).
|
||
- const-with-type `#foreign` parser path (`parser.zig:316`) is still on `foreign_expr`
|
||
but DEAD (registers no const); migrate or delete it at the Phase 8 cutover.
|
||
- The `decl.zig:2055` "foreign symbol … already bound" dedupe message is keyword-neutral
|
||
and fires for both forms — no churn, but reword to "extern" at cutover for consistency. Route the fn-decl `#foreign` path so a
|
||
`#foreign` fn builds the SAME extern AST that postfix `extern` already produces,
|
||
instead of a `foreign_expr` body. This is the highest-value path (the bulk of
|
||
`#foreign` usage). Key sub-questions to resolve before/while routing:
|
||
- The `foreign_expr` node carries `library_ref` + `c_name`; the `extern` fn carries
|
||
`extern_export = .extern_` + `extern_lib` + `extern_name` on the FnDecl with an
|
||
empty-block body. Migration = the parser's fn-body `#foreign` arm
|
||
(`parser.zig:~2062`) builds the extern shape (set `extern_export`, map
|
||
`library_ref→extern_lib`, `c_name→extern_name`) rather than a `foreign_expr`.
|
||
- Lowering ALREADY coalesces the two at every fn site checked this stream
|
||
(`decl.zig` 2088/2124/2132/2156/2324/2531 read `is_foreign OR extern_export`),
|
||
and the two prereq gates (visibility `decl.zig:2249`, variadic `decl.zig:2097` +
|
||
`pack.zig:302`) now do too — so the migration should be behavior-preserving with
|
||
ZERO snapshot churn. VERIFY with the A→B gate test (`lower.test.zig`) + a full
|
||
`zig build test` after routing; any churn means a site still reads `foreign_expr`
|
||
structurally and must be coalesced first.
|
||
- ⚠ This ALSO migrates the **const-with-type** path implicitly IF it shares the same
|
||
`foreign_expr`→extern reshape (it builds `const_decl{value=foreign_expr}`). Decide:
|
||
reshape the const path's value node alongside, or leave the dead const path on
|
||
`foreign_expr` until Phase 8 cutover. The const path is dead (see findings below),
|
||
so leaving it is acceptable; but the parser arm is shared-ish — check whether the
|
||
fn-body arm change touches it.
|
||
- Cadence: because the migration is behavior-preserving (no churn), it's a single
|
||
`refactor`/lock commit (like the 5.0 global-path commit `e5ddfbe`), NOT an
|
||
xfail→fix pair.
|
||
|
||
**Investigation findings (this session — reorder the remaining paths):**
|
||
- **const-with-type** (`parser.zig:316`, `name :: type_expr #foreign …`) is a
|
||
**DEAD path**: it builds `const_decl{value = foreign_expr}`, but
|
||
`registerTypedModuleConst` (`decl.zig:848-851`) bails on a `foreign_expr` value
|
||
(`else => return`), so it registers no const and emits no symbol — a probe
|
||
(`g_abs :: FP #foreign "abs";`) returns `unresolved 'g_abs'` at the use site, and
|
||
the form is used NOWHERE in `library`/`examples`/`issues`. Its migration target is
|
||
ambiguous because the `foreign_expr` value node is SHARED with the fn-decl path,
|
||
which isn't migrated yet. **Decision (user, 2026-06-14): defer it — migrate it
|
||
alongside the fn-decl path once `foreign_expr`'s extern shape is decided.** The
|
||
checkpoint's old "lowest-risk, route to the extern-named shape" note is wrong: the
|
||
"confirm the value-node lowering path coalesces" gate can't be met (nothing lowers it).
|
||
- **runtime-class prefix** (`parser.zig:~1351`, `#foreign #objc_class/#jni_class`) is
|
||
**ALREADY coalesced**: both prefix `#foreign` and postfix `extern` feed the single
|
||
`is_foreign_eff`→`is_foreign` field on `foreign_class_decl` (`parser.zig:1421-1432`),
|
||
so there is NO Phase 5.0 AST change for it — only the Phase 9.2 `Runtime*Class*`
|
||
rename remains. Drop it from the Phase 5.0 path list.
|
||
|
||
So Phase 5.0's real remaining work collapses to: the fn-path variadic prereq, then
|
||
the fn-decl `#foreign` body-marker migration. const-with-type + runtime-class need
|
||
no standalone Phase 5.0 commit.
|
||
|
||
Then Phase 5.1 (`lock`): unit test that `#foreign` and `extern` produce identical IR (the
|
||
A→B gate already covers fn/global/class — extend or reuse `lowerSrcToIr`). Then Phases 6–7
|
||
migrate stdlib + examples (empty snapshot diff per batch), Phase 8 cutover (hard-reject
|
||
`#foreign`), Phase 9 total `foreign` purge.
|
||
|
||
**⚠ CONFIRM BEFORE PART B (Open decisions 5 & 6):** runtime-class rename target
|
||
(`Runtime*Class*` recommended vs `Extern*Class*`) and the historical carve-out (keep
|
||
`issues/*.md` provenance, gate live tree only — recommended). These decide Phase 9 renames;
|
||
the plan says confirm before Phase 9, but worth raising with the user before sinking Part B
|
||
effort. **Also pick up the two Deferred items below at the start of Part B** (the
|
||
visibility-gate equivalence in particular needs a cross-module example).
|
||
|
||
**FUTURE MILESTONE — C→sx-by-name in JIT (`sx run`).** Investigated this session
|
||
(user-requested spike, RESOLVED feasible-but-blocked). Adding the C `#source` objects
|
||
directly into the ORC JITDylib (`LLVMOrcLLJITAddObjectFile`) instead of dlopen'ing a
|
||
dylib makes C↔sx cross-references resolve both ways in one link domain — proven: a
|
||
~20-line spike ran 1226 via `sx run` (37/82) and all 13 existing `#source` FFI examples
|
||
still passed. BLOCKER: C objects using `_Thread_local` (the return-trace runtime
|
||
`sx_trace.c`) SIGABRT under JITLink — MachO thread-local-variable handling needs the ORC
|
||
`MachOPlatform` set up (the bare `LLVMOrcCreateLLJIT` default doesn't), and C
|
||
constructors/`__mod_init_func` won't run without ORC initializer support. 42 `errors-*`
|
||
examples crashed in the spike. A real impl needs a C++ shim in `llvm_shim.c`
|
||
(`LLJITBuilder().setObjectLinkingLayerCreator(...)` + `MachOPlatform::Create`) — its own
|
||
milestone, NOT Phase 2/3 scope. The AOT `.aot`-marker corpus mode is the pragmatic test
|
||
path and works today. Spike fully reverted (target.zig/main.zig at HEAD).
|
||
|
||
**Deferred (carry into Part B):** (a) ~~docs~~ — DONE in Phase 4 (`specs.md`/`readme.md`
|
||
document `extern`/`export`; `#foreign` stays until the Part B cutover); (b) ~~visibility-gate
|
||
equivalence~~ — **DONE** (`717c35d`/`7d8ba1a`): the `c_import_bare` gate now polices a
|
||
lib-less `extern` fn identically to its lib-less `#foreign` twin (same C-specific
|
||
diagnostic); a library-bound `extern LIB` stays unconditionally visible. Locked by the
|
||
cross-module example 1228. (Empirical: the bare-extern twin was never a silent hole — the
|
||
general `isNameVisible` gate already denied it; only the diagnostic wording diverged.)
|
||
|
||
## Open decisions
|
||
Part A ratified (bare / postfix / `⇒ callconv(.c)` / lib-separate). Part B:
|
||
- **Decision 5 RATIFIED** (user, 2026-06-14): runtime-class rename target = `Runtime*Class*`
|
||
(object-model axis, not linkage). Drives the Phase 9.2 identifier renames.
|
||
- **Decision 6 STILL OPEN**: historical carve-out — keep `issues/*.md` (+ design-doc prose)
|
||
as provenance & gate only the live tree (recommended) vs purge everything. The user did
|
||
NOT confirm this at the Part A milestone; confirm before Phase 9.
|
||
- **Decision 7 RATIFIED** (user, 2026-06-15): **accept the churn** — `#foreign`-spelled
|
||
decls produce `extern`-worded diagnostics; example 1620 regenerated (only snapshot moved).
|
||
Aligns with Part B's extern-only end state; the interim oddity is cosmetic and removed at
|
||
the Phase 8 cutover. Landed in the fn-body flip `6b94bb6`. (Original framing below.)
|
||
— interim diagnostic wording for `#foreign`-spelled decls (gated the fn-body flip). Once the flip lands, a `#foreign`-spelled fn builds the extern AST, so any
|
||
diagnostic that reads the unified AST can no longer tell the user wrote `#foreign` vs
|
||
`extern`. Concretely, example 1620's lib-ref error flips "#foreign library…" →
|
||
"extern library…". Options: **(A, recommended)** accept the narrow churn — regen 1620 as
|
||
intentional; it aligns with Part B's `extern`-only end state and the interim oddity
|
||
(`#foreign` source → "extern" message) is cosmetic and short-lived (Phase 8 cutover
|
||
removes `#foreign`). **(B)** retain a one-bit surface marker on `FnDecl` (`wrote_foreign`)
|
||
so interim diagnostics stay keyword-accurate (zero churn, small extra plumbing, marker
|
||
deleted at cutover). Affects only diagnostic wording — IR/behavior identical either way.
|
||
|
||
## Log
|
||
- (5.0 fn-body flip) **PHASE 5.0 PARSER ROUTING COMPLETE.** Flipped the fn-body
|
||
`#foreign` parser arm (`parser.zig:~2062`) onto the extern AST (empty-block body +
|
||
`extern_export = .extern_` + extern_lib/extern_name); `extern_export` made `var` so
|
||
the body arm can route onto it. Updated the parser unit test to assert the extern
|
||
shape. Behavior-preserving via the four prereqs; only example 1620's lib-ref message
|
||
churned ("#foreign library"→"extern library", Decision 7, hand-edited). Suite green
|
||
(647 corpus / 444 unit). `refactor` `6b94bb6`.
|
||
- (5.0 prereq plain-free xfail) Added `1230-ffi-extern-same-name-authors` (two flat
|
||
authors of `absval` via `extern libc "abs"`; the `extern` twin of `#foreign` 0729).
|
||
RED — extern authors wrongly counted as ambiguous (646/1 fail). `test`/xfail `2706521`.
|
||
- (5.0 prereq plain-free fix) `isPlainFreeFn`/`isPlainFreeFnDecl` now also exclude
|
||
`extern_export == .extern_` (external C symbol, no sx body; name-keyed first-wins like
|
||
`#foreign`); `export` stays plain-free. 1230 green (`absval = 7`). Suite green (646/444).
|
||
`fix`/green `3c94c14`.
|
||
- (5.0 prereq lib-ref xfail) Added `1231-ffi-extern-undeclared-lib` (`extern nosuchunit
|
||
"abs"` — bogus lib ref). RED — compiles silently (extern lib ref unvalidated).
|
||
`test`/xfail `38c3240`.
|
||
- (5.0 prereq lib-ref fix) `checkForeignRefs` (c_import.zig) now reads the lib ref from
|
||
either spelling (foreign_expr.library_ref OR extern_lib) and names the surface keyword,
|
||
so 1620 (#foreign) is byte-unchanged and 1231 (extern) gets "extern library … not
|
||
declared". 1231 green. Suite green (647/444). `fix`/green `ad6aed3`. **ALL FOUR fn-path
|
||
prereqs DONE → fn-body flip de-risked; awaiting Decision 7 (interim wording).**
|
||
- (5.0 prereq variadic xfail) Added `1229-ffi-extern-cvariadic` (JIT `#source`,
|
||
int-sum + double-avg, `extern` C-variadic). Expected snapshot pins the DESIRED
|
||
correct output. RED (variadic `extern` slice-packs extras → garbage:
|
||
`sum_ints(3,10,20,30)` → 53316585; doubles → 0.0). `test`/xfail `9a2c78d`.
|
||
- (5.0 prereq variadic fix) Extended the two C-variadic gates — the `is_variadic`
|
||
drop in `declareFunction` (`decl.zig:2097`) and the early-out in
|
||
`packVariadicCallArgs` (`pack.zig:302`) — to fire for `extern_export == .extern_`
|
||
as well as a `foreign_expr` body. 1229 green (`60` / `2.000000`). Suite green
|
||
(645 corpus / 444 unit, 0 failed). `fix`/green `0fdc821`. **BOTH fn-path prereqs
|
||
DONE → fn-decl `#foreign` body-marker migration unblocked.**
|
||
- (5.0 prereq vis xfail) Added cross-module example `1228-ffi-extern-c-non-transitive`
|
||
(main → b → c). Main references c's lib-less `#foreign` + `extern` twins
|
||
transitively; expected snapshot pins the DESIRED equivalent C-specific
|
||
diagnostic for both. RED (extern twin gets the generic "not visible" wording —
|
||
443/444). `test`/xfail commit `717c35d`; the fix greens it.
|
||
- (5.0 prereq vis fix) Extended `isVisible(.c_import_bare)` (`decl.zig:2249`) to
|
||
switch on the body: a `foreign_expr` body OR an `extern_export == .extern_` decl
|
||
with no lib both route to `visibleOverEdges`; a library-bound decl stays
|
||
unconditionally visible. 1228 green — both twins emit "C function not visible".
|
||
Suite green (644 corpus / 444 unit, 0 failed). `fix`/green commit `7d8ba1a`.
|
||
**Deferred prereq (b) CLOSED.** Investigation this session also found
|
||
const-with-type is a DEAD parser path (defer per user) and the runtime-class
|
||
prefix is already coalesced (no Phase 5.0 change) — see Next step.
|
||
- (5.0 global) **PART B STARTED.** Routed the `#foreign` data-global parser path
|
||
(`parser.zig:425`) onto the extern-named `VarDecl` (`is_extern`/`extern_lib`/
|
||
`extern_name`) — the same AST postfix `extern` builds. Behavior-preserving
|
||
(lowering coalesces both at `decl.zig:1119,1127,1141`); zero snapshot churn. Suite
|
||
green (444/444 unit, 643 corpus). `refactor` lock, commit `e5ddfbe`. Remaining
|
||
Phase 5.0 paths: const-with-type (316), fn-body (2059, needs visibility+variadic
|
||
prereqs), runtime-class prefix (1305).
|
||
- (init) Plan written; FFI-linkage stream opened.
|
||
- (merge) Folded FOREIGN-MIGRATION in as Part B; deleted the split plan + checkpoint.
|
||
- (0.0) Added `kw_extern`/`kw_export` tokens + keyword-map entries + LSP keyword
|
||
classification + `lex linkage keywords` test. Suite green; no identifier collisions
|
||
in the corpus. `lock` commit.
|
||
- (0.1) Added `ast.ExternExportModifier` + `FnDecl.extern_export` +
|
||
`VarDecl.is_extern`/`extern_name` + `parseOptionalExternExport()` (unconsumed) + 2
|
||
parser unit tests. Suite green (443/633). `lock` commit.
|
||
- (1.0a) Wired fn-path extern parsing (`parseFnDecl` + both lookahead predicates) +
|
||
added `FnDecl.extern_lib`/`extern_name` + `VarDecl.extern_lib` per user feedback
|
||
(decision 4 revised: extern carries an optional lib axis). Unconsumed by lowering.
|
||
Suite green (443/633). `lock` commit.
|
||
- (1.0b) Added `examples/1223-ffi-extern-fn.sx` + hand-authored success snapshots.
|
||
RED (634 ran, 1 failed — sema `body produces no value`). `xfail` commit; 1.1 greens it.
|
||
- (1.1) Wired extern fn lowering (6 edits in `decl.zig`, all declare-only routing
|
||
mirroring `foreign_expr`): `funcWantsImplicitCtx` + `declareFunction` cc +
|
||
`lazyLowerFunction`/`lowerFunction`/`lowerFunctionBodyInto` guards. 1223 green;
|
||
`declare i32 @abs(i32)` (C ABI, no ctx). Suite green (634/443). `green` commit.
|
||
- (1.2a) Added `examples/1224-ffi-extern-fn-rename.sx` (`c_abs :: … extern "abs";`) +
|
||
hand-authored success snapshot (`c_abs(-42) = 42`). RED (635 ran, 1 failed — parse
|
||
error: `"abs"` after `extern` not yet accepted). `xfail`; 1.2b greens it. (Also
|
||
recovered a formatter-clobbered `parser.zig` — see Known issues.)
|
||
- (1.2b) `parseFnDecl` parses the optional `[LIB] ["csym"]` tail into
|
||
`extern_lib`/`extern_name`; `declareFunction` unifies the rename (foreign c_name OR
|
||
extern_name → declare under C name, map sx→C) and extends the dedupe guard to
|
||
extern. 1224 green (`c_abs`→`abs`); 1223 unregressed. Suite green (635/443).
|
||
`green` commit. extern_lib parsed+stored (lib linking stays the `#library` axis).
|
||
- (1.2c) Added `examples/1225-ffi-extern-global.sx` (`__stdinp : *void extern;`,
|
||
mirrors `#foreign` global 1205) + success snapshot. RED (636 ran, 1 failed — parse
|
||
error: var-decl `extern` not accepted). `xfail`; 1.2d greens it.
|
||
- (1.2d) Parser `kw_extern` branch in the var-decl path (`[LIB] ["csym"]` →
|
||
`is_extern`/`extern_lib`/`extern_name`) + `registerTopLevelGlobal`/`globalInitValue`
|
||
consume `is_extern`. 1225 green (`@__stdinp = external global ptr`). Suite green
|
||
(636/443). `green` commit. **PHASE 1 COMPLETE** — `extern` fns + globals fully work.
|
||
- (JIT spike) User-requested feasibility investigation of C→sx-by-name in `sx run`
|
||
(JIT). Verdict: feasible via `LLVMOrcLLJITAddObjectFile` (C objects into the ORC
|
||
JITDylib) — proven by a throwaway spike — but blocked by JITLink MachO TLV handling
|
||
(`sx_trace.c`'s `_Thread_local` SIGABRTs without the ORC `MachOPlatform`). Own future
|
||
milestone (see Next step). Spike reverted; no commit.
|
||
- (2.0) Added the **AOT corpus mode** (`expected/<name>.aot` → `sx build` + execute) to
|
||
`corpus_run.test.zig` + retired `tests/run_examples.sh` (verify-step.sh/CLAUDE.md
|
||
updated) + `examples/1226-ffi-export-fn.{sx,c,h}` (C calls `sx_square` back). RED (AOT
|
||
link fails: `_sx_square` undefined — export not lowered). `xfail`; 2.1 greens it.
|
||
- (2.1) Filled export gaps i/ii/iv in `decl.zig` (`.external` linkage + `.c` cc on both
|
||
define paths; `funcWantsImplicitCtx` false for any non-`.none` modifier) + force-lower
|
||
export fns as roots in `lowerMainAndComptime`. 1226 green via AOT (37/82). Suite green
|
||
(637/443). `green` commit.
|
||
- (2.2a) Added `examples/1227-ffi-export-fn-rename.sx` (`export "triple_c"`, C calls
|
||
`triple_c`). RED (define path emits `@sx_triple`, ignores `extern_name` → C ref
|
||
undefined). `xfail`; 2.2b greens it.
|
||
- (2.2b) `declareFunction` rename branch fires for `export` (stub under C name +
|
||
sx→C in `foreign_name_map`); `lazyLowerFunction` resolves the stub by that C name so
|
||
the body promotes into the C-named function (`define @triple_c`). sx-side call sites
|
||
resolve via the same map (probe: 5*5→25). 1227 green (22); 1226 unregressed. Suite
|
||
green (638/443). `green` commit. **PHASE 2 COMPLETE** — `export` fully works.
|
||
- (3.0) Added `examples/1348-ffi-objc-extern-class.sx` (postfix `extern` on `#objc_class`,
|
||
new spelling of `#foreign #objc_class`). RED (parser: `expected '{'` after the
|
||
directive). Hand-authored green snapshots. `xfail` commit; 3.1 greens it.
|
||
- (3.1a) Wired the postfix `extern`/`export` aggregate slot in `parseForeignClassDecl`
|
||
(optional modifier between `("X")` and `{`; `var is_foreign_eff` overrides the passed
|
||
`is_foreign`, threaded into the `foreign_class_decl` node). No lowering change — reuses
|
||
the existing `is_foreign` reference-vs-define path. 1348 green. Suite green (639/443).
|
||
`green` commit. **PHASE 3 COMPLETE.**
|
||
- (3.1b) Behavior-lock: added `examples/1426-ffi-jni-extern-class.sx` (jni `extern`,
|
||
parse-only) + `examples/1349-ffi-objc-export-class.sx` (objc `export` defined class,
|
||
`counter: 2`). Both pass against the 3.1a parser change (locked in their own commit per
|
||
the cadence rule). Suite green (641/443). `lock` commit. (Note: `-Dupdate-goldens`
|
||
newline-normalizes empty stderr → reverted unrelated 1226/1227 churn, kept new stderr
|
||
0-byte per repo convention; runner normalizes both.)
|
||
- (4.gate) **GATE A→B** — added `lowerSrcToIr` helper + "GATE A→B" test to `lower.test.zig`:
|
||
`#foreign` ≡ `extern`/`export` byte-identical printed IR for fn / global / Obj-C class.
|
||
Verified live via negative-probe (mutate one side → assertion fails). Behavior-lock; the
|
||
equivalence was prototyped first with `sx ir` (LLVM IR byte-identical for all three).
|
||
Suite green (641/444). `test` commit.
|
||
- (4.diag1) Added `examples/1174-diagnostics-foreign-postfix-conflict.sx` — prefix `#foreign`
|
||
+ postfix `export` on an aggregate previously surfaced a confusing internal
|
||
"emitObjcDefinedClassAllocImp … compiler bug". `xfail` (golden = clean message) → `green`:
|
||
`parseForeignClassDecl` rejects the combo at the postfix keyword (`failFmt`). Suite green.
|
||
- (4.docs) `specs.md` (new "`extern`/`export` linkage keywords" subsection after the
|
||
`#foreign` FFI docs) + `readme.md` (C Interop section) document the three axes. `docs` commit.
|
||
- (4.diag2) Added `examples/1175-diagnostics-extern-export-conflict.sx` — `extern export` on
|
||
one fn decl previously gave bare "expected ';'". `xfail` (golden = clean message) → `green`:
|
||
`parseFnDecl` rejects a second linkage keyword after `parseOptionalExternExport`. Suite
|
||
green (643/444). **PHASE 4 COMPLETE → PART A DONE.**
|
||
- (golden-fix) **`-Dupdate-goldens` churn RESOLVED.** Root cause was NOT a code bug:
|
||
`writeGolden` always writes `content + "\n"` (empty → canonical 1-byte `\n`, used by 484
|
||
of 489 empty goldens). The 5 churning stderr files [1226/1227/1348/1349/1426] were 0-byte
|
||
*outliers* (verify trims trailing `\n` so both forms passed, but regen always rewrote them
|
||
to 1-byte). Conformed all 5 to the 1-byte form → `-Dupdate-goldens` is now idempotent, no
|
||
more churn. (Separately: a flaky `0712-sha256-streaming` >10s timeout appears only under
|
||
concurrent `zig build` load — not a real failure; re-run serially.)
|
||
|
||
## Known issues
|
||
- **Workflow hazard (1.2):** an editor format-on-save (or `zig fmt`) clobbered the
|
||
working-tree `src/parser.zig` between commits — it reformatted one-liners AND
|
||
silently dropped my `hasFnBodyAfterArrow` extern edit, reverting 1223 to a parse
|
||
error. Recovered with `git checkout src/parser.zig` (HEAD had the correct,
|
||
committed version). **After any Edit-tool change to a file the IDE may have open,
|
||
rebuild + run the affected example before trusting the edit.**
|