diff --git a/current/CHECKPOINT-REIFY.md b/current/CHECKPOINT-REIFY.md index 62b360d4..6f5e7bc0 100644 --- a/current/CHECKPOINT-REIFY.md +++ b/current/CHECKPOINT-REIFY.md @@ -4,7 +4,20 @@ Companion to [PLAN-REIFY.md](PLAN-REIFY.md). Update after every step (one step a time, per the cadence rule). ## Last completed step -**Phase 0.2 (green) — Phase 0 COMPLETE.** Implemented `reify(.enum(...))`: +**Phase 1 (type-fn → reify identity) — COMPLETE.** A type-fn body that returns +`reify(...)` now mints the enum under the instantiation's name: +`instantiateTypeFunction` (`lower/generic.zig`) detects a reify-returning body +(`findReturnReifyCall`) and routes it to `reifyType(, +reify_call)` with the type-arg bindings active (so `payload = T` resolves to the +bound arg) — placed BEFORE the general case, which would otherwise route the +reify call to the inline-position loud bail. Registering under the mangled name +lets the fn's top-of-instantiation cache return the SAME `TypeId` on the second +instantiation → `Box(i64)` at two independent sites is ONE type (**Contract 1**). +`examples/0615` green (`build()`→`consume()` cross-site, plus `b : Box(i64) = +.none`); full suite green (671 examples, 447 unit). Cadence: 1.0 xfail (empty +marker, RED) → 1.1 green (this commit). + +### (prior) Phase 0.2 (green) — Phase 0 COMPLETE. Implemented `reify(.enum(...))`: `Lowering.reifyType` (in `lower/nominal.zig`) reads the flat-enum `TypeInfo` literal off the AST, synthesizes an `ast.EnumDecl`, and feeds it through the SAME `type_bridge.buildEnumInfo` path source enums use → the minted type is @@ -52,14 +65,16 @@ on-demand import keeps the prelude clean; reify users `#import "modules/std/meta.sx"`. (User-directed.) ## Next step -**Phase 1 (type-fn → reify identity).** `R :: ($T) -> Type { reify(...) }`; assert -`R(i64)` from two sites is ONE type (assignable / matchable across sites). xfail → -green by registering a reify-returning type-fn's result under the instantiation -mangled name (mirror `generic.zig:1663-1689`). NOTE: Phase 0's `reifyType` is hooked -only at the `E :: reify(...)` const-decl site (`decl.zig`) and reads a LITERAL -`TypeInfo` off the AST; Phase 1 must route a reify call returned from a type-fn body -(and likely generalize the literal-AST reader, or evaluate via the interpreter, for -non-literal `TypeInfo`). +**Phase 2 (`type_info` + `field_type`).** Reflect a struct/tuple → read variant / +field names + **types** (`field_type($T, i) -> Type`, `type_info($T) -> TypeInfo`). +xfail → green by implementing both over the type table (reuse the +`field_count`/`field_name` reflection path; both currently bail loudly — +`type_info` in `call.zig:tryLowerReflectionCall`, `field_type` in +`generic.zig:resolveTypeCallWithBindings`). NOTE: `reifyType` still reads a LITERAL +`TypeInfo` off the AST (works for the inline-literal and type-fn-over-literal cases +Phases 0–1 use); a `type_info`-derived (computed, non-literal) `TypeInfo` fed back +into `reify` would need the reader generalized (or interp evaluation) — call that out +when Phase 2 enables round-tripping. SELF-REFERENCE = Phase 4, API DECIDED (user-directed): explicit **`declare()` → `define(h, info)`** (the declaration-vs-definition split; NOT a `reify_rec((self)=>…)` @@ -77,6 +92,11 @@ in PLAN-REIFY Phase 4. None yet. ## Log +- **1.1 (green) — Phase 1 done.** Type-fn body `return reify(...)` routes through + `reifyType` under the instantiation name (`findReturnReifyCall` + + mangled-name registration); `Box(i64)` at two sites = one type (Contract 1); + `0615` green. +- **1.0 (xfail).** `0615` + empty `.exit` marker → RED (reify bailed in type-fn body). - **0.2 (green) — Phase 0 done.** `reifyType` mints a flat enum from a literal `TypeInfo` via the shared `buildEnumInfo` path; `0614` green; Contract 2 confirmed (reify'd enum == source enum, same construct/match). Payload-less variant idiom in diff --git a/current/PLAN-POST-REIFY.md b/current/PLAN-POST-REIFY.md new file mode 100644 index 00000000..64fbb398 --- /dev/null +++ b/current/PLAN-POST-REIFY.md @@ -0,0 +1,161 @@ +# PLAN-POST-REIFY — program plan for the async-first roadmap (everything after reify) + +Sequences every remaining stream after [PLAN-REIFY.md](PLAN-REIFY.md). This is the +**program-level** plan; each stream below is carved into its own +`PLAN-.md` + `CHECKPOINT-.md` (full step detail + kickoff prompt) +**when reached**, exactly as reify was. Rationale, the five reify contracts, risk +ranking (§8.1), and the testing strategy (§10) all live in the design-of-record: +[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md). + +**Cadence (IMPASSIBLE), every stream:** no commit both adds a test AND makes it pass +(lock, or xfail→green); `zig build && zig build test` green after every step; never +regenerate snapshots while red. On an unrelated compiler bug → file `issues/NNNN`, +mark the stream checkpoint BLOCKED, stop (CLAUDE.md rule). + +**Ordering = async-first** (design §7): the async story needs no JIT spine, so the +JIT/FFI cluster comes after. New corpus categories: `17xx` atomics, `18xx` concurrency. + +## Stream order (post-reify) + +| # | Stream | Roadmap steps | Depends on | Notes | +|---|--------|---------------|-----------|-------| +| **A** | Atomics | N1 (1) | — | independent foundation; gates B-parallel + channels | +| **B** | Async runtime | 4–12 | reify, A (for channels) | the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved | +| **C** | Parallel schedulers | 13–14 | A, B | N×(M:1) → M:N | +| **D** | Comptime JIT/FFI | 15–18 | — (independent of async) | S1 → C1 → C2 → C3 | +| **E** | Hot-reload (deferred) | 19–22 | D (S1/S2) | S2 → R1 → R2 → R3 | + +A and D are independent of each other and of B's core; B is the spine of the async +story. **Recommended execution order: A → B → C → D → E** (async-first; D can slot +earlier if FFI/`#compiler`-collapse becomes a priority). + +--- + +## Stream A — ATOMICS (N1) · `PLAN-ATOMICS.md` when carved + +**Goal:** LLVM atomic codegen — the net-new emit primitive. Surface = `Atomic($T)` +wrapper + `Ordering` enum (locked, design §4.6). Some IR/inference scaffolding exists; +**lowering is absent**. + +**Phases:** +- A.0 `Atomic($T)` + `Ordering` lib types + `load`/`store` → LLVM `load atomic`/`store + atomic` with orderings. +- A.1 RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`). +- A.2 `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**). +- A.3 `swap` + `fence(.ordering)`. + +**Gates:** unit `emit_llvm.test.zig` (correct op + ordering emission); corpus `17xx` +single-thread (deterministic); **arch-gated x86_64 + aarch64 `.ir`** (orderings lower +differently — x86 vs LL/SC). **Out of snapshot scope, state loudly:** ordering +*semantics* under weak memory (`.ir` proves the keyword emitted, not correctness). + +--- + +## Stream B — ASYNC RUNTIME (steps 4–12) · splits into `PLAN-FIBERS.md` + `PLAN-CHANNELS.md` + +The colorblind, stackful, pure-sx async runtime (design §4). Compiler floor is small; +the runtime is sx lib. Likely carved as two PLANs: + +### B1 — Fibers + Io + M:1 (the runtime; `PLAN-FIBERS.md`) +- B1.0 **`callconv(.naked)`** — extend `CallConv {default, c}` (types.zig:169) + skip + prologue/epilogue lowering. (Net-new; gates the context-switch.) +- B1.1 **Repointable-`context` codegen** — lower `context` as a swappable indirection + (never raw TLS) + per-fiber stack-limit. **Prerequisite of B1.3, not a successor.** +- B1.2 **A1 — `Io` interface + `context.io` + `Future` + `cancel()` API** (protocol/ + vtable threaded like `Allocator`). +- B1.3 **A2 — fiber runtime**: `callconv(.naked)` context-switch asm (per-arch), + bootstrap, `mmap` stacks. **sx lib, not a compiler builtin** (design §4 A2). +- B1.4 **A3 — `Io` impls: blocking → deterministic-sim (KEYSTONE) → event-loop** + (kqueue/epoll/io_uring). Build the deterministic `Io` *before* the event loop — it + is the test harness (§10.1). +- B1.5 **A5·M:1 scheduler** — validates the whole colorblind stack end-to-end. + +**Gates:** deterministic-`Io` **calibrated** against blocking `Io` (don't trust an +uncalibrated oracle — §8.1.3); corpus `18xx` under deterministic `Io`; **A2 +switch-stress test** (scribble every callee-saved reg + canary, deep fiber chains, +verify post-resume — §10.7) + arch-gated run tests. A2 is the highest-corruption-risk +piece (§8.1.1). + +### B2 — Channels + cancellation + stdlib (`PLAN-CHANNELS.md`) +- B2.0 **N3 — channels** (`Channel($T)`; `recv → RecvResult($T)` tagged union built via + **reify** type-fn) + fiber-aware `Mutex`/`WaitGroup` (atomic fast-path from A). +- B2.1 **A6 — cancellation** = `.canceled` in the existing `!` channel (model a); per- + fiber atomic flag (A); every `io.*` a cancellation point; structured cancel-and-join; + **masked during cleanup**. Rides ERR (`try`/`onfail`/`defer`). +- B2.2 **A4 — stdlib I/O rework** — fs/socket/process onto `context.io`. + +**Gates:** `18xx` under deterministic `Io`; cancellation cleanup asserted via stdout +ordering; `RecvResult` exercises the reify contracts. + +--- + +## Stream C — PARALLEL SCHEDULERS (steps 13–14) · `PLAN-PARALLEL.md` + +- C.0 **N×(M:1)** — per-thread M:1 loops + `std/thread.sx` spawn; shared state uses A + atomics; **errno-capture discipline + `context`-fiber-local** become mandatory. +- C.1 **M:N** — work-stealing (thread-safe steal queues + migration); **pinning** API + (`pin = .main | .any | .on(thread)`). M:N is **committed, not deferred** — just last. + +**Gates:** data races aren't snapshottable → **stress harness** (run-N / TSan-style), +*loudly* out of corpus scope (§10.2). **Named `context`-fiber-local + errno migration +test** (M:1 can't exercise migration — §10.7). + +--- + +## Stream D — COMPTIME JIT / FFI (steps 15–18) · `PLAN-JIT.md` + +Independent of async; can move earlier if `#compiler`→`extern` / bundler cleanup is +prioritized. + +- D.0 **S1 — persistent JIT executor** (long-lived ORC LLJIT + host-triple emitter + + fragment cache, plumbed into the interp). Foundational for C1/C3. +- D.1 **C1 — real comptime FFI = LLVM single ABI authority** (per-signature JIT + calling-thunks via S1 + trampoline fast-path). Adversarial **layout cases** (over- + aligned/empty structs, aarch64 small-struct split, `bool` — §8.1.6). +- D.2 **C2 — `#compiler`→`extern` collapse** (hooks → exported C symbols via C1; delete + `compiler_call`/Registry). Gate: bundler corpus byte-identical pre/post. +- D.3 **C3 — comptime asm via host-JIT** (un-bail `inline_asm`; lift→JIT→cache). + `06xx` host-arch `#run` asm + `11xx` cross-arch loud-bail diagnostic. +- (S2 only if a path hits TLS/constructors — see Stream E.) + +**Gates:** S1 lifecycle + cache unit tests; C1 behavior-lock trampoline cases → +xfail/green `12xx` float/struct/aggregate returns. + +--- + +## Stream E — HOT-RELOAD (deferred) (steps 19–22) · `PLAN-HOTRELOAD.md` + +Deferred; R1-vs-R2 chosen at pickup. Design constraint (not optional): runtime + +long-lived fibers stay **persistent**, only **leaf logic** reloads (can't hot-swap code +with live suspended fibers). + +- E.0 **S2 — ORC C++ shim** (`MachOPlatform` + redirectable symbols). **Highest risk + (§8.1.5):** only C++ in the tree, prior spike failed on `_Thread_local`, macOS- + specific — **Linux/Windows + non-Mac TLS/ctor JIT have no named plan yet.** +- E.1 **R1 — dylib hot-reload** (only needs shipped `export`; sidesteps S2). +- E.2 **R2 — JIT-resident hot-reload** (S1 + S2; ORC indirection stubs). +- E.3 **R3 — incremental compilation** (perf enabler; coarse per-file v1 first). + +**Gates (when picked up):** state-survival test; the live-suspended-fiber-into-stale- +module hazard; S2 TLS + C-constructor JIT test per host OS (the exact prior-spike case). + +--- + +## Cross-cutting (applies across streams) + +- **Testing keystone:** the deterministic-sim `Io` (B1.4) must exist + be calibrated + before *any* async test is trusted (§10.1). +- **Top risks to watch (§8.1):** A2 context-switch correctness (B1.3), reify→match + (de-risked, reify stream), deterministic-`Io` oracle calibration, `context`-fiber- + local/errno (C), S2 (E), C1 args-buffer layout (D). +- **The compiler floor stays small, but deep:** atomics, `callconv(.naked)`, repointable- + `context` codegen, `type_info`/`reify` (reify stream), the S1 JIT spine. Everything + else — schedulers, fibers, channels, the bundler — is sx lib. + +## Carving protocol + +When a stream is reached: copy this section into `current/PLAN-.md`, expand the +phases to xfail→green steps with file anchors (from the design doc's anchor list), add +a `CHECKPOINT-.md`, and write a Phase-0-scoped kickoff prompt (mirror +PLAN-REIFY's). Update [CHECKPOINT-REIFY.md](CHECKPOINT-REIFY.md)/this file's status as +streams complete. diff --git a/current/PLAN-REIFY.md b/current/PLAN-REIFY.md index 277dc663..0de9468e 100644 --- a/current/PLAN-REIFY.md +++ b/current/PLAN-REIFY.md @@ -128,7 +128,8 @@ Examples: `06xx` (comptime, deterministic), `11xx` (diagnostics for loud failure - [x] Phase 0 — `reify` flat enum (`reify(.enum(...))` mints a flat enum via the shared `buildEnumInfo` path; `examples/0614` green; Contract 2 confirmed) -- [ ] Phase 1 — type-fn identity +- [x] Phase 1 — type-fn identity (`Box :: ($T)->Type { reify(...) }` memoizes by + mangled name; `Box(i64)` at two sites is one type; `examples/0615` green) - [ ] Phase 2 — `type_info` + `field_type` - [ ] Phase 3 — `make_enum` + `RecvResult`/`TryResult` - [ ] Phase 4 — reference self-reference diff --git a/examples/expected/0615-comptime-reify-typefn-identity.exit b/examples/expected/0615-comptime-reify-typefn-identity.exit index e69de29b..573541ac 100644 --- a/examples/expected/0615-comptime-reify-typefn-identity.exit +++ b/examples/expected/0615-comptime-reify-typefn-identity.exit @@ -0,0 +1 @@ +0 diff --git a/examples/expected/0615-comptime-reify-typefn-identity.stderr b/examples/expected/0615-comptime-reify-typefn-identity.stderr new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/examples/expected/0615-comptime-reify-typefn-identity.stderr @@ -0,0 +1 @@ + diff --git a/examples/expected/0615-comptime-reify-typefn-identity.stdout b/examples/expected/0615-comptime-reify-typefn-identity.stdout new file mode 100644 index 00000000..84bc8be2 --- /dev/null +++ b/examples/expected/0615-comptime-reify-typefn-identity.stdout @@ -0,0 +1,2 @@ +some 7 +none diff --git a/src/ir/lower/generic.zig b/src/ir/lower/generic.zig index a1c589e5..94cdadcb 100644 --- a/src/ir/lower/generic.zig +++ b/src/ir/lower/generic.zig @@ -1709,6 +1709,19 @@ pub fn instantiateTypeFunction(self: *Lowering, alias_name: []const u8, template return self.instantiateTypeUnion(if (has_alias) alias_name else mangled_name, mangled_name, &enum_decl); } + // A type-fn body that RETURNS `reify(...)` — mint the enum under THIS + // instantiation's name (mangled for inline use, the alias name for + // `Foo :: Box(i64)`). The type-arg bindings are active here, so the reify + // payloads resolve against the instantiation's args (`payload = T` → the + // bound type). Registering under the mangled name lets the cache check at + // the top of this fn return the SAME TypeId on a second instantiation — + // so `Box(i64)` at two sites is ONE type (Contract 1). Must precede the + // general case below, whose `resolveTypeWithBindings` would route the + // reify call to the inline-position loud bail. + if (findReturnReifyCall(fd.body)) |reify_call| { + return self.reifyType(if (has_alias) alias_name else mangled_name, reify_call); + } + // General case: the body returns a TYPE EXPRESSION that is not an inline // struct/union/enum — `return [K]T`, `Vector(K, T)`, `*T`, an alias, etc. // Resolve it with the value/type bindings active (so `[K]T` folds K to a @@ -1739,6 +1752,19 @@ pub fn findReturnTypeExpr(body: *const Node) ?*const Node { return body; } +/// The `reify(...)` call a type-fn body returns (block `return reify(...)` or +/// arrow `=> reify(...)`), or null if the body's return is not a bare `reify` +/// call. Used to route a reify-returning type-fn through `reifyType` under the +/// instantiation name (Phase 1 nominal identity). +pub fn findReturnReifyCall(body: *const Node) ?*const ast.Call { + const ret = findReturnTypeExpr(body) orelse return null; + if (ret.data != .call) return null; + const callee = ret.data.call.callee; + if (callee.data != .identifier) return null; + if (!std.mem.eql(u8, callee.data.identifier.name, "reify")) return null; + return &ret.data.call; +} + /// Instantiate a tagged enum from a type function body. pub fn instantiateTypeUnion(self: *Lowering, alias_name: []const u8, mangled_name: []const u8, ed: *const ast.EnumDecl) ?TypeId { const table = &self.module.types;