18a4f9dd54
REIFY Phase 1.1 (Phase 1 complete). instantiateTypeFunction detects a type-fn body that returns reify(...) (findReturnReifyCall) and routes it to reifyType under the instantiation's name — mangled for inline use, the alias name for `Foo :: Box(i64)` — with the type-arg bindings active so reify payloads (`payload = T`) resolve against the instantiation args. Placed before the general case, whose resolveTypeWithBindings would route the reify call to the inline-position loud bail. Registering under the mangled name lets the top-of-instantiation cache return the SAME TypeId on a second instantiation, so Box(i64) resolved at two independent sites is ONE type (Contract 1). examples/0615 green (build()->consume() cross-site + `b : Box(i64) = .none`). Suite green (671 examples, 447 unit).
162 lines
8.6 KiB
Markdown
162 lines
8.6 KiB
Markdown
# PLAN-POST-REIFY — program plan for the async-first roadmap (everything after reify)
|
||
|
||
Sequences every remaining stream after [PLAN-REIFY.md](PLAN-REIFY.md). This is the
|
||
**program-level** plan; each stream below is carved into its own
|
||
`PLAN-<STREAM>.md` + `CHECKPOINT-<STREAM>.md` (full step detail + kickoff prompt)
|
||
**when reached**, exactly as reify was. Rationale, the five reify contracts, risk
|
||
ranking (§8.1), and the testing strategy (§10) all live in the design-of-record:
|
||
[../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md).
|
||
|
||
**Cadence (IMPASSIBLE), every stream:** no commit both adds a test AND makes it pass
|
||
(lock, or xfail→green); `zig build && zig build test` green after every step; never
|
||
regenerate snapshots while red. On an unrelated compiler bug → file `issues/NNNN`,
|
||
mark the stream checkpoint BLOCKED, stop (CLAUDE.md rule).
|
||
|
||
**Ordering = async-first** (design §7): the async story needs no JIT spine, so the
|
||
JIT/FFI cluster comes after. New corpus categories: `17xx` atomics, `18xx` concurrency.
|
||
|
||
## Stream order (post-reify)
|
||
|
||
| # | Stream | Roadmap steps | Depends on | Notes |
|
||
|---|--------|---------------|-----------|-------|
|
||
| **A** | Atomics | N1 (1) | — | independent foundation; gates B-parallel + channels |
|
||
| **B** | Async runtime | 4–12 | reify, A (for channels) | the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved |
|
||
| **C** | Parallel schedulers | 13–14 | A, B | N×(M:1) → M:N |
|
||
| **D** | Comptime JIT/FFI | 15–18 | — (independent of async) | S1 → C1 → C2 → C3 |
|
||
| **E** | Hot-reload (deferred) | 19–22 | D (S1/S2) | S2 → R1 → R2 → R3 |
|
||
|
||
A and D are independent of each other and of B's core; B is the spine of the async
|
||
story. **Recommended execution order: A → B → C → D → E** (async-first; D can slot
|
||
earlier if FFI/`#compiler`-collapse becomes a priority).
|
||
|
||
---
|
||
|
||
## Stream A — ATOMICS (N1) · `PLAN-ATOMICS.md` when carved
|
||
|
||
**Goal:** LLVM atomic codegen — the net-new emit primitive. Surface = `Atomic($T)`
|
||
wrapper + `Ordering` enum (locked, design §4.6). Some IR/inference scaffolding exists;
|
||
**lowering is absent**.
|
||
|
||
**Phases:**
|
||
- A.0 `Atomic($T)` + `Ordering` lib types + `load`/`store` → LLVM `load atomic`/`store
|
||
atomic` with orderings.
|
||
- A.1 RMW: `fetch_add/sub/and/or/xor` + `fetch_min/max` → `atomicrmw` (no `nand`).
|
||
- A.2 `compare_exchange`/`_weak` → `cmpxchg` (returns **`?T`, null = success**).
|
||
- A.3 `swap` + `fence(.ordering)`.
|
||
|
||
**Gates:** unit `emit_llvm.test.zig` (correct op + ordering emission); corpus `17xx`
|
||
single-thread (deterministic); **arch-gated x86_64 + aarch64 `.ir`** (orderings lower
|
||
differently — x86 vs LL/SC). **Out of snapshot scope, state loudly:** ordering
|
||
*semantics* under weak memory (`.ir` proves the keyword emitted, not correctness).
|
||
|
||
---
|
||
|
||
## Stream B — ASYNC RUNTIME (steps 4–12) · splits into `PLAN-FIBERS.md` + `PLAN-CHANNELS.md`
|
||
|
||
The colorblind, stackful, pure-sx async runtime (design §4). Compiler floor is small;
|
||
the runtime is sx lib. Likely carved as two PLANs:
|
||
|
||
### B1 — Fibers + Io + M:1 (the runtime; `PLAN-FIBERS.md`)
|
||
- B1.0 **`callconv(.naked)`** — extend `CallConv {default, c}` (types.zig:169) + skip
|
||
prologue/epilogue lowering. (Net-new; gates the context-switch.)
|
||
- B1.1 **Repointable-`context` codegen** — lower `context` as a swappable indirection
|
||
(never raw TLS) + per-fiber stack-limit. **Prerequisite of B1.3, not a successor.**
|
||
- B1.2 **A1 — `Io` interface + `context.io` + `Future` + `cancel()` API** (protocol/
|
||
vtable threaded like `Allocator`).
|
||
- B1.3 **A2 — fiber runtime**: `callconv(.naked)` context-switch asm (per-arch),
|
||
bootstrap, `mmap` stacks. **sx lib, not a compiler builtin** (design §4 A2).
|
||
- B1.4 **A3 — `Io` impls: blocking → deterministic-sim (KEYSTONE) → event-loop**
|
||
(kqueue/epoll/io_uring). Build the deterministic `Io` *before* the event loop — it
|
||
is the test harness (§10.1).
|
||
- B1.5 **A5·M:1 scheduler** — validates the whole colorblind stack end-to-end.
|
||
|
||
**Gates:** deterministic-`Io` **calibrated** against blocking `Io` (don't trust an
|
||
uncalibrated oracle — §8.1.3); corpus `18xx` under deterministic `Io`; **A2
|
||
switch-stress test** (scribble every callee-saved reg + canary, deep fiber chains,
|
||
verify post-resume — §10.7) + arch-gated run tests. A2 is the highest-corruption-risk
|
||
piece (§8.1.1).
|
||
|
||
### B2 — Channels + cancellation + stdlib (`PLAN-CHANNELS.md`)
|
||
- B2.0 **N3 — channels** (`Channel($T)`; `recv → RecvResult($T)` tagged union built via
|
||
**reify** type-fn) + fiber-aware `Mutex`/`WaitGroup` (atomic fast-path from A).
|
||
- B2.1 **A6 — cancellation** = `.canceled` in the existing `!` channel (model a); per-
|
||
fiber atomic flag (A); every `io.*` a cancellation point; structured cancel-and-join;
|
||
**masked during cleanup**. Rides ERR (`try`/`onfail`/`defer`).
|
||
- B2.2 **A4 — stdlib I/O rework** — fs/socket/process onto `context.io`.
|
||
|
||
**Gates:** `18xx` under deterministic `Io`; cancellation cleanup asserted via stdout
|
||
ordering; `RecvResult` exercises the reify contracts.
|
||
|
||
---
|
||
|
||
## Stream C — PARALLEL SCHEDULERS (steps 13–14) · `PLAN-PARALLEL.md`
|
||
|
||
- C.0 **N×(M:1)** — per-thread M:1 loops + `std/thread.sx` spawn; shared state uses A
|
||
atomics; **errno-capture discipline + `context`-fiber-local** become mandatory.
|
||
- C.1 **M:N** — work-stealing (thread-safe steal queues + migration); **pinning** API
|
||
(`pin = .main | .any | .on(thread)`). M:N is **committed, not deferred** — just last.
|
||
|
||
**Gates:** data races aren't snapshottable → **stress harness** (run-N / TSan-style),
|
||
*loudly* out of corpus scope (§10.2). **Named `context`-fiber-local + errno migration
|
||
test** (M:1 can't exercise migration — §10.7).
|
||
|
||
---
|
||
|
||
## Stream D — COMPTIME JIT / FFI (steps 15–18) · `PLAN-JIT.md`
|
||
|
||
Independent of async; can move earlier if `#compiler`→`extern` / bundler cleanup is
|
||
prioritized.
|
||
|
||
- D.0 **S1 — persistent JIT executor** (long-lived ORC LLJIT + host-triple emitter +
|
||
fragment cache, plumbed into the interp). Foundational for C1/C3.
|
||
- D.1 **C1 — real comptime FFI = LLVM single ABI authority** (per-signature JIT
|
||
calling-thunks via S1 + trampoline fast-path). Adversarial **layout cases** (over-
|
||
aligned/empty structs, aarch64 small-struct split, `bool` — §8.1.6).
|
||
- D.2 **C2 — `#compiler`→`extern` collapse** (hooks → exported C symbols via C1; delete
|
||
`compiler_call`/Registry). Gate: bundler corpus byte-identical pre/post.
|
||
- D.3 **C3 — comptime asm via host-JIT** (un-bail `inline_asm`; lift→JIT→cache).
|
||
`06xx` host-arch `#run` asm + `11xx` cross-arch loud-bail diagnostic.
|
||
- (S2 only if a path hits TLS/constructors — see Stream E.)
|
||
|
||
**Gates:** S1 lifecycle + cache unit tests; C1 behavior-lock trampoline cases →
|
||
xfail/green `12xx` float/struct/aggregate returns.
|
||
|
||
---
|
||
|
||
## Stream E — HOT-RELOAD (deferred) (steps 19–22) · `PLAN-HOTRELOAD.md`
|
||
|
||
Deferred; R1-vs-R2 chosen at pickup. Design constraint (not optional): runtime +
|
||
long-lived fibers stay **persistent**, only **leaf logic** reloads (can't hot-swap code
|
||
with live suspended fibers).
|
||
|
||
- E.0 **S2 — ORC C++ shim** (`MachOPlatform` + redirectable symbols). **Highest risk
|
||
(§8.1.5):** only C++ in the tree, prior spike failed on `_Thread_local`, macOS-
|
||
specific — **Linux/Windows + non-Mac TLS/ctor JIT have no named plan yet.**
|
||
- E.1 **R1 — dylib hot-reload** (only needs shipped `export`; sidesteps S2).
|
||
- E.2 **R2 — JIT-resident hot-reload** (S1 + S2; ORC indirection stubs).
|
||
- E.3 **R3 — incremental compilation** (perf enabler; coarse per-file v1 first).
|
||
|
||
**Gates (when picked up):** state-survival test; the live-suspended-fiber-into-stale-
|
||
module hazard; S2 TLS + C-constructor JIT test per host OS (the exact prior-spike case).
|
||
|
||
---
|
||
|
||
## Cross-cutting (applies across streams)
|
||
|
||
- **Testing keystone:** the deterministic-sim `Io` (B1.4) must exist + be calibrated
|
||
before *any* async test is trusted (§10.1).
|
||
- **Top risks to watch (§8.1):** A2 context-switch correctness (B1.3), reify→match
|
||
(de-risked, reify stream), deterministic-`Io` oracle calibration, `context`-fiber-
|
||
local/errno (C), S2 (E), C1 args-buffer layout (D).
|
||
- **The compiler floor stays small, but deep:** atomics, `callconv(.naked)`, repointable-
|
||
`context` codegen, `type_info`/`reify` (reify stream), the S1 JIT spine. Everything
|
||
else — schedulers, fibers, channels, the bundler — is sx lib.
|
||
|
||
## Carving protocol
|
||
|
||
When a stream is reached: copy this section into `current/PLAN-<STREAM>.md`, expand the
|
||
phases to xfail→green steps with file anchors (from the design doc's anchor list), add
|
||
a `CHECKPOINT-<STREAM>.md`, and write a Phase-0-scoped kickoff prompt (mirror
|
||
PLAN-REIFY's). Update [CHECKPOINT-REIFY.md](CHECKPOINT-REIFY.md)/this file's status as
|
||
streams complete.
|