Files
sx/current/PLAN-POST-METATYPE.md
agra 12e2ff7ef4 docs+rename: erase the reify name everywhere — stream is METATYPE
The compiler concept is declare/define (comptime type construction); the
old "reify" framing is gone from the entire repo.

- Rename: PLAN-REIFY → PLAN-METATYPE, CHECKPOINT-REIFY → CHECKPOINT-METATYPE,
  PLAN-POST-REIFY → PLAN-POST-METATYPE (both rewritten around declare/define);
  examples 0614/0615/0617 → comptime-metatype-* (+ their expected/ triplets),
  headers rewritten.
- Scrub reify from design/execution-evolution-roadmap.md (§7 step 3 contracts,
  §8.1, §9 decisions, §10 gates) → declare/define / comptime type construction.
- core.sx prelude pointer + parser.test.zig surface lock updated to the
  declare/define builtins (define(handle, info) -> Type; EnumInfo.name).

No behavior change; renamed examples match their renamed snapshots. Full
suite green (673), all unit tests pass. Zero `reify` tokens remain in
src/docs/sx/examples.
2026-06-16 21:23:05 +03:00

8.7 KiB
Raw Blame History

PLAN-POST-METATYPE — program plan for the async-first roadmap (everything after metatype)

Sequences every remaining stream after PLAN-METATYPE.md. This is the program-level plan; each stream below is carved into its own PLAN-<STREAM>.md + CHECKPOINT-<STREAM>.md (full step detail + kickoff prompt) when reached, exactly as metatype was. Rationale, the comptime type-construction design, risk ranking (§8.1), and the testing strategy (§10) all live in the design-of-record: ../design/execution-evolution-roadmap.md.

Cadence (IMPASSIBLE), every stream: no commit both adds a test AND makes it pass (lock, or xfail→green); zig build && zig build test green after every step; never regenerate snapshots while red. On an unrelated compiler bug → file issues/NNNN, mark the stream checkpoint BLOCKED, stop (CLAUDE.md rule).

Ordering = async-first (design §7): the async story needs no JIT spine, so the JIT/FFI cluster comes after. New corpus categories: 17xx atomics, 18xx concurrency.

Stream order (post-metatype)

# Stream Roadmap steps Depends on Notes
A Atomics N1 (1) independent foundation; gates B-parallel + channels
B Async runtime 412 metatype, A (for channels) the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved
C Parallel schedulers 1314 A, B N×(M:1) → M:N
D Comptime JIT/FFI 1518 — (independent of async) S1 → C1 → C2 → C3
E Hot-reload (deferred) 1922 D (S1/S2) S2 → R1 → R2 → R3

A and D are independent of each other and of B's core; B is the spine of the async story. Recommended execution order: A → B → C → D → E (async-first; D can slot earlier if FFI/#compiler-collapse becomes a priority).


Stream A — ATOMICS (N1) · PLAN-ATOMICS.md when carved

Goal: LLVM atomic codegen — the net-new emit primitive. Surface = Atomic($T) wrapper + Ordering enum (locked, design §4.6). Some IR/inference scaffolding exists; lowering is absent.

Phases:

  • A.0 Atomic($T) + Ordering lib types + load/store → LLVM load atomic/store atomic with orderings.
  • A.1 RMW: fetch_add/sub/and/or/xor + fetch_min/maxatomicrmw (no nand).
  • A.2 compare_exchange/_weakcmpxchg (returns ?T, null = success).
  • A.3 swap + fence(.ordering).

Gates: unit emit_llvm.test.zig (correct op + ordering emission); corpus 17xx single-thread (deterministic); arch-gated x86_64 + aarch64 .ir (orderings lower differently — x86 vs LL/SC). Out of snapshot scope, state loudly: ordering semantics under weak memory (.ir proves the keyword emitted, not correctness).


Stream B — ASYNC RUNTIME (steps 412) · splits into PLAN-FIBERS.md + PLAN-CHANNELS.md

The colorblind, stackful, pure-sx async runtime (design §4). Compiler floor is small; the runtime is sx lib. Likely carved as two PLANs:

B1 — Fibers + Io + M:1 (the runtime; PLAN-FIBERS.md)

  • B1.0 callconv(.naked) — extend CallConv {default, c} (types.zig:169) + skip prologue/epilogue lowering. (Net-new; gates the context-switch.)
  • B1.1 Repointable-context codegen — lower context as a swappable indirection (never raw TLS) + per-fiber stack-limit. Prerequisite of B1.3, not a successor.
  • B1.2 A1 — Io interface + context.io + Future + cancel() API (protocol/ vtable threaded like Allocator).
  • B1.3 A2 — fiber runtime: callconv(.naked) context-switch asm (per-arch), bootstrap, mmap stacks. sx lib, not a compiler builtin (design §4 A2).
  • B1.4 A3 — Io impls: blocking → deterministic-sim (KEYSTONE) → event-loop (kqueue/epoll/io_uring). Build the deterministic Io before the event loop — it is the test harness (§10.1).
  • B1.5 A5·M:1 scheduler — validates the whole colorblind stack end-to-end.

Gates: deterministic-Io calibrated against blocking Io (don't trust an uncalibrated oracle — §8.1.3); corpus 18xx under deterministic Io; A2 switch-stress test (scribble every callee-saved reg + canary, deep fiber chains, verify post-resume — §10.7) + arch-gated run tests. A2 is the highest-corruption-risk piece (§8.1.1).

B2 — Channels + cancellation + stdlib (PLAN-CHANNELS.md)

  • B2.0 N3 — channels (Channel($T); recv → RecvResult($T) tagged union built via metatype type-fn) + fiber-aware Mutex/WaitGroup (atomic fast-path from A).
  • B2.1 A6 — cancellation = .canceled in the existing ! channel (model a); per- fiber atomic flag (A); every io.* a cancellation point; structured cancel-and-join; masked during cleanup. Rides ERR (try/onfail/defer).
  • B2.2 A4 — stdlib I/O rework — fs/socket/process onto context.io.

Gates: 18xx under deterministic Io; cancellation cleanup asserted via stdout ordering; RecvResult exercises the metatype primitives.


Stream C — PARALLEL SCHEDULERS (steps 1314) · PLAN-PARALLEL.md

  • C.0 N×(M:1) — per-thread M:1 loops + std/thread.sx spawn; shared state uses A atomics; errno-capture discipline + context-fiber-local become mandatory.
  • C.1 M:N — work-stealing (thread-safe steal queues + migration); pinning API (pin = .main | .any | .on(thread)). M:N is committed, not deferred — just last.

Gates: data races aren't snapshottable → stress harness (run-N / TSan-style), loudly out of corpus scope (§10.2). Named context-fiber-local + errno migration test (M:1 can't exercise migration — §10.7).


Stream D — COMPTIME JIT / FFI (steps 1518) · PLAN-JIT.md

Independent of async; can move earlier if #compilerextern / bundler cleanup is prioritized.

  • D.0 S1 — persistent JIT executor (long-lived ORC LLJIT + host-triple emitter + fragment cache, plumbed into the interp). Foundational for C1/C3.
  • D.1 C1 — real comptime FFI = LLVM single ABI authority (per-signature JIT calling-thunks via S1 + trampoline fast-path). Adversarial layout cases (over- aligned/empty structs, aarch64 small-struct split, bool — §8.1.6).
  • D.2 C2 — #compilerextern collapse (hooks → exported C symbols via C1; delete compiler_call/Registry). Gate: bundler corpus byte-identical pre/post.
  • D.3 C3 — comptime asm via host-JIT (un-bail inline_asm; lift→JIT→cache). 06xx host-arch #run asm + 11xx cross-arch loud-bail diagnostic.
  • (S2 only if a path hits TLS/constructors — see Stream E.)

Gates: S1 lifecycle + cache unit tests; C1 behavior-lock trampoline cases → xfail/green 12xx float/struct/aggregate returns.


Stream E — HOT-RELOAD (deferred) (steps 1922) · PLAN-HOTRELOAD.md

Deferred; R1-vs-R2 chosen at pickup. Design constraint (not optional): runtime + long-lived fibers stay persistent, only leaf logic reloads (can't hot-swap code with live suspended fibers).

  • E.0 S2 — ORC C++ shim (MachOPlatform + redirectable symbols). Highest risk (§8.1.5): only C++ in the tree, prior spike failed on _Thread_local, macOS- specific — Linux/Windows + non-Mac TLS/ctor JIT have no named plan yet.
  • E.1 R1 — dylib hot-reload (only needs shipped export; sidesteps S2).
  • E.2 R2 — JIT-resident hot-reload (S1 + S2; ORC indirection stubs).
  • E.3 R3 — incremental compilation (perf enabler; coarse per-file v1 first).

Gates (when picked up): state-survival test; the live-suspended-fiber-into-stale- module hazard; S2 TLS + C-constructor JIT test per host OS (the exact prior-spike case).


Cross-cutting (applies across streams)

  • Testing keystone: the deterministic-sim Io (B1.4) must exist + be calibrated before any async test is trusted (§10.1).
  • Top risks to watch (§8.1): A2 context-switch correctness (B1.3), minted-enum → match codegen (de-risked, metatype stream), deterministic-Io oracle calibration, context-fiber-local/errno (C), S2 (E), C1 args-buffer layout (D).
  • The compiler floor stays small, but deep: atomics, callconv(.naked), repointable- context codegen, declare/define/type_info (metatype stream), the S1 JIT spine. Everything else — schedulers, fibers, channels, the bundler — is sx lib.

Carving protocol

When a stream is reached: copy this section into current/PLAN-<STREAM>.md, expand the phases to xfail→green steps with file anchors (from the design doc's anchor list), add a CHECKPOINT-<STREAM>.md, and write a Phase-0-scoped kickoff prompt (mirror PLAN-METATYPE's). Update CHECKPOINT-METATYPE.md/this file's status as streams complete.