diff --git a/current/CHECKPOINT-METATYPE.md b/current/CHECKPOINT-METATYPE.md new file mode 100644 index 00000000..8f34b5c0 --- /dev/null +++ b/current/CHECKPOINT-METATYPE.md @@ -0,0 +1,75 @@ +# CHECKPOINT-METATYPE — comptime type metaprogramming (`declare` / `define`) + +Companion to [PLAN-METATYPE.md](PLAN-METATYPE.md). Update after every step (one +step at a time, per the cadence rule). + +## Last completed step +**The `declare`/`define` floor — the compiler's only type-construction surface.** +Two comptime `#builtin`s mint types; every constructor (`RecvResult`, `TryResult`, +the one-shot form) is plain sx built over them. The compiler has ZERO knowledge of +any named constructor. + +- `declare() -> Type` mints an empty (undefined) nominal slot; `define(handle, + info) -> Type` decodes the `TypeInfo` value (variant names + payload Type-tags), + names the slot from `EnumInfo.name`, completes it byte-identical to a source + enum, and returns the handle (so the one-shot form chains: + `T :: define(declare(), info)`). Interp executes both against a `mint` TypeTable + handle (`setMintTable`); `defineEnum` + `decodeVariantElements` in `interp.zig`. +- A `::` binding or type-fn body that calls a `Type`-returning fn is + **comptime-evaluated** (`evalComptimeType`), running the `declare`/`define` + builtins to mint the type — no constructor-name pattern-matching. `decl.zig` + trigger = `fnReturnsTypeValue`; type-fn trigger = `returnExprMintsType` + (return is a `define(…)` call or a bodied `-> Type` helper). +- Nominal identity rides the type-fn instantiation cache (`renameNominalType` + re-keys the minted type to the mangled name); `RecvResult(i64)` at two sites is + ONE type. +- The type NAME travels in `EnumInfo.name` — the compiler derives no name from a + binding LHS. + +Examples green on the floor: `0614` (one-shot `define(declare(), …)`), `0615` +(type-fn identity), `0617` (channel result types). `field_type` reflection still +green (`0616`). Full suite green (673 examples). + +## Current state +- `modules/std/meta.sx`: `EnumVariant` / `EnumInfo{ name, variants }` / `TypeInfo` + data types; `declare` / `define` / `type_info` / `field_type` `#builtin`s; + `RecvResult($T)` / `TryResult($T)` sx constructors over `define(declare(), …)`. +- Compiler primitives only: `declare`/`define` (construction), `field_type` + (reflection). No constructor-name knowledge anywhere in the compiler — every + named constructor is sx. +- `type_info` still bails loudly (`call.zig:tryLowerReflectionCall`) — pending. + +## Decision (kept) +**Meta lives in `modules/std/meta.sx`, not the prelude.** Declaring its data types +in the always-loaded prelude interns them into every module's type table and +shifts every `.ir` snapshot. On-demand import keeps the prelude clean. + +## Next step +**Self-reference (recursive / mutually-recursive enums).** The two-statement form +`List :: declare(); define(List, .enum(.{ … payload = *List … }))`: `declare` +binds the handle, `define` completes it, and `*List` resolves because a pointer to +an undefined slot is legal. Needs the top-level `define(…)` statement form to run +at comptime in the window before any use of the type's layout (a bare top-level +call doesn't parse today — decide the surface). Pairs with a `make_enum(variants: +[]EnumVariant)` sx helper (a computed, non-literal variant list — exercises the +interpreter decoding a value-arg slice). + +Then: **`type_info($T) -> TypeInfo`** (reflect a type INTO a value — the inverse of +`define`'s decode; widen `TypeInfo` past `` `enum `` + construct a `[]EnumVariant` +value in the interp; its own focused effort) and **validation + loud diagnostics** +(dup variants, by-value self-ref, never-defined `declare`, use-before-define). + +## Known issues +None. + +## Log +- **declare/define floor established.** The comptime type-construction surface is + two primitives (`declare`/`define`); all named constructors are sx. A `::` binding + or type-fn body that calls a `Type`-returning fn is comptime-evaluated (the + builtins mint the type) — no syntactic constructor recognition in the compiler. + Name travels in `TypeInfo`. Examples 0614 (one-shot) / 0615 (type-fn identity) / + 0617 (channel results) run on the floor; `field_type` reflection (0616) unchanged. + Full suite green (673). +- **Stream carved (earlier).** Selected as the first async-first foundation: gates + channel result types (`RecvResult($T)`) and `race`'s synthesized union, fully + validated, self-contained, testable in isolation (`06xx` comptime). diff --git a/current/CHECKPOINT-REIFY.md b/current/CHECKPOINT-REIFY.md deleted file mode 100644 index b7e7004d..00000000 --- a/current/CHECKPOINT-REIFY.md +++ /dev/null @@ -1,217 +0,0 @@ -# CHECKPOINT-REIFY — comptime `type_info` / `reify` (async-first foundation, step 3) - -Companion to [PLAN-REIFY.md](PLAN-REIFY.md). Update after every step (one step at a -time, per the cadence rule). - -## Last completed step -**Phase 3.1 (green) — `RecvResult`/`TryResult` done.** Added `RecvResult($T)` and -`TryResult($T)` to `meta.sx` as type-fns over `reify` (a value-or-`closed`, and a -value-or-`empty`-or-`closed` enum). They needed NO new compiler machinery — they're -reify-of-a-literal in a type-fn body, i.e. exactly the Phase 1 path — so the channel -result types are pure sx library code. `examples/0617` green (both construct + -match, incl. payload-less `.closed`/`.empty`). Full suite green (673 examples). -Cadence: 3.0 xfail (undefined → RED) → 3.1 green. - -**Phase 3.1 is the last GREEN step.** Phase 3.2 was attempted + reverted; the stream -now pivots to the `declare`/`define` RE-ARCHITECTURE (see the section below + the -F1–F5 table in PLAN-REIFY). **`type_info` NOT done** (old Phase 2.2, orthogonal). - -### (prior) Phase 2.1 (green) — `field_type` done. `field_type($T, i) -> Type` is -implemented over the type table (`fieldTypeOf` in `lower/generic.zig`, re-exported -on `Lowering`): struct field / tagged-union + `union` variant payload (`.void` for -a tagless variant) / tuple element / array + vector element; OOB and memberless -types poison to `.unresolved` with a loud diagnostic (never a silent default). -It folds at lower time, so it composes inside `type_eq` / `type_name` / any type-arg -slot. `examples/0616` green (struct fields name+type, `type_eq` fold, tagged-union -payloads incl. `quit → void`). Full suite green (672 examples, 447 unit). Cadence: -2.0 xfail (empty marker, RED) → 2.1 green (this commit). - -**`type_info` is NOT done** — it still bails loudly in -`call.zig:tryLowerReflectionCall`. It builds a full `TypeInfo` *value* (inverse of -reify) and is the larger Phase 2.2 step (see Next step). - -### (prior) Phase 1 (type-fn → reify identity) — COMPLETE. A type-fn body that returns -`reify(...)` now mints the enum under the instantiation's name: -`instantiateTypeFunction` (`lower/generic.zig`) detects a reify-returning body -(`findReturnReifyCall`) and routes it to `reifyType(, -reify_call)` with the type-arg bindings active (so `payload = T` resolves to the -bound arg) — placed BEFORE the general case, which would otherwise route the -reify call to the inline-position loud bail. Registering under the mangled name -lets the fn's top-of-instantiation cache return the SAME `TypeId` on the second -instantiation → `Box(i64)` at two independent sites is ONE type (**Contract 1**). -`examples/0615` green (`build()`→`consume()` cross-site, plus `b : Box(i64) = -.none`); full suite green (671 examples, 447 unit). Cadence: 1.0 xfail (empty -marker, RED) → 1.1 green (this commit). - -### (prior) Phase 0.2 (green) — Phase 0 COMPLETE. Implemented `reify(.enum(...))`: -`Lowering.reifyType` (in `lower/nominal.zig`) reads the flat-enum `TypeInfo` -literal off the AST, synthesizes an `ast.EnumDecl`, and feeds it through the -SAME `type_bridge.buildEnumInfo` path source enums use → the minted type is -byte-identical to a hand-written `enum { value: i64; closed; }` and flows -through enum codegen unmodified (**Contract 2 confirmed** — a source enum -exhibits the exact same construct/match behavior, verified by probe). Wired at -the `E :: reify(...)` const-decl hook in `lower/decl.zig`. `examples/0614` -green (`value 3` / `closed`, exit 0); snapshots captured; full suite green -(670 examples, 447 unit). Unsupported reify shapes bail loudly via -`reifyBail` (never a silent default). - -Also landed (user-directed, separate commit `feat(parser):`): reserved keyword -as a member name after `.` (`.enum`, `case .enum:`, `x.enum`) — so the reify -example reads `reify(.enum(...))` without a backtick. readme updated. - -### Two earlier Phase-0 commits -- **0.1 (xfail).** Added `examples/0614` + empty `.exit` marker → RED. -- **0.0 (lock).** Added the comptime type-metaprogramming surface as the -on-demand module `library/modules/std/meta.sx` (NOT the prelude — see decision -below): `EnumVariant`/`EnumInfo`/`TypeInfo` data types + bodyless `#builtin` -decls `reify` / `type_info` / `field_type`. Each builtin bails LOUDLY when -reached unimplemented (no silent default). Unit test `src/parser.test.zig` -(registered in `src/root.zig`) locks that the decls parse. `zig build test` -green (447/447 unit, 669/669 examples). - -## Current state -- `modules/std/meta.sx` declares the surface; the variant uses the backtick raw - escape `` `enum `` (reads as the keyword, not a mangled `enum_`). -- **Loud bails wired (unimplemented → diagnostic, never a silent type):** - - `reify(...)` in a `::` type-alias position → `decl.zig` (the `.call` - const-decl branch) emits "reify is not yet implemented (REIFY Phase 0.2)" - and poisons the alias to `.unresolved`. This is also where Phase 0.2 will - hook the real construction. - - `reify` / `field_type` in any other type position → - `generic.zig:resolveTypeCallWithBindings` (defense-in-depth). - - `type_info(...)` in expression position → - `call.zig:tryLowerReflectionCall`. -- No interpreter-side construction yet — `reify` mints nothing. - -## Decision (0.0) -**Meta lives in `modules/std/meta.sx`, not the prelude (`core.sx`).** Declaring -the data types in the always-loaded prelude interns them into every module's -type table and shifts every `.ir` snapshot (broke 37 examples in a trial). An -on-demand import keeps the prelude clean; reify users `#import -"modules/std/meta.sx"`. (User-directed.) - -## RE-ARCHITECTURE in progress (user-directed, 2026-06-16) -**`declare`/`define` are to be the ONLY compiler primitives; `reify` / `make_enum` / -`RecvResult` / `TryResult` move to sx in `meta.sx`.** `reify` becomes literally -`{ h := declare(); define(h, info); return h; }`. The AST-walking `reifyType` and all -syntactic `reify` recognition (`decl.zig` `E :: reify` hook, `generic.zig` -`findReturnReifyCall`) get DELETED, replaced by generic comptime evaluation of a -Type-returning expression. Full F1–F5 phase table + the resolved naming/identity -design is in **PLAN-REIFY.md → "RE-ARCHITECTURE"**. - -**A Phase 3.2 attempt (make_enum + an eval-decode reader, `reifyEnumFromVariantsValue` -+ `reifyValueFn`) was implemented this session, then REVERTED** (`git reset --hard -9306ad5`) per this decision — the reader grew the compiler instead of shrinking it. -The eval-decode *decode logic* is salvage for `define` (see commit 4d6c463 in reflog -if needed); the *syntactic routing* is discarded. - -## Next step -**F1 — `declare()`/`define()` primitives + a directly-minted RECURSIVE enum.** -- `declare() -> Type`: intern an empty forward nominal slot (`reserveShadowEnumSlot` - shape, `nominal.zig:98`), anonymous, return a first-class `Type`. -- `define(handle, info)`: decode the `TypeInfo` *value* (reuse the reverted decode: - interp `Value` → names + payload `Type`-tags → `tagged_union` matching - `buildEnumInfo`), fill via `updatePreservingKey`. -- `X :: declare()` decl hook + scan-time `define(...)` side-effect (must complete - before codegen needs the layout — same scan-time window the old `reifyType` used). -- Example: `List :: declare(); define(List, .enum(.{ … payload = *List … }))` — - cons/nil list, construct + match. Self-ref works because the handle predates the body. -Then F2 (reify→sx + general comptime-Type-decl eval, delete `reifyType`), F3 (type-fn -body eval, delete `findReturnReifyCall`, re-green 0615/0617), F4 (make_enum in sx), -F5 (validation). `type_info` (old 2.2) stays orthogonal/pending. - -**END STATE IS NON-NEGOTIABLE: ZERO `reify` in the compiler.** `reifyType`, the -`E :: reify(...)` decl hook (`decl.zig` ~660), and `findReturnReifyCall` routing -(`generic.zig`) must ALL be deleted by the end of the floor work — `reify` lives only -in `meta.sx`. The compiler keeps only `declare`/`define` (+ the generic comptime -Type-expr evaluation that runs them). Removal can't precede the floor (the reify -examples would have no path), so the floor + removal land together; the suite must -never be left red across a session boundary. - -### F1 investigation findings (verified 2026-06-16 — don't re-derive) -- **Pass order** (`decl.zig:lowerRoot`): Pass 1 `scanDecls` (line 441 — registers all - types; the old reify hook mints here) → Pass 1f `UnknownTypeChecker` → Pass 2 - `lowerMainAndComptime` (1320 — force-lowers `main` via `lazyLowerFunction`, which - transitively lowers bodies; also *creates* `#run` comptime fns). `#run` side-effects - EXECUTE only at emit time (`emit_llvm.zig:runComptimeSideEffects`). -- **`define` timing:** must run AFTER scanDecls (needs all types) but BEFORE body - lowering (match/construct resolve variant indices at lower-time, Pass 2). Neither - existing hook fits → add a **new Pass 1g** (after `scanDecls`, before - `lowerMainAndComptime`) that executes recorded `declare`/`define` pairs via the - interp. Self-ref works because `declare` binds the name in Pass 1, so `*List` in the - `define` body resolves. -- **Parse constraint:** a bare top-level `define(List, ...)` does NOT parse - (`expected '::' / ':=' / ':' after identifier`). So `define` is invoked either via a - recognized top-level form or `#run define(...)` whose comptime fn Pass 1g executes - early (instead of waiting for emit). Decide the surface in F1. -- **Builtin dispatch:** `resolveBuiltin` (`call.zig:1297`) is a tiny name→`BuiltinId` - table; `reify`/`type_info`/`field_type` are NOT there — they're recognized - syntactically (decl hook / `tryLowerReflectionCall` at `call.zig:1679`). `declare`/ - `define` need their own dispatch (likely comptime interp builtins, since they mutate - the type table at comptime). -- **Salvage:** the reverted eval-decode (`reifyEnumFromVariantsValue` + - `decodeVariantElements`, commit `4d6c463` in reflog) IS `define`'s decode core. - -### (superseded) earlier Next step — `type_info` value construction Reflect a type into a `TypeInfo` -*value* — the inverse of reify. Two sub-pieces, both non-trivial: (a) widen the -`meta.sx` `TypeInfo` data model beyond `` `enum `` (struct / tuple variants); (b) -build the value at comptime — a `[]EnumVariant`-style slice of structs holding -strings (`name`) + `Type` tags (`payload`), populated from the type table. This is -comptime value-CONSTRUCTION (allocating slice/struct/string/type-tag values in the -interpreter), materially larger than 2.1's fold-to-a-TypeId. Currently bails loudly -in `call.zig:tryLowerReflectionCall`. Best done as its own session for context room. - -NOTE (round-tripping): `reifyType` still reads a LITERAL `TypeInfo` off the AST -(fine for the inline-literal + type-fn-over-literal cases of Phases 0–1). Once 2.2 -produces a COMPUTED `TypeInfo`, feeding it back into `reify` needs the reader -generalized (or interp evaluation of the `TypeInfo` value) — handle that when 2.2 -enables round-tripping. - -Alternatively jump to **Phase 3** (`make_enum` + `RecvResult`/`TryResult` sx lib over -`reify`) — that only needs reify (have it) + type-fns (have them), not `type_info`. - -SELF-REFERENCE = Phase 4, API DECIDED (user-directed): explicit **`declare()` → -`define(h, info)`** (the declaration-vs-definition split; NOT a `reify_rec((self)=>…)` -closure). `declare()` returns a forward nominal `Type` handle (named from the `::` -LHS) usable freely in any later `TypeInfo` — `*List`, `[]List`, and across types for -MUTUAL recursion the one-`self` closure couldn't express; `define(handle, info)` fills -the body. `reify(info)` stays as the one-shot sugar. Maps onto existing machinery: -declare = empty nominal slot (`reserveShadowEnumSlot`-style), define = `buildEnumInfo` -+ `updatePreservingKey`, struct-stub→tagged_union re-key already handled -(`internNamedTypeDecl`'s `adoptsForwardStructStub`). Loud invariants: never-defined -declare = hard error; by-value self-inclusion rejected (infinite size). Full write-up -in PLAN-REIFY Phase 4. - -## Known issues -None yet. - -## Log -- **3.1 (green) — `RecvResult`/`TryResult` done.** Added as type-fns over `reify` - in `meta.sx` (no new machinery — Phase 1 reify-of-literal-in-type-fn); `0617` - green. `make_enum` (3.2) pending on the generalized reify reader. -- **3.0 (xfail).** `0617` + empty marker → RED (RecvResult undefined). -- **2.1 (green) — `field_type` done.** `fieldTypeOf` over the type table - (struct/tagged-union/union/tuple/array/vector; OOB+memberless = loud poison); - folds at lower time, composes in `type_eq`/`type_name`; `0616` green. `type_info` - still pending (Phase 2.2 — builds a TypeInfo value). -- **2.0 (xfail).** `0616` + empty `.exit` marker → RED (field_type bailed). -- **1.1 (green) — Phase 1 done.** Type-fn body `return reify(...)` routes through - `reifyType` under the instantiation name (`findReturnReifyCall` + - mangled-name registration); `Box(i64)` at two sites = one type (Contract 1); - `0615` green. -- **1.0 (xfail).** `0615` + empty `.exit` marker → RED (reify bailed in type-fn body). -- **0.2 (green) — Phase 0 done.** `reifyType` mints a flat enum from a literal - `TypeInfo` via the shared `buildEnumInfo` path; `0614` green; Contract 2 confirmed - (reify'd enum == source enum, same construct/match). Payload-less variant idiom in - the example is `c : E = .closed;` (enum-literal target), same as a source enum. -- **parser (user-directed).** Keyword as member name after `.` — see `feat(parser)`. -- **0.1 (xfail).** `0614` + empty `.exit` marker → RED. -- **0.0 (lock).** Meta surface in `modules/std/meta.sx` (data types + 3 bodyless - `#builtin` decls), loud bails at all three reach points, `src/parser.test.zig` - parse-lock. Two user-directed refinements folded in: variant uses `` `enum `` - raw escape; surface moved out of the prelude into its own module to avoid - type-table / `.ir`-snapshot churn. `zig build test` green. -- **Stream carved.** Selected as the first async-first foundation: `reify` gates both - channel result types (`RecvResult($T)`) and `race`'s synthesized union, is fully - validated (3 reviewers), and is a self-contained compiler/type-system feature - testable in isolation (`06xx` comptime). Generic-enum syntax dropped in its favor. diff --git a/current/PLAN-METATYPE.md b/current/PLAN-METATYPE.md new file mode 100644 index 00000000..368287cb --- /dev/null +++ b/current/PLAN-METATYPE.md @@ -0,0 +1,115 @@ +# PLAN-METATYPE — comptime type metaprogramming (`declare` / `define` + reflection) + +## Goal + +Comptime type metaprogramming with the smallest possible compiler surface: + +- **`declare() -> Type`** — mint a NEW empty (undefined) nominal type, returned as + a first-class `Type` handle. +- **`define(handle, info) -> Type`** — fill a declared handle's body from a + `TypeInfo` *value* (which carries the type's name), and return the handle. +- **`type_info($T) -> TypeInfo`** — reflect a type INTO data (the inverse of + `define`'s decode). *Pending.* +- **`field_type($T, i) -> Type`** — the i-th field / variant-payload / element + type of `$T`. *Done.* + +These four `#builtin`s in `library/modules/std/meta.sx` are the **entire** +compiler surface. Every higher-level constructor is **plain sx built over +`declare`/`define`** — the compiler knows none of them by name: + +```sx +// one-shot (non-recursive): declare + define chained, define returns the handle +T :: define(declare(), .enum(.{ name = "T", variants = .[ … ] })); + +// recursive / mutually-recursive: keep the handle, reference it in its own body +List :: declare(); +define(List, .enum(.{ name = "List", variants = .[ + EnumVariant.{ name = "cons", payload = *List }, + EnumVariant.{ name = "nil", payload = void } ] })); + +// type-fns are ordinary sx (channel result types, etc.) +RecvResult :: ($T: Type) -> Type { + return define(declare(), .enum(.{ name = "RecvResult", variants = .[ + EnumVariant.{ name = "value", payload = T }, + EnumVariant.{ name = "closed", payload = void } ] })); +} +``` + +This gates channel result types (`RecvResult($T)`) and `race`'s synthesized +tagged-union (design [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) §7 step 3), and replaces a would-be `enum($T)` language feature. + +## How it works (the locked design) + +1. **Two comptime interp builtins.** `declare` mints an empty `tagged_union` slot + in the type table; `define` decodes the `TypeInfo` value (variant-name strings + + payload `Type`-tags) and completes the slot byte-identical to a source enum's + `buildEnumInfo` output, so it flows through enum codegen unmodified. The interp + mutates the type table via a `mint` handle the host sets (`setMintTable`). +2. **No syntactic constructor recognition.** A `::` binding or type-fn body that + calls a `Type`-returning fn is **comptime-evaluated** (`evalComptimeType`): the + expression runs through the interpreter, the `declare`/`define` builtins mint the + type, and the result `type_tag` is bound. `decl.zig` triggers on a non-generic + `-> Type` fn call; `instantiateTypeFunction` triggers on a type-fn body that + returns a `define(…)` call (or a bodied `-> Type` helper) — see + `generic.zig:returnExprMintsType`. +3. **Name travels in the data.** `define` names the slot from `EnumInfo.name`; the + compiler derives no name from a binding LHS. `type_info` round-trips it. +4. **Nominal identity** rides the existing type-fn mangled-name instantiation cache: + `RecvResult(i64)` at two sites memoizes to ONE `TypeId` (the body runs once; + `renameNominalType` re-keys the minted type to the mangled name). +5. **Comptime-only, JIT-free.** `declare`/`define` are interp ops; reaching them at + runtime / emit is a hard error. +6. **Undefined-until-defined.** `declare()` mints an undefined slot; *using* it + (construct / match / size) before its `define` is a loud diagnostic. A *pointer* + to an undefined slot (`*Self`) is fine — that's what self-reference needs. + +## Key code anchors + +- Builtins: `BuiltinId.declare` / `.define` (`src/ir/inst.zig`); lowering to + `callBuiltin` (`src/ir/lower/call.zig:tryLowerReflectionCall`); interp exec + + `defineEnum` + `decodeVariantElements` (`src/ir/interp.zig`); `mint` field + + `setMintTable`. +- Comptime evaluation: `evalComptimeType` / `renameNominalType` + (`src/ir/lower/comptime.zig`); decl trigger `fnReturnsTypeValue` + (`src/ir/lower/decl.zig`); type-fn trigger `returnExprMintsType` + + `instantiateTypeFunction` (`src/ir/lower/generic.zig`). +- Reflection: `field_type` → `fieldTypeOf` (`src/ir/lower/generic.zig`). +- Surface: `library/modules/std/meta.sx` (on-demand import — NOT the prelude, to + avoid shifting every `.ir` snapshot). + +## Cadence (IMPASSIBLE) + +No commit may both add a test AND make it pass (xfail-then-green, or a behavior +lock). `zig build && zig build test` after every step. Never regenerate snapshots +while red. Examples: `06xx` (comptime), `11xx` (diagnostics). + +## Status + +- [x] `declare` / `define` comptime builtins + the `mint` plumbing. +- [x] Comptime evaluation of a `Type`-returning `::` RHS and type-fn body + (the only triggers; no constructor-name knowledge in the compiler). +- [x] Name-in-`TypeInfo`; nominal identity via the instantiation cache. +- [x] `field_type` reflection (`examples/0616`). +- [x] Examples green on the floor: `0614` (one-shot), `0615` (type-fn identity), + `0617` (channel result types). +- [ ] **Self-reference** — the top-level `List :: declare(); define(List, …)` + two-statement form with a `*Self` payload (recursive / mutually-recursive + enums). Needs the `define(…)` top-level statement form to run at comptime in + the right window. Pairs with a `make_enum(variants: []EnumVariant)` sx helper + (computed variant list). +- [ ] **`type_info($T) -> TypeInfo`** — reflect a type INTO a value (inverse of + `define`'s decode): widen `TypeInfo` past `` `enum `` (struct/tuple) AND + construct a `[]EnumVariant`-style value in the interpreter. Bails loudly today + in `call.zig:tryLowerReflectionCall`. Round-trips through `define` once it lands. +- [ ] **Validation + loud diagnostics** — duplicate variant names, by-VALUE + self-reference (`payload = List` not `*List` → infinite size), a `declare()` + never `define()`d (hard error), use-before-define. Emit at the + `intern`/`internNominal` choke point; never a broken type. + +## Risks / watch + +- **Self-ref timing** — `define` for the two-statement form must complete before any + code uses the type's layout; a use-before-define must be a loud diagnostic, not a + silent empty enum. +- Keep `declare`/`define` **comptime-only**: reaching them at runtime is a hard error + (emit should bail loudly if one ever leaks into codegen). diff --git a/current/PLAN-POST-REIFY.md b/current/PLAN-POST-METATYPE.md similarity index 87% rename from current/PLAN-POST-REIFY.md rename to current/PLAN-POST-METATYPE.md index 64fbb398..d9a23215 100644 --- a/current/PLAN-POST-REIFY.md +++ b/current/PLAN-POST-METATYPE.md @@ -1,10 +1,10 @@ -# PLAN-POST-REIFY — program plan for the async-first roadmap (everything after reify) +# PLAN-POST-METATYPE — program plan for the async-first roadmap (everything after metatype) -Sequences every remaining stream after [PLAN-REIFY.md](PLAN-REIFY.md). This is the +Sequences every remaining stream after [PLAN-METATYPE.md](PLAN-METATYPE.md). This is the **program-level** plan; each stream below is carved into its own `PLAN-.md` + `CHECKPOINT-.md` (full step detail + kickoff prompt) -**when reached**, exactly as reify was. Rationale, the five reify contracts, risk -ranking (§8.1), and the testing strategy (§10) all live in the design-of-record: +**when reached**, exactly as metatype was. Rationale, the comptime type-construction +design, risk ranking (§8.1), and the testing strategy (§10) all live in the design-of-record: [../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md). **Cadence (IMPASSIBLE), every stream:** no commit both adds a test AND makes it pass @@ -15,12 +15,12 @@ mark the stream checkpoint BLOCKED, stop (CLAUDE.md rule). **Ordering = async-first** (design §7): the async story needs no JIT spine, so the JIT/FFI cluster comes after. New corpus categories: `17xx` atomics, `18xx` concurrency. -## Stream order (post-reify) +## Stream order (post-metatype) | # | Stream | Roadmap steps | Depends on | Notes | |---|--------|---------------|-----------|-------| | **A** | Atomics | N1 (1) | — | independent foundation; gates B-parallel + channels | -| **B** | Async runtime | 4–12 | reify, A (for channels) | the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved | +| **B** | Async runtime | 4–12 | metatype, A (for channels) | the bulk; likely splits into B1 (runtime) + B2 (channels/cancel/stdlib) when carved | | **C** | Parallel schedulers | 13–14 | A, B | N×(M:1) → M:N | | **D** | Comptime JIT/FFI | 15–18 | — (independent of async) | S1 → C1 → C2 → C3 | | **E** | Hot-reload (deferred) | 19–22 | D (S1/S2) | S2 → R1 → R2 → R3 | @@ -78,14 +78,14 @@ piece (§8.1.1). ### B2 — Channels + cancellation + stdlib (`PLAN-CHANNELS.md`) - B2.0 **N3 — channels** (`Channel($T)`; `recv → RecvResult($T)` tagged union built via - **reify** type-fn) + fiber-aware `Mutex`/`WaitGroup` (atomic fast-path from A). + **metatype** type-fn) + fiber-aware `Mutex`/`WaitGroup` (atomic fast-path from A). - B2.1 **A6 — cancellation** = `.canceled` in the existing `!` channel (model a); per- fiber atomic flag (A); every `io.*` a cancellation point; structured cancel-and-join; **masked during cleanup**. Rides ERR (`try`/`onfail`/`defer`). - B2.2 **A4 — stdlib I/O rework** — fs/socket/process onto `context.io`. **Gates:** `18xx` under deterministic `Io`; cancellation cleanup asserted via stdout -ordering; `RecvResult` exercises the reify contracts. +ordering; `RecvResult` exercises the metatype primitives. --- @@ -145,17 +145,17 @@ module hazard; S2 TLS + C-constructor JIT test per host OS (the exact prior-spik - **Testing keystone:** the deterministic-sim `Io` (B1.4) must exist + be calibrated before *any* async test is trusted (§10.1). -- **Top risks to watch (§8.1):** A2 context-switch correctness (B1.3), reify→match - (de-risked, reify stream), deterministic-`Io` oracle calibration, `context`-fiber- - local/errno (C), S2 (E), C1 args-buffer layout (D). +- **Top risks to watch (§8.1):** A2 context-switch correctness (B1.3), minted-enum → + match codegen (de-risked, metatype stream), deterministic-`Io` oracle calibration, + `context`-fiber-local/errno (C), S2 (E), C1 args-buffer layout (D). - **The compiler floor stays small, but deep:** atomics, `callconv(.naked)`, repointable- - `context` codegen, `type_info`/`reify` (reify stream), the S1 JIT spine. Everything - else — schedulers, fibers, channels, the bundler — is sx lib. + `context` codegen, `declare`/`define`/`type_info` (metatype stream), the S1 JIT spine. + Everything else — schedulers, fibers, channels, the bundler — is sx lib. ## Carving protocol When a stream is reached: copy this section into `current/PLAN-.md`, expand the phases to xfail→green steps with file anchors (from the design doc's anchor list), add a `CHECKPOINT-.md`, and write a Phase-0-scoped kickoff prompt (mirror -PLAN-REIFY's). Update [CHECKPOINT-REIFY.md](CHECKPOINT-REIFY.md)/this file's status as +PLAN-METATYPE's). Update [CHECKPOINT-METATYPE.md](CHECKPOINT-METATYPE.md)/this file's status as streams complete. diff --git a/current/PLAN-REIFY.md b/current/PLAN-REIFY.md deleted file mode 100644 index dce9f3e6..00000000 --- a/current/PLAN-REIFY.md +++ /dev/null @@ -1,205 +0,0 @@ -# PLAN-REIFY — comptime type reflection + construction (`type_info` / `reify`) - -## Goal - -Add the two comptime metaprogramming builtins — **`type_info($T) -> TypeInfo`** -(reflect a type → data) and **`reify(info: TypeInfo) -> Type`** (construct a *new -nominal type* from data) — plus the sx-lib helpers (`make_enum`, `field_type`, -`RecvResult`/`TryResult`) built over them. This is **step 3 of the async-first -sequence** ([../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md) -§7); it gates channel result types (`RecvResult($T)`) and `race`'s synthesized -tagged-union, and **replaces** a would-be `enum($T)` generic-enum language feature. - -> Rationale + the five validated contracts: design doc §7 step 3 + §8.1. The approach -> was grounded by three codebase reviewers — it is a **small extension reusing existing -> machinery**, not net-new architecture. - -## Locked design (the five reify contracts — all codebase-validated) - -1. **Nominal identity via type-fn memoization.** `RecvResult(i64)` is one `TypeId` - because type-fns dedup by mangled `(fn,args)` name (`generic.zig:1620-1629`) + - reify `findByName`. NOT structural dedup — enums are nominal (`types.zig:1110`). -2. **Functional through codegen.** A reify'd enum has **no backing AST decl**, and - every enum stage is type-table-driven (layout, construct, match+exhaustiveness, - `toLLVMType`, `type_name`/format) — so it flows through **unmodified**. -3. **Validate loudly** at the `intern`/`internNominal` choke point (`types.zig:411-439`). -4. **Comptime-only, JIT-free** — a type-table op in the interpreter; no S1 dependency. -5. **Reference-based self-reference** (`*Self`/`[]Self`) via an explicit - **`declare()` → `define()`** pair (reserve-placeholder→complete reuse of - `nominal.zig:86/108/120`, `types.zig:442`); **by-value recursion rejected**. (See - Phase 4 — the `declare`/`define` API supersedes the earlier `reify_rec` closure.) - -Surface follows the **`#builtin`** pattern of the existing reflection builtins -(`type_of`/`field_count`/`field_name` in `library/modules/std/core.sx`, -`specs.md:2594-2600`) — NOT the BuildOptions compiler-hook registry. - -## Key code anchors (verified by review) - -- Type minting: `TypeTable.intern` / `internNominal` — `src/ir/types.zig:411-439`. -- Type-fn instantiation + mangled-name cache — `src/ir/lower/generic.zig:1575-1689` - (cache check `:1620-1629`; register inline-struct result `:1663-1689`). -- Forward-declare reserve (recursive types) — `src/ir/lower/nominal.zig:86/108/120`; - complete a forward-declared type — `src/ir/types.zig:442`. -- Enum codegen (all type-table-driven, the reify target shape): size `types.zig:633-636`; - `resolveVariantIndex` `lower/expr.zig:1159-1177`; match `lower/control_flow.zig:748-945`; - `toLLVMType` `backend/llvm/types.zig:111-154`; `type_name` `types.zig:846-882`. -- Existing reflection builtins to mirror — `core.sx` (`#builtin`) + their interp/lower - handlers (`src/ir/interp.zig` `type_name`/reflection at ~`:1911`). -- Match form — `specs.md:408-424`. - -## Cadence (IMPASSIBLE) - -No commit may both add a test AND make it pass (xfail-then-green, or a behavior-lock). -`zig build && zig build test` after every step. Never regenerate snapshots while red. -Examples: `06xx` (comptime, deterministic), `11xx` (diagnostics for loud failures). - ---- - -## RE-ARCHITECTURE (user-directed, 2026-06-16) — `declare`/`define` are the ONLY compiler primitives - -**Decision:** the compiler must expose **only** the two comptime type-table -primitives `declare()` and `define(handle, info)`. **`reify`, `make_enum`, -`RecvResult`, `TryResult` all become plain sx** in `meta.sx` — `reify` is literally -`{ h := declare(); define(h, info); return h; }`. The AST-walking `reifyType` and -every *syntactic* recognition of `reify` (`decl.zig`'s `E :: reify(...)` hook, -`generic.zig`'s `findReturnReifyCall` type-fn routing) are **deleted** — replaced by -generic **comptime evaluation of a Type-returning expression**. - -This supersedes the old builtin-`reify` approach of Phases 0/1/3 below (which are -GREEN today but compiler-coupled). Phase 3.2 (`make_enum` + the eval-decode reader) -was implemented then **reverted** (commit reset to 9306ad5) to rebuild on this floor. - -### The irreducible compiler floor (cannot be sx — mints `TypeId`s / evaluates types) -1. **`declare() -> Type`** — intern a fresh EMPTY forward nominal slot (unique - `nominal_id`, anonymous name), return it as a first-class `Type` value. Reuses the - `reserveShadowEnumSlot` shape (`nominal.zig:98`). -2. **`define(handle: Type, info: TypeInfo)`** — decode the `TypeInfo` **value** (the - reverted `reifyEnumFromVariantsValue` decode logic moves here: interp `Value` → - names + payload `Type`-tags → `tagged_union` byte-identical to `buildEnumInfo`), - fill the slot via `updatePreservingKey` (`types.zig:442`). Loud on bad input. -3. **Comptime-evaluate a Type-returning `::` RHS / type-fn body** through the - interpreter to a `type_tag`, then bind / rename / identity-cache. This is the new - general capability that REPLACES the `reify`-name pattern-matching. - -### Naming + identity (resolved) -- `declare()` mints ANONYMOUS; the **binding site** names it: a direct `X :: ` - renames the handle to `X`; a type-fn instantiation renames to the mangled name. -- **Identity (Contract 1)** still rides the existing `instantiateTypeFunction` - mangled-name cache — `RecvResult(i64)` memoizes, body (hence `declare`) runs once. - -### Floor phases (REPLACES old Phases 0/1/3; re-greens 0614/0615/0617) -| Step | Commit | What | Files | -|---|---|---|---| -| F1.0 | xfail | `examples/06xx` mints a **recursive** enum directly: `List :: declare();` + `define(List, .enum(.{ variants = .[ … payload = *List … ] }))`; construct + match a cons/nil list. Red (declare/define undefined). | `examples/06xx-*` | -| F1.1 | green | `declare`/`define` as comptime builtins + the `X :: declare()` decl hook + scan-time `define(...)` side-effect + the value-decode (from the reverted 3.2). Self-ref works because the handle predates the body. | `src/ir/lower/{decl,nominal}.zig`, `src/ir/interp.zig`, `meta.sx` | -| F2.0 | xfail | Reimplement `reify` in `meta.sx` as `declare`+`define`+return; switch `E :: reify(...)` to the **general comptime-Type-expr** decl path; delete the `E :: reify` AST special-case + `reifyType` literal walk. Red until the general path lands. | `meta.sx`, `src/ir/lower/decl.zig` | -| F2.1 | green | General comptime evaluation of a Type-returning `::` RHS (run interp → `type_tag` → rename/bind). `examples/0614` green again with ZERO `reify` knowledge in the compiler. | `src/ir/lower/decl.zig`, `src/ir/interp.zig` | -| F3.0 | xfail | Delete `findReturnReifyCall` routing; make `instantiateTypeFunction` run a Type-returning body through the interp generally. Red for 0615/0617 until it lands. | `src/ir/lower/generic.zig` | -| F3.1 | green | General type-fn body comptime-eval + rename to mangled name + identity cache. `examples/0615` (cross-site identity) + `0617` (RecvResult/TryResult) green on the floor. | `src/ir/lower/generic.zig` | -| F4 | xfail→green | `make_enum(variants)` in `meta.sx` — now TRIVIAL (reify is sx, type-fns eval generally). New `examples/06xx` (inline + computed variants). | `meta.sx`, `examples/06xx-*` | -| F5 | xfail→green | Validation + loud diagnostics: dup variant names, by-VALUE self-ref (`payload = List` not `*List` → "infinite size"), a `declare()` never `define()`d (hard error at end of comptime). | `src/ir/interp.zig` / `src/ir/types.zig`, `examples/11xx-*` | - -> `type_info($T) -> TypeInfo` (old Phase 2.2 — reflect a type INTO a value) is -> orthogonal to this floor and still pending; it CONSTRUCTS a `TypeInfo` value -> (inverse of `define`'s decode). Round-trips through the floor once it lands. - ---- - -## (SUPERSEDED by the re-architecture above) Original builtin-`reify` phases - -### Phase 0 — `reify` of a flat enum (the core) -| Step | Commit | What | Files | -|---|---|---|---| -| 0.0 | lock | `TypeInfo`/`EnumInfo`/`EnumVariant` lib types in `core.sx` (data only); `reify`/`type_info`/`field_type` as bodyless `#builtin` decls (parsed, unimplemented → loud bail). Unit: decls parse. | `library/modules/std/core.sx`, `src/ir/interp.zig` | -| 0.1 | xfail | `examples/06xx-comptime-reify-enum.sx` — `reify(.enum_(.{variants=[.{name="value",payload=i64},.{name="closed",payload=void}]}))`, construct `.value(3)`, match it. Red (reify unimplemented). | `examples/06xx-*` | -| 0.2 | green | implement `reify(.enum_)` → build `EnumInfo`/`TaggedUnionInfo` `TypeInfo`, `internNominal(info, fresh_nominal_id)`, return `TypeId`. Example green; construct + match work unmodified (Contract 2). | `src/ir/interp.zig`, (`src/ir/types.zig` if a helper is wanted) | - -### Phase 1 — type-fn → reify identity -| Step | Commit | What | Files | -|---|---|---|---| -| 1.0 | xfail | `examples/06xx-comptime-reify-typefn-identity.sx` — `R :: ($T)->Type { reify(...) }`; assert `R(i64)` from two sites is ONE type (assignable/matchable across sites). Red if reify-result not registered by mangled name. | `examples/06xx-*` | -| 1.1 | green | register a reify-returning type-fn's result under the instantiation mangled name (mirror the inline-struct path `generic.zig:1663-1689`). Identity holds (Contract 1). | `src/ir/lower/generic.zig` | - -### Phase 2 — `field_type` (done) + `type_info` (reflect → value, pending) -| Step | Commit | What | Files | -|---|---|---|---| -| 2.0 | xfail | reflect a struct/tuple/tagged-union → read field/variant names + **types** (`field_type($T,i)`). Red. | `examples/0616-*` | -| 2.1 | green | **DONE.** `field_type($T, i) -> Type` over the type table (`fieldTypeOf` in `generic.zig`): struct field / tagged-union+union variant payload / tuple element / array+vector element; OOB + memberless → loud poison. Folds at lower time, composes in `type_eq`/`type_name`. | `src/ir/lower/generic.zig` | -| 2.2 | xfail→green | **PENDING.** `type_info($T) -> TypeInfo` — reflect a type into a `TypeInfo` *value* (the inverse of reify). Needs the `TypeInfo` data model widened (struct/tuple variants beyond `` `enum ``) AND comptime construction of a `[]EnumVariant`-style value (slice of structs holding strings + `Type` tags) read from the type table. Still bails loudly in `call.zig:tryLowerReflectionCall`. Larger than 2.1 — its own step. | `library/modules/std/meta.sx`, `src/ir/interp.zig` / `call.zig` | - -### Phase 3 — `RecvResult`/`TryResult` (done) + `make_enum` (pending) -| Step | Commit | What | Files | -|---|---|---|---| -| 3.0 | xfail | `examples/0617` uses `RecvResult(i64)`/`TryResult(i64)` (construct + match). Red (undefined). | `examples/0617-*` | -| 3.1 | green | **DONE.** `RecvResult($T)`/`TryResult($T)` as type-fns over `reify` in `meta.sx` — needs NO new machinery (reify-of-literal in a type-fn body = Phase 1). `0617` green. | `library/modules/std/meta.sx` | -| 3.2 | pending | **`make_enum(variants: []EnumVariant) -> Type`** — sx lib over `reify` with a RUNTIME (non-literal) `variants` arg. BLOCKED on the generalized reify reader: `reifyType` currently reads a LITERAL `TypeInfo` off the AST, so `reify(.enum(.{ variants = variants }))` (variable, not a literal) can't be read yet. Pairs with Phase 2.2 (interp-evaluate the `TypeInfo` value instead of AST-walking). | `library/modules/std/meta.sx`, `src/ir/lower/nominal.zig` | - -### Phase 4 — reference-based self-reference (explicit declare → define) -> **API decision (user-directed):** an explicit **`declare()` / `define(h, info)`** -> pair (the canonical declaration-vs-definition split), NOT a `reify_rec((self) => …)` -> closure. `declare()` returns a forward-declared nominal `Type` HANDLE — a real -> `Type` value usable freely in any later `TypeInfo` (`*List`, `[]List`, and across -> types for MUTUAL recursion, which a one-`self` closure can't express). -> `define(handle, info)` fills the body. `reify(info)` stays as the one-shot sugar -> (= declare + define + return) for the non-recursive case. No special self-ref node -> in `TypeInfo` — a declared type is just a `Type`. -> ```sx -> List :: declare(); // name from the `::` LHS -> define(List, .enum(.{ variants = .[ -> EnumVariant.{ name = "cons", payload = *List }, // reference it freely -> EnumVariant.{ name = "nil", payload = void } ] })); -> ``` -> Loud invariants: (a) a `declare()` never `define()`d = hard error at end of -> comptime (no silent empty type); (b) by-VALUE self-inclusion (`payload = List`, not -> `*List`) rejected — infinite size, same check recursive source types get. - -| Step | Commit | What | Files | -|---|---|---|---| -| 4.0 | xfail | `declare()` returns a forward nominal `Type`; `define()` fills it. Recursive enum via `*Self` (tree/list). Red. | `examples/06xx-*` | -| 4.1 | green | `declare` = intern empty nominal slot (`reserveShadowEnumSlot`-style); `define` = `buildEnumInfo` + `updatePreservingKey`; struct-stub→tagged_union re-key already handled by `internNamedTypeDecl` (`adoptsForwardStructStub`). | `src/ir/lower/nominal.zig`, `src/ir/lower/decl.zig`, `src/ir/types.zig` | - -### Phase 5 — validation + loud diagnostics -| Step | Commit | What | Files | -|---|---|---|---| -| 5.0 | xfail | `examples/11xx-diagnostics-reify-*` — dup variant names, non-integer backing, **by-value self-reference** ("infinite size; use `*Self`"). Pin the messages. | `examples/11xx-*` | -| 5.1 | green | validate `TypeInfo` at the `intern`/`internNominal` choke point; emit diagnostics, never a broken type (Contract 3). | `src/ir/interp.zig` / `src/ir/types.zig` | - -> `RaceResult` (tuple→tagged-union synthesis) is **not** in this stream — it lands with -> `race` (async cluster), but it consumes exactly the `type_info`+`field_type`+`reify` -> primitives built here. - -## Risks / watch - -- **Mangled-name plumbing (Phase 1)** is the one real unknown — confirm the type-fn - path registers a *reify-returned* result (not just inline `struct {…}` literals). - Fallback: have `reify` itself name the type by the instantiation key + `findByName`. -- **Self-ref completion (Phase 4)** must reuse the existing recursive-type - reserve→complete path; do not invent a new mutate-after-intern mechanism. -- Keep `reify` **comptime-only**: a `reify` reached at runtime is a hard error. - -## Status - -- [x] Phase 0 — `reify` flat enum (`reify(.enum(...))` mints a flat enum via the - shared `buildEnumInfo` path; `examples/0614` green; Contract 2 confirmed) -- [x] Phase 1 — type-fn identity (`Box :: ($T)->Type { reify(...) }` memoizes by - mangled name; `Box(i64)` at two sites is one type; `examples/0615` green) -- [~] Phase 2 — `field_type` DONE (`examples/0616` green); `type_info` (reflect → a - `TypeInfo` value) PENDING as step 2.2 -- [~] Phase 3 — `RecvResult($T)`/`TryResult($T)` DONE (`examples/0617` green, type-fns - over `reify` in `meta.sx`); `make_enum(variants)` PENDING (step 3.2 — needs the - generalized reify reader, pairs with 2.2) -- [ ] Phase 2 — `type_info` + `field_type` -- [ ] Phase 3 — `make_enum` + `RecvResult`/`TryResult` -- [ ] Phase 4 — reference self-reference -- [ ] Phase 5 — validation + diagnostics - -## Kickoff prompt (paste into a fresh session) - -> Work the REIFY stream per `current/PLAN-REIFY.md` (+ checkpoint -> `current/CHECKPOINT-REIFY.md`). Read the plan header (goal, five locked contracts, -> key anchors) first; rationale is in `design/execution-evolution-roadmap.md` §7 step 3 -> + §8.1. **This session = Phase 0 only** (`TypeInfo` lib types + `reify` of a flat -> enum: construct + match). Cadence (IMPASSIBLE): no commit both adds a test and makes -> it pass — lock, then xfail→green. `zig build && zig build test` after every step. If -> you hit an unrelated compiler bug, follow the CLAUDE.md IMPASSIBLE RULE (file an -> issue, stop). Stop at the end of Phase 0; update the checkpoint. diff --git a/design/execution-evolution-roadmap.md b/design/execution-evolution-roadmap.md index 1496c7ab..86244964 100644 --- a/design/execution-evolution-roadmap.md +++ b/design/execution-evolution-roadmap.md @@ -27,8 +27,8 @@ features into codegen. Concretely: The honest trade is **small *surface*, but each primitive is *deep*** — not "small compiler." The net-new **compiler** obligations this plan adds (all verified absent -today): **atomics lowering** (N1), **generic enums** `enum($T)`, **`type_info` + -`reify` + `field_type`** (comptime type construction), **`callconv(.naked)`**, +today): **atomics lowering** (N1), **generic enums** `enum($T)`, **`declare` + +`define` + `type_info` + `field_type`** (comptime type metaprogramming), **`callconv(.naked)`**, **repointable-`context` codegen** (+ per-fiber stack-limit), the **S1 persistent JIT spine**, **C1 thunk synthesis**, **comptime-asm lifting** (C3), and (later) the **S2 ORC C++ shim**. Async itself is genuinely a library; the *enabling primitives* are a @@ -363,36 +363,38 @@ grounding) are explicit steps, not buried. `atomicrmw`/`cmpxchg`/`fence` emission + orderings. Surface = `Atomic($T)` wrapper. Gates channels/N3 + parallel schedulers. 2. ~~**Generic enums** `enum($T)`~~ **DROPPED.** `RecvResult($T)`/`TryResult($T)` are - **type-fns over `reify`** (step 3), not a new `enum($T)` language feature — and - type-fns (user `($T)->Type` in type position) **already work** (e.g. + **type-fns over `declare`/`define`** (step 3), not a new `enum($T)` language + feature — and type-fns (user `($T)->Type` in type position) **already work** (e.g. [`Make`](../examples/0208-generics-value-param-type-function.sx), [`Complex`](../examples/0201-generics-generic-struct.sx)). A declarative `enum($T)` - surface, if ever wanted, is later *sugar* desugaring to a type-fn-over-`reify`. -3. **`type_info` + `reify` + `field_type`** — comptime metaprogramming floor. Gates - `race` synthesis **and** channel `RecvResult`/`TryResult` (all type-fns over - `reify`; **generic-enum syntax dropped**). **Validated against the codebase (3 - reviewers): a small extension reusing existing machinery throughout — not net-new - architecture.** Five contracts: + surface, if ever wanted, is later *sugar* desugaring to a type-fn over the primitives. +3. **`declare`/`define` (construction) + `type_info`/`field_type` (reflection)** — + comptime metaprogramming floor. Gates `race` synthesis **and** channel + `RecvResult`/`TryResult` (all sx type-fns over `declare`/`define`; **generic-enum + syntax dropped**). **Validated against the codebase (3 reviewers): a small + extension reusing existing machinery throughout — not net-new architecture.** + Contracts: 1. **Nominal identity via type-fn memoization** — type-fns dedup by mangled - `(fn,args)` name (generic.zig:1620-1629) + reify `findByName`, so `RecvResult(i64)` - is one `TypeId` and the body runs once. (NOT structural dedup — enums are - nominal via `nominal_id`, types.zig:1110.) + `(fn,args)` name (generic.zig) + `findByName`, so `RecvResult(i64)` is one + `TypeId` and the body runs once. (NOT structural dedup — enums are nominal via + `nominal_id`, types.zig.) 2. **Functional through codegen** — layout / construct / match+exhaustiveness / `toLLVMType` / `type_name`+format are **all type-table-driven, zero AST - coupling**, so a backing-decl-less reify'd enum flows through unmodified. + coupling**, so a backing-decl-less minted enum flows through unmodified. 3. **Validate loudly** at the single `intern`/`internNominal` choke point - (types.zig:411-439): reject dup variants / bad backing / unresolved payloads. + (types.zig): reject dup variants / bad backing / unresolved payloads. 4. **Comptime-only, JIT-free** — a type-table op in the interp; no S1 dependency - (keeps reify, hence channels + `race`, off the JIT critical path). - 5. **Reference-based self-reference (v1)** — `*Self`/`[]Self` payloads via the - reserve-placeholder→complete path recursive *source* types already use - (nominal.zig:86/108/120, types.zig:442); **by-value recursion rejected** (loud, - infinite size). reify gains a `reify_rec((self) => …)` builder form. + (keeps construction, hence channels + `race`, off the JIT critical path). + 5. **Reference-based self-reference** — `*Self`/`[]Self` payloads via the + explicit `declare()` → `define(handle, …)` split (the handle predates its + body, so it can be referenced inside it); **by-value recursion rejected** + (loud, infinite size). Reuses the reserve-placeholder→complete path recursive + *source* types already use (nominal.zig, types.zig). - **Type-minting precedents (7):** monomorphization, protocol vtables, tuples, vector/array, ptr/slice ctors, FFI stubs, **type-fn instantiation** — all construct `TypeInfo` programmatically + `intern()`. **Residual = plumbing, not - capability:** name reify-results by the instantiation's mangled name (done for - inline-struct bodies — extend to reify-results) + reify input validation. + capability:** name minted results by the instantiation's mangled name + input + validation. 4. **`callconv(.naked)`** — extend `CallConv {default, c}` (types.zig:169) + skip prologue/epilogue lowering. Gates A2. 5. **Repointable-`context` codegen** — lower `context` as a swappable indirection @@ -447,11 +449,11 @@ persistent, only leaf logic reloads). *scheduling*, not the *switch*); a one-register slip is invisible until it crashes on the right arch. Couples *library asm* to the *compiler ABI* — ABI drift breaks it silently later. → needs a dedicated **switch-stress test** (§10). -2. **`reify` → anonymous-tagged-union → match-codegen** (gates `race` + channels). +2. **`define` → tagged-union → match-codegen** (gates `race` + channels). **DE-RISKED by review** (§7 step 3): all enum stages are type-table-driven with zero AST coupling, identity is handled by existing type-fn mangled-name memoization, and forward-declaration for self-ref already exists. Residual is *plumbing* - (name reify-results by mangled name + input validation), not new architecture. + (name minted results by mangled name + input validation), not new architecture. 3. **Deterministic-`Io` is the test keystone yet itself uncalibrated** — a buggy deterministic scheduler yields deterministic-*wrong* stdout that snapshots lock in. → calibrate against the blocking `Io` / property-test fixed order (§10). @@ -477,9 +479,10 @@ precedes the FFI/JIT cluster (15–18) because async needs no JIT spine. **Cance the "keep calls clean" argument for the non-local-`raise` model is moot). Reuses `!`/`try`/`catch`/`onfail`; no new unwind primitive. **Net-new prereq surfaced by grounding:** `callconv(.naked)` (only `.default`/`.c` today). **Generic enums dropped** -— `RecvResult($T)`/`TryResult($T)` are **type-fns over `reify`** (type-fns already work -in type position, e.g. `Make`/`Complex`), so no `enum($T)` feature is needed; `reify` -gains two contracts (deterministic identity + functional-enum output, §7 step 3). +— `RecvResult($T)`/`TryResult($T)` are **type-fns over `declare`/`define`** (type-fns +already work in type position, e.g. `Make`/`Complex`), so no `enum($T)` feature is +needed; construction carries two contracts (deterministic identity + functional-enum +output, §7 step 3). **Locked (see §4.6 for the grounded surface):** - **N1 atomics surface = generic wrapper `Atomic($T)`** + `Ordering` enum, `.init`, @@ -493,10 +496,11 @@ gains two contracts (deterministic identity + functional-enum output, §7 step 3 `RecvResult($T){ value; closed }` (not `(v, ok)`), `try_recv` → `{ value; empty; closed }`; optional `for ch (v) {…}` iteration sugar. **locks** = `lock()` + `defer unlock()` (no guard sugar). `race`/`async`/`await` stay library, not keywords. -- **Comptime type metaprogramming = `type_info` + `reify` builtins only** (Zig - `@typeInfo`/`@Type` model). **Everything else is sx lib** — `make_enum`, - `field_type`, `RaceResult`. `reify` coverage starts at **enum/struct/tuple**, grows - later. `Future($T)` exposes `Value :: T` so `Future(X)→X` is plain member access +- **Comptime type metaprogramming = `declare`/`define` (construct) + `type_info` + (reflect) builtins only** (Zig `@Type`/`@typeInfo` model). **Everything else is sx + lib** — `make_enum`, the channel result types, `field_type`, `RaceResult`. + Construction coverage starts at **enum**, grows to struct/tuple later. `Future($T)` + exposes `Value :: T` so `Future(X)→X` is plain member access (no `type_arg` builtin). - **C1 FFI engine = LLVM as single ABI authority** — per-signature JIT calling-thunks via S1 (LLVM emits the ABI-correct call, same as runtime codegen); trampoline @@ -571,8 +575,8 @@ per-arch run tests on matching runners. ### 10.4 New corpus categories `17xx` atomics · `18xx` concurrency (fibers/channels/race/async, all under the -deterministic `Io`). Comptime metaprogramming (`type_info`/`reify`) + comptime-asm -extend `06xx`; C1 FFI extends `12xx`; the cross-arch comptime-asm **loud bail** and +deterministic `Io`). Comptime metaprogramming (`declare`/`define`/`type_info`) + +comptime-asm extend `06xx`; C1 FFI extends `12xx`; the cross-arch comptime-asm **loud bail** and the cancellation diagnostics are `11xx`. ### 10.5 Per-piece gates (design level) @@ -580,7 +584,7 @@ the cancellation diagnostics are `11xx`. | Piece | Locks via | |---|---| | **N1 atomics** | unit `emit_llvm.test.zig` (LLVM `atomicrmw`/`cmpxchg`/`fence` + ordering emission); corpus `17xx` single-thread (deterministic); arch-gated `.ir` (x86_64 + aarch64) | -| **type_info / reify** | unit (reflect round-trips; reify'd enum has correct layout/match codegen); corpus `06xx` comptime (deterministic) | +| **declare / define / type_info** | unit (reflect round-trips; a minted enum has correct layout/match codegen); corpus `06xx` comptime (deterministic) | | **C1 FFI** | **behavior-lock** existing trampoline cases first; then xfail→green `12xx` comptime extern with floats / structs-by-value / aggregate (`{ptr,len}`) returns; unit for thunk-synth + args-buffer marshal | | **S1 spine** | infra — exercised transitively via C1/C3 examples; unit for LLJIT lifecycle + thunk cache | | **C3 comptime asm** | corpus `06xx` host-arch `#run` asm computes a value; `11xx` diagnostic asserts the cross-arch loud bail | diff --git a/examples/0614-comptime-reify-enum.sx b/examples/0614-comptime-metatype-enum.sx similarity index 64% rename from examples/0614-comptime-reify-enum.sx rename to examples/0614-comptime-metatype-enum.sx index e15d3302..91346f78 100644 --- a/examples/0614-comptime-reify-enum.sx +++ b/examples/0614-comptime-metatype-enum.sx @@ -1,7 +1,8 @@ -// REIFY Phase 0: mint a NEW nominal enum from a `TypeInfo` value at comptime, -// then construct one of its variants and match on it — exercising that a -// reify'd enum (with NO backing AST decl) flows through enum codegen unmodified -// (layout / construct / match), Contract 2. +// Comptime type construction: mint a NEW nominal enum from a `TypeInfo` value +// via the `define(declare(), info)` primitives, then construct one of its +// variants and match on it — exercising that a programmatically-built enum +// (with NO backing AST decl) flows through enum codegen unmodified (layout / +// construct / match), byte-identical to a hand-written enum. // // The enum has two variants: `value` carrying an i64 payload, and `closed` with // no payload (`payload = void`). diff --git a/examples/0615-comptime-reify-typefn-identity.sx b/examples/0615-comptime-metatype-typefn-identity.sx similarity index 57% rename from examples/0615-comptime-reify-typefn-identity.sx rename to examples/0615-comptime-metatype-typefn-identity.sx index c2b34879..115d0770 100644 --- a/examples/0615-comptime-reify-typefn-identity.sx +++ b/examples/0615-comptime-metatype-typefn-identity.sx @@ -1,10 +1,10 @@ -// REIFY Phase 1: a type-fn that RETURNS `reify(...)` must memoize by the -// instantiation's mangled name, so `Box(i64)` resolved at two INDEPENDENT sites -// (here: a return type and a parameter type) is ONE `TypeId`. A value built at -// one site is therefore assignable / matchable at the other — nominal identity -// (Contract 1). If the reify result were not registered under the mangled -// instantiation name, the two sites would mint distinct types and -// `consume(build())` would be a type error. +// Comptime type construction — identity: a type-fn that builds a type with +// `define(declare(), ...)` must memoize by the instantiation's mangled name, so +// `Box(i64)` resolved at two INDEPENDENT sites (here: a return type and a +// parameter type) is ONE `TypeId`. A value built at one site is therefore +// assignable / matchable at the other — nominal identity. If the minted result +// were not registered under the mangled instantiation name, the two sites would +// mint distinct types and `consume(build())` would be a type error. #import "modules/std.sx"; #import "modules/std/meta.sx"; diff --git a/examples/0616-comptime-field-type.sx b/examples/0616-comptime-field-type.sx index 846a5c92..ed55d554 100644 --- a/examples/0616-comptime-field-type.sx +++ b/examples/0616-comptime-field-type.sx @@ -1,5 +1,6 @@ -// REIFY Phase 2: `field_type($T, i) -> Type` — the type-only field projection -// the value-level reflection (`field_value` / `type_of`) couldn't express. +// Comptime reflection: `field_type($T, i) -> Type` — the type-only field +// projection the value-level reflection (`field_value` / `type_of`) couldn't +// express. // Reflect a struct's fields by name (`field_name`) AND by type (`field_type`), // and a tagged-union's variant payloads. It folds at lower time, so it composes // inside `type_eq` / `type_name` / any type-arg slot. diff --git a/examples/0617-comptime-reify-recvresult.sx b/examples/0617-comptime-metatype-channel-results.sx similarity index 63% rename from examples/0617-comptime-reify-recvresult.sx rename to examples/0617-comptime-metatype-channel-results.sx index af49d7f1..a2737f3b 100644 --- a/examples/0617-comptime-reify-recvresult.sx +++ b/examples/0617-comptime-metatype-channel-results.sx @@ -1,9 +1,10 @@ -// REIFY Phase 3: RecvResult($T) / TryResult($T) — the channel result types, -// built ENTIRELY in sx library code as type-fns over `reify` (no new compiler -// machinery beyond Phases 0–1). A blocking recv yields a value or a `closed` -// marker; a non-blocking try-recv adds `empty` — three states a bool can't -// express. This locks that they construct and match like any enum, and that -// `RecvResult(i64)` is one nominal type across sites (the type-fn identity path). +// Comptime type construction — channel result types: RecvResult($T) / +// TryResult($T), built ENTIRELY in sx library code as type-fns over the +// `define(declare(), ...)` primitives (no compiler machinery). A blocking recv +// yields a value or a `closed` marker; a non-blocking try-recv adds `empty` — +// three states a bool can't express. This locks that they construct and match +// like any enum, and that `RecvResult(i64)` is one nominal type across sites +// (the type-fn identity path). #import "modules/std.sx"; #import "modules/std/meta.sx"; diff --git a/examples/expected/0614-comptime-reify-enum.exit b/examples/expected/0614-comptime-metatype-enum.exit similarity index 100% rename from examples/expected/0614-comptime-reify-enum.exit rename to examples/expected/0614-comptime-metatype-enum.exit diff --git a/examples/expected/0614-comptime-reify-enum.stderr b/examples/expected/0614-comptime-metatype-enum.stderr similarity index 100% rename from examples/expected/0614-comptime-reify-enum.stderr rename to examples/expected/0614-comptime-metatype-enum.stderr diff --git a/examples/expected/0614-comptime-reify-enum.stdout b/examples/expected/0614-comptime-metatype-enum.stdout similarity index 100% rename from examples/expected/0614-comptime-reify-enum.stdout rename to examples/expected/0614-comptime-metatype-enum.stdout diff --git a/examples/expected/0615-comptime-reify-typefn-identity.exit b/examples/expected/0615-comptime-metatype-typefn-identity.exit similarity index 100% rename from examples/expected/0615-comptime-reify-typefn-identity.exit rename to examples/expected/0615-comptime-metatype-typefn-identity.exit diff --git a/examples/expected/0615-comptime-reify-typefn-identity.stderr b/examples/expected/0615-comptime-metatype-typefn-identity.stderr similarity index 100% rename from examples/expected/0615-comptime-reify-typefn-identity.stderr rename to examples/expected/0615-comptime-metatype-typefn-identity.stderr diff --git a/examples/expected/0615-comptime-reify-typefn-identity.stdout b/examples/expected/0615-comptime-metatype-typefn-identity.stdout similarity index 100% rename from examples/expected/0615-comptime-reify-typefn-identity.stdout rename to examples/expected/0615-comptime-metatype-typefn-identity.stdout diff --git a/examples/expected/0617-comptime-reify-recvresult.exit b/examples/expected/0617-comptime-metatype-channel-results.exit similarity index 100% rename from examples/expected/0617-comptime-reify-recvresult.exit rename to examples/expected/0617-comptime-metatype-channel-results.exit diff --git a/examples/expected/0617-comptime-reify-recvresult.stderr b/examples/expected/0617-comptime-metatype-channel-results.stderr similarity index 100% rename from examples/expected/0617-comptime-reify-recvresult.stderr rename to examples/expected/0617-comptime-metatype-channel-results.stderr diff --git a/examples/expected/0617-comptime-reify-recvresult.stdout b/examples/expected/0617-comptime-metatype-channel-results.stdout similarity index 100% rename from examples/expected/0617-comptime-reify-recvresult.stdout rename to examples/expected/0617-comptime-metatype-channel-results.stdout diff --git a/library/modules/std/core.sx b/library/modules/std/core.sx index 72437786..289d41a0 100644 --- a/library/modules/std/core.sx +++ b/library/modules/std/core.sx @@ -31,10 +31,11 @@ field_value_int :: ($T: Type, idx: i64) -> i64 #builtin; field_index :: ($T: Type, val: T) -> i64 #builtin; error_tag_name :: (e: $T) -> string #builtin; -// Comptime type metaprogramming (`type_info` / `reify` / `field_type`) lives in -// the on-demand `modules/std/meta.sx`, NOT here — declaring its data types in -// the always-loaded prelude would intern them into every module's type table -// and shift every `.ir` snapshot. Import `modules/std/meta.sx` to use reify. +// Comptime type metaprogramming (`declare` / `define` / `type_info` / +// `field_type`) lives in the on-demand `modules/std/meta.sx`, NOT here — +// declaring its data types in the always-loaded prelude would intern them into +// every module's type table and shift every `.ir` snapshot. Import +// `modules/std/meta.sx` to construct or reflect types at comptime. // Call-site location, synthesized by the `#caller_location` directive when it // is a parameter's default value (ERR E4.1b). `process.exit` / `assert` use it diff --git a/src/parser.test.zig b/src/parser.test.zig index 75c32c9e..94c14a2c 100644 --- a/src/parser.test.zig +++ b/src/parser.test.zig @@ -6,12 +6,11 @@ const ast = @import("ast.zig"); const Node = ast.Node; const Parser = @import("parser.zig").Parser; -// REIFY Phase 0.0 (lock): the comptime type-metaprogramming surface added to -// `library/modules/std/meta.sx` must PARSE — the data types as struct/enum -// decls, and `reify`/`type_info`/`field_type` as bodyless `#builtin` consts. -// This locks the declared shape before the interpreter-side construction lands -// (Phase 0.2). Mirrors the exact spellings in meta.sx. -test "parser: reify TypeInfo data types + #builtin decls parse" { +// Lock: the comptime type-metaprogramming surface in `library/modules/std/meta.sx` +// must PARSE — the data types as struct/enum decls, and the four comptime builtins +// (`declare` / `define` / `type_info` / `field_type`) as bodyless `#builtin` +// consts. Mirrors the exact spellings in meta.sx. +test "parser: comptime type-metaprogramming surface parses" { var arena = std.heap.ArenaAllocator.init(std.testing.allocator); defer arena.deinit(); const alloc = arena.allocator(); @@ -22,12 +21,14 @@ test "parser: reify TypeInfo data types + #builtin decls parse" { \\ payload: Type; \\} \\EnumInfo :: struct { + \\ name: string; \\ variants: []EnumVariant; \\} \\TypeInfo :: enum { \\ `enum: EnumInfo; \\} - \\reify :: (info: TypeInfo) -> Type #builtin; + \\declare :: () -> Type #builtin; + \\define :: (handle: Type, info: TypeInfo) -> Type #builtin; \\type_info :: ($T: Type) -> TypeInfo #builtin; \\field_type :: ($T: Type, idx: i64) -> Type #builtin; \\ @@ -37,12 +38,12 @@ test "parser: reify TypeInfo data types + #builtin decls parse" { try std.testing.expect(root.data == .root); const decls = root.data.root.decls; - try std.testing.expectEqual(@as(usize, 6), decls.len); + try std.testing.expectEqual(@as(usize, 7), decls.len); const Found = struct { // A top-level `Name :: struct/enum {…}` parses to a `.struct_decl` / // `.enum_decl` node DIRECTLY (not wrapped in a const_decl); only the - // `#builtin` forms are `.const_decl`. Match on the shared `declName`. + // `#builtin` forms are `.fn_decl`. Match on the shared `declName`. fn byName(ds: []const *Node, name: []const u8) ?*const Node { for (ds) |d| { if (d.data.declName()) |n| { @@ -70,7 +71,7 @@ test "parser: reify TypeInfo data types + #builtin decls parse" { // Builtins: the `(params) -> Ret #builtin;` form parses as a `.fn_decl` // (the `->` triggers the function-def path) whose body is a `#builtin` // marker — same shape as the existing reflection builtins in core.sx. - for ([_][]const u8{ "reify", "type_info", "field_type" }) |bn| { + for ([_][]const u8{ "declare", "define", "type_info", "field_type" }) |bn| { const d = Found.byName(decls, bn) orelse return error.MissingDecl; try std.testing.expect(d.data == .fn_decl); try std.testing.expect(d.data.fn_decl.body.data == .builtin_expr);