ae27cffe9d
Record the verified pass-order / define-timing / parse / dispatch findings from F1 investigation, and make explicit that the floor work MUST delete reifyType + the E :: reify decl hook + findReturnReifyCall (reify lives only in meta.sx). Removal can't precede the floor, so they land together; suite never left red across a session boundary.
218 lines
14 KiB
Markdown
218 lines
14 KiB
Markdown
# CHECKPOINT-REIFY — comptime `type_info` / `reify` (async-first foundation, step 3)
|
||
|
||
Companion to [PLAN-REIFY.md](PLAN-REIFY.md). Update after every step (one step at a
|
||
time, per the cadence rule).
|
||
|
||
## Last completed step
|
||
**Phase 3.1 (green) — `RecvResult`/`TryResult` done.** Added `RecvResult($T)` and
|
||
`TryResult($T)` to `meta.sx` as type-fns over `reify` (a value-or-`closed`, and a
|
||
value-or-`empty`-or-`closed` enum). They needed NO new compiler machinery — they're
|
||
reify-of-a-literal in a type-fn body, i.e. exactly the Phase 1 path — so the channel
|
||
result types are pure sx library code. `examples/0617` green (both construct +
|
||
match, incl. payload-less `.closed`/`.empty`). Full suite green (673 examples).
|
||
Cadence: 3.0 xfail (undefined → RED) → 3.1 green.
|
||
|
||
**Phase 3.1 is the last GREEN step.** Phase 3.2 was attempted + reverted; the stream
|
||
now pivots to the `declare`/`define` RE-ARCHITECTURE (see the section below + the
|
||
F1–F5 table in PLAN-REIFY). **`type_info` NOT done** (old Phase 2.2, orthogonal).
|
||
|
||
### (prior) Phase 2.1 (green) — `field_type` done. `field_type($T, i) -> Type` is
|
||
implemented over the type table (`fieldTypeOf` in `lower/generic.zig`, re-exported
|
||
on `Lowering`): struct field / tagged-union + `union` variant payload (`.void` for
|
||
a tagless variant) / tuple element / array + vector element; OOB and memberless
|
||
types poison to `.unresolved` with a loud diagnostic (never a silent default).
|
||
It folds at lower time, so it composes inside `type_eq` / `type_name` / any type-arg
|
||
slot. `examples/0616` green (struct fields name+type, `type_eq` fold, tagged-union
|
||
payloads incl. `quit → void`). Full suite green (672 examples, 447 unit). Cadence:
|
||
2.0 xfail (empty marker, RED) → 2.1 green (this commit).
|
||
|
||
**`type_info` is NOT done** — it still bails loudly in
|
||
`call.zig:tryLowerReflectionCall`. It builds a full `TypeInfo` *value* (inverse of
|
||
reify) and is the larger Phase 2.2 step (see Next step).
|
||
|
||
### (prior) Phase 1 (type-fn → reify identity) — COMPLETE. A type-fn body that returns
|
||
`reify(...)` now mints the enum under the instantiation's name:
|
||
`instantiateTypeFunction` (`lower/generic.zig`) detects a reify-returning body
|
||
(`findReturnReifyCall`) and routes it to `reifyType(<mangled-or-alias name>,
|
||
reify_call)` with the type-arg bindings active (so `payload = T` resolves to the
|
||
bound arg) — placed BEFORE the general case, which would otherwise route the
|
||
reify call to the inline-position loud bail. Registering under the mangled name
|
||
lets the fn's top-of-instantiation cache return the SAME `TypeId` on the second
|
||
instantiation → `Box(i64)` at two independent sites is ONE type (**Contract 1**).
|
||
`examples/0615` green (`build()`→`consume()` cross-site, plus `b : Box(i64) =
|
||
.none`); full suite green (671 examples, 447 unit). Cadence: 1.0 xfail (empty
|
||
marker, RED) → 1.1 green (this commit).
|
||
|
||
### (prior) Phase 0.2 (green) — Phase 0 COMPLETE. Implemented `reify(.enum(...))`:
|
||
`Lowering.reifyType` (in `lower/nominal.zig`) reads the flat-enum `TypeInfo`
|
||
literal off the AST, synthesizes an `ast.EnumDecl`, and feeds it through the
|
||
SAME `type_bridge.buildEnumInfo` path source enums use → the minted type is
|
||
byte-identical to a hand-written `enum { value: i64; closed; }` and flows
|
||
through enum codegen unmodified (**Contract 2 confirmed** — a source enum
|
||
exhibits the exact same construct/match behavior, verified by probe). Wired at
|
||
the `E :: reify(...)` const-decl hook in `lower/decl.zig`. `examples/0614`
|
||
green (`value 3` / `closed`, exit 0); snapshots captured; full suite green
|
||
(670 examples, 447 unit). Unsupported reify shapes bail loudly via
|
||
`reifyBail` (never a silent default).
|
||
|
||
Also landed (user-directed, separate commit `feat(parser):`): reserved keyword
|
||
as a member name after `.` (`.enum`, `case .enum:`, `x.enum`) — so the reify
|
||
example reads `reify(.enum(...))` without a backtick. readme updated.
|
||
|
||
### Two earlier Phase-0 commits
|
||
- **0.1 (xfail).** Added `examples/0614` + empty `.exit` marker → RED.
|
||
- **0.0 (lock).** Added the comptime type-metaprogramming surface as the
|
||
on-demand module `library/modules/std/meta.sx` (NOT the prelude — see decision
|
||
below): `EnumVariant`/`EnumInfo`/`TypeInfo` data types + bodyless `#builtin`
|
||
decls `reify` / `type_info` / `field_type`. Each builtin bails LOUDLY when
|
||
reached unimplemented (no silent default). Unit test `src/parser.test.zig`
|
||
(registered in `src/root.zig`) locks that the decls parse. `zig build test`
|
||
green (447/447 unit, 669/669 examples).
|
||
|
||
## Current state
|
||
- `modules/std/meta.sx` declares the surface; the variant uses the backtick raw
|
||
escape `` `enum `` (reads as the keyword, not a mangled `enum_`).
|
||
- **Loud bails wired (unimplemented → diagnostic, never a silent type):**
|
||
- `reify(...)` in a `::` type-alias position → `decl.zig` (the `.call`
|
||
const-decl branch) emits "reify is not yet implemented (REIFY Phase 0.2)"
|
||
and poisons the alias to `.unresolved`. This is also where Phase 0.2 will
|
||
hook the real construction.
|
||
- `reify` / `field_type` in any other type position →
|
||
`generic.zig:resolveTypeCallWithBindings` (defense-in-depth).
|
||
- `type_info(...)` in expression position →
|
||
`call.zig:tryLowerReflectionCall`.
|
||
- No interpreter-side construction yet — `reify` mints nothing.
|
||
|
||
## Decision (0.0)
|
||
**Meta lives in `modules/std/meta.sx`, not the prelude (`core.sx`).** Declaring
|
||
the data types in the always-loaded prelude interns them into every module's
|
||
type table and shifts every `.ir` snapshot (broke 37 examples in a trial). An
|
||
on-demand import keeps the prelude clean; reify users `#import
|
||
"modules/std/meta.sx"`. (User-directed.)
|
||
|
||
## RE-ARCHITECTURE in progress (user-directed, 2026-06-16)
|
||
**`declare`/`define` are to be the ONLY compiler primitives; `reify` / `make_enum` /
|
||
`RecvResult` / `TryResult` move to sx in `meta.sx`.** `reify` becomes literally
|
||
`{ h := declare(); define(h, info); return h; }`. The AST-walking `reifyType` and all
|
||
syntactic `reify` recognition (`decl.zig` `E :: reify` hook, `generic.zig`
|
||
`findReturnReifyCall`) get DELETED, replaced by generic comptime evaluation of a
|
||
Type-returning expression. Full F1–F5 phase table + the resolved naming/identity
|
||
design is in **PLAN-REIFY.md → "RE-ARCHITECTURE"**.
|
||
|
||
**A Phase 3.2 attempt (make_enum + an eval-decode reader, `reifyEnumFromVariantsValue`
|
||
+ `reifyValueFn`) was implemented this session, then REVERTED** (`git reset --hard
|
||
9306ad5`) per this decision — the reader grew the compiler instead of shrinking it.
|
||
The eval-decode *decode logic* is salvage for `define` (see commit 4d6c463 in reflog
|
||
if needed); the *syntactic routing* is discarded.
|
||
|
||
## Next step
|
||
**F1 — `declare()`/`define()` primitives + a directly-minted RECURSIVE enum.**
|
||
- `declare() -> Type`: intern an empty forward nominal slot (`reserveShadowEnumSlot`
|
||
shape, `nominal.zig:98`), anonymous, return a first-class `Type`.
|
||
- `define(handle, info)`: decode the `TypeInfo` *value* (reuse the reverted decode:
|
||
interp `Value` → names + payload `Type`-tags → `tagged_union` matching
|
||
`buildEnumInfo`), fill via `updatePreservingKey`.
|
||
- `X :: declare()` decl hook + scan-time `define(...)` side-effect (must complete
|
||
before codegen needs the layout — same scan-time window the old `reifyType` used).
|
||
- Example: `List :: declare(); define(List, .enum(.{ … payload = *List … }))` —
|
||
cons/nil list, construct + match. Self-ref works because the handle predates the body.
|
||
Then F2 (reify→sx + general comptime-Type-decl eval, delete `reifyType`), F3 (type-fn
|
||
body eval, delete `findReturnReifyCall`, re-green 0615/0617), F4 (make_enum in sx),
|
||
F5 (validation). `type_info` (old 2.2) stays orthogonal/pending.
|
||
|
||
**END STATE IS NON-NEGOTIABLE: ZERO `reify` in the compiler.** `reifyType`, the
|
||
`E :: reify(...)` decl hook (`decl.zig` ~660), and `findReturnReifyCall` routing
|
||
(`generic.zig`) must ALL be deleted by the end of the floor work — `reify` lives only
|
||
in `meta.sx`. The compiler keeps only `declare`/`define` (+ the generic comptime
|
||
Type-expr evaluation that runs them). Removal can't precede the floor (the reify
|
||
examples would have no path), so the floor + removal land together; the suite must
|
||
never be left red across a session boundary.
|
||
|
||
### F1 investigation findings (verified 2026-06-16 — don't re-derive)
|
||
- **Pass order** (`decl.zig:lowerRoot`): Pass 1 `scanDecls` (line 441 — registers all
|
||
types; the old reify hook mints here) → Pass 1f `UnknownTypeChecker` → Pass 2
|
||
`lowerMainAndComptime` (1320 — force-lowers `main` via `lazyLowerFunction`, which
|
||
transitively lowers bodies; also *creates* `#run` comptime fns). `#run` side-effects
|
||
EXECUTE only at emit time (`emit_llvm.zig:runComptimeSideEffects`).
|
||
- **`define` timing:** must run AFTER scanDecls (needs all types) but BEFORE body
|
||
lowering (match/construct resolve variant indices at lower-time, Pass 2). Neither
|
||
existing hook fits → add a **new Pass 1g** (after `scanDecls`, before
|
||
`lowerMainAndComptime`) that executes recorded `declare`/`define` pairs via the
|
||
interp. Self-ref works because `declare` binds the name in Pass 1, so `*List` in the
|
||
`define` body resolves.
|
||
- **Parse constraint:** a bare top-level `define(List, ...)` does NOT parse
|
||
(`expected '::' / ':=' / ':' after identifier`). So `define` is invoked either via a
|
||
recognized top-level form or `#run define(...)` whose comptime fn Pass 1g executes
|
||
early (instead of waiting for emit). Decide the surface in F1.
|
||
- **Builtin dispatch:** `resolveBuiltin` (`call.zig:1297`) is a tiny name→`BuiltinId`
|
||
table; `reify`/`type_info`/`field_type` are NOT there — they're recognized
|
||
syntactically (decl hook / `tryLowerReflectionCall` at `call.zig:1679`). `declare`/
|
||
`define` need their own dispatch (likely comptime interp builtins, since they mutate
|
||
the type table at comptime).
|
||
- **Salvage:** the reverted eval-decode (`reifyEnumFromVariantsValue` +
|
||
`decodeVariantElements`, commit `4d6c463` in reflog) IS `define`'s decode core.
|
||
|
||
### (superseded) earlier Next step — `type_info` value construction Reflect a type into a `TypeInfo`
|
||
*value* — the inverse of reify. Two sub-pieces, both non-trivial: (a) widen the
|
||
`meta.sx` `TypeInfo` data model beyond `` `enum `` (struct / tuple variants); (b)
|
||
build the value at comptime — a `[]EnumVariant`-style slice of structs holding
|
||
strings (`name`) + `Type` tags (`payload`), populated from the type table. This is
|
||
comptime value-CONSTRUCTION (allocating slice/struct/string/type-tag values in the
|
||
interpreter), materially larger than 2.1's fold-to-a-TypeId. Currently bails loudly
|
||
in `call.zig:tryLowerReflectionCall`. Best done as its own session for context room.
|
||
|
||
NOTE (round-tripping): `reifyType` still reads a LITERAL `TypeInfo` off the AST
|
||
(fine for the inline-literal + type-fn-over-literal cases of Phases 0–1). Once 2.2
|
||
produces a COMPUTED `TypeInfo`, feeding it back into `reify` needs the reader
|
||
generalized (or interp evaluation of the `TypeInfo` value) — handle that when 2.2
|
||
enables round-tripping.
|
||
|
||
Alternatively jump to **Phase 3** (`make_enum` + `RecvResult`/`TryResult` sx lib over
|
||
`reify`) — that only needs reify (have it) + type-fns (have them), not `type_info`.
|
||
|
||
SELF-REFERENCE = Phase 4, API DECIDED (user-directed): explicit **`declare()` →
|
||
`define(h, info)`** (the declaration-vs-definition split; NOT a `reify_rec((self)=>…)`
|
||
closure). `declare()` returns a forward nominal `Type` handle (named from the `::`
|
||
LHS) usable freely in any later `TypeInfo` — `*List`, `[]List`, and across types for
|
||
MUTUAL recursion the one-`self` closure couldn't express; `define(handle, info)` fills
|
||
the body. `reify(info)` stays as the one-shot sugar. Maps onto existing machinery:
|
||
declare = empty nominal slot (`reserveShadowEnumSlot`-style), define = `buildEnumInfo`
|
||
+ `updatePreservingKey`, struct-stub→tagged_union re-key already handled
|
||
(`internNamedTypeDecl`'s `adoptsForwardStructStub`). Loud invariants: never-defined
|
||
declare = hard error; by-value self-inclusion rejected (infinite size). Full write-up
|
||
in PLAN-REIFY Phase 4.
|
||
|
||
## Known issues
|
||
None yet.
|
||
|
||
## Log
|
||
- **3.1 (green) — `RecvResult`/`TryResult` done.** Added as type-fns over `reify`
|
||
in `meta.sx` (no new machinery — Phase 1 reify-of-literal-in-type-fn); `0617`
|
||
green. `make_enum` (3.2) pending on the generalized reify reader.
|
||
- **3.0 (xfail).** `0617` + empty marker → RED (RecvResult undefined).
|
||
- **2.1 (green) — `field_type` done.** `fieldTypeOf` over the type table
|
||
(struct/tagged-union/union/tuple/array/vector; OOB+memberless = loud poison);
|
||
folds at lower time, composes in `type_eq`/`type_name`; `0616` green. `type_info`
|
||
still pending (Phase 2.2 — builds a TypeInfo value).
|
||
- **2.0 (xfail).** `0616` + empty `.exit` marker → RED (field_type bailed).
|
||
- **1.1 (green) — Phase 1 done.** Type-fn body `return reify(...)` routes through
|
||
`reifyType` under the instantiation name (`findReturnReifyCall` +
|
||
mangled-name registration); `Box(i64)` at two sites = one type (Contract 1);
|
||
`0615` green.
|
||
- **1.0 (xfail).** `0615` + empty `.exit` marker → RED (reify bailed in type-fn body).
|
||
- **0.2 (green) — Phase 0 done.** `reifyType` mints a flat enum from a literal
|
||
`TypeInfo` via the shared `buildEnumInfo` path; `0614` green; Contract 2 confirmed
|
||
(reify'd enum == source enum, same construct/match). Payload-less variant idiom in
|
||
the example is `c : E = .closed;` (enum-literal target), same as a source enum.
|
||
- **parser (user-directed).** Keyword as member name after `.` — see `feat(parser)`.
|
||
- **0.1 (xfail).** `0614` + empty `.exit` marker → RED.
|
||
- **0.0 (lock).** Meta surface in `modules/std/meta.sx` (data types + 3 bodyless
|
||
`#builtin` decls), loud bails at all three reach points, `src/parser.test.zig`
|
||
parse-lock. Two user-directed refinements folded in: variant uses `` `enum ``
|
||
raw escape; surface moved out of the prelude into its own module to avoid
|
||
type-table / `.ir`-snapshot churn. `zig build test` green.
|
||
- **Stream carved.** Selected as the first async-first foundation: `reify` gates both
|
||
channel result types (`RecvResult($T)`) and `race`'s synthesized union, is fully
|
||
validated (3 reviewers), and is a self-contained compiler/type-system feature
|
||
testable in isolation (`06xx` comptime). Generic-enum syntax dropped in its favor.
|