3328d3fe52
Phase 5 of PLAN-IO-UNIFY retired the bespoke fiber-task API (Task/go/wait/cancel) and re-homed race onto *Future. Prepend SUPERSEDED banners to the three historical planning docs (PLAN-FIBERS, CHECKPOINT-FIBERS, PLAN-RACE) pointing readers to PLAN-IO-UNIFY for the current design, noting that the Scheduler engine primitives + the race type-machinery they document remain accurate. Bodies kept as the historical record. Completes the plan's 'update roadmap/checkpoints' item. Verified by parallel agents: a completeness pass (no retired API in compilable code; sched.sx surface fully retired; migrated examples meaningful; readme on the unified API) and an adversarial memory-safety pass over the heap reclamation (no UAF/double-free; await/worker orderings, cancel timings, 8-way double-free+deinit, blocking impl all reclaim cleanly with exact values).
146 lines
10 KiB
Markdown
146 lines
10 KiB
Markdown
> **SUPERSEDED (2026-06-28).** The `race` LOGIC described here shipped, then was
|
||
> RE-HOMED onto `*Future` + the `Io` protocol as `context.io.race` in PLAN-IO-UNIFY
|
||
> Phase 4. The Task-based `sched.race` over `*Task` documented below is RETIRED. See
|
||
> `current/PLAN-IO-UNIFY.md` (`## Status (2026-06-28)`) for the current design. The
|
||
> type-level machinery this doc documents (`RaceResult` / `make_variant` / tuple
|
||
> reflection) is UNCHANGED and still in use. Body below is a historical record.
|
||
|
||
# PLAN-RACE — Stream B2/A1: `race` over the M:1 fiber scheduler
|
||
|
||
Carved from the async roadmap ([../design/execution-evolution-roadmap.md](../design/execution-evolution-roadmap.md)
|
||
§4.5, §4.6, §7 step 3). The headline A1 feature still missing after Stream B1: **`race`** — start N
|
||
async tasks, return when the FIRST completes, and **structurally** cancel + join the losers before
|
||
returning. The result is a synthesized tagged-union mirroring the input named tuple's labels.
|
||
|
||
```sx
|
||
fa := s.go(() -> A => read_a(conn)); // *Task(A)
|
||
fb := s.go(() -> B => read_b(conn)); // *Task(B)
|
||
winner := s.race((a: fa, b: fb)); // RaceResult = enum { a: A; b: B }
|
||
if winner == {
|
||
case .a: (v) { handle_a(v); } // v : A (fb cancelled + joined)
|
||
case .b: (v) { handle_b(v); } // v : B (fa cancelled + joined)
|
||
}
|
||
// positional form: s.race((fa, fb)) → tags ._0 / ._1
|
||
```
|
||
|
||
## Design decisions (grounded against the tree, 2026-06-26)
|
||
|
||
- **Built over the M:1 `Task` layer, NOT `context.io`/`Future`.** The suspending async is
|
||
`sched.go`/`wait`/`cancel` over `*Task($R)` (B1.4a); `context.io.async`→`Future` is the BLOCKING
|
||
impl (workers run inline → racing is meaningless there). The roadmap's "Future" maps to our
|
||
`*Task`. `race` is a `Scheduler`/`Task` UFCS function in `library/modules/std/sched.sx`.
|
||
- **The result is a comptime-synthesized nominal tagged-union** (`RaceResult`), one variant per input
|
||
tuple element: variant NAME = the tuple label (positional → `_0`/`_1`), payload = the task's result
|
||
type. Synthesis uses the proven `declare`/`define`/`make_enum` + `field_count`/`field_name`/
|
||
`field_type` reflection (examples 0619–0623, 0646). The input arrives as an inferred type PARAMETER
|
||
`$T` (a named tuple of `*Task(..)`), which reflects correctly (issue 0195 fixed; the tuple-*alias*
|
||
gap is issue 0196, NOT on this path).
|
||
- **Cancellation rides the existing cooperative `Task.cancel`** (sets the flag + `.canceled` state).
|
||
`race` cancels every loser, then `wait`s each (joins) so no loser fiber outlives the `race` call —
|
||
structured. Reuses `suspend_self`/`wake`; no new scheduler machinery.
|
||
|
||
## The one net-new compiler primitive (step 1)
|
||
|
||
**`pointee($P: Type) -> Type #builtin`** — given a pointer type `*X`, return `X`. This is the only
|
||
missing reflection capability: `race` must project each tuple element `*Task(A)` to its result type
|
||
`A`, and there is currently NO way to get a pointer's target type at comptime (`field_count(*X)`=0,
|
||
`type_info` has no pointer variant). With it the projection is pure sx:
|
||
```sx
|
||
TaskResult :: ($P: Type) -> Type { // P = *Task(A) → A
|
||
return field_type(pointee(P), 0); // pointee → Task(A); field 0 = `value: A`
|
||
}
|
||
```
|
||
Small + generally useful (reflection is currently complete for aggregates but blind to pointer
|
||
targets). Mirror `field_type`'s `#builtin` plumbing (`src/ir/lower/call.zig` + `src/ir/calls.zig`),
|
||
backed by the pointer TypeInfo's pointee TypeId (`src/ir/types.zig`). Lock with a comptime example.
|
||
|
||
## Steps (each: implement → lock with an example → `zig build test` green → both platforms)
|
||
|
||
1. **`pointee` reflection builtin.** Add `pointee($P: Type) -> Type` (core.sx + compiler). Example:
|
||
`pointee(*i64)` = `i64`, `field_type(pointee(*Task(i64)), 0)` = the task value type. (worker+review)
|
||
2. **`RaceResult($T) -> Type` synthesis.** Type-fn: reflect the named-tuple `$T` of `*Task(..)`,
|
||
project each element via `TaskResult`, mint the tagged-union (labels → variants). Comptime-only
|
||
example asserting the minted type's `field_count`/`field_name` match the input tuple.
|
||
3. **`Task.Value` projection + result construction.** Confirm a winner's value can be boxed into the
|
||
minted variant by label/index (uses the existing variant-construction path).
|
||
4. **Runtime `race(tasks: $T) -> RaceResult(T)`.** Suspend the caller until the first task is
|
||
`.ready`; build the winner variant; then cancel + `wait`-join every loser before returning.
|
||
Single-winner (first by completion order; FIFO tiebreak). Example: 2 tasks, deterministic winner
|
||
via `sleep` ordering (like 1817), asserting the loser is cancelled + joined.
|
||
5. **Positional tuple form** (`._0`/`._1`) + edge cases (already-ready task → immediate, single-task
|
||
race, all-cancelled). Examples.
|
||
6. **Validate** every new example byte-identical on aarch64-macOS host AND aarch64-linux container;
|
||
full `zig build test` green; adversarially review each step.
|
||
|
||
## Status
|
||
|
||
Prereqs DONE (each committed + adversarially reviewed + suite-green):
|
||
- **issue 0195** (tuple/array/vector field reflection) fixed (`8ac6c573`). Tuple reflection works on
|
||
inline + `$T`-param forms. Issue 0196 (tuple *alias*) filed, not on the critical path.
|
||
- **`pointee($P) -> Type`** builtin added (`f1d29876`) — projects `*Task(A)` → `Task(A)`.
|
||
- **`field_count`/`size_of`/`align_of` fold as comptime constants** (`2a6ef398`) — so a generic
|
||
`($T) -> Type` builder can `inline for 0..field_count(T)` and size `[field_count(T)]EnumVariant`.
|
||
Verified: the variable-arity loop + array dim now work inside `RaceResult`.
|
||
|
||
- **comptime type-call composition** fixed (`eb18bbc6`) — a `field_type(...)`/`pointee(...)` result is
|
||
now usable as a `Type`-typed struct-field value, a generic `$P: Type` arg, and a nested type-call
|
||
arg (incl. with an `inline for` loop-var index). The **variable-arity `RaceResult` synthesis works
|
||
end-to-end** (proven by `examples/comptime/0649-comptime-typecall-composition.sx`: reflect a named
|
||
tuple of `*Box(..)` handles → mint a tagged-union with the tuple's labels, projecting `*Box(A)`→`A`).
|
||
|
||
- **`return` inside `inline if` fixed** (`84c2ae4f`) — the natural early-return-per-arm pattern (a
|
||
`return` in an `inline if`/comptime-`case` branch inside an `inline for`) no longer drops the
|
||
function's trailing statements. Lets `race` build the winner variant with a clean
|
||
`inline if i == { case 0: … else: … }` per-arm form.
|
||
- **`make_variant($E, idx, payload)`** added to `modules/std/meta.sx` (`1c26944e`) — the WRITE side of
|
||
the metatype triad: construct a minted tagged-union value by variant INDEX (the winner is chosen at
|
||
runtime; its label can't be a literal). Pure sx (writes the i64 tag@0 + payload@8). Verified for
|
||
complex payloads (struct / string / 40-byte struct). **This resolves the variant-construction gap.**
|
||
|
||
**GAP 1 — comptime-cursor indexing of a named-tuple VALUE — DONE** (`fee86adf`). `tasks[i]` with a
|
||
comptime cursor now reads the i-th element with its concrete type (option (a), a `structGet`).
|
||
|
||
**GAP 2 — surfaced during the runtime, all DONE** (`6a976287`): three compiler enablers the runtime
|
||
needed beyond the read path. (1) a named-tuple LITERAL passed directly as `$T` lost its element names
|
||
(`field_name` → ""), breaking `RaceResult`'s `make_enum`; (2) tuple-element L-VALUES by comptime index
|
||
(`tasks[i].waiter = …`) panicked at LLVM emit (an `index_gep` with `ptrTo(.unresolved)`); (3) a user
|
||
`($X) -> Type` call couldn't bind a `$E: Type` arg, blocking `make_variant(RaceResult(T), …)`. All
|
||
fixed + adversarially reviewed; OOB comptime tuple indices now diagnose loudly on every L-value path.
|
||
|
||
**Runtime in `sched.sx` — DONE** (`9099735e`).
|
||
- `RaceResult :: ($T) -> Type` over `*Task(..)` (the 0649 shape, with `Task`).
|
||
- `race :: ufcs (self: *Scheduler, tasks: $T) -> RaceResult(T)`: Phase 1 suspend until the FIRST
|
||
`.ready` (register waiter on all pending; on wake DEREGISTER from all; lowest-index winner). Phase 2
|
||
build the winner with `make_variant`. Phase 3 cancel + JOIN each loser one-at-a-time (only the joined
|
||
loser carries a waiter → no mid-join double-wake). Join rides a new `Task.finished` flag (set at the
|
||
end of the `go` body, checked before parking). Cooperative-cancel: a loser parked mid-`sleep` runs to
|
||
its natural end before `race` returns.
|
||
- Locked by `examples/concurrency/1821-concurrency-fiber-race.sx` (3 tasks i64/bool/f64, sleep 10/20/30,
|
||
shortest wins, losers cancelled + joined). Byte-identical on aarch64-macOS host AND aarch64-linux
|
||
container; full `zig build test` green (826/0).
|
||
|
||
**Remaining (step 5, future work):**
|
||
- POSITIONAL tuple form (`._0`/`._1`): `field_name` yields "" for an unnamed element → `make_enum`
|
||
rejects the duplicate. Needs a `_N` fallback in `RaceResult` (a comptime int→string) or a compiler
|
||
`field_name` default for positional tuples.
|
||
- Bare named-arg call form `s.race(a = ta, b = tb)` (no `.(…)` wrapper) — see the note below.
|
||
- Edge cases: already-ready-at-call (works today), all-cancelled (would deadlock-abort loudly).
|
||
- By-design caveat to document: `race` returns only when the SLOWEST loser finishes (cooperative
|
||
cancel can't preempt a mid-`sleep` loser; a loser blocked on `block_on_fd` that never fires blocks
|
||
the join forever — `cancel` doesn't unblock an fd waiter).
|
||
|
||
### Future work — bare named-arg call form `s.race(a = ta, b = tb)`
|
||
Today the result-tuple is passed as a named-tuple LITERAL: `s.race(.(a = ta, b = tb))`. The user asked
|
||
about dropping the `.(…)` so it reads `s.race(a = ta, b = tb)`. That is a CALL-SITE syntax/binding
|
||
feature, independent of the race runtime:
|
||
- The parser would have to accept `name = expr` call arguments and, for a `(tasks: $T)` parameter whose
|
||
type is inferred, MATERIALIZE them into a single named-tuple value bound to `$T` (rather than treating
|
||
each `name = expr` as a separate positional/keyword arg). Effectively "collect trailing `k = v` call
|
||
args into one anonymous named-tuple when the callee has a single inferred aggregate param".
|
||
- Risk: `name = expr` in call position currently has no meaning (or would collide with a future
|
||
keyword-argument feature). Decide whether `k = v` call args are (a) always tuple-materialized for an
|
||
inferred aggregate param, or (b) a dedicated `..` / sugar. Option (a) is the least new syntax.
|
||
- Once the call site produces the same named-tuple value, the entire race runtime is unchanged (it
|
||
already reflects `$T`). So this is purely front-end sugar — schedule it with the keyword-args work,
|
||
not the concurrency stream.
|