lab/sx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fibers: end-to-end M:1 capstone (B1.5) — Stream B1 complete

Browse Source 
1817 composes the whole colorblind pure-sx async stack: the M:1
scheduler, suspending go/wait async, and deterministic virtual-time
sleep/now_ms, over the naked swap_context on guarded mmap stacks. A
coordinator launches three async tasks (sleep 30/10/20 -> return
100/20/3), awaits all three in spawn order, and sums them; tasks
complete in DEADLINE order (task 2@10, 3@20, 1@30), sum 123, final
virtual clock 30 -- fully deterministic.

Stream B1 (fibers + Io + M:1 scheduler) is feature-complete: examples
1800-1817, suite 755/0. Checkpoint + plan marked COMPLETE; next carve
is Stream B2 (channels / cancel / async stdlib).
This commit is contained in:
agra
2026-06-21 19:43:22 +03:00
parent 1b0d640f73
commit 5949a88439
7 changed files with 149 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

83
current/CHECKPOINT-FIBERS.md
View File

@@ -4,7 +4,28 @@ Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step
per the cadence rule). New corpus category: `18xx` concurrency.
## Last completed step
**B1.4c — REAL fd-readiness blocking via kqueue (macOS).** `library/modules/std/sched.sx` now lets a
**B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE.** A single capstone exercises the whole
colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async
`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)`
`swap_context` on guarded `mmap` stacks (B1.0–B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`:
a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in
SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in
DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual
clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**.
**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the
`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler,
suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via
kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx`
(1800–1817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1,
zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a;
0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`),
0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`;
routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env
/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve.
### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS)
`library/modules/std/sched.sx` now lets a
fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready.
Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes),
`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT`
@@ -339,25 +360,25 @@ body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positi
(unparseable — `isLambda` breaks on the `abi` keyword).
## Current state
**B1.4c COMPLETE — real fd-readiness blocking via kqueue (macOS) exists.** `library/modules/std/sched.sx`
now carries: the M:1 scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), the
suspending fiber-task async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic timers (B1.4b:
`clock_ms` virtual clock, `timers` list, `now_ms`/`sleep`, timer-driven `run`), AND real fd readiness
(B1.4c: lazy `kq`, `io_waiters` list, `block_on_fd`, a kqueue-blocking run-loop Mode 2 that wakes the
fiber whose fd fired). It reuses the verified `std/net/kqueue.sx` bindings (imported as `kqb`) rather
than re-deriving the FFI. Fibers can now block on either virtual `sleep(ms)` OR a real fd; both park
paths are balanced through `wake` (which evicts a stale timer AND a stale fd-waiter, the UAF guard).
Locked by `1811` (round-robin), `1812` (suspend/wake), `1813` (async go/wait/cancel), `1814` (sim-timer
deadline ordering), `1815` (timer early-wake eviction), `1816` (pipe fd block→kqueue-wake→read). Suite
GREEN **754/0**.
**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime:
the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task
async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b:
`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`,
`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded
`mmap` stacks (B1.0–B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw
suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). Locked
by `18xx` 1800–1817 (naked-asm, context-snapshot, blocking async, the switch + §10.7 stress gate +
guarded stacks + Win64 sibling, scheduler round-robin, suspend/wake, async go/wait/cancel, sim-timer
ordering, timer early-wake eviction, kqueue pipe I/O, and the **1817 end-to-end capstone**). Suite
GREEN **755/0**, master committed.
The remaining B1 work: **B1.5** end-to-end M:1 validation under the deterministic timers / fd readiness;
a **linux epoll twin** of `block_on_fd` (mirror via `std/net/epoll`; the OS-neutral facade is
`std.event`) is future work — B1.4c wired the **macOS kqueue** path only. NOTE: the suspending async +
deterministic timers live as `sched.*` methods (M:1, receiver-driven), NOT routed through the erased
`context.io` (which would force sched.sx into every std consumer + duplicate the `_fib_tramp` global
asm); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/`arm_timer`/`poll` remain reserved for the
M:N evolution / when a program wants the capability-threaded form.
Future work (none blocking B1): a **linux epoll twin** of `block_on_fd` (mirror via `std/net/epoll`;
OS-neutral facade `std.event`) — B1.4c wired macOS kqueue only; routing the suspending async through
the erased `context.io` (forces sched.sx into every std consumer + duplicates the `_fib_tramp` global
asm — deferred to the M:N model, where the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/
`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task /
closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream
B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started.
### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists
`library/modules/std/sched.sx` drives N fibers
@@ -449,12 +470,12 @@ fibers/Io/scheduler code yet. Grounded floor facts:
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
## Next step
**→ B1.5 — end-to-end M:1 validation under the deterministic timers / fd readiness.** B1.4c (real
fd-readiness blocking via kqueue, `sched.block_on_fd` + the kqueue-blocking run-loop Mode 2) is done —
the macOS event-loop path exists. Build an `18xx` example that exercises the full M:1 story together
(multiple fibers, a mix of `sleep`/`go`/`wait` and `block_on_fd`, reaping, the orphan-deadlock guard).
The §10.7 gate (1808) + guarded-stack (1809) + Win64 (1810) + scheduler/async/timers/fd
(1811-1816) must keep passing throughout.
**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature-
complete and committed (1800–1817 green, 755/0). Pick up **Stream B2** (channels / structured cancel /
async stdlib) as a fresh carve (PLAN-CHANNELS.md), OR one of the documented non-blocking follow-ups:
the linux `epoll` twin of `block_on_fd`, a `Scheduler.deinit` (free the kq fd / heap Tasks / drain
leaks), `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the
erased `context.io` for the M:N model. None of these block B1.
**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS
kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via
@@ -782,3 +803,15 @@ incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended
accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination.
Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll).
- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a
lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx`
(worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd
lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a
never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment
corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0.
- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack
(`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order
(task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async
runtime is feature-complete end-to-end (1800–1817), all adversarially reviewed. Suite GREEN
**755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in
B1.2). Next carve: Stream B2 (channels / cancel / async stdlib).

20
current/PLAN-FIBERS.md
View File

@@ -1,6 +1,14 @@
# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
> **STATUS: 🚧 in progress.** B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
> **STATUS: ✅ COMPLETE.** The pure-sx M:1 async runtime is feature-complete end-to-end
> (`library/modules/std/sched.sx`, examples 1800–1817, suite 755/0): `abi(.naked)` context switch
> (aarch64 + x86_64/Win64), M:1 scheduler, suspending `go`/`wait`/`cancel`, deterministic virtual-time
> timers (`sleep`/`now_ms`), and real fd readiness via kqueue (`block_on_fd`). Five compiler bugs fixed
> en route (0151/0152/0153/0154/0156-P1/0157). Deferred non-blocking follow-ups: linux `epoll` twin,
> `Scheduler.deinit`, `Future(void)`/`timeout` (0150), `context.io`-routed async (M:N). Next carve:
> Stream B2 (channels / cancel / async stdlib). Historical step-status below.
>
> B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
> (`Io` interface + `async`/`await`/`cancel` over blocking `CBlockingIo`) ✅ · B1.3 (fiber
> runtime: naked `swap_context` + §10.7 stress gate + guarded `mmap` stacks, proven on aarch64
> AND x86_64/Win64) ✅ · **B1.5a (M:1 scheduler CORE — `std/sched.sx`: `spawn`/`yield_now`/
@@ -9,7 +17,7 @@
> blockers 0156-Part1 + 0157 en route; locked `1813`).
> **B1.4b (deterministic virtual-time timers — sched.sleep/now_ms/timer-run) ✅** (reviewed; fixed a CRITICAL timer-vs-early-wake UAF; locked 1814/1815).
> **B1.4c (event-loop — real fd readiness via kqueue: `block_on_fd` + run-loop Mode 2) ✅** (reviewed; fixed a CRITICAL same-fd lost-wakeup hang; locked 1816). macOS only — linux epoll twin deferred.
> **→ NOW: B1.5** — end-to-end M:1 validation under the deterministic timers / fd readiness. Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
> **B1.5 (end-to-end M:1 capstone — `go`/`wait`+`sleep`+scheduler, deterministic ordering) ✅** (locked 1817). **STREAM B1 COMPLETE.** Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
> deterministic timers live as `sched.*` methods (M:1), NOT routed through the erased `context.io` (avoids forcing sched.sx into every std consumer + the `_fib_tramp` dup-symbol
> trap); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready` stay reserved for M:N. Deferred:
> issue 0150 (`Future(void)`/`timeout`); 0156-Part2 (deferred `..` spread); the `::` callable-param
@@ -223,9 +231,11 @@ Blocking exists (io.sx `CBlockingIo`). Next the deterministic-sim `Io`, **calibr
blocking** before any `18xx` test trusts it; then the event loop. The deterministic `Io` is the
test harness for *all* of B1.5 + Stream B2.
### B1.5 — A5: M:1 scheduler
End-to-end validation of the colorblind stack. `18xx` corpus under the deterministic `Io`,
asserting program-emitted ordering contracts.
### B1.5 — A5: M:1 scheduler — ✅ COMPLETE
End-to-end validation of the colorblind stack. The `18xx` corpus asserts program-emitted ordering
contracts under the scheduler + deterministic timers; the capstone `1817` composes `go`/`wait` +
`sleep`/`now_ms` + the scheduler (three tasks complete in deadline order, deterministic sum). Stream
B1 is feature-complete.
---