fibers: end-to-end M:1 capstone (B1.5) — Stream B1 complete
1817 composes the whole colorblind pure-sx async stack: the M:1 scheduler, suspending go/wait async, and deterministic virtual-time sleep/now_ms, over the naked swap_context on guarded mmap stacks. A coordinator launches three async tasks (sleep 30/10/20 -> return 100/20/3), awaits all three in spawn order, and sums them; tasks complete in DEADLINE order (task 2@10, 3@20, 1@30), sum 123, final virtual clock 30 -- fully deterministic. Stream B1 (fibers + Io + M:1 scheduler) is feature-complete: examples 1800-1817, suite 755/0. Checkpoint + plan marked COMPLETE; next carve is Stream B2 (channels / cancel / async stdlib).
This commit is contained in:
@@ -4,7 +4,28 @@ Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step
|
||||
per the cadence rule). New corpus category: `18xx` concurrency.
|
||||
|
||||
## Last completed step
|
||||
**B1.4c — REAL fd-readiness blocking via kqueue (macOS).** `library/modules/std/sched.sx` now lets a
|
||||
**B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE.** A single capstone exercises the whole
|
||||
colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async
|
||||
`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)`
|
||||
`swap_context` on guarded `mmap` stacks (B1.0–B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`:
|
||||
a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in
|
||||
SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in
|
||||
DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual
|
||||
clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**.
|
||||
|
||||
**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the
|
||||
`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler,
|
||||
suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via
|
||||
kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx`
|
||||
(1800–1817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1,
|
||||
zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a;
|
||||
0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`),
|
||||
0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`;
|
||||
routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env
|
||||
/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve.
|
||||
|
||||
### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS)
|
||||
`library/modules/std/sched.sx` now lets a
|
||||
fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready.
|
||||
Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes),
|
||||
`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT`
|
||||
@@ -339,25 +360,25 @@ body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positi
|
||||
(unparseable — `isLambda` breaks on the `abi` keyword).
|
||||
|
||||
## Current state
|
||||
**B1.4c COMPLETE — real fd-readiness blocking via kqueue (macOS) exists.** `library/modules/std/sched.sx`
|
||||
now carries: the M:1 scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), the
|
||||
suspending fiber-task async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic timers (B1.4b:
|
||||
`clock_ms` virtual clock, `timers` list, `now_ms`/`sleep`, timer-driven `run`), AND real fd readiness
|
||||
(B1.4c: lazy `kq`, `io_waiters` list, `block_on_fd`, a kqueue-blocking run-loop Mode 2 that wakes the
|
||||
fiber whose fd fired). It reuses the verified `std/net/kqueue.sx` bindings (imported as `kqb`) rather
|
||||
than re-deriving the FFI. Fibers can now block on either virtual `sleep(ms)` OR a real fd; both park
|
||||
paths are balanced through `wake` (which evicts a stale timer AND a stale fd-waiter, the UAF guard).
|
||||
Locked by `1811` (round-robin), `1812` (suspend/wake), `1813` (async go/wait/cancel), `1814` (sim-timer
|
||||
deadline ordering), `1815` (timer early-wake eviction), `1816` (pipe fd block→kqueue-wake→read). Suite
|
||||
GREEN **754/0**.
|
||||
**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime:
|
||||
the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task
|
||||
async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b:
|
||||
`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`,
|
||||
`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded
|
||||
`mmap` stacks (B1.0–B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw
|
||||
suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). Locked
|
||||
by `18xx` 1800–1817 (naked-asm, context-snapshot, blocking async, the switch + §10.7 stress gate +
|
||||
guarded stacks + Win64 sibling, scheduler round-robin, suspend/wake, async go/wait/cancel, sim-timer
|
||||
ordering, timer early-wake eviction, kqueue pipe I/O, and the **1817 end-to-end capstone**). Suite
|
||||
GREEN **755/0**, master committed.
|
||||
|
||||
The remaining B1 work: **B1.5** end-to-end M:1 validation under the deterministic timers / fd readiness;
|
||||
a **linux epoll twin** of `block_on_fd` (mirror via `std/net/epoll`; the OS-neutral facade is
|
||||
`std.event`) is future work — B1.4c wired the **macOS kqueue** path only. NOTE: the suspending async +
|
||||
deterministic timers live as `sched.*` methods (M:1, receiver-driven), NOT routed through the erased
|
||||
`context.io` (which would force sched.sx into every std consumer + duplicate the `_fib_tramp` global
|
||||
asm); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/`arm_timer`/`poll` remain reserved for the
|
||||
M:N evolution / when a program wants the capability-threaded form.
|
||||
Future work (none blocking B1): a **linux epoll twin** of `block_on_fd` (mirror via `std/net/epoll`;
|
||||
OS-neutral facade `std.event`) — B1.4c wired macOS kqueue only; routing the suspending async through
|
||||
the erased `context.io` (forces sched.sx into every std consumer + duplicates the `_fib_tramp` global
|
||||
asm — deferred to the M:N model, where the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/
|
||||
`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task /
|
||||
closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream
|
||||
B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started.
|
||||
|
||||
### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists
|
||||
`library/modules/std/sched.sx` drives N fibers
|
||||
@@ -449,12 +470,12 @@ fibers/Io/scheduler code yet. Grounded floor facts:
|
||||
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
|
||||
|
||||
## Next step
|
||||
**→ B1.5 — end-to-end M:1 validation under the deterministic timers / fd readiness.** B1.4c (real
|
||||
fd-readiness blocking via kqueue, `sched.block_on_fd` + the kqueue-blocking run-loop Mode 2) is done —
|
||||
the macOS event-loop path exists. Build an `18xx` example that exercises the full M:1 story together
|
||||
(multiple fibers, a mix of `sleep`/`go`/`wait` and `block_on_fd`, reaping, the orphan-deadlock guard).
|
||||
The §10.7 gate (1808) + guarded-stack (1809) + Win64 (1810) + scheduler/async/timers/fd
|
||||
(1811-1816) must keep passing throughout.
|
||||
**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature-
|
||||
complete and committed (1800–1817 green, 755/0). Pick up **Stream B2** (channels / structured cancel /
|
||||
async stdlib) as a fresh carve (PLAN-CHANNELS.md), OR one of the documented non-blocking follow-ups:
|
||||
the linux `epoll` twin of `block_on_fd`, a `Scheduler.deinit` (free the kq fd / heap Tasks / drain
|
||||
leaks), `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the
|
||||
erased `context.io` for the M:N model. None of these block B1.
|
||||
|
||||
**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS
|
||||
kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via
|
||||
@@ -782,3 +803,15 @@ incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
|
||||
regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended
|
||||
accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination.
|
||||
Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll).
|
||||
- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a
|
||||
lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx`
|
||||
(worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd
|
||||
lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a
|
||||
never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment
|
||||
corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0.
|
||||
- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack
|
||||
(`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order
|
||||
(task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async
|
||||
runtime is feature-complete end-to-end (1800–1817), all adversarially reviewed. Suite GREEN
|
||||
**755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in
|
||||
B1.2). Next carve: Stream B2 (channels / cancel / async stdlib).
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
# PLAN-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
|
||||
|
||||
> **STATUS: 🚧 in progress.** B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
|
||||
> **STATUS: ✅ COMPLETE.** The pure-sx M:1 async runtime is feature-complete end-to-end
|
||||
> (`library/modules/std/sched.sx`, examples 1800–1817, suite 755/0): `abi(.naked)` context switch
|
||||
> (aarch64 + x86_64/Win64), M:1 scheduler, suspending `go`/`wait`/`cancel`, deterministic virtual-time
|
||||
> timers (`sleep`/`now_ms`), and real fd readiness via kqueue (`block_on_fd`). Five compiler bugs fixed
|
||||
> en route (0151/0152/0153/0154/0156-P1/0157). Deferred non-blocking follow-ups: linux `epoll` twin,
|
||||
> `Scheduler.deinit`, `Future(void)`/`timeout` (0150), `context.io`-routed async (M:N). Next carve:
|
||||
> Stream B2 (channels / cancel / async stdlib). Historical step-status below.
|
||||
>
|
||||
> B1.0 (`abi(.naked)`) ✅ · B1.1 (per-fiber `context`) ✅ · B1.2
|
||||
> (`Io` interface + `async`/`await`/`cancel` over blocking `CBlockingIo`) ✅ · B1.3 (fiber
|
||||
> runtime: naked `swap_context` + §10.7 stress gate + guarded `mmap` stacks, proven on aarch64
|
||||
> AND x86_64/Win64) ✅ · **B1.5a (M:1 scheduler CORE — `std/sched.sx`: `spawn`/`yield_now`/
|
||||
@@ -9,7 +17,7 @@
|
||||
> blockers 0156-Part1 + 0157 en route; locked `1813`).
|
||||
> **B1.4b (deterministic virtual-time timers — sched.sleep/now_ms/timer-run) ✅** (reviewed; fixed a CRITICAL timer-vs-early-wake UAF; locked 1814/1815).
|
||||
> **B1.4c (event-loop — real fd readiness via kqueue: `block_on_fd` + run-loop Mode 2) ✅** (reviewed; fixed a CRITICAL same-fd lost-wakeup hang; locked 1816). macOS only — linux epoll twin deferred.
|
||||
> **→ NOW: B1.5** — end-to-end M:1 validation under the deterministic timers / fd readiness. Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
|
||||
> **B1.5 (end-to-end M:1 capstone — `go`/`wait`+`sleep`+scheduler, deterministic ordering) ✅** (locked 1817). **STREAM B1 COMPLETE.** Detailed progress in [CHECKPOINT-FIBERS.md](CHECKPOINT-FIBERS.md). NOTE: suspending async +
|
||||
> deterministic timers live as `sched.*` methods (M:1), NOT routed through the erased `context.io` (avoids forcing sched.sx into every std consumer + the `_fib_tramp` dup-symbol
|
||||
> trap); the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready` stay reserved for M:N. Deferred:
|
||||
> issue 0150 (`Future(void)`/`timeout`); 0156-Part2 (deferred `..` spread); the `::` callable-param
|
||||
@@ -223,9 +231,11 @@ Blocking exists (io.sx `CBlockingIo`). Next the deterministic-sim `Io`, **calibr
|
||||
blocking** before any `18xx` test trusts it; then the event loop. The deterministic `Io` is the
|
||||
test harness for *all* of B1.5 + Stream B2.
|
||||
|
||||
### B1.5 — A5: M:1 scheduler
|
||||
End-to-end validation of the colorblind stack. `18xx` corpus under the deterministic `Io`,
|
||||
asserting program-emitted ordering contracts.
|
||||
### B1.5 — A5: M:1 scheduler — ✅ COMPLETE
|
||||
End-to-end validation of the colorblind stack. The `18xx` corpus asserts program-emitted ordering
|
||||
contracts under the scheduler + deterministic timers; the capstone `1817` composes `go`/`wait` +
|
||||
`sleep`/`now_ms` + the scheduler (three tasks complete in deadline order, deterministic sum). Stream
|
||||
B1 is feature-complete.
|
||||
|
||||
---
|
||||
|
||||
|
||||
66
examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx
Normal file
66
examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx
Normal file
@@ -0,0 +1,66 @@
|
||||
// Stream B1 (fibers) B1.5 — the M:1 colorblind async stack, end-to-end.
|
||||
//
|
||||
// One program exercises the whole pure-sx runtime together: the M:1 scheduler
|
||||
// (B1.5a), the suspending fiber-task async layer `go`/`wait` (B1.4a), and the
|
||||
// deterministic virtual-time timers `sleep`/`now_ms` (B1.4b) — all over the
|
||||
// `abi(.naked)` `swap_context` on guarded `mmap` stacks (B1.0–B1.3).
|
||||
//
|
||||
// A coordinator fiber launches three async tasks; each `sleep`s a different
|
||||
// duration, records its completion (id @ virtual-ms) into a shared log, then
|
||||
// returns a value. The coordinator `wait`s on all three (in SPAWN order) and
|
||||
// sums their results. Because tasks complete in DEADLINE order — not spawn
|
||||
// order, not await order — the completion log is the deterministic contract:
|
||||
//
|
||||
// task A: sleep 30 → returns 100
|
||||
// task B: sleep 10 → returns 20
|
||||
// task C: sleep 20 → returns 3
|
||||
// completion order (by deadline): B@10, C@20, A@30
|
||||
// coordinator awaits A,B,C → sum = 123, final virtual clock = 30
|
||||
//
|
||||
// `wait(A)` parks the coordinator until A finishes at t=30; B and C finish
|
||||
// earlier (at 10 and 20) and are already `.ready` by the time their `wait`s run,
|
||||
// so they return without re-parking — the values are correct regardless of await
|
||||
// order, while the timer-driven schedule fixes the completion ORDER. Fully
|
||||
// deterministic + reproducible (virtual time, no real clock).
|
||||
//
|
||||
// aarch64-macOS-pinned (the scheduler's per-arch asm + Apple mmap constants):
|
||||
// runs end-to-end on a matching host, ir-only on a mismatch.
|
||||
#import "modules/std.sx";
|
||||
sched :: #import "modules/std/sched.sx";
|
||||
|
||||
Log :: struct { id: [8]i64; at: [8]i64; n: i64; }
|
||||
rec :: (l: *Log, id: i64, at: i64) { l.id[l.n] = id; l.at[l.n] = at; l.n = l.n + 1; }
|
||||
|
||||
main :: () -> i64 {
|
||||
lg : Log = ---; lg.n = 0;
|
||||
s := sched.Scheduler.init();
|
||||
ps := @s; pl := @lg;
|
||||
|
||||
// The coordinator runs as a fiber so `wait` has a `current` to park.
|
||||
s.spawn(() => {
|
||||
// Launch three async tasks; each sleeps, logs its completion, returns.
|
||||
a := ps.go(() -> i64 => { ps.sleep(30); rec(pl, 1, ps.now_ms()); 100 });
|
||||
b := ps.go(() -> i64 => { ps.sleep(10); rec(pl, 2, ps.now_ms()); 20 });
|
||||
c := ps.go(() -> i64 => { ps.sleep(20); rec(pl, 3, ps.now_ms()); 3 });
|
||||
|
||||
// Await in SPAWN order; results come back correct regardless.
|
||||
va := a.wait() or { -1 };
|
||||
vb := b.wait() or { -1 };
|
||||
vc := c.wait() or { -1 };
|
||||
sum := va + vb + vc;
|
||||
|
||||
rec(pl, 9, sum); // sentinel row: id=9 carries the sum in `at`
|
||||
});
|
||||
s.run();
|
||||
|
||||
print("completion order (id @ virtual-ms):\n");
|
||||
i := 0;
|
||||
while i < lg.n {
|
||||
if lg.id[i] == 9 { print("sum: {}\n", lg.at[i]); }
|
||||
else { print(" task {} @ {}ms\n", lg.id[i], lg.at[i]); }
|
||||
i = i + 1;
|
||||
}
|
||||
print("final virtual clock: {}ms\n", s.now_ms());
|
||||
print("tasks: {}\n", s.n_spawned);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{ "target": "macos" }
|
||||
@@ -0,0 +1 @@
|
||||
0
|
||||
@@ -0,0 +1 @@
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
completion order (id @ virtual-ms):
|
||||
task 2 @ 10ms
|
||||
task 3 @ 20ms
|
||||
task 1 @ 30ms
|
||||
sum: 123
|
||||
final virtual clock: 30ms
|
||||
tasks: 4
|
||||
Reference in New Issue
Block a user