cc13700237
Add library/modules/std/net/epoll.sx — raw epoll bindings, the linux twin of
std/net/kqueue.sx — and branch std.event.Loop on `inline if OS` so the
OS-neutral readiness Loop runs on linux (epoll) as well as darwin (kqueue);
callers never see the backend.
epoll_event has no packed-struct primitive in sx, so it is modelled as an
arch-branched struct of u32 fields — { events, data_lo, data_hi } → 12 bytes on
x86_64 (matching __attribute__((packed))), { events, pad, data_lo, data_hi } →
16 bytes on aarch64 — every field 4-aligned, so the layout is byte-exact for the
kernel ABI with no packed attribute and no unaligned access. The fd is stashed
in data_lo (epoll echoes one data word, not the fd separately).
epoll.sx is self-contained (libc only, no build.sx): the `inline if ARCH`
selecting the struct is resolved by the compiler's flatten pre-pass, so the
module's IR stays small. The epoll backend is imported INSIDE event.sx's
`inline if OS == .linux` branch (not top level): event.sx rides the std.sx
barrel, so a top-level import would register epoll's types into every std
program's type table on darwin and drift every .ir snapshot.
The epoll Loop keeps a small per-fd registration table (combined EPOLLIN/OUT
mask via EPOLL_CTL_ADD/MOD/DEL), maps the fd back to the caller's udata, arms
EPOLLRDHUP so a peer half-close surfaces as Event.eof (matching kqueue EV_EOF),
and uses an eventfd as the cross-thread wake channel (kqueue's EVFILT_USER).
Validation: the kqueue path runs end-to-end on the macOS host (1632 unchanged);
the epoll bindings + ABI layout are corpus-locked ir-only by
examples/event/1633 (x86_64-linux, both arches probe-verified). The epoll Loop
is verified to lower clean for both linux arches and self-reviewed, but is not
corpus-snapshotted (a Loop example drags the std barrel → ~18k-line brittle IR);
runtime behavior validates on a linux runner.
929 lines
78 KiB
Markdown
929 lines
78 KiB
Markdown
# CHECKPOINT-FIBERS — Stream B1 (fibers + Io + M:1 scheduler)
|
||
|
||
Companion to [PLAN-FIBERS.md](PLAN-FIBERS.md). Update after every step (one step at a time,
|
||
per the cadence rule). New corpus category: `18xx` concurrency.
|
||
|
||
## Last completed step
|
||
**B1 follow-up — `Scheduler.deinit` (close the bounded leaks).** Post-B1 non-blocking cleanup: a
|
||
terminal `deinit` on `library/modules/std/sched.sx`'s `Scheduler` releases the resources B1 documented
|
||
as leaked. Frees, in order: (1) any fibers still enqueued ready (leak-safety net for `spawn`/`go`
|
||
without `run()` — `munmap` stack + free struct; a suspended off-queue fiber is unreachable, but a clean
|
||
`run()` aborts on orphans so none survive it); (2) every heap `*Task` from `go` — newly tracked via a
|
||
`task_allocs: List(*void)` field appended in `go` (the scheduler otherwise has no handle on its generic
|
||
`Task($R)`s); (3) the three `List` backings (`task_allocs`/`timers`/`io_waiters`, all grown through
|
||
`own_allocator`); (4) the lazily-opened kqueue fd (`close`, reset to `-1`). NOT freed (unchanged
|
||
language limitation): the per-`spawn`/`go` closure env (sx exposes no env-free). Idempotent (rests on
|
||
`List.deinit` nulling `items` + the `kq`/`ready_head` resets); TERMINAL contract — no scheduler-owned
|
||
handle (`*Task`, `*Fiber`, the scheduler) is usable after `deinit`.
|
||
- Added a canonical `close :: (i32) -> i32 extern libc` (matches the dedupe-canonical signature 1816
|
||
already uses) + the `task_allocs` field.
|
||
- Locked by `examples/concurrency/1820-concurrency-fiber-scheduler-deinit.sx` (aarch64-macOS `.build
|
||
{"target":"macos"}`, runs end-to-end): one run touches every freed resource — a SLEEPER (`timers`), a
|
||
pipe READER `block_on_fd` + WRITER (kqueue fd + `io_waiters`), two `go` tasks (`Task`s + `task_allocs`)
|
||
— then `deinit`. Verified by a tracking `GPA`: `freed by deinit: 5`, `live after deinit: 5` (the
|
||
RESIDUAL = the 5 documented closure envs, not a bug), `kq open after run: true` → `kq after deinit:
|
||
-1` (the genuinely-open kqueue fd is closed), `read: 3 [97 98 99]` (the fd path actually ran). Counts
|
||
captured into locals BEFORE printing (`print` allocates format temporaries through the same GPA).
|
||
- **Adversarially reviewed (worker):** no real memory-safety bug in the supported (deinit-after-`run`)
|
||
path — reap-loop reads `f.next` before freeing `f`, the three freed List backings + Tasks + kq are all
|
||
disjoint + scheduler-owned, no over-free, idempotent. The one CRITICAL it raised was a DOC contradiction
|
||
(step-(1) defensive reap vs step-(2) "post-run only"), reconciled by spelling out the terminal contract.
|
||
Its 0154-over-store concern (`.{}`→`List` writes in `init` could clobber `kq`) was PROBED and cleared:
|
||
`kq == -1` immediately after `init`, all fields clean. Suite GREEN **759/0**.
|
||
|
||
### Earlier — B1.5 — END-TO-END M:1 validation — STREAM B1 COMPLETE
|
||
A single capstone exercises the whole
|
||
colorblind pure-sx async runtime together: the M:1 scheduler (B1.5a) + suspending fiber-task async
|
||
`go`/`wait` (B1.4a) + deterministic virtual-time `sleep`/`now_ms` (B1.4b), over the `abi(.naked)`
|
||
`swap_context` on guarded `mmap` stacks (B1.0–B1.3). `examples/concurrency/1817-concurrency-fiber-m1-end-to-end.sx`:
|
||
a coordinator fiber launches three `go` tasks (sleep 30/10/20 → return 100/20/3), awaits all three in
|
||
SPAWN order, and sums them. The completion log is the deterministic contract — tasks finish in
|
||
DEADLINE order (`task 2@10, task 3@20, task 1@30`), not spawn/await order; `sum: 123`; final virtual
|
||
clock 30. Fully reproducible (virtual time, no real clock). Suite GREEN **755/0**.
|
||
|
||
**Stream B1 is feature-complete.** The pure-sx async runtime exists end-to-end: fibers behind the
|
||
`abi(.naked)` context switch (proven on aarch64 + x86_64/Win64), the M:1 cooperative scheduler,
|
||
suspending `go`/`wait`/`cancel` async, deterministic virtual-time timers, and real fd-readiness via
|
||
kqueue — all in `library/modules/std/sched.sx`, all adversarially reviewed, locked by `18xx`
|
||
(1800–1817). Compiler floor delivered: `abi(.naked)` emission (B1.0) + per-fiber `context` (B1.1,
|
||
zero-change). Five compiler bugs fixed en route (0151/0152/0153 in B1.2; 0154 in B1.5a;
|
||
0156-Part1 + 0157 in B1.4a). Deferred (documented, non-blocking): issue 0150 (`Future(void)`/`timeout`),
|
||
0155 (scalar-pointer index), 0156-Part2 (deferred `..` spread); a linux `epoll` twin of `block_on_fd`;
|
||
routing the suspending async through the erased `context.io` (M:N evolution); the heap-Task / closure-env
|
||
/ kq-fd leaks (bounded, default-GPA-invisible). Stream B2 (channels/cancel/stdlib) is the next carve.
|
||
|
||
### Earlier — B1.4c — REAL fd-readiness blocking via kqueue (macOS)
|
||
`library/modules/std/sched.sx` now lets a
|
||
fiber park on a file descriptor and the run loop block on `kevent` until the kernel reports it ready.
|
||
Reuses the existing verified `library/modules/std/net/kqueue.sx` bindings (`Kevent` (32 bytes),
|
||
`kqueue`/`kevent`/`kq_apply`/`kq_wait` + the `EVFILT_READ`/`EV_ADD`/`EV_ENABLE`/`EV_ONESHOT`
|
||
constants) rather than re-deriving the FFI — sched.sx imports it as `kqb`. Added to `Scheduler`:
|
||
- `kq: i32` (LAZY — `-1` in `init`, opened by the first `block_on_fd`, so a pure-compute /
|
||
virtual-timer scheduler never opens a kqueue fd; leaks one fd at exit once opened, same class as the
|
||
documented spawn-env / go-Task leaks — no deinit yet);
|
||
- `io_waiters: List(IoWaiter)` (`IoWaiter :: struct { fd: i32; fiber: *Fiber; }`, grown through
|
||
`own_allocator` per the long-lived-container rule);
|
||
- `block_on_fd(self, fd, want_read)` — lazily opens `kq`, arms a one-shot `EVFILT_READ` registration,
|
||
records an `IoWaiter{fd, current}`, then `suspend_self()`. Guards a null `current` (loud abort, like
|
||
`sleep`); `want_read=false` (write-readiness) is not wired yet → loud abort rather than silently
|
||
arming a read filter.
|
||
- Run-loop: after the ready queue drains, **Mode 1 (virtual time)** fires the earliest pending timer
|
||
(takes precedence — a program uses `sleep` OR fds, documented non-unification limitation); **Mode 2
|
||
(real fd)** — if `io_waiters` is non-empty, BLOCK on `kq_wait(kq, evbuf, MAXEV=16, -1)` (null
|
||
timeout), then for each fired event match `ev.ident` back to its waiter, evict it, and `wake` the
|
||
fiber; **else** break. Orphan-deadlock check unchanged in spirit but now correct: an fd waiter is NOT
|
||
an orphan (while `io_waiters.len > 0` the loop blocks on kqueue rather than reaching the check), and
|
||
a genuine no-timer/no-fd suspend still aborts loudly (verified with a probe: exit 134).
|
||
- `wake` now also evicts a pending fd-waiter (`cancel_io_waiter_for`, mirror of `cancel_timer_for`) —
|
||
same UAF reasoning: a fiber woken by another path must not leave a stale `IoWaiter` pointing at a
|
||
reaped `*Fiber`. The kqueue registration is `EV_ONESHOT` so we never `EV_DELETE` (a never-fired
|
||
one-shot lingers harmlessly; the drain ignores an unmatched ident; closing the fd auto-removes it).
|
||
- DE-RISK probe (run first, no scheduler): confirmed `size_of(Kevent) == 32`, the pipe roundtrip
|
||
(`kq_wait` returned 1 with `out.ident == read_fd`, `out.filter == -1` (EVFILT_READ), `out.data == 1`
|
||
byte readable) — the struct layout reads back the fd correctly.
|
||
- Locked by `examples/concurrency/1816-concurrency-fiber-io-pipe.sx`: a `pipe`; a reader fiber spawned
|
||
FIRST blocks on the empty read end, then a writer fiber writes `a b c` → the run loop blocks on
|
||
kqueue, wakes the reader, which reads the 3 bytes. Output `log: wrote read 3 [97 98 99]` /
|
||
`n_suspended: 0` (the "wrote" before "read" ordering proves the reader actually blocked then woke via
|
||
kqueue readiness). `.build` `{ "target": "macos" }` (matches host arch → runs end-to-end; ir-only on
|
||
a mismatch, like 1814/1815 — no `.ir` snapshot needed since it runs here). The example declares its
|
||
own `read`/`write`/`close` externs with the CANONICAL signatures std already binds
|
||
(`(i32,[*]u8,usize)->isize` / `(i32)->i32`) — a divergent re-binding is rejected by the extern dedupe.
|
||
- **Adversarial review (worker) of the run-loop change — found 2 CRITICALs:**
|
||
- **(1) two fibers on the SAME fd → lost wakeup + permanent hang.** macOS `EV_ADD` for an existing
|
||
`(ident, filter)` REPLACES the registration (doesn't stack), so two waiters share one registration:
|
||
the fd fires once, one wakes, the other is stranded in `io_waiters` and the next `kq_wait(-1)` blocks
|
||
forever. FIXED: `block_on_fd` now enforces one-waiter-per-fd with a loud abort (the model already
|
||
assumed it). Verified: dup-fd → `sched: block_on_fd: fd N already has a waiter`, not a hang.
|
||
- **(2) an fd-waiter on a never-ready fd hangs instead of the timer path's loud abort.** Re-examined:
|
||
this is CORRECT event-loop semantics — blocking on I/O until ready (possibly forever, like a server
|
||
idling on a socket) is the point; the scheduler cannot know an fd will never become ready, so it must
|
||
keep waiting. NOT a scheduler deadlock. Fixed the MISLEADING comment that implied the orphan check
|
||
covers fd-waiters: it does not, by design (it covers only pure `suspend_self` parks). No code change —
|
||
the "hang" is a caller-side logic issue (waiting on input that never arrives), not a bug to abort on.
|
||
- Review CLEARED: the IoWaiter UAF parity (early-wake evicts the waiter; a lingering one-shot that later
|
||
fires hits no match → clean no-op), ident width/sign, `kq_wait` EINTR/error handling, timer-vs-io
|
||
precedence (timer wins; no hang). All probed safe.
|
||
- Suite GREEN **754/0** (incl. the dup-fd guard, no new example needed — the abort is host-fragile to
|
||
pin like 1809's guard-firing). Next: **B1.5** (end-to-end M:1 validation under the deterministic timers
|
||
/ fd readiness); a linux epoll twin of `block_on_fd` (mirror via `std/net/epoll`, the OS-neutral facade
|
||
is `std.event`) is future work.
|
||
|
||
### Earlier — B1.4b — deterministic VIRTUAL-TIME timer scheduling (the KEYSTONE) — landed + adversarially
|
||
reviewed (caught a CRITICAL UAF, fixed).
|
||
`library/modules/std/sched.sx` gained a virtual clock +
|
||
sleep timers so fibers schedule in reproducible simulated time (no real clock): `clock_ms` (advances
|
||
ONLY as timers fire), a `timers: List(Timer)` (insertion-order, linear min-scan, FIFO tiebreak),
|
||
`now_ms()`, `sleep(ms)` (arm `{clock_ms+ms, current}` + `suspend_self`), and a timer-driven `run`
|
||
(drain ready → fire earliest timer → advance clock → wake → repeat; orphan-deadlock check preserved
|
||
for a genuine no-timer suspend). Locked by `1814` (5 fibers sleep 30/10/20/15/15 → wake order
|
||
B@10, D@15, E@15 (FIFO), C@20, A@30 — deadline order, not spawn order; `now_ms()` reads each virtual
|
||
deadline; final clock 30). §8.1.3 calibration note in the header: the deterministic wake ORDER
|
||
equals what real `sleep`s produce, reproducing blocking semantics' observable ordering without real
|
||
time. The deterministic-sim `Io` is realized at the scheduler level (`sleep`/`now_ms`/timer-`run`),
|
||
not as an erased `Io`-protocol impl (same erasure reason as FiberIo).
|
||
- **Adversarial review (worker) of the run-loop change: found a CRITICAL use-after-free** — a fiber
|
||
that armed a `sleep` timer but was woken EARLY by another path (a manual/`Task` `wake`) ran to
|
||
completion + was reaped (stack `munmap`'d, `Fiber` freed) while its `Timer` still held a dangling
|
||
`*Fiber`; a later fire would `wake` freed memory (silent-corruption: "passes" only because the
|
||
freed slot coincidentally read `state != .suspended`). FIXED: `wake` now evicts the woken fiber's
|
||
pending timer (`cancel_timer_for`) — every re-ready path funnels through `wake` (the timer-fire in
|
||
`run` already removed the fired timer, so it's a harmless re-scan there), so no stale timer can
|
||
outlive its fiber. Regression `1815-concurrency-fiber-timer-early-wake.sx` (early wake → `clock: 0`,
|
||
the stale 100ms timer evicted, not fired). Review CLEARED: `n_suspended` accounting,
|
||
orphan-deadlock false-positives, timer-list integrity (re-arm during fire), clock monotonicity,
|
||
termination — all traced/probed safe.
|
||
- Suite GREEN (count below). Next: **B1.4c** (event-loop `Io` — real fd readiness, kqueue/epoll).
|
||
|
||
### Earlier — B1.4a — a truly-SUSPENDING fiber-task async layer (`go`/`wait`/`cancel`)
|
||
landed + adversarially reviewed; cleared two more compiler blockers en route. `library/modules/std/sched.sx`
|
||
now carries `Task($R)` + `Scheduler.go(work) -> *Task($R)` + `wait`/`cancel` (a `ufcs` layer over
|
||
the M:1 scheduler). `s.go(work)` runs the nullary thunk `work` as a REAL fiber; `t.wait()` SUSPENDS
|
||
the caller until it completes (vs io.sx's blocking `context.io.async`, which runs inline). Locked by
|
||
`examples/concurrency/1813-concurrency-fiber-async-suspend.sx`: two tasks interleave (A yields
|
||
mid-body so B runs first → `1 2 3`), awaited values `42`/`100`, and a canceled task's `wait` raises
|
||
`.Canceled` → `or -99` → `sequence: 1 2 3 42 100 -99`.
|
||
- **Design: a NULLARY thunk, not `async(worker, ..args)`.** A comptime variadic pack can't cross a
|
||
deferred (fiber) boundary — `..args` captured into a closure re-expands from the spawner's
|
||
now-gone locals (issue 0156 Part 2). So `go` takes `work: Closure() -> $R`; the user captures
|
||
inputs in the lambda at the call site (the `go func(){…}()` idiom). **Self-contained in sched.sx**
|
||
(NOT io.sx): io.sx importing sched.sx duplicates the `_fib_tramp` global asm when a program also
|
||
imports sched.sx directly (global asm emits per import-path) — so the Io-protocol
|
||
`spawn_raw`/`suspend_raw`/`ready` hooks stay reserved for the future M:N model; M:1 uses
|
||
`go`/`wait` directly. Heap `*Task` (must outlive `go`'s frame; leak documented). `TaskErr` is
|
||
LOCAL (the `!` failable detection doesn't see through io.sx's `IoErr` re-export alias).
|
||
- **Two compiler blockers hit + FIXED (user-authorized in-session):**
|
||
- **issue 0156 Part 1** — a single-type generic `$R` (parsed as `comptime_pack_ref`) used as a
|
||
type-arg (`Box($R)`, `size_of(Box($R))`) inside a pack-fn body hit a missing arm in
|
||
`resolveTypeWithBindings` → `.unresolved` → LLVM panic. Fix: mirror `resolveTypeArg`'s
|
||
`comptime_pack_ref` arm (look up `type_bindings`, else a loud diagnostic). Regression
|
||
`examples/generics/0216-generics-typearg-in-pack-fn-body.sx`. (Part 2 — deferred `..` spread
|
||
crashes — reframed OPEN/non-blocking, `issues/0156`.)
|
||
- **issue 0157** — a user generic `ufcs` method whose name collides with a stdlib re-export
|
||
(`cancel` on `*Task` vs io.sx's `cancel` on `*Future`) resolved via last-wins `fn_ast_map` with
|
||
NO receiver filtering → wrong overload → `$R` unbound → LLVM panic. Fix
|
||
(`src/ir/lower/call.zig` `selectUfcsGenericByReceiver`): every generic-ufcs dispatch enumerates
|
||
ALL module authors (`module_decls`), keeps receiver-binding ones, picks the most
|
||
receiver-SPECIFIC (concrete > bare `$T`), dedups re-exports, and flags a genuine 2-specific tie
|
||
as a deterministic "ambiguous — qualify" diagnostic (never a silent order-dependent pick).
|
||
Regression `examples/generics/0217-generics-ufcs-method-name-collides-stdlib.sx`.
|
||
- **Adversarial review (worker) of the 0157 fix + Task layer.** Caught the determinism CRITICAL
|
||
(fixed: always-run selection + specificity + ambiguity), `wait`-outside-a-fiber null-deref (fixed:
|
||
loud guard in `suspend_self`/`yield_now`), and cancel-doesn't-skip-work (fixed: worker skips
|
||
`work()` if already canceled). Lost-wakeup / cancel-after-complete / reap traced safe. Also
|
||
simplified `1812` (`**Fiber` shared handle → a `Sh.parked` field; output identical).
|
||
- Suite GREEN 751/0 (749 + 1813 + 0217). Next: **B1.4b** (deterministic-sim `Io`).
|
||
|
||
### Earlier — B1.5a — the M:1 cooperative fiber scheduler CORE — landed + adversarially reviewed
|
||
The hand-bootstrapped ping-pong (1807-1810) is now a reusable scheduler API in pure sx:
|
||
`library/modules/std/sched.sx` — a generic `Fiber` (`body: Closure() -> void`) + `Scheduler`
|
||
with `init`/`spawn`/`yield_now`/`suspend_self`/`wake`/`run` over the proven `swap_context` on
|
||
guarded `mmap` stacks. The ONE generic dispatch (`fib_dispatch`, reached from the `_fib_tramp`
|
||
trampoline) runs ANY stored closure body on a fresh stack — replacing the fixed `bl _fib_body`.
|
||
Reaping `munmap`s the stack + frees the heap `Fiber` on completion; an intrusive FIFO gives
|
||
round-robin order.
|
||
- **Foundational design de-risked by probe before building:** a fiber can store + call a
|
||
`Closure() -> void` on its fresh stack via the generic dispatch; outputs flow OUT through
|
||
pointers captured in the closure (capture-by-value does NOT write back — pushed onto the user).
|
||
- **Hit + FIXED a blocker compiler bug — issue 0154** (user-authorized in-session fix). `null` /
|
||
`---` assigned to a struct field picked up a leaked enclosing `target_type` (the function's
|
||
RETURN type, set for the whole body at decl.zig:2691) and built a WHOLE-STRUCT-typed null →
|
||
an oversized `zeroinitializer` store through the field's GEP that overran the field's slot and
|
||
clobbered the saved x29/x30, so the fn `ret`'d to 0x0. This was EXACTLY the `Scheduler.init()`
|
||
by-value-return shape (`sched_ctx: [13]u64` before `current: *Fiber`). Fix: added
|
||
`.null_literal, .undef_literal` to the `needs_target` switch in `lowerAssignment`
|
||
(`src/ir/lower/stmt.zig`) so the field's type is used. Repro → regression test
|
||
`examples/types/0193-types-sret-array-before-pointer.sx`; `issues/0154-*.md` RESOLVED.
|
||
- **Adversarial review (worker): asm/bootstrap/lifetime SOUND** (the headline closure-env-lifetime
|
||
fear was disproven — envs are heap-promoted, survive the spawn frame). Found **1 CRITICAL** +
|
||
robustness gaps, ALL hardened: (CRITICAL) `wake` re-enqueued an already-queued fiber →
|
||
FIFO corruption/segfault → now GUARDED on `.suspended` (spurious/double/stale wake = safe
|
||
no-op); orphan-suspend leak/deadlock → `n_suspended` accounting + a loud `run()`-drain
|
||
diagnostic+abort; `mmap` `MAP_FAILED` (=-1, not null) / `mprotect` / Fiber-OOM → loud bails
|
||
(per §8.1.1 the guard is mandatory); the per-fiber closure-env leak (sx exposes no env-free) →
|
||
documented as a KNOWN LIMITATION (bounded by spawn count; invisible under the default GPA).
|
||
- **Locked two `18xx` examples** (aarch64-macos `.build`-pinned, ir-only on a mismatch):
|
||
`1811-concurrency-fiber-scheduler.sx` (3 fibers round-robin via `yield_now` → ordering contract
|
||
`sequence: 0 1 2 0 1 2 0 1 2`, all `.done`) + `1812-concurrency-fiber-suspend-wake.sx` (park via
|
||
`suspend_self`, resumed by another fiber's `wake`, + the spurious-wake no-op — the CRITICAL-fix
|
||
regression → `log: 10 20 21 11` / `suspended-left: 0`).
|
||
- **Filed issue 0155 (NON-blocking, NOT fixed)** — found incidentally in the review: indexing a
|
||
scalar pointer (`pc[0]`, `pc: *i64`) panics codegen (`.unresolved` reaching LLVM emission). The
|
||
scheduler uses array-field indexing + `.*`, never this, so it's filed for its own session.
|
||
- Suite GREEN **748/0** (746 base + 1811 + 1812 + 0193 regression). Next: **B1.4a** (FiberIo —
|
||
wire `Io.spawn_raw`/`suspend_raw`/`ready` onto the scheduler so `async`/`await` truly suspend).
|
||
|
||
### Earlier — B1.3b-1 — the x86_64 / Win64 `swap_context` sibling — VALIDATED on real hardware
|
||
The
|
||
context switch is now proven on a SECOND architecture + ABI. A Win64 `swap_context` saves the
|
||
COMPLETE Win64 callee-saved set — 8 GP (rbx, rbp, rdi, rsi, r12-r15) + rsp **and xmm6-xmm15**
|
||
(10 XMM, 128-bit via `movups` — Win64 has callee-saved XMM, unlike SysV/aarch64) — plus a Win64
|
||
`scribble_verify` (32-byte shadow + 16-align at each `call`, COFF symbols, rsp-carried return
|
||
addr). Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
|
||
ir-only here): the 2-fiber mutual scribble printed **`0 0 P`** when built `--target
|
||
x86_64-windows-gnu --self-contained` and **run on a Windows 7 x64 VM (UTM)** — every GP + XMM
|
||
callee-saved survived. **Adversarially reviewed before the VM run** (worker emitted the real `.s`
|
||
and verified every `call` alignment, the 264-byte frame offsets, the rsp/return-addr round-trip,
|
||
swap ordering, and COFF naming against the Win64 ABI — no critical/minor bugs). The build→VM→run
|
||
loop was set up this session (cross-build needs `--self-contained`; output via the Win32
|
||
`WriteFile` boundary, the 1660 pattern). Suite green. Note: this is the GOOD-swap-only mutual
|
||
scribble (self-validating by construction; the in-process negative control was dropped to avoid an
|
||
sx fn-ptr-convention rabbit hole — the detection of this exact logic was negative-controlled on
|
||
aarch64 in 1808). The SysV/Linux x86_64 sibling (different reg set: no callee-saved XMM, args
|
||
rdi/rsi) remains for a Linux x86_64 host.
|
||
|
||
### Earlier — B1.3b-2 — mmap guard-page stacks (commit `dd532ab`)
|
||
Fiber stacks are `mmap`'d with a `PROT_NONE` GUARD PAGE at the low end (§8.1.1: a
|
||
fixed stack without a guard silently corrupts neighbors on overflow). `mmap` the `[guard |
|
||
usable]` region, `mprotect` the low 16KB page `PROT_NONE`; SP descends into the guard and faults
|
||
loudly at the boundary instead of corrupting a neighbor. Locked by
|
||
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
|
||
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
|
||
- **Guard FIRING validated** (manually, not corpus-pinned — a deliberate overflow crash is
|
||
host-fragile): a fiber recursing past its 128KB stack faults with `Bus error` at the guard page
|
||
(`region+GUARD`); the sx crash handler turns it into exit 134. Documented in the example header.
|
||
- **x86_64 sibling:** was deferred here (couldn't run x86_64 on this arm64 host), then DONE as
|
||
Win64 once a Windows 7 x64 VM became available — see B1.3b-1 above (`examples/1810`, `0 0 P`).
|
||
|
||
### Earlier — B1.3a-2 — the context-switch STRESS GATE (design §10.7) — DONE + adversarially reviewed
|
||
The explicit every-callee-saved-register scribble that B1.3a-1 owed. `swap_context` now saves the
|
||
COMPLETE AAPCS64 callee-saved set — integer x19-x28 + fp/lr + sp AND FP **d8-d15** (per §6.1.2
|
||
only the low 64 bits of v8-v15 are callee-saved, so `d8-d15` is exactly sufficient; x18 is Apple's
|
||
reserved platform reg, untouched). A naked `scribble_verify(self_ctx, peer, base)` loads a unique
|
||
sentinel into all 18 callee-saved regs, yields, and on resume counts the ones that didn't survive
|
||
(honoring its own caller ABI via a 176-byte frame that saves+restores the caller's callee-saved;
|
||
base reloaded from the frame post-swap; the original lr round-trips through the swap). The gate is
|
||
a **2-fiber MUTUAL scribble** (A and B scribble DISTINCT sentinels into the same physical regs, so
|
||
each survives only if `swap_context` saved+restored it — a lone fiber yielding to an idle peer
|
||
would NOT exercise preservation). Locked by `examples/1808-concurrency-fiber-switch-stress.sx`
|
||
(aarch64-pinned): `A mismatches: 0` / `B mismatches: 0`.
|
||
- **Validity proven by NEGATIVE controls:** dropping the d8-d15 save/restore → 8/8 mismatches
|
||
(exactly the FP regs); dropping x27/x28 → 2/2. The gate genuinely catches a broken switch.
|
||
- **Adversarial review (worker, per the plan): no CRITICAL bugs.** Verified the callee-saved set
|
||
is complete + correct, all frame offsets/16-alignment, the lr/sp dance, and swap read-ordering
|
||
against AAPCS64. Applied its one recommendation: `boot` now zeroes the FP ctx slots [13..20] so a
|
||
first switch-to loads 0 (not garbage) into d8-d15. Residual gaps it flagged (all spec-correct
|
||
for a call-boundary swap, documented in the example header): NZCV/FPSR not swapped; **FPCR**
|
||
(rounding mode — thread-global, bleeds across fibers if changed) and **TPIDR_EL0/TLS** (errno,
|
||
allocator thread-caches — shared by same-thread fibers) not swapped; fp=0 bootstrap blocks
|
||
unwind/signal walking past a fiber trampoline. These bite at the N×M:1 / signals stages, not the
|
||
single-thread switch.
|
||
- Suite green **734/0**, master clean. WIP probes: `.sx-tmp/scribble2.sx` (+ `_broken`/`_gp`).
|
||
|
||
### Earlier — B1.3a-1 — the foundational stackful context switch (commit `b234b7d`)
|
||
Pure sx over `abi(.naked)`: naked `swap_context` (GP-only 13-slot save) + by-hand fiber bootstrap
|
||
(SP = `alloc_bytes` stack top, LR = global-asm trampoline, x19 = `*Fiber`). Locked by
|
||
`examples/1807-concurrency-fiber-context-switch.sx`: 2-fiber ping-pong (`rounds: 6` / `canary
|
||
fails: 0`) + 64-frame deep recursion (`frames verified: 64` / `depth fails: 0`). Indirect
|
||
register/stack survival; 1808 supersedes its switch with the complete GP+FP save area + the
|
||
explicit gate.
|
||
|
||
### Earlier — B1.2 COMPLETE — the async surface works end-to-end
|
||
All three surface blockers (0151, 0152, 0153) FIXED + committed; async examples landed + green.
|
||
- **0151 fixed** (`362674f`): generic `$T` infers through generic-struct / pointer / UFCS-pack
|
||
params. Regression `0214` + `0215`.
|
||
- **0152 fixed** (`e5586f6`): `Atomic(bool)` load/store byte-promoted to `i8` in the codegen
|
||
emitters. Regression `1705`.
|
||
- **0153 fixed** (`68c1991`): `inferGenericReturnType` now pins return-type resolution to the
|
||
fn's DEFINING module (mirroring `monomorphizeFunction`), so a re-exported value-failable's
|
||
`!E` resolves to the real `.error_set` TypeId — the failable channel survives the re-export
|
||
alias. Regression `1058-errors-reexport-value-failable-channel.sx`.
|
||
- **Async examples landed:** `examples/1805-concurrency-io-blocking-async.sx`
|
||
(`context.io.async((a,b)->i64 => a+b, 40, 2).await() or {…}` → `sum: 42` / `double: 42` /
|
||
`clock ok`) + `examples/1806-concurrency-io-cancel.sx` (`f.cancel()` → `await` raises
|
||
`.Canceled` → `or` default; `ok: 7` / `canceled: -99`). Both green, snapshots captured.
|
||
|
||
### Earlier — the three B1.2 surface fixes (committed)
|
||
Generic `$T` inference, `Atomic(bool)` byte-promotion, and re-export failable-channel pin —
|
||
details below.
|
||
- **0151 fix (committed):** four gaps closed on the inference + UFCS-dispatch path —
|
||
(1) `extractTypeParam`/`matchTypeParam(Static)` got a `parameterized_type_expr` arm
|
||
(recover the arg instance's recorded per-param bindings via `struct_instance_bindings` +
|
||
the template's ordered `type_params`, recurse positionally; this also fixes `*Box($T)` —
|
||
it recurses into its `Box($T)` pointee); (2) the `pointer_type_expr` arm now falls through
|
||
to match the pointee against a non-pointer arg (auto-address-of: a `*Box($T)` param accepts
|
||
a by-value `Box($T)`, e.g. a UFCS receiver `b.m()`); (3) `ExprTyper.inferType` got a
|
||
`.lambda` arm building the closure type from the lambda's annotations (the UFCS binder types
|
||
args from the raw AST before they're lowered, so it can now bind `Closure(..) -> $R` from
|
||
the worker's declared return type); (4) a pack UFCS target routes through the SAME
|
||
`lowerPackFnCall` the direct call uses, with the receiver spliced in as `args[0]`.
|
||
- Regression tests: `examples/0214-generics-ufcs-closure-return-pack.sx` (direct + UFCS
|
||
closure-return pack) + `examples/0215-generics-infer-through-pointer.sx` (by-value /
|
||
pointer / multi-param / nested / UFCS-auto-ref struct-head inference). Issue 0151 marked
|
||
RESOLVED; repro moved into the suite.
|
||
|
||
### Earlier — B1.2 (Io capability) — LANDED + adversarially reviewed
|
||
Commits `a1b14f0` (lock) + `45d869d` (Io capability) + `3eeb965` (issue 0151 lock).
|
||
- **LANDED + review-confirmed correct** (commit 45d869d): `Io :: protocol #inline`
|
||
(spawn_raw/suspend_raw/ready/poll/now_ms/arm_timer) + `io` field on `Context`
|
||
(`{allocator; data; io}`, io LAST); BOTH `__sx_default_context` materializers
|
||
(protocol.zig + comptime_vm.zig) build an identical CBlockingIo→Io vtable (review verified
|
||
byte-for-byte agreement; `context.io.now_ms()` dispatches at runtime AND comptime); the
|
||
`push Context.{…}` omitted-field-**inherits-ambient** fix (review: correct, right fix, no
|
||
bad blast radius); `library/modules/std/io.sx` (`Future($R)`, `CBlockingIo`,
|
||
`async`/`await`/`cancel`); the `!`-protocol-impl-lint suppression; 37 `.ir` regens
|
||
(review: pure layout/type-table, no error text, zero .exit/.stdout/.stderr change).
|
||
- **BLOCKED — async surface non-functional:** `await`/`cancel` take `*Future($R)` and are
|
||
**uncallable in EVERY form** (not just UFCS) — sx can't infer a generic `$T` from a
|
||
pointer-wrapped arg (`*Future($R)`). `async(...)` (create) works via explicit call and
|
||
produces a correct `.ready` Future, but you can't `await` it. Root bug = **issue 0151
|
||
(WIDENED)**: infer `$T` from `*T`-wrapped params + closure-return-via-pack + UFCS dispatch.
|
||
Minimal repro: `unbox :: (b: *Box($T)) -> $T` fails to infer `T`.
|
||
- **No async example in the corpus** (1805 was removed because it needs the blocked surface)
|
||
→ the green suite does NOT cover async. Restore `1805` (async/await) + add `1806` (cancel)
|
||
once 0151 is fixed.
|
||
|
||
### Earlier — B1.1 (per-fiber `context` root) — DONE. Zero compiler change (confirmed by probe).
|
||
The fiber-spawn context convention works end-to-end with ordinary language features:
|
||
- `snap := context` captures the spawner's `Context` as a value;
|
||
- the snapshot is stored in a struct (the stand-in `Fiber`);
|
||
- a trampoline running under a *different* ambient context installs the fiber's stored root
|
||
with `push f.root { … }`, and the body reads the snapshot — not the trampoline's ambient
|
||
context — because `context` is an implicit slot-0 `*Context` param (call-carried, rides the
|
||
callee's own stack) and `push` allocates on the caller frame (no global, no TLS).
|
||
- Locked by `examples/1804-concurrency-context-snapshot.sx`: prints `fiber root: 42` (the
|
||
installed snapshot wins over ambient 99) + `ambient after: 99` (the `push` scope restores
|
||
the ambient context on exit). No fiber runtime yet (that's B1.3) — this proves the plumbing
|
||
it will build on. No `.build` pin (pure sx, host-independent).
|
||
- **Probe result:** the design doc's "lower as swappable indirection, never raw TLS" guarded
|
||
a non-problem — context was already param-carried, never TLS. No path re-reads
|
||
`__sx_default_context` mid-stack, so there is **no compiler obligation** here.
|
||
- `zig build && zig build test` green: **726 ran, 0 failed**.
|
||
|
||
### Earlier — B1.0 (`abi(.naked)` codegen) — complete
|
||
Replaced the emit bail with real LLVM `naked` emission:
|
||
- `emit_llvm` declaration pass: for `func.is_naked`, add the LLVM `naked` + `noinline` +
|
||
`nounwind` attributes and **skip** the `frame-pointer=all` attribute (incompatible with a
|
||
frameless function). Pass 2 now emits the `.naked` body normally — `naked` makes the
|
||
backend emit it verbatim (the inline asm + its own `ret`) with no prologue/epilogue.
|
||
- IR shape (verified): `; Function Attrs: naked noinline nounwind` / `define internal i64
|
||
@answer() #0 { entry: call void asm sideeffect "…ret…", ""() unreachable }` /
|
||
`attributes #0 = { naked noinline nounwind }`. The caller invokes it as an ordinary
|
||
`() -> i64` call (`.naked` is `call_conv == .default`).
|
||
- `examples/1800-concurrency-naked-asm.sx` — now GREEN, aarch64-pinned (`.build {"target":
|
||
"macos"}`): runs end-to-end → **exit 42** on this host, ir-only on a mismatch; `.ir`
|
||
snapshot captured.
|
||
- `examples/1801-concurrency-naked-generic.sx` (renamed from `-bail`) — the generic `.naked`
|
||
now emits a correct naked `answer__i64` (exit 42), proving generic.zig produces a naked
|
||
body, not a framed one. aarch64-pinned.
|
||
- `examples/1802-concurrency-naked-asm-x86.sx` — x86_64 cross sibling (`.build {"target":
|
||
"x86_64-linux"}`, ir-only here): `.ir` locks `naked` + `movl $42, %eax` / `ret`.
|
||
- Unit test `emit: abi(.naked) function gets the naked attribute (no frame-pointer)` in
|
||
`emit_llvm.test.zig` (asserts `naked` present, `frame-pointer` absent).
|
||
- **B1.0c (review-hardening):** a param-bearing `.naked` fn emitted invalid LLVM (loud
|
||
verifier error "cannot use argument of naked function") because the param-alloca loop
|
||
wasn't gated. Fixed forward (this *enables* the B1.3 context-switch use case rather than
|
||
rejecting it): gated the param-alloca loop on `fd.abi != .naked` in decl.zig (both paths) +
|
||
generic.zig; a naked fn's args stay in registers (read by asm), declared-but-unused in
|
||
LLVM. Locked by `examples/1803-concurrency-naked-asm-param.sx` (`add(a,b)` → x0+x1 → 42).
|
||
- `zig build && zig build test` green: **725 ran, 0 failed** + unit tests.
|
||
|
||
### Earlier — B1.0a (lock + review hardening)
|
||
Plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites + generic.zig +
|
||
pack.zig); `funcWantsImplicitCtx` skips `.naked` (no synthetic ctx, like `.c`); all
|
||
body-lowering paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx
|
||
return); `emit_llvm` Pass 2 bailed loudly (since flipped to real emission). Adversarial
|
||
review caught the generic/pack `is_naked` gap (a generic `.naked` silently shipped a framed
|
||
body); closed + locked. The review's `.naked`-lambda CRITICAL was a false positive
|
||
(unparseable — `isLambda` breaks on the `abi` keyword).
|
||
|
||
## Current state
|
||
**STREAM B1 FEATURE-COMPLETE.** `library/modules/std/sched.sx` is the whole pure-sx M:1 async runtime:
|
||
the scheduler core (B1.5a: `spawn`/`yield_now`/`suspend_self`/`wake`/`run`), suspending fiber-task
|
||
async (B1.4a: `Task($R)`/`go`/`wait`/`cancel`), deterministic virtual-time timers (B1.4b:
|
||
`clock_ms`/`now_ms`/`sleep`, timer-driven `run`), AND real fd readiness via kqueue (B1.4c: lazy `kq`,
|
||
`io_waiters`, `block_on_fd`, run-loop Mode 2) — all over the `abi(.naked)` `swap_context` on guarded
|
||
`mmap` stacks (B1.0–B1.3), reusing `std/net/kqueue.sx`. Every park path (timer sleep, fd block, raw
|
||
suspend) is balanced through `wake` (which evicts stale timer + fd waiters — the UAF guards). A terminal
|
||
`deinit` (B1 follow-up) closes the previously-documented leaks: heap `Task`s (tracked via `task_allocs`),
|
||
the `timers`/`io_waiters`/`task_allocs` List backings, and the kqueue fd; the per-`spawn`/`go` closure
|
||
env remains unfreeable (language limitation). Locked by `18xx` 1800–1820 (naked-asm, context-snapshot,
|
||
blocking async, the switch + §10.7 stress gate + guarded stacks + Win64 sibling, scheduler round-robin,
|
||
suspend/wake, async go/wait/cancel, sim-timer ordering, timer early-wake eviction, kqueue pipe I/O, the
|
||
**1817 end-to-end capstone**, sleep-negative/double-wait guards, and **1820 scheduler-deinit**). Suite
|
||
GREEN **759/0**, committed.
|
||
|
||
Future work (none blocking B1): a **linux epoll twin** of `block_on_fd` (mirror via `std/net/epoll`;
|
||
OS-neutral facade `std.event`) — B1.4c wired macOS kqueue only; routing the suspending async through
|
||
the erased `context.io` (forces sched.sx into every std consumer + duplicates the `_fib_tramp` global
|
||
asm — deferred to the M:N model, where the `Io` protocol's `spawn_raw`/`suspend_raw`/`ready`/
|
||
`arm_timer`/`poll` hooks take over); `Future(void)`/`timeout` (issue 0150); freeing the heap-Task /
|
||
closure-env / kq-fd (a Scheduler `deinit` + closure-env-ownership affordance). **Next carve: Stream
|
||
B2** (channels / structured cancel / async stdlib) — see PLAN-CHANNELS.md when started.
|
||
|
||
### Earlier — B1.5a COMPLETE — the M:1 scheduler CORE exists
|
||
`library/modules/std/sched.sx` drives N fibers
|
||
(generic `Closure() -> void` bodies) cooperatively over the proven `swap_context`, on guarded
|
||
`mmap` stacks: `spawn` / `yield_now` (round-robin) / `suspend_self` + `wake` (off-queue park/resume)
|
||
/ `run` (drives to drain, reaps on `.done`). Adversarially reviewed + hardened (wake guarded, loud
|
||
mmap/mprotect/OOM/deadlock bails, env-leak documented). Locked by `1811` (round-robin ordering
|
||
contract) + `1812` (suspend/wake park-resume + spurious-wake guard). Suite GREEN **748/0**.
|
||
|
||
The remaining B1.4 work wires this scheduler under the `Io` capability: **B1.4a (FiberIo)** makes
|
||
`context.io` route `spawn_raw`/`suspend_raw`/`ready` onto the `Scheduler` so `async`/`await` truly
|
||
SUSPEND (today's `CBlockingIo` runs the worker to completion inline); **B1.4b** the deterministic-sim
|
||
`Io` (virtual clock + timer queue, calibrated against blocking — the KEYSTONE test harness);
|
||
**B1.4c** the event-loop `Io` (kqueue/epoll). Then **B1.5** is the end-to-end M:1 validation under
|
||
the deterministic `Io`.
|
||
|
||
### Earlier — B1.2 COMPLETE
|
||
The full async surface (Io capability on Context + `async`/`await`/`cancel` +
|
||
blocking `CBlockingIo`) works end-to-end. Master GREEN (732/0), installed `sx` clean. All four
|
||
B1.2 surface bugs resolved or deferred:
|
||
- **0151 fixed** (`362674f`): generic `$T` through generic-struct / pointer / UFCS-pack params.
|
||
Regression `0214` + `0215`.
|
||
- **0152 fixed** (`e5586f6`): `Atomic(bool)` byte-promoted to `i8` in the load/store emitters.
|
||
Regression `1705`.
|
||
- **0153 fixed** (`68c1991`): `inferGenericReturnType` pins return-type resolution to the fn's
|
||
defining module, so a re-exported value-failable keeps its `!` channel. Regression `1058`.
|
||
- Issue **0150** (`void` struct field → SIGTRAP) DEFERRED — only `Future(void)` / `timeout`,
|
||
which are B1.4.
|
||
|
||
The async examples are landed + green: `1805` (`async`/`await` + `now_ms` → `sum: 42` /
|
||
`double: 42` / `clock ok`) + `1806` (`cancel` → `await` raises `.Canceled` → `or` default).
|
||
The `18xx` concurrency category now covers naked-asm (1800-1803), context-snapshot (1804), and
|
||
the async surface (1805-1806).
|
||
|
||
### B1.2 Io capability — what is LANDED + verified (commit 45d869d)
|
||
- `Io :: protocol #inline { spawn_raw; suspend_raw -> !; ready; poll; now_ms; arm_timer; }`
|
||
in `core.sx` next to `Allocator`, with `SpawnOpts{ pin: PinTarget }` + `ParkToken{ handle }`.
|
||
Six methods, each justified by a downstream consumer (B1.3-B1.5).
|
||
- `Context :: struct { allocator; data; io: Io; }` — `io` appended LAST so `allocator` stays
|
||
index 0 (the `call.zig:1229` hardcode) and `data` keeps index 1 (minimal VM-fallback churn).
|
||
- Both `__sx_default_context` materializers updated in lockstep + verified: `protocol.zig`
|
||
`emitDefaultContextGlobal` (extended `ctx_fields` 2→3, built the `CBlockingIo→Io` inline
|
||
7-word vtable `{null-ctx, fn0..fn5}` via `getOrCreateThunks("Io","CBlockingIo")`) and
|
||
`comptime_vm.zig` `materializeDefaultContext` fallback (wrote the 6 thunk func-refs at
|
||
`io_base = addr + 4*ps`, offset `+ (i+1)*ps`). The global path auto-followed the 3-field
|
||
Context type. **`context.io.now_ms()` printed `clock ok` live — the capability threads + the
|
||
vtable dispatches correctly.**
|
||
- Stateless `CBlockingIo :: struct {}` + `impl Io for CBlockingIo` (mirror of `CAllocator`):
|
||
blocking semantics — `spawn_raw`/`ready`/`poll`/`arm_timer` no-op/0, `now_ms` → `time.mono_ms()`.
|
||
- **push-inherit-omitted fix** (`stmt.zig` `lowerPush`): a `push Context.{...}` now SEEDS the
|
||
new slot from the ambient context (load+store), then overwrites ONLY the literal's named
|
||
fields — so omitted fields (now incl. `io`) are INHERITED, never zero-inited to a null
|
||
vtable. Eliminates the omitted-field footgun globally (zero per-site churn across the 17
|
||
partial-literal sites). This is the correct capability-bag semantics; it compiled clean.
|
||
- **`!`-protocol-method warning fix** (`error_analysis.zig` + a new `Lowering.impl_method_names`
|
||
set populated in `protocols.zig` `registerImplBlock`): a protocol impl method may be declared
|
||
`!` by contract (e.g. `Io.suspend_raw`) yet never raise; the "declared `!` but never errors —
|
||
drop the `!`" hint is a false positive for impl methods, now suppressed for them.
|
||
|
||
Status of the blockers that originally stopped B1.2:
|
||
- **issue 0151 — FIXED this session** (generic `$T` through generic-struct / pointer /
|
||
UFCS-pack params). `async`/`await`/`cancel` are callable. See "Last completed step".
|
||
- **issue 0152 — NEW, the current blocker** (`Atomic(bool)` → sub-byte i1 atomic; LLVM reject).
|
||
Blocks the async examples via `Future.canceled: Atomic(bool)`. Filed; codegen-level fix.
|
||
- **issue 0150** — `void` struct field SIGTRAP; only `Future(void)`/`timeout` (B1.4). DEFERRED.
|
||
|
||
Per the IMPASSABLE STOP rule: 0151 fix shipped (suite green 728/0), 0152 filed, STOPPED.
|
||
Resume B1.2's async examples once 0152 lands.
|
||
|
||
### Earlier — B1.0 + B1.1 complete
|
||
Stream A (atomics) is feature-complete (✅). Stream B1: **B1.0 + B1.1 complete.** The two
|
||
compiler-floor preconditions for the fiber runtime are in place: (1) `abi(.naked)` emits a
|
||
real LLVM `naked` function end-to-end (decl, generic, pack paths) — the context-switch
|
||
substrate; (2) per-fiber `context` root needs **no compiler change** — the spawn convention
|
||
(snapshot `context`, store, `push` it from the trampoline) is pure library sx. No
|
||
fibers/Io/scheduler code yet. Grounded floor facts:
|
||
- `context` is an implicit slot-0 `*Context` param + `push Context` is a stack `alloca` ⇒
|
||
**fiber-local for free** (confirmed by the B1.1 probe — never TLS, never re-read from the
|
||
`__sx_default_context` global mid-stack). A spawn passes the snapshot as the fiber-entry
|
||
fn's slot-0 ctx via `push f.root { entry(args) }`. Locked by `1804-...-context-snapshot`.
|
||
- Inline asm works end-to-end (lower→emit→JIT, aarch64 + x86_64) — the `.naked` body reuses it.
|
||
- **`.naked` with PARAMS works** (B1.0c, the B1.3 substrate): the param-alloca loop is gated
|
||
on `fd.abi != .naked` in decl.zig (both paths) + generic.zig — a naked fn's args stay in
|
||
ABI registers (read by the asm body), declared-but-unused in LLVM (verifier-legal).
|
||
Example `1803-concurrency-naked-asm-param.sx` (`add(a,b)` reads x0/x1). **Unsupported (loud,
|
||
not silent):** a `.naked` *variadic-pack* fn (pack.zig's param loop is intertwined with
|
||
comptime-param/`#insert` handling, and a naked fn can't read a runtime-sized pack from
|
||
registers anyway) → loud LLVM-verifier error for that nonsensical construct. Acceptable
|
||
boundary; a sharper sx diagnostic for it is a candidate polish, not a blocker.
|
||
|
||
## Next step
|
||
**Stream B1 is COMPLETE — no next step in this stream.** The pure-sx M:1 async runtime is feature-
|
||
complete and committed (1800–1820 green, 759/0), now WITH a `Scheduler.deinit` closing the bounded
|
||
leaks. Pick up **Stream B2** (channels / structured cancel / async stdlib) as a fresh carve
|
||
(PLAN-CHANNELS.md), OR one of the remaining non-blocking follow-ups: the linux `epoll` twin of
|
||
`block_on_fd`, `Future(void)`/`timeout` (needs issue 0150), or routing the suspending async through the
|
||
erased `context.io` for the M:N model. (`Scheduler.deinit` — DONE, see Last completed step.) None of
|
||
these block B1. The closure-env leak survives `deinit` (no language affordance to free a closure env);
|
||
revisit if/when sx grows closure-env ownership.
|
||
|
||
**Deferred (future B1.4c sibling): the linux epoll twin of `block_on_fd`.** B1.4c wired the **macOS
|
||
kqueue** path only (the host is aarch64-macOS). The linux mirror would register interest via
|
||
`std/net/epoll` and the OS-neutral facade is `std.event` — keep the two as separate run modes inside
|
||
`run`, branching on the platform, exactly as the timer-vs-fd modes are kept separate now. Documented
|
||
non-unification: virtual-time timers and real kqueue timeouts are NOT merged — `run` fires a pending
|
||
timer before ever blocking on kqueue (a program uses `sleep` OR fds); a true "fd-or-real-timeout" wants
|
||
a kqueue `EVFILT_TIMER`, future work.
|
||
|
||
> **▶ LINUX EPOLL — in progress (2026-06-26), via `std.event.Loop` (the OS-neutral facade).**
|
||
> Chosen over the sched.sx `block_on_fd` twin because the facade is the named home for epoll, is pure
|
||
> sx + libc (zero compiler change), is consumed by http.sx, and has a runnable darwin sibling. Landed:
|
||
> (A) **`library/modules/std/net/epoll.sx`** — raw bindings, the linux twin of `std/net/kqueue.sx`.
|
||
> `epoll_event` is modelled as an **arch-branched struct** (`{events, data_lo, data_hi}` u32 fields →
|
||
> 12 B x86_64 packed / 16 B aarch64), so layout is byte-exact with NO packed attribute, NO unaligned
|
||
> access, NO scalar-pointer indexing (issue 0155) — the struct-per-arch approach the user flagged as
|
||
> better than raw byte poking. Self-contained (libc only — NO build.sx import; the top-level `inline if
|
||
> ARCH` resolves via the compiler's flatten pre-pass, keeping the IR small). Locked by
|
||
> `examples/event/1633-event-epoll-bindings-linux.sx` (ir-only x86_64-linux, durable 244-line .ir;
|
||
> aarch64 16 B layout also probe-verified). (B) **`std.event.Loop` branched on `inline if OS`** into two
|
||
> top-level OS-selected structs (sx has no conditional struct fields): the kqueue Loop unchanged
|
||
> (darwin, runs — 1632 green), a new epoll Loop (linux) with the per-fd registration table (combined
|
||
> EPOLLIN/OUT mask via ADD/MOD/DEL), eventfd wake channel, and EPOLLRDHUP→eof. **Verified to LOWER**
|
||
> clean for both linux arches (every epoll syscall emits) + self-reviewed; NOT corpus-snapshotted (a
|
||
> Loop example drags the std barrel → ~18k-line brittle IR — documented in event.sx). Runtime validation
|
||
> pends a linux runner. **Remaining:** a linux CI run to validate end-to-end; optionally route sched.sx
|
||
> `block_on_fd` through `std.event` (still needs the linux sched.sx port — mmap consts, tramp symbol,
|
||
> errno, x86_64 SysV switch).
|
||
|
||
> **✅ issue 0192 FIXED (2026-06-26) — epoll work UNBLOCKED.** A qualified-import-member const
|
||
> (`m.EV_SIZE`) now folds as a compile-time constant in every position the bare/flat form does
|
||
> (array dim, arithmetic, Vector lane, generic value-param, inline-for) — so the clean
|
||
> `[MAXEV * ep.EV_SIZE]u8` event buffer the bindings want will work. Fix: a `lookupQualifiedConst`
|
||
> ctx hook resolving the namespace alias → target module's per-source const, wired into the int/float
|
||
> const folders (`src/ir/program_index.zig` + `src/ir/lower/comptime.zig`). Regression:
|
||
> `examples/modules/0842-modules-qualified-import-const-comptime.sx`. The hint stands for the rebuild:
|
||
> **a struct-per-arch `EpollEvent` (arch-branched u32 fields, 12 B x86_64 / 16 B aarch64) beat raw
|
||
> byte access** — idiomatic field reads, no issue-0155 scalar-pointer indexing, no unaligned u64.
|
||
> Resume: rebuild `std/net/epoll.sx`, branch `std.event.Loop` on `inline if OS`, lock with a darwin run
|
||
> + ir-only linux example.
|
||
|
||
> **⛔ (HISTORICAL) BLOCKED on issue 0192 (filed 2026-06-26).** Started the epoll work: chose the `std.event.Loop`
|
||
> backend (pure sx + libc externs, zero compiler change — per "do this in sx as much as possible") as
|
||
> the first deliverable, since event.sx already names epoll as its linux backend and it's runnable
|
||
> (darwin via kqueue) + ir-only-verifiable (linux). De-risked four landmines by probe — arch-dependent
|
||
> layout const via module-scope `inline if ARCH` (folds + validates in linux IR), slice-based byte access
|
||
> (sidesteps issue 0155), no unaligned u64 (store the 32-bit fd in epoll `data`), and comptime-dead linux
|
||
> externs don't break the darwin corpus (just an unreferenced `declare`). Then hit a compiler bug while
|
||
> sizing the event buffer: a **qualified-import-member const is not a compile-time constant** —
|
||
> `[m.CAP]u8` / `A :: m.CAP` fail (a *flat*-imported const works). Root cause located:
|
||
> `evalConstIntExpr` (`src/ir/program_index.zig:325`) has no namespace-member-const arm. Per the STOP
|
||
> rule the half-built `std/net/epoll.sx` (which used a struct-based layout to route around the bug) was
|
||
> **removed**, not landed — the unblock session rebuilds it cleanly with the fix in hand. Repro +
|
||
> investigation prompt: `issues/0192-qualified-import-const-not-comptime.{md,sx}`.
|
||
|
||
Design note carried forward: an event-loop `Io` needs a current-`Scheduler` handle. `sched.*` methods
|
||
thread it via `self`/the `Task`; if B1.4c wants the capability-threaded `context.io` form it'll need
|
||
an ambient current-scheduler accessor in sched.sx (still deferred — the `sched.*`-method form
|
||
suffices). The `Io` protocol's `poll`/`arm_timer` map onto this when/if that wiring is built.
|
||
|
||
**Side thread (optional, low priority): the SysV/Linux x86_64 sibling.** A THIRD switch variant
|
||
for `x86_64-linux`: SysV callee-saved = rbx, rbp, r12-r15 + rsp (6 GP + sp; **no** callee-saved
|
||
XMM, unlike Win64) — a 7-slot ctx, args rdi/rsi/rdx, the rsp-carried return addr. Needs a Linux
|
||
x86_64 host (or a working cross-run) to RUN + the mutual-scribble gate. Not blocking — the switch
|
||
is already validated on two arch/ABI pairs.
|
||
|
||
**Deferred (do NOT block on these):** issue **0150** (`void` struct field SIGTRAP) — only
|
||
`Future(void)`/`timeout` (B1.4). The **`::` callable-parameter feature** (named-fn async workers
|
||
`async(read_a, conn)`) — WIP at `.sx-tmp/wip-callable-params/patch.diff` (parser done, inference
|
||
incomplete); a dedicated effort; lambda workers are the idiom meanwhile.
|
||
|
||
`Context` layout settled: `{ allocator; data; io; }` (allocator index 0 fixed by
|
||
`call.zig:1229`, io last). Io protocol + materializers + push-inherit are LANDED + reviewed.
|
||
|
||
## Known issues / capability gaps
|
||
- **issue 0157 (OPEN, BLOCKING B1.4a)** — a user-defined generic ufcs method whose NAME collides
|
||
with a stdlib re-export (`cancel`, re-exported by `std.sx` from `io.sx` as `ufcs (f: *Future($R))`),
|
||
called via UFCS on a different generic struct (`*Task($R)`), leaves `$R` unresolved → `.unresolved`
|
||
reaches LLVM emission → panic (`src/backend/llvm/types.zig:196`). Renaming → works; the non-UFCS
|
||
call form already diagnoses `cannot infer generic type parameter 'R'`, so the UFCS path skips that
|
||
diagnostic. Surfaced by `cancel :: ufcs (t: *Task($R))` in `std/sched.sx`. Minimal repro (no
|
||
fibers/closures): `issues/0157-ufcs-generic-method-name-collides-stdlib-unresolved.{md,sx}`.
|
||
- **✅ issue 0154 — FIXED** (`null`/`---` to a struct field over-stored a whole-struct null when
|
||
the function's return type leaked as `target_type`, corrupting the frame → `ret` to 0x0;
|
||
surfaced building `Scheduler.init()`'s by-value return). Fix: `.null_literal`/`.undef_literal`
|
||
added to `needs_target` in `lowerAssignment` (`src/ir/lower/stmt.zig`). Regression:
|
||
`examples/types/0193`.
|
||
- **issue 0155 (OPEN, NON-blocking)** — indexing a scalar pointer (`pc[0]`, `pc: *i64`) panics
|
||
codegen (`.unresolved` reaching LLVM emission, `src/backend/llvm/types.zig:196`). Found in the
|
||
B1.5a review; the scheduler doesn't use it (array-field index + `.*` only). Filed for its own
|
||
session: `issues/0155-scalar-pointer-index-llvm-panic.{md,sx}`.
|
||
- **✅ issue 0158 — FIXED** — a plain `union` struct-literal (`b : Overlay = .{ f = 3.14 }`) fell
|
||
through the generic struct-literal path (`getStructFields` empty for a union → malformed
|
||
`structInit`, overlapping zero-fill clobbered the member → silent `0.0`). Fix: `lowerStructLiteral`
|
||
detects a plain-union target → new `lowerUnionLiteral` (`src/ir/lower/stmt.zig`) writes each named
|
||
member into a union-sized slot via the assignment-path lvalue resolver, then loads it back.
|
||
Single-arm only (one direct member, or same-arm promoted members); overlapping/different-arm/
|
||
positional literals are diagnosed. specs.md updated. Regressions: `examples/types/0194` +
|
||
`examples/diagnostics/1191`.
|
||
- **✅ issue 0157 — FIXED** (B1.4a) — a user generic `ufcs` method whose name collides with a
|
||
stdlib re-export resolved via last-wins `fn_ast_map` with no receiver filtering → wrong overload →
|
||
`$R` unbound → LLVM panic. Fix: `selectUfcsGenericByReceiver` (`src/ir/lower/call.zig`) — most
|
||
receiver-specific binding author across ALL module authors, deterministic, ambiguity-diagnosing.
|
||
Regression: `examples/generics/0217`.
|
||
- **✅ issue 0156 Part 1 — FIXED** (B1.4a) — single-type generic `$R` as a type-arg in a pack-fn
|
||
body (`Box($R)`/`size_of(Box($R))`) → `.unresolved` → panic. Fix: `comptime_pack_ref` arm in
|
||
`resolveTypeWithBindings`. Regression: `examples/generics/0216`.
|
||
- **Part 2 (OPEN, NON-blocking)** — a deferred `..` spread (a comptime pack captured into a
|
||
closure, or a tuple `..t` spread) crashes instead of working/diagnosing. The fiber async layer
|
||
avoids it by design (nullary thunks), so it's filed for its own session: `issues/0156`.
|
||
- **Heap leaks in the fiber runtime (documented limitations, NOT bugs):** `spawn`'s closure env +
|
||
`go`'s heap `Task` are never freed (sx exposes no closure-env free; Task ownership is deferred).
|
||
Bounded by spawn/go count, invisible under the default GPA. Revisit for a long-running
|
||
arena-backed scheduler.
|
||
- **✅ issue 0153 — FIXED** (re-exported generic value-failable `($R, !E)` kept its `!` channel:
|
||
`inferGenericReturnType` now pins return-type resolution to the fn's defining module).
|
||
Regression: `examples/1058`. Was the LAST B1.2 surface blocker.
|
||
- **✅ issue 0152 — FIXED** (`Atomic(bool)` sub-byte i1 atomic → byte-promoted to i8 in the
|
||
load/store emitters). Regression: `examples/1705`. Unblocked `Future.canceled`.
|
||
- **✅ issue 0151 — FIXED** (generic `$T` through generic-struct / pointer / UFCS-pack params).
|
||
Regression: `examples/0214` + `0215`. Was the original B1.2 surface blocker.
|
||
- **issue 0150** (deferred) — a `void` struct field crashes the compiler (unsized-type SIGTRAP
|
||
in LLVM `getTypeSizeInBits`). Blocks `Future(void)` → `timeout` (B1.4). Repro: `issues/0150-...`.
|
||
- (Note: **issue 0149**, filed by another session against an earlier dirty binary, was a
|
||
manifestation of the pre-fix 0151 — now moot.)
|
||
- **Orthogonal (not a B1 blocker):** default VALUES for comptime params don't bind on
|
||
generic-struct methods (free-fn defaults DO work) — inherited from Stream A. Only matters
|
||
if a B2 lib type wants a defaulted comptime param; atomics/fibers require explicit, so
|
||
unaffected.
|
||
- **Issue 0144 (open, independent):** calling an unrecognized bodiless `#builtin` silently
|
||
returns 0 / exit 0 — a silent-fallback footgun in the generic builtin-call path. Filed;
|
||
leave for its own fix session unless prioritized. Not a B1 blocker.
|
||
- **Deferred design gap (documented):** the B1.4 event-loop `Io` does not yet cooperate with
|
||
a platform UI run loop (CFRunLoop/NSRunLoop/ALooper); pinning gives thread-affinity, not
|
||
run-loop integration — a §6 app-target concern, out of B1 scope.
|
||
|
||
## Decisions (Stream B1 specifics; surface locked in design §4 / §4.6)
|
||
- **The async runtime is sx LIBRARY code.** The compiler provides only: the general
|
||
primitives (inline asm ✅, `abi(.naked)` naked [B1.0], atomics ✅) + fiber-safe codegen
|
||
(`context` already fiber-local — B1.1). Schedulers, fibers, channels, futures, `Io`
|
||
vtables, `mmap` stacks are all sx.
|
||
- **`abi(.naked)` is the real spelling of the design's `callconv(.naked)`** — postfix slot,
|
||
`name :: (sig) -> Ret abi(.naked) { asm { … }; }`. B1.0 = carry it into IR + emit LLVM
|
||
`naked` + skip prologue/ctx (mirror the existing `.c` skip), NOT extend the enum (it's
|
||
already there, just inert).
|
||
- **`.naked` ≠ `.c`:** a `.c` epilogue would restore SP from the wrong stack across a context
|
||
switch (SP-in ≠ SP-out by design). `.naked` = no prologue/epilogue/frame; the asm emits its
|
||
own `ret`. This is why the switch must be `.naked`.
|
||
- **Naming:** sx-facing name is **`naked`** (keyword `abi(.naked)`, field `is_naked`, the
|
||
diagnostic), matching LLVM's `naked` attribute and the industry term (Zig/Rust/GCC/Clang).
|
||
The ABI variant was renamed `.pure → .naked` (user direction): "pure" universally means
|
||
*side-effect-free*, the opposite of a register-clobbering context switch.
|
||
- **B1.0 snapshot scope:** a `.naked` body is raw per-arch asm; LLVM's `naked` attr text is
|
||
arch-invariant. **B1.0a** = one host example locked to the emit bail (host-independent —
|
||
fires before instruction selection; no `.build` pin). **B1.0b** = pin aarch64 + add an
|
||
x86_64 cross sibling (`.build` target-gated, ir-only on mismatch), like the asm corpus
|
||
split. The `.ir` proves the `naked` attr + asm emitted, NOT register-save correctness
|
||
(that's B1.3's stress harness).
|
||
- **B1.1 — per-fiber context is library-only (CONFIRMED by probe):** push frames are
|
||
stack-`alloca`'d and the implicit ctx rides slot 0, so the spawn convention — snapshot
|
||
`context`, store it, `push f.root { entry(args) }` from the trampoline — installs the
|
||
fiber's root with no compiler change. Verified: the body reads the snapshot over a different
|
||
ambient context, and `push` restores ambient on exit (`1804-...-context-snapshot`). The
|
||
design doc's "never raw TLS" guarded a non-problem (context was never TLS).
|
||
- **Test keystones (design §10):** the **B1.3 switch-stress harness** gates the
|
||
context-switch (the one piece the deterministic `Io` can't test — §8.1.1, §10.7); the
|
||
**B1.4 deterministic-sim `Io`** (calibrated against blocking `Io` — §8.1.3) gates all
|
||
scheduling tests. Both must exist + be calibrated before the async tests they gate are
|
||
trusted. `18xx` asserts program-emitted ordering contracts, not raw interleaving.
|
||
|
||
## Log
|
||
- **B1 follow-up — `Scheduler.deinit`.** Closes the bounded leaks B1 documented. Added a `task_allocs:
|
||
List(*void)` field (appended in `go` so the scheduler can reach its generic `Task($R)`s) + a canonical
|
||
`close` extern, then a terminal idempotent `deinit`: reap leftover ready fibers (`munmap` + free) →
|
||
free tracked Tasks → `List.deinit` the 3 backings → `close` the lazy kqueue fd (reset `-1`). Closure
|
||
envs stay unfreeable (documented). Probe-observed the accounting under a tracking GPA (deinit drives
|
||
live allocs 7→3 in a spawn+sleep+2×go run; residual = envs). Locked by
|
||
`1820-concurrency-fiber-scheduler-deinit.sx` (one run hits timers + kqueue fd + Tasks; `freed by
|
||
deinit: 5`, `live after deinit: 5` (env residual), `kq open after run: true`→`kq after deinit: -1`,
|
||
`read: 3 [97 98 99]`), `.build {"target":"macos"}`. Adversarial review: no real UAF/over-free in the
|
||
supported deinit-after-`run` path; reconciled a doc contradiction (terminal-contract wording); 0154
|
||
over-store concern probed + cleared (`kq == -1` right after `init`). Suite GREEN **759/0**.
|
||
- **B1.4c — real fd-readiness blocking via kqueue (macOS).** De-risked first with a no-scheduler probe
|
||
(confirmed `size_of(Kevent)==32` and the pipe→kevent roundtrip: `kq_wait` returned 1, `out.ident ==
|
||
read_fd`, `out.filter == -1`, `out.data == 1` — the struct layout reads the fd back correctly). Then
|
||
added to `library/modules/std/sched.sx` (importing the existing verified `std/net/kqueue.sx` as `kqb`
|
||
rather than re-deriving the FFI): a lazy `kq: i32` (-1 until first use), `io_waiters: List(IoWaiter)`,
|
||
`block_on_fd(fd, want_read)` (arm one-shot `EVFILT_READ`, record waiter, `suspend_self`), a run-loop
|
||
Mode 2 (block on `kq_wait(kq, evbuf, MAXEV=16, -1)` when only fd waiters remain, wake the fiber whose
|
||
fd fired), and `wake` now also evicts a stale fd-waiter (`cancel_io_waiter_for`, the same UAF guard as
|
||
`cancel_timer_for`). Timers keep precedence over fds (documented non-unification). Orphan-deadlock
|
||
check still fires for a genuine no-timer/no-fd suspend (probed: exit 134). Locked by
|
||
`1816-concurrency-fiber-io-pipe.sx` (reader blocks on empty pipe → writer writes `a b c` → kqueue
|
||
wakes reader → reads 3 bytes; `log: wrote read 3 [97 98 99]`, `n_suspended: 0`), `.build`
|
||
`{ "target": "macos" }`, runs end-to-end on host. The example's `read`/`write`/`close` externs use the
|
||
canonical signatures std already binds (extern-dedupe rejects a divergent re-binding). Suite GREEN
|
||
**754/0**. Next: B1.5 (end-to-end M:1 validation); linux epoll twin deferred.
|
||
- **carve** — wrote PLAN-FIBERS.md + CHECKPOINT-FIBERS.md. Grounded the B1 compiler floor:
|
||
`ABI.naked` inert (type_resolver.zig:237), IR `Function` has no naked flag (inst.zig:605),
|
||
attribute API pattern (emit_llvm.zig:1339 nounwind), `.c` ctx-skip precedent
|
||
(decl.zig:515), `push Context` stack-alloca + slot-0 implicit ctx (stmt.zig:1263,
|
||
lower.zig:259), `__sx_default_context` root (decl.zig:2667/2815), inline-asm corpus
|
||
(1645/1651). Corrected the design's `callconv(.naked)` → real `abi(.naked)` spelling and
|
||
the B1.0 snapshot story. B1.1 grounded as likely library-only. Baseline green (721/0).
|
||
- **B1.0a** — plumbed `Function.is_naked` (set from `fd.abi == .naked` at both decl sites);
|
||
`funcWantsImplicitCtx` skips `.naked` (no implicit ctx, like `.c`); both body-lowering
|
||
paths bypass `lowerValueBody` for `.naked` (asm body + `unreachable` cap — no sx return);
|
||
`emit_llvm` Pass 2 bails loudly on `func.is_naked`. `examples/1800-concurrency-naked-asm.sx`
|
||
locked to the bail (exit 1 + diagnostic). Suite green (722/0). (ABI variant later renamed
|
||
`.pure → .naked` — see the Naming decision above — so all `is_*`/`abi(.*)`/example names
|
||
here read `naked`.)
|
||
- **B1.0a review-hardening** — adversarial review found generic/pack Function-creation paths
|
||
left `is_naked` false (silent framed body for a generic `.naked` instance — returned 42 but
|
||
corrupted the stack). Fixed generic.zig + pack.zig (set `is_naked` + asm-only `unreachable`
|
||
cap); locked by `examples/1801-concurrency-naked-generic-bail.sx`. The review's `.naked`-
|
||
lambda CRITICAL was a false positive (unparseable — `isLambda` breaks on `abi`). Suite
|
||
green (723/0).
|
||
- **B1.0b** — real `naked` emission: emit_llvm declaration pass adds LLVM `naked`/`noinline`/
|
||
`nounwind` + skips `frame-pointer` for `func.is_naked`; Pass 2 emits the body verbatim (no
|
||
prologue). `1800` green aarch64-pinned (exit 42 + `.ir`); renamed `1801` → `-generic`
|
||
(generic `.naked` emits a naked body, exit 42); added x86_64 sibling `1802` (ir-only, `.ir`
|
||
locks `naked` + `movl $42, %eax`). Unit test asserts `naked` present + `frame-pointer`
|
||
absent. Suite green (724/0).
|
||
- **B1.0c** — review-hardening: param-bearing `.naked` emitted invalid LLVM (loud verifier
|
||
error). Gated the param-alloca loop on `fd.abi != .naked` (decl.zig both paths + generic.zig)
|
||
— naked args stay in registers, read by the asm body (the B1.3 context-switch shape).
|
||
Locked by `examples/1803-concurrency-naked-asm-param.sx`. Pack `.naked` left unsupported
|
||
(loud, nonsensical). **B1.0 complete.** Suite green (725/0).
|
||
- **rename** — ABI variant `.pure → .naked` (keyword, `Function.is_naked`, diagnostics,
|
||
examples 1800-1803 `*-pure-* → *-naked-*`, docs). "pure" universally means side-effect-free
|
||
— wrong for a register-clobbering switch; "naked" matches LLVM/Zig/Rust/GCC/Clang. Pure
|
||
cosmetics, no semantic change. Suite green (725/0).
|
||
- **B1.1** — per-fiber `context` root: **zero compiler change** (probe-confirmed). The spawn
|
||
convention (snapshot `context` → store in a struct → `push f.root { entry() }` from the
|
||
trampoline) installs the fiber's root via the implicit slot-0 `*Context` param; the body
|
||
reads the snapshot, not the trampoline's ambient ctx, and the `push` scope restores ambient
|
||
on exit. Locked by `examples/1804-concurrency-context-snapshot.sx` (prints `fiber root: 42`
|
||
/ `ambient after: 99`). Suite green (726/0). **Next: B1.2 (Io interface + context.io).**
|
||
- **B1.2 (BLOCKED)** — built the full `Io` capability (protocol on `Context`, stateless
|
||
`CBlockingIo` blocking default, both `__sx_default_context` materializers, push-inherit-omitted
|
||
fix, `!`-impl-method warning fix) and VERIFIED the core works live (`context.io.now_ms()` →
|
||
`clock ok`). Two independent compiler bugs blocked the `async`/`await`/`timeout` layer:
|
||
**0150** (`void` struct field → unsized SIGTRAP, blocks `Future(void)`) and **0151** (type-var
|
||
from a fn-ptr param's return type not bound in the body, blocks `async`'s `Future(R)`). Both
|
||
filed with standalone repros + investigation prompts. Per the STOP rule: reverted ALL B1.2
|
||
working changes (master green again, 726/0; the dirty binary had broken the photo project —
|
||
see the now-moot 0149), saved WIP to `.sx-tmp/b12-wip/`, STOPPED. Resume after 0150 + 0151.
|
||
- **0151 FIXED** — generic inference now binds `$T` through a generic-struct param head, a
|
||
pointer (`*Box($T)`, incl. UFCS auto-ref), and a closure-return-via-pack on the UFCS path.
|
||
Four gaps closed: `parameterized_type_expr` arm in `extractTypeParam`/`matchTypeParam(Static)`
|
||
(recovers the arg instance's recorded per-param bindings, recurses positionally); pointer arm
|
||
falls through to match a value arg (auto-address-of); `ExprTyper.inferType` `.lambda` arm
|
||
(closure type from annotations — UFCS types args from raw AST pre-lowering); pack UFCS target
|
||
routes through `lowerPackFnCall` with the receiver spliced in as `args[0]`. Issue 0151 marked
|
||
RESOLVED; repro → `examples/0214-generics-ufcs-closure-return-pack.sx`; widened cases →
|
||
`examples/0215-generics-infer-through-pointer.sx`. Suite green 728/0. The now-callable async
|
||
surface immediately exposed a SEPARATE codegen bug — **issue 0152** (`Atomic(bool)` → sub-byte
|
||
i1 atomic, LLVM reject; `Future.canceled` hits it). Filed with standalone repro + fix prompt.
|
||
Per the STOP rule: shipped the 0151 fix, filed 0152, STOPPED. Resume the async examples
|
||
(1805/1806) after 0152.
|
||
- **0152 FIXED** — the atomic load/store emitters (`src/backend/llvm/ops.zig`) byte-promote a
|
||
sub-byte (`bool`→`i1`) access to its `i8` storage type and `trunc`/`zext` the value at the
|
||
boundary (new `atomicByteType` helper). rmw/cmpxchg left as-is (a `bool` rmw/CAS is rejected
|
||
at the sx level — integer-only — so a sub-byte element never reaches them; comments record
|
||
this). Regression `examples/1705-atomics-bool-byte-promoted.sx` (load/store round-trip). Issue
|
||
0152 marked RESOLVED. Suite green 729/0. With `Atomic(bool)` working, the async surface
|
||
exposed the TRUE remaining blocker — **issue 0153**: a re-exported generic value-failable
|
||
`($R, !E)` loses its `!` channel at the call site (the earlier "secondary `or` PHI" symptom
|
||
was this, NOT an `Atomic` cascade — confirmed it persists after 0152). Narrowed to the
|
||
generic+re-export co-requirement (non-generic re-export OK; direct generic import OK; only the
|
||
combination drops `!`). Root cause: the monomorphized return-type's error-set, reached via the
|
||
re-export alias, resolves to a non-`.error_set` TypeId, so `errorChannelOf`
|
||
(`lower/error.zig:148`) misses the channel. Filed `issues/0153-...` with a minimal co-located
|
||
2-file repro + a single-file stdlib-`await` repro + investigation prompt. Per the STOP rule:
|
||
shipped the 0152 fix, filed 0153, STOPPED. Resume the async examples after 0153.
|
||
- **0153 FIXED → B1.2 COMPLETE** — `inferGenericReturnType` (`src/ir/generics.zig`) resolved the
|
||
return-type AST in the CALL-SITE module, so a re-exported error set (`LE :: lib.LE`) resolved
|
||
to a non-`.error_set` alias and the planned call-result was a plain tuple (channel lost). Fix:
|
||
pin the source to `fd.body.source_file` around the return-type resolution, exactly as
|
||
`monomorphizeFunction` does — the `!E` now resolves to the real `.error_set`. One-function
|
||
change; full suite green (732/0), no regression. Issue 0153 RESOLVED; repro →
|
||
`examples/1058-errors-reexport-value-failable-channel.sx` (+ companion `lib.sx`). With the
|
||
channel preserved, landed the async examples: **`1805`** (`async`/`await` + `now_ms` → `sum:
|
||
42` / `double: 42` / `clock ok`) + **`1806`** (`cancel` → `await` raises `.Canceled` → `or`
|
||
default; `ok: 7` / `canceled: -99`). **B1.2 (Io capability + M:1 async surface) is COMPLETE.**
|
||
Next: B1.3 (fiber runtime) on the `.naked` context-switch substrate.
|
||
- **B1.3a-1 — context switch works.** Implemented the stackful switch in pure sx over
|
||
`abi(.naked)`: `swap_context(from, to)` (save callee-saved x19-x28 + fp/lr + sp into `*from`,
|
||
load from `*to`, `ret` onto `to`'s stack) + by-hand fiber bootstrap (SP = top of an
|
||
`alloc_bytes` stack, LR = a `.global _fib_tramp` global-asm trampoline that does `mov x0, x19;
|
||
bl _fib_body`, x19 = `*Fiber`). Proven via a probe (main↔fiber), then locked by
|
||
`examples/1807-concurrency-fiber-context-switch.sx` (aarch64-pinned): a 2-fiber ping-pong
|
||
(`rounds: 6`, `canary fails: 0` — a per-fiber stack canary survives every switch) + a 64-frame
|
||
deep recursive chain suspended at the bottom and resumed (`frames verified: 64` / `depth fails:
|
||
0`). The `bl _fib_body` reaches the sx body via `export "fib_body"` (the 1655 asm→sx pattern);
|
||
runs under JIT, ir-only on a non-arm host (`.ir` captured — `swap_context` shows `naked noinline
|
||
nounwind`). Suite green 733/0. **Honest scope:** indirect register/stack survival only; the
|
||
EXPLICIT every-callee-saved + FP scribble (§10.7) is B1.3a-2, still owed. Next: B1.3a-2.
|
||
- **B1.3a-2 — the §10.7 stress gate, adversarially reviewed.** Extended `swap_context` to the
|
||
COMPLETE AAPCS64 callee-saved set (added FP d8-d15 → 21-slot ctx) and wrote a naked
|
||
`scribble_verify` that loads a unique sentinel into all 18 callee-saved regs, yields, and counts
|
||
non-survivors on resume (176-byte frame saves/restores the caller's callee-saved + base; lr
|
||
round-trips the swap). The gate is a 2-fiber MUTUAL scribble (each clobbers the other's regs, so
|
||
survival ⇒ the switch saved+restored them). Locked by
|
||
`examples/1808-concurrency-fiber-switch-stress.sx` (`A/B mismatches: 0`). Validity proven by
|
||
negative controls (drop d8-d15 → 8/8; drop x27/x28 → 2/2). **Spawned an adversarial-review
|
||
worker (per the plan + user request): NO critical bugs** — callee-saved set complete (x18 rightly
|
||
excluded; d8-d15 suffices per §6.1.2), offsets/alignment/lr-sp dance all verified. Applied its
|
||
one rec: `boot` zeroes FP ctx slots so first-entry loads 0, not garbage. Honest residual gaps
|
||
(spec-correct for a call-boundary swap; in the example header): FPCR/FPSR/NZCV + TPIDR/TLS not
|
||
swapped, fp=0 blocks unwind — relevant at N×M:1 / signals, not here. Suite green 734/0.
|
||
Next: B1.3b (x86_64 sibling + mmap guard-page stacks).
|
||
- **B1.3b — mmap guard-page stacks (x86_64 sibling deferred).** Fiber stacks now `mmap` a
|
||
`[guard | usable]` region and `mprotect` the low 16KB page `PROT_NONE`, so a stack overflow
|
||
faults at the guard boundary instead of silently corrupting a neighbor (§8.1.1). Locked by
|
||
`examples/1809-concurrency-fiber-guard-stack.sx` (aarch64-macos-pinned): `guard armed: 1`
|
||
(`mprotect`→0) + `sum: 20100` (a fiber runs real recursion on the guarded stack + yields).
|
||
Guard FIRING validated manually (overflow → `Bus error` at `region+GUARD`, exit 134 via the sx
|
||
crash handler) — not corpus-pinned because a deliberate-overflow crash is host-fragile (and a
|
||
mere "child faulted" fork test wouldn't prove the BOUNDARY catch). The x86_64 `swap_context`
|
||
sibling was DEFERRED: `--target x86_64-macos` mislinks on this arm64 host and `x86_64-linux`
|
||
can't run here, so it could only ship un-run/un-negative-controlled — which §10.7 forbids for the
|
||
highest-risk asm. SysV target notes (rbx/rbp/r12-r15/rsp, no callee-saved XMM, rsp-carried return
|
||
addr) recorded in Next step. Suite green **735/0**. Next: x86_64 sibling (needs an x86_64 host)
|
||
OR B1.4 (`Io` impls / scheduler) on the proven aarch64 substrate.
|
||
- **B1.3b-1 — x86_64 / Win64 switch sibling VALIDATED on real hardware.** The user provided a
|
||
Windows 7 x64 VM (UTM), so the x86_64 switch became RUNNABLE (as Win64). Validated the
|
||
cross-build→VM→run loop (`--target x86_64-windows-gnu --self-contained` → PE32+; output via the
|
||
Win32 `WriteFile` boundary, the 1660 pattern). Wrote a Win64 `swap_context` (8 GP rbx/rbp/rdi/
|
||
rsi/r12-r15 + rsp + **xmm6-xmm15** via `movups` — Win64 has callee-saved XMM) + a Win64
|
||
`scribble_verify` (264-byte frame, 32-byte shadow + 16-align at each `call`, COFF symbols,
|
||
rsp-carried return addr) driving the 2-fiber mutual scribble. **Adversarially reviewed (worker
|
||
emitted the real `.s`, verified every alignment/offset/round-trip against the Win64 ABI — no
|
||
critical/minor bugs), THEN run on the VM → `0 0 P`** (all 8 GP + 10 XMM callee-saved survived).
|
||
Locked by `examples/1810-concurrency-fiber-switch-win64.sx` (pinned `x86_64-windows-gnu`,
|
||
ir-only on this host; the VM run is the runtime-correctness provenance). Good-swap-only (the
|
||
in-process negative control was dropped to avoid an sx fn-ptr-convention rabbit hole; the
|
||
detection of this exact logic was negative-controlled on aarch64 in 1808). Suite green **736/0**.
|
||
The B1.3 context switch is now proven on TWO arch/ABI pairs. Next: **B1.4** (Io impls / M:1
|
||
scheduler) on the proven substrate. (Side thread: the SysV/Linux x86_64 sibling, when a Linux
|
||
x86_64 host is available.)
|
||
- **B1.5a — M:1 scheduler CORE + a fixed blocker bug.** Built `library/modules/std/sched.sx`: a
|
||
generic `Fiber`/`Scheduler` over `swap_context` on guarded `mmap` stacks. `spawn` heap-allocs a
|
||
fiber, bootstraps its ctx, enqueues it; the ONE generic dispatch (`fib_dispatch` via `_fib_tramp`)
|
||
runs ANY stored `Closure() -> void` on a fresh stack (replacing the fixed `bl _fib_body`);
|
||
`yield_now` round-robins, `suspend_self`/`wake` park/resume off-queue, `run` drives to drain +
|
||
reaps `.done` fibers (`munmap` + free). **De-risked first by probe** (closure-on-fiber + output
|
||
via captured pointer). **Hit blocker bug 0154** (user-authorized fix): `null`/`---` to a struct
|
||
field over-stored a whole-struct null when the fn return type leaked as `target_type`, corrupting
|
||
the frame (`ret` 0x0) — exactly the `Scheduler.init()` by-value-return shape. Fixed in `stmt.zig`
|
||
(`needs_target` += `null`/`undef` literals); regression `examples/types/0193`; `0154` RESOLVED.
|
||
**Adversarial review:** asm/bootstrap/lifetime sound (env-lifetime fear disproven — heap-promoted);
|
||
1 CRITICAL (`wake` re-enqueue → FIFO segfault) + robustness gaps ALL hardened (wake guarded on
|
||
`.suspended`, `n_suspended` deadlock diagnostic+abort, loud mmap/mprotect/OOM bails, env-leak
|
||
documented). Locked `1811` (round-robin `0 1 2 ×3`) + `1812` (suspend/wake + spurious-wake guard,
|
||
`log: 10 20 21 11`). Filed NON-blocking `0155` (scalar-pointer index panics codegen — review
|
||
incidental, unused by sched). Suite GREEN **748/0**. Next: **B1.4a** (FiberIo).
|
||
- **B1.4a (truly-suspending fiber-task async, nullary-thunk design) — BLOCKED on issue 0157.**
|
||
Implemented the async layer SELF-CONTAINED in `library/modules/std/sched.sx` (kept its lone
|
||
`#import "modules/std.sx"` to avoid the duplicate-`_fib_tramp` trap): `TaskState`, a LOCAL
|
||
`TaskErr :: error { Canceled }` (the re-exported `IoErr` alias is NOT seen through by the
|
||
`raise`/failable-type check — verified), `Task($R)`, and `go`/`wait`/`cancel` ufcs. Design is
|
||
the validated nullary-thunk (`.sx-tmp/pnullary.sx` → `log: 1 2 3 42 100`): `work` is a
|
||
`Closure() -> $R`, user captures inputs at the call site, NO `..args` crosses the fiber boundary
|
||
(deliberately sidesteps 0156). `go`+`wait` run correctly; both wake-orderings traced. Wrote the
|
||
example `examples/concurrency/1813-concurrency-fiber-async-suspend.sx` (+ `{ "target": "macos" }`
|
||
`.build`) but its `cancel` ufcs surfaced a NEW compiler bug — issue **0157**: a user generic
|
||
ufcs whose name collides with a stdlib re-export (`cancel` from io.sx) is mis-resolved on UFCS
|
||
call over a different generic struct, leaving `$R` unresolved → LLVM panic. Bisected to a minimal
|
||
no-fiber repro (name is the sole trigger; non-UFCS form diagnoses correctly). Example NOT seeded
|
||
into the corpus (no `.exit` marker) — do NOT regen its goldens until 0157 lands. Per the STOP
|
||
rule: filed `issues/0157-*.{md,sx}`, marked state BLOCKED, paused.
|
||
- **B1.4a COMPLETE (this session) — suspending fiber-task async + two compiler fixes.** Built the
|
||
`Task($R)` + `go`/`wait`/`cancel` layer in `sched.sx` (nullary-thunk design; self-contained to
|
||
avoid the `_fib_tramp` duplicate-symbol trap). Locked `1813` (`sequence: 1 2 3 42 100 -99`).
|
||
FIXED the two blockers the worker had filed: **0156 Part 1** (`comptime_pack_ref` arm in
|
||
`resolveTypeWithBindings`; regression `0216`) and **0157** (receiver-driven UFCS overload
|
||
selection `selectUfcsGenericByReceiver`; regression `0217`). Adversarial review of the 0157 fix +
|
||
Task layer found a determinism CRITICAL (always-run selection + specificity + ambiguity
|
||
diagnostic), a `wait`-outside-fiber null-deref (loud guard), and cancel-not-skipping-work (skip
|
||
if pre-canceled) — all fixed. Simplified `1812` (`**Fiber` → `Sh.parked`). 0156 Part 2 reframed
|
||
OPEN/non-blocking. Suite GREEN **751/0**. Next: B1.4b (deterministic-sim `Io`, the KEYSTONE).
|
||
- **B1.4b COMPLETE (this session) — deterministic virtual-time timers + a CRITICAL UAF fix.** Added
|
||
`clock_ms`/`timers`/`now_ms`/`sleep` + a timer-driven `run` to `sched.sx` (worker-built): fibers
|
||
sleep in reproducible simulated time, waking in deadline order (FIFO tiebreak). Locked `1814`
|
||
(5 fibers, wake order B@10/D@15/E@15/C@20/A@30). Adversarial review of the run-loop change found a
|
||
CRITICAL use-after-free — a fiber woken EARLY (manual/Task `wake`) before its `sleep` timer fired
|
||
was reaped while its `Timer` kept a dangling `*Fiber`; a later fire dereferenced freed memory
|
||
(silent "pass" only by luck). Fixed: `wake` evicts the fiber's pending timer (`cancel_timer_for`);
|
||
regression `1815` (early wake → `clock: 0`, stale timer never fires). Review cleared n_suspended
|
||
accounting, deadlock false-positives, timer-list integrity, clock monotonicity, termination.
|
||
Suite GREEN **753/0**. Next: B1.4c (event-loop `Io`, kqueue/epoll).
|
||
- **B1.4c COMPLETE (this session) — real fd readiness via kqueue + 2 CRITICAL review fixes.** Added a
|
||
lazy `kq` + `io_waiters` + `block_on_fd` + a kqueue-blocking run-loop Mode 2 to `sched.sx`
|
||
(worker-built, reusing `std/net/kqueue.sx`). Adversarial review found two CRITICALs: same-fd
|
||
lost-wakeup hang (FIXED — `block_on_fd` enforces one-waiter-per-fd with a loud abort) and a
|
||
never-ready-fd "hang" (RECLASSIFIED as correct event-loop semantics; misleading orphan-check comment
|
||
corrected). Locked `1816` (pipe block→kqueue-wake→read). Suite green 754/0.
|
||
- **B1.5 COMPLETE → STREAM B1 DONE (this session).** Capstone `1817` composes the whole stack
|
||
(`go`/`wait` + `sleep`/`now_ms` + scheduler) — three tasks complete in DEADLINE order
|
||
(task 2@10 / 3@20 / 1@30), `sum: 123`, final virtual clock 30. The pure-sx colorblind M:1 async
|
||
runtime is feature-complete end-to-end (1800–1817), all adversarially reviewed. Suite GREEN
|
||
**755/0**. Five compiler bugs fixed across the stream (0151/0152/0153/0154/0156-P1/0157 — 0151-3 in
|
||
B1.2). Next carve: Stream B2 (channels / cancel / async stdlib).
|
||
- **Post-review hardening (this session) — 6 findings from an adversarial review of the B1 commits.**
|
||
Fixed: **P1-a** the UFCS generic PLANNER (`calls.zig`) used the last-wins `fn_ast_map` winner while
|
||
lowering reselected by receiver → plan/lowering could disagree and MISBOX the result; now both share
|
||
`selectUfcsGenericByReceiver`. **P1-b** the selection scanned `module_decls` globally, flagging a
|
||
transitively-hidden same-named overload as a FALSE ambiguity; now two-tier — directly-visible authors
|
||
first (ambiguity only among those), global fallback for receiver-reachable namespaced methods (e.g.
|
||
`Task.cancel`) that defers to `fd0` on a hidden tie. **P2-b** boolean specificity tied `*$T` with
|
||
`*Box($T)`; now peels pointer layers so the structurally-narrower receiver wins. **P1-c** a second
|
||
concurrent `Task.wait` overwrote the single waiter slot → silent deadlock; now one-awaiter-per-task
|
||
loud abort. **P2-c** `sleep(negative)` rewound the virtual clock; now rejected loudly. (**P2-a**
|
||
non-generic-winner-hides-generic did not reproduce — the non-generic arm already falls through.)
|
||
Regressions: `examples/generics/0218` (receiver specificity + plan/lowering agreement),
|
||
`examples/concurrency/1818` (negative-sleep abort), `1819` (double-wait abort). Suite GREEN **758/0**.
|