sx/CLAUDE.md

sx compiler — session instructions

IMPASSIBLE RULES — no exceptions

When you hit a sx compiler bug during normal work

STOP. File the issue. Wait for a fix in another session. Do NOT work around the bug. Do NOT continue with adjacent work. Do NOT land code that depends on the not-yet-fixed behaviour.

Procedure:

1. Create issues/NNNN-slug.md (next free NNNN, see existing issues/).
2. The file must contain:
   • Symptom — one-line summary + observed vs expected.
   • Reproduction — minimal sx code (inline fenced block). Must reproduce the bug standalone, no project dependencies beyond modules/std.sx / modules/allocators.sx.
   • Investigation prompt — a ready-to-paste prompt the user can drop into a fresh session to fix the bug. Should include: the suspected area of the compiler (file + function), what the fix likely needs to do, and the verification step (run the repro, expect new output).
3. Update current/CHECKPOINT-MEM.md (or the relevant stream's checkpoint) — mark ## Current state as BLOCKED on issue NNNN.
4. Tell the user: bug filed at issues/NNNN-slug.md, work paused pending fix.
5. STOP. Do not migrate, do not refactor, do not look for an alternative path. The session ends until the user returns with confirmation the fix landed.

This rule is uncontestable. None of the following override the stop:

• "But the workaround is small."
• "But this just affects one file."
• "But I can route around it."
• "But the bug is pre-existing — not introduced by my change."
• "But it doesn't block what I was just doing — only future work."
• "But I already finished the step I was on; this is adjacent."
• "But the fix is obviously safe to land alongside."

If filing an issue is on the table, the answer is STOP. Period. Do not weigh "blocking vs non-blocking". Do not bundle the issue filing with continued work in the same session. Do not finalise the checkpoint, regen snapshots, or move to the next phase after filing. Filing the issue IS the last action of the session.

Every workaround during execution becomes technical debt that hides the bug from the next person who hits it. Every "I'll just keep going since it's pre-existing" leaves the next session believing the bug is gated behind their work and not yours. The cost of stopping is one paused session; the cost of working around — or "just finishing this last thing" — is permanent silent fragility.

If you genuinely think the bug is not a bug (e.g. you've misunderstood specs.md), STILL file the issue with that hypothesis in the prompt — let the fix session confirm or correct.

The two acceptable actions after recognising a compiler bug:

1. File issues/NNNN-slug.md, mark the checkpoint BLOCKED, stop.
2. If the bug surfaced AFTER you've already shipped a complete step (built + tests pass + checkpoint logged), still file as in (1) and still stop — do not roll forward into the next step.

There is no "wrap up first" option.

REJECTED PATTERNS — never generate these

Silent fallback defaults in the compiler

❌ Forbidden: returning a "reasonable-looking" default value when a lookup fails in the compiler. Examples of the pattern to root out:

// NEVER write this — lookup fails, return s64 and pretend nothing
// happened. Any caller asking `what type is this?` gets a lie.
return self.module.types.findByName(name_id) orelse .s64;

// NEVER write this — same shape, dressed up:
return scope.lookup(name) orelse default_type;
return resolved_field orelse .void;

These defaults silently produce wrong results in cases the implementer didn't think of. The classic failure mode: the default coincidentally matches the size/shape of one common case, so the test suite passes and the bug ships invisibly. issue-0042 lived for years because resolveTypeArg's orelse .s64 returned 8 bytes for unresolved type-alias names — coincidentally correct for any 8-byte target (s64, *T, f64, function pointers), and silently wrong for everything else.

✅ Required: when a lookup that must succeed fails, emit a diagnostic via self.diagnostics.addFmt(.err, span, "...", .{...}) and return a dedicated, unmistakable sentinel — one that can never be confused with a legitimate result — or a ?T return that forces the caller to handle the null. Errors must surface to the user as text, not as a silently-corrupted size or alignment.

❌ .void is an UNACCEPTABLE sentinel for a failed type lookup. void is a real, heavily-checked type (void returns, void params, "no value" markers), and pervasive if (ty == .void) { skip / return-nothing } checks would silently swallow the failure — trading one silent default (.s64) for another (.void) one layer down. The same objection rules out noreturn (diverging expressions) and any other load-bearing builtin. Instead, add a distinct .unresolved-style TypeId whose sole meaning is "resolution failed". A dedicated value (1) can't be mistaken for a real type by any downstream check, (2) makes the exhaustive switches in the type table fail to compile until every site handles it (forcing coverage), and (3) can be a hard tripwire — if (ty == .unresolved) @panic(...) in codegen/emit guarantees it never silently ships. The one-time plumbing cost is exactly the trade this file mandates ("the field plumbing is a one-time cost; silent-clobber debugging is forever"). The same principle applies to non-type sentinels: prefer a Ref.none-style value that is distinct from every valid result, not a real one that "looks broken enough".

If you find an existing default-return in the compiler that swallows a lookup failure, treat it as a discovered bug — file an issue per the IMPASSIBLE RULES above, do not just delete the default in place without surfacing what it was hiding.

Silent unimplemented arms (catch-all else branches)

❌ Forbidden: a switch / if-chain over a Value tag, Op variant, TypeId, etc. whose else branch silently does the wrong thing — returns the input unchanged, returns a zero/null/undef default, picks one common width and writes that many bytes, swallows an error into .void_val so the caller fills a zero-init const, etc. The pattern is identical in spirit to the silent-fallback-defaults rule above: a case the implementer didn't think of falls through to behaviour that looks like it worked but corrupts the data downstream.

Examples of patterns we've burned ourselves on:

// NEVER write this — `.int` value at a raw destination, write 8
// bytes regardless of actual IR type. Silently clobbers neighbors
// when the destination is sub-8.
.int => |v| {
    const bytes = std.mem.toBytes(v);
    @memcpy(dst[0..bytes.len], &bytes);
},

// NEVER write this — `.deref` of anything but slot_ptr passes the
// value through unchanged. Looks like a successful deref to callers;
// silently wrong for raw pointers.
.deref => |u| switch (frame.getRef(u.operand)) {
    .slot_ptr => |s| return frame.loadSlot(s),
    else => return val,  // ← silently wrong
},

// NEVER write this — comptime init error becomes void_val, which the
// LLVM emitter happily turns into a zero-init constant. The user sees
// the const evaluating to 0 with no diagnostic.
const result = interp.call(func_id, &.{}) catch .void_val;

✅ Required: either (1) implement the arm correctly in the same step as the one that introduces the new shape, or (2) bail loudly with a one-line diagnostic that names the specific case. For the interp we have bailDetail(comptime msg) which sets Interpreter.last_bail_detail so the host diagnostic surfaces "op=X: " instead of a bare CannotEvalComptime. Mirror the same pattern in any new evaluator / interpreter / serializer.

Preferred order: implement the arm. Only fall back to "bail loudly" when the implementation requires plumbing that's out of scope for the current step. In that case, leave a one-line comment explaining what would be needed to implement it properly — so the next person hitting the diagnostic has a head-start.

If a path requires width / type / layout information that isn't threaded into the IR op yet, prefer to add the field to the op struct (Store.val_ty-style) over leaving an "8 bytes assumed" shortcut. The field plumbing is a one-time cost; silent-clobber debugging is forever.

When in doubt: else => return bailDetail("clear one-line reason") beats else => unreachable beats else => /* hope */.

Allocator construction

✅ Required shape: init returns the concrete state by value. The caller binds it to a local (or embeds it in a struct field); that local IS the allocator's storage. xx local borrows the local's address into the Allocator protocol value — no heap allocation for the state struct, no free of the state needed, no caller-provides- storage ceremony at the call site.

gpa := GPA.init();                          // GPA (value, stack-local)
arena := Arena.init(xx gpa, 4096);          // Arena (value)
tracker := TrackingAllocator.init(xx gpa);  // TrackingAllocator (value)

push Context.{ allocator = xx tracker, data = null } { ... }

print("gpa allocs: {}\n", gpa.alloc_count);   // direct field access
tracker.report();                              // direct method call
arena.reset();                                 // direct method call
arena.deinit();                                // frees the chunks; the
                                               // Arena struct itself
                                               // goes away with the local

Why by-value:

• No state-struct leak. The local is reclaimed when its scope ends; no explicit deinit is needed to free the struct (chunks/buffers the allocator manages downstream still need cleanup — that's orthogonal).
• One fewer libc_malloc per allocator instance.
• Composition stays clean: a struct that owns an allocator embeds it directly (arena_a: Arena;) rather than holding a pointer (arena_a: *Arena;). The owning struct's heap-alloc covers it.
• xx local is borrow-mode under sx's protocol-erasure rule (see specs.md §3 — Ownership and Lifetime). Mutations through the protocol are visible to the local.

❌ Forbidden: the manual "caller provides storage" pattern, because it pushes raw-struct construction at the user. This is a different shape from the value-return rule above — the user writes out the type, declares uninitialised state, and invokes a separate create/init_in_place that mutates it. Verbose, fragile, easy to forget the init step:

// NEVER write this — explicit @ptr:
g_gpa : GPA = ---;
GPA.create(@g_gpa);

// NEVER write this — UFCS-disguised same pattern:
gpa_state : GPA = .{ alloc_count = 0 };
gpa_state.create();

// NEVER write this — in-place init on a struct field:
self.arena_a.create(parent, size);

The value-return pattern subsumes these use cases without the gotcha: gpa := GPA.init(); already gives the caller a local; if they want the storage in a struct field, self.arena_a = Arena.init(parent, size); works directly.

❌ Also forbidden: wrapping an init result through a cast just to bind a "typed pointer" you don't actually need (it's a value now):

// NEVER write this — tracker is already a TrackingAllocator value:
tracker := TrackingAllocator.init(xx gpa);
t : *TrackingAllocator = xx @tracker;   // redundant rename
t.report();

Call methods directly on the local — tracker.report(); works via UFCS auto-address-of, no manual pointer juggling required.

When migrating an existing allocator from the old init() -> *T shape to the new init() -> T, also drop the trailing parent.dealloc(xx a) from any deinit — the caller's local owns the storage now, deinit only frees downstream resources (chunks, counters' backing, etc.).

Long-lived containers growing through context.allocator

A struct's lifetime can outlast its caller's current context.allocator. When that happens, any internal allocation made via context.allocator (directly, or via a List.append(item) that uses the default) binds to a transient allocator, and dies the moment that transient scope is torn down — even though the owning struct is still alive and reachable.

❌ Forbidden: the implicit-context capture in any struct whose lifetime crosses a push Context { ... } boundary:

LongLived :: struct {
    items: List(Entry);

    add :: (self: *LongLived, e: Entry) {
        // BAD — `items` grows through whichever allocator happens to
        // be current at this call. If the caller is inside a transient
        // `push Context { allocator = ... }`, the new backing lives in
        // that transient allocator. When the push scope ends, the
        // backing is freed/reset, but `items.items` still points at it.
        self.items.append(e);
    }
}

The same trap applies to direct context.allocator.alloc(...) / .dealloc(...) calls inside such structs.

✅ Required: capture the owning allocator at construction time and forward it explicitly to every internal growth point. The container's API supports this directly — List(T)'s mutations take an optional trailing alloc: Allocator = context.allocator:

LongLived :: struct {
    items: List(Entry);
    own_allocator: Allocator;

    init :: (self: *LongLived) {
        // Snapshot whatever allocator is in scope at construction.
        // That same allocator must outlive this struct.
        self.own_allocator = context.allocator;
    }

    add :: (self: *LongLived, e: Entry) {
        self.items.append(e, self.own_allocator);
    }

    // Direct allocs too:
    grow_buf :: (self: *LongLived, n: s64) {
        self.buf = self.own_allocator.alloc(n);
    }
}

Two-question test for whether a struct needs this pattern:

1. Can a caller's push Context { ... } wrap a method on this struct?
2. Does any method allocate (directly, or by triggering List growth)?

If both yes, capture the owning allocator at init. Field name is by convention (parent_allocator, owner, own_allocator — pick the project's existing one and follow it).

Sibling case (do NOT migrate): a container whose backing is intentionally tied to the caller's scope — typically a per-scope scratch buffer that is reset/zeroed at the top of every scope. Those SHOULD use context.allocator so they live and die with the scope. A clear comment at the declaration site is mandatory.

On every session start

Five active workstreams run in parallel — IR (the language compiler), FFI (Obj-C / JNI ceremony reduction), MEM (memory module overhaul, mem.sx + protocol expansion), LANG (user-facing language features — diagnostics renderer, heterogeneous variadic packs), and ERR (error handling: separate-channel ! errors, try / catch / or / onfail, return traces). They touch mostly disjoint files; any can be advanced independently.

1. Read all five checkpoints to see where each stream is paused:
   • current/CHECKPOINT.md — IR progress tracker.
   • current/CHECKPOINT-FFI.md — FFI progress tracker.
   • current/CHECKPOINT-MEM.md — MEM progress tracker + issues log.
   • current/CHECKPOINT-LANG.md — LANG progress tracker.
   • current/CHECKPOINT-ERR.md — ERR progress tracker.
2. Read the plan that corresponds to the stream the user wants to advance:
   • current/PLAN.md — IR implementation plan.
   • current/PLAN-FFI.md — FFI ceremony reduction plan.
   • ~/.claude/plans/tidy-doodling-cray.md — MEM (mem.sx) implementation plan.
   • current/PLAN-LANG.md — LANG implementation plan.
   • current/PLAN-ERR.md — ERR implementation plan.
3. Read specs.md if you need to understand language behavior.
4. Pick up from the next incomplete step in the relevant CHECKPOINT*.md. If the user hasn't said which stream to work on, ask before picking.

Note: implementation_plan.md is the archive of completed work (closures, protocols, auto type erasure, init blocks). Do NOT pick up unchecked items from it — those are on hold until the IR work is done.

While working

• Scratch files go in .sx-tmp/ (in the repo root), never /tmp. /tmp is outside the workspace, so every write there triggers an approval prompt; .sx-tmp/ is gitignored and approval-free. mkdir -p .sx-tmp once, then write probes/snippets there (e.g. .sx-tmp/probe.sx).
• Work on one step at a time. Complete it fully before moving on.
• After completing a step, immediately update the relevant checkpoint (current/CHECKPOINT.md for IR, current/CHECKPOINT-FFI.md for FFI, current/CHECKPOINT-MEM.md for MEM, current/CHECKPOINT-LANG.md for LANG, current/CHECKPOINT-ERR.md for ERR):
  • Update ## Last completed step with the step you just finished.
  • Update ## Current state with what exists now.
  • Update ## Next step with what comes next.
  • Add a log entry under ## Log.
• If a step fails or you get stuck:
  • Add the issue to ## Known issues in the relevant checkpoint.
  • Do NOT skip the step — fix the blocker or ask the user.
• If you make a design decision not already in specs.md, add it to ## Decisions Log in implementation_plan.md.
• FFI cadence rule (from current/PLAN-FFI.md): no commit may both add a test AND make it pass. Either lock in current behavior with a passing test, or land an expected-failing test that the very next commit turns green.

IR-specific rules (from current/PLAN.md)

• Never modify src/codegen.zig in Phases 0–1. It is the safety net.
• In Phase 3, only read specific sections of codegen.zig (grep for the relevant handler).
• No step should require reading more than ~1,000 lines of existing code. If it does, split it.
• No step should produce more than ~500 lines of new code. If it does, split it.
• If Claude gets confused mid-step, stop, update current/CHECKPOINT.md with partial progress, and tell the user to start a new session.

Context management

• Each step is scoped so it can be completed in a single session without exhausting context.
• If you're running low on context, stop at the current step boundary, update current/CHECKPOINT.md with your progress, and tell the user to start a new session.
• Never try to complete multiple phases in one session unless each is small.

Build and verify

After any code change:

zig build                              # must compile
zig build test                         # must pass

After completing a phase's final step, run the phase's end-to-end verification command listed in current/PLAN.md.

Testing

After any compiler change:

1. Build: zig build && zig build test
2. Run regression tests: bash tests/run_examples.sh
   • Every test must show ok (currently 324)
   • Zero failures, zero timeouts

Test layout

Examples and pinned issue repros use the XXXX-category-test-name scheme — a 4-digit number in per-category 100-blocks: basic 00xx, types 01xx, generics 02xx, closures 03xx, protocols 04xx, packs 05xx, comptime 06xx, modules 07xx, memory 08xx, optionals 09xx, errors 10xx, diagnostics 11xx, ffi 12xx, ffi-objc 13xx, ffi-jni 14xx, vectors 15xx, platform 16xx.

Expected output lives in an expected/ directory next to the test file, split into three streams (no more merged 2>&1) plus an optional IR snapshot:

<root>/XXXX-category-name.sx
<root>/expected/XXXX-category-name.exit      # process exit code
<root>/expected/XXXX-category-name.stdout    # normalized stdout
<root>/expected/XXXX-category-name.stderr    # normalized stderr
<root>/expected/XXXX-category-name.ir        # optional `sx ir` snapshot

A test is any <name>.sx with an expected/<name>.exit marker. The runner scans two roots: examples/ (the feature suite) and issues/ (pinned bug repros). Multi-file tests keep companions (.c/.h, imported .sx, fixture dirs) under the same XXXX- prefix.

Snapshot integrity

Never run --update while tests are failing. The --update flag blindly overwrites expected output with whatever the compiler produces — including error messages. If you update snapshots during a broken state, the test suite will "pass" against garbage output and real regressions become invisible.

Safe workflow:

1. Fix the code until bash tests/run_examples.sh passes against the existing snapshots.
2. Only run --update when you've intentionally changed output (new feature, new test, changed formatting).
3. After --update, review the diff (git diff examples/expected/ issues/expected/) to confirm no error messages or empty output were captured.

Adding a new language feature

There is no monolithic smoke file — each feature is its own focused example.

1. Create examples/XXXX-<category>-<name>.sx (next free number in the matching category block).
2. Run it: ./zig-out/bin/sx run examples/XXXX-<category>-<name>.sx
3. Seed the marker and capture expected output: : > examples/expected/XXXX-<category>-<name>.exit then bash tests/run_examples.sh --update
4. Verify all tests still pass: bash tests/run_examples.sh

Test file roles

File	Purpose
examples/XXXX-category-name.sx	Focused feature example — one feature per file.
examples/expected/XXXX-category-name.{exit,stdout,stderr}	Expected exit code + the two output streams. Regenerate with --update.
examples/expected/XXXX-category-name.ir	Optional sx ir snapshot — present only where lowering shape is locked.
issues/NNNN-slug.md	Open-issue / bug-report writeup (mark RESOLVED in a banner when fixed; the .md stays).
issues/NNNN-slug.sx (+ issues/NNNN-slug/)	The issue's minimal repro, co-located with the .md. A repro with an issues/expected/NNNN-slug.exit marker runs in the suite; unpinned ones don't.
tests/run_examples.sh	Test runner. Scans examples/ and issues/; compares stdout/stderr/exit (+ optional IR) per test.

Unit test file convention

All Zig unit tests live in separate *.test.zig files alongside the source they test:

Source file	Test file
src/ir/types.zig	src/ir/types.test.zig
src/ir/interp.zig	src/ir/interp.test.zig
src/ir/lower.zig	src/ir/lower.test.zig
...	...

• Never put test blocks directly in source files. All tests go in the corresponding .test.zig file.
• Each .test.zig file must be imported in the barrel file (src/ir/ir.zig) so zig build test discovers them via refAllDecls.
• When adding a new source file that needs tests, create the .test.zig file and add pub const foo_tests = @import("foo.test.zig"); to the barrel.

Creating a new standalone test

1. Create examples/XXXX-<category>-<name>.sx (focused example) or, for an open bug, issues/NNNN-slug.{md,sx} (repro co-located with the writeup).
2. Run it: ./zig-out/bin/sx run <path>.sx
3. Seed the marker (: > <root>/expected/<name>.exit) and capture expected: bash tests/run_examples.sh --update
4. Verify: bash tests/run_examples.sh

Resolving an open issue

When a bug filed under issues/NNNN-slug.{md,sx} is fixed:

1. Move the repro into the feature suite as a regression test: git mv issues/NNNN-slug.sx examples/XXXX-<category>-<name>.sx.
2. Seed examples/expected/XXXX-<category>-<name>.exit, capture with --update, and review the diff.
3. Tighten the example's comment header to describe the feature (keep a one-line Regression (issue NNNN) note for provenance).
4. Mark issues/NNNN-slug.md RESOLVED with a short banner (root cause + fix + regression-test path). The .md stays as the writeup.
5. Run the suite to confirm nothing else broke.

The set of issues/*.md without a RESOLVED banner is the open-issue list.

Known bugs

When you encounter a known bug during unrelated work (e.g. a closure returning a pointer instead of a value while fixing forward references), do NOT fix it inline. Instead:

1. Add it to implementation_plan.md under a ## Known Bugs section with a short description, reproduction steps, and the file/line where it manifests.
2. Continue with the current task, working around the bug if needed.
3. Fix it at the end of the session (or in a future session) as a separate step.

This keeps the current task focused and avoids scope creep from non-trivial side-fixes.

Git commits

• Never add Co-Authored-By lines to commit messages.

Bundling lives in sx

Platform-specific bundling (Apple .app, Android .apk) is sx code. The compiler shrinks to: parse → IR → codegen → link → invoke a sx function. Codesigning / Info.plist / AndroidManifest / javac / d8 / aapt2 / zipalign / apksigner / framework embed / entitlements / asset trees all run in the IR interpreter post-link via libc / process.run foreign calls.

File	Role
library/modules/platform/bundle.sx	All four targets (macOS, iOS sim, iOS device, Android). Branches on BuildOptions.is_macos / is_ios / is_ios_device / is_ios_simulator / is_android accessors.
library/modules/fs.sx	POSIX file stdlib (open / read / write / copy / mkdir / unlink / chmod / rename / exists / basename / dirname).
library/modules/process.sx	popen-based run(cmd) -> ?ProcessResult + env(name) + find_executable(name).
library/modules/compiler.sx	BuildOptions setters + accessors. Adding a new bundling parameter = add a setter here + a hook in compiler_hooks.zig.
library/modules/platform/android.sx	AndroidPlatform (state-on-struct, no module globals). sx_android_* helpers take plat: *AndroidPlatform as first arg. logical_w field drives dpi_scale = pixel_w / logical_w so consumer's design-width fits any physical resolution.
src/ir/compiler_hooks.zig	BuildConfig + every BuildOptions.* hook. Hook registry is in Registry.registerDefaults.
src/ir/host_ffi.zig	dlsym(RTLD_DEFAULT) + arity-switched cdecl trampolines. Lets #foreign("c") decls resolve at #run / post-link time against host libc.
src/main.zig	After target.link(), threads target_triple + frameworks + jni_main emissions into BuildConfig, then invokes the post-link callback by FuncId (or by <module>.bundle_main name). --bundle / --apk flags feed bundle_path; auto-fallback to post_link_module = "platform.bundle" when bundle_path is set without a registered callback.

Specifics in specs.md §10.5. The full bundling pipeline spec — what runs per Apple target vs Android, what each accessor returns, the BuildConfig forwarded from main.zig — lives there.

Wiring a new bundling step:

1. Add the parameter as a setter on BuildOptions :: struct #compiler { ... } in library/modules/compiler.sx.
2. Add the BuildConfig field + setter hook + accessor hook in src/ir/compiler_hooks.zig. Register both in Registry.registerDefaults.
3. Optionally forward a CLI flag in src/main.zig before the post-link invocation.
4. Read the accessor from library/modules/platform/bundle.sx.

File roles

File	Role
specs.md	Language specification. Source of truth for syntax/semantics.
current/PLAN.md	Active IR implementation plan.
current/CHECKPOINT.md	Active IR progress tracker. Update after every step.
current/PLAN-FFI.md	Active FFI ceremony reduction plan (Obj-C / JNI intrinsics, JNI DSL, Obj-C header import).
current/CHECKPOINT-FFI.md	Active FFI progress tracker. Update after every step.
current/PLAN-LANG.md	Active LANG implementation plan (diagnostics renderer, heterogeneous variadic packs).
current/CHECKPOINT-LANG.md	Active LANG progress tracker. Update after every step.
current/PLAN-ERR.md	Active ERR implementation plan (! errors, try / catch / or / onfail, return traces).
current/CHECKPOINT-ERR.md	Active ERR progress tracker. Update after every step.
implementation_plan.md	Archive of completed work (closures, protocols, etc.). Do not pick up tasks from here.
readme.md	Original syntax sketches. Do not modify.
CLAUDE.md	This file. Session instructions.
library/modules/platform/bundle.sx	sx-side .app / .apk bundler. See "Bundling lives in sx" above.
library/modules/fs.sx, library/modules/process.sx	POSIX stdlib for the bundler + general consumer use.