feat(dist): bundled-zig link backend for hermetic macOS/Linux/Windows builds
Drive a bundled `zig` as `zig cc` for the AOT link step, supplying lld + CRT + libc (musl/glibc/mingw) so `sx build` produces native binaries with no host toolchain. Default Linux output is static musl (portable-anywhere). - src/zig_backend.zig: discover zig ($SX_ZIG / bundled-next-to-exe / PATH); bundled-vs-PATH provenance gates auto-activation. - src/target.zig: selectZigLinker + emitZigLinkArgv + zigTargetTriple, dispatched before the per-OS branches; macOS/Linux/Windows in scope. - src/ir/emit_llvm.zig: LLVMNormalizeTargetTriple so vendor-less zig triples (e.g. x86_64-windows-gnu) parse to the correct OS/object format (COFF not ELF). - src/main.zig: --self-contained / --no-self-contained; linux-musl, linux-musl-arm, windows-gnu shorthands; de-vendor linux/linux-arm to match the corpus runner. - examples/1660: Windows Win32 print-42 + exit(0) via kernel32 (ir-only off-Windows). Auto-activates only for a bundled zig; a PATH-only zig engages under --self-contained, so native dev/CI builds are never silently rerouted. Docs: readme Cross-Compilation, design/bundled-zig-link-backend-design.md, current/PLAN-DIST.md.
This commit is contained in:
agra
124
current/PLAN-DIST.md
Normal file
124
current/PLAN-DIST.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# PLAN-DIST — bundle `zig` as sx's hermetic link/libc backend
|
||||
|
||||
## Goal
|
||||
|
||||
`sx build` produces a native binary by driving a **bundled `zig`**
|
||||
(`zig cc`) as the linker, so a distributed sx on Linux needs no system
|
||||
`cc`/lld/libc/CRT. `sx run` (JIT) is unaffected — it never links.
|
||||
|
||||
This is the "be like Zig" move: reuse Zig's hermetic toolchain (lld +
|
||||
crt objects + musl/glibc, all bundled in the `zig` distribution) instead
|
||||
of building our own lld-in-process + libc-from-source pipeline.
|
||||
|
||||
> **Configuration surface** (env vars, flags, resolution order,
|
||||
> activation truth table, target→ABI map, distribution layout) is
|
||||
> specified in [../design/bundled-zig-link-backend-design.md](../design/bundled-zig-link-backend-design.md) — the design-of-record
|
||||
> for how the backend is configured. Keep the two files in sync.
|
||||
|
||||
## Locked decisions
|
||||
|
||||
1. **Default Linux output ABI = static musl** (`x86_64-linux-musl`,
|
||||
`-static`). Output runs on ANY Linux with zero deps — the property
|
||||
that makes Zig binaries portable. glibc/dynamic only via explicit
|
||||
`--target x86_64-linux-gnu`.
|
||||
2. **Activation = auto** when a bundled/resolvable `zig` exists AND the
|
||||
user passed no `--linker`. Falls back to system `cc` otherwise.
|
||||
3. **Dev uses PATH `zig`** (0.16.0 already installed). Defer copying a
|
||||
vendored toolchain into `libexec/` until Phase 3 packaging.
|
||||
|
||||
## Why `zig cc`, not raw `ld.lld`
|
||||
|
||||
`zig cc` is a clang-compatible driver, so it slots into the **existing**
|
||||
cc-style argv branch in `src/target.zig` almost unchanged, and supplies
|
||||
lld + crt objects + musl/glibc automatically per `-target`. Driving
|
||||
`ld.lld` directly would force us to locate/pass crt1.o/crti.o/libc
|
||||
ourselves — exactly the work we're avoiding.
|
||||
|
||||
## Key code anchors (verified)
|
||||
|
||||
- Linker selection hook: `TargetConfig.getLinker()` — `src/target.zig:194-196`
|
||||
(`self.linker orelse "cc"`).
|
||||
- Unix `cc`-style link branch: `src/target.zig:524-564` (this is where
|
||||
the zig backend hooks in; `-o`/`-L`/`-l`/extra objects already pass
|
||||
through clang-compatibly).
|
||||
- Exe-relative resolution pattern to mirror for finding zig:
|
||||
`src/imports.zig:204-227` (`discoverStdlibPaths`, `$SX_STDLIB_PATH`
|
||||
override + `<exe>/..` candidates).
|
||||
- `--linker` CLI flag parsing: `src/main.zig:87-90`.
|
||||
- Emit triple (must agree with link target): `src/ir/emit_llvm.zig`
|
||||
(`LLVMSetTarget`, ~L246-284).
|
||||
|
||||
## Phases
|
||||
|
||||
### Phase 0 — Resolve a bundled/host zig
|
||||
- New `src/zig_backend.zig`: `discoverZig(alloc) -> ?[]const u8`.
|
||||
Resolution order:
|
||||
1. `$SX_ZIG` env override.
|
||||
2. `<exe>/../libexec/zig/zig` (install layout, Phase 3).
|
||||
3. `<exe>/../../zig-bundle/zig` (dev vendored layout, Phase 3).
|
||||
4. `zig` on `PATH` (dev fallback — active now).
|
||||
- Add `SX_DEBUG_ZIG` trace, matching existing `SX_DEBUG_*` hooks.
|
||||
- No behavior change yet; just resolution + a debug/print hook to confirm.
|
||||
|
||||
### Phase 1 — `zig cc` link backend (core change)
|
||||
- `src/target.zig`: generalize the linker from a single token to a
|
||||
**driver argv**. Today `getLinker()` returns one string at `argv[0]`;
|
||||
introduce a `LinkBackend` so the internal backend contributes
|
||||
`{zigPath, "cc"}` as leading entries.
|
||||
- In the Unix branch (L524-564), when backend = zig:
|
||||
- prepend `zig cc`,
|
||||
- append `-target <mapped triple>`,
|
||||
- add `-static` for musl,
|
||||
- everything else (`-o`, `-L`, `-l`, extra objects, extra link flags)
|
||||
passes through unchanged.
|
||||
- Add `sxTripleToZig()` mapping (sx shorthand/triple → zig `-target`);
|
||||
unspecified-on-Linux → `x86_64-linux-musl`.
|
||||
- Align emit triple: when the zig backend is selected, set the LLVM
|
||||
module triple in `emit_llvm.zig` to match the link target
|
||||
(x86_64-linux), so the `.o` links cleanly against musl crt.
|
||||
|
||||
### Phase 2 — Activation
|
||||
- Auto-enable: if `discoverZig()` succeeds and no `--linker` override,
|
||||
use the zig backend for `sx build`. System `cc` remains the fallback.
|
||||
- Optional explicit `--self-contained` / `--no-self-contained` to force.
|
||||
- Confirm `sx run`/JIT path is untouched (no link step).
|
||||
|
||||
### Phase 3 — Distribution packaging
|
||||
- `build.zig`: a `dist` step assembling
|
||||
- `bin/sx` (built with `-Dstatic-llvm`),
|
||||
- `libexec/zig/` (vendored zig binary **and its `lib/`**, copied from a
|
||||
pinned ziglang.org release per host arch),
|
||||
- `library/` (stdlib),
|
||||
into a relocatable tarball.
|
||||
- Pin the zig version (currently 0.16.0).
|
||||
|
||||
### Phase 4 — Verify & lock
|
||||
- Manual first: `sx build hello.sx` (auto zig backend) then `file`/`ldd`
|
||||
the output → expect "statically linked".
|
||||
- Honor snapshot-integrity + FFI-cadence rules before adding a corpus
|
||||
test (host/arch-gated, likely a `.build` sidecar).
|
||||
|
||||
## Risks / watch
|
||||
|
||||
- **Bundle size**: zig + its `lib/` ≈ 50–60 MB.
|
||||
- **gnu vs musl ABI**: pure codegen objects link fine against musl;
|
||||
TLS/stack-protector are the only realistic friction. Aligning the emit
|
||||
triple (Phase 1) covers the common path.
|
||||
- **macOS/Windows cross** via the same `zig cc -target` is nearly free
|
||||
after Phase 1, but Apple-SDK linking has caveats — scope to Linux
|
||||
target first; treat the rest as follow-up.
|
||||
- **c_import.zig** also shells `cc` for C imports (JIT). Out of scope
|
||||
here; same backend can absorb it later.
|
||||
|
||||
## Status
|
||||
|
||||
- [x] Phase 0 — resolve zig (`src/zig_backend.zig`)
|
||||
- [x] Phase 1 — zig cc link backend (`target.zig` + `emit_llvm` triple normalize)
|
||||
- [x] Phase 2 — activation (`--self-contained`/`--no-self-contained`; auto on bundled zig)
|
||||
- [ ] Phase 3 — dist packaging (vendor `zig` into `libexec/`)
|
||||
- [ ] Phase 4 — verify & lock (manual ✓ macOS/Linux/Windows; corpus test pending runner `--self-contained` support)
|
||||
|
||||
Scope landed as **macOS + Linux + Windows** (not Linux-first). See the
|
||||
"Implementation status" section in
|
||||
[../design/bundled-zig-link-backend-design.md](../design/bundled-zig-link-backend-design.md)
|
||||
for what refined the original locked decisions.
|
||||
Reference in New Issue
Block a user