plan: final direction — full migration, no legacy; all bundling/codesign in default_pipeline
Records the user's decision: drop gate-OFF entirely (VM is the sole comptime evaluator; delete interp.zig); migrate BuildOptions directly to VM-native abi(.compiler) arms with NO legacy handlers; ALL bundling + code signing for every target (macOS/iOS-device/iOS-sim/Android) lives in the sx default_pipeline; validate against ~/projects/m3te + ~/projects/distribution. Phase 5 steps P5.5-P5.8 in PLAN-COMPILER-VM.md.
This commit is contained in:
@@ -102,15 +102,16 @@ with ONE welded mechanism. Branch: `reify` (off `master`). Update after every st
|
||||
> a failure BAILS (hard build error), no `-> !`/failable-tuple needed. New VM readers `readStringList`/
|
||||
> `readStringArg`. Smoke test `1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's
|
||||
> objects to a temp output via sx `link` — the relinked binary RUNS; negative-probe verified (bad path → bail →
|
||||
> build exit 1). The Zig driver STILL auto-links (1663 links a separate temp output); removing the auto-link is
|
||||
> P5.4. **704/0 both gates.** **ALL build primitives now exist** (queries `emit_object`/`c_object_paths`/
|
||||
> `link_libraries` + action `link`). **NEXT — P5.3: the `on_build` slot.** A comptime-assignable compiler global
|
||||
> (generalizes `post_link_callback_fn`: an assignable typed global w/ a stdlib default, vs a setter). `#run
|
||||
> on_build = build;` captures the FuncId; the compiler invokes `on_build(config)` post-codegen WITH the
|
||||
> CLI-derived `BuildConfig` as an ARG — which needs a VM entry that MARSHALS args (the gap `core.invokeByFuncId`
|
||||
> rejects today with `ComptimeVmArgsUnsupported`). Then P5.4 (sx `default_build` that calls the primitives +
|
||||
> RESTRUCTURE the driver to stop auto-emit/auto-link + delete `#compiler`/`compiler_call`/`compiler_hooks` + the
|
||||
> S5a `build_options`/`set_post_link_callback`) — P5.4 kills the 4 strict `compiler_call` bails (1609/1614/1615/1616).
|
||||
> build exit 1). **P5.3 (`on_build` registrar) + P5.4 CORE DONE (2026-06-19, newest Log entries):** the whole
|
||||
> build is sx-driven via `default_pipeline` (force-lowered + auto-invoked; NO Zig auto-emit/auto-link);
|
||||
> `on_build(cb)` is the sole callback mechanism; `set_post_link_callback` deleted. **703/0 both gates.**
|
||||
> **NEXT — the FULL MIGRATION (no legacy left), spec'd as Phase 5 steps P5.5–P5.8 in `PLAN-COMPILER-VM.md`:**
|
||||
> P5.5 migrate the 36 `BuildOptions` `#compiler` methods → VM-native `abi(.compiler)` arms (NO legacy handler —
|
||||
> direct migration; thread a persistent allocator for setter strings; kills the 4 strict `compiler_call` bails
|
||||
> 1609/1614/1615/1616) · P5.6 ALL bundling + code signing for EVERY target (macOS/iOS-device/iOS-sim/Android) in
|
||||
> the sx `default_pipeline` · P5.7 DELETE `#compiler`/`compiler_call`/`compiler_hooks`/`interp.zig` + the
|
||||
> `regToValue` bridge + VM→legacy fallback (drop gate-OFF; VM is the SOLE evaluator) · P5.8 build
|
||||
> `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test + add `.app`/`.apk` smoke tests.
|
||||
> **FINAL atomic step (4F):** (`out` already done — VM-native via libc `write`) handle `interp_print_frames` +
|
||||
> flip strict-to-default (remove the fallback) + delete `interp.zig`/`Value` + re-express `define`/`make_enum`.
|
||||
> See `PLAN-COMPILER-VM.md` → Phase 4 for the full plan + top risks (bundler test coverage).
|
||||
@@ -440,33 +441,23 @@ when reached (sentinels or accessor fns; see the design doc Risks).
|
||||
deleted 1661/1663 (primitives now exercised by EVERY AOT build). `sx run` (JIT) is UNTOUCHED (emits in-process,
|
||||
never invokes `default_pipeline`). Benign `.ir` churn each step; **703/0 both gates.**
|
||||
> **REMAINING P5.4 (the BuildOptions-surface migration — large, mechanical, dual-path, string-lifetime-sensitive;
|
||||
> NOT YET DONE — paused to avoid rushing core-IR deletion on a depleted context budget):**
|
||||
> - **Migrate the 36 `BuildOptions :: struct #compiler` methods → free `ufcs … abi(.compiler)` fns** (the
|
||||
> pattern S5a used for `set_post_link_callback`). Families: string SETTERS (`set_bundle_path`/`set_bundle_id`/
|
||||
> `set_codesign_identity`/`set_provisioning_profile`/`set_manifest_path`/`set_keystore_path`/`add_framework`/
|
||||
> `add_link_flag`/`set_output_path`/`set_wasm_shell`/`set_post_link_module`/`add_asset_dir`) — write/append to
|
||||
> `BuildConfig`; string GETTERS (`binary_path`/`bundle_path`/`bundle_id`/`codesign_identity`/
|
||||
> `provisioning_profile`/`target_triple`/`manifest_path`/`keystore_path`) — read `BuildConfig`; BOOL getters
|
||||
> (`is_macos`/`is_ios`/`is_ios_device`/`is_ios_simulator`/`is_android`) — compute from the triple; LIST/index
|
||||
> getters (`framework_count`/`framework_at`/`framework_path_*`/`asset_dir_*`/`jni_main_*`).
|
||||
> - **Each method = dual-path:** a VM `callCompilerFn` arm (flat memory) + a legacy `compiler_lib` handler. The
|
||||
> legacy handlers can REUSE the existing `compiler_hooks` hook bodies (make them `pub`; the hook already dupes
|
||||
> strings into a long-lived allocator + reads/writes `BuildConfig`). **String lifetime:** a setter called at
|
||||
> `#run` (emit-time VM eval) must dupe the flat-memory string into a PERSISTENT allocator (NOT the per-eval VM
|
||||
> arena which frees at `Vm.deinit`) — thread `emit_llvm.alloc` in (e.g. a `BuildConfig.string_alloc` field) so
|
||||
> real bundle builds keep the strings; for the strict-bail JIT case the config is set-but-never-read so any
|
||||
> alloc works, but do it right for real `sx build`.
|
||||
> - **This kills the 4 strict `compiler_call` bails** (1609 `add_framework`; 1614/1615/1616 `set_bundle_*`/
|
||||
> `set_codesign_*`/`set_provisioning_*` — all SETTERS at `#run configure()`). Migrating the SETTERS alone
|
||||
> suffices for strict-green (the GETTERS run only at post-link, which JIT/strict-run never fires).
|
||||
> - **Then DELETE** `#compiler` (parse/lower attribute), the `compiler_call` IR op (`inst.zig` + every
|
||||
> switch arm + `interp.zig:1130` dispatch), and `compiler_hooks.zig` `HookFn`/`Registry` (keep the pub hook
|
||||
> bodies only if the legacy handlers call them — else inline + delete). The `BuildOptions` struct becomes a
|
||||
> plain (zero-field) handle type, or is dropped if `opt` is unused. **Migrate the bundler** (`platform/bundle.sx`
|
||||
> `bundle_main`) so `default_pipeline` calls it when `bundle_path` is set (the `#compiler` accessors it reads
|
||||
> become the migrated `abi(.compiler)` getters) + the `--bundle` shim / `post_link_module` path is reworked or
|
||||
> removed (default_pipeline owns bundling). **No corpus AOT bundle test exists** → add dedicated bundle smoke
|
||||
> tests (the stream's top risk). After strict goes green: flip VM-to-default + delete `interp.zig` (4F).
|
||||
> NOT YET DONE):** **FINAL DIRECTION (user 2026-06-19): FULL MIGRATION — NO LEGACY. Drop gate-OFF entirely;
|
||||
> the VM is the SOLE evaluator; delete `interp.zig`. Migrate DIRECTLY to VM-native `abi(.compiler)` arms — NO
|
||||
> legacy `compiler_lib` handlers, NO dual-path.** See `PLAN-COMPILER-VM.md` → Phase 5 steps **P5.5–P5.8** for
|
||||
> the full spec. In brief:
|
||||
> - **P5.5** — migrate all 36 `BuildOptions :: struct #compiler` methods → free `ufcs … abi(.compiler)` decls +
|
||||
> `comptime_vm.callCompilerFn` arms (NO legacy handler). Setters dupe strings into a PERSISTENT allocator
|
||||
> (thread `emit_llvm.alloc` via e.g. `BuildConfig.string_alloc`). Kills the 4 strict `compiler_call` bails.
|
||||
> - **P5.6** — ALL bundling + code signing for EVERY target (macOS `.app`, iOS device/sim, Android `.apk`:
|
||||
> Info.plist/codesign/provisioning/entitlements/framework-embed/AndroidManifest/javac/d8/aapt2/zipalign/
|
||||
> apksigner) runs in the sx `default_pipeline` (via the migrated getters + `fs`/`process` host-FFI). Remove the
|
||||
> `--bundle`/`post_link_module` Zig shim. Compiler keeps ONLY the linker primitive (Option B).
|
||||
> - **P5.7** — DELETE `#compiler` + `compiler_call` op + `compiler_hooks` (Registry/HookFn) + `interp.zig`
|
||||
> (Interpreter/Value/reflectTypeInfo/callExtern) + `regToValue`/`valueToReg` + the VM→legacy fallback; make
|
||||
> `-Dcomptime-flat` permanent. A VM bail is ALWAYS a build diagnostic now. Re-express `define`/`make_enum` as
|
||||
> sx. Land the 0141 repro; reconcile 1654.
|
||||
> - **P5.8** — build `~/projects/m3te` + `~/projects/distribution` end-to-end as the acceptance test that
|
||||
> `default_pipeline` covers all targets; add `.app` + `.apk` bundle smoke tests (no corpus coverage today).
|
||||
- **P5.3 (`on_build` registrar) — the build-callback registration mechanism; callback takes `BuildOptions` (2026-06-19).**
|
||||
Per the user's design: `on_build(cb)` is the build-callback registrar (a FREE fn), generalizing
|
||||
`set_post_link_callback` — the callback is `(opt: BuildOptions) -> bool abi(.compiler)` and the compiler invokes
|
||||
|
||||
@@ -523,23 +523,67 @@ The compiler's whole post-IR role: codegen → build the CLI-derived `BuildConfi
|
||||
readers `readStringList`/`readStringArg` (inverse of `makeStringList`). Smoke test
|
||||
`1663-platform-build-pipeline-link` (AOT): a post-link callback re-links the build's objects to a temp output —
|
||||
the relinked binary RUNS; negative-probe verified. The Zig driver still auto-links (removed in P5.4). 704/0.
|
||||
- **P5.3 — `on_build` slot:** a comptime-assignable compiler slot (GENERALIZES today's `post_link_callback_fn`:
|
||||
an assignable typed global with a stdlib default, vs a setter). `#run on_build = build;` captures the
|
||||
`FuncId`; the compiler invokes it post-codegen with the CLI-derived `BuildConfig`.
|
||||
- **P5.4 — sx `default_build` + `BuildConfig`:** write the stdlib pipeline; move config/orchestration into sx.
|
||||
**Delete** `#compiler` / `compiler_call` / `compiler_hooks` (`HookFn`/`Registry`) + the S5a
|
||||
`build_options`/`set_post_link_callback` (config is now sx data passed as primitive args).
|
||||
- **4E — post-link bundler on the VM (role C).** Depends on the FFI escape (done) + the
|
||||
`BuildConfig`-on-the-VM threading above. Route `core.invokeByFuncId` / `main.zig`'s post-link
|
||||
call through the VM. **No corpus coverage** (only runs on `sx build --bundle/--apk`) — add
|
||||
dedicated bundle smoke tests (min `.app` + `.apk`); gate on real bundle builds, not just
|
||||
`zig build test`. Riskiest phase.
|
||||
- **4F — flip + delete (the end state).** Flip the VM to default (retire `-Dcomptime-flat`);
|
||||
delete `interp.zig` (`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`); drop
|
||||
the `regToValue`/`valueToReg` bridge; simplify `core.zig` (`invokeByFuncId` → VM) + `main.zig`
|
||||
(`last_bail_*` → VM diagnostics); remove the dual-path `compiler_lib` handlers (keep only
|
||||
VM-native `callCompilerFn`); re-express `define`/`make_enum` as sx over the compiler-API
|
||||
(allocation works on the sole evaluator) and land the original 0141 repro as a corpus test.
|
||||
- **P5.3 — `on_build` registrar — DONE (2026-06-19).** `on_build(cb)` registers the build callback
|
||||
(`cb: (opt: BuildOptions) -> bool abi(.compiler)`); the compiler force-lowers + auto-invokes the well-known
|
||||
`default_pipeline` when no override. (Implemented as a registrar, not an assignable slot — the opaque
|
||||
`BuildOptions` handle is one word, so arg-passing needs no struct marshaling.)
|
||||
- **P5.4 core — DONE (2026-06-19).** `default_pipeline` in `build.sx` drives the whole build; NO Zig
|
||||
auto-emit/auto-link; `emit_object`/`link` are sx-called actions via the `BuildHooks` vtable;
|
||||
`set_post_link_callback` deleted (all callers on `on_build`). Build-path auto-imports `modules/build.sx`.
|
||||
703/0 both gates.
|
||||
|
||||
### THE FINAL DIRECTION (user, 2026-06-19): FULL MIGRATION — NO LEGACY LEFT.
|
||||
|
||||
**Decision: DROP gate-OFF entirely.** The VM becomes the SOLE comptime evaluator; `-Dcomptime-flat` is made
|
||||
permanent then removed; `interp.zig` (the legacy tagged-`Value` `Interpreter`) is DELETED. There is no
|
||||
dual-path, no legacy `compiler_lib` handler, no `regToValue`/`valueToReg` bridge, no VM→legacy fallback. We
|
||||
migrate the BuildOptions surface DIRECTLY to VM-native `abi(.compiler)` arms (no legacy handler — there is no
|
||||
legacy to handle). **All bundling + code signing for EVERY target lives in the sx `default_pipeline`.**
|
||||
|
||||
- **P5.5 — migrate the 36 `BuildOptions :: struct #compiler` methods → VM-native `abi(.compiler)`.** Each
|
||||
becomes a free `ufcs (self: BuildOptions, …) abi(.compiler)` decl (so `opt.method(...)` still resolves via
|
||||
UFCS) with a `comptime_vm.callCompilerFn` arm — and **NO legacy `compiler_lib` handler** (the user's directive;
|
||||
the legacy interp is going away). Families: string SETTERS (`set_bundle_path`/`set_bundle_id`/
|
||||
`set_codesign_identity`/`set_provisioning_profile`/`set_manifest_path`/`set_keystore_path`/`add_framework`/
|
||||
`add_link_flag`/`set_output_path`/`set_wasm_shell`/`set_post_link_module`/`add_asset_dir`) — write/append to the
|
||||
threaded `BuildConfig`; string GETTERS (`binary_path`/`bundle_path`/`bundle_id`/`codesign_identity`/
|
||||
`provisioning_profile`/`target_triple`/`manifest_path`/`keystore_path`); BOOL getters (`is_macos`/`is_ios`/
|
||||
`is_ios_device`/`is_ios_simulator`/`is_android` — compute from the triple); LIST/index getters
|
||||
(`framework_count`/`framework_at`/`framework_path_*`/`asset_dir_*`/`jni_main_*`, built via `makeStringList`).
|
||||
**String lifetime:** a setter at `#run` must dupe the flat-memory string into a PERSISTENT allocator (NOT the
|
||||
per-eval VM arena) — thread `emit_llvm.alloc` into the VM (e.g. `BuildConfig.string_alloc`) so the strings
|
||||
survive to post-link. This kills the 4 strict `compiler_call` bails (1609/1614/1615/1616).
|
||||
- **P5.6 — ALL bundling + code signing in `default_pipeline` (every target).** `default_pipeline` (or a
|
||||
`bundle()` it calls, in `platform/bundle.sx`) performs, after `link`, the full per-target bundle when
|
||||
`bundle_path()` is set — branching on `is_macos`/`is_ios_device`/`is_ios_simulator`/`is_android`:
|
||||
- **macOS `.app`** — `Contents/{MacOS,Resources,Frameworks}`, `Info.plist`, embed `-framework` dylibs +
|
||||
`install_name_tool` fixups, `codesign` (ad-hoc or with `codesign_identity`).
|
||||
- **iOS device `.app`** — device slice, embedded `.mobileprovision` (`provisioning_profile`), entitlements,
|
||||
`codesign` with the real identity; **iOS simulator `.app`** — sim slice, no provisioning, ad-hoc sign.
|
||||
- **Android `.apk`** — `AndroidManifest.xml` (or `manifest_path` override), asset tree (`add_asset_dir`),
|
||||
`#jni_main` Java → `javac` → `d8` → `classes.dex`, `aapt2` package, `zipalign`, `apksigner` with the
|
||||
debug/`keystore_path` keystore.
|
||||
All of it runs on the VM via the migrated `abi(.compiler)` getters + `fs`/`process` host-FFI (the existing
|
||||
`platform/bundle.sx` logic, now reading the VM-native accessors instead of `#compiler` hooks). The compiler
|
||||
keeps ONLY the linker as a primitive (Option B). Remove the `--bundle`/`post_link_module` Zig shim — bundling
|
||||
is `default_pipeline`'s job; CLI flags feed `BuildConfig` and `default_pipeline` branches on it.
|
||||
- **P5.7 — DELETE all legacy.** Remove the `#compiler` attribute (parse + lower), the `compiler_call` IR op
|
||||
(`inst.zig` + every switch arm + the `interp.zig:1130` dispatch), `compiler_hooks.zig`
|
||||
(`HookFn`/`Registry`/all hooks). Make `-Dcomptime-flat` permanent (VM always) and **delete `interp.zig`**
|
||||
(`Interpreter`/`Value`/`defineEnum`…/`reflectTypeInfo`/`callExtern`/`last_bail_*`); drop the
|
||||
`regToValue`/`valueToReg` bridge and the VM→legacy fallback in `emit_llvm` (`#run`/const-init) and
|
||||
`comptime.zig` (type-fn / `#insert`) — a VM bail is now ALWAYS a build-gating diagnostic (4B wiring), never a
|
||||
fallback. `core.invokeByFuncId` is already VM-only. Re-express `define`/`make_enum` as sx over the
|
||||
compiler-API. Land the 0141 repro as a corpus test. Reconcile 1654 (asm-global at comptime) to the VM wording.
|
||||
- **P5.8 — real-project validation (integration).** Build `~/projects/m3te` and `~/projects/distribution` with
|
||||
the new pipeline end-to-end (their real bundle/codesign/target configs) — these are the acceptance test that
|
||||
`default_pipeline` covers all targets. Fix gaps surfaced there. Add dedicated bundle smoke tests (min `.app` +
|
||||
`.apk`) to the corpus (the bundler still has no `zig build test` coverage — the stream's top risk).
|
||||
|
||||
**End state:** ONE evaluator (the VM); ZERO legacy; the entire build — emit, link, and all bundling + code
|
||||
signing for macOS/iOS-device/iOS-sim/Android — is sx in `default_pipeline`, overridable via `#run on_build(...)`.
|
||||
The compiler is: parse → IR → codegen → invoke `on_build`/`default_pipeline` on the VM (which calls back into
|
||||
the linker primitive). `m3te` + `distribution` build clean.
|
||||
|
||||
**Dependencies:** 4A → (4B, 4C independent) ; `abi(.compiler)` S1+S2(done) → S3 → S4 → S5 (BuildOptions) ;
|
||||
FFI(done)+`BuildConfig`-on-VM → (S5, 4E) → 4F.
|
||||
|
||||
Reference in New Issue
Block a user