compiler-API: welded structs by reflection + memory-order validation
Replace the explored byte-layout-override engine (offset-ordered LLVM structs /
weld plans / byte-blobs — all unnecessary) with a much simpler design: a welded
`struct abi(.zig) extern compiler { … }` is a bodied header declaring its fields
in the bound compiler type's MEMORY order. The compiler reflects the real Zig
type (field names via @typeInfo, offsets via @offsetOf, size via @sizeOf —
nothing hand-maintained) and validates the header matches, with loud diagnostics.
On pass it is an ordinary struct whose natural layout already equals the Zig
layout — no reorder, no padding, no index/remap tables, no special LLVM path — so
@ptrCast'ing it to the compiler's own type and dereferencing is byte-identical.
When types.zig shifts, the header stops matching and the developer gets a specific
message to fix it.
- compiler_lib.zig: weldStruct reflects field names and bakes bound_types fields
in ascending-offset (memory) order; deleted computeWeldPlan/WeldPlan/WeldElement.
- nominal.zig validateWeldedStruct: precise diagnostics — field-not-found,
wrong-field-order (+ expected memory order), type-layout (size) mismatch,
total-size mismatch.
- Examples: 0627 (StructInfo in memory order, byte-identical, usable),
1186 (source-order StructInfo -> wrong-field-order diagnostic); 1183 refreshed.
- Design doc + checkpoint updated.
This commit is contained in:
@@ -7,25 +7,59 @@ Companion to the design-of-record
|
|||||||
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
|
with ONE welded mechanism. Branch: `reify` (off `master`). Update after every step.
|
||||||
|
|
||||||
## ⏯ Resume (fresh session)
|
## ⏯ Resume (fresh session)
|
||||||
Phase 1 is COMPLETE and committed (`cd5b958`); Phase 2 (full byte-layout weld)
|
Phase 1 done; Phase 2 **welded structs are working** via a much simpler design than
|
||||||
just started. **Do sub-step 2.2 next** — make `src/backend/llvm/types.zig`'s
|
the original byte-layout-override "GEP engine" (that plan — `computeWeldPlan`,
|
||||||
`.@"struct"` case build a welded struct's LLVM type from `compiler_lib.computeWeldPlan`
|
offset-ordered LLVM structs, byte-blobs — was explored and DROPPED). The locked
|
||||||
(offset-ordered field elements + `[N x i8]` padding) with a build-time
|
design: a welded `Name :: struct abi(.zig) extern compiler { … }` is a bodied
|
||||||
`LLVMOffsetOfElement == plan offset` + `LLVMABISizeOfType == total_size` assertion;
|
header declaring fields in the compiler type's MEMORY order; the compiler reflects
|
||||||
cache the plan per TypeId for the GEP sites. The plan math (sub-step 2.1) is done,
|
the bound Zig type (`@typeInfo` names + `@offsetOf` offsets + `@sizeOf`, nothing
|
||||||
pure, and unit-tested — see `computeWeldPlan` in `src/ir/compiler_lib.zig`. Full
|
maintained by hand) and VALIDATES the header matches, with loud diagnostics. On
|
||||||
2.2–2.6 breakdown under **## Next step**. Read order: this file → the design doc →
|
pass it's an ordinary byte-identical struct — so `@ptrCast` to the compiler's own
|
||||||
`src/ir/compiler_lib.zig`. Build/verify: `zig build && zig build test` (green now).
|
type + deref just works; no index tables, no reorder, no special emit.
|
||||||
|
|
||||||
> ⚠ Snapshot gotcha: `zig build test -Dupdate-goldens` on this aarch64 host clobbers
|
**Next:** Phase 2 continues — re-express `type_info`/`define` (struct) as sx over
|
||||||
> cross-arch examples' CI-captured `.stdout` (1228/1231/1639/1651/1657–1660) with
|
welded `register_struct`/`find_type` (host-call bridge, Phase 2.5/2.6); see
|
||||||
> host-specific empties. After regenerating, revert those (`git checkout` / `rm`)
|
**## Next step**. Read order: this file → `src/ir/compiler_lib.zig` (registry +
|
||||||
> before committing — they are NOT part of this stream.
|
reflection) → `src/ir/lower/nominal.zig` `validateWeldedStruct`. Build/verify:
|
||||||
|
`zig build && zig build test`.
|
||||||
|
|
||||||
|
> ⚠ Snapshot workflow: use `-Dname=examples/NNNN-foo.sx[,…] -Dupdate-goldens` to
|
||||||
|
> regenerate ONLY the named example(s) — a full `-Dupdate-goldens` re-runs all ~690
|
||||||
|
> and a flaky/host-divergent example (AOT/cross-arch) can clobber good snapshots.
|
||||||
|
> See CLAUDE.md → Snapshot integrity.
|
||||||
|
|
||||||
## Last completed step
|
## Last completed step
|
||||||
**Phase 2, sub-step 1 — the weld-plan layout math + `StructInfo` registered.**
|
**Phase 2 — welded structs by reflection + memory-order validation (byte-identical,
|
||||||
The de-risked core of the byte-layout-override ("GEP") engine, pure + unit-tested,
|
no GEP engine).** A welded `struct abi(.zig) extern compiler { … }` now works
|
||||||
no emit/interp wiring yet (suite trivially green).
|
end-to-end as a byte-identical mirror of the bound Zig type.
|
||||||
|
|
||||||
|
Design (locked, supersedes the byte-layout-override plan):
|
||||||
|
- The sx header declares fields in the compiler type's MEMORY order. The compiler
|
||||||
|
REFLECTS the bound Zig type — field names from `@typeInfo`, offsets from
|
||||||
|
`@offsetOf`, size from `@sizeOf` — and validates the header matches. Nothing is
|
||||||
|
maintained by hand; a `types.zig` change re-reflects on the next compiler build.
|
||||||
|
- On pass it's an ORDINARY struct whose natural layout already equals the Zig
|
||||||
|
layout → `@ptrCast` to the compiler type + deref is byte-identical. No
|
||||||
|
byte-blob, no index/remap tables, no reorder, no special LLVM path.
|
||||||
|
- Loud, precise diagnostics on any drift: *field not found* (+ memory order),
|
||||||
|
*wrong field order at position N* (+ expected memory order), *type layout
|
||||||
|
mismatch* (field size), *layout mismatch* (total size / count).
|
||||||
|
|
||||||
|
What changed from the dropped plan:
|
||||||
|
- `compiler_lib.zig`: `weldStruct` now REFLECTS field names (`@typeInfo`) and bakes
|
||||||
|
`bound_types` fields in ascending-OFFSET (memory) order — no hand-listed names.
|
||||||
|
Deleted `computeWeldPlan`/`WeldPlan`/`WeldElement`. `validateStructLayout` checks
|
||||||
|
the sx header against the memory-ordered registry.
|
||||||
|
- `nominal.zig` `validateWeldedStruct`: renders the precise diagnostics
|
||||||
|
(+ `weldedFieldOrderStr`).
|
||||||
|
- Examples: `0627` (StructInfo in memory order, byte-identical, usable);
|
||||||
|
`1186` (source-order StructInfo → wrong-field-order diagnostic). `1183` message
|
||||||
|
refreshed.
|
||||||
|
- `zig build` + `zig build test` green (692 corpus, unit tests pass).
|
||||||
|
|
||||||
|
### Earlier — Phase 2.1 (weld-plan layout math, now removed)
|
||||||
|
**The weld-plan offset math + `StructInfo` registered.** Was the core of the
|
||||||
|
byte-layout-override engine; superseded by the reflection+validation design above.
|
||||||
|
|
||||||
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
|
Decision (locked 2026-06-17): **full byte-layout weld** — a welded sx struct is
|
||||||
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
|
laid out byte-identically to the bound Zig type (Zig's `@offsetOf`, reordering +
|
||||||
@@ -204,54 +238,48 @@ What landed:
|
|||||||
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
|
- **Deferred**: offset-override / LLVM byte-offset GEP for non-natural layouts
|
||||||
(needed by `StructInfo`'s slice field, Phase 2).
|
(needed by `StructInfo`'s slice field, Phase 2).
|
||||||
|
|
||||||
## Next step — Phase 2 decomposition (byte-layout weld for `StructInfo`)
|
## Next step — Phase 2: welded compiler FUNCTIONS over the real types
|
||||||
|
|
||||||
The weld plan (sub-step 1) is the pure layout math. The remaining sub-steps wire
|
Welded structs are byte-identical mirrors now, so the API surface can grow:
|
||||||
it through emit + interp so a non-natural welded struct actually works. Each must
|
|
||||||
stay green; do ONE per session (the IR-stream split rule).
|
|
||||||
|
|
||||||
- **2.2 — LLVM type honours the plan.** In `src/backend/llvm/types.zig` `.@"struct"`
|
- **Bind `register_struct` / `find_type`** over the host-call bridge
|
||||||
case: if the struct's name is in `compiler_lib.findType`, build the LLVM struct
|
(`compiler_lib.zig` `bound_fns`, like `intern`/`text_of`). `register_struct`
|
||||||
from `computeWeldPlan` — elements in offset order (real field types + `[N x i8]`
|
takes a welded `StructInfo` and mints a real `TypeId` (guarded: dup field names,
|
||||||
padding), and **assert** `LLVMOffsetOfElement(elem) == plan.elements[e].offset`
|
kind well-formedness — the checks `define` does today). Because the welded
|
||||||
for every field element + `LLVMABISizeOfType == total_size` (the build-time
|
`StructInfo` is byte-identical, the handler can read it as the real Zig
|
||||||
layout-equality assertion; mismatch = a loud emit failure). Cache the plan per
|
`*StructInfo` (cast + deref) rather than marshalling a `Value` field-by-field —
|
||||||
TypeId (the GEP sites + interp need the remap). Prove: a welded struct's LLVM
|
the payoff of the byte-weld. `find_type(StringId) -> ?Type` reads the table.
|
||||||
type has the Zig offsets (an emit-level test or an `.ir`/codegen check).
|
Prove: build a struct programmatically + round-trip a source one.
|
||||||
- **2.3 — field access honours the remap.** Every `struct_gep` / field load+store
|
- **Re-express `type_info`/`define` (struct) as sx** over `register_struct`/
|
||||||
for a welded struct maps the sx field index → `plan.sx_to_llvm[i]` before
|
|
||||||
`LLVMBuildStructGEP2` (`src/backend/llvm/ops.zig` — `emitFieldAccess` /
|
|
||||||
struct-literal init / the `field_ptr` paths). Prove with a REORDERED welded
|
|
||||||
struct used as runtime data: construct + read each field back correct.
|
|
||||||
- **2.4 — interp comptime layout.** The comptime interp represents structs as
|
|
||||||
`Value.aggregate` by logical index — fine for field access. The byte layout
|
|
||||||
matters at the handler boundary: serialize a welded-struct `Value` into
|
|
||||||
Zig-layout memory (via the plan's offsets) so a handler can take `*ZigType`,
|
|
||||||
and read a Zig-layout result back into a `Value`. (Or: keep handlers reading
|
|
||||||
`Value` aggregates logically — decide when wiring `register_struct`.)
|
|
||||||
- **2.5 — `register_struct` / `find_type` handlers.** Bind
|
|
||||||
`register_struct(StructInfo) -> Type` (guarded: dup field names, kind) +
|
|
||||||
`find_type(StringId) -> ?Type` over the host-call bridge, consuming a welded
|
|
||||||
`StructInfo`. Prove: build a struct programmatically + round-trip a source one.
|
|
||||||
- **2.6 — re-express `type_info`/`define` (struct) as sx** over `register_struct`/
|
|
||||||
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
|
`find_type`; migrate `examples/0622`; delete the bespoke struct interp arms
|
||||||
(`defineStruct`/`reflectTypeInfo` struct path). Design build-order steps 2–3.
|
(`defineStruct` / the `reflectTypeInfo` struct path).
|
||||||
|
|
||||||
Then Phase 3+: widen to enum/tuple (`EnumInfo`/`TaggedUnionInfo`/`TupleInfo`,
|
Then Phase 3+: widen the welded types to `EnumInfo`/`TaggedUnionInfo`/`TupleInfo`
|
||||||
optional fields → sentinels), migrate `BuildOptions` to `abi(.zig) extern
|
(optional fields → sentinels) — each just needs an sx header in the compiler
|
||||||
compiler` (the `#compiler` registry re-homes under the `compiler` lib), delete
|
type's memory order + the matching `register_*` fn. Finally migrate `BuildOptions`
|
||||||
|
to `abi(.zig) extern compiler` (re-home the `#compiler` registry) and delete
|
||||||
`#compiler`.
|
`#compiler`.
|
||||||
|
|
||||||
|
Note: a welded struct with an `?T` / `union(enum)` field (e.g. `EnumInfo`'s
|
||||||
|
`backing_type: ?TypeId`, `explicit_values: ?[]const i64`) is the next layout
|
||||||
|
wrinkle — the sx header must mirror Zig's optional/union representation. Handle
|
||||||
|
when reached (sentinels or accessor fns; see the design doc Risks).
|
||||||
|
|
||||||
## Known issues
|
## Known issues
|
||||||
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
|
- None for this stream. (Metatype's deferred enhancement is issue 0141 — comptime
|
||||||
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
|
`List` growth; orthogonal, see `current/CHECKPOINT-METATYPE.md`.)
|
||||||
|
|
||||||
## Log
|
## Log
|
||||||
- **Phase 2.1 — weld-plan layout math + `StructInfo` registered.** Decision:
|
- **Phase 2 — welded structs by reflection + memory-order validation.** Dropped
|
||||||
full byte-layout weld (not logical-field marshalling). `computeWeldPlan`
|
the byte-layout-override engine (computeWeldPlan / offset-ordered LLVM struct /
|
||||||
(offset-order elements + padding + sx→element remap), pure + unit-tested
|
byte-blob — all explored, all unnecessary). Instead: the sx header declares
|
||||||
against `Field` (identity) and `StructInfo` (reordered, remap `[1,0,3,2]`).
|
fields in the compiler type's memory order; the compiler reflects the bound Zig
|
||||||
No emit/interp wiring yet. Build + suite green.
|
type (`@typeInfo`/`@offsetOf`/`@sizeOf`) and validates the header matches with
|
||||||
|
loud diagnostics (field-not-found, wrong-order+expected-order, size mismatch).
|
||||||
|
On pass it's an ordinary byte-identical struct — cast + deref just works.
|
||||||
|
Examples 0627 (usable) / 1186 (wrong-order diagnostic). Suite green (692).
|
||||||
|
- **Phase 2.1 — weld-plan layout math (REMOVED).** The byte-layout-override math;
|
||||||
|
superseded by the reflection+validation design and deleted.
|
||||||
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
|
- **Phase 1 polish — comptime-only enforcement.** A runtime call to a welded fn is
|
||||||
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
|
a clean build-gating error (`emitCall` gate, guarded by enclosing-`is_comptime`
|
||||||
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
|
so `#run`/`::` uses stay green), not a link failure. Example 1185. Build + suite
|
||||||
|
|||||||
@@ -93,6 +93,19 @@ build error — the sx side is a header checked against the implementation). Bec
|
|||||||
the same compiler builds both, they're guaranteed identical, and a `types.zig`
|
the same compiler builds both, they're guaranteed identical, and a `types.zig`
|
||||||
change re-bakes the offsets on the next build — both sides move together.
|
change re-bakes the offsets on the next build — both sides move together.
|
||||||
|
|
||||||
|
> **Implementation note (how it's exact, concretely).** No layout-override engine
|
||||||
|
> is needed. The sx header DECLARES its fields in the compiler type's **memory
|
||||||
|
> order** (Zig may reorder a struct from source order). The compiler REFLECTS the
|
||||||
|
> bound Zig type — field names from `@typeInfo`, offsets from `@offsetOf`, size
|
||||||
|
> from `@sizeOf`, nothing hand-maintained — and VALIDATES the header matches that
|
||||||
|
> memory order, with loud diagnostics on drift (*field not found*, *wrong field
|
||||||
|
> order* + the expected order, *type/layout size mismatch*). On pass the sx
|
||||||
|
> struct's NATURAL layout already equals the Zig layout, so it is an ordinary
|
||||||
|
> struct — no reorder, no padding tricks, no index/remap tables, no special LLVM
|
||||||
|
> path — and `@ptrCast`ing it to the compiler's own type and dereferencing is
|
||||||
|
> byte-identical. When `types.zig` shifts, the header stops matching and the
|
||||||
|
> developer gets a specific message to fix it.
|
||||||
|
|
||||||
This is what C-ABI `extern` can't do: it copies Zig's REAL layout, so Zig slices
|
This is what C-ABI `extern` can't do: it copies Zig's REAL layout, so Zig slices
|
||||||
(`{ptr,len}`), field reordering, and `union(enum)` tag placement all "just work" —
|
(`{ptr,len}`), field reordering, and `union(enum)` tag placement all "just work" —
|
||||||
no slice→ptr+len surgery on `types.zig`, no version fragility.
|
no slice→ptr+len surgery on `types.zig`, no version fragility.
|
||||||
|
|||||||
30
examples/0627-comptime-weld-struct-reflected-layout.sx
Normal file
30
examples/0627-comptime-weld-struct-reflected-layout.sx
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
// Comptime compiler API — a welded struct mirrors the compiler's real Zig type
|
||||||
|
// byte-for-byte by declaring its fields in the compiler type's MEMORY order.
|
||||||
|
//
|
||||||
|
// `StructInfo` is the real `types.TypeInfo.StructInfo`, which Zig reorders from
|
||||||
|
// source order to (fields, name, nominal_id, is_protocol). The sx header declares
|
||||||
|
// the fields in that memory order; the compiler reflects the bound Zig type
|
||||||
|
// (@offsetOf/@sizeOf) and validates the header matches — so the struct is laid
|
||||||
|
// out identically and a pointer to it can be cast to the compiler's own type and
|
||||||
|
// dereferenced. Nothing is maintained by hand: a types.zig change re-reflects.
|
||||||
|
|
||||||
|
#import "modules/std.sx";
|
||||||
|
|
||||||
|
compiler :: #library "compiler";
|
||||||
|
|
||||||
|
Field :: struct abi(.zig) extern compiler { name: u32; ty: u32; }
|
||||||
|
|
||||||
|
StructInfo :: struct abi(.zig) extern compiler {
|
||||||
|
fields: []Field;
|
||||||
|
name: u32;
|
||||||
|
nominal_id: u32;
|
||||||
|
is_protocol: bool;
|
||||||
|
}
|
||||||
|
|
||||||
|
main :: () {
|
||||||
|
si : StructInfo = ---;
|
||||||
|
si.name = 42;
|
||||||
|
si.nominal_id = 7;
|
||||||
|
si.is_protocol = true;
|
||||||
|
print("name={} nominal={} proto={}\n", si.name, si.nominal_id, si.is_protocol);
|
||||||
|
}
|
||||||
20
examples/1186-diagnostics-weld-struct-wrong-order.sx
Normal file
20
examples/1186-diagnostics-weld-struct-wrong-order.sx
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
// Diagnostic: a welded struct whose fields are NOT in the compiler type's memory
|
||||||
|
// order is a loud build error — the sx header must mirror the real Zig layout so
|
||||||
|
// the two are byte-identical. The message names the offending position and shows
|
||||||
|
// the expected memory order. (Declaring StructInfo in source order trips this:
|
||||||
|
// Zig reorders it to fields-first.)
|
||||||
|
|
||||||
|
#import "modules/std.sx";
|
||||||
|
|
||||||
|
compiler :: #library "compiler";
|
||||||
|
|
||||||
|
Field :: struct abi(.zig) extern compiler { name: u32; ty: u32; }
|
||||||
|
|
||||||
|
StructInfo :: struct abi(.zig) extern compiler {
|
||||||
|
name: u32;
|
||||||
|
fields: []Field;
|
||||||
|
is_protocol: bool;
|
||||||
|
nominal_id: u32;
|
||||||
|
}
|
||||||
|
|
||||||
|
main :: () { print("unreached\n"); }
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
0
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
name=42 nominal=7 proto=true
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
error: welded type 'Field' has 2 field(s) in the compiler library but the declaration has 1
|
error: welded type 'Field': the compiler type has 2 field(s) but the declaration has 1 — declare them in memory order: name, ty
|
||||||
--> examples/1183-diagnostics-weld-struct-field-count.sx:12:51
|
--> examples/1183-diagnostics-weld-struct-field-count.sx:12:51
|
||||||
|
|
|
|
||||||
12 | Field :: struct abi(.zig) extern compiler { name: u32; }
|
12 | Field :: struct abi(.zig) extern compiler { name: u32; }
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
1
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
error: welded type 'StructInfo': wrong field order at position 0 — found 'name', the compiler type has 'fields' here (memory order: fields, name, nominal_id, is_protocol)
|
||||||
|
--> examples/1186-diagnostics-weld-struct-wrong-order.sx:14:11
|
||||||
|
|
|
||||||
|
14 | name: u32;
|
||||||
|
| ^^^
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
|
||||||
@@ -104,66 +104,32 @@ test "compiler_lib: validateStructLayout flags each kind of drift" {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Lock: `Field` (natural two-u32 layout) has the trivial weld plan — two field
|
// Lock: `StructInfo` is reflected in MEMORY order — Zig reorders it from source
|
||||||
// elements in declaration order, no padding, identity remap.
|
// order (name, fields, is_protocol, nominal_id) to (fields@0, name@16,
|
||||||
test "compiler_lib: weld plan for Field is the identity (no reorder, no pad)" {
|
// nominal_id@20, is_protocol@24). The registry must present the fields in that
|
||||||
const alloc = std.testing.allocator;
|
// memory order, since an sx welded header must declare them so to be
|
||||||
const bt = compiler_lib.findType("Field").?;
|
// byte-identical.
|
||||||
var plan = try compiler_lib.computeWeldPlan(alloc, bt.fields, bt.size);
|
test "compiler_lib: StructInfo is reflected in Zig memory order" {
|
||||||
defer plan.deinit(alloc);
|
const StructInfoZig = types.TypeInfo.StructInfo;
|
||||||
|
|
||||||
try std.testing.expectEqual(@as(usize, 8), plan.total_size);
|
|
||||||
try std.testing.expectEqual(@as(usize, 2), plan.elements.len);
|
|
||||||
// Identity remap.
|
|
||||||
try std.testing.expectEqual(@as(usize, 0), plan.sx_to_llvm[0]);
|
|
||||||
try std.testing.expectEqual(@as(usize, 1), plan.sx_to_llvm[1]);
|
|
||||||
// Both elements are real fields at 0 and 4.
|
|
||||||
try std.testing.expectEqual(@as(?usize, 0), plan.elements[0].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 0), plan.elements[0].offset);
|
|
||||||
try std.testing.expectEqual(@as(?usize, 1), plan.elements[1].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 4), plan.elements[1].offset);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Lock: `StructInfo` is the first NON-natural layout — Zig reorders it to
|
|
||||||
// (fields@0, name@16, nominal_id@20, is_protocol@24) with a 7-byte alignment
|
|
||||||
// tail. The plan must reproduce exactly that order + the sx→element remap, so the
|
|
||||||
// LLVM type built from it is byte-identical to the Zig type. sx declaration order
|
|
||||||
// is (name, fields, is_protocol, nominal_id) = sx indices 0,1,2,3.
|
|
||||||
test "compiler_lib: weld plan for StructInfo reorders + pads to the Zig layout" {
|
|
||||||
const alloc = std.testing.allocator;
|
|
||||||
const bt = compiler_lib.findType("StructInfo").?;
|
const bt = compiler_lib.findType("StructInfo").?;
|
||||||
var plan = try compiler_lib.computeWeldPlan(alloc, bt.fields, bt.size);
|
|
||||||
defer plan.deinit(alloc);
|
|
||||||
|
|
||||||
try std.testing.expectEqual(@as(usize, 32), plan.total_size);
|
try std.testing.expectEqual(@sizeOf(StructInfoZig), bt.size);
|
||||||
|
try std.testing.expectEqual(@as(usize, 4), bt.fields.len);
|
||||||
|
|
||||||
// Elements in ascending offset order: fields, name, nominal_id, is_protocol,
|
// Memory order: fields, name, nominal_id, is_protocol.
|
||||||
// then a trailing 7-byte pad (25 → 32).
|
try std.testing.expectEqualStrings("fields", bt.fields[0].name);
|
||||||
try std.testing.expectEqual(@as(usize, 5), plan.elements.len);
|
try std.testing.expectEqual(@offsetOf(StructInfoZig, "fields"), bt.fields[0].offset);
|
||||||
|
try std.testing.expectEqualStrings("name", bt.fields[1].name);
|
||||||
|
try std.testing.expectEqual(@offsetOf(StructInfoZig, "name"), bt.fields[1].offset);
|
||||||
|
try std.testing.expectEqualStrings("nominal_id", bt.fields[2].name);
|
||||||
|
try std.testing.expectEqual(@offsetOf(StructInfoZig, "nominal_id"), bt.fields[2].offset);
|
||||||
|
try std.testing.expectEqualStrings("is_protocol", bt.fields[3].name);
|
||||||
|
try std.testing.expectEqual(@offsetOf(StructInfoZig, "is_protocol"), bt.fields[3].offset);
|
||||||
|
|
||||||
// elem 0: fields (sx index 1) @ 0, size 16
|
// Offsets are strictly ascending (memory order).
|
||||||
try std.testing.expectEqual(@as(?usize, 1), plan.elements[0].sx_field);
|
try std.testing.expect(bt.fields[0].offset < bt.fields[1].offset);
|
||||||
try std.testing.expectEqual(@as(usize, 0), plan.elements[0].offset);
|
try std.testing.expect(bt.fields[1].offset < bt.fields[2].offset);
|
||||||
try std.testing.expectEqual(@as(usize, 16), plan.elements[0].size);
|
try std.testing.expect(bt.fields[2].offset < bt.fields[3].offset);
|
||||||
// elem 1: name (sx index 0) @ 16, size 4
|
|
||||||
try std.testing.expectEqual(@as(?usize, 0), plan.elements[1].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 16), plan.elements[1].offset);
|
|
||||||
// elem 2: nominal_id (sx index 3) @ 20, size 4
|
|
||||||
try std.testing.expectEqual(@as(?usize, 3), plan.elements[2].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 20), plan.elements[2].offset);
|
|
||||||
// elem 3: is_protocol (sx index 2) @ 24, size 1
|
|
||||||
try std.testing.expectEqual(@as(?usize, 2), plan.elements[3].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 24), plan.elements[3].offset);
|
|
||||||
// elem 4: trailing pad @ 25, size 7
|
|
||||||
try std.testing.expectEqual(@as(?usize, null), plan.elements[4].sx_field);
|
|
||||||
try std.testing.expectEqual(@as(usize, 25), plan.elements[4].offset);
|
|
||||||
try std.testing.expectEqual(@as(usize, 7), plan.elements[4].size);
|
|
||||||
|
|
||||||
// sx → element remap: name→1, fields→0, is_protocol→3, nominal_id→2.
|
|
||||||
try std.testing.expectEqual(@as(usize, 1), plan.sx_to_llvm[0]);
|
|
||||||
try std.testing.expectEqual(@as(usize, 0), plan.sx_to_llvm[1]);
|
|
||||||
try std.testing.expectEqual(@as(usize, 3), plan.sx_to_llvm[2]);
|
|
||||||
try std.testing.expectEqual(@as(usize, 2), plan.sx_to_llvm[3]);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Lock: the welded-function export list resolves the round-trip readers and
|
// Lock: the welded-function export list resolves the round-trip readers and
|
||||||
|
|||||||
@@ -49,26 +49,31 @@ pub const BoundType = struct {
|
|||||||
const FieldZig = types.TypeInfo.StructInfo.Field; // { name: StringId, ty: TypeId } — two u32s
|
const FieldZig = types.TypeInfo.StructInfo.Field; // { name: StringId, ty: TypeId } — two u32s
|
||||||
const StructInfoZig = types.TypeInfo.StructInfo; // { name, fields: []Field, is_protocol, nominal_id } — Zig-reordered
|
const StructInfoZig = types.TypeInfo.StructInfo; // { name, fields: []Field, is_protocol, nominal_id } — Zig-reordered
|
||||||
|
|
||||||
/// Bake a `BoundType` from a real Zig struct type `T`. Field offsets/sizes come
|
/// Bake a `BoundType` by REFLECTING the real Zig struct type `T` — field names
|
||||||
/// from `@offsetOf`/`@sizeOf` on `T`; `sx_field_names` supplies the sx-visible
|
/// from `@typeInfo`, offsets from `@offsetOf`, sizes from `@sizeOf`. Nothing is
|
||||||
/// names positionally (must match `T`'s field order and count — a mismatch is a
|
/// maintained by hand: a `types.zig` change re-bakes on the next compiler build.
|
||||||
/// compile error, never a silent truncation).
|
/// Fields are returned in ascending-OFFSET (memory) order, which is the order an
|
||||||
fn weldStruct(
|
/// sx welded header must declare them in to be byte-identical (Zig may reorder a
|
||||||
comptime sx_name: []const u8,
|
/// struct's fields from source order). The sx-visible field name IS the Zig
|
||||||
comptime T: type,
|
/// field identifier.
|
||||||
comptime sx_field_names: []const []const u8,
|
fn weldStruct(comptime sx_name: []const u8, comptime T: type) BoundType {
|
||||||
) BoundType {
|
|
||||||
const zig_fields = @typeInfo(T).@"struct".fields;
|
const zig_fields = @typeInfo(T).@"struct".fields;
|
||||||
if (zig_fields.len != sx_field_names.len)
|
|
||||||
@compileError("compiler-lib weld '" ++ sx_name ++ "': sx field count != Zig field count");
|
|
||||||
comptime var layouts: [zig_fields.len]FieldLayout = undefined;
|
comptime var layouts: [zig_fields.len]FieldLayout = undefined;
|
||||||
inline for (zig_fields, 0..) |zf, i| {
|
inline for (zig_fields, 0..) |zf, i| {
|
||||||
layouts[i] = .{
|
layouts[i] = .{
|
||||||
.name = sx_field_names[i],
|
.name = zf.name,
|
||||||
.offset = @offsetOf(T, zf.name),
|
.offset = @offsetOf(T, zf.name),
|
||||||
.size = @sizeOf(zf.type),
|
.size = @sizeOf(zf.type),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
// Sort into memory order so the sx header is checked against the layout the
|
||||||
|
// compiler actually uses (declaration order != memory order under Zig's
|
||||||
|
// auto-layout).
|
||||||
|
comptime std.sort.insertion(FieldLayout, &layouts, {}, struct {
|
||||||
|
fn lt(_: void, a: FieldLayout, b: FieldLayout) bool {
|
||||||
|
return a.offset < b.offset;
|
||||||
|
}
|
||||||
|
}.lt);
|
||||||
const frozen = layouts;
|
const frozen = layouts;
|
||||||
return .{
|
return .{
|
||||||
.sx_name = sx_name,
|
.sx_name = sx_name,
|
||||||
@@ -78,14 +83,13 @@ fn weldStruct(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The welded-type export list. `Field` (two u32s, natural layout) proved the
|
/// The welded-type export list. Each entry reflects a real internal Zig type;
|
||||||
/// weld in Phase 1; `StructInfo` (Phase 2) is the first NON-natural layout —
|
/// the sx header that binds it must mirror these fields IN THIS (memory) ORDER.
|
||||||
/// Zig reorders its fields (`fields`@0, `name`@16, `nominal_id`@20,
|
/// `Field` (two u32s) is naturally ordered; `StructInfo` is Zig-reordered
|
||||||
/// `is_protocol`@24), so it exercises the offset-override engine. `EnumInfo` /
|
/// (`fields`@0, `name`@16, `nominal_id`@20, `is_protocol`@24).
|
||||||
/// `TaggedUnionInfo` / `TupleInfo` join later.
|
|
||||||
pub const bound_types = [_]BoundType{
|
pub const bound_types = [_]BoundType{
|
||||||
weldStruct("Field", FieldZig, &.{ "name", "ty" }),
|
weldStruct("Field", FieldZig),
|
||||||
weldStruct("StructInfo", StructInfoZig, &.{ "name", "fields", "is_protocol", "nominal_id" }),
|
weldStruct("StructInfo", StructInfoZig),
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Look up a welded type by its sx name. Returns null when the name is not on
|
/// Look up a welded type by its sx name. Returns null when the name is not on
|
||||||
@@ -150,91 +154,6 @@ pub fn validateStructLayout(
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Weld plan (byte-layout override) ────────────────────────────────────────
|
|
||||||
//
|
|
||||||
// A welded struct must be laid out byte-identically to the bound Zig type, whose
|
|
||||||
// fields Zig may REORDER (and pad). The sx struct's natural layout generally
|
|
||||||
// won't match — so the compiler imposes the Zig layout: it builds the struct's
|
|
||||||
// LLVM type as the fields in ascending-OFFSET order, with explicit padding
|
|
||||||
// elements filling the gaps, and remaps each sx field index to its LLVM element
|
|
||||||
// index. `computeWeldPlan` is that pure layout math; the LLVM type builder + the
|
|
||||||
// struct-GEP / field-access sites consume the plan (later sub-steps), and the
|
|
||||||
// interp serializes comptime struct Values through the same offsets.
|
|
||||||
|
|
||||||
/// One element of a welded struct's LLVM layout: either a real field (carrying
|
|
||||||
/// its sx field index) or a padding gap. Always in ascending `offset` order.
|
|
||||||
pub const WeldElement = struct {
|
|
||||||
/// The sx field index this element holds, or null for a padding gap.
|
|
||||||
sx_field: ?usize,
|
|
||||||
/// Byte offset of this element within the struct (the bound Zig offset).
|
|
||||||
offset: usize,
|
|
||||||
/// Byte width of this element (the field's size, or the gap width).
|
|
||||||
size: usize,
|
|
||||||
};
|
|
||||||
|
|
||||||
/// The byte-layout plan for a welded struct: its LLVM elements in offset order
|
|
||||||
/// (fields + padding) and the sx-field → LLVM-element-index remap. Owns its
|
|
||||||
/// slices — `deinit` with the same allocator passed to `computeWeldPlan`.
|
|
||||||
pub const WeldPlan = struct {
|
|
||||||
elements: []const WeldElement,
|
|
||||||
/// `sx_to_llvm[i]` is the index into `elements` of sx field `i`.
|
|
||||||
sx_to_llvm: []const usize,
|
|
||||||
total_size: usize,
|
|
||||||
|
|
||||||
pub fn deinit(self: *WeldPlan, alloc: std.mem.Allocator) void {
|
|
||||||
alloc.free(self.elements);
|
|
||||||
alloc.free(self.sx_to_llvm);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
/// Compute the byte-layout plan for a struct whose fields carry their bound Zig
|
|
||||||
/// offsets (`fields[i].offset`/`.size`, e.g. from a `BoundType`). `total_size` is
|
|
||||||
/// the bound Zig `@sizeOf`. The result lists LLVM elements in ascending-offset
|
|
||||||
/// order — real fields interleaved with padding gaps — plus the sx-field →
|
|
||||||
/// element-index remap that struct-GEP uses. Pure; allocates the result slices.
|
|
||||||
pub fn computeWeldPlan(
|
|
||||||
alloc: std.mem.Allocator,
|
|
||||||
fields: []const FieldLayout,
|
|
||||||
total_size: usize,
|
|
||||||
) !WeldPlan {
|
|
||||||
// Order the sx field indices by ascending byte offset (stable).
|
|
||||||
const order = try alloc.alloc(usize, fields.len);
|
|
||||||
defer alloc.free(order);
|
|
||||||
for (order, 0..) |*o, i| o.* = i;
|
|
||||||
std.sort.insertion(usize, order, fields, struct {
|
|
||||||
fn lessThan(fs: []const FieldLayout, a: usize, b: usize) bool {
|
|
||||||
return fs[a].offset < fs[b].offset;
|
|
||||||
}
|
|
||||||
}.lessThan);
|
|
||||||
|
|
||||||
var elements = std.ArrayList(WeldElement).empty;
|
|
||||||
errdefer elements.deinit(alloc);
|
|
||||||
const sx_to_llvm = try alloc.alloc(usize, fields.len);
|
|
||||||
errdefer alloc.free(sx_to_llvm);
|
|
||||||
|
|
||||||
var cursor: usize = 0;
|
|
||||||
for (order) |sx_i| {
|
|
||||||
const f = fields[sx_i];
|
|
||||||
// Fill any gap before this field with a padding element.
|
|
||||||
if (f.offset > cursor) {
|
|
||||||
try elements.append(alloc, .{ .sx_field = null, .offset = cursor, .size = f.offset - cursor });
|
|
||||||
}
|
|
||||||
sx_to_llvm[sx_i] = elements.items.len;
|
|
||||||
try elements.append(alloc, .{ .sx_field = sx_i, .offset = f.offset, .size = f.size });
|
|
||||||
cursor = f.offset + f.size;
|
|
||||||
}
|
|
||||||
// Trailing padding up to the bound total size (alignment tail).
|
|
||||||
if (total_size > cursor) {
|
|
||||||
try elements.append(alloc, .{ .sx_field = null, .offset = cursor, .size = total_size - cursor });
|
|
||||||
}
|
|
||||||
|
|
||||||
return .{
|
|
||||||
.elements = try elements.toOwnedSlice(alloc),
|
|
||||||
.sx_to_llvm = sx_to_llvm,
|
|
||||||
.total_size = total_size,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Functions (comptime-only, host-call bridged) ────────────────────────────
|
// ── Functions (comptime-only, host-call bridged) ────────────────────────────
|
||||||
|
|
||||||
/// A welded `compiler` function: dispatched under the comptime interpreter to its
|
/// A welded `compiler` function: dispatched under the comptime interpreter to its
|
||||||
|
|||||||
@@ -753,14 +753,40 @@ fn validateWeldedStruct(self: *Lowering, sd: *const ast.StructDecl, tid: TypeId,
|
|||||||
const total = table.typeSizeBytes(tid);
|
const total = table.typeSizeBytes(tid);
|
||||||
|
|
||||||
const mismatch = compiler_lib.validateStructLayout(bt, sx_fields.items, total) orelse return;
|
const mismatch = compiler_lib.validateStructLayout(bt, sx_fields.items, total) orelse return;
|
||||||
|
// The compiler type's fields, in the memory order an sx header must mirror —
|
||||||
|
// included in the order/count diagnostics so the fix is obvious.
|
||||||
|
const order = weldedFieldOrderStr(self.alloc, bt);
|
||||||
|
defer if (order.len > 0) self.alloc.free(order);
|
||||||
switch (mismatch) {
|
switch (mismatch) {
|
||||||
.field_count => |m| diags.addFmt(.err, span, "welded type '{s}' has {d} field(s) in the compiler library but the declaration has {d}", .{ sd.name, m.expected, m.got }),
|
.field_count => |m| diags.addFmt(.err, span, "welded type '{s}': the compiler type has {d} field(s) but the declaration has {d} — declare them in memory order: {s}", .{ sd.name, m.expected, m.got, order }),
|
||||||
.field_name => |m| diags.addFmt(.err, span, "welded type '{s}' field {d} is named '{s}' in the compiler library, not '{s}'", .{ sd.name, m.index, m.expected, m.got }),
|
.field_name => |m| {
|
||||||
.field_size => |m| diags.addFmt(.err, span, "welded type '{s}' field '{s}' is {d} byte(s) in the compiler library but {d} as declared", .{ sd.name, m.name, m.expected, m.got }),
|
// Distinguish "this name isn't a field at all" from "right field set,
|
||||||
.total_size => |m| diags.addFmt(.err, span, "welded type '{s}' is {d} byte(s) in the compiler library but {d} as declared (padding/alignment mismatch)", .{ sd.name, m.expected, m.got }),
|
// wrong order".
|
||||||
|
const exists = blk: {
|
||||||
|
for (bt.fields) |bf| if (std.mem.eql(u8, bf.name, m.got)) break :blk true;
|
||||||
|
break :blk false;
|
||||||
|
};
|
||||||
|
if (exists)
|
||||||
|
diags.addFmt(.err, span, "welded type '{s}': wrong field order at position {d} — found '{s}', the compiler type has '{s}' here (memory order: {s})", .{ sd.name, m.index, m.got, m.expected, order })
|
||||||
|
else
|
||||||
|
diags.addFmt(.err, span, "welded type '{s}': field '{s}' is not a field of the compiler type (its fields, in memory order: {s})", .{ sd.name, m.got, order });
|
||||||
|
},
|
||||||
|
.field_size => |m| diags.addFmt(.err, span, "welded type '{s}': type layout mismatch — field '{s}' is {d} byte(s) in the compiler type but {d} as declared", .{ sd.name, m.name, m.expected, m.got }),
|
||||||
|
.total_size => |m| diags.addFmt(.err, span, "welded type '{s}': layout mismatch — the compiler type is {d} byte(s) but the declaration is {d} (alignment/padding)", .{ sd.name, m.expected, m.got }),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The bound type's field names in memory order, `, `-joined, for diagnostics.
|
||||||
|
/// Returns an owned string; empty (no free needed) on allocation failure.
|
||||||
|
fn weldedFieldOrderStr(alloc: std.mem.Allocator, bt: *const compiler_lib.BoundType) []const u8 {
|
||||||
|
var buf = std.ArrayList(u8).empty;
|
||||||
|
for (bt.fields, 0..) |bf, i| {
|
||||||
|
if (i > 0) buf.appendSlice(alloc, ", ") catch return "";
|
||||||
|
buf.appendSlice(alloc, bf.name) catch return "";
|
||||||
|
}
|
||||||
|
return buf.toOwnedSlice(alloc) catch "";
|
||||||
|
}
|
||||||
|
|
||||||
/// Register a top-level ENUM decl under a per-decl nominal identity (E6a) —
|
/// Register a top-level ENUM decl under a per-decl nominal identity (E6a) —
|
||||||
/// the enum twin of `registerStructDecl`. A GENUINE same-name shadow already
|
/// the enum twin of `registerStructDecl`. A GENUINE same-name shadow already
|
||||||
/// reserved its DISTINCT slot up-front in `scanDecls` (the first at id 0, the
|
/// reserved its DISTINCT slot up-front in `scanDecls` (the first at id 0, the
|
||||||
|
|||||||
Reference in New Issue
Block a user