101 lines
31 KiB
Markdown
101 lines
31 KiB
Markdown
<!-- formerly generated by the retired flow harness; hand-maintained since the 2026-06-11 sx re-evaluation -->
|
|
|
|
# Distribution Platform — Milestone 1, Slice A: local content-addressed `dist ci publish`
|
|
|
|
First independently-shippable slice of PLAN.md's Milestone 1, scoped to what is implementable AND verifiable entirely inside /Users/agra/projects/distribution. The deliverable is the local CI publish path: `dist ci publish --manifest dist.json --local-store .dist --json` parses a manifest, creates/finds an app, drafts a release, stores each artifact content-addressed by SHA-256 in a local object store, runs common validation, publishes, promotes the requested channel, persists state to a human-readable db.json, and prints stable JSON (release id, artifact ids, digests, local download URLs). This is subplan 03 Slice 2 (local publish) standing on subplan 02 Slice 1+3 (in-memory domain + content-addressed storage) and the common-validation subset of subplan 05 — deliberately the no-server, no-SQLite, no-auth core so it dogfoods without a running distd.
|
|
|
|
PHASE-0 / CROSS-REPO DECISION (hard constraint 2). Subplan 01 (sx language + stdlib) lives in the SEPARATE sx repo (/Users/agra/projects/sx) and is OUT OF SCOPE — no step here edits the sx compiler or stdlib. RE-EVALUATED 2026-06-11 against current sx: the stdlib restructure moved everything under `modules/std/` behind the `modules/std.sx` facade — the flat legacy `modules/fs.sx`-style files are deleted. There is no `pub` keyword; aliases are the re-export mechanism, and the facade's namespace tail (`mem`, `fs`, `process`, `socket`, `json`, `xml`, `cli`, `hash`, `log`, `test`) carries one level into flat importers. sx primitives this milestone USES: `List(T)`, `print`/`format`, string helpers (`concat`, `substr`, `int_to_string`, `int_to_hex_string`, `path_join`), `std.fs` (read_file/write_file/exists/create_dir_all/copy_file/rename/basename/dirname), `std.process` (run/env/find_executable/exit/assert), `std.json` (Value/Object/Sink reader+writer), `std.cli` (argv, `<group> <command>` dispatch, flag specs, exit codes), `std.hash` (streaming SHA-256), `?T` optionals and the `!` error channel (`catch`/`onfail` bindings take parens), `Allocator.alloc_bytes`/`dealloc_bytes` plus the typed `std.mem` helpers, and reflection builtins. The originally planned TEMPORARY IN-REPO shims under `src/infra/` (hash/json/cli) were SUPERSEDED before being written — sx shipped `std.hash`, `std.json`, and `std.cli` natively, the product code uses them directly, and `src/infra/` has been removed. Pieces still missing from sx and intentionally avoided for this slice: HashMap (use linear scan over `List` pairs for tiny N), StringBuilder (use `concat`/`format`), std.time (publish.sx shims `time(2)` locally via FFI), real SQLite (replaced by a human-readable `db.json` in the store), HTTP/TLS (no server), archive/zip inspection (deep IPA/APK validation deferred), token-at-rest (no network auth in local mode).
|
|
|
|
DEFERRED to later slices/runs (named per constraint 3): subplan 01 sx-language work (`pub` exports, alias imports, namespace re-exports) and the bulk of the Phase 1 stdlib upstreaming; real SQLite persistence (subplan 02 Slice 2); retention/cleanup (02 Slice 4); token security at rest + scoped auth (02 Slice 5); the HTTP API and install pages (subplan 04 / Phases 5,7); remote publish + streaming upload + retries (03 Slice 3); deep platform-specific IPA/APK/macOS/Linux/Windows archive validation (subplan 05); admin UI (subplan 06 / Phase 8); Docker + UGREEN NAS packaging (subplan 07 / Phase 9); CI templates and observability polish (03 Slices 4-5).
|
|
|
|
## Acceptance criteria
|
|
|
|
- A build/test gate exists in the distribution repo (a `Makefile` + `tests/run.sh` that locate the sx binary via `SX ?= /Users/agra/projects/sx/zig-out/bin/sx`): `make test` runs every `tests/**/*.sx` test through the sx compiler, prints per-test ok/FAIL, exits 0 when all pass and non-zero on any failure; an intentionally-failing probe test makes it exit non-zero (gate proven, not vacuous).
|
|
- (SUPERSEDED 2026-06-11) The three planned `src/infra/` pure-sx shims were replaced by the sx stdlib before being written: SHA-256 comes from `std.hash`, JSON from `std.json`, and subcommand/flag parsing from `std.cli`, used directly from `modules/std/` with no edits to `/Users/agra/projects/sx`. Their behavior is exercised in-repo by the store, manifest, and CLI tests under `tests/`.
|
|
- `./dist ci publish --manifest examples/dist.json --local-store <dir> --json` (invoked via the gate, e.g. `make publish-example`) runs end-to-end against a fixture app whose manifest lists at least one Android APK and one iOS IPA artifact, exits 0, and prints a single pure-JSON object on stdout containing the release id, one artifact id + sha256 digest per artifact, and a local download URL/path per artifact (no human-only text on stdout under `--json`).
|
|
- Each published artifact's bytes are physically present in the local store under its SHA-256 key (e.g. `<dir>/objects/<sha256>`), the recorded digest equals an independent `shasum -a 256` of the source file, and re-running publish with byte-identical artifacts does not create a second copy of those bytes (content-addressed dedup).
|
|
- Platform state persists across separate CLI invocations via a human-readable `db.json` (or per-entity sidecars) under the store recording apps, releases, artifacts, channels, and audit events; a follow-up `./dist release rollback` (and `release promote`) invocation reads that state, moves the channel pointer, appends an audit event, and the change is verifiable by inspecting the store between invocations.
|
|
- Failure paths are loud and machine-readable: a malformed manifest, a missing artifact file, an unknown platform id, or a SHA-256/size mismatch each yields a non-zero exit and a JSON error object under `--json` (error code + message), with no partially-published release left pointed at by a channel.
|
|
- Every step is implemented and verified inside /Users/agra/projects/distribution using the sx compiler; the git working tree builds and `make test` is green at each step boundary; no change is made to the sx compiler or stdlib.
|
|
|
|
## Phases
|
|
|
|
### P1 — Build/test gate + in-repo sx infra shims (the greenfield foundation)
|
|
|
|
- **P1.1** Establish repo skeleton + sx build/test gate
|
|
- intent: Stand up the greenfield gate: a product source layout (`src/`, `src/infra/`, `tests/`, `examples/`) and a reproducible way to compile+run sx in this repo. Add a `Makefile` with `build`/`test`/`publish-example` targets and a `tests/run.sh` runner (mirroring sx's own tests/run_examples.sh convention) that locates the compiler via `SX ?= /Users/agra/projects/sx/zig-out/bin/sx`, runs each `tests/**/*.sx` via `$SX run`, and reports per-test pass/fail with a non-zero aggregate exit on any failure. Include one trivial passing sx program (imports `modules/std.sx`, prints, exits 0) to prove toolchain wiring.
|
|
- acceptance: `make test` exits 0 and prints `ok` for the smoke test; temporarily adding a deliberately-failing test makes `make test` exit non-zero (then remove it). `make build` compiles the smoke program without error. No edits outside the distribution repo.
|
|
- **P1.2** infra/hash.sx — SHA-256 (pure sx, known-answer tested)
|
|
- SUPERSEDED 2026-06-11: sx shipped `std.hash` (streaming SHA-256, lowercase hex) in `modules/std/hash.sx`; the store and validation code use it directly (see P2.2/P3.3). No in-repo shim exists.
|
|
- **P1.3** infra/json.sx — minimal JSON reader + writer (round-trip tested)
|
|
- SUPERSEDED 2026-06-11: sx shipped `std.json` (reader + writer, `Value`/`Object`/`Sink`) in `modules/std/json.sx`; manifest parsing, `db.json` persistence, and `--json` output use it directly (see P3.2/P2.3/P3.1). No in-repo shim exists.
|
|
- **P1.4** infra/cli.sx — minimal subcommand/flag parser (tested)
|
|
- SUPERSEDED 2026-06-11: sx shipped `std.cli` (argv access, `<group> <command>` dispatch, flag specs, exit-code contract) in `modules/std/cli.sx`; the `dist` entrypoint uses it directly (see P3.1). No in-repo shim exists.
|
|
|
|
### P2 — Product domain model + content-addressed store (in-memory, file-backed)
|
|
|
|
- **P2.1** Domain structs + boundary validation
|
|
- intent: Define the core domain in sx under `src/domain/` per subplan 02: App (id, slug, display_name, bundle ids by platform, owner, visibility, timestamps), Platform enum (ios, android_apk, macos, linux, windows), Release (id, app_id, version, build, channel, notes, created_by, timestamps), Artifact (id, app_id, release_id, platform, filename, content_type, size_bytes, sha256, storage_key, validation_status), Channel (app_id, name, current_release_id, policy, rollout_percent), AuditEvent (id, actor, action, target_type, target_id, metadata, created_at). Add domain validation for slugs, semantic-ish versions, channel names, and platform ids (subplan 02 Slice 1). Lookups use linear scan over `List` (HashMap deferred).
|
|
- acceptance: A test constructs valid and invalid Apps/Releases/Artifacts and asserts the validator accepts the valid ones and rejects each invalid case (bad slug, empty version, unknown platform, bad channel name) with a typed error. `make test` green.
|
|
- **P2.2** Content-addressed artifact store (staging → atomic move, dedup)
|
|
- intent: Implement a local artifact store under `src/store/` (subplan 02 Slice 3): write incoming bytes to a staging path, compute SHA-256 (P1.2), place the object at `<root>/objects/<sha256>` via atomic rename, and skip the copy when the object already exists. Storage key = the digest. Interrupted/failed writes must not leave a published object.
|
|
- acceptance: A test stores a fixture file, asserts the object exists at `objects/<sha256>` with matching bytes, asserts the returned storage_key equals the digest, and asserts storing identical bytes twice yields one object (dedup) while a staging failure leaves no object. `make test` green.
|
|
- **P2.3** In-memory repository + db.json persistence (SQLite stand-in)
|
|
- intent: Implement an in-memory repository over the domain (`create`/`get`/`list`/`update` for apps, releases, artifacts, channels, audit events) with create/find-by-slug, find-artifact-by-digest, and an atomic-ish publish transaction wrapper, plus serialize/reload of the whole model to a human-readable `<root>/db.json` via infra/json (subplan 02 Slice 1; the explicit temporary boundary replacing subplan 02 Slice 2's SQLite). Persistence is required so separate CLI invocations share state.
|
|
- acceptance: A test creates an app+release+artifacts, writes db.json, reloads into a fresh repository, and asserts the reloaded model equals the original; a test asserts a failed publish transaction leaves no dangling release referenced by a channel. `make test` green and db.json is valid JSON re-parseable by infra/json.
|
|
|
|
### P3 — `dist` CLI — local content-addressed publish path (the shippable product)
|
|
|
|
- **P3.1** `dist` entrypoint: parser wiring, help, exit-code + --json contract
|
|
- intent: Create the `dist` program under `src/` wiring infra/cli to subcommand handlers (`ci publish`, `release promote`, `release rollback` as stubs for now), with help text and the exit-code contract. Enforce `--json` purity: in JSON mode only machine-readable JSON goes to stdout; human text goes to stderr (subplan 03 Slice 1).
|
|
- acceptance: `make build` builds `dist`; running it with no args / an unknown command prints readable help/error and a non-zero exit; running a stub command with `--json` emits only JSON on stdout (verified by piping stdout through infra/json's parser in a test). `make test` green.
|
|
- **P3.2** Manifest model + parse + validation
|
|
- intent: Define the v0 manifest shape (app slug, version, channel, artifacts[] each with platform, path, optional filename/content-type override, metadata) and parse it via infra/json into a typed struct, validating required fields and that each artifact path exists and platform is known (subplan 03 Manifest Shape). Add `examples/dist.json` plus small fixture APK/IPA byte files under `examples/`.
|
|
- acceptance: A test parses `examples/dist.json` into the manifest struct and asserts fields; tests assert that a missing required field, an unknown platform, and a non-existent artifact path each fail with a machine-readable error. `make test` green.
|
|
- **P3.3** Common artifact validation subset
|
|
- intent: Implement the platform-agnostic validation pass (subplan 05 Common Validation subset): file exists, size matches, SHA-256 matches the stored digest, content type allowed, extension matches platform policy (.apk→android_apk, .ipa→ios, etc.). Result is a validation_status (pending/passed/failed) with reason. Deep IPA/APK zip inspection is explicitly deferred.
|
|
- acceptance: A test runs validation on a good fixture (passed) and on tampered cases — wrong size, digest mismatch, .apk declared as ios — each yielding failed with a specific reason. `make test` green.
|
|
- **P3.4** `dist ci publish --local-store --json` end-to-end
|
|
- intent: Implement the local publish path (subplan 03 Slice 2 + the PLAN CI contract, local mode): validate manifest → create/find app → create draft release → store each artifact content-addressed (P2.2) → run common validation (P3.3) → publish the release → promote the requested channel if specified → write an audit event per upload/publish/promotion → persist db.json (P2.3) → print stable JSON (release id, artifact ids, digests, local download URLs). On any validation failure, abort without leaving a channel pointing at the release.
|
|
- acceptance: `make publish-example` runs `dist ci publish --manifest examples/dist.json --local-store <tmp> --json` on a 2-artifact (APK+IPA) fixture, exits 0, and emits JSON whose release id, per-artifact ids, sha256 digests, and local URLs match the store contents; objects exist under `<tmp>/objects/<sha256>`; db.json records the release, artifacts, channel pointer, and audit events; a digest/size-mismatch fixture run exits non-zero with a JSON error and leaves no promoted channel. Captured as a pinned test.
|
|
- **P3.5** `dist release promote` / `dist release rollback` over the local store
|
|
- intent: Implement standalone channel promotion and rollback against the persisted store (subplan 03; subplan 02 Slice 2 rollback acceptance): `promote` points a channel at a given release (policy gate check stub-allowed for v0), `rollback` moves the channel pointer to the previous valid release for that channel; both append audit events and re-persist db.json. Rollback never points a channel at a non-published/invalid release.
|
|
- acceptance: A scripted test: publish release A (channel beta) then release B (channel beta), assert beta→B; run `dist release rollback --app <slug> --channel beta`, assert beta→A and an audit event recorded; assert `dist release promote` of an unknown release id exits non-zero with a JSON error. `make test` green; full suite (`make test`) passes.
|
|
|
|
### P4 — `distd` over the local store (subplan 04, read-only start; added 2026-06-12)
|
|
|
|
- **P4.1** `dist server run` — read-only HTTP API + downloads
|
|
- intent: Serve the store over HTTP on 0.0.0.0:<port> (default 8787): `/healthz`, `/api/apps`, `/api/apps/<slug>` (app + releases + channels), `/download/<sha256>` (object bytes, X-Checksum-SHA256). Errors reuse the CLI's JSON error shape with matching HTTP statuses. db.json reloads per request so CLI writes are visible immediately. HTTP/1.1 is a TEMPORARY IN-REPO shim over `std.socket` (`src/server/http.sx`, liftable into the sx stdlib later): GET-only, Connection: close, no TLS (reverse proxy terminates), sequential accept loop. KNOWN sx BOUNDARY: stack arrays of 64K+ in one frame crash the sx LLVM backend (DAGCombiner segfault) — response buffers are heap slices from the per-request arena.
|
|
- acceptance: `tests/server_http.sx` publishes a fixture, starts `build/dist server run` on a test port, and asserts every route via curl (healthz, apps index, app detail, unknown-slug/route/digest JSON errors, byte-identical download) plus per-request freshness (a publish while the server runs appears on the next request). `make test` green.
|
|
- DEFERRED within subplan 04: token auth + write endpoints (Slice 2), streaming uploads (Slice 4 write half), install pages (Slice 5).
|
|
- **P4.2** HTML index at `/` — the install-page seed (DONE 2026-06-12, cf39589)
|
|
- intent: `GET /` serves a dense server-rendered console page: each app with its channels' current releases and every release's artifacts as direct `/download/<sha256>` links. Read-only off the per-request db.json reload; all dynamic text HTML-escaped. The full per-app install pages (platform detection, iOS install modes) stay in subplan 04 Slice 5; the admin UI stays subplan 06.
|
|
- acceptance: `tests/server_http.sx` asserts `/` returns 200 text/html containing the published app's slug, channel→release row, and a working artifact download link. `make test` green (14/14 after the sN→iN lang migration, 6c19f10).
|
|
- **P4.3** Token security at rest + `dist token` CLI (subplan 02 Slice 5)
|
|
- intent: The auth groundwork the P4.4 write endpoints will consume. Token domain entity (id, name, token_hash, scopes, app_slug + channel scoping, created_at, expires_at, last_used_at, revoked_at) with boundary validation; secret generation (`dist_` + 64 hex chars from 32 `arc4random_buf` bytes — libc FFI, the same thin-platform-backend boundary as publish's `time(2)`); ONLY the SHA-256 of the secret is persisted (`std.hash`); `check_token` enforces revocation, expiry, scope membership, and app/channel scope match; `mark_token_used` records last-use. CLI: `dist token create` (prints the raw secret exactly once; absent db.json starts an empty store, so CI tokens can be minted before the first publish), `dist token list` (status: active/revoked/expired; never the secret), `dist token revoke` (strict: unknown id and double-revoke are distinct errors; aborts before db.save). Every mutation appends an audit event and re-persists db.json; `tokens` joins the persisted arrays (an absent member loads as empty, so older db.json files stay readable).
|
|
- acceptance: `tests/token_ops.sx` drives `build/dist`: create → exit 0, JSON carries the secret once, db.json stores sha256(secret) and never the raw secret; list shows both tokens active; revoke flips status + audit event; unknown-id and already-revoked revokes exit 1 with distinct JSON codes and leave db.json unchanged; a publish into the same store still works. `tests/token_check.sx` unit-covers check_token (ok / revoked / expired / missing scope / app mismatch / channel mismatch / unscoped-matches-all) and that a db.json without `tokens` loads as zero tokens. `make test` green.
|
|
- DEFERRED: bearer parsing + write endpoints over HTTP (P4.4); remote `dist ci publish --server --token` (subplan 03 Slice 3, needs an HTTP client shim).
|
|
- **P4.4** Bearer auth + write endpoints on distd (subplan 04 Slice 2 + the write half of 3/4)
|
|
- intent: distd stops being read-only. `src/server/http.sx` learns the request surface writes need: header capture (case-insensitive lookup), a Content-Length-bounded body read loop (8K header cap, hard body cap with 413; 411 when POST omits the length), and the 401/403/411/413 status texts. Auth (`src/server/auth.sx`): `Authorization: Bearer <secret>` is re-hashed (`std.hash`) and resolved via `find_token_by_hash`, then gated through the domain's `check_token` — missing/malformed/unknown credentials are 401, a live token refused for revocation/expiry/scope/app/channel is 403, each with a refusal-specific dotted code; successful auth stamps `mark_token_used` and persists. Reads stay public. Write routes (all POST, all token-gated with scope `publish`): `/api/upload` (raw bytes → content-addressed store, returns the sha256 key, dedup is free), `/api/apps/<slug>/releases` (JSON body naming version/channel + artifacts by sha256 of ALREADY-UPLOADED objects → the same find/create-app → transaction → audit → save pipeline as the CLI, via a `commit_publish` core extracted from publish.sx; unknown object/app/platform aborts before any channel moves), `/api/apps/<slug>/channels/<name>/promote` + `/rollback` (delegate to the P3.5 `run_promote`/`run_rollback`, so CLI and HTTP semantics cannot drift). Server-side artifact checks reuse the P3.3 policy tables (content-type allow-list, filename-extension-vs-platform).
|
|
- acceptance: `tests/server_write.sx` publishes+mints tokens via the CLI, starts the server, and asserts over curl: 401 for no/garbage/unknown bearer; 403 with distinct codes for a read-scoped and a wrong-app-scoped token; upload returns the fixture's sha256 and the object lands in the store (re-upload dedups); a release POST referencing the uploaded digest publishes, moves the channel, and is visible on the next GET; promote/rollback move the pointer like their CLI twins; a release POST naming an unknown digest exits 4xx with a JSON error and leaves no channel moved; db.json records `last_used_at` > 0 for the used token. `make test` green.
|
|
- DEFERRED: remote `dist ci publish --server --token` (the CLI's HTTP client half, subplan 03 Slice 3); streaming/resumable uploads (bodies are read whole into the per-request arena, capped); install pages (Slice 5).
|
|
- **P4.5** Remote `dist ci publish --server --token` (subplan 03 Slice 3 — the PLAN.md CI contract, remote mode)
|
|
- intent: The CLI learns to publish against a running distd instead of a local store directory. `src/client/http_client.sx` is the TEMPORARY IN-REPO client shim over `std.socket` (mirroring the server shim): `connect(2)` via libc FFI, `--server` URLs of the form `http://<ipv4-or-localhost>:<port>` (DNS/getaddrinfo and TLS are explicit v0 boundaries — refuse `https://` loudly; the deployment story terminates TLS at a reverse proxy), one POST per call with Bearer/Content-Type/Content-Length, response read to EOF (the server speaks Connection: close) and split into status + body. `src/publish/remote.sx` drives the same manifest front half as local publish (parse + validate + on-disk existence), then per artifact: read bytes → POST `/api/upload` → trust-but-verify the returned digest against a manifest-declared sha256 when present; then POST `/api/apps/<slug>/releases` naming the uploaded digests with the manifest's filename/content-type/metadata. The server's response IS the publish outcome — it is parsed back into a `PublishOutcome` so `--json` and human rendering reuse the exact P3.4a paths, and every non-2xx is surfaced by passing the server's JSON error code/message through `report_failure` (exit 1). `dist ci publish` now takes EITHER `--local-store <dir>` OR `--server <url> --token <secret>`; supplying both or neither is a usage error.
|
|
- acceptance: `tests/remote_publish.sx` mints a token in a fresh store, starts `build/dist server run`, and drives `build/dist ci publish --manifest <m> --server http://127.0.0.1:<port> --token <secret> --json`: exit 0 with `status:"published"` JSON naming the release and per-artifact sha256; the SERVER's store contains `objects/<sha>` and a db.json release + channel pointer + `token:<name>` audit actors; re-running the same manifest exits 1 with the server's `transaction.integrity` code; a wrong secret exits 1 with `auth.unknown_token`; an `https://` or malformed `--server` is a loud usage failure; `--local-store` together with `--server` is a usage error. `make test` green.
|
|
- DEFERRED: DNS hostnames + TLS in the client shim; retries/resume on flaky links (subplan 03 Slice 3 hardening); install pages (Slice 5).
|
|
- **P4.6** Install pages with accurate iOS modes (subplan 04 Slice 5; PLAN.md "iOS Install Policy")
|
|
- intent: The human install/download surface, honest about what each platform can do. App gains the iOS policy fields: `ios_mode` (`artifact_only` default / `testflight` / `enterprise`) and `testflight_url`; absent db.json members load as defaults (the token-compat pattern). `dist app set --app <slug> --local-store <dir> [--display-name] [--ios-mode] [--testflight-url] [--ios-bundle-id]` is the admin mutator (apps are created by publish; set requires the app to exist), enforcing the policy preconditions as loud CLI errors — testflight requires a URL, enterprise requires an iOS bundle id — and appending an `app.update` audit event. Server: `GET /install/<slug>/<channel>` renders the channel's current release as platform sections — the requester's platform (User-Agent sniff: iPhone/iPad, Android, Macintosh, Windows, Linux) is ordered first and marked — with per-mode iOS actions: TestFlight link, `itms-services://?action=download-manifest&url=https://<host>/install/<slug>/<channel>/manifest.plist` (https hardcoded — itms-services requires it; the proxy terminates TLS), or an IPA download explicitly labeled as NOT installable from the page. Every artifact row shows size + sha256 (the Android acceptance). `GET /install/<slug>/<channel>/manifest.plist` serves the enterprise OTA manifest XML (software-package URL from the Host header, bundle-identifier from the app's iOS bundle id, bundle-version from the release) and 404s for any other mode or a missing iOS artifact/bundle id. The HTML index links every channel row to its install page.
|
|
- acceptance: `tests/server_install.sx` publishes the APK+IPA example, then walks the modes via `dist app set`: default page labels the IPA artifact-only (no install claim); testflight without URL exits 1 (`app.testflight_url_required`), with URL the page links TestFlight; enterprise without bundle id exits 1 (`app.bundle_id_required`), with id the page carries the itms-services link and manifest.plist serves XML naming the bundle id, version, and an https `/download/<sha>` URL; flipping back to testflight 404s the plist; an Android User-Agent gets the android section marked detected, an iPhone UA the ios section; unknown slug/channel are JSON 404s; the index links `/install/acme-app/stable`. `make test` green.
|
|
- DEFERRED: QR rendering; access-gated downloads for private apps (visibility exists in the model; pages and downloads stay public in v0); remote-mode `dist app set` (admin writes stay local-store).
|
|
|
|
### P5 — SQLite persistence (subplan 02 Slice 2; unblocks retiring db.json)
|
|
|
|
- **P5.1** Vendor SQLite + sx bindings (the foundation)
|
|
- intent: Vendor the SQLite amalgamation (3.53.2, public domain) under `vendor/sqlite/` with provenance in its README. `make build` compiles it twice from one source: `build/vendor/libsqlite3.a` (statically linked into `dist` via `-L build/vendor`) and `build/vendor/jit/libsqlite3.dylib` (dlopen'd by `sx run`, which is how the test runner executes tests) — two DIRECTORIES because the macOS linker prefers a dylib over an archive in one search dir. KNOWN sx BOUNDARY: the JIT resolves `#foreign` symbols via `dlsym(RTLD_DEFAULT)`, which searches all images already loaded into the compiler process — the OS libsqlite3 is among them and wins by load order. So the vendored build renames its API surface to `dist_sqlite3_*` (`vendor/sqlite/rename.h`, injected with `-include`), making resolution unambiguous in BOTH modes: those symbols exist only in the vendored products. `src/db/sqlite.sx` binds the renamed surface (open/exec/prepare/bind/step/column/finalize, errmsg, last_insert_rowid, changes, libversion) behind `Sqlite`/`SqliteStmt` wrappers; handles cross the FFI as `usize`, strings read from SQLite are copied into `context.allocator` before sqlite's buffers die.
|
|
- acceptance: `tests/sqlite_smoke.sx` asserts `sqlite3_libversion()` equals the VENDORED version (3.53.2 — the OS copy is 3.51.0, so a fallback fails loudly), a typed create/insert/select round trip with text/int64/null bindings, persistence across close+reopen, BEGIN/ROLLBACK atomicity, and errmsg detail on bad SQL. Verified under `sx run` (vendored dylib) AND as an AOT binary (static archive, no libsqlite3 in `otool -L`). `make test` green.
|
|
- EXTENDED 2026-06-12 (full mapping): `src/db/sqlite.sx` now binds the complete practical C API (~100 functions, one variant per duplicate family): open_v2 + flags, extended errcodes + error_offset, txn_state/autocommit, changes64, limits, full bind/column families (double/blob/zeroblob), parameter + column introspection (built with `SQLITE_ENABLE_COLUMN_METADATA`), `table_column_metadata`, statement introspection (sql/expanded_sql/readonly/status), incremental blob I/O (`SqliteBlob`), online backup (`SqliteBackup` + `sqlite_backup_run`), serialize/deserialize, and library utilities (complete, strglob/strlike, randomness, memory). NOT bound by design: callback-taking APIs (hooks/UDFs/collations need C→sx callbacks), `sqlite3_value_*` (UDF-coupled), varargs config, UTF-16, and omitted subsystems. `build/vendor/rename.h` is now GENERATED by make from the bindings' `#foreign` names — single source of truth, no drift. `tests/sqlite_api.sx` pins all wrapper families (15 cases). KNOWN sx BOUNDARY (filed): `if !e` on an error binding evaluates wrong (true on a set error) — tests use plain bools for negated error logic.
|
|
- **P5.2** Store persistence on SQLite — db.json retired (subplan 02 Slice 2)
|
|
- intent: `src/repo/db.sx` persists the whole Repo to `<store>/dist.db` through the P5.1 bindings, keeping the load-whole/save-whole call shape so publish/ops/server code stays storage-agnostic. Schema: one table per entity (apps + app_bundle_ids, releases, artifacts, channels, tokens, audit_events), enums stored as lowercase variant names, list order round-tripped via rowid (save inserts in list order into emptied tables, load reads ORDER BY rowid). Uniqueness the domain guarantees is enforced — apps.slug, channels(app_id, name), tokens.token_hash — plus lookup indexes releases(app_id) and artifacts(sha256) (NON-unique: identical bytes may ship in several releases; audit event ids repeat by design, so audit_events carries no constraints). `save` is DELETE-all + INSERT-all inside BEGIN IMMEDIATE…COMMIT, rolled back on any failure; every connection sets busy_timeout(5000), so the CLI and a running distd interleave safely on one store (db.json never guaranteed that). A store holding only a pre-SQLite db.json imports ONCE on first load, then the file is renamed db.json.imported; a store with neither starts empty (token create still works on a fresh store). Consumers gate on `db.store_exists` instead of probing for db.json; the JSON read-back stays for the import path and the entity→json writers for distd's /api responses. SX BOUNDARY FIXED IN-FLIGHT: the first product chain nesting the bindings two aliased imports deep (dist → ops → db → sqlite) exposed sx issue 0130 — `#library` declared behind ≥2 aliased imports was dropped from the AOT link line and the JIT dlopen list; fixed in sx (extractLibraries/extractFrameworks now recurse over nested namespaces) before this step landed.
|
|
- acceptance: `tests/repo_roundtrip.sx` (field-for-field SQLite round-trip + an independent SQL read of dist.db; no db.json written), `tests/db_import.sx` (one-time import: every entity survives, db.json → db.json.imported, writes work after, no re-import, a store with no database refuses with store.load), and the migrated pinned suite — token_ops / release_ops / publish_happy / publish_persist / publish_fail / remote_publish / server_http / server_write assert store state by QUERYING dist.db through the SQLite bindings (no test reads db.json). `make test` green (22/22).
|
|
|