Files
distribution/PLAN.md

305 lines
9.0 KiB
Markdown

# Distribution Platform in sx
## Goal
Build an App Store-like distribution platform in `sx`, but not limited to iOS
apps. It should distribute mobile, desktop, and server/client artifacts through
a CI-friendly release workflow.
Supported targets for the first product direction:
- iOS: IPA metadata, TestFlight links, Enterprise/MDM manifests, artifact-only
downloads
- Android: APK first, AAB later
- macOS: app archives, dmg/pkg metadata, signing and notarization status
- Linux: tarballs first, then deb/rpm/AppImage metadata
- Windows: zip/installer first, then MSIX/signing metadata
The main commands are:
- `distd`: the HTTP API, install page, and artifact server
- `dist`: the CI/admin CLI
The release workflow must be easy to integrate into CI. A CI job should be able
to upload artifacts, create a release, attach platform variants, promote a
channel, and receive machine-readable JSON output.
Product code should live in `sx`. C or system APIs are acceptable only as thin
platform backends for capabilities that `sx` cannot express yet.
## Product Shape
This is an operational distribution tool first, not a public marketplace. The
initial product should feel like a release console: quiet, dense, fast to scan,
and optimized for repeated internal/private release work.
Core principles:
- CI is the primary writer.
- Humans inspect, promote, revoke, download, and audit.
- Releases are immutable.
- Channels are mutable pointers to releases.
- Artifacts are content-addressed and never silently replaced.
- Every upload, publish, promotion, token change, and deletion writes an audit
event.
- The first version should be understandable from the filesystem and SQLite
database during development.
## Users And Actors
- CI runner: publishes releases with `dist ci publish`.
- Release manager: promotes channels, checks validation, and reads audit logs.
- Developer/tester: downloads the current artifact for a platform/channel.
- Admin: creates apps, manages tokens, and configures identity constraints.
- `distd`: server/API/artifact store.
- `dist`: CLI for CI and local admin work.
## Core Concepts
App:
- Product being distributed.
- Owns identity rules, allowed platforms, releases, channels, and tokens.
Release:
- Immutable version/build record for one app.
- Contains one or more platform artifacts.
Artifact:
- Uploaded file for a platform.
- Stored by digest and linked to release metadata.
Channel:
- Mutable pointer such as stable, beta, internal, nightly.
- Promotion changes the pointer, not the release.
Token:
- Scoped credential for CI and automation.
- Stored hashed, expires or can be revoked.
Audit event:
- Immutable record of important actions.
## Main Workflows
CI publish:
1. CI builds artifacts.
2. CI writes or updates `dist.json`.
3. CI runs `dist ci publish --server "$DIST_SERVER" --token "$DIST_TOKEN"
--manifest dist.json --json`.
4. `dist` validates the manifest and streams artifact uploads.
5. `distd` validates metadata and stores artifacts by digest.
6. The release is published.
7. A requested channel is promoted if policy allows it.
8. CI receives JSON with release ids, artifact ids, digests, and URLs.
Human release management:
1. Release manager opens the admin UI.
2. They inspect release status, artifact validation, and audit history.
3. They promote, rollback, revoke tokens, or download artifacts.
Install/download:
1. User opens an app/channel install page.
2. Platform detection chooses the most relevant artifact or install path.
3. The page shows accurate platform-specific actions and metadata.
4. Access policy determines whether the user can download or install.
## iOS Install Policy
iOS needs special handling. The platform must not imply that a normal iPhone can
install an arbitrary IPA from a web page.
Supported iOS modes:
- TestFlight: store a TestFlight link and open Apple's install flow.
- Enterprise/MDM: serve a valid HTTPS manifest plist for enrolled devices.
- Artifact only: allow authenticated IPA download, without presenting it as a
normal mobile install action.
This distinction must be visible in both the API model and admin/install UI.
## Architecture Overview
The intended stack is:
```text
sx language and std primitives
-> infra libraries: http, json, hashing, storage, db, auth, archive
-> distribution domain: app, artifact, release, channel, token, audit
-> interfaces: CLI, HTTP API, install pages, admin UI
-> deployment: Docker image, NAS-friendly config, persistent data volume
```
Storage direction:
- SQLite first.
- Local filesystem artifact storage first.
- Object storage adapter later.
Deployment direction:
- Docker/OCI image.
- One persistent `/data` volume for database, artifacts, uploads, config, and
logs.
- One HTTP port.
- `/healthz` endpoint.
- Runs behind a reverse proxy for TLS.
- Runs as a non-root user.
- UGREEN NAS deployment through Docker/Container Manager is a first-version
requirement.
## sx Foundation Work
Before the product can be implemented well, `sx` needs a stronger language and
standard library foundation.
Language/module needs:
- `pub` exports so std modules do not leak private helpers.
- Alias imports and curated namespace barrels.
- Namespace member re-export syntax such as `pub print :: core.print`.
- Error handling that follows the real sx model. `!` is an error channel, not a
generic result wrapper.
Standard library needs, at overview level:
- Collections: extended `List`, `HashMap`
- Strings: validated UTF-8 `String`, `StringBuilder`, explicit Unicode model
- Bytes and paths
- Filesystem and process APIs
- Time, random, hashing, and encoding
- JSON, URL, MIME, config, CLI, and logging
- HTTP server/client and TLS boundary
- SQLite
- Archive inspection
- Testing helpers
Unicode must be specified precisely:
- `String` is validated UTF-8 bytes.
- Byte length, scalar values, grapheme clusters, and display width are distinct.
- APIs must clearly say which unit they operate on.
- Invalid UTF-8 must not silently become a `String`.
## Implementation Phases
Phase 0 - sx language/module prerequisites:
- Add `pub` support, alias imports, and namespace member re-exports.
Phase 1 - standard library foundation:
- Build the std primitives needed for CLI, HTTP, storage, validation, and tests.
Phase 2 - product domain:
- Define apps, releases, artifacts, channels, tokens, policies, and audit
events.
Phase 3 - storage:
- Add SQLite persistence and filesystem artifact storage.
Phase 4 - CLI first:
- Make `dist ci publish --manifest dist.json --json` work before the admin UI.
Phase 5 - HTTP API:
- Expose app, release, upload, download, channel, token, and audit endpoints.
Phase 6 - artifact validation:
- Validate APK and IPA first, then desktop artifact metadata.
Phase 7 - channels and install pages:
- Add promotion, rollback, download URLs, and platform-specific install pages.
Phase 8 - admin UI:
- Build a dense operational UI for apps, releases, artifacts, tokens, settings,
install pages, and audit logs.
Phase 9 - packaging:
- Package `distd` and the admin UI into a Docker image suitable for UGREEN NAS.
## CI Contract
The primary CI command should be:
```sh
dist ci publish \
--server "$DIST_SERVER" \
--token "$DIST_TOKEN" \
--manifest dist.json \
--json
```
The command should:
1. Validate the manifest.
2. Create or find the app.
3. Create a draft release.
4. Upload all artifacts with streaming SHA-256.
5. Validate platform metadata.
6. Publish the release.
7. Promote the requested channel if specified.
8. Print JSON containing release id, artifact ids, digests, and download URLs.
## First Milestone
Milestone 1 is complete when:
- Required `sx` language/std primitives exist or have explicit temporary
boundaries.
- `distd` can run locally.
- `dist ci publish` can publish a release with at least APK and IPA artifacts.
- Artifacts are stored by digest.
- SQLite stores apps, releases, artifacts, channels, tokens, and audit events.
- A channel can be promoted and rolled back.
- Install/download pages accurately handle iOS, Android, and desktop artifacts.
- The admin UI can inspect apps, releases, validations, tokens, and audit logs.
- A Docker image can run on a UGREEN NAS with a persistent data volume.
## Non-goals For Version 1
- Public marketplace payments.
- Reviews and ratings.
- Organization billing.
- Sophisticated staged rollout targeting.
- CDN integration.
- Full signing/notarization automation.
- Object storage.
- Postgres.
## Detailed Execution
`PLAN.md` is the overview. Detailed implementation breakdowns live in
`.agents/subplans/`.
Before starting or resuming work, read:
- `.agents/ORCHESTRATION.md`
- `.agents/CHECKPOINT.md`
- `.agents/checkpoint.json`
- the active `.agents/subplans/*.md` file
The workflow is sequential and branch-based:
- Codex manages orchestration and validation.
- Snarky owns product briefs and final product acceptance.
- Opus owns layout/design decisions.
- Opus is the only role that writes code during Opus implementation phases.
- Implementation uses git branches, not worktrees.
- Checkpoints are updated after every completed slice and before stopping work.