Baseline: distribution workspace before observability redesign
This commit is contained in:
agra
304
PLAN.md
Normal file
304
PLAN.md
Normal file
@@ -0,0 +1,304 @@
|
||||
# Distribution Platform in sx
|
||||
|
||||
## Goal
|
||||
|
||||
Build an App Store-like distribution platform in `sx`, but not limited to iOS
|
||||
apps. It should distribute mobile, desktop, and server/client artifacts through
|
||||
a CI-friendly release workflow.
|
||||
|
||||
Supported targets for the first product direction:
|
||||
|
||||
- iOS: IPA metadata, TestFlight links, Enterprise/MDM manifests, artifact-only
|
||||
downloads
|
||||
- Android: APK first, AAB later
|
||||
- macOS: app archives, dmg/pkg metadata, signing and notarization status
|
||||
- Linux: tarballs first, then deb/rpm/AppImage metadata
|
||||
- Windows: zip/installer first, then MSIX/signing metadata
|
||||
|
||||
The main commands are:
|
||||
|
||||
- `distd`: the HTTP API, install page, and artifact server
|
||||
- `dist`: the CI/admin CLI
|
||||
|
||||
The release workflow must be easy to integrate into CI. A CI job should be able
|
||||
to upload artifacts, create a release, attach platform variants, promote a
|
||||
channel, and receive machine-readable JSON output.
|
||||
|
||||
Product code should live in `sx`. C or system APIs are acceptable only as thin
|
||||
platform backends for capabilities that `sx` cannot express yet.
|
||||
|
||||
## Product Shape
|
||||
|
||||
This is an operational distribution tool first, not a public marketplace. The
|
||||
initial product should feel like a release console: quiet, dense, fast to scan,
|
||||
and optimized for repeated internal/private release work.
|
||||
|
||||
Core principles:
|
||||
|
||||
- CI is the primary writer.
|
||||
- Humans inspect, promote, revoke, download, and audit.
|
||||
- Releases are immutable.
|
||||
- Channels are mutable pointers to releases.
|
||||
- Artifacts are content-addressed and never silently replaced.
|
||||
- Every upload, publish, promotion, token change, and deletion writes an audit
|
||||
event.
|
||||
- The first version should be understandable from the filesystem and SQLite
|
||||
database during development.
|
||||
|
||||
## Users And Actors
|
||||
|
||||
- CI runner: publishes releases with `dist ci publish`.
|
||||
- Release manager: promotes channels, checks validation, and reads audit logs.
|
||||
- Developer/tester: downloads the current artifact for a platform/channel.
|
||||
- Admin: creates apps, manages tokens, and configures identity constraints.
|
||||
- `distd`: server/API/artifact store.
|
||||
- `dist`: CLI for CI and local admin work.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
App:
|
||||
|
||||
- Product being distributed.
|
||||
- Owns identity rules, allowed platforms, releases, channels, and tokens.
|
||||
|
||||
Release:
|
||||
|
||||
- Immutable version/build record for one app.
|
||||
- Contains one or more platform artifacts.
|
||||
|
||||
Artifact:
|
||||
|
||||
- Uploaded file for a platform.
|
||||
- Stored by digest and linked to release metadata.
|
||||
|
||||
Channel:
|
||||
|
||||
- Mutable pointer such as stable, beta, internal, nightly.
|
||||
- Promotion changes the pointer, not the release.
|
||||
|
||||
Token:
|
||||
|
||||
- Scoped credential for CI and automation.
|
||||
- Stored hashed, expires or can be revoked.
|
||||
|
||||
Audit event:
|
||||
|
||||
- Immutable record of important actions.
|
||||
|
||||
## Main Workflows
|
||||
|
||||
CI publish:
|
||||
|
||||
1. CI builds artifacts.
|
||||
2. CI writes or updates `dist.json`.
|
||||
3. CI runs `dist ci publish --server "$DIST_SERVER" --token "$DIST_TOKEN"
|
||||
--manifest dist.json --json`.
|
||||
4. `dist` validates the manifest and streams artifact uploads.
|
||||
5. `distd` validates metadata and stores artifacts by digest.
|
||||
6. The release is published.
|
||||
7. A requested channel is promoted if policy allows it.
|
||||
8. CI receives JSON with release ids, artifact ids, digests, and URLs.
|
||||
|
||||
Human release management:
|
||||
|
||||
1. Release manager opens the admin UI.
|
||||
2. They inspect release status, artifact validation, and audit history.
|
||||
3. They promote, rollback, revoke tokens, or download artifacts.
|
||||
|
||||
Install/download:
|
||||
|
||||
1. User opens an app/channel install page.
|
||||
2. Platform detection chooses the most relevant artifact or install path.
|
||||
3. The page shows accurate platform-specific actions and metadata.
|
||||
4. Access policy determines whether the user can download or install.
|
||||
|
||||
## iOS Install Policy
|
||||
|
||||
iOS needs special handling. The platform must not imply that a normal iPhone can
|
||||
install an arbitrary IPA from a web page.
|
||||
|
||||
Supported iOS modes:
|
||||
|
||||
- TestFlight: store a TestFlight link and open Apple's install flow.
|
||||
- Enterprise/MDM: serve a valid HTTPS manifest plist for enrolled devices.
|
||||
- Artifact only: allow authenticated IPA download, without presenting it as a
|
||||
normal mobile install action.
|
||||
|
||||
This distinction must be visible in both the API model and admin/install UI.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The intended stack is:
|
||||
|
||||
```text
|
||||
sx language and std primitives
|
||||
-> infra libraries: http, json, hashing, storage, db, auth, archive
|
||||
-> distribution domain: app, artifact, release, channel, token, audit
|
||||
-> interfaces: CLI, HTTP API, install pages, admin UI
|
||||
-> deployment: Docker image, NAS-friendly config, persistent data volume
|
||||
```
|
||||
|
||||
Storage direction:
|
||||
|
||||
- SQLite first.
|
||||
- Local filesystem artifact storage first.
|
||||
- Object storage adapter later.
|
||||
|
||||
Deployment direction:
|
||||
|
||||
- Docker/OCI image.
|
||||
- One persistent `/data` volume for database, artifacts, uploads, config, and
|
||||
logs.
|
||||
- One HTTP port.
|
||||
- `/healthz` endpoint.
|
||||
- Runs behind a reverse proxy for TLS.
|
||||
- Runs as a non-root user.
|
||||
- UGREEN NAS deployment through Docker/Container Manager is a first-version
|
||||
requirement.
|
||||
|
||||
## sx Foundation Work
|
||||
|
||||
Before the product can be implemented well, `sx` needs a stronger language and
|
||||
standard library foundation.
|
||||
|
||||
Language/module needs:
|
||||
|
||||
- `pub` exports so std modules do not leak private helpers.
|
||||
- Alias imports and curated namespace barrels.
|
||||
- Namespace member re-export syntax such as `pub print :: core.print`.
|
||||
- Error handling that follows the real sx model. `!` is an error channel, not a
|
||||
generic result wrapper.
|
||||
|
||||
Standard library needs, at overview level:
|
||||
|
||||
- Collections: extended `List`, `HashMap`
|
||||
- Strings: validated UTF-8 `String`, `StringBuilder`, explicit Unicode model
|
||||
- Bytes and paths
|
||||
- Filesystem and process APIs
|
||||
- Time, random, hashing, and encoding
|
||||
- JSON, URL, MIME, config, CLI, and logging
|
||||
- HTTP server/client and TLS boundary
|
||||
- SQLite
|
||||
- Archive inspection
|
||||
- Testing helpers
|
||||
|
||||
Unicode must be specified precisely:
|
||||
|
||||
- `String` is validated UTF-8 bytes.
|
||||
- Byte length, scalar values, grapheme clusters, and display width are distinct.
|
||||
- APIs must clearly say which unit they operate on.
|
||||
- Invalid UTF-8 must not silently become a `String`.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
Phase 0 - sx language/module prerequisites:
|
||||
|
||||
- Add `pub` support, alias imports, and namespace member re-exports.
|
||||
|
||||
Phase 1 - standard library foundation:
|
||||
|
||||
- Build the std primitives needed for CLI, HTTP, storage, validation, and tests.
|
||||
|
||||
Phase 2 - product domain:
|
||||
|
||||
- Define apps, releases, artifacts, channels, tokens, policies, and audit
|
||||
events.
|
||||
|
||||
Phase 3 - storage:
|
||||
|
||||
- Add SQLite persistence and filesystem artifact storage.
|
||||
|
||||
Phase 4 - CLI first:
|
||||
|
||||
- Make `dist ci publish --manifest dist.json --json` work before the admin UI.
|
||||
|
||||
Phase 5 - HTTP API:
|
||||
|
||||
- Expose app, release, upload, download, channel, token, and audit endpoints.
|
||||
|
||||
Phase 6 - artifact validation:
|
||||
|
||||
- Validate APK and IPA first, then desktop artifact metadata.
|
||||
|
||||
Phase 7 - channels and install pages:
|
||||
|
||||
- Add promotion, rollback, download URLs, and platform-specific install pages.
|
||||
|
||||
Phase 8 - admin UI:
|
||||
|
||||
- Build a dense operational UI for apps, releases, artifacts, tokens, settings,
|
||||
install pages, and audit logs.
|
||||
|
||||
Phase 9 - packaging:
|
||||
|
||||
- Package `distd` and the admin UI into a Docker image suitable for UGREEN NAS.
|
||||
|
||||
## CI Contract
|
||||
|
||||
The primary CI command should be:
|
||||
|
||||
```sh
|
||||
dist ci publish \
|
||||
--server "$DIST_SERVER" \
|
||||
--token "$DIST_TOKEN" \
|
||||
--manifest dist.json \
|
||||
--json
|
||||
```
|
||||
|
||||
The command should:
|
||||
|
||||
1. Validate the manifest.
|
||||
2. Create or find the app.
|
||||
3. Create a draft release.
|
||||
4. Upload all artifacts with streaming SHA-256.
|
||||
5. Validate platform metadata.
|
||||
6. Publish the release.
|
||||
7. Promote the requested channel if specified.
|
||||
8. Print JSON containing release id, artifact ids, digests, and download URLs.
|
||||
|
||||
## First Milestone
|
||||
|
||||
Milestone 1 is complete when:
|
||||
|
||||
- Required `sx` language/std primitives exist or have explicit temporary
|
||||
boundaries.
|
||||
- `distd` can run locally.
|
||||
- `dist ci publish` can publish a release with at least APK and IPA artifacts.
|
||||
- Artifacts are stored by digest.
|
||||
- SQLite stores apps, releases, artifacts, channels, tokens, and audit events.
|
||||
- A channel can be promoted and rolled back.
|
||||
- Install/download pages accurately handle iOS, Android, and desktop artifacts.
|
||||
- The admin UI can inspect apps, releases, validations, tokens, and audit logs.
|
||||
- A Docker image can run on a UGREEN NAS with a persistent data volume.
|
||||
|
||||
## Non-goals For Version 1
|
||||
|
||||
- Public marketplace payments.
|
||||
- Reviews and ratings.
|
||||
- Organization billing.
|
||||
- Sophisticated staged rollout targeting.
|
||||
- CDN integration.
|
||||
- Full signing/notarization automation.
|
||||
- Object storage.
|
||||
- Postgres.
|
||||
|
||||
## Detailed Execution
|
||||
|
||||
`PLAN.md` is the overview. Detailed implementation breakdowns live in
|
||||
`.agents/subplans/`.
|
||||
|
||||
Before starting or resuming work, read:
|
||||
|
||||
- `.agents/ORCHESTRATION.md`
|
||||
- `.agents/CHECKPOINT.md`
|
||||
- `.agents/checkpoint.json`
|
||||
- the active `.agents/subplans/*.md` file
|
||||
|
||||
The workflow is sequential and branch-based:
|
||||
|
||||
- Codex manages orchestration and validation.
|
||||
- Snarky owns product briefs and final product acceptance.
|
||||
- Opus owns layout/design decisions.
|
||||
- Opus is the only role that writes code during Opus implementation phases.
|
||||
- Implementation uses git branches, not worktrees.
|
||||
- Checkpoints are updated after every completed slice and before stopping work.
|
||||
Reference in New Issue
Block a user