Baseline: distribution workspace before observability redesign

This commit is contained in:
agra
2026-06-02 16:08:17 +03:00
commit 055c8ced15
38 changed files with 6001 additions and 0 deletions

View File

@@ -0,0 +1,153 @@
# Subplan 01 - sx Language And Standard Library
## Goal
Build the language and std primitives required before the distribution platform
can be implemented cleanly in sx.
## Work Location
- Primary repo: `/Users/agra/projects/sx`
- Planning repo: `/Users/agra/projects/distribution`
## Prerequisites
- Read `/Users/agra/projects/sx/readme.md`
- Read `/Users/agra/projects/sx/specs.md`
- Read `/Users/agra/projects/sx/docs/error-handling.md`
- Re-read `PLAN.md` sections:
- `Phase 0 - sx Language and Module Prerequisites`
- `Standard Library API Surface`
- `Detailed Std Struct And Method Sketches`
## Slice 1 - Public Exports And Namespace Aliases
Deliver:
- Add `pub` declarations for public module members.
- Add public namespace member re-export aliases:
- `pub print :: core.print`
- `pub format :: core.format`
- Support alias imports without forcing every caller to know the source module.
- Add parser, AST, resolver, and compiler tests.
- Add examples for module barrels such as `std.sx`.
Acceptance:
- Private module members are not importable outside their namespace.
- Re-exported members keep correct identity for diagnostics and docs.
- Cycles and duplicate public names produce clear errors.
## Slice 2 - Error Handling Alignment
Deliver:
- Audit all proposed std signatures against sx error handling.
- Replace any incorrect `!T` assumptions with the actual sx shape.
- Add tests that show success and error return forms.
Acceptance:
- New std APIs follow the actual sx error model.
- Distribution platform examples compile with the same error style.
## Slice 3 - Unicode And String Model
Deliver:
- Define `String` as validated UTF-8 bytes.
- Define explicit terms:
- byte length
- Unicode scalar value
- grapheme cluster
- code point index is not a random-access string index
- Add `std.unicode` primitives for validation, scalar iteration,
normalization policy, case mapping policy, and ASCII fast paths.
- Add `StringBuilder` for efficient construction.
Acceptance:
- Invalid UTF-8 cannot silently become `String`.
- APIs that operate on bytes, scalars, and display width are named distinctly.
- Platform identifiers and paths do not accidentally use display width logic.
## Slice 4 - Collections
Deliver:
- Extend `List` with reserve, resize, insert, remove, pop, clear, sort, find,
contains, map/filter style helpers where idiomatic.
- Add `HashMap` with deterministic iteration option or clear documentation that
order is unspecified.
- Add hash/equality hooks for custom key types.
- Add tests for resize, collisions, deletion, and iteration.
Acceptance:
- Manifest parsing, route tables, and in-memory product stores can be written
without ad hoc arrays.
## Slice 5 - Bytes, Paths, FS, And Process
Deliver:
- `std.bytes` for slices, builders, compare, copy, starts_with, ends_with.
- `std.path` with platform-aware join, normalize, basename, dirname, extension.
- `std.fs` with open/read/write/stat/mkdir/rename/remove/temp_dir/walk.
- `std.process` with args, env, cwd, exit, spawn, wait, stdout/stderr capture.
Acceptance:
- A CLI can read a manifest, inspect files, spawn validation helpers, and write
artifacts safely.
## Slice 6 - Time, Random, Hashing, Encoding
Deliver:
- `std.time` timestamps, durations, monotonic clock, RFC3339 parse/format.
- `std.random` secure bytes and deterministic PRNG for tests.
- `std.hash` SHA-256 streaming.
- `std.encoding` hex, base64url, percent encoding.
Acceptance:
- Release ids, token hashes, artifact digests, and audit timestamps can be
implemented without platform-specific shims.
## Slice 7 - JSON, Config, CLI, And Logging
Deliver:
- `std.json` parser/writer with useful typed access.
- `std.config` for env/file/CLI config layering.
- `std.cli` parser with subcommands, flags, help, exit codes.
- `std.log` structured events and levels.
Acceptance:
- `dist ci publish --manifest dist.json --json` can parse args and emit stable
JSON output.
## Slice 8 - HTTP, TLS, SQLite, Archive, Testing
Deliver:
- `std.http` server/client basics.
- `std.tls` client/server wiring or explicit temporary boundary.
- `std.db.sqlite` minimal prepared statements and transactions.
- `std.archive` zip/tar/gzip reading needed for artifacts.
- `std.testing` assertions, fixtures, temp directories, golden files.
Acceptance:
- `distd` can expose endpoints, persist releases, inspect artifacts, and run
repeatable tests.
## Checkpoint Notes
After each slice, update:
- `.agents/CHECKPOINT.md`
- `.agents/checkpoint.json`
- Any changed API signatures in `PLAN.md`

View File

@@ -0,0 +1,151 @@
# Subplan 02 - Product Domain And Storage
## Goal
Define the distribution platform's core data model and persistence layer once
the required std primitives exist.
## Core Structs
App:
- id
- slug
- display_name
- bundle identifiers by platform
- owner
- visibility
- created_at
- updated_at
Platform:
- ios
- android_apk
- macos
- linux
- windows
Artifact:
- id
- app_id
- release_id
- platform
- filename
- content_type
- size_bytes
- sha256
- storage_key
- metadata
- validation_status
Release:
- id
- app_id
- version
- build
- channel
- notes
- created_by
- created_at
- published_at
Channel:
- app_id
- name
- current_release_id
- policy
- rollout_percent
Token:
- id
- name
- token_hash
- scopes
- created_at
- expires_at
- last_used_at
- revoked_at
AuditEvent:
- id
- actor
- action
- target_type
- target_id
- metadata
- created_at
## Slice 1 - In-Memory Model
Deliver:
- In-memory repository interfaces.
- Basic create/list/get/update operations.
- Domain validation for slugs, versions, channels, and platform ids.
Acceptance:
- CLI and HTTP tests can run without SQLite.
- Invalid app/release/artifact state is rejected at the domain boundary.
## Slice 2 - SQLite Schema
Deliver:
- Schema migrations.
- Tables for apps, releases, artifacts, channels, tokens, audit events.
- Indexes for slug lookup, app releases, artifact digest lookup, token hash.
- Transaction wrapper for publish flows.
Acceptance:
- A release and all artifacts can be created atomically.
- Channel promotion can be rolled back if a validation gate fails.
## Slice 3 - Artifact Storage
Deliver:
- Content-addressed storage by SHA-256.
- Staging directory for uploads.
- Atomic move from staging to final storage.
- Metadata sidecar or DB rows for content type, size, and platform metadata.
Acceptance:
- Duplicate artifact bytes are not stored twice unless policy requires it.
- Interrupted uploads do not create published artifacts.
## Slice 4 - Retention And Cleanup
Deliver:
- Retention policy per app/channel.
- Cleanup of unreferenced staged files.
- Audit events for deletion.
Acceptance:
- Stable releases can be retained longer than beta/internal builds.
- Cleanup never deletes the release currently pointed at by a channel.
## Slice 5 - Token Security
Deliver:
- Token generation.
- Token hashing at rest.
- Scope checks.
- Expiration and revocation.
- Last-used audit updates.
Acceptance:
- Raw tokens are shown only once.
- CI publish tokens can be scoped to one app and one channel.

View File

@@ -0,0 +1,125 @@
# Subplan 03 - CLI And CI Integration
## Goal
Make `dist` useful from CI before the admin UI exists.
## Commands
`dist server`:
- starts `distd`
- accepts config file and env overrides
`dist doctor`:
- checks config, database, storage directory, HTTPS base URL, and platform tools
`dist app create`:
- creates an app
- accepts slug, display name, owner, platform ids
`dist token create`:
- creates scoped CI tokens
- prints raw token once
`dist ci publish`:
- validates manifest
- creates/fetches app
- creates draft release
- uploads artifacts
- validates artifacts
- publishes release
- optionally promotes a channel
- prints JSON output
`dist release promote`:
- promotes a release to a channel
- checks policy gates
`dist release rollback`:
- moves a channel pointer to the previous valid release
## Manifest Shape
Required fields:
- app slug
- version
- channel
- artifacts list
Artifact fields:
- platform
- path
- filename override
- content type override
- metadata
## Slice 1 - Parser And Help
Deliver:
- Top-level parser.
- Subcommand parser.
- Help text.
- Exit code contract.
Acceptance:
- Unknown commands produce readable errors.
- `--json` never emits human-only text on stdout.
## Slice 2 - Local Publish
Deliver:
- `dist ci publish --manifest dist.json --local-store .dist`
- In-memory or file-backed local mode for early testing.
Acceptance:
- CI flow can be dogfooded without a running HTTP server.
## Slice 3 - Remote Publish
Deliver:
- HTTP client integration.
- Token auth.
- Streaming uploads with SHA-256.
- Retry policy for safe requests.
Acceptance:
- Upload output includes release id, artifact ids, digests, and URLs.
## Slice 4 - CI Templates
Deliver:
- GitHub Actions example.
- GitLab CI example.
- Generic shell example.
Acceptance:
- Users can copy the command and only set server, token, and manifest path.
## Slice 5 - Observability
Deliver:
- Structured logs.
- `--verbose`.
- Machine-readable errors under `--json`.
Acceptance:
- CI logs explain what failed without leaking tokens.

View File

@@ -0,0 +1,104 @@
# Subplan 04 - HTTP API And Install Experience
## Goal
Expose release management and install/download flows through `distd`.
## API Slices
Slice 1 - Server Skeleton:
- config loading
- routing
- JSON request/response helpers
- error response shape
- health endpoint
Slice 2 - Auth:
- bearer token parsing
- scope checks
- audit actor resolution
Slice 3 - Apps And Releases:
- create/list/get apps
- create/list/get releases
- publish release
- promote/rollback channel
Slice 4 - Uploads And Downloads:
- streaming upload endpoint
- digest validation
- resumable upload optional for later
- download endpoint with content length and SHA-256 headers
Slice 5 - Public Install Pages:
- app page by slug/channel
- platform detection
- QR/deep link support
- authenticated and public modes
## iOS Install Rules
Normal iOS devices cannot install arbitrary IPA files from a web page.
Supported modes:
- TestFlight link: open Apple's TestFlight flow.
- Enterprise/MDM: serve signed HTTPS manifest plist for enrolled devices.
- Artifact only: allow authenticated IPA download without claiming mobile
install support.
Acceptance:
- The UI labels iOS install mode accurately.
- Enterprise install requires HTTPS and a valid manifest.
- Artifact-only mode does not display a misleading "Install on iPhone" action.
## Android APK Install Rules
Supported modes:
- Direct APK download.
- Optional install instructions shown only when relevant.
- SHA-256 visible in download metadata.
Acceptance:
- APK download is access-controlled according to app/channel policy.
## Desktop Install Rules
macOS:
- notarization status displayed when available
- download zip/dmg/pkg
Linux:
- tar/appimage/deb/rpm metadata
Windows:
- installer zip/exe/msi metadata
- signature status displayed when available
## API Contract
All JSON endpoints should return:
- stable status code
- error code
- message
- request id
- optional details
Downloads should include:
- `Content-Length`
- `Content-Type`
- `X-Artifact-SHA256`
- cache policy by channel

View File

@@ -0,0 +1,105 @@
# Subplan 05 - Artifact Validation
## Goal
Validate uploaded platform artifacts before publication and channel promotion.
## Common Validation
All artifacts:
- file exists
- size matches upload metadata
- SHA-256 matches
- content type is allowed
- extension matches platform policy
- archive is readable when applicable
## iOS IPA
Validate:
- zip structure
- `.app` bundle exists
- Info.plist can be read
- bundle id matches app policy
- version/build metadata matches release
- provisioning profile mode is classified
- install mode is classified as TestFlight, Enterprise/MDM, or Artifact only
Acceptance:
- The system never implies browser IPA install unless enterprise/MDM manifest
requirements are met.
## Android APK
Validate:
- APK structure
- manifest package id
- version code/name
- min sdk
- signature presence
- basic metadata extraction
Acceptance:
- APK validator can reject package id mismatch before publication.
## macOS
Validate:
- archive opens
- app bundle or installer package exists
- Info.plist metadata when relevant
- code signing status if tool support exists
- notarization state when provided or checked
Acceptance:
- Notarization can be pending for beta but must satisfy stable policy if
configured.
## Linux
Validate:
- archive/package opens
- architecture metadata when available
- executable presence
- optional package metadata for deb/rpm/appimage
Acceptance:
- Platform and architecture metadata can drive download labels.
## Windows
Validate:
- exe/msi/zip exists
- authenticode status if supported
- malware scan placeholder status
- architecture metadata when available
Acceptance:
- Stable promotion can require completed scan policy.
## Validation Pipeline
Statuses:
- pending
- passed
- failed
- warning
- skipped
Policy:
- beta/internal may allow warnings.
- stable requires all configured required checks.
- every validation result is auditable.

View File

@@ -0,0 +1,100 @@
# Subplan 06 - Admin UI
## Goal
Build a dense, operational admin UI for managing apps, releases, artifacts,
channels, tokens, install pages, and audit logs.
## Authority
- Snarky defines product requirements and acceptance criteria.
- Opus owns layout, visual hierarchy, and interaction design.
- Opus is the only role that writes UI code during implementation.
## Screens
Apps:
- app list
- platform coverage
- channel status
- latest release summary
App detail:
- metadata
- releases table
- channels
- install page visibility
Release detail:
- artifact list
- validation status
- timeline
- promote/rollback actions
Install preview:
- public install page preview
- platform detection states
- iOS mode selector
- QR/deep link preview
Tokens:
- token list
- create token
- scopes
- revoke
Audit:
- filter by actor, action, app, release
- event details
Settings:
- server URL
- storage
- policy gates
- NAS deployment values
## UX Requirements
- First screen is the working product, not a landing page.
- Operational SaaS density: quiet, scan-friendly, no oversized marketing hero.
- Layout works on desktop and mobile without text overlap.
- Tables and panels should have stable dimensions.
- Repeated items can be cards; avoid nested cards.
- The UI must distinguish iOS TestFlight, Enterprise/MDM, and Artifact-only
modes clearly.
## Mock Redesign Slice
Current static mock files:
- `index.html`
- `styles.css`
- `app.js`
The current layout was rejected by the user.
Next redesign branch:
- `opus/redesign-distribution-mock`
Allowed paths for Opus:
- `index.html`
- `styles.css`
- `app.js`
- `assets/**`
Acceptance:
- App console is usable in the first viewport.
- Navigation is not visually heavy.
- Release workflow and install workflow are immediately legible.
- Mobile layout prioritizes content before secondary chrome.
- No misleading iOS install claim.

View File

@@ -0,0 +1,84 @@
# Subplan 07 - Packaging, Docker, And UGREEN NAS
## Goal
Package the distribution platform so it can run on a UGREEN NAS through Docker.
## Runtime Shape
Container:
- `distd` server binary
- static admin UI assets
- config file support
- healthcheck
Volumes:
- database
- artifact storage
- logs
- optional TLS certs
Environment:
- `DIST_BASE_URL`
- `DIST_DATA_DIR`
- `DIST_DB_PATH`
- `DIST_STORAGE_DIR`
- `DIST_ADMIN_BIND`
- `DIST_PUBLIC_BIND`
- `DIST_LOG_LEVEL`
## Slice 1 - Local Container
Deliver:
- Dockerfile
- docker compose file
- healthcheck endpoint
- local volume layout
Acceptance:
- `docker compose up` starts `distd`.
- The admin UI loads.
- Data survives container restart.
## Slice 2 - NAS Deployment
Deliver:
- UGREEN NAS deployment notes.
- Recommended volume paths.
- Reverse proxy/TLS guidance.
- Backup/restore guidance.
Acceptance:
- A user can deploy with Docker UI or compose.
- Required external ports and volumes are clear.
## Slice 3 - Updates And Rollback
Deliver:
- image tag policy
- migration policy
- backup before migration
- rollback instructions
Acceptance:
- Updating the platform does not risk artifact loss without warning.
## Slice 4 - Dogfood
Deliver:
- Publish the distribution platform's own release through `dist`.
- Document the self-hosted release flow.
Acceptance:
- The platform can distribute its own updates.

View File

@@ -0,0 +1,200 @@
# Subplan 08 - Orchestration, Checkpoints, And QA
## Goal
Keep agent work resumable, auditable, and constrained.
## Required Files
- `.agents/ORCHESTRATION.md`
- `.agents/CHECKPOINT.md`
- `.agents/checkpoint.json`
- `.agents/subplans/README.md`
- `.agents/runs/<run-id>/...`
## Run Creation
For each substantial task:
1. Create `.agents/runs/<run-id>/`.
2. Copy relevant acceptance criteria into `brief.md`.
3. Record the active branch.
4. Record allowed write paths.
5. Update checkpoint before invoking Opus.
Manager planning sessions count as substantial tasks when the user expects
observability. Create a run record for planning work too, with Codex manager as
the active agent.
## Agent Liveness
Each active run should include:
```txt
.agents/runs/<run-id>/
state.json
agents.json
```
`state.json` records:
- run id
- current phase
- current branch
- input artifact
- input hash
- expected output artifact
- retry count
- next action
- blocker, if any
`agents.json` records:
- role
- status: queued, running, completed, failed, dead, restarted
- started_at
- heartbeat_at
- lease_expires_at
- thread id, process id, or tool call id when available
- last_error
## Status And Progress Tail
Use the local status command from the workspace root:
```sh
node .agents/scripts/status.mjs --tail 40
```
For a browser dashboard:
```sh
node .agents/scripts/observe.mjs --port 4317
```
Then open `http://127.0.0.1:4317`.
The command reads:
- `.agents/checkpoint.json`
- every `.agents/runs/<run-id>/state.json`
- every `.agents/runs/<run-id>/agents.json`
It prints:
- all known runs
- current phase and branch
- all recorded agents and their lease status
- expired leases
- blockers
- the next action
- the tail of the active run's progress file
Progress files are checked in this order:
- `progress.log`
- `implementation-log.md`
- `validation.md`
- `opus-proposal.md`
- `snarky-review.md`
Managers should append progress events to `progress.log` whenever possible.
Human-readable phase artifacts still stay in their named markdown files.
## Agent Restart Policy
If an agent dies, the manager restarts the role from durable files, not memory.
Snarky restart:
- Read `PLAN.md`, `.agents/ORCHESTRATION.md`, checkpoint files, and active run
artifacts.
- Re-run the current Snarky phase using the same input artifact.
- Replace only the expected Snarky output for that phase.
Opus proposal/review restart:
- Re-run the same `opus-runner` planning tool with the same input artifact.
- Keep previous failed output, if any, as diagnostic context.
- Do not advance until the expected output validates.
- Use a lease and CLI/tool timeout of at least 30 minutes.
Opus implementation restart:
- Check current branch.
- Check dirty state.
- If the branch is clean, retry the same implementation instruction.
- If the branch is dirty, manager must inspect the diff and decide whether to
continue, ask Opus to repair, or ask the user.
- Never auto-reset or discard partial Opus edits.
- Use a lease and CLI/tool timeout of at least 30 minutes.
Retry limits:
- Retry a dead planning phase up to 2 times.
- Retry an implementation phase up to 1 time without user input.
- After the retry cap, record a blocker in checkpoint files.
## Checkpoint Policy
Update checkpoints:
- at the start of a run
- after Snarky brief
- after Opus proposal
- after concern resolution
- before Opus implementation
- after implementation
- after validation
- before ending the turn
Checkpoint must include:
- timestamp
- current phase
- current branch
- active run id
- completed artifacts
- next action
- blockers
- commands/checks already run
## Validation Layers
Manager validation:
- git branch and diff check
- out-of-scope file check
- syntax checks
- unit/integration tests when available
- browser/screenshot checks for UI work when available
Snarky validation:
- product requirements
- acceptance criteria
- install flow accuracy
- scope discipline
Opus validation:
- layout/design quality
- interaction clarity
- technical design concerns
## Resume Procedure
After power loss or interruption:
1. Read `.agents/CHECKPOINT.md`.
2. Read `.agents/checkpoint.json`.
3. Check git branch and dirty state.
4. Read the active run directory if present.
5. Continue from `next_action`.
6. Do not assume an Opus implementation completed unless validation is recorded.
## Current Known Setup Issue
The distribution workspace is not currently a git repository. Branch-based Opus
implementation requires initializing git or moving these files into a repo with
a clean baseline commit.

View File

@@ -0,0 +1,44 @@
# Detailed Subplan Index
These subplans break `PLAN.md` into implementation slices that can be handed to
Snarky and Opus without reloading the whole plan every time.
## Reading Order
1. `01-language-and-stdlib.md`
2. `02-domain-and-storage.md`
3. `03-cli-and-ci.md`
4. `04-http-api-and-install.md`
5. `05-artifact-validation.md`
6. `06-admin-ui.md`
7. `07-packaging-nas.md`
8. `08-orchestration-and-qa.md`
## Session Contract
- Read `PLAN.md`, `.agents/ORCHESTRATION.md`, `.agents/CHECKPOINT.md`, and
the active subplan before doing work.
- Keep work sequential. Do not use parallel implementation agents.
- Use git branches for implementation. Do not use worktrees.
- For product or UX scope, Snarky writes acceptance criteria and has final say.
- For layout and visual design, Opus has final say.
- For technical problems, consult Opus and resolve by consensus.
- Opus is the only role that writes application code during Opus phases.
- Update `.agents/CHECKPOINT.md` and `.agents/checkpoint.json` after every
completed slice, before any risky handoff, and after validation.
## Slice Exit Criteria
Every slice should finish with:
- Changed files listed.
- Tests/checks run listed.
- Known risks listed.
- Next slice named.
- Checkpoint updated.
## Current Priority
The first blocking priority is `01-language-and-stdlib.md`. Product code should
wait until the required `sx` primitives exist or an explicit temporary shim is
approved.