README.md (6475B)
1 unos-repository 2 =============== 3 4 The UNOS package repository: every `.apk` that makes up the OS, the tooling 5 that builds and signs them, and the CI pipeline that publishes them. 6 7 This repo produces a signed apk repository. It does not assemble a rootfs, an 8 image or an installer -- that is the `unos` repo, which consumes what is 9 published here over HTTP. The `linkd` daemon's source lives in its own repo and 10 arrives here as a tag-pinned tarball like any other dependency. 11 12 Published to `https://repo.unos.finwo.net/apk/<channel>/<arch>`. 13 14 15 Layout 16 ------ 17 18 channels/<layer>/channel.conf upstream= and publish= 19 channels/<layer>/packages/<name>/template build recipe (mk/build.sh) 20 channels/<layer>/packages/<name>/files/ static files 21 channels/<layer>/packages/<name>/patches/ *.patch, applied to WRKSRC 22 builder/ the CI image: Dockerfile + VERSION 23 mk/ the build driver and its tooling 24 .conductor.yml CI pipeline, GENERATED from the dep graph 25 crossed with the channel list 26 27 Channels chain `base` -> `lts` -> `stable` -> `edge`, each layer replacing 28 whole package directories from the one above. `base` holds every package and 29 is never published; `lts` is empty, so it is base. 30 31 32 Building 33 -------- 34 35 ./mk/bootstrap-host.sh build pinned muon + apk into build/host/bin 36 ./mk/build.sh <pkg> build one package (--channel, default lts) 37 ./mk/build-all.sh build everything in dependency order 38 ./mk/repo-index.sh (re)generate and sign APKINDEX.tar.gz 39 40 `mk/bootstrap-host.sh` must run first: `apk-tools` declares `muon` as a 41 hostmakedepends, and it is built here rather than installed from a 42 distribution, so `mk/deps.sh check` reports it missing until bootstrap has 43 produced it. 44 45 See `mk/README.md` for the full tooling reference and 46 `channels/base/packages/README.md` for the template format. 47 48 49 Where source comes from 50 ----------------------- 51 52 Two mutually exclusive mechanisms, kept apart deliberately: 53 54 `distfiles=` is third-party upstream source. It **must** carry a matching 55 `checksum=` sha256; `mk/build.sh` refuses to build without one. We are trusting 56 a server nobody here controls, so the tarball is pinned by content. 57 58 `srcfiles=` is our own source, fetched by tag from `git.finwo.net`. It carries 59 no checksum: the tag is the pin and TLS is the transport, and a forge 60 regenerates tag archives, so they are not byte-stable and a sha256 would fail 61 for reasons indistinguishable from tampering. Only `linkd` uses this today. 62 63 Keeping them separate means "no checksum" can never silently apply to an 64 upstream tarball. 65 66 Cached source lives in `build/work/sources/`, and is only fetched when absent. 67 `srcfiles` entries are cached under `<pkg>-<basename>` because these URLs end 68 in the tag, so the bare basename is often just `v0.1.0.tar.gz` and two 69 first-party packages at the same version would otherwise share one entry. 70 71 Signing 72 ------- 73 74 Packages and the index are signed with RSA-4096 (apk mandates RSA). The key is 75 resolved by `mk/sign-key.inc`, in order: 76 77 $UNOS_KEY_APK_RSA_PRI provided by the unos-sign-key CI worker feature 78 $UNOS_SIGN_KEY manual override 79 ./.sign-key local selection, via mk/keymgmt.sh use 80 the single *.rsa in ~/.unos-keys/ 81 82 `mk/build.sh` names the signature record after the key that signed it 83 (`.SIGN.RSA.<keyname>.rsa.pub`), and the apk client verifies it against 84 `/etc/apk/keys/<keyname>.rsa.pub`, shipped by the `unos-keys` package. 85 86 **Every CI worker has a distinct keypair**, so the signer is whichever worker 87 the conductor scheduled the build on. `unos-keys` therefore trusts all 88 four. Ship fewer and verification fails for an arbitrary subset of packages 89 that changes run to run -- a failure that looks like corruption rather than a 90 missing key. Adding a worker means publishing `unos-keys` *before* that worker 91 builds anything. 92 93 Any one worker can consequently sign packages every client trusts. The intended 94 end state is a release step that re-signs with a single key and merely accepts 95 these as inputs; it does not exist yet. 96 97 98 CI 99 -- 100 101 `.conductor.yml` is **generated** -- do not edit it: 102 103 ./mk/gen-pipeline.sh --write regenerate 104 ./mk/gen-pipeline.sh --check fail if stale (run by tests/unit/test_deps.sh) 105 106 The conductor treats the pipeline as data and will not execute repository code 107 to build its task graph, so the per-package `needs` edges have to be written 108 out literally. Generating them from `makedepends` keeps the templates the 109 single source of truth; drift is a test failure rather than a package building 110 against a stale dependency. 111 112 Tasks run in ephemeral containers with no mounts and no artifact sharing, so 113 each package seeds its sysroot from the published repo (`mk/sysroot.sh`) and 114 publishes its own result (`mk/publish.sh`). `needs` provides ordering only. 115 116 Every package task builds on the prebuilt `finwo/unos-builder` image, and the 117 `builder` task rebuilds and pushes it when `builder/VERSION` moves. Each 118 package task probes `mk/check-revision.sh` first and skips when that exact 119 `pkgver-r` is already published, so a rerun rebuilds only what changed. 120 121 Two worker features are required. **`unos-sign-key`** mounts the signing key 122 and exports `UNOS_KEY_APK_RSA_PRI`, `UNOS_KEY_APK_RSA_PUB`, 123 `UNOS_KEY_SUPERCOP_PRI` and `UNOS_KEY_SUPERCOP_PUB`. **`docker`** is needed by 124 the `builder` task alone, which runs a docker-in-docker image rather than the 125 builder it is producing. Feature names are global to the conductor, which 126 hosts more than UNOS, hence the project prefix. A task requiring a feature no 127 worker advertises is never scheduled -- it does not fail, it silently never 128 runs. 129 130 S3 credentials arrive as `BUCKET_ACCESS_KEY`, `BUCKET_SECRET_KEY`, 131 `BUCKET_NAME`, `BUCKET_ENDPOINT` (a URL), `BUCKET_REGION` (a region name) and 132 `BUCKET_PUBLIC_URL` (the bare public base, with no `apk/` segment -- the 133 publish and sysroot steps append it). Pushing the builder image additionally 134 needs `REGISTRY_USERNAME` and `REGISTRY_TOKEN`, plus `REGISTRY_HOST` for a 135 registry other than Docker Hub. 136 137 138 Testing 139 ------- 140 141 ./tests/run.sh 142 143 Graph validation, pipeline freshness, applet policy and packaging layout. Tests 144 that need a package built will skip rather than fail when it is absent, so the 145 suite is meaningful in a fresh container; anything needing a booted machine 146 lives in the `unos` repo.