conductor.md (7500B)
1 # conductor 2 3 A small CI server. Accepts triggers, schedules jobs, serves the worker 4 API and hosts the application that drives it. One service, one image. 5 6 Source and issues: https://github.com/finwo/conductor 7 8 ## Tags 9 10 | Tag | What it is | 11 | ---------------- | ------------------------------------------ | 12 | `latest` | the most recent release | 13 | `0.10.0` | an exact version | 14 | `0.10` | the latest patch of that minor series | 15 | `main` | the tip of the default branch | 16 17 Built for `linux/amd64`, `linux/arm64` and `linux/riscv64`. 18 19 **0.10.0 closed several ways in.** Workers are unaffected; the protocol 20 they speak is unchanged. 21 22 - A project with no trigger secret no longer accepts anonymous triggers. 23 Starting a build now takes a valid signature, or a session that may 24 manage the project. Anything driving `/trigger` without either stops 25 working, deliberately. 26 - A trigger secret starts builds and no longer reads them back. Reading a 27 private job, task, log or artifact needs a session. 28 - `visibility` in `.conductor.yml` is refused, and a file still carrying it 29 fails to compile. Visibility is a project setting: use 30 `PATCH /api/v1/projects/:id`. 31 - `repo_url` must be `https`, `ssh`, `git` or an `scp`-style 32 `user@host:path`. Other schemes select a git transport rather than a 33 location and are rejected. 34 - Changing your own password now takes `current_password` alongside the new 35 one. An administrator resetting someone else's does not. 36 37 **0.9.0 changed how people sign in.** Workers are unaffected; the protocol 38 they speak is unchanged. 39 40 - The server rendered interface is gone. The image now serves a single page 41 application that talks to `/api/v1`, and sessions are bearer tokens 42 rather than a cookie. Everyone is signed out once on upgrade. 43 - `CONDUCTOR_OIDC_CLIENT_SECRET` no longer exists. An OIDC client must be 44 reconfigured from confidential to **public**, with PKCE required, the 45 conductor's origin in Web Origins, and that origin registered as a 46 redirect URI. Leaving it confidential means nobody can sign in. 47 - `CONDUCTOR_SESSION_TTL` is replaced by `CONDUCTOR_ACCESS_TTL` and 48 `CONDUCTOR_REFRESH_TTL`. 49 50 **Upgrade the conductor and its workers together.** Two releases have 51 changed what a worker has to understand: 52 53 - **0.5.0** put every API response in an envelope: `{"ok":true,"data":...}` 54 on success, `{"ok":false,"error":{"name":...}}` on failure. A worker older 55 than 0.5.0 reads the claim response one level too high and will not run 56 anything for a 0.5 or newer conductor. 57 - **0.3.0** made a job one run of a pipeline and a task the unit a worker 58 runs. A 0.2.x worker cannot claim from a 0.3 or newer conductor. 59 60 ## Quick start 61 62 ```sh 63 docker run -d \ 64 --name conductor \ 65 -p 8080:8080 \ 66 -v conductor-data:/data \ 67 -e CONDUCTOR_PUBLIC_URL=http://localhost:8080 \ 68 finwo/conductor 69 ``` 70 71 The administrator password is printed once in the log on first start. 72 Set `CONDUCTOR_ADMIN_PASSWORD` to choose it instead. 73 74 Tasks need a worker, which is a separate image: see 75 [finwo/conductor-worker](https://hub.docker.com/r/finwo/conductor-worker). 76 A compose file running both is in the repository under `deploy/`. 77 78 ## State 79 80 Everything lives in `/data`, so mount a volume there: 81 82 ``` 83 /data/conductor.db sqlite, unless a database url is configured 84 /data/mirrors one bare git mirror per project 85 /data/logs the live log spool 86 /data/storage artifacts and archived logs, unless S3 is configured 87 ``` 88 89 ## Configuration 90 91 Every setting has a default and can come from the environment, so no 92 configuration file is needed. Worth setting in production: 93 94 | Variable | Why | 95 | -------------------------- | ------------------------------------------------ | 96 | `CONDUCTOR_PUBLIC_URL` | Workers get absolute callback URLs built from it, and it is the issuer tokens are signed under. | 97 | `CONDUCTOR_SESSION_SECRET` | Otherwise every access token stops verifying at restart. | 98 | `CONDUCTOR_SECRET_KEY` | Otherwise stored secrets are kept in the clear. | 99 | `CONDUCTOR_ADMIN_PASSWORD` | Otherwise one is generated and logged once. | 100 101 Generate the two secrets with `openssl rand -hex 32`. 102 103 ## Signing in 104 105 The image serves the application and a login page for it. 106 107 | Variable | Default | What it does | 108 | ------------------------------- | -------------- | ------------------------------- | 109 | `CONDUCTOR_ACCESS_TTL` | `900` | Access token lifetime, seconds. | 110 | `CONDUCTOR_REFRESH_TTL` | `2592000` | Session lifetime, seconds. | 111 | `CONDUCTOR_REFRESH_SLIDING` | `true` | Measure that from last use rather than from signing in. | 112 | `CONDUCTOR_LOCAL_REDIRECT_URIS` | `CONDUCTOR_PUBLIC_URL` | Origins the browser may be sent back to, comma separated. | 113 | `CONDUCTOR_LOCAL_AUTH` | derived | Force the built-in login page even with a provider configured. | 114 115 `localhost` and `127.0.0.1` are different origins. An unregistered origin 116 is refused, so list every one in use. `*` allows any, for local testing. 117 118 An external provider instead of the built-in page: 119 120 ```sh 121 -e CONDUCTOR_OIDC_DISCOVERY_URL=https://idp.example.com/realms/ci/.well-known/openid-configuration \ 122 -e CONDUCTOR_OIDC_CLIENT_ID=conductor \ 123 -e CONDUCTOR_OIDC_ADMIN_ROLE=conductor-admin 124 ``` 125 126 Register conductor as a **public** client with PKCE. There is no client 127 secret: the application redeems the code itself and the conductor never 128 does. Only one provider is ever used, so configuring this turns the 129 built-in login page off. 130 131 Postgres or MySQL instead of sqlite: 132 133 ```sh 134 -e CONDUCTOR_DATABASE_URL=postgres://conductor:secret@postgres:5432/conductor 135 ``` 136 137 S3 compatible storage instead of the volume: 138 139 ```sh 140 -e CONDUCTOR_S3_ENDPOINT=http://minio:9000 \ 141 -e CONDUCTOR_S3_BUCKET=conductor \ 142 -e CONDUCTOR_S3_ACCESS_KEY_ID=... \ 143 -e CONDUCTOR_S3_SECRET_ACCESS_KEY=... 144 ``` 145 146 Artifacts and finished logs then go to the bucket, and downloads are 147 handed over with a presigned redirect rather than proxied. 148 149 ## Retention 150 151 Build output is deleted on a schedule, or a busy server fills its disk 152 with logs nobody will read. The defaults keep artifacts for 30 days or 153 the last 10 jobs, and logs for 14 days, and a project may override any of 154 it. Upgrading an installation that has been running for a while will 155 delete a lot on the first sweep, so set these before starting if that 156 matters: 157 158 ```sh 159 -e CONDUCTOR_RETENTION_ARTIFACT_DAYS=0 \ 160 -e CONDUCTOR_RETENTION_LOG_DAYS=0 161 ``` 162 163 Zero means keep forever. 164 165 ## Documentation 166 167 - [Deployment](https://github.com/finwo/conductor/blob/main/docs/deployment.md): 168 images, state, production settings 169 - [Authentication](https://github.com/finwo/conductor/blob/main/docs/auth.md): 170 accounts, OIDC, who may see what 171 - [Pipelines](https://github.com/finwo/conductor/blob/main/docs/pipeline.md): 172 the `.conductor.yml` reference 173 - [Workers](https://github.com/finwo/conductor/blob/main/docs/worker.md): 174 running a worker 175 - [HTTP API](https://github.com/finwo/conductor/blob/main/docs/api.md): 176 the JSON API and its credentials 177 - [Contributing](https://github.com/finwo/conductor/blob/main/docs/contributing.md): 178 layout, testing, building 179 180 Overview and quick start: 181 [README](https://github.com/finwo/conductor/blob/main/README.md).