conductor-worker.md (5421B)
1 # conductor-worker 2 3 Runs tasks for a [conductor](https://hub.docker.com/r/finwo/conductor) 4 server. Polls for work, runs each task in its own container, streams the 5 log back and uploads artifacts. 6 7 Source and issues: https://github.com/finwo/conductor 8 9 ## Tags 10 11 | Tag | What it is | 12 | ---------------- | ------------------------------------------ | 13 | `latest` | the most recent release | 14 | `0.10.0` | an exact version | 15 | `0.10` | the latest patch of that minor series | 16 | `main` | the tip of the default branch | 17 18 Built for `linux/amd64`, `linux/arm64` and `linux/riscv64`. 19 20 **Upgrade the conductor and its workers together.** Two releases have 21 changed what this image has to understand: 22 23 - **0.5.0** put every API response in an envelope: `{"ok":true,"data":...}` 24 on success, `{"ok":false,"error":{"name":...}}` on failure. A worker older 25 than 0.5.0 reads the claim response one level too high and will not run 26 anything for a 0.5 or newer conductor. 27 - **0.3.0** made a job one run of a pipeline and a task the unit this image 28 runs. A 0.2.x worker cannot claim from a 0.3 or newer conductor. 29 30 Pin both to the same version if they are upgraded separately. 31 32 ## Quick start 33 34 A worker needs no inbound connectivity, so this works from behind NAT. 35 Mint a token under Workers in the application, or with the admin CLI on the 36 server, then: 37 38 ```sh 39 docker run -d \ 40 --name conductor-worker \ 41 --restart unless-stopped \ 42 -e CONDUCTOR_URL=https://ci.example.com \ 43 -e CONDUCTOR_WORKER_TOKEN=... \ 44 -v /var/run/docker.sock:/var/run/docker.sock \ 45 finwo/conductor-worker 46 ``` 47 48 A worker you register is only ever offered tasks from your own projects. 49 50 ## It keeps nothing 51 52 The socket is the only thing this image needs. A task's tree is unpacked 53 into the task's own container over that socket, and its artifacts are read 54 back the same way, so the worker shares no directory with what it runs. 55 56 That means no volumes, nothing to persist between restarts, and no paths 57 that have to mean the same thing inside and outside the container. The 58 task containers are siblings on the host daemon rather than children, but 59 since nothing of the worker's filesystem is handed to them, that stops 60 being something you have to think about. 61 62 Where a task runs is decided by the build, not by this machine: the 63 `workdir` key in the pipeline, the project's setting, or `/work`. 64 65 ## Configuration 66 67 | Variable | Default | What it does | 68 | ------------------------------ | -------------- | ----------------------------------- | 69 | `CONDUCTOR_URL` | required | Where the conductor is. | 70 | `CONDUCTOR_WORKER_TOKEN` | required | Issued by the conductor, shown once. | 71 | `CONDUCTOR_WORKER_NAME` | the hostname | How this worker appears in tasks. | 72 | `CONDUCTOR_WORKER_CONCURRENCY` | `1` | Tasks at once. | 73 | `CONDUCTOR_WORKER_ARCHES` | any | Architectures offered, comma separated. | 74 | `CONDUCTOR_WORKER_TOKEN_FILE` | none | Read the token from a file instead. | 75 | `CONDUCTOR_WORKER_DOCKER` | `docker` | Runtime CLI: docker, podman, nerdctl. | 76 | `CONDUCTOR_WORKER_POLL_INTERVAL` | `5` | Seconds between polls when idle. | 77 78 Leaving `CONDUCTOR_WORKER_ARCHES` unset offers every architecture, which 79 is what you want unless one worker serves several. 80 81 ## Features 82 83 A feature is a name this worker advertises, together with the local 84 resources a task asking for it should be given. A pipeline task asks with 85 `requires:`, and the conductor will not send that task to a worker which 86 does not advertise every name it lists. The conductor only ever learns 87 the names. 88 89 Features cannot be expressed in an environment variable, because each one 90 carries mounts, devices and environment of its own. They need a 91 configuration file, JSON or YAML, mounted into the container: 92 93 ```json 94 { 95 "conductor_url": "https://ci.example.com", 96 "features": { 97 "docker": {}, 98 "sign-key": { 99 "mounts": ["/srv/keys/build.rsa:/keys/build.rsa:ro"], 100 "env": { "SIGN_KEY": "/keys/build.rsa" } 101 } 102 } 103 } 104 ``` 105 106 ```sh 107 -v /etc/conductor/worker.json:/etc/conductor/worker.json:ro 108 ``` 109 110 A file at that path is picked up on its own, with the token still coming 111 from the environment if you would rather keep it out of the file. 112 Nothing enforces what a name means beyond matching it, so `docker` is 113 simply how you say this worker's tasks may use the socket mounted above. 114 115 ## Documentation 116 117 - [Workers](https://github.com/finwo/conductor/blob/main/docs/worker.md): 118 running a worker 119 - [Pipelines](https://github.com/finwo/conductor/blob/main/docs/pipeline.md): 120 the `.conductor.yml` reference 121 - [Deployment](https://github.com/finwo/conductor/blob/main/docs/deployment.md): 122 images, state, production settings 123 - [Authentication](https://github.com/finwo/conductor/blob/main/docs/auth.md): 124 accounts, OIDC, who may see what 125 - [HTTP API](https://github.com/finwo/conductor/blob/main/docs/api.md): 126 the JSON API and its credentials 127 - [Contributing](https://github.com/finwo/conductor/blob/main/docs/contributing.md): 128 layout, testing, building 129 130 Overview and quick start: 131 [README](https://github.com/finwo/conductor/blob/main/README.md).