README.md (10236B)
1 linkd 2 ===== 3 4 A network interface management daemon: an `ifupdown` replacement with 5 pluggable dataplane backends. It owns `/etc/network/ports` and 6 `/etc/network/interfaces`, and applies that configuration to the kernel over 7 raw netlink. 8 9 `linkd` is a multicall binary. It dispatches on `basename(argv[0])`, and the 10 package ships `ifup`, `ifdown`, `ifquery`, `ifreload` and `linkctl` as symlinks 11 to it. `linkctl` is the exception: it is a thin shim that takes its subcommand 12 from `argv[1]` and does not dispatch on `argv[0]`. 13 14 This repository contains only the daemon. Packaging, OS images and any 15 integration tests that boot a real machine belong to whatever OS consumes it. 16 17 18 Design constraints 19 ------------------ 20 21 **No shelling out to `ip`.** Every link, address, route, VLAN, bridge and VRF 22 operation goes through `src/netlink/rtnl.c` against a raw `AF_NETLINK` socket. 23 This is not a style preference. Calling `ip` reintroduces a `PATH` dependency 24 on the switch and, worse, makes operations asynchronous: the command returns 25 before the kernel has applied anything, which is what forces tests to sleep and 26 retry. Netlink acks before we reply, so callers are synchronous by 27 construction. `tests/unit/test_netlink.sh` enforces this. 28 29 **No libnl or libmnl.** `src/netlink/netlink.h` talks to the kernel directly. 30 There are no link-time dependencies: plugins are separate processes, so 31 nothing is dlopen'd. 32 33 **Accepted IPC connections are blocking.** The listener is non-blocking so it 34 can sit in `poll()`, but the accepted connection must not be: a non-blocking 35 accepted fd makes the server read `EAGAIN`, stdio report EOF, and the parser 36 see an empty stream, so it replies with nothing. That was the actual cause of 37 what looked like IPC flakiness. Bounded instead by `SO_RCVTIMEO`/`SO_SNDTIMEO`. 38 39 **Clients half-close after writing.** Without `shutdown(sock, SHUT_WR)` the 40 server waits for a second command until its receive timeout expires. 41 42 43 Building 44 -------- 45 46 make # build for linux-glibc-amd64 47 make targets # list available target triples 48 make clean 49 50 Requires [`dep`](https://github.com/finwo/dep) on `PATH` and network access to 51 `git.finwo.net`. 52 53 The top-level `Makefile` is an assembler, not a compiler. It stages a 54 self-contained tree by layering three things into `build/<triple>/`: 55 56 src/ -> build/<triple>/src the daemon sources 57 target/common/ -> build/<triple>/ the real Makefile + .dep 58 target/<triple>/ -> build/<triple>/ per-triple overlay 59 60 then runs `dep install` inside it and builds there. `target/<triple>/` is an 61 extension point for a `Makefile.target`, which `target/common/Makefile` picks 62 up with `-include`; the one existing triple has none, so it is currently empty. 63 64 `build/` is generated and gitignored in full. 65 66 ### Dependencies 67 68 `target/common/.dep` declares five: `cofyc/argparse`, `finwo/cnfparse`, 69 `finwo/pbkdf2`, `finwo/resp` and `rxi/log`. A sixth, `finwo/buf`, arrives 70 transitively through cnfparse's own manifest. All are compiled from source 71 straight into the binary; nothing is linked against a prebuilt library. 72 73 They are fetched from branch tips rather than tags, and are not checksummed. 74 That means a build is not reproducible across upstream changes, and a sha256 on 75 a release tarball of *this* repository pins its own code but not the libraries 76 compiled into the resulting binary. This is a deliberate, known deviation from 77 the pin-everything rule this project otherwise follows, and the fix (tagging 78 those repositories) is upstream of here. 79 80 81 Testing 82 ------- 83 84 ./tests/run.sh 85 86 All tests are unit tests: they build the daemon and exercise it directly. 87 `test_parse.sh` links the real `ifaces`/`ports`/`config` objects and parses a 88 generated config, so it fails when the grammar changes rather than when a 89 fixture drifts. 90 91 Anything requiring a booted system -- bridges and VLANs against a real kernel, 92 VRF table allocation, address application -- is an integration test and lives 93 in the consuming OS repository, which can build a rootfs and run a VM. 94 95 Note for anyone editing the source-grepping guards in `test_netlink.sh`: each 96 one asserts the file exists before grepping it. A grep that matches nothing 97 because a file moved is indistinguishable from one that matches nothing because 98 the code is correct, and only the second should pass. 99 100 101 Configuration 102 ------------- 103 104 `/etc/linkd.cnf`, same grammar as the rest: 105 106 config_ports /etc/network/ports /etc/network/ports.d/*.cnf 107 config_iface /etc/network/interfaces /etc/network/interfaces.d/*.cnf 108 listen unix:///var/run/linkd.sock 109 listen tcp://127.0.0.1:6789 110 authfile /etc/linkd.passwd 111 dhcp_client udhcpc -f -i {{interface}} 112 plugin /usr/lib/linkd/bcm 113 114 A missing file is not an error; the defaults above are what you get. Patterns 115 are expanded by `source`, so one matching nothing is fine -- a system with no 116 drop-ins is normal. 117 118 When `/etc/linkd.cnf` does not exist, `/etc/linkd/*.cnf` is read instead. That 119 is how a package configures linkd for something it ships without owning the 120 main config file. It is a fallback, not a merge: an explicit `--config`, or an 121 existing `/etc/linkd.cnf`, is the whole configuration. 122 123 124 Clients and authorization 125 ------------------------- 126 127 The daemon speaks RESP on every address it is told to `listen` on, so 128 `redis-cli` works as a client. `linkctl` sends whatever you give it: 129 130 linkctl PING 131 linkctl COMMAND 132 linkctl IFQUERY lo 133 134 `ifup`, `ifdown`, `ifquery` and `ifreload` are symlinks that send the 135 corresponding command. 136 137 A connection has one of three roles, and every command names the minimum it 138 needs: 139 140 | Role | Commands | 141 | ---------- | --------------------------------------------- | 142 | *none* | `PING`, `AUTH`, `QUIT` | 143 | `readonly` | + `IFQUERY`, `INFO`, `COMMAND` | 144 | `full` | + `IFUP`, `IFDOWN`, `IFRELOAD` | 145 146 **A root peer on a unix socket starts with `full`**: the kernel vouches for the 147 uid via `SO_PEERCRED`, which beats any password. Everyone else starts at *none* 148 and authenticates with `AUTH <user> <password>`. 149 150 Credentials live in the file named by `authfile`: 151 152 admin:$pbkdf2-sha256$29000$<salt>$<hash>:full 153 watcher:$pbkdf2-sha256$29000$<salt>$<hash>:readonly 154 155 Generate entries with `linkctl hash [user]`. The format is passlib's 156 `pbkdf2_sha256`, adapted base64 and all, so entries made by passlib or Django 157 work unchanged. A missing role field means `readonly`. 158 159 The file is refused outright if it is world-writable, and warned about if it is 160 readable beyond its owner. **Binding a `tcp://` listener without an `authfile` 161 is a hard error**, so an unauthenticated network listener cannot be created by 162 forgetting something. 163 164 165 DHCP 166 ---- 167 168 `iface eth0 inet dhcp` spawns the command named by `dhcp_client`, one per 169 interface, stopped again on `ifdown`. No client is known by name: the package 170 that ships one ships the config naming it. On UNOS that is busybox, which 171 drops `/etc/linkd/udhcpc.cnf`. 172 173 `{{interface}}` is substituted in each argv element, as many times as it 174 appears, with the name exactly as written in `/etc/network/interfaces`. 175 Substitution is per element, so no shell ever sees the name. 176 177 The command must stay in the foreground -- `-f` for udhcpc and sdhcp, `-B` for 178 dhcpcd. A client that forks leaves linkd holding a pid that has already exited, 179 so `ifdown` could not stop it and a detached client would keep renewing a lease 180 on an interface that is meant to be down. 181 182 linkd does not apply the lease. The client does, or the script it runs does, 183 and linkd observes the result through its netlink mirror like any other change. 184 For udhcpc that script is `/usr/share/udhcpc/default.script`; without it a 185 lease is obtained and nothing is configured. 186 187 188 Dataplane plugins 189 ----------------- 190 191 A plugin is a **program**, not a shared library. linkd talks to it in RESP 192 (the Redis wire protocol), so a plugin can be written in anything that can read 193 and write a pipe. `tests/fixtures/plugin-echo.sh` is a working one in POSIX 194 shell, and the test suite drives it, so that claim is tested rather than 195 asserted. 196 197 Plugins are declared in `/etc/linkd.cnf`: 198 199 plugin /usr/lib/linkd/bcm spawned, RESP over stdin/stdout 200 plugin tcp://user:pass@host:6789 connected to, AUTH if credentials 201 optional failures do not fail operations 202 203 On startup linkd asks each plugin `COMMAND` for the verbs it implements and 204 `INFO` for its identity. `INFO` must report `hardware_detected:0|1` in its 205 `# Stats` section; a plugin answering `0` is refused, which is how a driver for 206 absent hardware declines rather than failing later and opaquely. 207 208 Operations are **broadcast** to every plugin advertising the verb, and all of 209 them are called even if an earlier one fails, so plugins cannot end up 210 disagreeing about what was applied: 211 212 PORT ADMIN <if> <up|down> RIF ADD|DEL <if> <cidr> table <n> 213 PORT MTU <if> <mtu> NEIGH ADD|DEL <if> <ip> <mac> table <n> 214 PORT APPLY <if> [speed N] [fec X] [autoneg 0|1] 215 ROUTE ADD|DEL <dst> [via <gw> dev <if>]... [metric N] table <n> 216 RESYNC 217 218 Any `-ERR` from a plugin fails the operation unless that plugin is `optional`. 219 220 A plugin may also own configuration. When the `plugin` stanza hits a 221 sub-directive linkd does not recognise, it starts the plugin, asks `CONFIG 222 LIST` for the directives it accepts, and forwards matching ones as 223 `CONFIG SET <directive> <args...>`. A directive neither side claims ends the 224 stanza and is re-dispatched normally. 225 226 A plugin that dies or stops answering is restarted with backoff (0.5s 227 doubling to 30s). While it is down its operations fail, so a switch never 228 silently accepts configuration it is not programming into hardware. 229 230 `plugins/bcm/` is the Broadcom XGS plugin, currently a skeleton that answers 231 `COMMAND`/`INFO` and errors on everything else. It is not built by the 232 top-level Makefile and requires `SDK=` pointing at a built OpenBCM tree: 233 234 make -C plugins/bcm SDK=/path/to/opennsl 235 236 It is intended to be built and shipped by the `openbcm` package, which does not 237 exist yet.