README.md (8846B)
1 linkd 2 ===== 3 4 A network interface management daemon: an `ifupdown` replacement with 5 pluggable dataplane backends. It owns `/etc/network/ports` and 6 `/etc/network/interfaces`, and applies that configuration to the kernel over 7 raw netlink. 8 9 `linkd` is a multicall binary. It dispatches on `basename(argv[0])`, and the 10 package ships `ifup`, `ifdown`, `ifquery`, `ifreload` and `linkctl` as symlinks 11 to it. `linkctl` is the exception: it is a thin shim that takes its subcommand 12 from `argv[1]` and does not dispatch on `argv[0]`. 13 14 This repository contains only the daemon. Packaging, OS images and any 15 integration tests that boot a real machine belong to whatever OS consumes it. 16 17 18 Design constraints 19 ------------------ 20 21 **No shelling out to `ip`.** Every link, address, route, VLAN, bridge and VRF 22 operation goes through `src/netlink/rtnl.c` against a raw `AF_NETLINK` socket. 23 This is not a style preference. Calling `ip` reintroduces a `PATH` dependency 24 on the switch and, worse, makes operations asynchronous: the command returns 25 before the kernel has applied anything, which is what forces tests to sleep and 26 retry. Netlink acks before we reply, so callers are synchronous by 27 construction. `tests/unit/test_netlink.sh` enforces this. 28 29 **No libnl or libmnl.** `src/netlink/netlink.h` talks to the kernel directly. 30 There are no link-time dependencies: plugins are separate processes, so 31 nothing is dlopen'd. 32 33 **Accepted IPC connections are blocking.** The listener is non-blocking so it 34 can sit in `poll()`, but the accepted connection must not be: a non-blocking 35 accepted fd makes the server read `EAGAIN`, stdio report EOF, and the parser 36 see an empty stream, so it replies with nothing. That was the actual cause of 37 what looked like IPC flakiness. Bounded instead by `SO_RCVTIMEO`/`SO_SNDTIMEO`. 38 39 **Clients half-close after writing.** Without `shutdown(sock, SHUT_WR)` the 40 server waits for a second command until its receive timeout expires. 41 42 43 Building 44 -------- 45 46 make # build for linux-glibc-amd64 47 make targets # list available target triples 48 make clean 49 50 Requires [`dep`](https://github.com/finwo/dep) on `PATH` and network access to 51 `git.finwo.net`. 52 53 The top-level `Makefile` is an assembler, not a compiler. It stages a 54 self-contained tree by layering three things into `build/<triple>/`: 55 56 src/ -> build/<triple>/src the daemon sources 57 target/common/ -> build/<triple>/ the real Makefile + .dep 58 target/<triple>/ -> build/<triple>/ per-triple overlay 59 60 then runs `dep install` inside it and builds there. `target/<triple>/` is an 61 extension point for a `Makefile.target`, which `target/common/Makefile` picks 62 up with `-include`; the one existing triple has none, so it is currently empty. 63 64 `build/` is generated and gitignored in full. 65 66 ### Dependencies 67 68 `target/common/.dep` declares three: `cofyc/argparse`, `finwo/cnfparse` and 69 `rxi/log`. A fourth, `finwo/buf`, arrives transitively through cnfparse's own 70 manifest. All four are compiled from source straight into the binary; nothing 71 is linked against a prebuilt library. 72 73 They are fetched from branch tips rather than tags, and are not checksummed. 74 That means a build is not reproducible across upstream changes, and a sha256 on 75 a release tarball of *this* repository pins its own code but not the four 76 libraries compiled into the resulting binary. This is a deliberate, known 77 deviation from the pin-everything rule this project otherwise follows, and the fix 78 (tagging those repositories) is upstream of here. 79 80 81 Testing 82 ------- 83 84 ./tests/run.sh 85 86 All tests are unit tests: they build the daemon and exercise it directly. 87 `test_parse.sh` links the real `ifaces`/`ports`/`config` objects and parses a 88 generated config, so it fails when the grammar changes rather than when a 89 fixture drifts. 90 91 Anything requiring a booted system -- bridges and VLANs against a real kernel, 92 VRF table allocation, address application -- is an integration test and lives 93 in the consuming OS repository, which can build a rootfs and run a VM. 94 95 Note for anyone editing the source-grepping guards in `test_netlink.sh`: each 96 one asserts the file exists before grepping it. A grep that matches nothing 97 because a file moved is indistinguishable from one that matches nothing because 98 the code is correct, and only the second should pass. 99 100 101 Configuration 102 ------------- 103 104 `/etc/linkd.cnf`, same grammar as the rest: 105 106 config_ports /etc/network/ports /etc/network/ports.d/*.cnf 107 config_iface /etc/network/interfaces /etc/network/interfaces.d/*.cnf 108 listen unix:///var/run/linkd.sock 109 listen tcp://127.0.0.1:6789 110 authfile /etc/linkd.passwd 111 plugin /usr/lib/linkd/bcm 112 113 A missing file is not an error; the defaults above are what you get. Patterns 114 are expanded by `source`, so one matching nothing is fine -- a system with no 115 drop-ins is normal. 116 117 118 Clients and authorization 119 ------------------------- 120 121 The daemon speaks RESP on every address it is told to `listen` on, so 122 `redis-cli` works as a client. `linkctl` sends whatever you give it: 123 124 linkctl PING 125 linkctl COMMAND 126 linkctl IFQUERY lo 127 128 `ifup`, `ifdown`, `ifquery` and `ifreload` are symlinks that send the 129 corresponding command. 130 131 A connection has one of three roles, and every command names the minimum it 132 needs: 133 134 | Role | Commands | 135 | ---------- | --------------------------------------------- | 136 | *none* | `PING`, `AUTH`, `QUIT` | 137 | `readonly` | + `IFQUERY`, `INFO`, `COMMAND` | 138 | `full` | + `IFUP`, `IFDOWN`, `IFRELOAD` | 139 140 **A root peer on a unix socket starts with `full`**: the kernel vouches for the 141 uid via `SO_PEERCRED`, which beats any password. Everyone else starts at *none* 142 and authenticates with `AUTH <user> <password>`. 143 144 Credentials live in the file named by `authfile`: 145 146 admin:$pbkdf2-sha256$29000$<salt>$<hash>:full 147 watcher:$pbkdf2-sha256$29000$<salt>$<hash>:readonly 148 149 Generate entries with `linkctl hash [user]`. The format is passlib's 150 `pbkdf2_sha256`, adapted base64 and all, so entries made by passlib or Django 151 work unchanged. A missing role field means `readonly`. 152 153 The file is refused outright if it is world-writable, and warned about if it is 154 readable beyond its owner. **Binding a `tcp://` listener without an `authfile` 155 is a hard error**, so an unauthenticated network listener cannot be created by 156 forgetting something. 157 158 159 Dataplane plugins 160 ----------------- 161 162 A plugin is a **program**, not a shared library. linkd talks to it in RESP 163 (the Redis wire protocol), so a plugin can be written in anything that can read 164 and write a pipe. `tests/fixtures/plugin-echo.sh` is a working one in POSIX 165 shell, and the test suite drives it, so that claim is tested rather than 166 asserted. 167 168 Plugins are declared in `/etc/linkd.cnf`: 169 170 plugin /usr/lib/linkd/bcm spawned, RESP over stdin/stdout 171 plugin tcp://user:pass@host:6789 connected to, AUTH if credentials 172 optional failures do not fail operations 173 174 On startup linkd asks each plugin `COMMAND` for the verbs it implements and 175 `INFO` for its identity. `INFO` must report `hardware_detected:0|1` in its 176 `# Stats` section; a plugin answering `0` is refused, which is how a driver for 177 absent hardware declines rather than failing later and opaquely. 178 179 Operations are **broadcast** to every plugin advertising the verb, and all of 180 them are called even if an earlier one fails, so plugins cannot end up 181 disagreeing about what was applied: 182 183 PORT ADMIN <if> <up|down> RIF ADD|DEL <if> <cidr> table <n> 184 PORT MTU <if> <mtu> NEIGH ADD|DEL <if> <ip> <mac> table <n> 185 PORT APPLY <if> [speed N] [fec X] [autoneg 0|1] 186 ROUTE ADD|DEL <dst> [via <gw> dev <if>]... [metric N] table <n> 187 RESYNC 188 189 Any `-ERR` from a plugin fails the operation unless that plugin is `optional`. 190 191 A plugin may also own configuration. When the `plugin` stanza hits a 192 sub-directive linkd does not recognise, it starts the plugin, asks `CONFIG 193 LIST` for the directives it accepts, and forwards matching ones as 194 `CONFIG SET <directive> <args...>`. A directive neither side claims ends the 195 stanza and is re-dispatched normally. 196 197 A plugin that dies or stops answering is restarted with backoff (0.5s 198 doubling to 30s). While it is down its operations fail, so a switch never 199 silently accepts configuration it is not programming into hardware. 200 201 `plugins/bcm/` is the Broadcom XGS plugin, currently a skeleton that answers 202 `COMMAND`/`INFO` and errors on everything else. It is not built by the 203 top-level Makefile and requires `SDK=` pointing at a built OpenBCM tree: 204 205 make -C plugins/bcm SDK=/path/to/opennsl 206 207 It is intended to be built and shipped by the `openbcm` package, which does not 208 exist yet.