Agent updates

Two independent facts have to line up before a new binary runs on your host: it has to hash to the value the platform published, and it has to carry a valid signature over its raw bytes made with the platform's release key.

• The checksum proves the download is intact. It says nothing about who produced it.
• The signature proves who produced it. It is checked by the agent before anything is staged, and again by a script that only root can modify before the binary is installed.
• The public key that second check uses is owned by root. The service account the agent runs as cannot replace it, so compromising the agent still does not get a binary executed as root.
• A release without a signature is refused. So is a release whose signature does not match, even when the checksum does.

When an agent is behind, the panel says so at the top of its page and in the agent list. Press the button and the platform hands the agent a short-lived download link over the connection that is already open.

• One agent: the banner on its page, or the update card on its settings tab.
• Several at once: select them in the agent list and update the selection. The count on the button only includes the agents that can actually be updated right now.
• The dashboard has a direct button per agent that is behind, so you do not have to go looking.
• Progress survives a page reload: it is read from the command that is running, not from something the browser remembers.

Every agent has a schedule, and it is on out of the box at 03:00 in Central European Time. When a newer release is available at one of the times you set, the platform hands it to that agent the way the button does. Everything else stays the same: the same signed binary, the same checks on the host, the same rollback.

• Several times per agent, at most six, on a five minute grid. Six is enough for once every four hours and small enough to read at a glance.
• The times are read in the same wall clock the rest of the panel uses. A time that does not exist on the night the clocks go forward runs an hour later rather than being skipped, and a time that exists twice runs once.
• There is a fifteen minute catch up window and no more. An update that was meant for 03:00 is not something you want at 09:00: you picked the night on purpose.
• An agent that was offline at its time skips the night without using up its slot, and takes the next one. An agent that was reachable but already up to date does use the slot, so its history shows that somebody looked.
• At most five agents of one account leave per minute. A fleet of forty downloads roughly 22 MB each, out of one network, and spreading that over eight minutes fits inside the catch up window.
• You get a notification when a scheduled update succeeds and when one fails, grouped per account. A failing agent reports at most once per day, so a host that is stuck does not fill your notifications with the same sentence.

The agent downloads the new binary next to itself, verifies the checksum and the signature, and only then exits. Systemd restarts the service, a root-owned step verifies the staged binary once more, installs it atomically and probes that it actually runs before keeping it. If any of that fails, the old binary stays in place and the service comes back on it.

You do not have to use the panel. Everything the button does is also a handful of commands, and the panel prints them for you with the current checksum and signature filled in. The two steps worth keeping when you script it yourself are the verification steps:

Verify the checksum of what you downloaded.

echo "<sha256-from-the-panel>  provibr-agent-linux-x86_64" | sha256sum -c -

Verify the signature against the public key on the host. This is the step that a plain download does not give you, and it is the one that matters.

openssl pkeyutl -verify -pubin -inkey /usr/local/lib/provibr-agent/license-signing.pub -rawin \
  -in ./provibr-agent-linux-x86_64 -sigfile ./provibr-agent-linux-x86_64.sig