The case against port forwarding your panel

If a management platform is going to run your hypervisors, something has to cross the boundary between their network and yours. There are two directions to choose from, and the choice determines almost everything else about the security model.

The traditional direction is inward. The platform holds credentials for your Proxmox cluster, your DirectAdmin box and your game panel, and connects to each of them over the internet. That means an API answering to the internet, a firewall rule or a VPN, and a control plane reachable by anyone who can find it.

The other direction is outward. You run a small agent inside your own network. It opens one connection to the platform and keeps it open, commands travel down that connection, and results travel back up it. Nothing on your side listens.

This piece is about what the second option costs and what it gives you, from having built one.

The firewall change is two rules, and both are outbound

The whole network requirement of an outbound agent fits in a table:

Two ports rather than one is a deliberate split. Enrolment happens before the agent has a client certificate, so that port can only offer server-side TLS. The session port requires mutual TLS and rejects anything without a valid client certificate. One listener for both would mean a listener where the client certificate is optional and the authorisation decision is conditional. Two make the boundary structural: on 50052 there is no code path in which an unauthenticated peer gets past the handshake.

If your egress filtering is strict, those two rules are the entire ask. There is no inbound rule to write, no NAT entry, no VPN concentrator, and nothing to renew when your public address changes.

What "outbound" means when the network is hostile

An outbound TCP connection through NAT is not free: the state that makes it work lives in someone else's table, and that table has a timeout. Carrier NAT, a corporate firewall and a cloud load balancer all quietly forget an idle connection, and the failure mode is the worst kind, because both ends still believe the socket is fine.

Two mechanisms handle that, and you need both.

The first is HTTP/2 keepalive, at 30 second intervals with a 10 second timeout. That is the layer that beats NAT and load balancer timeouts, and the only one that notices a TCP connection that died silently underneath the stream.

The second is an application heartbeat every 20 seconds, with a watchdog that ends the session if no acknowledgement arrives within 45 seconds. Keepalive proves the socket is alive; the heartbeat proves the other end is still processing.

When the session does end, the agent reconnects with exponential backoff and full jitter: the wait is drawn uniformly from zero to the current cap, doubling from one second up to sixty. The jitter is not decoration. Without it, every agent belonging to every customer reconnects at the same instant after a gateway restart, and your recovery turns into a self-inflicted stampede.

One detail that only shows up in production: the backoff counter resets only after a session has been healthy for a full minute. Without that rule, an agent with a half-broken connection connects, dies, resets its counter to one second, and hammers the gateway once a second forever.

Identity: a certificate, not a password

An inbound model authenticates the platform to your panel, usually with an API key that you paste into a form. An outbound model has to authenticate the agent to the platform, and it can do that with something better than a shared secret.

The agent enrols once with a single-use token: 36 random bytes, 48 base64url characters, stored server-side only as a SHA-256 hash and valid for 72 hours. It is consumed inside a transaction that re-checks the conditions it was selected on, because between the read and the write is where a duplicate enrolment gets in.

What comes back is a client certificate. From then on the session is mutually authenticated, and the gateway pins the exact peer: it takes the SHA-256 fingerprint of the presented certificate and looks the agent up by that fingerprint, before the agent has said who it thinks it is. The identity is fixed by the certificate; the first message on the stream can only confirm it.

Rotation runs on the agent's own clock: certificates live 90 days and renewal starts at the halfway mark, day 45. That leaves 45 days of retries before anything is at risk, and the panel warns a human only at 21 days left, so renewal has to have failed for three weeks before anyone is bothered. Each rotation generates a new key pair: renewing on the same key extends the life of something that may already have leaked.

Revocation deserves its own sentence, because the honest version is less elegant than the diagram. There is no CRL and no OCSP. Revoking an agent is a database update, and a connected session is checked against it once a minute. That check is fail-closed, with a budget of three consecutive failures, so a database hiccup cannot silently keep a revoked agent alive, and the worst case is bounded and computable: sixty seconds times three.

Exactly once, across a connection that will drop

Once you accept that the link is a long-lived stream over an unreliable path, delivery semantics stop being a theoretical concern. The platform will re-deliver a command after a reconnect, because from its side the difference between "the agent did not get it" and "the result did not come back" is invisible.

So the agent remembers. Completed commands are kept on disk with their results, keyed by command ID, in a bounded LRU. A repeat of a finished command returns the cached result and executes nothing. A repeat of a command that is still running is ignored, because the answer is already on its way.

On disk, specifically, and that is the point: an in-memory cache is empty exactly when the platform re-offers everything, which is right after a restart.

The reverse direction matches. Events leave through a sequenced outbox; on reconnect the platform states the sequence number it has and the agent replays from there. The outbox is bounded, and when something falls out of it the agent says so on the next connect, by declaring the lowest sequence it still holds minus one, so the platform stops waiting for events that no longer exist.

There is one honest crack in the promise, worth stating rather than hiding: if writing the dedupe record to disk fails, the result is still sent. The alternative would be dropping a completed command's result to protect a guarantee, and at worst a later re-delivery runs it a second time.

Credentials stay on your side

This is the part that changes the shape of a breach.

In the inbound model, the platform stores credentials that grant control of your infrastructure, and that store is the prize. In the outbound model, the credentials for your hypervisors and panels live in an encrypted vault on your own host, pushed there once when you add an integration.

Our vault uses XChaCha20-Poly1305 with a key derived through HKDF-SHA256 from a local key file and the machine ID, with the agent ID mixed into the derivation info. A consequence worth spelling out: copying vault.enc to another machine gives you a file that will not decrypt, because the machine it was bound to is part of the key.

And the limit, which we wrote into the source next to the implementation, is that this does not protect against root on the host. It does not need to. The customer owns the host and owns the credentials. The vault protects against a leaked file, not against the owner.

The audit trail is shaped by the same rule. Every command is a row that is never hard-deleted, and the payload of a credential push is stored redacted. An audit table that contains the secrets it is auditing is a second copy of the problem.

Four things this does not solve

An architecture piece that only lists advantages is marketing. Here is what moves rather than disappears.

Software on your host. The trade is real: you no longer expose a port, and in exchange you run a binary that has to be installed, updated and trusted. That moves the question from "who can reach my panel" to "who signs the code I run", which is a better question but not a free one.

Wide privileges. An agent that manages a hypervisor holds credentials that manage a hypervisor. The connection direction changes nothing about what those credentials can do, only where they live and who else can reach them.

Outbound is still an exit. A connection that leaves your network can carry data out of it. The destination is fixed, the transport is mutually authenticated and the protocol is narrow, but "two outbound rules" is a smaller claim than "nothing leaves".

Real-time replies. This one shows up immediately in the API. Because the platform cannot call the agent, a write cannot answer with its result: the correct answer to "create this machine" is an acknowledgement and something to poll. Reads are unaffected, since those come from the platform's own database. Anyone designing an API on top of an outbound agent has to accept that asymmetry rather than paper over it.

There is also exactly one case where the agent does listen, and pretending otherwise would be dishonest. Installing bare metal over the network means answering DHCP and serving files, which cannot be done from an outbound socket. So the agent binds those services, on one named interface, only for machines with a job in progress, and only while there is work to do. "Enabled" means "allowed here", not "listening now".

When inbound is still the right answer

If you run one hypervisor and manage it yourself, none of this applies: an agent buys nothing over a firewall rule and a strong password.

The case sharpens with distance and with count. Several sites, networks you do not own, addresses that change, customers whose firewalls you cannot edit, and a control plane you would rather not have on the public internet at all. At that point the agent is not an extra component; it is the component that lets you delete the rest.

This is how Provibr works: one outbound connection per agent, no inbound port, and credentials that never leave the customer's network. The agent documentation covers installation, sizing and revocation, getting started walks the first connection end to end, and the API documents the acknowledge-and-poll shape that the outbound design forces.