Proving you own a hostname without a certificate

Any product that lets a customer type a hostname and then checks it every minute has built two things. The first is monitoring. The second is a distributed request generator, pointed wherever the customer likes, with your company's name in the user agent and your customer's invoice address behind it.

The usual objection is that the target has a public address, so anybody could send it traffic anyway. That is true and it is not the point. A public address is not permission to measure somebody, and "anybody could" is not a defence that survives the first abuse report, particularly when the traffic is periodic, sustained and originates from an identifiable commercial platform.

So a hostname target has to be earned. This is the design we settled on, including the parts that are less obvious than they look.

Two methods that prove different things

There are two practical ways for somebody to demonstrate that a name is theirs, and it is worth being clear that they are not equivalent.

A DNS TXT record proves control of the zone. Whoever can put a value there decides where the name points, which is the strongest claim about a name that exists.

A file on the web server proves control of the machine the name currently resolves to. That is weaker: on shared hosting, more people can produce it than can edit the zone. It is also much more achievable for a customer who has a server but no access to their employer's DNS provider.

That is the same pair of trade-offs that certificate authorities make with their two common challenge types, and the same conclusion follows: offer both, record which one was used, and do not pretend they carry equal weight.

The shape of the challenge

The TXT record goes under its own label rather than on the domain itself. That is not tidiness. A record on the apex lands next to the customer's mail policy records, where every additional string pushes the answer closer to the size where resolvers start having a bad day, and where somebody else's cleanup of "old TXT records" removes it without malice. Under its own label it stands alone.

The file lives under a reserved path with the token as the filename, and the file contains the token and nothing else.

The token itself is thirty two hexadecimal characters, which is a hundred and twenty eight bits. It is not a secret and it is not a key: it is going to sit in public DNS or on a public web server. What it has to be is unguessable, because anybody who can predict a valid token can claim a name they do not own by putting it on their own server first. Lowercase hex keeps it safe for a DNS record, safe in a URL, and free of characters that can be misread when a customer copies it by hand.

One structural rule: exactly one piece of code writes out the challenge, and the result travels with the request to whoever performs the check. If both sides knew the prefix independently, there would eventually be a day when the panel displays one label and the checker looks up another, and the resulting message is "the record is not there" about a record that is there. Nothing is more expensive to support than a correct answer to a question nobody asked.

Why the agent does the fetching

This is the decision the rest hangs on.

The platform does not resolve the name and does not fetch the file. If it did, our own infrastructure would be making connections to a name typed by a user, which is server side request forgery with a form field as the attack surface. The classic result is a request to a cloud metadata endpoint from inside our own network, made by us, on request.

Instead the customer's own agent does it: the same agent that will run the checks afterwards, sitting inside the customer's own network and connecting outbound only. Lying about the outcome gains nothing, because the reward is permission to measure something the agent could already reach. The verification is performed by the party who already has the access, which is the only arrangement where the incentive to cheat disappears.

There is a consequence worth stating out loud, because it looks like a missing safeguard. Guidance for this kind of feature usually says to block private ranges, loopback, link-local and metadata addresses. That rule belongs to hosted probes, meaning measurement points that we operate and that sit in our network. For an agent inside the customer's own network, private addresses are the normal case: their hypervisor, their switch, their storage. Blocking them would disable the only measurement source for exactly the situation it exists to serve.

What both paths share is a single function deciding which addresses are allowed. If verification and measurement each had their own list, there would be a day when they disagreed, and on that day one of them is a detour around the other.

Failing closed on mixed answers

A name can resolve to several addresses, and they do not have to be equally acceptable.

If a name answers with both the customer's public address and a metadata endpoint, then every execution is a fresh draw. Nine runs measure the public address and the tenth talks to something else entirely. So the count of acceptable answers carries no information at all: one forbidden address in the set makes the whole name unusable, and the customer is told which address it was, because a refusal they cannot act on is just an error message.

The same holds during verification and during ordinary checks, which is again why there is one address policy and not two.

Details you only learn by running it

Three of these came out of building the checker; the fourth is a design decision that belongs with them.

Compare with "contains", not equality. A DNS provider is allowed to present a TXT value with quoting around it, and some control panels add quotes that they then strip again in their own display. The token is long and opaque: it is either in there or it is not, and demanding an exact match only punishes customers who did everything right.

Send the name in the host header and in the TLS server name. The connection goes to an address that already passed the policy check, but a web server with several sites needs to be told which one is being asked for. Without both, a correctly configured customer fails verification because they were served somebody else's default site.

Allow exactly one redirect. A web server that pushes everything to HTTPS is normal, and refusing the redirect would fail half the internet. Allowing many would let the proof end up somewhere else entirely. One hop, and only to the same host.

Do not require HTTPS for the file challenge. The proof is the token. A certificate adds nothing to it, and requiring one turns an ownership check into a certificate check, which is a different question with different failure modes. That is the same reasoning behind the plain HTTP challenge type in certificate issuance. Note that this default is specific to verification: when the scheme is the thing being measured, guessing it would produce a green measurement of something other than what the customer configured, and there the right behaviour is to refuse.

Proof expires, because domains change hands

A verified name should not stay verified forever, and the reason is not paranoia about our own records.

Domains change owner without telling anybody. A customer lets a registration lapse, somebody else picks it up, and from that moment we are measuring a stranger's machine every minute with our customer's name attached to the schedule. Nobody in that chain has done anything wrong and the outcome is still unacceptable.

Ninety days is the validity we chose, which is the same order as a short-lived certificate and for the same reason: short enough that a transfer is noticed quickly, long enough that customers are not doing paperwork every week. Re-verification starts fourteen days before expiry, which is deliberately generous, because a failed re-verification usually means the agent was offline or a resolver was slow rather than that ownership changed.

Attempts are rate limited to one every six hours, not tied to the monitoring interval. A verification is not a measurement: it says nothing about availability, it does not belong in an uptime figure, and asking more often only generates traffic against the customer's DNS. Six hours over fourteen days is more than fifty attempts, which turns "expired" into a real signal instead of bad luck.

The states that come out of this are pending, verified, expiring, expired and revoked, and one of them is not a gate. "Expiring" exists so the interface can warn somebody; a proof that lapses in ten days is valid today, and the checks keep running. Treating it as a separate gate state would take a customer's monitoring away on a day when nothing changed.

The database enforces the pairing rather than trusting the code: a row either has both a verification time and an expiry time or neither of them. A verified row with no expiry is a permanent grant that nobody chose, which is precisely the thing this whole design exists to prevent.

What we would write down

1. 01Decide what the challenge proves, zone control or machine control, and store which one was used.
2. 02Put the challenge under a label of your own, never on the apex next to somebody's mail records.
3. 03Make the checker the party that already has the access. If your platform fetches a user-supplied name, you have built a request forgery feature.
4. 04One address policy for verification and measurement. Two lists become two behaviours.
5. 05Fail closed on mixed answers, and name the offending address in the refusal.
6. 06Give the proof an expiry, and start renewing long before it, because the failure you are guarding against is silent by nature.

To be clear about our own state: this is a design that exists in code with its own tests, monitoring is not something you can buy from us today, and no part of the above should be read as a product announcement. It is the shape we would build again.

From the monitoring work on Provibr. The related question of measuring without touching, and how many bytes a check is allowed to send, is in why a scanner should send zero bytes; how an agent decides which of several measurement points to use is in how an agent picks the right edge server.