Hostinger's CDN blocks GPTBot with HTTP 429 on every cache miss: the evidence, the diagnosis and the ticket

Any request that reaches Hostinger's edge with the versioned GPTBot user agent and no cached copy to serve gets HTTP 429 with an empty body. The response carries Server: hcdn and a CDN request ID and never touches the origin server, so nothing in the site's own configuration is involved. Requests that hit a warm cache pass, which is why the block hides from casual checks.

One detail worth stating exactly. The bare word GPTBot with no version and no Mozilla prefix passed 8 of 8 on this run. On 11 September the string GPTBot/1.2 on its own failed 5 of 5. The rule matches something in the real crawler's string, and OpenAI's real crawler sends the full string, so the practical effect is total.

A host will assume the customer's configuration is at fault, reasonably, so the job was to remove every customer-side variable by direct test before asking anyone to look at the edge. Six checks did it, and the sixth is the one that ended the argument.

1. The 429 never reaches origin. It carries Server: hcdn, a CDN request ID and Content-Length: 0. No PHP, no .htaccess, no logging script can be in the path.
2. CDN security level Medium and Off produce identical 429s. The setting is irrelevant to the rule.
3. hPanel exposes no user-agent rule of any kind. Zero IP blocks, zero country blocks. A rule that matches a user agent cannot be something the customer configured.
4. No Cloudflare in front. Nameservers and IPs are Hostinger's, and no cf- header appears anywhere.
5. Hostinger's own edge diagnostic clears the client. /hcdn-cgi/myip returns every classification flag at zero, and it returns 200 under the GPTBot identity while content paths return 429 in the same minute. The TLS fingerprint is identical across identities. The rule is on the user agent and the path, not the client.
6. A second, unrelated domain on the same host fails identically. Different site, different configuration, same 429, 8 of 8. Two unrelated domains failing the same way is not customer configuration, and Hostinger listed both in the escalation.

The response headers, request by request

Four different edge nodes (edge10, edge6, edge7, edge8, edge9) returned the same result, so it is not one misbehaving node. Every 200 is marked DYNAMIC, meaning it went to origin; every 429 has no cache status at all, because it was refused before the cache was consulted.

Ticket #21396332, opened 2026-09-11 and escalated to the infrastructure team on 2026-09-14. The support agent's confirmation, quoted from the ticket:

"The 429 is being generated at the Hostinger CDN edge, before your origin is ever reached."

"It affects two unrelated domains on our platform with no shared configuration."

"Security level and traffic blocking rules have no effect."

"Every other major crawler passes without issue; only GPTBot is affected."

"This points to a platform-level edge rule that is specifically matching GPTBot, something that exists at the infrastructure layer and is not configurable from hPanel. There is no setting available to you (or to us at the support level) that can whitelist a bot UA or IP range at the CDN edge."

"Googlebot and Bingbot are on a verified crawler allowlist that bypasses CDN security checks. GPTBot is not currently on that list."

"We cannot commit to a timeline or outcome."

Correction, 15 September 2026

This page originally said the mechanism was "not rate limiting" and described it as a verified-crawler allowlist with the largest AI crawler missing from it. That came from the first agent's explanation, and Hostinger's final answer supersedes it. On 15 September they closed the ticket and stated that their servers "apply a rate limit specifically to OpenAI's training crawler (GPTBot/1.x) at the infrastructure level", that the rule exists to "protect shared server resources for all customers on the platform", and that it is "not configurable per domain".

Every measurement on this page still stands. What changed is the label. A rate limit set low enough that 15 consecutive cache-busted requests all fail, and that still refuses the first request after nine idle hours, is indistinguishable from a blanket block when you are testing it from outside. We could not tell the difference from the client side, and we should have said so rather than ruling one out.

"The 429 responses for GPTBot are intentional and server-wide, this is not caused by anything on your website or your hPanel configuration."

"A different User-Agent string returning 200, it simply doesn't match the rate-limit rule; it does not mean your site behaves differently for that crawler."

"The issue appearing on multiple domains, because the rule is server-wide, not domain-specific."

"While we are unable to allowlist GPTBot on shared/cloud hosting, here are your available options."

"We're sorry we can't offer a per-domain exception on the current hosting plan."

The three options they offered: raise the cache TTL so more requests hit a warm cache, upgrade to a VPS or dedicated server where "you have full control over server-level rules, including the ability to remove or adjust crawler rate limits entirely", or do nothing, on the grounds that OAI-SearchBot and ChatGPT-User already reach the site. Their own words on that last point: "Your site is not invisible to ChatGPT", which matches what we measured in section 02.

So the finding is narrower and more deliberate than we first described, and in one way worse. It is not an oversight in a list. A major shared host has chosen to rate-limit OpenAI's training crawler across its platform, will not make exceptions, and prices the remedy as an upgrade. Anyone publishing on shared hosting is opted into that by default and has no way to see it.

Five minutes, one terminal, no tool. The method has three rules: defeat the cache with a random query string on every request, send the real crawler's full user-agent string rather than the bare name, and run the controls from the same machine within the same minute so nothing but the identity differs.

# 1. GPTBot, full string, cache-busted
curl -sI -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot" \
  "https://YOURSITE.com/?miss=$RANDOM" | head -n 1

# 2. Control: an ordinary browser, same second
curl -sI -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0 Safari/537.36" \
  "https://YOURSITE.com/?miss=$RANDOM" | head -n 1

# 3. Control: OpenAI's other crawler
curl -sI -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot" \
  "https://YOURSITE.com/?miss=$RANDOM" | head -n 1

# 4. Control: Googlebot
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  "https://YOURSITE.com/?miss=$RANDOM" | head -n 1

Read the result in this order. If all four return 200, you are fine. If GPTBot alone returns 429 or 403, look at the response headers: a Server value naming the CDN and a request ID with no cache status means the edge refused it before your site was involved. Repeat GPTBot ten times to rule out a budget. Then run the same four commands against a second site on the same host, ideally one you did not build. If it fails identically, you have the evidence a support agent needs, and the words to use are "reproduced on two unrelated domains, controls pass from the same IP in the same minute."

Two traps. A 200 on a page you visited a minute ago proves nothing, because the CDN served the copy it already had; the random query string is not optional. And a short sequence of 200s from a cold start proves nothing either; we saw first requests pass and then everything fail. Ten cache-busted requests is the minimum for a claim.

The raw run files behind this page are published alongside it: the 64-request run, the header capture and the log split. CC BY 4.0.

Measured

2026-09-14 10:59:13 UTC

Client

One IP, 103.147.86.116, Pakistan

Requests

64 across 4 paths, all cache-busted

Identities

8, full user-agent strings as sent, listed above

Edge nodes

edge10, edge6, edge7, edge8, edge9

Ticket

#21396332, quoted verbatim

• Single vantage point. Everything here was measured from one IP in Pakistan. The UA-scoped result, with seven identities passing from the same IP in the same minutes, makes an IP-reputation explanation very unlikely, and Hostinger's own diagnostic showed the IP clean, but the test has not been run from OpenAI's published crawler ranges.
• The production log undercounts. After the cache change, hits never reach origin, so the server-side log undercounts every crawler equally. The zero for GPTBot is against 373 for OpenAI's other bots in the same log, so the comparison holds; the absolute numbers do not.
• The bare-word result is reported, not explained. We do not know the exact pattern the rule matches, only that the string OpenAI's crawler actually sends is caught.
• This is one host. Whether other hosts' edges behave the same way is exactly what the method in section 07 is for.

Written by Abdul Subkhan. Published 2026-09-14. Our measurement rules are on the methodology page. Figures regenerate from the run files, so the page and the data cannot disagree.