HasanPresident · Head of EngineeringA site can be perfectly online for your visitors and completely invisible to Google. The most insidious cause is neither content nor backlinks: it is a server configuration rule that blocks Googlebot, Google’s indexing robot. We experienced it with a client — a national network of shops investing six figures a year in advertising — whose site came up for no query at all, not even its own name.
The paradox of this type of failure: everything works. Visitors browse, ad campaigns send traffic, sales come in. Only organic traffic is at zero — and since many companies only look at total sessions, the loss goes unnoticed for months.
At our client’s, the initial symptom wasn’t even an SEO one: we had taken over the ad account, the campaigns were running well, but the brand could not be found in organic search. Basic reflex: type site:nomdusite.fr into Google. Result: almost no pages indexed.
Search Console is the first stop — but beware: in our case, it showed no obvious error. That is the trap with server-level blocks: if the server rejects Googlebot before even serving the page, the reports can stay deceptively quiet.
The decisive test is done on the command line, by pretending to be Googlebot:
curl -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" -I https://www.exemple.fr/A normal visitor received a 200 OK. Googlebot received an error. The site was literally telling Google: “don’t come in”.
On a WordPress site, two files can block a robot: robots.txt (the “polite”, visible block) and .htaccess (the invisible, server-level block). Ours was in the .htaccess: a user-agent filtering rule that rejected Googlebot, most likely added during a security intervention — nobody knew any more when or by whom. That is the key point: nobody had sabotaged the site. One well-intentioned rule, one overly broad copy-paste, and the entire indexing disappears.
The fix itself takes five minutes: delete or correct the rule. The real work comes afterwards:
| Step | Tool | Observed time frame |
|---|---|---|
| Delete the blocking rule | .htaccess |
Immediate |
| Check Googlebot’s access | curl -A + Search Console URL test |
Immediate |
| Submit the sitemap | Search Console | 24-72 h |
| Request indexing of key pages | Search Console, page by page | 1-2 weeks |
| Return of organic traffic | — | Gradual over 4-8 weeks |
site:votredomaine.fr into Google. An abnormally low number of results is the first warning sign.curl above, or with a Googlebot rendering test tool.robots.txt and ask your hosting provider whether user-agent filtering rules exist on the server side — including in a WAF or a CDN.Because your advertising budget pays twice when the site is invisible in organic search: you buy traffic that SEO should bring you for free, and Google Ads itself evaluates your landing pages. At 3 Goats, the media buyer who spots the anomaly and the engineer who opens the .htaccess are on the same team — which is what made it possible to find in a few hours what had gone unnoticed for months. The details of what we cover are on our acquisition page.
Probably not. A Googlebot block causes a sudden drop or a total absence of indexing, not erosion. A slow decline points instead to content, competition or a quality issue.
Not necessarily. If the server rejects the robot upstream, Search Console may simply stop collecting data without raising a clear alert. The live URL test, on the other hand, always tells the truth.
Allow 4 to 8 weeks for a substantial recovery, longer if the block lasted a long time. Strategic pages can be sped up through manual indexing requests.
AI audit
Our tool analyses your Google Ads account and delivers a prioritised action plan.
Read next