Certain Sites Encounter Reduced Crawl Rates in Google Search Console

Select Language:

Multiple SEO specialists and website owners have reported notable reductions in crawl rates within Google Search Console. Most of those experiencing this began noticing a dip around August 8th, which has persisted through to the present day. Only some users have observed this downturn, indicating it may not be a widespread issue but significant enough to warrant attention.

Google’s John Mueller acknowledged the problem, confirming it was an internal issue on their servers. He stated that the problem has since been fixed and that normal crawl activities are expected to resume automatically soon. Mueller also clarified that the reduced crawling wasn’t just a problem with reporting but an actual decrease in Google’s crawling activity, impacting some sites more than others.

While most sites haven’t felt the impact, several large websites across different platforms—including Vercel, WP Engine, and Fastly—have experienced noticeable crawling disruptions. These issues aren’t limited to Google; Bing appears to be unaffected, suggesting a problem specific to Google’s system or a broader crawling bug.

On social media and discussion forums, there are numerous threads detailing this decline. Screenshots from these discussions reveal a sharp drop in crawl requests, with some threads showing a 99% reduction from typical volumes. The decline started around August 8th, with sporadic reports appearing even later in the month, such as on August 20th and beyond.

Some experts have indicated that this disturbance might be linked to specific hosting platforms. Malte Ubl, CTO of Vercel, shared insights about a significant event where the global Googlebot crawl rate plummeted by 30%, particularly affecting large sites. He noted that crawling behavior returned to normal after the issue was addressed on Google’s end. Numerous reports also suggest this problem may tie to traffic patterns seen by tools like Cloudflare Radar, which documented an uptick in Googlebot requests during this period.

The situation remains somewhat ambiguous, with patterns indicating that the issue could stem from unforeseen bugs within Google’s crawling infrastructure or possibly interactions with certain hosting platforms. Some evidence hints at similar behaviors impacting Bing’s crawler, further suggesting a potential systemic flaw rather than an isolated platform problem.

Checking your crawl stats may help determine if your site was impacted, although rankings appear to remain stable despite the crawls’ reduction. The consensus among experts suggests that this might be a temporary glitch affecting specific large sites or platforms, but the full scope and cause are still under investigation.