Updated on February 6th, 2025 at 02:50 pm
Where in the World does GoogleBot crawl your website from? Where is GoogleBot based and should you optimise your website for crawl efficiency from a specific Geo location or physical realm?
Here is a list of confirmed GoogleBot IP addresses used for crawling and the locations where these IP’s originate so you can see where your website is getting crawled from when you see a specific IP address in your crawl logs.
- Googlebot IP Address List
- IPv6 Address List
- Allow GoogleBot to Crawl
- Don't Automatically Redirect Traffic from Different Geos
- Block Unwanted URLs from Crawling
- Consolidate Duplicate Content
- Find and Fix your Sitemap.xml Files
- Use a CDN
- Improve Server Response Times
- Use Http Status Codes to Specify Changes
- Improve Page Speed
- Get Links
Googlebot IP Address List
Here’s the updated list of GoogleBot IP addresses Google uses to crawl your website from. Essentially, these are Googlebot’s IP addresses.
The list was updated on 04th February 2025, with:
+ 192.178.6.0/27 (Added)
+ 192.178.6.32/27 (Added)
- 66.249.64.192/27 (Removed)
- 66.249.66.128/27 (Removed)We’ve updated our list below to reflect the changes.
GoogleBot IP address list updated 2025 (2025-02-04T15:50:54)
192.178.6.0/27 - United States, Unknown, Unknown (new)
192.178.6.32/27 - United States, Unknown, Unknown (new)
192.178.5.0/27 - United States, Unknown, Unknown
34.100.182.96/28 - India, Maharashtra, Mumbai
34.101.50.144/28 - Indonesia, Jakarta, Jakarta
34.118.254.0/28 - Belgium, Brussels Capital, Brussels
34.118.66.0/28 - Poland, Mazovia, Warsaw
34.126.178.96/28 - Singapore, Unknown, Singapore
34.146.150.144/28 - Japan, Tokyo, Tokyo
34.147.110.144/28 - The Netherlands, Groningen, Groningen
34.151.74.144/28 - Australia, New South Wales, Sydney
34.152.50.64/28 - Canada, Quebec, Montreal
34.154.114.144/28 - Italy, Milan, Milan
34.155.98.32/28 - France, Paris, Paris
34.165.18.176/28 - Israel, Tel Aviv, Tel Aviv
34.175.160.64/28 - Spain, Madrid, Madrid
34.176.130.16/28 - Chile, Santiago Metropolitan, Santiago
34.22.85.0/27 - South Korea, Seoul, Seoul
34.64.82.64/28 - South Korea, Seoul, Seoul
34.65.242.112/28 - Switzerland, Zurich, Zurich
34.80.50.80/28 - Taiwan, Taipei City, Taipei
34.88.194.0/28 - Finland, South Karelia Region, Lappeenranta
34.89.10.80/28 - United Kingdom, England, London
34.89.198.80/28 - Germany, Hesse, Frankfurt am Main
34.96.162.48/28 - Hong Kong, Unknown, Hong Kong
35.247.243.240/28 - Brazil, Sao Paulo, São Paulo
66.249.64.0/27 - United States, Unknown, Unknown
66.249.64.128/27 - United States, Unknown, Unknown
66.249.64.160/27 - United States, Unknown, Unknown
66.249.64.224/27 - United States, Unknown, Unknown
66.249.64.32/27 - United States, Unknown, Unknown
66.249.64.64/27 - United States, Unknown, Unknown
66.249.64.96/27 - United States, Unknown, Unknown
66.249.65.0/27 - United States, Unknown, Unknown
66.249.65.160/27 - United States, Unknown, Unknown
66.249.65.192/27 - United States, Unknown, Unknown
66.249.65.224/27 - United States, Unknown, Unknown
66.249.65.32/27 - United States, Unknown, Unknown
66.249.65.64/27 - United States, Unknown, Unknown
66.249.65.96/27 - United States, Unknown, Unknown
66.249.66.0/27 - United States, Unknown, Unknown
66.249.66.160/27 - United States, Unknown, Unknown
66.249.66.192/27 - United States, Unknown, Unknown
66.249.66.32/27 - United States, Unknown, Unknown
66.249.66.64/27 - United States, Unknown, Unknown
66.249.66.96/27 - United States, Unknown, Unknown
66.249.68.0/27 - United States, Unknown, Unknown
66.249.68.32/27 - United States, Unknown, Unknown
66.249.68.64/27 - United States, Unknown, Unknown
66.249.69.0/27 - United States, Unknown, Unknown
66.249.69.128/27 - United States, Unknown, Unknown
66.249.69.160/27 - United States, Unknown, Unknown
66.249.69.192/27 - United States, Unknown, Unknown
66.249.69.224/27 - United States, Unknown, Unknown
66.249.69.32/27 - United States, Unknown, Unknown
66.249.69.64/27 - United States, Unknown, Unknown
66.249.69.96/27 - United States, Unknown, Unknown
66.249.70.0/27 - United States, Unknown, Unknown
66.249.70.128/27 - United States, Unknown, Unknown
66.249.70.160/27 - United States, Unknown, Unknown
66.249.70.192/27 - United States, Unknown, Unknown
66.249.70.224/27 - United States, Unknown, Unknown
66.249.70.32/27 - United States, Unknown, Unknown
66.249.70.64/27 - United States, Unknown, Unknown
66.249.70.96/27 - United States, Unknown, Unknown
66.249.71.0/27 - United States, Unknown, Unknown
66.249.71.128/27 - United States, Unknown, Unknown
66.249.71.160/27 - United States, Unknown, Unknown
66.249.71.192/27 - United States, Unknown, Unknown
66.249.71.224/27 - United States, Unknown, Unknown
66.249.71.32/27 - United States, Unknown, Unknown
66.249.71.64/27 - United States, Unknown, Unknown
66.249.71.96/27 - United States, Unknown, Unknown
66.249.72.0/27 - United States, Unknown, Unknown
66.249.72.128/27 - United States, Unknown, Unknown
66.249.72.160/27 - United States, Unknown, Unknown
66.249.72.192/27 - United States, Unknown, Unknown
66.249.72.224/27 - United States, Unknown, Unknown
66.249.72.32/27 - United States, Unknown, Unknown
66.249.72.64/27 - United States, Unknown, Unknown
66.249.72.96/27 - United States, Unknown, Unknown
66.249.73.0/27 - United States, Unknown, Unknown
66.249.73.128/27 - United States, Unknown, Unknown
66.249.73.160/27 - United States, Unknown, Unknown
66.249.73.192/27 - United States, Unknown, Unknown
66.249.73.224/27 - United States, Unknown, Unknown
66.249.73.32/27 - United States, Unknown, Unknown
66.249.73.64/27 - United States, Unknown, Unknown
66.249.73.96/27 - United States, Unknown, Unknown
66.249.74.0/27 - United States, Unknown, Unknown
66.249.74.128/27 - United States, Unknown, Unknown
66.249.74.32/27 - United States, Unknown, Unknown
66.249.74.64/27 - United States, Unknown, Unknown
66.249.74.96/27 - United States, Unknown, Unknown
66.249.75.0/27 - United States, Unknown, Unknown
66.249.75.128/27 - United States, Unknown, Unknown
66.249.75.160/27 - United States, Unknown, Unknown
66.249.75.192/27 - United States, Unknown, Unknown
66.249.75.224/27 - United States, Unknown, Unknown
66.249.75.32/27 - United States, Unknown, Unknown
66.249.75.64/27 - United States, Unknown, Unknown
66.249.75.96/27 - United States, Unknown, Unknown
66.249.76.0/27 - United States, Unknown, Unknown
66.249.76.128/27 - United States, Unknown, Unknown
66.249.76.160/27 - United States, Unknown, Unknown
66.249.76.192/27 - United States, Unknown, Unknown
66.249.76.224/27 - United States, Unknown, Unknown
66.249.76.32/27 - United States, Unknown, Unknown
66.249.76.64/27 - United States, Unknown, Unknown
66.249.76.96/27 - United States, Unknown, Unknown
66.249.77.0/27 - United States, Unknown, Unknown
66.249.77.128/27 - United States, Unknown, Unknown
66.249.77.160/27 - United States, Unknown, Unknown
66.249.77.192/27 - United States, Unknown, Unknown
66.249.77.224/27 - United States, Unknown, Unknown
66.249.77.32/27 - United States, Unknown, Unknown
66.249.77.64/27 - United States, Unknown, Unknown
66.249.77.96/27 - United States, Unknown, Unknown
66.249.78.0/27 - United States, Unknown, Unknown
66.249.78.32/27 - United States, Unknown, Unknown
66.249.79.0/27 - United States, Unknown, Unknown
66.249.79.128/27 - United States, Unknown, Unknown
66.249.79.160/27 - United States, Unknown, Unknown
66.249.79.192/27 - United States, Unknown, Unknown
66.249.79.224/27 - United States, Unknown, Unknown
66.249.79.32/27 - United States, Unknown, Unknown
66.249.79.64/27 - United States, Unknown, Unknown
66.249.79.96/27 - United States, Unknown, Unknown| IPv4 Addresses | Location |
|---|---|
| 192.178.5.0/27 | United States, Unknown, Unknown |
| 34.100.182.96/28 | India, Maharashtra, Mumbai |
| 34.101.50.144/28 | Indonesia, Jakarta, Jakarta |
| 34.118.254.0/28 | Belgium, Brussels Capital, Brussels |
| 34.118.66.0/28 | Poland, Mazovia, Warsaw |
| 34.126.178.96/28 | Singapore, Unknown, Singapore |
| 34.146.150.144/28 | Japan, Tokyo, Tokyo |
| 34.147.110.144/28 | The Netherlands, Groningen, Groningen |
| 34.151.74.144/28 | Australia, New South Wales, Sydney |
| 34.152.50.64/28 | Canada, Quebec, Montreal |
| 34.154.114.144/28 | Italy, Milan, Milan |
| 34.155.98.32/28 | France, Paris, Paris |
| 34.165.18.176/28 | Israel, Tel Aviv, Tel Aviv |
| 34.175.160.64/28 | Spain, Madrid, Madrid |
| 34.176.130.16/28 | Chile, Santiago Metropolitan, Santiago |
| 34.22.85.0/27 | South Korea, Seoul, Seoul |
| 34.64.82.64/28 | South Korea, Seoul, Seoul |
| 34.65.242.112/28 | Switzerland, Zurich, Zurich |
| 34.80.50.80/28 | Taiwan, Taipei City, Taipei |
| 34.88.194.0/28 | Finland, South Karelia Region, Lappeenranta |
| 34.89.10.80/28 | United Kingdom, England, London |
| 34.89.198.80/28 | Germany, Hesse, Frankfurt am Main |
| 34.96.162.48/28 | Hong Kong, Unknown, Hong Kong |
| 35.247.243.240/28 | Brazil, Sao Paulo, São Paulo |
| 66.249.64.0/27 | United States, Unknown, Unknown |
| 66.249.64.128/27 | United States, Unknown, Unknown |
| 66.249.64.160/27 | United States, Unknown, Unknown |
| 192.178.6.0/27 | United States, Unknown, Unknown |
| 66.249.64.224/27 | United States, Unknown, Unknown |
| 66.249.64.32/27 | United States, Unknown, Unknown |
| 66.249.64.64/27 | United States, Unknown, Unknown |
| 66.249.64.96/27 | United States, Unknown, Unknown |
| 66.249.65.0/27 | United States, Unknown, Unknown |
| 66.249.65.160/27 | United States, Unknown, Unknown |
| 66.249.65.192/27 | United States, Unknown, Unknown |
| 66.249.65.224/27 | United States, Unknown, Unknown |
| 66.249.65.32/27 | United States, Unknown, Unknown |
| 66.249.65.64/27 | United States, Unknown, Unknown |
| 66.249.65.96/27 | United States, Unknown, Unknown |
| 66.249.66.0/27 | United States, Unknown, Unknown |
| 192.178.6.32/27 | United States, Unknown, Unknown |
| 66.249.66.160/27 | United States, Unknown, Unknown |
| 66.249.66.192/27 | United States, Unknown, Unknown |
| 66.249.66.32/27 | United States, Unknown, Unknown |
| 66.249.66.64/27 | United States, Unknown, Unknown |
| 66.249.66.96/27 | United States, Unknown, Unknown |
| 66.249.68.0/27 | United States, Unknown, Unknown |
| 66.249.68.32/27 | United States, Unknown, Unknown |
| 66.249.68.64/27 | United States, Unknown, Unknown |
| 66.249.69.0/27 | United States, Unknown, Unknown |
| 66.249.69.128/27 | United States, Unknown, Unknown |
| 66.249.69.160/27 | United States, Unknown, Unknown |
| 66.249.69.192/27 | United States, Unknown, Unknown |
| 66.249.69.224/27 | United States, Unknown, Unknown |
| 66.249.69.32/27 | United States, Unknown, Unknown |
| 66.249.69.64/27 | United States, Unknown, Unknown |
| 66.249.69.96/27 | United States, Unknown, Unknown |
| 66.249.70.0/27 | United States, Unknown, Unknown |
| 66.249.70.128/27 | United States, Unknown, Unknown |
| 66.249.70.160/27 | United States, Unknown, Unknown |
| 66.249.70.192/27 | United States, Unknown, Unknown |
| 66.249.70.224/27 | United States, Unknown, Unknown |
| 66.249.70.32/27 | United States, Unknown, Unknown |
| 66.249.70.64/27 | United States, Unknown, Unknown |
| 66.249.70.96/27 | United States, Unknown, Unknown |
| 66.249.71.0/27 | United States, Unknown, Unknown |
| 66.249.71.128/27 | United States, Unknown, Unknown |
| 66.249.71.160/27 | United States, Unknown, Unknown |
| 66.249.71.192/27 | United States, Unknown, Unknown |
| 66.249.71.224/27 | United States, Unknown, Unknown |
| 66.249.71.32/27 | United States, Unknown, Unknown |
| 66.249.71.64/27 | United States, Unknown, Unknown |
| 66.249.71.96/27 | United States, Unknown, Unknown |
| 66.249.72.0/27 | United States, Unknown, Unknown |
| 66.249.72.128/27 | United States, Unknown, Unknown |
| 66.249.72.160/27 | United States, Unknown, Unknown |
| 66.249.72.192/27 | United States, Unknown, Unknown |
| 66.249.72.224/27 | United States, Unknown, Unknown |
| 66.249.72.32/27 | United States, Unknown, Unknown |
| 66.249.72.64/27 | United States, Unknown, Unknown |
| 66.249.72.96/27 | United States, Unknown, Unknown |
| 66.249.73.0/27 | United States, Unknown, Unknown |
| 66.249.73.128/27 | United States, Unknown, Unknown |
| 66.249.73.160/27 | United States, Unknown, Unknown |
| 66.249.73.192/27 | United States, Unknown, Unknown |
| 66.249.73.224/27 | United States, Unknown, Unknown |
| 66.249.73.32/27 | United States, Unknown, Unknown |
| 66.249.73.64/27 | United States, Unknown, Unknown |
| 66.249.73.96/27 | United States, Unknown, Unknown |
| 66.249.74.0/27 | United States, Unknown, Unknown |
| 66.249.74.128/27 | United States, Unknown, Unknown |
| 66.249.74.32/27 | United States, Unknown, Unknown |
| 66.249.74.64/27 | United States, Unknown, Unknown |
| 66.249.74.96/27 | United States, Unknown, Unknown |
| 66.249.75.0/27 | United States, Unknown, Unknown |
| 66.249.75.128/27 | United States, Unknown, Unknown |
| 66.249.75.160/27 | United States, Unknown, Unknown |
| 66.249.75.192/27 | United States, Unknown, Unknown |
| 66.249.75.224/27 | United States, Unknown, Unknown |
| 66.249.75.32/27 | United States, Unknown, Unknown |
| 66.249.75.64/27 | United States, Unknown, Unknown |
| 66.249.75.96/27 | United States, Unknown, Unknown |
| 66.249.76.0/27 | United States, Unknown, Unknown |
| 66.249.76.128/27 | United States, Unknown, Unknown |
| 66.249.76.160/27 | United States, Unknown, Unknown |
| 66.249.76.192/27 | United States, Unknown, Unknown |
| 66.249.76.224/27 | United States, Unknown, Unknown |
| 66.249.76.32/27 | United States, Unknown, Unknown |
| 66.249.76.64/27 | United States, Unknown, Unknown |
| 66.249.76.96/27 | United States, Unknown, Unknown |
| 66.249.77.0/27 | United States, Unknown, Unknown |
| 66.249.77.128/27 | United States, Unknown, Unknown |
| 66.249.77.160/27 | United States, Unknown, Unknown |
| 66.249.77.192/27 | United States, Unknown, Unknown |
| 66.249.77.224/27 | United States, Unknown, Unknown |
| 66.249.77.32/27 | United States, Unknown, Unknown |
| 66.249.77.64/27 | United States, Unknown, Unknown |
| 66.249.77.96/27 | United States, Unknown, Unknown |
| 66.249.78.0/27 | United States, Unknown, Unknown |
| 66.249.78.32/27 | United States, Unknown, Unknown |
| 66.249.79.0/27 | United States, Unknown, Unknown |
| 66.249.79.128/27 | United States, Unknown, Unknown |
| 66.249.79.160/27 | United States, Unknown, Unknown |
| 66.249.79.192/27 | United States, Unknown, Unknown |
| 66.249.79.224/27 | United States, Unknown, Unknown |
| 66.249.79.32/27 | United States, Unknown, Unknown |
| 66.249.79.64/27 | United States, Unknown, Unknown |
| 66.249.79.96/27 | United States, Unknown, Unknown |
You can verify Googlebot IP addresses via their website.
Google also list IP addresses for:
- User triggered fetchers, like Google Read Aloud and Google Publisher Centre
- Special crawlers, like AdsBot and AdSense crawlers
The obvious emergent pattern is that the majority of IP addresses GoogleBot originates from are geographically based in the USA.
However it is possible for GoogleBot to crawl your website from other geos too.
So, what does the geo-location of Google’s data centres have to do with SEO?
IPv6 Address List
Google updated their range of ipv6 addresses on February 02nd 2025. They have added approximately 4 ipv6 ranges to their list from 104 in 2024 to 108 now.
{
"creationTime": "2025-02-04T15:50:54.000000",
"prefixes": [
{
"ipv6Prefix": "2001:4860:4801:10::/64"
},
{
"ipv6Prefix": "2001:4860:4801:11::/64"
},
{
"ipv6Prefix": "2001:4860:4801:12::/64"
},
{
"ipv6Prefix": "2001:4860:4801:13::/64"
},
{
"ipv6Prefix": "2001:4860:4801:14::/64"
},
{
"ipv6Prefix": "2001:4860:4801:15::/64"
},
{
"ipv6Prefix": "2001:4860:4801:16::/64"
},
{
"ipv6Prefix": "2001:4860:4801:17::/64"
},
{
"ipv6Prefix": "2001:4860:4801:18::/64"
},
{
"ipv6Prefix": "2001:4860:4801:19::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1a::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1b::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1c::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1d::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1e::/64"
},
{
"ipv6Prefix": "2001:4860:4801:1f::/64"
},
{
"ipv6Prefix": "2001:4860:4801:20::/64"
},
{
"ipv6Prefix": "2001:4860:4801:21::/64"
},
{
"ipv6Prefix": "2001:4860:4801:22::/64"
},
{
"ipv6Prefix": "2001:4860:4801:23::/64"
},
{
"ipv6Prefix": "2001:4860:4801:24::/64"
},
{
"ipv6Prefix": "2001:4860:4801:25::/64"
},
{
"ipv6Prefix": "2001:4860:4801:26::/64"
},
{
"ipv6Prefix": "2001:4860:4801:27::/64"
},
{
"ipv6Prefix": "2001:4860:4801:28::/64"
},
{
"ipv6Prefix": "2001:4860:4801:29::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2a::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2b::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2c::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2d::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2e::/64"
},
{
"ipv6Prefix": "2001:4860:4801:2f::/64"
},
{
"ipv6Prefix": "2001:4860:4801:31::/64"
},
{
"ipv6Prefix": "2001:4860:4801:32::/64"
},
{
"ipv6Prefix": "2001:4860:4801:33::/64"
},
{
"ipv6Prefix": "2001:4860:4801:34::/64"
},
{
"ipv6Prefix": "2001:4860:4801:35::/64"
},
{
"ipv6Prefix": "2001:4860:4801:36::/64"
},
{
"ipv6Prefix": "2001:4860:4801:37::/64"
},
{
"ipv6Prefix": "2001:4860:4801:38::/64"
},
{
"ipv6Prefix": "2001:4860:4801:39::/64"
},
{
"ipv6Prefix": "2001:4860:4801:3a::/64"
},
{
"ipv6Prefix": "2001:4860:4801:3b::/64"
},
{
"ipv6Prefix": "2001:4860:4801:3c::/64"
},
{
"ipv6Prefix": "2001:4860:4801:3d::/64"
},
{
"ipv6Prefix": "2001:4860:4801:3e::/64"
},
{
"ipv6Prefix": "2001:4860:4801:40::/64"
},
{
"ipv6Prefix": "2001:4860:4801:41::/64"
},
{
"ipv6Prefix": "2001:4860:4801:42::/64"
},
{
"ipv6Prefix": "2001:4860:4801:43::/64"
},
{
"ipv6Prefix": "2001:4860:4801:44::/64"
},
{
"ipv6Prefix": "2001:4860:4801:45::/64"
},
{
"ipv6Prefix": "2001:4860:4801:46::/64"
},
{
"ipv6Prefix": "2001:4860:4801:47::/64"
},
{
"ipv6Prefix": "2001:4860:4801:48::/64"
},
{
"ipv6Prefix": "2001:4860:4801:49::/64"
},
{
"ipv6Prefix": "2001:4860:4801:4a::/64"
},
{
"ipv6Prefix": "2001:4860:4801:4b::/64"
},
{
"ipv6Prefix": "2001:4860:4801:50::/64"
},
{
"ipv6Prefix": "2001:4860:4801:51::/64"
},
{
"ipv6Prefix": "2001:4860:4801:53::/64"
},
{
"ipv6Prefix": "2001:4860:4801:54::/64"
},
{
"ipv6Prefix": "2001:4860:4801:55::/64"
},
{
"ipv6Prefix": "2001:4860:4801:60::/64"
},
{
"ipv6Prefix": "2001:4860:4801:61::/64"
},
{
"ipv6Prefix": "2001:4860:4801:62::/64"
},
{
"ipv6Prefix": "2001:4860:4801:63::/64"
},
{
"ipv6Prefix": "2001:4860:4801:64::/64"
},
{
"ipv6Prefix": "2001:4860:4801:65::/64"
},
{
"ipv6Prefix": "2001:4860:4801:66::/64"
},
{
"ipv6Prefix": "2001:4860:4801:67::/64"
},
{
"ipv6Prefix": "2001:4860:4801:68::/64"
},
{
"ipv6Prefix": "2001:4860:4801:69::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6a::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6b::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6c::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6d::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6e::/64"
},
{
"ipv6Prefix": "2001:4860:4801:6f::/64"
},
{
"ipv6Prefix": "2001:4860:4801:70::/64"
},
{
"ipv6Prefix": "2001:4860:4801:71::/64"
},
{
"ipv6Prefix": "2001:4860:4801:72::/64"
},
{
"ipv6Prefix": "2001:4860:4801:73::/64"
},
{
"ipv6Prefix": "2001:4860:4801:74::/64"
},
{
"ipv6Prefix": "2001:4860:4801:75::/64"
},
{
"ipv6Prefix": "2001:4860:4801:76::/64"
},
{
"ipv6Prefix": "2001:4860:4801:77::/64"
},
{
"ipv6Prefix": "2001:4860:4801:78::/64"
},
{
"ipv6Prefix": "2001:4860:4801:79::/64"
},
{
"ipv6Prefix": "2001:4860:4801:80::/64"
},
{
"ipv6Prefix": "2001:4860:4801:81::/64"
},
{
"ipv6Prefix": "2001:4860:4801:82::/64"
},
{
"ipv6Prefix": "2001:4860:4801:83::/64"
},
{
"ipv6Prefix": "2001:4860:4801:84::/64"
},
{
"ipv6Prefix": "2001:4860:4801:85::/64"
},
{
"ipv6Prefix": "2001:4860:4801:86::/64"
},
{
"ipv6Prefix": "2001:4860:4801:87::/64"
},
{
"ipv6Prefix": "2001:4860:4801:88::/64"
},
{
"ipv6Prefix": "2001:4860:4801:90::/64"
},
{
"ipv6Prefix": "2001:4860:4801:91::/64"
},
{
"ipv6Prefix": "2001:4860:4801:92::/64"
},
{
"ipv6Prefix": "2001:4860:4801:93::/64"
},
{
"ipv6Prefix": "2001:4860:4801:94::/64"
},
{
"ipv6Prefix": "2001:4860:4801:95::/64"
},
{
"ipv6Prefix": "2001:4860:4801:a0::/64"
},
{
"ipv6Prefix": "2001:4860:4801:c::/64"
},
{
"ipv6Prefix": "2001:4860:4801:f::/64"
},Specific changes to the 2025 list are included here:
- 2001:4860:4801:3::/64 (Removed)
+ 2001:4860:4801:1f::/64 (New)
+ 2001:4860:4801:2d::/64 (New)
+ 2001:4860:4801:4b::/64 (New)
+ 2001:4860:4801:a0::/64 (New)
+ 2001:4860:4801:94::/64 (New)
+ 2001:4860:4801:95::/64 (New)
Google also include a range of IPv6 addresses which GoogleBot may originate from for the purposes of web crawling. Due to the nature of Google’s infrastructure and IPv6 addresses, there’s no way to tie these to a specific Geo location.
However, you should still take care to make sure you’re not blocking these IPs from accessing your website.
Allow GoogleBot to Crawl
The first point almost seems too obvious to state but don’t block GoogleBot from crawling. Don’t configure bot blocking based on on IP addresses included in the list above and don’t block any bots unless you’re sure they’re imitating Googlebot, rather than the real thing.
Check things like:
- your server to make sure it is reachable
- your network to understand downtime or issues that could prevent crawling
- robots.txt rules, these can block pages, site sections or the entirety of a website from being properly crawled
Once you’ve verified GoogleBot can crawl your website effectively, check BingBot IP addresses to ensure you appear in Microsoft’s search engine.
Don’t Automatically Redirect Traffic from Different Geos
If your site serves multiple versions of content to users accessing pages from different locations around the world, then make sure you’re not automatically re-routing users to those pages.
If you always do this, you risk redirecting GoogleBot to one particular set of pages making it impossible for Google to access and crawl every version of geo specific content you serve.
Automatic redirection can also be frustrating for human users who may want to browse content specific to a different geo than they currently live in or place an order for your products if they are traveling away from home for when they return.
Block Unwanted URLs from Crawling
Blocking crawling with robots.txt is an effective way to improve the crawl budget of your website.
If you have URLs that you don’t want indexed or would add nothing by being indexed, then you can write rules for your robots.txt file to keep them out of the index.
Note that using meta robots noindex rules won’t necessarily help your crawl budget if that’s the issue causing your site pages not to be indexed.
Consolidate Duplicate Content
Google’s wording here isn’t related to using canonical tags, pages with proper canonical tags, those that reference a separate URL as the canonical version of the page, still take crawl budget.
If you’re looking to optimise crawl budget on your website look for issues of identical duplication, issues like where your site loads on both http://www.yourdomain.com/some-path/ and yourdomain.com/some-path/ both with a self referential canonical tag.
Other similar examples of issues like this can include:
- URLs loading both with and without trailing slashes, both with self referential canonical tags; e.g. yourdomain.com/some-path/ and yourdomain.com/some-path
- canonical tag errors, where a space, additional character or self generating URL parameter can create multiple versions of an indexable page with the same content.
Find and Fix your Sitemap.xml Files
Find the sitemap.xml for your website and ensure that URLs listed within the sitemap aren’t non-indexable URLs or not needed in the index.
By removing and optimising your sitemap.xml files, you can ensure that Googlebot and search engine spiders prioritise crawling of your most important, indexable URLs.
Remember to ping your sitemap to Google if you make changes to multiple pages, site hierarchy or URL structures, so Google can quickly discover the changes to your content.
Use a CDN
Even if you’re running a website based in another geo, like the U.K. solely for London, you’ll still benefit from hosting your site via a CDN (content delivery network).
A content delivery network uploads the code running your site to data centres around the World meaning the site is faster to load for both users visiting from anywhere in the World and crucially, search engines who, in the case of Google, can crawl from anywhere on the planet based on the Googlebot IP address list. However, they most commonly crawl from the United States.
The recommendation of moving your hosting to data centres based in the US is likely overkill and at worst, potentially detrimental to your SEO efforts because it is often argued that IP location of your hosting server is important for how well you rank for queries in a particular Geo.
Unless your business website is truly appealing to a Global audience, you’re best off hosting your site on a hosting platform with hosting servers close to where your organisation is based.
Improve Server Response Times
The faster your server response times, the more pages Googlebot may be able to crawl across your website.
Achieve this by paying more for better hosting, finding and fixing redirect chains that can slow crawls, blocking large resources on pages that add little or no context to the content of the pages.
Use Http Status Codes to Specify Changes
Googlebot may use If-Modified-Since to send a conditional request. If your page and content has been updated since a certain date, the content may be crawled.
If it has, a 200 status response will be returned, if not, a 304 status code will be returned.
This potential utility of If-Modified-Since underlines the critical importance of frequently updating content across all indexable pages of your website.
If you’re struggling with pages not being crawled frequently or worse, not being indexed, ensure you’ve updated the content recently and that your pages are fresh and indexable.
Including the last modified date in your meta tags is important here.
If you’re not intending to , or your pages don’t need updating, then you needn’t worry about returning 304s.
Improve Page Speed
Improving the page speed of indexable pages across your website is a critical factor for helping improve crawl budget.
Google’s resources are limited, so there’s not an infinite resource standing by to index newly published pages on your website.
Improving page speed can help Googlebot collect more data, more quickly and efficiently, when it is crawling.
Page speed performance improvements can help your users to enjoy using your website more too!
Here are even more ranking factors.
Get Links
High quality and useful content is what Google is seeking out. If your content meets these criteria and helps human users to achieve their goals, then your site could be crawled more frequently.
One way to prove how useful your content is, would be to gain authoritative backlinks from high quality websites within your niche.
With a lot of links pointing to various different pages across your website, you are more likely to see an uptick in crawl frequency because;
- Googlebot considers your site as higher quality because of the high number of links
- Googlebot will stumble across your website more frequently because of the inbound links
It is difficult to prove that proximity to Google’s data centres and crawling locations makes a difference because the benefits would likely be so negligible.
However, because Google begins crawling from the US, then US sites could be something to consider for your link building efforts.
US sites are generally higher authority, have larger visit counts and are generally more profitable than sites elsewhere. So even if the crawl frequency increases because of proximity of US sites to Google Data centres doesn’t provide measurable impacts, the higher quality of some US sites could.
If you have any kind of ambitions to grow your website internationally then it could be argued that links from US sites could help to accelerate this growth because your site will be more likely to get crawled via an external link from a US site.
Published on: 07/06/2023