What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Google is not indexing each country/language subfolder on the ranks?
Hi folks, We use Magento 2 for the multi-country shops (its a multistore). The URL: www.avarcas.com The first days Google indexed the proper url in each country: avarcas.com/uk avarcas.com/de ... Some days later, all the countries are just indexing / (the root). I correctly set the subfolders in Webmaster tools. What's happening? Thanks
International SEO | | administratorwibee0 -
Google Places and Hotel Finder
I work with a few hotels in Costa Rica and Nicaragua. Google Places has not launched in these regions so it has been impossible for me to start an account and get my properties listed in hotel finder. Somehow my competition has managed to get one of their properties listed and it's killing me! Does anyone know of any way to speed up the process? I'm very involved with Google+, FB, Twitter, Pinterest but G Places and Hotel finder or Google Maps don't want to pick me up. Any ideas? Thanks Mozzers. aSrDvhr
International SEO | | janders0 -
Google suggesting a translation
Hi everybody, I notice since some months that Google when used for german language results proposes a translation next to the listing of one of my websites. When searching for english results (hl=en) it does not propose a translation! My website is clearly in german (given as target in GWMT and by meta tag). Other pages on the same domain are not treated this way by Google. No translation is proposed for all subpages of this website. Obviously, Google considers the homepage of this website english instead of german. Any fix for that? It is a *.org Wolfgang
International SEO | | wgr_strategic0 -
CDN and the Google ranking?
Does anyone have any experience on the change in the rankings (positive or negative) after a site implemented a content delivery network (CDN) like Akamai or Level 3?
International SEO | | IvanIrishRecruiter1 -
Getting ranked in French on Google UK ?
Hellooooo the Moz community ! (#superexcited, # firstpost) Here's my problem. I'm working for a client specialised in Corporate Relocation to London for French families. (I'm reworking the entire site from the ground up, so I can manoeuvre pretty easily) The thing is, these families will either be : Searching on Google FR but mostly in English (French as well) Searching on Google UK but mostly in French ! (and of course, English as well) To be honest, I'm really not sure what strategy I should go with. Should I just target each local market in its native language and google will pick up the right language if people are searching in the "opposite" language ? I'd love some tips to help get me started. Sadly, I don't have a lot of data yet. (Client didn't even have tracking up on their site before I came in). So far here's what I got (on very small number of visitors): Location: 50+% from UK / 20+% from France.
International SEO | | detailedvision
Language : 60+% En / 35+% Fr Thank you. Tristan0 -
Http://us.burberry.com/: Big traffic change for top URL (error 593f1ceb2d67)
Please forgive duplicating this question on the SEOMoz & Webmaster Tools forum but I'm hoping to hit both audiences with this question... A few days ago I noticed that our US homepage (us.burberry.com) had dropped from PR5 to PR0, and the page has been deindexed by Google. After checking Webmaster Tools I also received the following message: http://us.burberry.com/: Big traffic change for top URL April 2, 2012Search results clicks for http://us.burberry.com/ have decreased significantly.Message ID: 593f1ceb2d67.We're not doing any link building at all (we've enough on-site issues to deal with). The only changes I have made are adding Google Analytics to the website, uploading sitemaps via Webmaster Tools (it's not linked to from robots.txt yet), and setting the burberry.com and www.burberry.com geo-location settings to 'unlisted' (we want uk.burberry.com appearing in the UK results, us.burberry.com appearing in the US results etc rather than www.burberry.com).I've reversed the geo-location settings but I doubt this would have caused this. We've duplicate copies of our homepage (such as us.burberry.com/store//) from typos in inbound links (and bad programming that allows them to work rather than 404'ing) but I don't think any of this is new. What I don't understand is (a) why this is happening now and (b) why is this just affecting our US homepage? We've ~40 different duplicates of the homepage (us, uk, ca, pt, ro, sk etc etc) so why is the US site being affected and not the others? Does anyone know if this is due to an algorithm change by Google or something else all together? Background:Our website www.burberry.com has 46 subdomains such as uk.burberry.com, ca.burberry.com and us.burberry.com. There is a lot of duplicate content on each subdomain (including basic things like tracking parameters in URLs) and across subdomains (uk.burberry.com/store & us.burberry.com/store are exactly the same), there's very little text on the site (its nearly all images), as well as poor redirects, inaccessible content (AJAX/Flash) and a whole host of basic SEO things that aren't being done correctly. I've joined the company in the last few months and have started addressing these issues but I've got a LOT of work to do yet.One thing that we have in our favour is a link profile that is as clean and natural as they come - there was only ever one link building campaign performed (which was before my time) and I had all of those links removed as soon as I joined the company.Any help would be greatly appreciated! Thanks for your timeDean RoweEdit: us.burberry.com 301 redirects to us.burberry.com/store/ as explained on the webmaster tools forum, but I don't believe this is the cause as its the same across all subdomains.
International SEO | | FashionLux0 -
Same website in different countries, best practices for SEO?
Hey Guys, I have read several similar questions regarding mine, but none seem to truly cover my question. Basically, we have a company named Junair. We created the website for the company here in Australia (http://www.junair.com.au). As can be seen throughout the page, it mentions that it caters for both Australia and NZ (NZ has its own phone number). It does ok in the rankings at the moment, but rankings will continue to rise in the future once more links are getting picked up. Now however, the Junair team in NZ purchased the NZ domain http://www.junair.co.nz and redirected it to the Australian page. No matter which page you visit on the NZ URL, the URL will never change, and neither will the page title. They have now contacted us and asked to perform SEO on the NZ domain so the NZ domain would show up in searches on Google NZ. At the moment, when searching for "Junair" on google.co.nz, the Australian domain is coming up. How could I change this so the NZ URL would show instead? And what would be the best practices to perform SEO on the NZ URL, should I just create links pointing to http://www.junair.co.nz ? Thank you, Roderic
International SEO | | Michael-Goode0 -
Subdomain hosted in a different country - what are the implications?
Hello, We are looking at creating an eCommerce section to a website and we are just weighing up the options: Magento - host on hour own server - great but it can often be very slow when hosting a shared server. Shopify - hosted solution but hosting is in the US and we are in the UK and shop will be hosted on a subdomain as a result Build our own solution - time consuming and costly There are two issues that have arisen from this situation.... Is it worse for SEO to host your store in a different country or to host in your country but your store potentially run slower? I'm swaying to the side of the argument that says give your users a good and fast experience instead of worrying about where you host the store. Bearing in mind that the main website will be hosted in the UK anyway and it is just the subdomain that will be hosted in the US. Just wondered if anybody has had experience with this or if I'm missing something? All feedback greatly appreciated! Thanks, Elias
International SEO | | A_Q0