Google crawling different content--ever ok?
-
Here are a couple of scenarios I'm encountering where Google will crawl different content than my users on initial visit to the site--and which I think should be ok. Of course, it is normally NOT ok, I'm here to find out if Google is flexible enough to allow these situations:
1. My mobile friendly site has users select a city, and then it displays the location options div which includes an explanation for why they may want to have the program use their gps location. The user must choose the gps, the entire city, or he can enter a zip code, or choose a suburb of the city, which then goes to the link chosen. OTOH it is programmed so that if it is a Google bot it doesn't get just a meaningless 'choose further' page, but rather the crawler sees the page of results for the entire city (as you would expect from the url), So basically the program defaults for the entire city results for google bot, but for for the user it first gives him the initial ability to choose gps.
2. A user comes to mysite.com/gps-loc/city/results The site, seeing the literal words 'gps-loc' in the url goes out and fetches the gps for his location and returns results dependent on his location. If Googlebot comes to that url then there is no way the program will return the same results because the program wouldn't be able to get the same long latitude as that user.
So, what do you think? Are these scenarios a concern for getting penalized by Google?
Thanks, Ted
-
Thanks Cyrus. Very good points!
-
Thanks Sheena. In the second scenario good point--they are generated via user POST so in theory Google should never see them or index them, but since they can be shared Google ends up finding them, so I do need to make sure Google doesn't index them if possible.
-
This is not the definition of cloaking and I wouldn't worry too much about any penalty.
That said, anytime you redirect googlebot to a different experience than users it's a situation you want to be very careful with, and in most situations avoid. Often this is solved by serving different experiences via javascript. Even though Google is pretty darn good at parsing javascript, they will often interpret the default version of a page as if the javascript is turned off.
Regardless, I'd keep an eye on search results, Google Webmaster Tools, cached versions of your site and make ample use of "Fetch and Render" in GWT to ensure Google interprets your site they way you think it should.
-
I do not have experience with any site using this type of selector, but theoretically you should not encounter any problems as you're showing different content with the intent of improving the experience, not to deceive. If Google handles this like an ip-redirect, then you should be fine.
In scenario 2, however, I'm wondering if you even want Google to index these URLs - since it sounds like these URLs will be dynamically generated & might end up being duplicates of other pages on the site (similar to internal search pages). Something to watch out for!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Mobile site crawl returns poorer results on 100% responsive site
Has anyone experienced an issue where Google Mobile site crawl returns poorer results than their Desktop site crawl on a 100% responsive website that passes all Google Mobile tests?
Intermediate & Advanced SEO | | MFCommunications0 -
Google Adsbot crawling order confirmation pages?
Hi, We have had roughly 1000+ requests per 24 hours from Google-adsbot to our confirmation pages. This generates an error as the confirmation page cannot be viewed after closing or by anyone who didn't complete the order. How is google-adsbot finding pages to crawl that are not linked to anywhere on the site, in the sitemap or linked to anywhere else? Is there any harm in a google crawler receiving a higher percentage of errors - even though the pages are not supposed to be requested. Is there anything we can do to prevent the errors for the benefit of our network team and what are the possible risks of any measures we can take? This bot seems to be for evaluating the quality of landing pages used in for Adwords so why is it trying to access confirmation pages when they have not been set for any of our adverts? We included "Disallow: /confirmation" in the robots.txt but it has continued to request these pages, generating a 403 page and an error in the log files so it seems Adsbot doesn't follow robots.txt. Thanks in advance for any help, Sam
Intermediate & Advanced SEO | | seoeuroflorist0 -
Content with Read More..?
How does google see content that's static on page & content that has a "see more" or "read more" tag. Where the content collapses & de-collapses on a mouse click. On a condition that the complete is readable via the source code view as well as crawl-able by spiders?
Intermediate & Advanced SEO | | welcomecure0 -
Google images
Hi, I am working on a website with a large number (millions) of images. For the last five months Ihave been trying to get Google Images to crawl and index these images (example page: http://bit.ly/1ePQvyd). I believe I have followed best practice in the design of the page, naming of images etc. Whilst crawlng and indexing of the pages is going reasonably well with the standard crawler, the image bot has only crawled about half a million images and indexed only about 40,000. Can anyone suggest what I could do to increase this number 100 fold? Richard
Intermediate & Advanced SEO | | RichardTay0 -
URL Parameter Being Improperly Crawled & Indexed by Google
Hi All, We just discovered that Google is indexing a subset of our URL’s embedded with our analytics tracking parameter. For the search “dresses” we are appearing in position 11 (page 2, rank 1) with the following URL: www.anthropologie.com/anthro/category/dresses/clothes-dresses.jsp?cm_mmc=Email--Anthro_12--070612_Dress_Anthro-_-shop You’ll note that “cm_mmc=Email” is appended. This is causing our analytics (CoreMetrics) to mis-attribute this traffic and revenue to Email vs. SEO. A few questions: 1) Why is this happening? This is an email from June 2012 and we don’t have an email specific landing page embedded with this parameter. Somehow Google found and indexed this page with these tracking parameters. Has anyone else seen something similar happening?
Intermediate & Advanced SEO | | kevin_reyes
2) What is the recommended method of “politely” telling Google to index the version without the tracking parameters? Some thoughts on this:
a. Implement a self-referencing canonical on the page.
- This is done, but we have some technical issues with the canonical due to our ecommerce platform (ATG). Even though page source code looks correct, Googlebot is seeing the canonical with a JSession ID.
b. Resubmit both URL’s in WMT Fetch feature hoping that Google recognizes the canonical.
- We did this, but given the canonical issue it won’t be effective until we can fix it.
c. URL handling change in WMT
- We made this change, but it didn’t seem to fix the problem
d. 301 or No Index the version with the email tracking parameters
- This seems drastic and I’m concerned that we’d lose ranking on this very strategic keyword Thoughts? Thanks in advance, Kevin0 -
Google Local oddity
So I spotted something a little weird... one of my client's Google Local placements in blended results has the domain name - complete with the .com extension appearing where the business name typically appears: Businessxyz.com www. businessxyz .com of Google reviews Has anyone seen this? I setup their Google Places account quite some time ago and used the business name - not the url. I also setup their Google+ and Local page - using the name. None of the page titles on the website contain the url. I simply can not pinpoint where G is pulling this from or why for that matter. All competitors are appearing with business name - only my client has the domain name visible for the particular local search query. Any ideas?
Intermediate & Advanced SEO | | SCW0 -
Are links to on-page content crawled / have any effect on page rank?
Lets say I have a really long article that begins with links to <a name="something">anchors on the same page.</a> <a name="something"></a> <a name="something">E.g.,</a> Chapter 1, Chapter 2, etc, allowing the user to scroll down to different content. There are also other links on this page that link to other pages. A few questions: Googlebot arrives on the page. Does it crawl links that point to anchors on the same page? When link juice is divided among all the links on the page, do these links count and page rank is then lost? Thanks!
Intermediate & Advanced SEO | | anthematic0 -
How to position in local Google
Hello, It's been easy for me to jump in .com - English only results. But, regional google is making me problems. How to position there? What are the top 3 key elements for ranking locally that don't matter much internationally? Thanks!
Intermediate & Advanced SEO | | DaBomb110