How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Phantom search results showing up in analytics
Hi, I'm getting these weird phantom search results showing up in my analytics account which is skewing my results. I have a wordpress site, and I do have some custom search results pages, but analytics is telling me these hits are coming from google- which I doubt. The end of the query string (&cat=plus-5-results) is not coming from a link created within my site. Anyone have any ideas? Failing that, would somebody be kind enough to explain how I can ignore these hits, but retain other, valid search result requests? Thanks reading! Will Screenshot%202019-05-16%20at%2022.32.10.jpg
Reporting & Analytics | | madegood0 -
Blocking Standard pages with Robots.txt (t&c's, shipping policy, pricing & privacy policies etc)
Hi I've just had best practice site migration completed for my old e-commerce store into a Shopify environment and I see in GSC that it's reporting my standard pages as blocked by robots.txt, such as these below examples. Surely I don't want these blocked ? is that likely due to my migrators or s defaults setting with Shopify does anyone know? : t&c's shipping policy pricing policy privacy policy etc So in summary: Shall I unblock these? What caused it Shopify default settings or more likely my migration team? All Best Dan
Reporting & Analytics | | Dan-Lawrence0 -
Do modal pop-ups impact the Google Analytics of the host page?
We have an instance of a page where visitors can click a button to start an interactive quiz. The quiz pops up in a modal window that references another domain (the interactive content provider). Will the person completing the quiz in the modal pop-up still be counted as an active visitor on the original host page during the time they are completing the quiz?
Reporting & Analytics | | MuhammadInc0 -
% Change - Google analytics - how to calculate?
Hi All,
Reporting & Analytics | | JohnPalmer
I have two dates with two different numbers I want to calculate the "% Change" like google analytics, The numbers of June 2015 - 127,931 sessions
The numbers of June 2014 - 90914 sessions please tell me what is the %Change. Best. J0 -
Behavior Flow vs. All Pages report in Google Analytics
In the interest of determining why our ecommerce site isn't converting, I've been spending some quality time with GA. I've suspected that our front page is part of the problem, especially where our organic traffic is concerned (we get a good deal of referral traffic from a link on an OEM's site). According to the Behavior Flow report under the Behavior section of GA, organic traffic to our home page is hemorrhaging (roughly 60% bounce rate). But when I went to the All Pages report (Behavior > Site Content > All Pages) and looked at organic traffic to our home page, then looked at the Medium as a secondary dimension, I'm getting a bounce rate of 35%. Why the massive discrepancy? Can somebody assist?
Reporting & Analytics | | ufmedia0 -
Google Analytics
Is there any way around being able to set up 20 goals in Analytics? What if one of my clients want to set up 10 pages with separate actions to track on each page. Will then each actions count as a separate goal? If this is the case, how could I work around creating more than 20 goals? Would I have to look into hiring/buying a third party or separate software? Your help is greatly appreciated.
Reporting & Analytics | | flcity150 -
Setting up Google Analytics for Subsites
I currently have one main .com site and am planning on launching geo-location subsites .co.uk, .com.au, .ru, etc... Traffic will flow between both sites and some of the content on the subsites will be duplicate and therefore include a canonical tag to the main site. I want to set up GA to capture who is going to the subsites and vice versa and correctly capture crossover traffic. Any advice on implementing advanced analytics directly (or links to sources that will direct me the right direction for this project)
Reporting & Analytics | | theLotter0 -
Weird info from google analytics?
Hi Could anyone explain what these visits are in Google Analytics? Under traffic sources and organic I am seeing lots of entries with data like below. Any ideas what kind of traffic this is? Is it a bot and if so what is their purpose of it and is it recommended that you block it? Pages/Visit 1.00 Avg. Time on Site 00:00:00 % New Visits : 100% Bounce Rate: 100.00% Many Thanks
Reporting & Analytics | | ocelot0