We are still seeing duplicate content on SEOmoz even though we have marked those pages as "noindex, follow." Any ideas why?
-
We have many pages on our website that have been set to "no index, follow." However, SEOmoz is indexing them as duplicate content. Why is that?
-
Hi Gary,
Great answer from Daniel.
One thing that you can do is to create a list of noindexed pages in excel, then add all pages identified by SEOmoz as duplicates and run a simple comparison in excel. This will identify any pages that do not match. You will easily see whether the new pages in the report can be ignored.
There is already a feature request in the works with the SEOmoz engineering team which will enable us to "turn off" pages that can be ignored (like those that are already noindexed). In the meantime, keeping track of the pages you can ignore is probably the best option.
You can keep track of progress by following updates on the Feature Request here.
Hope that helps,
Sha
-
Go to Google and search site:yourdomain.com and see if the pages in question come up. If so, Google has indexed them. If not, Google has not indexed them. Like SEOMoz, Google can crawl any page. Doesn't mean they will index the page. If you have noindexed a page, it should not be indexed by Google and should not be problematic for you.
-
So, it indexes issues that Google does see and doesn't see. How do we differentiate between the two?
Additionally, what would be some suggestions as to what we should do?
-
SEOMoz is not a search engine index, it uses a crawler. If those pages are not blocked by the robots.txt file, then SEOMoz will crawl them. They ignore the noindex tag because they don't index anything. Search engines will honor the noindex tag and not index a page if you specify with the robots meta tag. However, to remove pages from the crawl, disallow them in the robots.txt.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate pages with "/" and without "/"
I seem to have duplicate pages like the examples below: https://example.com https://example.com/ This is happening on 3 pages and I'm not sure why or how to fix it. The first (https://example.com) is what I want and is what I have all my canonicals set too, but that doesn't seem to be doing anything. I've also setup 301 redirects for each page with "/" to be redirected to the page without it. Doing this didn't seem to fix anything as when I use the (https://example.com/) URL it doesn't redirect to (https://example.com) like it's supposed to. This issue has been going on for some time, so any help would be much appreciated. I'm using Squarespace as the design/hosting site.
Technical SEO | | granitemountain0 -
Old Content Pages
Hello we run a large sports website. Since 2009 we have been doing game previews for most games every day for all the major sports..IE NFL, CFB, NBA, MLB etc.. Most of these previews generate traffic for 1-2 days leading up to or day of the event. After that there is minimal if any traffic and over the years almost nothing to the old previews. If you do a search for any of these each time the same matchup happens Google will update its rankings and filter out any old matchups/previews with new ones. So our question is what would you do with all this old content? Is it worth just keeping? Google Indexes a majority of it? Should we prune some of the old articles? The other option we thought of and its not really practical is to create event pages where we reuse a post each time the teams meet but if there was some sort of benefit we could do it.
Technical SEO | | dueces0 -
Partial duplicate content (reviews) on product pages - is this ok?
Hello, we recently received some really good reviews about a range of products we sell (there are normally 8 products in a range). Due to the industry we are in it made no sense to try and get reviews on each individual product within the range as they differ only ever so slightly. So my question is we want to add these reviews to each of the 8 products that lie within each range, but by adding them it would mean that each page has around 600 words of unique product description followed by approx 600 words of reviews that are the same on each of the products within that range. Is this ok? my only other option would be to screenshot the reviews and upload them as images below each product description. If anyone could offer advice here that would be much appreciated. Thanks
Technical SEO | | livs20130 -
Joomla creating duplicate pages, then the duplicate page's canonical points to itself - help!
Using Joomla, every time I create an article a subsequent duplicate page is create, such as: /latest-news/218-image-stabilization-task-used-to-develop-robot-brain-interface and /component/content/article?id=218:image-stabilization-task-used-to-develop-robot-brain-interface The latter being the duplicate. This wouldn't be too much of a problem, but the canonical tag on the duplicate is pointing to itself.. creating mayhem in Moz and Webmaster tools. We have hundreds of duplicates across our website and I'm very concerned with the impact this is having on our SEO! I've tried plugins such as sh404SEF and Styleware extensions, however to no avail. Can anyone help or know of any plugins to fix the canonicals?
Technical SEO | | JamesPearce0 -
Duplicate Content Errror
I am getting a duplicate content error for urls for the "tags" or categories pages for my blog. These are some the URLs that SEOmoz is saying are errors, or duplicate pages. http://sacmarketingagency.com/blog/?Tag=Facebook http://sacmarketingagency.com/blog/?Tag=content+marketing http://sacmarketingagency.com/blog/?Tag=inbound+marketing As you can see, they are just the pages that are aggregating certain blog post based on how we tagged them with the appropriate category. Is this really a problem for our SEO, if so any suggestions on how to fix this?
Technical SEO | | TalkingSheep0 -
I am trying to correct error report of duplicate page content. However I am unable to find in over 100 blogs the page which contains similar content to the page SEOmoz reported as having similar content is my only option to just dlete the blog page?
I am trying to correct duplicate content. However SEOmoz only reports and shows the page of duplicate content. I have 5 years worth of blogs and cannot find the duplicate page. Is my only option to just delete the page to improve my rankings. Brooke
Technical SEO | | wianno1680 -
Noindex duplicate content penalty?
We know that google now gives a penalty to a whole duplicate if it finds content it doesn't like or is duplicate content, but has anyone experienced a penalty from having duplicate content on their site which they have added noindex to? Would google still apply the penalty to the overall quality of the site even though they have been told to basically ignore the duplicate bit. Reason for asking is that I am looking to add a forum to one of my websites and no one likes a new forum. I have a script which can populate it with thousands of questions and answers pulled direct from Yahoo Answers. Obviously the forum wil be 100% duplicate content but I do not want it to rank for anyway anyway so if I noindex the forum pages hopefully it will not damage the rest of the site. In time, as the forum grows, all the duplicate posts will be deleted but it's hard to get people to use an empty forum so need to 'trick' them into thinking the section is very busy.
Technical SEO | | Grumpy_Carl0 -
Duplicate Page Content
Hi within my campaigns i get an error "crawl errors found" that says duplicate page content found, it finds the same content on the home pages below. Are these seen as two different pages? And how can i correct these errors as they are just one page? http://poolstar.net/ http://poolstar.net/Home_Page.php
Technical SEO | | RouteAccounts0