"Issue: Duplicate Page Content " in Crawl Diagnostics - but these pages are noindex
-
Saw an issue back in 2011 about this and I'm experiencing the same issue. http://moz.com/community/q/issue-duplicate-page-content-in-crawl-diagnostics-but-these-pages-are-noindex
We have pages that are meta-tagged as no-everything for bots but are being reported as duplicate. Any suggestions on how to exclude them from the Moz bot?
-
Technically that could be done in your robots.txt file but I wouldn't recommend that if you want Google to crawl them too. I'm not sure if Rogerbot can do that. Sorry I couldn't be more help.
If you don't get one of the staffers on here in the next few days, I would send a ticket to them for clarification.
If you decide to go with robots.txt here is a resource from Google on implementing and testing it. https://support.google.com/webmasters/answer/156449?hl=en
-
Thanks for the information on Rogerbot. I understand the difference between the bots from Google and Moz.
Some errors reported in Moz are not real. For example we use a responsive slider on the home page that generates the slides from specific pages. These pages are tagged to no-everything so as to be invisible to bots, yet they are generating errors in the reports.
Is there anyway to exclude some pages from the reports?
-
Don't forget that Rogerbot (moz's crawler) is a robot and not an index like Google. Google used robots to gather the data but the results we see is an index. Rogerbot will crawl the pages regardless of noindex or nofollow.
Here is more info on RogerBot http://moz.com/help/pro/rogerbot-crawler
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to fix thin content issue?
Hello! I've checked my website via Moz and received "thin content" issue: "Your page is considered to have "thin content" if it has less than 50 words" But I definitely know that we have 5 text blocks with unique content, each block consist of more than 50 words. Do you have any ideas what may cause this issue? Thanks in advance, Yana
On-Page Optimization | | yanamazault0 -
"Avoid Multiple Page Title Elements"
Hi, in page recommondation I get a "Acoid Multiple Page Tile Elements" Fix. Make sure your page has only one <title>tag. </span><em>"Web pages are meant to have a single title, and for both accessibility and search engine optimization reasons, we strongly recommend following this practice"</em></p> <p>Well I'm trying....I am not able to find where og why I have multiple titles in this page?</p> <p>This is a norwegian page, but maybe someone can look through it?</p> <p>http://www.proplantime.no/bransjer/bygganlegg/mannskapsliste</p></title>
On-Page Optimization | | Marked_Proplan0 -
How to Handle duplicate pages/titles in Wordpress
The wordpress blog causes problems with page titles. If you go to the second page of blog posts it there's a different URL but with the same page title. for example: page 1: site/blog page 2: site/blog/page/2 Each page gets flagged for duplicate page titles. Thanks in advance for your thoughts,
On-Page Optimization | | heymarshall1 -
How to check duplicate content with other website?
Hello, I guest that my website may be duplicate contents with other websites. Is this a important factor on SEO? and how to check and fix them? Thanks,
On-Page Optimization | | JohnHuynh1 -
Category Descriptions if Category Pages are Noindexed?
I have noindexed my category pages. Do I still need to write a description for my categories? I am using the plugin SEO Content Control to see what I can spiff up but I don't want to waste my time. Thank you!
On-Page Optimization | | dealblogger0 -
Is there a tool that will "grade" content?
Does anybody know of a tool that can "grade" content for Panda compliance. For example, it might look at: • the total number of words on the page • the average number of words in sentences • grammar • spelling • repetitious words and/or phrases • Readability—using algorithms such as: Flesch Kincaid Reading Ease Flesch Kincaid Grade Level Gunning Fog Score Coleman Liau Index Automated Readability Index (ARI) For the last 5 months I've been writing and rewriting literally 100s of catalog descriptions—adhering to the "no duplicate content" and "adding value" rubrics—but in an extremely informal style. I would like to know if I'm at least meeting Google Panda's minimum standards.
On-Page Optimization | | RScime250 -
Which pages should use rel="canonical" links?
I have many pages showing up as multiple content. Most of the them belong to product pages for my store, login pages that show up everywhere on the site, etc. I know that I need to use the rel=canonical link in the header but after searching the forum I'm still unsure of what pages need it. Is it the pages that I don't want searched/crawled by Google or the other way around? Thanks! Crystal
On-Page Optimization | | COfashionista0 -
How Pandas Define "Thin" content
Many websites like www.geico.com have little content on the homepage, but instead a ton of graphics. I've been told before to watch out for pages/posts less than 200 words, but 95% of websites have "main pages" that are graphically driven and have very very few words. So, if Panda is cracking down on thin content, how does Panda define "thin" with regards to major pages of a site? Thanks!
On-Page Optimization | | terran0