Why so many crawl errors?
-
Our site is showing it has a ton of crawl errors in the back end, mostly concerning duplicate content within our blog. The content is unique however. We know this for certain because it's done in house or put together by some of the freelance writers we work with.
The site is for an RV dealership and we're using a template-based system from a well known company.
Any ideas on what may be causing this?
-
Hi Darrin,
Can you provide your website URL or a few examples of the URLs that are getting the duplicate content error?
If you can't provide them, I would recommend referencing http://www.seomoz.org/help/fixing-crawl-diagnostic-issues which says the following about fixing duplicate page content:
"Duplicate Page Content
Duplicate Content means there are pages that are identical (or nearly identical) to content on other pages of your site, which can force your pages to unnecessarily compete with each other for rankings.
Here are some things you can do about duplicate pages:
Delete content that is similar on each page.
Add some new and unique content to each page that is on the report. This can be done by adding more information, ideas, product descriptions, or anything that can make it differ from other pages on the domain.
You can also add a rel=canonical to one of the duplicate pages. Here are a few ways to do this:
Add a rel="canonical" link in between the and elements. This should be done on the version of the page you want to be ranking or that non-canonical version of the two (or multiple) pages.
To specify a canonical link to the page http://www.seomoz.org/blog.php?item=seomoz-iscool, create a element that looks like this: <link rel="canonical"href="<a href="http://www.seomoz.com/blog.php?item=seomoz-iscool">http://www.seomoz.com/blog.php?item=seomoz-iscool"/></link rel="canonical"href="<a>
Copy this link into the section of all non-canonical versions of the page, such as http://www.seomoz.com/blog.php?item=seomoz-iscool&sort=fun.
Keep in mind that that canonicals will stop the pages from ranking against each other, but they will still show up as duplicate content from a UI perspective, so we will still count them as duplicate."
Thanks,
Mike
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How get rid of 403 crawl error?
My wordpress website has 162 crawl 403 errors. Based on what I read it means that the server is denying crawlers to access the pages. The pages itself will load so guessing it's just an issue with crawlers only. How do I go about fixing this issue?
On-Page Optimization | | emrekeserr30 -
406 http error
Hi I am getting about 10 of these errors and when I click on the link it goes to make login wordpress page can anyone help 406 : Received 406 (Not Acceptable) error response for page http://reseolve.com/wp-login.php?redirect_to=http%3A%2F%2Freseolve.com%2Frecent-posts%2F
On-Page Optimization | | ReSEOlve0 -
Getting 404 Errors on Fonts After Website Redesign
Hi, I recently redesigned my website and I changed a few URLs so I have been keeping an eye on 404 errors and setting up redirects as needed. However, my 404 error log keeps getting filled font related URLs like this: <colgroup><col width="960"></colgroup>
On-Page Optimization | | DohenyDrones
| /fonts/AvenirLTStd-Heavy.otf), url(/fonts/AvenirLTStd-Heavy.woff), url(/fonts/AvenirLTStd-Heavy.eot), url(/fonts/AvenirLTStd-Heavy.eot? |
| /fonts/FrutigerLTStd-Black.otf), url(/fonts/FrutigerLTStd-Black.woff), url(/fonts/FrutigerLTStd-Black.eot), url(/fonts/FrutigerLTStd-Black.eot? |
| /fonts/AvenirLTStd-Heavy.otf), url(/fonts/AvenirLTStd-Heavy.woff), url(/fonts/AvenirLTStd-Heavy.eot), url(/fonts/AvenirLTStd-Heavy.eot? |
| /fonts/AvenirLTStd-Book.otf), url(/fonts/AvenirLTStd-Book.woff), url(/fonts/AvenirLTStd-Book.eot), url(/fonts/AvenirLTStd-Book.eot? |
| /fonts/FrutigerLTStd-Bold.otf), url(/fonts/FrutigerLTStd-Bold.woff), url(/fonts/FrutigerLTStd-Bold.eot), url(/fonts/FrutigerLTStd-Bold.eot? |
| /fonts/FrutigerLTStd-Bold.otf), url(/fonts/FrutigerLTStd-Bold.woff), url(/fonts/FrutigerLTStd-Bold.eot), url(/fonts/FrutigerLTStd-Bold.eot? |
| /fonts/FrutigerLTStd-Black.otf), url(/fonts/FrutigerLTStd-Black.woff), url(/fonts/FrutigerLTStd-Black.eot), url(/fonts/FrutigerLTStd-Black.eot? |
| /fonts/AvenirLTStd-Book.otf), url(/fonts/AvenirLTStd-Book.woff), url(/fonts/AvenirLTStd-Book.eot), url(/fonts/AvenirLTStd-Book.eot? |
| /fonts/AvenirLTStd-Heavy.otf), url(/fonts/AvenirLTStd-Heavy.woff), url(/fonts/AvenirLTStd-Heavy.eot), url(/fonts/AvenirLTStd-Heavy.eot? | Any idea what is causing these 404 errors? Is this something that might hurt my SERP?0 -
How to fix Medium Priority Issues by mozpro crawled report??
How to resolve this issues crawled by mozpro?
On-Page Optimization | | renukishor
Some Medium priority issues like that: Missing Meta Description Tag: 2669
Title Element is Too Long: 523
Duplicate Page Title: 37 How to add missing meta description tag in these pages and how to short title element ?0 -
How to deal with 404 errors from deleted categories?
I was reorganizing my blog and deleted several categories. Of course now Google Webmasters is coming up with a bunch of 404 errors. How do I fix this?
On-Page Optimization | | blogger20130 -
Too many page links warning... but each link has canonical back to main page? Is my page OK?
The Moz crawl warns me many of my pages have too many links, like this page http://www.webjobz.com/jobs/industry/Accounting ...... has 269 links but many of the links are like this /jobs/jobtitles/Accounting?k=&w=3&hiddenLocationID=463170&depth=2 and are used to refine search criteria.... when you click on those links they all have a canonical link back to http://www.webjobz.com/jobs/industry/Accounting Is my page being punished for this? Do I have to put "no follow" tags on every link I do not want the bots to follow and if I do so is Roger (moz bot) not going to count this as a link?
On-Page Optimization | | Webjobz0 -
Html and css errors - what do SE spiders do if they come across coding errors? Do they stop crawling the rest of the code below the error
I have a client who uses a template to build their websites (no problem with that) when I ran the site through w3c validator it threw up a number of errors, most of which where minor eg missing close tags and I suggested they fix them before I start their off site SEO campaigns. When I spoke to their web designer about the issues I was told that some of the errors where "just how its done" So if that's the case, but the validator still registers the error, do the SE spiders ignore them and move on, or does it penalize the site in some way?
On-Page Optimization | | pab10 -
Changing Subfolder that has been crawled before
Question: I am using a wordpress multisite and I enabled the crawl options yesterday www.abc.com/subfolder <-original but i find that www.abc.com/sub is good enough I checked the site:abc.com but I find that my pages in the /subfolder has been crawled before. Can I just change it to www.abc.com/sub or it will raise duplicate content issue?
On-Page Optimization | | joony20080