Crawl Diagnostics Update
-
I have corrected some errors in my SEOMoz Crawl Diagnostics, however the errors are still showing. It says a crawl has happen since. Any idea's why?
-
You can do an on-demand crawl test of up to 3000 URLs at http://pro.seomoz.org/tools/crawl-test, which might help.
-
Hi Brad
Can I ask what applications you use and would you recommend them?
Thanks
Pete
-
I agree...is there really no way to crawl before the next scheduled crawl date? With other applications, I am able to do this type of thing whenever I want.
-
Thanks for the reply. It says my next crawl starts: May 14th, 2012. Can I make it happen any earlier or do I need to wait?
-
Hey Pete,
Sorry about the confusion. The crawls kick off a little earlier (3 days before) your actual crawl date, so that may the reason why. It is hard to say without specific dates.
Best,
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rookie question re Moz Crawl errors after deleting a property from console.
Hi all, I stupidly removed the "http" url of my one website a few days back (it is one of three, the other two being the https), then re-added it around a day later and, while google console isn't reporting back any errors, Moz Crawl is going to town on this now for one critical "4xx" issues, canonicals and various other content issues that I addressed days previously...last Moz crawl performed an hour ago, url deleted and re-added two days ago. I have resubmitted a sitemap, will this smooth itself out or shall I go and make changes? Many thanks in advance.
Moz Pro | | UkPod0 -
Block Moz (or any other robot) from crawling pages with specific URLs
Hello! Moz reports that my site has around 380 duplicate page content. Most of them come from dynamic generated URLs that have some specific parameters. I have sorted this out for Google in webmaster tools (the new Google Search Console) by blocking the pages with these parameters. However, Moz is still reporting the same amount of duplicate content pages and, to stop it, I know I must use robots.txt. The trick is that, I don't want to block every page, but just the pages with specific parameters. I want to do this because among these 380 pages there are some other pages with no parameters (or different parameters) that I need to take care of. Basically, I need to clean this list to be able to use the feature properly in the future. I have read through Moz forums and found a few topics related to this, but there is no clear answer on how to block only pages with specific URLs. Therefore, I have done my research and come up with these lines for robots.txt: User-agent: dotbot
Moz Pro | | Blacktie
Disallow: /*numberOfStars=0 User-agent: rogerbot
Disallow: /*numberOfStars=0 My questions: 1. Are the above lines correct and would block Moz (dotbot and rogerbot) from crawling only pages that have numberOfStars=0 parameter in their URLs, leaving other pages intact? 2. Do I need to have an empty line between the two groups? (I mean between "Disallow: /*numberOfStars=0" and "User-agent: rogerbot")? (or does it even matter?) I think this would help many people as there is no clear answer on how to block crawling only pages with specific URLs. Moreover, this should be valid for any robot out there. Thank you for your help!0 -
Crawl report - duplicate page title/content issue
When the crawl report is finished, it is saying that there are duplicate content/page titles issues. However there is a canonical tag that is formatted correctly so just wondered if this was a bug or if anyone else was having the same issues? For example, I'm getting a error warning for this page http://www.thegreatgiftcompany.com/categories/categories_travel?sort=name_asc&searchterm=&page=1&layout=table
Moz Pro | | KarlBantleman0 -
Crawl Diagnostics - unexpected results
I received my first Crawl Diagnostics report last night on my dynamic ecommerce site. It showed errors on generated URLs which simply are not produced anywhere when running on my live site. Only when running on my local development server. It appears that the Crawler doesn't think that it's running on the live site. For example http://www.nordichouse.co.uk/candlestick-centrepiece-p-1140.html will go to a Product Not Found page, and therefore Duplicate Content errors are produced. Running http://www.nhlocal.co.uk/candlestick-centrepiece-p-1140.html produces the correct product page and not a Product Not Found page Any thoughts?
Moz Pro | | nordichouse0 -
In my crawl diagnostics, there are links to duplicate content. How can I track down where these links originated in?
How can I find out how SEOMOz found these links to begin with? That would help fix the issue. Where's the source page where the link was first encountered listed at?
Moz Pro | | kirklandsl0 -
Set crawl frequency
Current crawl frequency is weekly, is it possible for me to set this frequency our-self?
Moz Pro | | bhanu22170 -
Domain authority decrease after open site explorer update. Reasons?
Hi! I just noticed a decrease on our domain authority and also the page authority of several of our pages on the SeoMoz toolbar. For instance, DA went from 34 to 29. I was scared and confused, so I went to check it on the opensite explorer and saw that it had been updated. I want to ask you guys if the same happened with your metrics or is it just us. Because the strange thing about it is that we have been doing some good optimization work lately and also some linkbuilding, and actually our SERP’s have been improving in general, and some specific ones have been performing great. Our website is http://www.inmonova.com as a reference. Can anyone cast some light into this?
Moz Pro | | inmonova0