Why is my crawl STILL in progress?
-
I'm a bit new here, but we've had a few crawls done already. They are always finished by Wednesday night.
Our website is not large (by any means), but the crawl still says it's in progress now 3 days later.
What's the deal here?!?
-
Hi there!
Thanks for writing in! I checked your account it seems that the crawl was finished. Various factors sometimes can affect the speed at which Roger will check out your site, it usually tries to be as efficient as possible and not take the scenic route, every once in a while he gets stuck trying to resolve something on the site (crossword puzzle maybe) which can cause the crawl to last a little longer. We're constantly keeping track of how long the crawls take for our customers and try to come up with a better more efficient Roger!
If your next crawl is taking more than a day (24 hours), i would send us a ticket at [email protected]. Thanks again for writing in!
Best,
Peter
SEOmoz Help Team. -
Cool, thanks Andrea. Glad it's not just me...
-
I've had mine take several days and Moz has told me that's not unexpected (even if it hasn't happened to you before).
If it goes longer than let's say today, contact their help desk. It's possible something got stuck (it's happened to me) and they have a great help team who will get you taken care of. They are very responsible and approachable.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Its been over a month, rogerbot hasn't crawled the entire website yet. Any ideas?
Rogerbot has stopped crawling the website at 308 pages past week and has not crawled the website with over 1000+ pages. Any ideas on what I can do to get this fixed & crawling?
Moz Pro | | TejaswiNaidu0 -
Order of urls in SEOMoz crawl report
Is there any rhyme or reason to the order of urls in the SEOMoz crawl report, or are the urls just listed in random order?
Moz Pro | | LynnMarie0 -
How does SEOMoz crawl sites? Does it follow the sitemap?
I only ask because in my report it returned a few "no follow" links. Was wondering how SEOMoz operates.
Moz Pro | | Pikimal0 -
Does the SEOMoz weekly crawl that highlights no meta description tag, take into account if there is a meta robots noindex,follow tag on the pages it indicates the missing meta descriptions?
The weekly crawl website report is telling me that there are pages that have missing meta description tags, yet I've implemented meta robots tags to 'noindex, follow' those pages which are visible in those page source files. As far as Google Is concerned, surely this then won't be a problem since it is being instructed NOT to consider these specific pages for indexing. I am assuming that the weekly SEOmoz website crawl is simply throwing the missing meta description crawl findings into its report without itself observing that the particluar URL references contain the meta robots 'noindex,follow' tag ???? Appreciate if you can clairfy if this is the case. It would help me understand that (at least in terms of my efforts towards Google) your own crawl doesn't observe the meta robots tag instruction, hence the resultant report's flagging the discrepancy.
Moz Pro | | callassist0 -
Bad on page optimization grade, still displays high in SERP
I'm looking at the keyword on page optimization report for my site and I have keywords where my landing page is showing up first on the SERP, but my SEOmoz on-page optimization grade for them is an F. Am I ranking high for these terms because no one is optimized for the terms? They are almost all directed at my home page, does domain authority play a role in me ranking high for them? Thanks for any insight.
Moz Pro | | ClaytonKendall0 -
On the Crawl Diagnostics Summary, its reporting over 100 "Title Missing or Empty" issues, but they all check out fine?
Wondering if there Is a bug with the crawler or known timeout issues? Site speed is fast, but we do run a couple of large cron jobs out of hours, which may be the cause of any timeouts, but shouldn't the crawler report that, rather saying no title tags on 100 pages, when there are? SEOmoz newbie, so still finding my feet 🙂
Moz Pro | | sjr4x40 -
Does it really take 3-7 days to crawl a site?
This is what you say on your website: Please note: depending on the size and speed of your site, it may take between 3 and 7 days to complete your crawl. Are you guys kidding? This is unacceptable for almost all of my deadlines, and I suspect 99% of the SEO world... you can fly to the moon faster.
Moz Pro | | rldonnell0