Why Did My Google Crawls Hit A Wall?
-
Hello,
One my the sites I work with, http://www.oransi.com, has seen a significant decrease in crawl Googlebot activity in the last 90 days. See screenshot.
This decrease in crawl stats runs in conjunction with less Kb downloaded per day & an increase in how much time it took Google to download a page.
The client did just go through a redesign, however that happened on 4/16/15, which was after the decrease in Googlebot activity, so that should not be the issue. Same could be said for the mobilegeddan algorithm change.
Any help would be greatly appreciated.
-
Hi BrandLabs,
Thanks for your question! You've had several community members respond, and they've asked some important questions. Did you see them? Please update, thanks!
Christy
-
Did they recently try and do some 301 redirects to point all non-www pages to the www version? It looks like the redirects may have been setup wrong and they're going to 404 pages instead of redirecting. There are a handful of other broken links on the site as well that could be contributing to the decline in Googlebot crawling. That's where I would start.
-
Well it's clear that something has happened.
Was any work carried out prior to to drop? Are you seeing any increase in 404's or problems in Google Search Console (Webmaster Tools after the daft re-brand)?
-Andy
-
Has there been any changes to website traffic or more specifically organic traffic over this same period?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is my content being fully read by Google?
Hi mozzers, I wanted to ask you a quick question regarding Google's crawlability of webpages. We just launched a series of content pieces but I believe there's an issue.
Intermediate & Advanced SEO | | TyEl
Based on what I am seeing when I inspect the URL it looks like Google is only able to see a few titles and internal links. For instance, when I inspect one of the URLs on GSC this is the screenshot I am seeing: image.pngWhen I perform the "cache:" I barely see any content**:** image.pngVS one of our blog post image.png Would you agree with me there's a problem here? Is this related to the heavy use of JS? If so somehow I wasn't able to detect this on any of the crawling tools? Thanks!0 -
Client has an inexplicable jump in crawled pages being reported in Google Search Console
Recently a client of mine noticed an inexplicable jump in crawled pages being reported in Google Search Console. We researched the following culprits and found nothing: Rel=canonicals are put in place No SSL/non SSL duplication We used a tool to extrapolate search query page data from Google Search Insights; nothing unusual No dynamic pages being made on the website All necessary landing pages are in the XML sitemap Could this be a glitch in GSC? We are wondering what the heck is going on. 7eaeS
Intermediate & Advanced SEO | | BigChad20 -
Google Webmaster tools -Fixing over 20,000+ crawl errors
Hi, I'm trying to gather all the 404 crawl errors on my website after a recent hacking that I've been trying to rectify and clean up. Webmaster tools states that I have over 20 000+ crawl errors. I can only download a sample of 1000 errors. Is there any way to get the full list instead of correcting 1000 errors, marking them as fixed and waiting for the next batch of 1000 errors to be listed in Webmaster tools? The current method is quite timely and I want to take care of all errors in one shot instead of over a course of a month.
Intermediate & Advanced SEO | | FPK0 -
Is it possible to rank a RE-DIRECT in Google ?
Hi Guys Some help here please would be much appreciated. I have created a super URL re-direct to my Amazon product page. So my question is If I spend time and money build SEO links to my re-direct URL will it still rank high in Google or does Google NOT like or rank re-direct URLs ??? Thanks Gareth
Intermediate & Advanced SEO | | GAZ090 -
Google can't access/crawl my site!
Hi I'm dealing with this problem for a few days. In fact i didn't realize it was this serious until today when i saw most of my site "de-indexed" and losing most of the rankings. [URL Errors: 1st photo] 8/21/14 there were only 42 errors but in 8/22/14 this number went to 272 and it just keeps going up. The site i'm talking about is gazetaexpress.com (media news, custom cms) with lot's of pages. After i did some research i came to the conclusion that the problem is to the firewall, who might have blocked google bots from accessing the site. But the server administrator is saying that this isn't true and no google bots have been blocked. Also when i go to WMT, and try to Fetch as Google the site, this is what i get: [Fetch as Google: 2nd photo] From more than 60 tries, 2-3 times it showed Complete (and this only to homepage, never to articles). What can be the problem? Can i get Google to crawl properly my site and is there a chance that i will lose my previous rankings? Thanks a lot
Intermediate & Advanced SEO | | granitgash
Granit FvhvDVR.png dKx3m1O.png0 -
Not ranking in Google - why???
This will be a bit long, so please bare with me. I have a client in the auto parts industry who wants to rank their homepage for 13 different keywords. We are ranked first page for all keywords in Yahoo! Mexico and Bing Mexico, but not ranking first page at all in Google Mexico. My client's competitor, however, is clearly outranking my client in Google. When comparing both pages, my client's, while not 100% optimized, looks better optimized than their competitor's. Looking at all metrics using Moz, SEMRush, ahrefs, etc... my client's site looks MUCH better on all fronts. I know ranking a single homepage for more than 10 keywords is a difficult task. Our competitor is however, ranking for them, so it's not impossible. The keywords are not even that competitive according to Moz's analysis. I decided to create an optimized page for each keyword to try to rank these pages, but still my client wants the homepage to rank (again, if the competitor is ranking, then it's possible to do this) and I am afraid these pages I created could result in keyword cannibalization ultimately affecting the homepage's possibility to rank. My client had a previous SEO agency working for them and basically all they did was create fake blogs and have lots of keyword rich links directed to the site's homepage. I got the complete link profile from several tools and submitted a disavow requests for as many fishy links I could find, but that hasn't shown any results so far. Note: when looking at the competitor link profile, they have basically just a few links and no external links of real value whatsoever. My client is obviously very frustrated, and so am I. In my SEO experience, it shouldn't be such a difficult task to accomplish, however nothing seems to work even though everything seems to point that my client should rank higher. So now I'm running out of ideas regarding what to do with this site. Any insight you could provide would be SO helpful to me and my client. If needed I can provide my client's homepage URL and also their competitors homepage for you to review. i can also give you any extra information you need. Thanks a lot!
Intermediate & Advanced SEO | | EduardoRuiz0 -
Site not indexed in Google UK
This site was moved to a new host by the client a month back and is still not indexed in Google UK if you search for the site directly. www.loftconversionswestsussex.com Webmaster tools shows that 55 pages have been crawled and no errors have been detected. The client also tried the "Fetch as Google Bot" tactic in GWT as well as running a PPC campaign and the site is still not appearing in Google. Any thoughts please? Cheers, SEO5..
Intermediate & Advanced SEO | | SEO5Team0 -
Does Google read code as is or as rendered?
Question - Does Google read code as is or as rendered? So for example, with a Facebook Like box, it has all the profile pictures of people...will Google see these as all separate links or ignore them?
Intermediate & Advanced SEO | | jhinchcliffe0