Google crawl rate almost zero since re-launch, organic search up 50% though!
-
We're confused as to why Google's crawl of our site has dropped hugely since our new site went live. The URLs of almost all pages changed, and were 301d to the new site. About 20% of our pages were blocked by robots.txt for the re-launch. The re-launch has been great for organic search, with hits up about 50%. Yet our new content is taking a lot longer to get indexed than before. Our KB downloaded a day according to webmaster tools are well down, as is time spent downloading a page. Any ideas as to why this is?i7hwX.png
-
Hi David,
First, my preference would always be to use 301 redirect to send pages that no longer exist to a new URL.
Using a 301 passes link juice and signals search engines that the old page should be de-indexed and the new page indexed in its place.
Barring some issue with your rel=canonical tags, which would be the first place to look, I would check other things that can send signals to the search engines about freshness:
- meta refresh tags
- update frequency reported in your sitemap
- custom crawl rate setting in Webmaster Tools
Then of course, there is an analysis of the differences between your old design and the new...have you made a significant change to your menu(s) and/or internal linking structure which has made it harder for crawlers to follow links to all pages in your site?
Hope that helps,
Sha
-
It's the same domain but we've rebuilt the website and improved the url and content structure.
We've consolidated urls (e.g. redirecting to canonical url) and published the update xml sitemap.
Pages are crawled correctly, we just can't figure out why the graph changed like that.
-
Hi David,
Just a few questions to help understand your situation:
- When you say "new site" do you mean that you have changed to a new domain?
If it is a new domain:
- Did you 301 individual pages to relevant pages on the new domain, or do you have a catch-all sending all traffic to home?
- Is the chart you uploaded from the new Webmaster Tools account (for the new domain)?
- Have you created and uploaded xml sitemap(s) for the new site?
If it is not a new domain:
- Have you created and uploaded xml sitemap(s) for the new version of the site?
- Have you checked the crawl rate setting in the Webmaster Tools account in case it has been manually altered?
These answers will give us a clearer idea of where to start.
Thanks,
Sha
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Manual Action - When requesting links be removed, how important to Google is the address you're sending the requests from?
We're starting a campaign to get rid of a bunch of links, and then submitting a disavow report to Google, to get rid of a manual action. My SEO vendor said he needs an @email domain from the website in question @travelexinsurance.com, to send and receive emails from vendors. He said Google won't consider the correspondence to and from webmasters if sent from a domain that is not the one with the manual action penalty. Due to company/compliance rules, I can't allow a vendor not in our building to have an email address like that. I've seen other people mention they just used a GMAIL.com account. Or we could use a similar domain such as @travelexinsurancefyi.com. My question, how critical is it that the domain the correspondence with the webmasters be from the exact website domain?
Technical SEO | | Patrick_G0 -
Google not indexing my website
Hi guys, We have this website http://www.m-health-expo.nl/ but it is not indexed by google. In webmaster tools google says that it can not fetch the site due to the robots.txt but i do not see any faults in it. http://www.m-health-expo.nl/robots.txt Do you see something strange, it really bothers me.
Technical SEO | | RuudHeijnen0 -
Why are Google search results different if you are log'd into Google or not?
I get different results when I'm log'd into my Google account associated with my website than if I'm not. The same country is occurring. So how can I rely on the google results I'm seeing? For instance my site is page 1 with the improvements I made based on SEOMOZ if I'm log'd in. Yet I'm not on the first 25 pages if I'm not logged in.
Technical SEO | | Romana0 -
How to re-claim Google Places listing?
I have 2 new clients whose Google Places pages were claimed & edited by other SEO companies. No info is available to me on the Google Accounts used to claim them. Both Places listings need significant edits. Is there a way to re-claim these listings with new Google Accounts, in order to edit/optimize them? Or, do I have to create new Places listings and then ask Google to delete the old ones? This situation must be occurring every day, it seems....
Technical SEO | | DonB1 -
Why is Google stripping/replacing my TITLE tag for the site with the BRAND Name only when looking at BRAND level search
When doing a search in Google (US Proxy) - Google is stripping and replacing my functional TITLE with the brand name only (say 'Nike'), but if you do a specific search term like ('buy nike shoes') and see a top 10 listing for my site's homepage, now the title works and shows correctly. I saw this a few years ago with another one of my company domains, but didn't ask the question as it worked out. Thanks for any insight.. NOTE: It's not damaging any results, or rankings for the site.. but: when searching for BRAND name of the company, like I explained, it's replacing a optimized title for the BRAND name, and then re-placing it naturally when deep search brings up the homepage and the TITLE looks fine.. Very weird at best! Thanks, Rob
Technical SEO | | RobMay0 -
What Google uses in search result descriptions
Recently, Google has started including certain information from our web pages in their search results description that is a bit puzzling. For example if you google 'Wedding Band Raleigh' the description they are using for our site's (GigMasters) page begins with the text 'Results 1 - 10 of 1005' Not sure why they are pulling that information. That is in on the page but its not high up on the page or marked with any special h1, h2, or h3 tag. We do have that information inside of a div which we have named 'Results'. Maybe that's why? Did we inadvertently use some sort of Google rich snippet or schema.org naming convention?! Any insight would be hugely appreciated.
Technical SEO | | gigmasters0 -
Crawl report showing only 1 crawled page
Hi, I´m really new to this and have just setup some Campaigns. I have setup a Campaign for the root domain: portaldeldiablo.com.uy which returned only 2 crawled pages.. As this page had a 301 redirect from the non-www to the www version, I deleted this Campaign and setup a new one for www.portaldeldiablo.com.uy which returned only 1 crawled page.. I really don´t know why is my website not being crawled..Thanks in advance for your help.
Technical SEO | | ceci27100 -
Title Tag for homepage Not displaying correctly in Google Search Results
Hi We have a problem with how Google is is displaying our title tag when a user does a search on our company name Increation. I have attached two images of how the title tag for our Homepage is dispayed in Google Search Results. One of them is "increations" - this is the result which is displaying incorrectly. The other shows Kitchens | Bathrooms | | Luxury Kitchens | - which is how we want the title tag to be dispalyed. This inorreclt result is only displayed when a user types in Increation into Google. (BTW it only happens on Google, not the other search engines). To give you some background an external SEO company did some work for us (before I joined as usual :-)). They built a lot of anchor links with the word "Increations" pointing back to our home page. Is this whats causing the probelm? Also how can we resolve this issue so that the correct Title Tag is displayed. Thanks for your help. Mik increation_search_results_google#hyhfZ
Technical SEO | | increation0