How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Malware, Google Ranking and cleanup
Hi My website was infected with the malware. We have cleaned it and during the process we had to delete the cache. Now website is showing like this on the google. How can i fix this? 3e8e9111-eac7-4238-a691-9678542eb4cf-image.png Secondly, As i have lost all the ranking, how can i get back quickly? Please help Thanks in advance
On-Page Optimization | | fslpso0 -
Google changing my Title
I noticed today that Google is showing a different Title in searches for the homepage. It is showing a title which I believe was active as of late last year or early January. The new title is in the title tag, in Google cache and in MOZ crawls as well. Not sure why it would still be showing a different title in the search and wanted to see if anyone can explain the reason. Thanks
On-Page Optimization | | campaigneast0 -
Google Search Console issue: "This is how Googlebot saw the page" showing part of page being covered up
Hi everyone! Kind of a weird question here but I'll ask and see if anyone else has seen this: In Google Search Console when I do a fetch and render request for a specific site, the fetch and blocked resources all look A-OK. However, in the render, there's a large grey box (background of navigation) that covers up a significant amount of what is on the page. Attaching a screenshot. You can see the text start peeking out below (had to trim for confidentiality reasons). But behind that block of grey IS text. And text that apparently in the fetch part Googlebot does see and can crawl. My question: is this an issue? Should I be concerned about this visual look? Or no? Never have experienced an issue like that. I will say - trying to make a play at a featured snippet and can't seem to have Google display this page's information, despite it being the first result and the query showing a featured snippet of a result #4. I know that it isn't guaranteed for the #1 result but wonder if this has anything to do with why it isn't showing one. VmIqgFB.png
On-Page Optimization | | ChristianMKG0 -
Google Indexing
Hi, We recently launched a new version of our site on the Magento platform. I submitted a new sitemap and on the first crawl only 7 pages out of 132 were indexed...a few days later and we now have 107 indexed (phew). My question is this....how on earth do i find out which pages are indexed and more importantly not indexed? For all i know they might be really important ones so I need to be able to identify the missing pages so i can work on getting them indexed. Nic
On-Page Optimization | | nicc19760 -
Ajax url returns an error by google. Is there another way besides creating a HTML version?
We trying to find out if there is anything to make it so google does not keep returning errors cuase of our ajax urls. Is there any other option besides creating it all in a HTML format for google to read? Any tips or help would be great!
On-Page Optimization | | DoRM0 -
Slash at the end of a url
I keep reading contradicting information, so I figured I'll ask here. What's the best practice for slash '/' at the end of a URL? Should it be idealchooser.com/search/laptop/ or idealchooser.com/search/laptop (no trailing slash)? The options: 1. Accept both equally 2. Accept 1 and redirect the other with 301 3. Accept 1 and treat the other as a wrong URL returning 404 Which would be the best for SEO? Thank you.
On-Page Optimization | | corwin0 -
Which method should I use for my URL structure?
I have an existing site that is currently utilizing a structure that is like this: http://www.mysite.com/Ohio/City-of-Cleveland-PRODUCT-NAME Should I restructure it like: http://www.mysite.com/Ohio/City-of-Cleveland/Product-Name We are doing very well with very specific searches already but are sometimes coming in 2nd and 3rd place. For example: If I search for CLEVELAND PRODUCT NAME I always come up in the top three and about 60% of the time I am #1. I want to make it better. We have only launched in 4 states but plan on launching an additional 4 states over the next few weeks and I want to make sure we are building things properly. Any feedback would be wonderful. As usual, thanks everyone!! -Alex
On-Page Optimization | | dbuckles0 -
Google Reconsideration
Our site fell from grace last July and landed on page five of the Google search results for our primary keyword. For 6 months I tried a number of strategies with no results, including reconfiguring our site based on the SOEmoz on-page grading tool. More recently, after receiving your advice in a Q&A, I took down all of my paid links and submitted a reconsideration request to Google. Interestingly, 3 days later we popped up 20 spots. This left us on the top of page three. Better than page 5, but still not prime time! A few days ago (two weeks after our reconsideration request was submitted) I got a message back in my Webmaster Tools, that they had completed a review of our site - but oddly enough they provided no info on the outcome, positive or negative. And there has been no additional movement in the rankings since I received the message. Was the original 20 point jump the result of the reconsideration request, or just a coincidence? Or, is it possible that they did a review and the results will only occur later during some organic re-indexing process? What do you think?
On-Page Optimization | | JimSkychief0