How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Google penalize you for reindexing multiple URLS?
Hello, Just a quick, question! I was wanting to know if multiple page indexing (site overhaul) could cause a drop in organic traffic ranking or be penalized by Google for submitting multiple pages at one time. Thanks
On-Page Optimization | | InternetRep0 -
Url structure
Hi Guys, Wondering what is better for url structure say for example a key word "slow cooker" example.com/slowcooker or example.com/slow-cooker ? Thank you 🙂
On-Page Optimization | | GetApp0 -
I have more pages in my site map being blocked by the robot file than I have being allowed to be crawled. Is Google going to hate me for this?
Using some rules to block all pages which start with "copy-of" on my website because people have a bad habit of duplicating new product listings to create our refurbished, surplus etc. listings for those products. To avoid Google seeing these as duplicate pages I've blocked them in the robot file, but of course they are still automatically generated in our sitemap. How bad is this?
On-Page Optimization | | absoauto0 -
Title tag and URL Optimization
Hello guys, Should the URL reflect the structure of the title of a webpage? This is the old title with the Url: 20mm O/D Black Polypropylene LSZH Flexible Conduit 100m Coil /Product/20mm-o-d-black-polypropylene-lszh-conduit-100m-coil/1352 I changed the keyword position and it looks like this: 20mm Flexible Conduit | O/D Black Polypropylene LSZH | 100m I kept the same Url for now, should I change that too? Thanks
On-Page Optimization | | PremioOscar0 -
Only homepage in Google
Hello SEOmozzers, A colleague of mine started up a site here: www.wikiboedel.nl. Sadly only the homepage is indexed in Google. It's a wordpress install. Robots.txt seems fine, all pages are on index and follow. Can't see why only the home is indexed in Google. Do you know whats going wrong? Please tell me. Tank you.
On-Page Optimization | | B.Great0 -
Errors in URL´s
SEOMOZ is showing quite a lot of URL Errors like this: http://trampoliny.net.pl/akcesoria/pokrowiec-basic?frontend=1825cb1eea3af8ee6ee2d96617d32ff6 All these URL´s use the parameter "?frontend=". In webmaster tools we told google not to index this parameter. Unfortunately at the moment we cannot set this parameter as "NOINDEX". We also dont want to use a robots.txt file. How to get rid of the URLS in Seomoz?
On-Page Optimization | | drgoodcat0 -
Google Results Title Tag HELP
Can anybody tell us why Google changes your title tag in the SERP? If you check out the below link or type in 'days inn', you will see the 2nd result for www.daysinnrc.co.uk just says 'Days Inn' but on the actual site the title tag for this page is 'Days Inn UK | Days Inn | Daysinnrc.co.uk' http://www.google.co.uk/#hl=en&sclient=psy-ab&q=days+inn&oq=days+inn&gs_l=hp.3...4110.4110.4.4297.1.1.0.0.0.0.0.0..0.0...0.0...1c.1.kWVC24EnCHE&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=7680231318a44bb0&bpcl=35466521&biw=1920&bih=934 This has happened with another site too, does anybody know why? Thanks
On-Page Optimization | | SEOwins0 -
Canonical URL problem
On page analysis wanted me to add a canonical url tag. However I added then re ran the on page analysis and it came up with an error. What is the proper way to add a canonical url tag in the head of an index page? ie. add a canonical tag to www.hompeage.com/index.html would it be ? Or should I ignore this for a home page? Because I add it then run the analysis again and get this? Appropriate Use of Rel Canonical Moderate fix <dl> <dt>Canonical URL</dt> <dd>"http://www.ensoplastics.com/index.html"</dd> <dt>Explanation</dt> <dd>If the canonical tag is pointing to a different URL, engines will not count this page as the reference resource and thus, it won't have an opportunity to rank. Make sure you're targeting the right page (if this isn't it, you can reset the target above) and then change the canonical tag to reference that URL.</dd> <dt>Recommendation</dt> <dd>We check to make sure that IF you use canonical URL tags, it points to the right page. If the canonical tag points to a different URL, engines will not count this page as the reference resource and thus, it won't have an opportunity to rank. If you've not made this page the rel=canonical target, change the reference to this URL. NOTE: For pages not employing canonical URL tags, this factor does not apply.</dd> <dd>So do I add it or not? If I don't I get a lower page rating if I take it off I get a higher page rating with room for improvement. </dd> </dl>
On-Page Optimization | | ENSO0