How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Possible issues with 301 redirecting to a new domain name
Ive got a current domain and after a bit of a rebrand Im considering 301 rediecting the current site to a newly purchased domain. Id redirect each age to idential pages. Am I likely to see any issues. I know this is the recomended way from Google but just wondering how smoothly it works and whether Im likely to see any ranking drops or other problems?
Intermediate & Advanced SEO | | paulfoz16090 -
Adventurous 301 redirection chain
Picture this - if you have a spirit for adventure! Client builds Alpha****Domain.com Then builds a number of backlinks to Alpha****Domain.com Client also creates a number of 301 redirects from several older domains to AlphaDomain.com Client then changes Alpha****Domain.com to Beta****Domain.com They create 301 redirects from Alpha****Domain.com to Beta****Domain.com But then... they 'park' Alpha****Domain.com (ie. no longer accessible)! About one year later, client changes a whole bunch of URLs on Beta****Domain.com without keeping track of changes. Thankfully, the hosting service (Shopify) automatically creates some redirects, but it's more by accident than design! Questions: After step 6 above, are the 301 redirects created in steps 3 and 5 now totally redundant and broken? If AlphaDomain.com no longer exists, surely all redirects to and from this domain are broken? Or can they be recovered? What happens to all the backlinks originally created in step 2? Finally, can anything be done to recover lost URLs in step 7? Yes. What a mess!
Intermediate & Advanced SEO | | muzzmoz0 -
Redirect Plugin: Redirecting or Rewriting?
Hey everybody! It's been a while since off the boards! I am reworking a site and I have been looking into their Redirection Plugin. I personally tend to lean towards just using the .htaccess because, well, why not. However, when looking deeper into the plugin I found myself a little confused with their redirection wording. RewriteRule ^/products/landing-page-october-2015/$ /products/special-education-news-october-2015/ [R=301,L] Is that the same thing as a classic Redirect 301?
Intermediate & Advanced SEO | | HashtagHustler0 -
301 redirects broken - problems - please help!
Hi, I have a bit of an issue... Around a year ago we launched a new company. This company was launched out of a trading style of another company owned by our parent group (the trading style no longer exists). We used a lot of the content from the old trading style website, carefully mapping page-to-page 301 redirects, using the change of address tool in webmaster tools and generally did a good job of it. The reason I know we did a good job is that although we lost some traffic in the month we rebranded, we didn't lose rankings. We have since gained traffic exponentially and have managed to increase our organic traffic by over 200% over the last year. All well and good. However, a mistake has recently occurred whereby the old trading style website domain was deleted from the server for a period of around 2-3 weeks. It has since been reinstated. Since then, although we haven't lost rankings for the keywords we track I can see in webmaster tools that a number of our pages have been deindexed (around 100+). It has been suggested that we put the old homepage back up, and include a link to the XML sitemap to get Google to recrawl the old URLs and reinstate our 301 redirects. I'm OK with this (up to a point - personally I don't think it's an elegant solution) however I always thought you didn't need a link to the xml sitemap from the website and that the crawlers should just find it? Our current plan is not to put the homepage up exactly as it was (I don't believe this would make good business sense given that the company no longer exists), but to make it live with an explanation that the website has moved to a different domain with a big old button pointing to the new site. I'm wondering if we also need a button to the xml sitemap or not? I know I can put a sitemap link in the robots file, but I wonder if that would be enough for Google to find it? Any insights would be greatly appreciated. Thank you, Amelia
Intermediate & Advanced SEO | | CommT0 -
Is it ok to 301 redirect this previously algorithmicly penalised site?
Hi All, Is it OK to 301 redirect site A to site B? Site A: http://goo.gl/P9Zp2y Site B: http://goo.gl/ySDCzb The story - in 2013 site a seemed to be penalised with some kind of anchor text algorithm penalty - SEO couldnt fix, so created site B and turned site A into a holding page with a no follow link to new site. SEO company worked on disavow file etc, implemented in late 2013 301 redirect site A to B in late 2013 - SEO advised to stop 301 about 8 weeks later... This was my fault i didnt realise the implications of a redirect... Stopped the redirect, but too late, as site B dropped in rankings in early 2014 - new disavow files uploaded to both sites, but damage seems done now. No longer have a SEO company, and i would ideally like to 301 redirect site A to B, as it looks messy having a holding page - but wanted to check if SEO would still strongly advise against that? please advise James
Intermediate & Advanced SEO | | isntworkdull0 -
Shall I use a 301 or 302 redirect when people leave the company?
Hello, At my company, we have instances where client-facing people leave the company and so we need to remove their profile page from the website. As opposed to people receiving a 404 when they search for them, I thought it would be best to divert visitors to a generic landing page to explain that the person they are looking for has left the company with details on how to get in touch. I'm tempted to use a 302 redirect so the person they are searching for stays in the search results longer. But longer-term, will this cause any harm? Should it be eventually be turned into a 301 redirect? Or should I just use a 301 in the first instance. Thanks in advance, Stu
Intermediate & Advanced SEO | | Stuart260 -
Lost all ranking after site-wide 301 redirect
Hi all I did a complete site-wide 310 redirect about 3 weeks ago for a site that had consistently been in Pos 1-5 for my targeted keyword ("low glycemic foods"). I changed the domain from low-glycemic-foods-org to low-glycemic-diet.com because I thought that was a more appropriate title and thru my readings I believed that if I carefully followed the recommended procedures I would quickly regain my SERP. Webmaster tools is showing that I have over 800 inbound links - many from very trustworthy sources including .edu, etc BUT my home page is nowhere to be found for the keyword search "low glycemic diet". My Seomoz onpage SEO score is an "A" Any enlightenment would be much appreciated.
Intermediate & Advanced SEO | | veezer0 -
Can penalties be passed via 301 redirect?
I have a well established domain that's been hit with some penalties. It hasn't been nuked off the map, just downgraded, especially on short-tail, one word type queries. I'm planning on redirecting this domain to another well established domain. The domains already have a history of lots of interlinking and are very similar from a subject matter standpoint. I feel that the penalized domain has been hit with an "over-optimization" of link anchor text penalty (I'm hoping it's algorithmic, but it could be manual). My question is if anyone has ever heard of a penalty like this being transferred to another domain through a 301 redirect. My hope is that the penalty just puts a cap on how much juice the redirect can pass, rather than transferring the penalty to the other domain itself. Any thoughts on this?
Intermediate & Advanced SEO | | SEOMG1