How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Relaunching website in two phases - 301 redirect approach
Hey Mozzers, Interested to know your thoughts on the following situation. I am relaunching a website with an updated URL structure in TWO phases. Phase one will be a much smaller version of the site, with 30% of the pages going live - the remaining 70% of page won't be available until Phase two. In Phase one, these 30% of pages will be 301 redirected from their like-for-like versions - old site to new site. The remaining 70%... because the like-for-like pages won't be available until Phase two, which is likely to be launched in 3 months time, should I do a temporary redirect on these pages (302) to the new homepage for the time being, until the new versions of the pages are live - and then implement the 301 from old url to new url. A bit of a messy situation, and not ideal for SEO, but my hands are tied as the organisation is pushing ahead with this phased approach. So, interested to hear your thoughts on an appropriate 301 migration plan.
Intermediate & Advanced SEO | | RWesley0 -
If my website uses CDN does thousands of 301 redirect can harm the website performance?
Hi, If my website uses CDN does thousands of 301 redirect can harm the website performance? Thanks Roy
Intermediate & Advanced SEO | | kadut1 -
Huge organic drop following new site go live
Hi Guys, I am currently working on a site that's organic traffic suffered ( and is still suffering ) a huge drop in organic traffic. From a consistent 3-400 organic visits a day to almost zero. This happened as soon as the new site went live. I am now digging to find out why. 301s were put in place ( over 2, 500 over them ) and there are still over 1,100 outstanding after review search console this morning. Having looked at the redirect file that was put in place when the new site went live, it all look OK, apart from the redirects look like this... http://www.physiotherapystore.com/ to http://physiotherapystore.com/ Where the new URL is missing www. - I am concerned this is causing a large duplicate issue as both www. and non www. work fine. I am right to have concern or is this something not to worry about?
Intermediate & Advanced SEO | | HappyJackJr0 -
Ecommerce catalog update: 301 redirects?
Hello mozers, We run an ecommerce store and are planning a massive catalog update this month. Essentially, 100% of our product listings will be deleted, and an all new catalog will be uploaded. The new catalog contains mostly new products, however there are some products that already existing in the old catalog as well. The new catalog has a bunch of improvements to the product pages, included optimized meta titles and descriptions, multiple language, optimized URLs and more. My question is the following: When we delete the existing catalog, all indexed URLs will return 404 errors. Setting up 301 redirects from old to new products (for products which existing previously) is not feasible given the number of products. Also, many products are simply being remove entirely. So should we go ahead and delete all products, upload the new catalog, update the sitemap, resubmit it for crawling, and live with a bunch of 404 errors until these URLs get dropped from Google? The alternative I see is setting 301 redirects to the home page, but I am not sure this would be correct use of 301 redirects. Thanks for your input.
Intermediate & Advanced SEO | | yacpro130 -
Hundreds of 301 Redirects. Remove Pages or Not?
Hi Mozers, I have a website that has literally got hundreds of 301 redirects. I had a close look at these URLs and only some of them have backlinks to it and remaining all of them are not indexing in Google and has got not backlinks at all. Based on what I have noticed experts mentioning, loads of 301 redirects can potentially slow down the site speed. In a case like the website I have, should I completely take off the pages from website to reduce the number of 301 redirects or should I leave 301 redirects? There is no traffic or backlinks coming from these URLs. Malika
Intermediate & Advanced SEO | | Malika10 -
What to do about similar product pages on major retail site
Hi all, I have a dilemma and I'm hoping the community can guide me in the right direction. We're working with a major retailer on launching a local deals section of their website (what I'll call the "local site"). The company has 55 million products for one brand, and 37 million for another. The main site (I'll call it the ".com version") is fairly well SEO'd with flat architecture, clean URLs, microdata, canonical tag, good product descriptions, etc. If you were looking for a refrigerator, you would use the faceted navigation and go from department > category > sub-category > product detail page. The local site's purpose is to "localize" all of the store inventory and have weekly offers and pricing specials. We will use a similar architecture as .com, except it will be under a /local/city-state/... sub-folder. Ideally, if you're looking for a refrigerator in San Antonio, Texas, then the local page should prove to be more relevant than the .com generic refrigerator pages. (the local pages have the addresses of all local stores in the footer and use the location microdata as well - the difference will be the prices.) MY QUESTION IS THIS: If we pull the exact same product pages/descriptions from the .com database for use in the local site, are we creating a duplicate content problem that will hurt the rest of the site? I don't think I can canonicalize to the .com generic product page - I actually want those local pages to show up at the top. Obviously, we don't want to copy product descriptions across root domains, but how is it handled across the SAME root domain? Ideally, it would be great if we had a listing from both the .com and the /local pages in the SERPs. What do you all think? Ryan
Intermediate & Advanced SEO | | RyanKelly0 -
Redirecting a redirect - thoughts?
Hi! A client has just had 14k 404s pop up in his WMT. I think this is because a page that they redirected to had moved. My question is, can I clean these up by redirecting the page the original redirect was one? If so, will it have any negative impact?
Intermediate & Advanced SEO | | neooptic0 -
Choose of destination for a 301 redirection
Hi, I had a website paris-football.com which ranked quite well on specific request as 'paris football" and "paris foot". I decided 2 months ago to stop this website as I had no time to update it and it was quite rubish in terms of content and make a redirection to a better quality website. I decided to redirect to the deep url http://www.sportytrader.com/paris-foot.php . The destination Url has not beneft from the redirection and has even seen its rankings drop since the redirection. do you think that it would have been better to redirect to the Home Page http://www.sportytrader.com ? Do you think that I can still change the destination url ? Thanks a lot for your help,
Intermediate & Advanced SEO | | jarnac0