How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 redirects cross domains
Hi Moz Community. We have a client that has Website A and Website B. Website A is going to be replaced by Website C, a new website and brand. Some products sold on Website A are going to be split out to Website B & C. i.e. Say Website A sells eight products - then four will go to Website B and four to Website C. OUR QUESTION Technically we know we can 301 redirect the Website A products to the relevant Website B & Website C products. 1. Given this convoluted structure, will there be any negative ramifications for SEO? and; 2. Which website would you redirect the homepage to, B or C?
Intermediate & Advanced SEO | | WCR0 -
301 Redirect Showing Up as Thousands Of Backlinks?
Hi Everyone, I'm currently doing quite a large back link audit on my company's website and there's one thing that's bugging me. Our website used to be split into two domains for separate areas of the business but since we have merged them together into one domain and have 301 redirected the old domain the the main one. But now, both GWT and Majestic are telling me that I've got 12,000 backlinks from that domain? This domain didn't even have 12,000 pages when it was live and I only did specific 301 redirects (ie. for specific URL's and not an overall domain level 301 redirect) for about 50 of the URL's with all the rest being redirected to the homepage. Therefore I'm quite confused about why its showing up as so many backlinks - Old redirects I've done don't usually show as a backlink at all. UPDATE: I've got some more info on the specific back links. But now my question is - is having this many backlinks/redirects from a single domain going to be viewed negatively in Google's eyes? I'm currently doing a reconsideration request and would look to try and fix this issue if having so many backlinks from a single domain would be against Google's guidelines. Does anybody have any ideas? Probably somthing very obvious. Thanks! Sam
Intermediate & Advanced SEO | | Sandicliffe0 -
New site started ranking, lost ground after 301...?
Hi everyone. So we decided to re-brand a website starting with a fresh domain and fresh website (new everything). The first 4 weeks the site was performing really well and started bringing organic traffic for several targeted keywords (small amounts, but something). Around 5 weeks later, we decided to perform a 301 Redirect from an older site we had that really hadn't changed since 2007. As soon as we performed the 301, we started loosing ground on the keywords that were starting to perform. I may help to mention that we were targeting different keywords on the new site, versus the keywords/industry that were targeted on the older site... because we were focusing on another but similar industry. Now, 10 weeks later we are still not showing up for the keywords that we were starting to make headway on in the first four weeks of starting.... any ideas why? suggestions? The 301's were performed properly. We verified them, and we check Google WMT for any additional ones that may still be out there, but try to fix them as soon as possible. We have grown the site from just a few pages to over 60+ pages in the last 2 months with unique and fresh content targetting those keywords... Thank you in advance
Intermediate & Advanced SEO | | co.mc0 -
Effect SERP's internal 301 redirects?
I'm considering installing Wordpress for my website. So I have to change the static URL's from /webpage.html to /webpage/. Yet I don't want to lose in the SERP's. What should I expect?
Intermediate & Advanced SEO | | wellnesswooz1 -
301 entire site
Is there a good 301 code snippet to change just the root domain but keep the ending extensions? I just bid on a domain that I think would be much better for me moving forward, but do not want to have to try going through thousands of pages to do their 301 individually My site is almost 4 yrs old. Well established and has a large fanbase. Several of our social networks are under the name of the new branded domain, hence part of the desire to switch.
Intermediate & Advanced SEO | | Atomicx0 -
301 redirect a old site that has been "dead" for a while?
Hi guys, A quick question. I have a client who has an old business website that had some great links (Forbes.com, CocaCola.com, etc). The problem is that he knew nothing about SEO and let the hosting expire. He still owns the domain, but the site is no longer listed in Google. He did no SEO, so I am not worried about being hit by any artificial anchor text penalties, since the links are as natural as it gets. So my questions is, would there be any benefit from 301 redirecting that site to his new business? The new business is in almost exactly the same niche as the old site. I am thinking of 301'ing to a sub-page which will refer to his past venture with the old business, not to the homepage of the new site. Thanks in advance for your help.
Intermediate & Advanced SEO | | rayvensoft0 -
Are there any negatives to channeling my links through a 301 redirect?
I'm channeling 1000's of links through another url with a 301 redirect. I've thought this through and can't see any downside to doing this, but I want to get your opinion. Can you see any downside to doing this? With regards to passing anchor text, PR, PA, etc? Since this is done with sites all the time when they change urls, I can't see Google being able to penalize me for this....can you? What do you think?
Intermediate & Advanced SEO | | Brian-M0 -
Any way to find which domains are 301 redirected to competitors' websites?
By looking at the work from an SEO collegue it became clear that his weak linkbuilding graph probably is not the cause for his good rankings for a pretty competitive keyword. (also no social mentions where found) I was wondering what it could be, site structure and other on page optimization factors seems to be ok and I don't think there will be exceptionally good or bad user behavior... Finally I looked at the competitors and found that they have more links, better content en better design, so I got a little stuck. The only reason I can think of is that he is doing 301 redirects (or is rel=canonical tags). Is there a way to trace these redirects back to the source in order to include this important variable in your competitor research? thnx
Intermediate & Advanced SEO | | djingel10