Google Dropped 3,000+ Pages due to 301 Moved !! Freaking Out !!
-
We may be the only people stupid enough to accidentally prevent the google bot from indexing our site.
In our htaccess file someone recently wrote the following statement
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301]Its almost funny because it was a rewrite that rewrites back to itself... We found in webmaster tools that the site was not able to be indexed by the google bot due to not detecting the robots.txt file. We didn't have one before as we didn't really have much that needed to be excluded. However we have added one now for kicks really. The robots.txt file though was never the problem with regard to the bot accessing the site.
Rather it was the rewrite statement above that was blocking it. We tested the site not knowing what the deal was so we went under webmaster tools then health and then selected "Fetch as Google" to have the website. This was our way of manually requesting the site be re-indexed so we could see what was happening. After doing so we clicked on status and it provided the following:
HTTP/1.1 301 Moved Permanently
Content-Length: 250
Content-Type: text/html
Location: http://www.mystie.com/
Server: Microsoft-IIS/7.5
MicrosoftOfficeWebServer: 5.0_Pub
MS-Author-Via: MS-FP/4.0
X-Powered-By: ASP.NET
Date: Wed, 22 Aug 2012 02:27:49 GMT
Connection: close<title>301 Moved Permanently</title>
Moved Permanently
The document has moved here.
We changed the screwed up rewrite mistake in the htaccess file that found its way in there but now our issue is that all of our pages have been severely penalized with regard to where they are now ranking compared to just before the indecent.
We are essentially freaking out because we don't know the real time consequences of this and if or how long it will take for the certain pages to regain their prior ranks. Typical pages when down anywhere between 9-40 positions on high volume search terms. So to say the least our company is already discussing the possibilities of fairly large layoffs based on what we anticipate with regard to the drop in traffic. This sucks because this is peoples lives but then again a business must make money and if you sell less you have to cut the overhead and the easiest one is payroll.
I'm on a team with three other people that I work with to keep the SEO side up to snuff as much as we can and we sell high ticket items so the potential effects if Google doesn't restore matters could be significant. My question is what would you guys do? Is there any way we can contact Google about such a matter? If you can I've never seen such a thing. I'm sure the pages that are missing from the index now might make their way back in but what will there rank look like next time and with that type of rewrite has it permanently effected every page site wide, including those that are still in the index but severely effected by the index.
Would love to see things bounce back quick but I don't know what to expect and neither do my counterparts. Thanks for any speculation, suggestions or insights of any kind!!!
-
Interested to know if the indexing and rankings of the pages came back, David?
Paul
-
If every single page was set to 301 back to your home page and subsequently Google has removed all other URLs from the index, then disable the 301's, make sure the pages are working, and resend the sitemap to Google.
Do a scan with link sleuth to confirm all your pages are working correctly.
Greg
-
Thanks for the reply. Actually we were not trying to redirect the site anywhere. That was a redirect that literally redirected mysite.com to the identical mysite.com. Therefore we had no want or need to redirect anything other than internal url's. I think someone may have intended this to be used as a temporary overall site redirect during a maintenance schedule or something so that every page went to the same page but I assuming this was messed up because someone didn't put in the correct redirect code or url for that matter. Hope that makes sense.
So the issue was the google seen thousands of pages that were looking like a 301 redirect to the homepage and even the homepage redirected to the homepage as a permanently moved.
-
Ouch, sorry to hear that. I just added the htaccess code to one of my dev sites and it seems to redirect just fine (200 when visiting www, 301 when visiting non-www). I'm guessing that Google is going to the non-www page and is being redirected to the www version. Maybe switch the code to do the opposite (assuming your goal was to canonical/consolidate the urls). Message me if you want the htaccess code.
For fixing your site rankings, they will most likely bounce back on their own in due time so the best thing to do is try to get G to recrawl all the pages ASAP. I would:
-
Make sure you add and verify both the www and non-www urls of the site in GWT, then choose which one you want to be primary.
-
Change you sitemap to have have frequency/priority crawl rates, have the correct www/non-ww links and resubmit your sitemap in GWT.
-
Share links on twitter (they are crawled very often)
-
Create an rss feed with your pages and ping it
Good luck getting it ranking again.
-Oleg
-
-
First off a 301 redirect takes a few weeks (2-12) to fully work.
Second off you can send google a letter of reconsideration if you feel you were penalized. However I would give the 301 redirect a little boit of time to transfer your link juice. Also when you do a 301 i've heard that only 80% of your link juice will transfer. I like to contact my best links and have them link directly to the new site.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rank dropped for website
We have seen rankings drop considerably for www.think-ins.co.uk in the last 48 hours, I know Penguin 2.0 has been released but anyone got any ideas on what specifically we might have been penalised for? Thanks
Algorithm Updates | | instinctive0 -
Canonicalization on more than one page?
is it proper to "canocalize" more than one page in a site? Or should it only be on the home page? eg: http://www.sundayschoolnetwork.com">
Algorithm Updates | | sakeith0 -
Organic listing & map listing on 1st page of Google
Hi, Back then, a company could get multiple listings in SERP, one in Google Maps area and a homepage or internal pages from organic search results. But lately, I've noticed that Google are now putting together the maps & organic listings. This observation has been confirmed by a couple of SEO people and I thought it made sense, but one day I stumble with this KWP "bmw dealership phoenix" and saw that www.bmwnorthscottsdale.com has separate listing for google places and organic results. Any idea how this company did this? Please see the attached image
Algorithm Updates | | ao5000000 -
Google doesnt index my Google+ Profile
Hey guys! I know it sounds like a novice question, but I have checked ALL THE BOXES THAT TELL GOOGLE TO INDEX MY GOOGLE+ PROFILE. It is Visible for search - 100%. It's been 3 weeks since I opened a Google+ profile and it still hasn't been indexed for its name. Any guesses what's going on? (It's not this name so don't try to google me)
Algorithm Updates | | Yoav_Vilner0 -
Specific Page Penalty?
Having trouble to figure out why one of our pages is not ranking in SERPs, on-page optimisation looks decent to me. Checked by using gInfinity extension and searched for the page URL. Can one page be penalised from Google engines (.ie / .com ) and the rest of the website not penalised? The (possible) penalised page is showing in Google places in SERPs. I assume this would not show if it was penalised. Would appreciate any advice. Thanks
Algorithm Updates | | notnem0 -
Google above the fold update
Hi everyone, Ever since the Jan 19th Google 'above the fold update' I have noticed some strange ranking changes in some of my sites. 1. rankings increased dramatically (not in top 50 to page 2) on Jan 19th for about 5 days then dropped out completely from the top 50. 2. our rankings then did the same thing again around Feb 2nd for about 5 -6 days then has bottomed out ever since. We do not have any ads on the site but our pages are dominated by images for most of the 'above the fold' section then followed by the content down the page. Any insight into this would be much appreciated. Cheers, Andrew
Algorithm Updates | | jay.raman0 -
Domain Authority and Google keywords
Hi there, We have a domain authority of 33, one of our competitors has an authority of 10, yet they appear to list higher on many keyword searches in google. Is there a reason for this? Our site is 5 months old, and their site is over 3 yrs old. Thanks for your feedback 🙂
Algorithm Updates | | PHDAustralia680 -
Does Google index Wordpress pages with frames
Does Google or other search engines index Wordpress pages that use frames? Here is the site in question: http://www.source-nutrition.com/son/
Algorithm Updates | | BradBorst0