What is the fastest way to deindex content from Google?
-
Yesterday we had a client discover that our staging URLs were being indexed in Google. This was due to a technical oversight from our development team (forgot to upload meta robots tags).
We are trying to remove this content as quickly as possible. Are there any methods in the Google Search Console to expedite this process?
Thanks
-
Excellent answer. Thank you very much.
-
Rosemary, in order to remove the content quickly, you have to do several things. You see, Google's processes for crawling, etc. and removing content from the index don't always happen all at once. So, it's best to do several things:
-
Remove the content. When visitors or bots visit the URL, use the "410 Gone" server header code to ensure that it's not just a 404 error being used.
-
If the content must stay and cannot be removed but still needs to be removed from Google's index, consider password protecting the content, putting it behind a paywall, making users log in to see the content, and/or adding a meta robots noindex tag on the page.
-
Add a robots.txt file on the subdomain so that it tells the bots to stop crawling. If you use something like dev.yourdomain.com for a dev section of the site, make sure that you have a robots.txt file at dev.yourdomain.com/robots.txt.
-
Use Google Search Console to remove the content. Once logged in, use the removal tool: https://www.google.com/webmasters/tools/removals?pli=1
By using several approaches, this is going to be the fastest way to remove the content.
-
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Blog with all copied content, should it be rewritten?
Hi, I am auditing a blog where their goal is to get approved to on ad networks but the whole blog has copied content from different sources, so no ad network is approving them. Surprisingly (at least to me), is that the blog ranks really well for a few keywords (#1's and rich snippets ), has a few hundred of natural backlinks, DA is high, has never been penalized (they have always used canonical tags to the original content), traffic is a few thousand sessions a month with mostly 85% organic search, etc. overall Google likes it enough to show them high on search. So now the owner wants to monetize it. I suggested that the best approach was to rewrite their most visited articles and deleted the rest with 301 redirects to the posts that stay. But I actually haven't worked on a similar project before and can't find precise information online so I'm looking to know if anyone has a similar experience to this. A few of my questions are: If they rewrite most of the pages and delete the rest so there is no repeated/copied content, would ad networks (eg. adsense) approve them? Assuming the new articles are at least as good quality as the current ones but with original content, is there a risk on losing DA? since pretty much it will look like a new site once they are done They have thousands of articles but only about 200 hundred get most visits, which would be the ones getting rewritten, so it should be fine to redirect the deleted ones to the remaining? Thanks for any suggestions and/or tips on this 🙂
Intermediate & Advanced SEO | | ArturoES0 -
Content suggestions
Hi, In moz pro you get content suggestions. I was wondering if you can still rank if the topics you cover for a specific keyword on your page are not listed there ? I guess the key is that all the topics covered are related to each other, correct ? Thank you,
Intermediate & Advanced SEO | | seoanalytics0 -
Sitemap and content question
This is our primary sitemap https://www.samhillbands.com/sitemaps/sitemap.xml We have a about 750 location based URL's that aren't currently linked anywhere on the site. https://www.samhillbands.com/sitemaps/locations.xml Google is indexing most of the URL because we submitted the locations sitemap directly for indexing. Thoughts on that? Should we just create a page that contains all of the location links and make it live on the site? Should we remove the locations sitemap from separate indexing...because of duplicate content? # Sitemap Type Processed Issues Items Submitted Indexed --- --- --- --- --- --- --- --- --- 1 /sitemaps/locations.xml Sitemap May 10, 2016 - Web 771 648 2 /sitemaps/sitemap.xml Sitemap index May 8, 2016 - Web 862 730
Intermediate & Advanced SEO | | brianvest0 -
Content Audit Questions
Hi Mozzers Having worked on my companies site for a couple of months now correcting many issues, im now ready to begin looking at a content review, many areas of the site contain duplicate content, the main causes being 1. Category Page Duplications
Intermediate & Advanced SEO | | ATP
e.g.
Widget Page Contains ("Blue Widget Extract")
Widget Page Contains ("Red Widget Extract")
Blue Widget Page Contains ("Same Blue Widget Extract")
Red Widget Page Contains ("Same Red Widget Extract") 2. Product Descriptions
Item 1 (Identical to item 2 with the exception of a few words and technical specs)
Item 2 Causing almost all the content on the site to get devalued. Whilst i've cleared all moz errors and warnings im certain this is causing devaluation of most of the website. I was hoping you could answer these questions so I know what to expect once i have made the changes. Will the pages that had duplicate content recover once they possess unique content or should i expect a hard and slow climb back? The website has never receive any warnings from Google, does this mean recovery for penalties like duplicate content will be quicker Several pages rank on page 1 for fairly competitive keywords despite having duplicate content and keyword spammy content. What are the chances of shooting myself in the foot by editing this content? I know I will have to wait for google to crawl the pages before i see any reflection in the changes, but how long after google has crawled the page should I get a realistic idea of how positive the changes were? As always, thanks for you time!0 -
Best way to move the content to a different domain without inviting any SERP penalty?
Hi all, We are in a bit of a fix right now. We have around 60-70 articles (Wordpress pages / posts) that we intend to move to another domain of ours. What's the best way to do so such that we do not invite any Google penalty. Here's a detailed information about our case:
Intermediate & Advanced SEO | | stj
Let's say, our site example.com has more 2000 articles. To help us better position our content for one of the sections on example.com, we have started another website, example2.com and want to move those 60-70 articles from example.com to example2.com. What is the best way to do it such that we are not penalised by Google? Is it (a) Move all the said content (60-70 articles) from example.com to example2.com and (b) do a permanent redirect (301) of each of the older article URLs to newer article URLs. What are the other options?0 -
Content Aggregation Site: How much content per aggregated piece is too much?
Let's say I set up a section of my website that aggregated content from major news outlets and bloggers around a certain topic. For each piece of aggregated content, is there a bad, fair, and good range of word count that should be stipulated? I'm asking this because I've been mulling it over—both SEO (duplicate content) issues and copyright issues—to determine what is considered best practice. Any ideas about what is considered best practice in this situation? Also, are there any other issues to consider that I didn't mention?
Intermediate & Advanced SEO | | kdaniels0 -
Google Experiment
Hi there, are there any implications for organic seo for implementing google experiment. Where you send users to a new page?
Intermediate & Advanced SEO | | pauledwards0 -
Google places
Is there away to get to the top of google places? Can it be manipulated?
Intermediate & Advanced SEO | | dynamic080