Blocked By Meta Robots
-
Hi
I logged in the other day to find that over night I received 8347 notices saying certain pages are being kept out of the search engine indexes by meta-robots.
I have not changed my robots.txt in years and I certainly didn't block Google from visiting those pages.
Is this a fault on Roger Mozbot behalf? Or is there a bot preventing 8000+ pages being indexed?
Is there a way to find out what meta-robot is doing this and where? And how I can get rid of it?
I usually rank between #3 and #5 for the term 'sex toys' on google.com.au, but I now rank #7 to #9 so it would seem some of my pages/content is being blocked.
My website is www.theloveshop.com THIS IS AN ADULT TOYS SITE.
There is no porn videos or anything like that on it, but just in case you don't wish to look at sex toys or are around kids I thought I would mention it.
Blake
-
Hi Nakul
Thank you very much for the suggestions and I will start checking the code on those pages today. Thanks for heads up on the .com.au site as well I will be changing the hosting today to Aus, and thanks to you that site may get pushed up the rankings for a highly competitive term that I have been fighting to get for years
As for following my competitors links, I have been doing that for awhile and it would seem most of them either buy their links or have them on pages that are irrelevant to our niche, one of my competitors has even acquired more links than SEOmoz.com but has no were near the content. In saying that I am by no means a pure white hat SEO, more of a grey hat if such a thing exists, simply because I find it hard to get links to an adult site without it being spammy of some sorts. So I have started writing articles and linking to products mentioned, but trying to make the articles interesting/funny at the same time. Hopefully soon I will start to see some improvement.
Again thank you for your help and I will let you know how I go with those pages blocked by bots.
Blake
-
The meta robots block sounds like it's a block from your CMS or something in your header include file or something. I would check on the pages that show as blocked to see if there's any noindex sort of code in those pages. Other then that, shift in rankings could be based on your competition optimizing for the same set of keywords. I would recommend to spend some time analyzing the backlinks of the competition and see if you can gain some of those kinds of links.
On a side note: It also looks like you own both the .com and the .com.au site. The .com is hosted in AU and the .com.au is hosted in USA. Did you intend to do it the other way around ? I would imagine your rankings for the .com.au get better if it was hosted in Australia instead of USA.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the meta title written only for google (try to stuff in as many keywords as possible) or is there a user experience aspect as well?
Is the meta title written only for google (try to stuff in as many keywords as possible) or is there a user experience aspect as well?
On-Page Optimization | | whiteonlySEO0 -
The Keyword density in a landing page is very low . by adding meta keyword tag can improve keyword density?
The Keyword density in a landing page is very low . by adding meta keyword tag can improve keyword density?
On-Page Optimization | | socialhi50 -
Whats wrong with meta title? In SERP it looks different.
Hi, can someone tell me why my meta titles looks different in SERP? I use Prestashop CMS and meta title in many pages looks similair to this one: Puodų Rinkiniai | Puodai | Iki -40% Akcija | ŠefoSvajonė.lt And in SERP it looks like this: Puodų Rinkiniai - sefosvajone.lt. sefosvajone.lt Why can this be? Its freshly indexed. Rather than set Meta Title text it shows just name of the Product or Category and then shops adress twice or sometimes once. Thanks!
On-Page Optimization | | Absurdus0 -
Meta titles and description for subcategory pages in CMS websites
I have a CMS based site and for that I am getting duplicate page title and description error message, for my all sub-category pages. What can I do for this because write Meta title and description for each page for CMS based website is not possible.
On-Page Optimization | | 1akal0 -
Which Meta Tags would you recommend having on the pages?
Hello, There are so many different meta tags which you can implement to the header of your website and I'm wanting to get people's opinions on which you feel are worth putting in and which are completely pointless? For example: Author Copyright Language Content-Type Content-Language Distribution Abstract Keywords Description Classification Canonical Expires Revisit-after Rating I look forward to hearing your comments and suggestions. Kind Regards, Shaun Swales Twitter: Https://www.twitter.com/CryptCommerce
On-Page Optimization | | ShaunSwales0 -
Can you expound why i have to avoid using meta keywords?
I'm using the on page report card and it tells me that i have to avoid using meta keywords.I'm a little bit confused. I thought that it's important to use it all the time so search engine can better index the site. if I use SEO Quake it will tell me in the diagnostic test that I need to input keywords.
On-Page Optimization | | jsevilla0 -
How unique should meta descriptions be?
I understand that it is a best practice to write a different meta description for each page. However if you have 50 pages on a topic that are similar, is it okay to just change a couple words. For instance if you have a page about how to get an SEO job in each state, can you use the same meta description 50 times but change just the state name. Or is it much better to vary the meta descriptions significantly. Thanks!
On-Page Optimization | | SparkplugDigital0