Search engines have been blocked by robots.txt., how do I find and fix it?
-
My client site royaloakshomesfl.com is coming up in my dashboard as having Search engines have been blocked by robots.txt, only I have no idea where to find it and fix the problem. Please help! I do have access to webmaster tools and this site is a WP site, if that helps.
-
Here's the screen in Wordpress to check to see if you've blocked search engines through WP.
-
Ah, I see. Thanks!!
-
nothing comes back when I run this
site:www.royaloakshomesfl.com
So looks like its not indexed
-
When I search their brand name, they pop up in Google, so they must be indexed somehow, no?
-
I would do as Lavellester suggest, then i would then i would re-submit as i can see you are not in either index.
-
You can see it here:
http://royaloakshomesfl.com/robots.txt
So its in the webroot of your hosting. You can change it via FTP to and change it to the following to grant full access until you have figured out robot access rights. I think you need at least one Disallow: line without the / slash in your existing configuration to make it work.
User-agent: * Disallow:
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search visibility degrading gradually
We have several web pages with the same structure released in several countries. Each website contains information about spam callers in the country the website has been released for. Now I have the problem that I see a slow degradation of search traffic in the US. The UK website on the other hand is doing quite well, actually improving. According to MOZ our mobile search visibility dropped significantly in the last week and I am at the moment not able to pin this down. Can anyone please give me a hint at what data best to analyze to find the source of this problem? TIA Best
Technical SEO | | Roverandom
Thomas1 -
Do you think have to re-submit my site to search engines after I made improvements?
Some time ago I started to do SEO for a one-page website and didn't get any positive result: no traffic, no filled in online booking form (yet another, multiple page website offering the same service yielded in multiple filled-in "schedule an appointment" forms). I found out my one-page website was considered to be "keyword-spamming" and converted it to a multiple page one. Its domain authority went up, but it doesn't still bring any traffic. I am thinking maybe I have to let the search engines know that it has been updated so they stop penalizing it? Do you think it might help and if yes, what exactly I should do? Will be thankful very much for any suggestion!
Technical SEO | | kirupa0 -
Quality Issues: My blog is blocked on Google Search Engine
Hi Webmasters, I got an email from google team. The email is included below. **Google Webmaster Tools: Quality Issues on http://abcdblogger.com/**August 8, 2012 Dear site owner or webmaster of http://abcdblogger.com/, We've detected that some of your site's pages may be using techniques that are outside Google's Webmaster Guidelines. If you have any questions about how to resolve this issue, please see ourWebmaster Help Forum for support. Sincerely, Google Search Quality Team My blog is completely blocked on Google Search engine. I removed all existing posts and reinstalled a fresh version of wordpress and wrote a good article. I redirected all broken links my homepage with a 301. After making those changes I submitted a reconsideration request to Google, But they declined it. I doubt that the reason for blocking could be due to the backlinks pointing to my domain. I think Google's Disavow Tool help me to remove low quality backlinks, But how can I sort low quality backlinks using Opensite Explorer? If possible can you create a text file with all possible low quality links, So that I could submit it using Google Disavow Tool. Thanks.
Technical SEO | | hafiskani0 -
Question about construction of our sitemap URL in robots.txt file
Hi all, This is a Webmaster/SEO question. This is the sitemap URL currently in our robots.txt file: http://www.ccisolutions.com/sitemap.xml As you can see it leads to a page with two URLs on it. Is this a problem? Wouldn't it be better to list both of those XML files as separate line items in the robots.txt file? Thanks! Dana
Technical SEO | | danatanseo0 -
Using Robots.txt
I want to Block or prevent pages being accessed or indexed by googlebot. Please tell me if googlebot will NOT Access any URL that begins with my domain name, followed by a question mark,followed by any string by using Robots.txt below. Sample URL http://mydomain.com/?example User-agent: Googlebot Disallow: /?
Technical SEO | | semer0 -
How do I fix these duplicate URLs?
HI guys, I ran a report on my site and it shows some duplicate titles (example below). Do I need to add something to the htaccess file or another file to fix this? I understand that the search engines should only see 1 URL for the page. 2 pages have "Bikes for sale | used bikes | second hand bicycles" title pauslwebsite.com/bikes/ paulswebsite.com/bikes/index.asp Thanks
Technical SEO | | paulmund0 -
Robots.txt blocking site or not?
Here is the robots.txt from a client site. Am I reading this right --
Technical SEO | | 540SEO
that the robots.txt is saying to ignore the entire site, but the
#'s are saying to ignore the robots.txt command? See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file To ban all spiders from the entire site uncomment the next two lines: User-Agent: * Disallow: /0 -
How to do a no follow on site search
We have a site search that is causing a huge amount of errors as the SEOmoz crawler is showing these as duplicate content. Our first thought was to do a no-follow on the site-search directory, but we realized that the site search is /site-search.aspx and URl strings appear at the end for hundreds of pages. How dow we/how can we no-follow an undetermined amount of URL strings?
Technical SEO | | Apptixweb0