What is the appropriate Robot.txt to unblock if Google cannot get all the resources from my homepage?
-
Hello everyone. I did some research as to why my website has decreased in the Google search rankings recently. After reading this Yoast article I believe it's because the robot.txt files I have set up on my wordpress website. The following is a screen shot of the results of a fetch & render query of my webpage.Googlebot couldn't get all resources for this page. Here's a list:
My current robot.txt settings are as follows;
User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: */xmlrpc.php Disallow: */wp-*.php Disallow: */trackback/ Disallow: *?wptheme= Disallow: *?comments= Disallow: *?replytocom Disallow: */comment-page- Disallow: *?s= Disallow: */wp-content/ Allow: */wp-content/uploads/ ```What to I need to allow/disallow to allow Google spiders to properly read my website?
-
Remove the following line:
Disallow: */wp-content/ And add: Allow: /wp-includes/js/
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
44 terms dropped out of the top 3 results on google this past week.
Can anyone take a look at the site and offer suggestions as to what I can do. My visibility went from 13% to 2% over the past two months. I had 800 organic terms prior to this now I am falling off a cliff. Have no idea what happened. I just switched over to HTTPS 3 weeks ago and it has helped a little.
Algorithm Updates | | Boodreaux1 -
The Google Algo update that happened 1-8 is KILLING my rankings
Does anyone know what happened?? I have a great website, we ranked very highly for a slew of industry keywords, #1 in most of our top-money kws....and our keywords have been in freefall since the update. Help?!
Algorithm Updates | | Sean_Gutermuth0 -
Google cant read my robots.txt from past 10 days
http://awesomescreenshot.com/08d1s6aybc hi, my robots.txt is http://wallpaperzoo.com/robots.txt google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with http://hdwallpaperzones.com/robots.txt
Algorithm Updates | | toxicpls0 -
Images not getting indexed in google image search :( " site: hdwallpaperzones.com " )
hi as i have mentioned in title.. my website images are not getting indexed in google image search engine.. out of 360 images only 5 got indexed from 3 days.. please help me out.. thanks
Algorithm Updates | | toxicpls0 -
Did anyone else get a massive drop off in Bing recently?
Only for one set of my keywords. But I fell from first page to off the map in one fell swoop. Did this happen to anyone else? Should I be worried?
Algorithm Updates | | CleanEdisonInc0 -
Would Google Remove Pages for Inactivity?
Hi, I've been watching the Total Indexed number for 4 domains that I work with for the last few months. In Google Webmaster Tools three of them were holding steady up until August-September, when suddenly they started declining by hundreds of thousands of URLs a week. I've asked my IT department and they say they haven't done anything technically different in the last few months that would affect indexation. I've also searched on google and on search marketing blogs to see if anyone else has experience this to no avail. As you can see in the image, the "Not Selected" pages have not increased so it appears this is not due to duplicate content (of which we have a lot). However, the "Ever Crawled" number is increasing. The only reasonable answer that I can conclude is that Google is now de-indexing inactive URLs? Anyone have a better answer? yIYDm.jpg
Algorithm Updates | | OfficeFurn0 -
Google Page Rank?
We have had a quality website for 12 years now, and it seems no matter how many more links we get and how much new content we add daily, we have stayed at PR3 for the past 10 years or so. Our SEOMoz domain authority is 52. We have over 950,000 pages linking to us from 829 unique root domains. Is this in line with PR3 or should we be approaching PR4 soon? We do daily blog posts with all unique, fresh quality content that has not been published elsewhere. We try to do everything with 'white hat' methods, and we are constantly trying to provide genuine content and high quality products, and customer service. How can we improve our PR and how important is PR today?
Algorithm Updates | | applesofgold0 -
Will signing up for Google Places affect my national rankings
OK, Here is a question which I can't find but think people have thought about. I would like to know others opinion. I have had a site that ranks well under generic national keyword terms. (not geographically specific) Its a small website, only 10 pages. We get 85% of our business from online applications. These applications come from all over the united states.Our SERP rankings generate 70% of all our traffic. My question is this: we operate in a state where we don't do business. We are a virtual business. Should I sign up for google places? Will It hurt my national SERP rankings?
Algorithm Updates | | FidelityOne0