Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Toxic URL???
Hi I have a URL that produced page 1, number 1 to 3 for most of our industries top phrases. Then we received a google penalty, (as did several of our competitors on the same day). We were effectively wiped from google. After much disavowing we were allowed back into the search results, this took about 3 months. I have employed the services of a top London SEO company for over a year now and have seen no significant improvement. I believe they are doing there best, however there results are VERY poor. According to the various tools, (searchmetrics, woorank, semrush) to name but a few, our site scores very well, yet we are not getting the results. Page one seems to be full of totally new websites, most of which I have never heard of, and have appeared from nowhere. Should I scrap our URL and put up a completely new one, and put a redirect from the original one? This would be a biggy since our url has been around for 20 years. Thanks for reading. Andy
On-Page Optimization | | First-VehicleLeasing0 -
Blog Site Set-Up/Frequency
Hello! I am a new blogger that is just getting started. All I have done so far is purchase a domain name and signed up with Blue Host. From what I am reading, WordPress seems to be the most recommended plug-in to get going. There is a lot of material out there on SEO and I wanted the community's thoughts on where to start. My blog is intended to provide readers with information on a particular topic. It is a vast area with much to write about - I intend on populating the content myself/with the help of experts in the field. It is not news so it is not time sensitive. Here are my questions: 1. As this is a brand new site, should I be blogging every day or is couple of times a week sufficient? I am able to produce content fairly quickly in the beginning as there is a lot to write about. Should I write a whole lot of content first and then release it on a schedule (twice a week, daily, etc.)? As it is a new site, I don't want it to launch with just one article.. 2. I am taking a WordPress class next week, but as someone that is not too familiar with copy/backlinks, other mechanisms to boost SEO. does WordPress make it easy enough to optimize your site for search results or does it require more expertise? As far as I see, my first tasks are to 1. ) set up the site effectively so people can find it and 2.) create valuable/engaging content. Appreciate any advice on setting up the site, blogging frequency, other tips to get going. I don't want to launch a site and get dinged by Google for reasons I am unware of..At some point, when I have a robust set of content, I am thinking of FB advertising to increase traffic to the site. Seems a bit premature to do that at this point.. Thanks all in advance for your feedback!
On-Page Optimization | | mmprakash0 -
Wordpress category url problem.
I have set up wordpress categories but the permalinks are showing as www.mydomain.com/?cat=12 as opposed to the category name. The child categories though work fine and show as www.mydomain.com/category/chidcatgegory I've obviously got my permalink settings wrong somewhere. How do I fix this?
On-Page Optimization | | SamCUK0 -
Question Regarding Site Structure
I have a quick question regarding site structure that I hope some of you guys could share your opinion on. I watched a white board friday from Rand a little while back where he explains that you need to try and make the site structure as flat as possible. He was saying try having no more that 3 links from the home page to get to the desired location. My question is this. I am looking at a site that has a pretty complex structure that I am trying to clear up as much as possible without making any of there rankings suffer. So they have www.domian.com/general-category/district/town/ and sometimes www.domian.com/general-category/district/town/item-specifics Now i know it is not good as it is, but they are dubious about changing too much as they have some serious traffic coming to the site. But, my question is that all the pages can be found from the home page through the menus/sub-menus. But do these count as a direct link from the home page. Also a problem is that because of this mozbot has detected that there are too many links from the home page and suggested that it should be below 200. But should I make these menu links no index or no follow. Obviously, by doing this, if the link does count as direct from the home page it wont after doing this. Thanks Jenson
On-Page Optimization | | jensonseo0 -
URL 404 errors after crawl? HELP!
I am getting Crawl errors. It shows multiple pages as. I know this is more of a technical question however, I cannot find the answer anywhere. I'm using wordpress www.mydomain.com/title-of-page/mydomain.com/contact WHAT IS THIS?!
On-Page Optimization | | ChristineWeinbrecht0 -
Keyword in url, which way better?
Hello, is there a difference between urls for targeting keyword "brazil tourist visa" fastbrazilvisas.com/tourist or fastbrazilvisas.com/brazil-tourist-visa ? ran the report In-Page Optimization it tells "no keyword usage in url". is there an idea behind that? thanks
On-Page Optimization | | Kotkov0 -
SEF URLs. Should I use / or - ?
I have o activate SEF URLs in a website. Regarding SEO, is there any difference between using / or - ? I mean, Is it better to write URLs like this: http://www.domain.com/folder/folder/page or like this: http://www.domain.com/folder-folder-page ? Is there any difference? Thanks!
On-Page Optimization | | ociosu0 -
Meta refresh - nojavascript url
seomox is telling me that I am getting a page that is not being indexed or crawled and since the crawl status code is 200 and there are no robots the meta-refresh url must be the problem. the meta refresh url is different than the on page report card url as it's the nojavascript url which my developer says should be ok. see his comments below. The is redirecting to http://mastermindtoys.com/store/nojavascript.html only in case if the JavaScript is disabled in the client browser. This is the right way to do it, I don’t understand why this might be a problem, otherwise MM has to implement Noscript pages that have a real content. I didn’t get what’s wrong about accessibility. The code 200 means it is accessible, and yes there is nothing to access if JavaScript is disabled on browser. I think there are no modern retail sites that would do any sensible business with the scripting disabled in browsers.The H1 is really present 2 times and second occurrence can be removed, though I highly doubt about importance of this change.Regarding duplicates – what URLs are considered duplicates? Can you please send me examples?I am not aware of canonical URL problem for MM site unless we consider old .asp links as duplicate links of the canonical product pages. I would appreciate if SEOMoz gave us an example what they mean.I suspect that the page is not getting indexed as a result of this or I'm just not getting a good score. Which is it?
On-Page Optimization | | mastermindtoys0