Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When does updating site content affect SERPS?
Can any of you shine a light as to when updating content on a website had any effect on its 'general' placement in the search engines? I appreciate that for an online newspaper, it must be important, but has anyone noticed from theirs or a client's site that by not uploading i.e. product descriptions or articles in general, that their site has taken a hit?
On-Page Optimization | | SDavis110 -
Help finding someone to handle crashing site/site optimization.
I need someone who can handle website/WordPress issues as they come up.For example, my site has gone down 4 times tonight, and my host can't figure it out. They also keep recommending that I optimize my site, but I don't know how. I need a go-to web person for this sort of thing. Any recommendations?
On-Page Optimization | | cbrant7770 -
Will German meta data help an English USA-based site get found in Germany?
I think I already know the answer to this, but I'd love a second opinion. If you add German meta data to some web pages will it help those pages turn up in search results in Germany?I'm assuming not. It's an English, USA based site, doing well here for English searches - and I don't want to mess that up. A German site is being planned, but this client is hoping the English site could be found in the meantime. ~Caro
On-Page Optimization | | Caro-O0 -
Why Does This Site Rank Well?
I noticed recently that the website of a local church (http://www.lifepointfc.com/) has climbed toward the top of the SERPS, and I cannot figure out why. The on-site SEO seems mediocre, at best, and their domain authority is 15 while their homepage authority is just 19. Does anyone see how they are working their SEO magic?
On-Page Optimization | | cbizzle0 -
Url structure
Hi Guys, Wondering what is better for url structure say for example a key word "slow cooker" example.com/slowcooker or example.com/slow-cooker ? Thank you 🙂
On-Page Optimization | | GetApp0 -
Is a site map necessary or recommended?
We have a website that has been up for the past 4 years without a site map. Google is indexing it. Do we need a site map? Do you recommend we create one and submit it to goggle and bing? The site is www.logobids.com Thank you.
On-Page Optimization | | IsaacH0 -
URL extensions naming
I have always wrote URL extensions as www.mysite.com/two_words.html .... when I need to separate two words, I use _ as the separator ... I am a first time SEO Moz user ... I While looking around the tools on SEO Moz, I happened to stumble across the on-page analysis. A great tool indeed, rather worryingly though, one issue it flagged to me was my URL extension "Characters which are less commonly used in URLs may cause problems with accessibility, interpretation and ranking in search engines. It is considered a best practice to stick to standard URL structures to avoid potential problems." Can someone advice me if this really is a problem, its just not this project, its tons of sites I have already developed that I am also worried about ... I always write file extensions with more than one word using _ to separate the words. How should I write the extension, I am almost embarrassed to ask this question ... Surely, even Google's algorithms are not smart enough to decipher two words without some some sort of spacing .... Regards J
On-Page Optimization | | Johnny4B0 -
Fixing large number of 404s
Hi Newbie SEO here.... I converted quite a massive site to wordpress (www.yourbusinesschannel.com). For a number of reasons, over 1000 pages were not converted and the URLs could not easily be preserved in any event I am now going to add these pages back by importing a large CSV into wordpress. However what is the best way to tell google that the pages are now back and at new addresses? Is it to submit a site map and get the site recrawled from scratch?
On-Page Optimization | | JamesKirkyBCtv0