Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will German meta data help an English USA-based site get found in Germany?
I think I already know the answer to this, but I'd love a second opinion. If you add German meta data to some web pages will it help those pages turn up in search results in Germany?I'm assuming not. It's an English, USA based site, doing well here for English searches - and I don't want to mess that up. A German site is being planned, but this client is hoping the English site could be found in the meantime. ~Caro
On-Page Optimization | | Caro-O0 -
Two sites into one
I have two sites owned by one client, he wants to merge them into one keeping one website, but which one? I've been using the Moz Pro to look at the stats for both sites; page authority, inbound links etc, but they're both fairly close in results. The client wants to know what would be the best course to take with these two sites, what site should he keep and which should he merge? Any advice?
On-Page Optimization | | barrowr0 -
Canonical URL Category and Tags
Hello, I would like to know that I want to use both category and tags in my blog StylishMahi. If I index both category and tags, should I use canonical URL tag to pass referring to main category. As I want more my categories in SERP results ranking higher? I have also attached a picture. Can someone please confirm? Photo by Moz ZigdWMx
On-Page Optimization | | PratapSingh0 -
23000 Links are not found- Should I redirect them?
Hi I have been deleting product links from my website but never redirect them. On my google webmaster, it shows there is total 23000 products are not found. Should I redirect them all back to the home page? For the pages with soft 404 response.. should I also redirect those original URL back to home page ? Thanks
On-Page Optimization | | ilovebodykits0 -
Site structure suggestions/feedback
I asked this on Reddit and got some some decent answers. I'm curious to see what the pro's of SEOmoz think. I've got a lead generation site for forklift parts--liftxparts.com. You can think of it similar to car parts, where we have sections for specific brands (e.x. Toyota forklift parts) and sections for specific categories (e.x. forklift filters). Right now, the site is structured in two main levels: the top level is a dozen or so brands (separate pages for Toyota forklift parts, Clark forklift parts, etc), and then the second level is the categories (separate pages for a dozen or so different categories like forklift filters, forklift engine parts, etc.).If you check out one of the pages, like Clark forklift parts for example (our top landing page)--liftxparts.com/clark-forklift-parts.html, you'll see that on the brand pages (they're all structured the same), we list all the different categories (with links to the same second level category pages) and "search" buttons. All pages point to the same lead capture form.This has been working pretty well--about 90% of visitors end up on our lead capture form, and a high percentage of those convert. We're working on increasing organic traffic now and I'm thinking our structure could use some improvement.Looking at the analytics, there are a lot more impressions for keywords like "clark forklift" than "clark forklift parts". One gap I've uncovered is while our average position, and by extension CTR and traffic, for phrases like "clark forklift parts" is quite good, it's not so good for broader and higher searched terms like "clark forklift". Should we add another level of hierarchy targeted to just general brands? So now we have content for clark forklift parts, but should we add a page for terms like "clark forklift"? Or should we just add some broader content to the existing brand pages? The pages are quite long already, I'm afraid adding more content to the bottom of the page isn't very functional. Our thinking is that we can increase average position for higher searched terms by adding content targeted to those terms. The question is how exactly to go about it and how to work it into our current site structure? Any feedback related to our site structure or even just related ideas about other ways to approach our goal of increasing organic traffic would be very much appreciated! Thanks!
On-Page Optimization | | wisamabdulla0 -
New Url Structure
Hey Guys We are working on a new site and in order to implement some of the new functions we need to restructure our url's , Will redirect everything correctly but I was looking for advice on the structure we need the word product / category subfolder for speed but would there be any benefit making them shorter ? what would you guys advise ? Category Current http://www.freestylextreme.com/uk/Home/Brands/DC-Shoe-Co-/default.aspx **New ** freestylextreme.com/uk/category/dc-shoe-co Product Current http://www.freestylextreme.com/uk/Home/Brands/DC-Shoe-Co-/Mens-DC-Shoe-Co-T-shirts/DC-Black-Unwind-T-Shirt---.aspx New freestylextreme.com/uk/product/dc-black-unwind-t-shirt
On-Page Optimization | | elbeno0 -
Is my H1 and H2 site structure proper?
If I have an h1 tag on the page that describes the purpose of the page, is it a bad idea to make all of the sub-headings h2 tags? A designer mentioned that it was frowned upon to make all of the secondary headings into h2 tags, or to use CSS to selectively style the heading tags.
On-Page Optimization | | dbuckles0 -
Is having the word catalog in an ecommerce site url detrimental to seo.
IS: www.example.com/catalog/category%/product% better than www.example.com/category%/product% category and product are dynamic values that change with the diff. categ. and products displayed while catalog is constant.
On-Page Optimization | | no6thgear0