Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
With 301 Redirects Does Changing URLs Matter?
We are redesigning our website in order to give it a more modern visual look. For the most part all the content will remain the same. Our old site is hosted on .asp so all of our current URLs look something like this: www.example.com/products/food.asp We plan on using 301 redirects in order to update every URL and remove the .asp. Since we are going to be doing 301 redirects for every existing URL anyways, does it matter from an SEO and ranking standpoint, if we also change the content and structure of the URL? For example, would we see a ranking impact if we were to change the above example URL to www.example.com/food? Obviously we want to try to retain as much link juice and ranking factors as possible during this redesign. Another issue we are seeing is with the image file names of our existing website images. We are moving to a new CMS platform (WordPress) that automatically saves images using a folder path similar to this: wp-uploads/2015-08/food. Will that change affect our SEO or ranking at all? When Google crawls an image does it care about the full path? Any insight would be much appreciated! 🙂
On-Page Optimization | | BlueLinkERP0 -
Unsure about improvement of inner sites
Hi ! I want to improve the ranking of Home on my site. Now I did some on-page SEO on Home itself for the keyword "Webdesign Freiburg". How much does it make sense to put this keyword also into inner sites - although I primarily want to rank Home for this keyword? Does Home profit from putting the keyword into subpages? Cheers and thanks Marc
On-Page Optimization | | RWW0 -
Do sites with more pages rank better?
If a site has more pages and also has good internal linking, then logically it would rank better. Is this the case? Should I be building big (but high quality) sites?
On-Page Optimization | | T0BY0 -
Site Duplicated despte redirect
Buon pormeriggio from I can smell Whaler Chips Through the window Wetherby,
On-Page Optimization | | Nightwing
When you Google Thakray Medical Museum 2 sites appear in the SERPS, yikes! Now the .org site is no longer hosted & point to the .co.uk site when clicked on but in a nutshell I wantto get rid of the .org site
as illustrated here: http://s216.photobucket.com/user/zymurgy_bucket/media/two-versions-same-website-yikes_zps182e6e12.jpg.html Actions taken so far:
1: Wembaster tools re index request for the .co.uk site
2: Redirect configured to point .org site to the .co.uk What else is left apart from updating the xml site but ultimating i do not want to see the the .org site but it doesnt exist (well id did a few month back but is no longer hosted so i am told) Any insights welcome,
GRazie tanto,
David0 -
Should I rewrite all my URLs ?
Hi all, I'm pretty new here and this is a question I'm struggling with since years ! All my URLs are very long. Years ago I wanted to put as many keywords as possible but today I'm not sure anymore it was such a good idea. Example: http://www.spirit-of-metal.com/album-groupe-Take_Me_To_Janus-nom_album-Ripping_the_Heart_from_the_Chest_of_the_Earth-l-en.html The problem is I have more than 300K of these pages. I'm afraid to create a huge mess even if I 301 them all to the new pages. What's your opinion ? Is it worth the effort ? Many thanks in advance for your precious help !
On-Page Optimization | | kivanSOM0 -
URL extensions naming
I have always wrote URL extensions as www.mysite.com/two_words.html .... when I need to separate two words, I use _ as the separator ... I am a first time SEO Moz user ... I While looking around the tools on SEO Moz, I happened to stumble across the on-page analysis. A great tool indeed, rather worryingly though, one issue it flagged to me was my URL extension "Characters which are less commonly used in URLs may cause problems with accessibility, interpretation and ranking in search engines. It is considered a best practice to stick to standard URL structures to avoid potential problems." Can someone advice me if this really is a problem, its just not this project, its tons of sites I have already developed that I am also worried about ... I always write file extensions with more than one word using _ to separate the words. How should I write the extension, I am almost embarrassed to ask this question ... Surely, even Google's algorithms are not smart enough to decipher two words without some some sort of spacing .... Regards J
On-Page Optimization | | Johnny4B0 -
How to deal with tracking numbers in URLs
I am working on a site at the minute that has links like this: Jobs in London URL looks like: domain.com/jobs-in-london/ However, my developers insist that they need to use tracking codes, so everytime someone clicks on the above link, they are redirected (301) to a new URL that looks like: domain.com/search/1234567(unique search id) This is killing me when I am trying to get internal pages, like /jobs-in-london/ ranked. What to do?
On-Page Optimization | | MirandaP0 -
Site wide internal links in footer
What is your take on site wide inernal links in the footer, e.g. for the most competitive keywords? Do you think those links are still useful or are those links of little value from your experience? It is an old school tactic but I still see many sites doing it.
On-Page Optimization | | Florakel0