Question about robots.txt
-
Solved!
-
Just a friendly reminder. Please don't delete your question after it's been answered. It's very likely that someone in the future could have the same question and they would have been able to find the answer if you hadn't deleted the question.
-
Consider deleting all of this:
Disallow: /&limit
Disallow: /?limit
Disallow: /&sort
Disallow: /?sort
Disallow: /?route=checkout/
Disallow: /?route=account/
Disallow: /?route=product/search
Disallow: /?route=affiliate/
Disallow: /?marca
Disallow: /&manufacturer
Disallow: /?manufacturer
Disallow: /?filter
Disallow: /&filter
Disallow: /?order
Disallow: /&order
Disallow: /?price
Disallow: /&price
Disallow: /?filter_tag
Disallow: /&filter_tag
Disallow: /?mode
Disallow: /&mode
Disallow: /?cat
Disallow: /&cat
Disallow: /?product_id
Disallow: /&product_id
Disallow: /?route=affiliate/
Disallow: /*?keywordThose rules are telling Google not to crawl domain.com/EVERYTHING(then the URL parameter). This could be where the issue stems from. If you're worried about URLs with these things ranking, consider implementing canonical tags instead to point to the proper pages
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Question about my website ranking
Hi everyone I have sports website. My website DA, DR is good. I have created good quality backlinks. But still my website ranking is not improving. Can you guide me how to improve my website ranking. Thanks in advance!
On-Page Optimization | | fktos.tam.906n0 -
One more question about rel=canonical
I'm still trying to wrap my head around rel=canonical and its importance. Thanks to the community, I've been able to understand most of it. Still, I have a couple of very specific questions: I share certain blog posts on the Huffington Post. Here's an example: http://www.huffingtonpost.ca/cedric-lizotte/munich-travel-guide_b_13438956.html - Of course I post these on my blog as well. Here: http://www.continentscondiments.com/things-munich-classics/ - Obviously the HuffPo has a huge DA, and I'll never match it. However the original post is mine, on my blog, and not on the HuffPo. They wont - obviously - add a rel=canonical just for me and for the sake of it, they have a million other things to do. QUESTION: Should I add a rel=canonical to my own site pointing to the post on the HuffPost? What would be the advantage? Should I just leave this alone? I share blog posts on Go4TravelBlog too. Example: http://www.go4travelblog.com/dallmayr-restaurant-munich/ - but, once again, the original post is on one of my blogs. In this case, it's on another blog of mine: http://www.thefinediningblog.com/dallmayr-restaurant-in-munich/ QUESTION: Well it's pretty much the same! Should I beg Go4TravelBlog to add a rel=canonical pointing to mine? If they refuse, what do I do? Would it be better to add a rel=canonical from my site to theirs, or do I fight it out and have a rel=canonical pointing to my own post? Why? Thanks a million for your help!
On-Page Optimization | | cedriklizotte0 -
Magento Duplicate Content Question - HELP!
In Magento, when entering product information, does the short description have to be different than the meta description? If they are both the same is this considered duplicate content? Thanks for the help!!!
On-Page Optimization | | LeapOfBelief0 -
Question about url structure for large real estate website
I've been running a large real estate rental website for the past few years and on May 8, 2013 my Google traffic dropped by about 50%. I'm concerned that my current url structure might be causing thin content pages for certain rental type + location searches. My current directory structure is:
On-Page Optimization | | Amped
domain.com/home-rentals/california/
domain.com/home-rentals/california/beverly-hills/
domain.com/home-rentals/california/beverly-hills/90210/
domain.com/apartment-rentals/california/
domain.com/apartment-rentals/california/beverly-hills/
domain.com/apartment-rentals/california/beverly-hills/90210/
etc.. I was thinking of changing it to the following:
domain.com/rentals/california/
domain.com/rentals/california/beverly-hills/
domain.com/rentals/california/beverly-hills/90210/ ** Note: I'd provide users the ability to filter their results by rental type - by default all types would be displayed. Another question - my listing pages are currently displayed as:
domain.com/123456 And I've been thinking of changing it to:
domain.com/123456-123-Street-City-State-Zip Should I proceed with both changes - one or the one - neither - or something else I'm not thinking of? Thank you in advance!!0 -
Usability-question on 2 of my websites
Hi all, It has been a while since I have been in this forum, but that's normally a great sign 🙂 Anyway these days I'm work on a "face-lift" on a couple of my sites (www.krydsord.dk, www.bingoforum.dk and www.alutagrender.dk), but also this face-lift should be more than just that. Hopefully it will help the usability to be better, and as a result of that - I hope it will make it easier for my visitors to easily find what they are looking for. The question is: I would like to ad, say, 25 words of description (mouse-over) in my top menu buttons. Why? Because, I believe that it's a possible problem that the menu just says "krydsord" (crosswords) in stead of a describing text saying something about "Push this button and you will see a complete list of all the cross words on Krydsord.dk). This way, I get the chance to add more text (for the Search Engines) and to guide my visitors in a way that I'm not capable of with the limitations my top menus give me as they are at this moment. Naturally, my visitors should be able to easily decline this "extra information", once they have got it once. Is this completely way off, or do you like the idea? More specifically: Will this be annoying for the visitors, will it have any positive effect on SEO matters? Actually I got partly inspired during a SEOmoz webinar (http://www.seomoz.org/webinars/designing-for-seo), even though it's not exactly what they are talking about in thwe webinar. For anybody who are curious (or just need to kill some time), you can have a look from 19:00 and watch it for about two minutes. Thanks a lot! Nicolai
On-Page Optimization | | MPO0 -
Do I need a robots meta tag on the homepage of my site?
Is it recommended to include on the homepage of your site site? I would like Google to index and follow my site. I am using WordPress and noticed my homepage is not including this meta tag, therefore wondering if I should include it?
On-Page Optimization | | asc760 -
Anchor Texts on Internal Links on my Site Question
[Hey folks, Anchor text question for internal links on my site. We all know that proper anchor text is important for rankings, both in links pointing to your site and links on your site. Google webmaster tools is showing my top internal anchor text links to be letters. This is because of navigation on the site which is in a global footer on hundreds of thousands of pages. Do you think that this dilutes or minimizes the relevance of the real anchor texts that we are targeting which trail behind these, or if this went away it would have no effect on the rest of the anchor texts on the site. How would you handle this, we can just get rid of this navigation because it's passing PR and helping the pages get spidered. Thanks in advance. Anchor text 1. g 2. q 3. d 4. v 5. k 6. t 7. o 8. u 9. a 10. p 11. m 12. r 13. e 14. x 15. y 16. l 17. i 18. z 19. c 20. j 21. s 22. b 23. f 24. h 25. w 26. n](mailto:[email protected])
On-Page Optimization | | irvingw0 -
What's the best practice for implementing a "content disclaimer" that doesn't block search robots?
Our client needs a content disclaimer on their site. This is a simple "If you agree to these rules then click YES if not click NO" and you're pushed back to the home page. I have this gut feeling that this may cause an upset with the search robots. Any advice? R/ John
On-Page Optimization | | TheNorthernOffice790