Dates in URL's
-
I have an issue of duplicate content errors and duplicate page titles which is penalising my site. This has arisen because a number of URLs are suffixed by date(s) and have been spidered . In principle I do not want any url with a suffixed date to be spidered.
Eg:-
www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/06_07_13/13_07_13
http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/20_07_13/27_07_13
Only this URL should be spidered:-
http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm
I have over 10,000 of these duplicates and firstly wish to remove them on block from Google ( not one by one ) and secondly wish to amend my robots.txt file so the URL's are not spidered. I do not know the format for either.
Can anyone help please.
-
Thanks Kyle.
Particularly grateful for the Disallow format, they are the only URL's using an underscore so will work for me. WIll be checking why these are being created.
Do I need to remove them using the Removal Tool in Google, is there a format for doing this on block ?
Thanks again,
Alan
-
Hi Alan,
I would probably start by adding a disallow rule to robots.txt.
**Disallow: /*_** _may work and block all your dated URLs from being indexed but may also have adverse affects if you have any URLs containing underscores. To test whether this solution would work I would firstly implement a disallow directly on a chosen dated URL, _**Disallow: /20_07_13 **_for example, and then test whether Google has noindexed the page. GWT should tell you whether you have inadvertently blocked any other pages by doing so.
You should also be thinking about how these URLs are being created and taking actions to prevent it. Consider implementing canonical tags if you haven't already to clean up any potential duplication issues.
Cheers,
K
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect chains error on the home page URL
Hello Everyone, I'm getting redirect chains error on the home page URL:
On-Page Optimization | | Nikhil_Falcon
http://ebitdacatalyst.com in Wordpress. I've checked my redirection list in the plugin, and haven't found any redirections on http://ebitdacatalyst.com. Can anyone please help me in solving this issue? I don't know where from it's coming.0 -
Is it better to shorten my existing url to use only keyword after domain with a 301 redirect from existing url
I have a long existing URL that has included my key word but the url has about 5 additional words in the text ( eg url would have " /super widgets in stock at the widget store " as url text after domain. keywords is super widget The URL was at the top of search results for my keyword for many years until recently. Is it better to shorten my url text to now use only my keyword " /super-widgets " after the domain with a 301 direct from my existing url to optimize it Thanks
On-Page Optimization | | mrkingsley2 -
How important are clean URLs?
Just wanting to understand the importance of clean URLs in regards to SEO effectiveness. Currently, we have URLs for a site that reads as follows: http://www.interhampers.com.au/c/90/Corporate Gift Hampers Should we look into modifying this so that the URL does not have % or figures?
On-Page Optimization | | Gavo1 -
Table text : Does that influences the website's ranking ?
We are currently working on a new page for one of our clients and there is A LOT of writing, so A LOT of differents pages. If I decide to put the texts on table text instead of creating a link for another page, will that influence the ranking in any way ? I've been a web writer for only a year now and still try to figure out how to optimize the written part. Thank you for your answers.
On-Page Optimization | | marketingmedia.ca0 -
Website redesign and it's impact on ranking
Hey Everyone, I have had a website in place for over 7 years and I am now at the point where I need to implement a redesign in order to sell our product more effectively. -URL's will stay the same
On-Page Optimization | | Justin45
-Content will change but be very similar page-to-page
-Title & Meta tags will remain the same I'm planning on taking the site from a non-backend site to a wordpress site so the navigation will change. Does anyone foresee that this change will have any dramatic effect on site ranking? Thank You!0 -
How do I cure 'overly dynamic' url's on an e-commerce website?
I've just launched an e-commerce website selling hosiery and have received aa report from SEO Moz regarding overly dynamic URL's. How do I resolve this issue - in words of one syllable please, I'm new to SEO! Here are three exapmles of over 120: http://www.yosassy.com/index.php?route=product/category&path=1&page=2 http://www.yosassy.com/index.php?route=product/product&filter_tag=&page=1&product_id=57 http://www.yosassy.com/index.php?route=product/product&filter_tag=&page=1&product_id=64 Thank you.
On-Page Optimization | | lindsayjhopkins0 -
Should ".html" in the URL or not?
Does URL like /wedding-dresses.html/f/price/2,100 with "html" in the middle do harm to SEO? Should I drop the ".html"?
On-Page Optimization | | StevenRoland0 -
Number ID's in URL's
If you have to place a number ID into a URL. Does it matter from an seo perspective whether it is placed in the middle or at the end? Thanks
On-Page Optimization | | simmo2350