Duplicate content warning: Same page but different urls???
-
Hi guys i have a friend of mine who has a site i noticed once tested with moz that there are 80 duplicate content warnings, for instance
Page 1 is http://yourdigitalfile.com/signing-documents.html
the warning page is http://www.yourdigitalfile.com/signing-documents.html
another example
Page 1 http://www.yourdigitalfile.com/
same second page http://yourdigitalfile.com
i noticed that the whole website is like the nealry every page has another version in a different url?, any ideas why they dev would do this, also the pages that have received the warnings are not redirected to the newer pages you can go to either one???
thanks very much
-
Thanks Tim. Do you have any examples of what those problems might be? With such a large catalog managing those rel canonical tags will be difficult (I don't even know if the store allows them, it's a hosted store solution and little code customization is allowed).
-
Hi there AspenFasteners, in this instance rather than a .HTAccess rule I would suggest applying a rel canonical tag which points to the page you deem as the original master source.
Using the robots to try and hide things could potentially cause you more issues as your categories may struggle to be indexed correctly.
-
We have a similar problem, but much more complex to handle as we have a massive catalog of 80,000 products and growing.
The problem occurs legitimately because our catalog is so large that we offer different navigation paths to the same content.
http://www.aspenfasteners.com/Self-Tapping-Sheet-Metal-s/8314.htm
http://www.aspenfasteners.com/Self-Tapping-Sheet-Metal-s/8315.htm
(If you look at the "You are here" breadcrumb trail, you will see the subtle differences in the navigation paths, with 8314.htm, the user went through Home > Screws, with 8315.htm, via Home > Security Fasteners > Screws).
Our hosted web store does not offer us htaccess, so I am thinking of excluding the redundant navigation points via robots.txt.
My question: is there any reason NOT to do this?
-
Oh ok
The only reason i was thinking it is duplicate content is the warnings i got on the moz crawl, see below.
75 Duplicate Page Content
6 4xx Client Error
5 Duplicate Page Title
44 Missing Meta Description Tag
5 Title Element is Too Short
I have found over 80 typos, grammatical errors, punctuation errors and incorrect information which was leading me to believe the quality of the work and their attention to detail was rather bad, which is why i thought this was a possibility.
Thanks again for your time its really appreciated
-
I wouldn't say that they have created two pages, it is just that because you have two versions of the domain and not set a preferred version that you are getting it indexing twice. .HTaccess changes are under the hood of the website and could have simply been an oversight.
-
Hey Tim
Thanks for your answer. It's really weird, other than lazyness on the devs part not to remove old or previous versions of pages?, have you any idea why they would create multiple versions of the same page with different url's?? is there any legit reason like ones severs mobile or something??
Just wondering thanks for replying
-
OK, so in this instance the only issue you have is that you need to choose your preferred start point - www or non www.
I would add a bit of code to your htaccess file to point to your preferred choice. I personally prefer a www. domain. Something like the below would work.
RewriteCond %{HTTP_HOST} ^example.com$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]As your site is already indexed I would also for the time being and as more of a safety measure add canonicals to the pages that point to the www. version of your site.
Also if you have a Google Search Console account, you can select your prefered domain prefix in there. this will again help with your indexation.
Hopefully I have covered most things.
Cheers
Tim
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Script must not be placed outside HTML tag? If not, how Google treats the page?
Hi, We have recently received the "deceptive content" warning from Google about some of our website pages. We couldn't able to find the exact reason behind this. However, we placed some script outside the HTML tag in some pages (Not in the same pages with the above warning). We wonder whether this caused an issue to Google to flag our pages. Please help. Thanks
White Hat / Black Hat SEO | | vtmoz0 -
Canonical tags being direct to "page=all" pages for an Ecommerce website
I find it alarming that my client has canonical tags pointing to "page=all" product gallery pages. Some of these product gallery pages have over 100 products and I think this could effect load time, especially for mobile. I would like to get some insight from the community on this, thanks!
White Hat / Black Hat SEO | | JMSCC0 -
Are links on a press page considered "reciprocal linking"?
Hi, We have a press page with a list of links to the articles that have mentioned us (most of which also have a link to our website). Is there any SEO impact with this approach? Does Google consider these reciprocal links? And if so, would making the links on the press page 'nofollow' solve the issue?
White Hat / Black Hat SEO | | mikekeeper0 -
The differences between XXX.domain.com and domain.com/XXX?
hi guys i would like to know which seo value is better? for example if i would put a link in xxx.domain.com or domain.com/XXX which one will give me a better seo value? does it give the same? assuming that domain.com have a huge PR RANK itself. why do people bother making XXX.domain.com instead? hope for clarification thanks!
White Hat / Black Hat SEO | | andzon0 -
Creating duplicate site for testing purpose. Can it hurt original site
Hello, We are soon going to upgrade the cms to latest version along with new functionlaities - the process may take anywhere from 4 week to 6 weeks. may suggest - we need to work on live server, what we have planned take exact replica of site and move to a test domain, but on live server Block Google, Bing, Yahoo - User-agent: Google Disallow: / , User-agent: Bing Disallow: / User-agent: Yahoo Disallow: / in robots.txt Will upgrade CMS and add functionality - will test the entire structure, check url using screaming frog or xenu and move on to configure the site on original domain The process upgradation and new tools may take 1 - 1.5 month.... Concern is that despite blocking Google, Bing & Yahoo through User agent disallow - can still the url can be crawled by the search engines - if yes - it may hurt the original site as will read on as entire duplicate or is there any alternate way around.. Many thanks
White Hat / Black Hat SEO | | Modi1 -
How does Google decide what content is "similar" or "duplicate"?
Hello all, I have a massive duplicate content issue at the moment with a load of old employer detail pages on my site. We have 18,000 pages that look like this: http://www.eteach.com/Employer.aspx?EmpNo=26626 http://www.eteach.com/Employer.aspx?EmpNo=36986 and Google is classing all of these pages as similar content which may result in a bunch of these pages being de-indexed. Now although they all look rubbish, some of them are ranking on search engines, and looking at the traffic on a couple of these, it's clear that people who find these pages are wanting to find out more information on the school (because everyone seems to click on the local information tab on the page). So I don't want to just get rid of all these pages, I want to add content to them. But my question is... If I were to make up say 5 templates of generic content with different fields being replaced with the schools name, location, headteachers name so that they vary with other pages, will this be enough for Google to realise that they are not similar pages and will no longer class them as duplicate pages? e.g. [School name] is a busy and dynamic school led by [headteachers name] who achieve excellence every year from ofsted. Located in [location], [school name] offers a wide range of experiences both in the classroom and through extra-curricular activities, we encourage all of our pupils to “Aim Higher". We value all our teachers and support staff and work hard to keep [school name]'s reputation to the highest standards. Something like that... Anyone know if Google would slap me if I did that across 18,000 pages (with 4 other templates to choose from)?
White Hat / Black Hat SEO | | Eteach_Marketing0 -
Include placename in URL, or not?
Hi Mozzers, I'm wondering whether to put placename in URL or not. This is for a hotel so it's very focused on the county. I have loads of sub pages along the lines of www.hotelname.com/short-breaks-somerset www.hotelname.com/eat-out-somerset and so on but I was wondering whether that placename element would help or hinder. For example, may want to rank for short breaks in other searches (not just those seeking short breaks in Somerset) and was wondering whether the somerset bit may actually hinder this in the future. Also noticed Somerset is mentioned in nearly all of the page urls through the site. Perhaps this is a bit spammy and just not neccesary. I can include the address of the hotel on every page anyway. What do you think? Thanks in advance for your help 🙂 Luke
White Hat / Black Hat SEO | | McTaggart0 -
The Link in Profile Page is it good BackLink or not?
Well, i see that we need 200mozpoints to be able to put our Website Link (DOFOLLOW) into our profile in SEOMOZ.. the way i know it, it would be a good BackLink for my site. Here is the questions, please do answer from top to bottom, because if you have answered "NOT GOOD" for the first question, then the rest of the question will definitely be "NOT GOOD" too Every single back link source i used below (for question #2 and #3), comes from a good domain (it is an extremely wellknown website in Indonesia) 1. Is the DOFOLLOW link from my SEOMOZ Profile Page , a good back link? 2. is the DOFOLLOW from http://www.indonesiaindonesia.com/m4g1c14n a good back link 3. is the DOFOLLOW from http://www.kaskus.us/member.php?u=10407 (click the Contact Info), a good back link? okay, only if you answered the first 3 questions with "It is a good backlink, and it will definitely help your SEO Standing for your site", then i ask you my real question.. i was planning to use the service from http://www.monsterbacklinks.com , and i asked them to show me what kind of "High Quality Backlink" they will be giving me, here is their reply, 10 examples of profile they use to backlink to one of their client Domain PR 4--http://www.sanramon.org/user/12548
White Hat / Black Hat SEO | | IKT
Domain PR 5--http://extratasty.com/profile/42069/paulc4312
Domain PR 5--http://www.bug.co.uk/forums/members/paulc4312.html
Domain PR 5--http://www.offspring.com/forums/member.php?u=84973
Domain PR 5--http://www.massify.com/profiles/paulcpaul
Domain PR 6--http://www.gamezone.com/member/159751/
Domain PR
5--http://www.indyarocks.com/profile/profile_vview_main.php?uid=6155724
Domain PR 6--http://classic.mapmywalk.com/user_profile?u=866130762956343886
Domain PR 5--http://www.netbookreviews.com/forum/members/paulc4312.html
Domain PR 5--http://www.thepoint.com/users/paul-c-2/profile
Domain PR 5--http://forums.cagepotato.com/members/paulc4312.html In my eyes, all of those links are as good as the one link coming from SEOMOZ Profile, hell in fact i have already purchased from them the 750 High Quality BackLink package (cost 197$), but my PayPal is being lock down just now, because i login to my account from both my cellphone and pc (they think my account is hacked)... so will i increase my SEO Standing if i used their service? if they are, i will finalized my purchase tomorrow (after i settled the problem with paypal) Their FAQ Page is also very convincing .. such these 2 questions Will I get penalized for paying you to do my backlinks? There is no way you will get penalized for paying us to do your backlinks. It is possible to get penalized for paying people to put links on their sites but that's not what you're buying from us. When purchasing from us you are paying us to place thousands of free backlinks. There is absolutely no way Google can penalize you for this. Will Google ban/sandbox me for getting so many backlinks? We have never had any problems with getting sanboxed or banned by google. None of our customers have had any problems either. If our methods of placing backlinks were to get a site penalized or banned then we would be sending thousands of links towards our competitors sites. But since our methods work great for increasing search engine rankings, we would never use our backlinking on our competitors because that will damage our rankings and boost theirs. Please enlighten me 🙂0