Tool for scanning the content of the canonical tag
-
Hey All,
question for you. What is your favorite tool/method for scanning a website for specific tags? Specifically (as my situation dictates now) for canonical tags?
I am looking for a tool that is flexible, hopefully free, and highly customizable (for instance, you can specify the tag to look for). I like the concept of using google docs with the import xml feature but as you can only use 50 of those commands at a time it is very limiting (http://www.distilled.co.uk/blog/seo/how-to-build-agile-seo-tools-using-google-docs/).
I do have a campaign set up using the tools which is great! but I need something that returns a response faster and can get data from more than 10,000 links. Our cms unfortunately puts out some odd canonical tags depending on how a page is rendered and I am trying to catch them quickly before it gets indexed and causes problems. Eventually I would also like to be able to scan for other specific tags, hence the customizable concern. If we have to write a vb script to get it into excel I suppose we can do that.
Cheers,
Josh
-
No idea on that one - it's still pretty new. The developers actually chimed in on the post, so you could ask them in the comments.
-
Thanks Dr. Pete and Marcus.
I just finished reading the post. I have looked at Screaming Frog before but was hoping to be able to find a way to do it myself. Just didn't want to plop money down on something that seemed like it should be able to be done using tools I already had. But the software does look good. Any thought on if they will come out with a one time purchase instead of a yearly subscription?
Cheers!
Josh
-
Hey Dr. Pete, Joshua
I was just coming here to say that I had read the Dr. Pete post and this may do the job. It's a paid bit of a software but I will be picking it up later. I have my guys knocking up a canonical checker that will be free for all but that may take a day or so to get perfect.
Let me know if you have a play with Screaming Frog!
Marcus
-
I'm pretty sure that Screaming Frog SEO Spider will do it, but you need the paid version to custom-filter on the canonical tag. I've got a post going up about it tomorrow.
-
Great, really appreciate it! Many thumbs up
-
Hey Josh,
Right, cool. I have got a few jobs to sort out but I am going to have a bash at knocking this up this afternoon. Should be easy enough (he said, damning himself to hours of problems).
Leave it with me for 24 hours.
Marcus
-
Hey Marcus,
thanks for the quick response. That is exactly what I would be looking for. I do have a list of url's and that is also simple enough to get from something like xenu. Would love to work with you on this.
Thanks.
Josh
-
Hey, I am not aware of any such tool, but it should not be too hard to put one together, maybe a useful little tool as well.
If you have all of your pages in spreadsheet or database, it should be easy enough to write a little script that cycles through them.
Start Loop
-
request page
-
parse code to get canonical URL
-
compare page to canonical
-
output problem URLs
End Loop
Slightly over simplified and requires a list of all your URLs but would be willing to help put something like this together, could be useful for all of us, especially for those (like me) that work with a lot of CMS sites.
Cheers
Marcus
-
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When Should I Ignore Moz's Report Canonical Missing?
I'm dealing with an eCommerce website which has a category, subcategory, products. Moz is showing all of these and the individual products as missing a canonical. The site is very thin on content at the moment, but all the pages are clearly different, and I don't see why they need a canonical unless this is some rule that eCommerce sites have to follow. Should I ignore Moz's missing canonical report? My understanding is if the product appears in multiple categories, then a canonical should be put in place to the product. Any advice would be appreciated. Christina
Moz Pro | | ChristinaRadisic0 -
Duplicate content report - question on best practice
Hello all, New to MOZ Pro and SEO - so lots to get my head round! I’m working through the Duplicate Content section of the Crawl report and am not sure what the best practice is for my situation. Background: We are a reference guide for luxury hotels around the world, but the hotels that are featured on the site vary year on year. When we add a new hotel page, it sets up the url as ourwebsite.com/continent/country/regionORcity/hotel. When the hotels come off, I redirect their URL to the country or region where we have other hotels. Example: http://www.johansens.com/europe/switzerland/zermatt/ The hotel in Zermatt has come off the site, showing 0 results on this landing page. Question: My duplicate content report is showing a number of these regional pages that are displaying the copy “0 places - Region’ because the hotel has come off, but the landing page is still live. Should I redirect the regional page back to the main country page? And then if I add a new hotel to the site from that region in the future, simply remove the redirect? Should I also delete the page? Any tips would be much appreciated!
Moz Pro | | CN_Johansens0 -
Duplicate Content: Marketing Page / Content Page
So I am getting duplicate content warnings on my website for my pages white paper and webinar video pages. Each white paper / webinar video page is behind a marketing form page that must be filled out. I am getting a lot of warnings that the marketing page and the content page are being picked up as duplicated content. In the past, both the marketing page and the content page were given the same title and url, the body content is not similar. My question: Is the URL / Title similarity enough to set off the duplicate content warnings and would changing one or the other solve the issue?
Moz Pro | | AllMedSeo0 -
Mac Alternatives for Netpeak & SEO Tools For Excel?
Does anybody know of any mac alternatives for Netpeak & SEO tools for excel? I haven't been able to find any. I just need something to pull PA & DA quickly for a list of domains and URLs. Will I just have to create something custom with the Moz API?
Moz Pro | | kking41200 -
Keywords Data Tool: Why is volume metrics unavailable for all of my keywords?
When I use the SEOMoz Pro Tool for Keyword Resarch, I get the notice that the tool is getting improvements. But when I run my keywords all of the volume metric data is unavailable. Why is this?
Moz Pro | | seocoppercupimages0 -
Is there a keyword suggestion tool available in the SEOMOZ suite of tools?
Is there a keyword suggestion tool available in the SEOMOZ suite of tools that is similar to semrush.com? semrush allows you to put in a URL and then will tell you what keywords you rank for. Looking for a good tool that is similar.
Moz Pro | | webestate0 -
Excel tips or tricks for duplicate content madness?
Dearest SEO Friends, I'm working on a site that has over 2,400 instances of duplicate content (yikes!). I'm hoping somebody could offer some excel tips or tricks to managing my SEOMoz crawl diagnostics summary data file in a meaningful way, because right now this spreadsheet is not really helpful. Here's a hypothetical situation to describe why: Say we had three columns of duplicate content. The data is displayed thusly: | Column A | Column B | Column C URL A | URL B | URL C | In a perfect world, this is easy to understand. I want URL A to be the canonical. But unfortunately, the way my spreadsheet is populated, this ends up happening: | Column A | Column B | Column C URL A | URL B | URL C URL B | URL A | URL C URL C | URL A | URL B | Essentially all of these URLs would end up being called a canonical, thus rendering the effect of the tag ineffective. On a site with small errors, this has never been a problem, because I can just spot check my steps. But the site I'm working on has thousands of instances, making it really hard to identify or even scale these patterns accurately. This is particularly problematic as some of these URLs are identified as duplicates 50+ times! So my spreadsheet has well over 100K cells!!! Madness!!! Obviously, I can't go through manually. It would take me years to ensure the accuracy, and I'm assuming that's not really a scalable goal. Here's what I would love, but I'm not getting my hopes up. Does anyone know of a formulaic way that Excel could identify row matches and think - "oh! these are all the same rows of data, just mismatched. I'll kill off duplicate rows, so only one truly unique row of data exists for this particular set" ? Or some other work around that could help me with my duplicate content madness? Much appreciated, you Excel Gurus you!
Moz Pro | | FMLLC0 -
Do crawl reports see canonical tags?
Greetings, I just redesigned my site, www.funderstanding.com, and have the old site pointing to the new site via canonical URLs. I had a new crawl test run and it showed a large amount of duplicate content. Does the SEO Moz crawl tool validate canonical urls and adjusts the duplicate content count or is this note considered? FYI, I sent from no duplicate content to having 865 errors since the redesign went up so that seems suspicious. I would think though that assuming the canonical tag were used properly, and I hope it is?, that this would not be a problem?? All help with this is most appreciated. Eric
Moz Pro | | Ericc220