Any tools for scraping blogroll URLs from sites?
-
This question is entirely in the whitehat realm...
Let's say you've encountered a great blog - with a strong blogroll of 40 sites.
The 40-site blogroll is interesting to you for any number of reasons, from link building targets to simply subscribing in your feedreader. Right now, it's tedious to extract the URLs from the site. There are some "save all links" tools, but they are also messy.
Are there any good tools that will
a) allow you to grab the blogroll (only) of any site into a list of URLs (yeah, ok, it might not be perfect since some sites call it "sites I like" etc.)
b) same, but export as OPML so you can subscribe.
Thanks!
Scott
-
Not at all. I guess my feeling here is that there is a sort of untapped social graph defined by blogrolls. If it were simple to harvest them upon visiting a blog (e.g. this blogger recommends...) one could do a stumble-on-steroids approach to a niche.
-
I thought you might be able to use the outbound link scraper to grab the outbound link onto the page. Pop in your URLS of the pages you want to scrape and it will spit out our a list of those domaind and urls. You can take those urls and put them into the contact finder and it will return the contact details for those sites. Combine the two spreadsheets for an epiuc list of blogs to contact for your outreach.
This is obviously for link building rather than subscribing - sorry if I have misunderstood what you were trying to do
-
Hi Keri,
That is a very cool tool, but is overkill for this. It takes far too many steps to accomplish only part of the desired goal of grabbing all blogroll URLs (within the blogroll DIV tag) and exporting the list to a valid OMPL file or URL list.
thanks!
-
nothing I saw there would do this. It looks like it could manage to list all external links, and I suppose you could manually pick the blogroll out of it.
-
Hi there,
Well, Keris response reminded me of this question and the fact that I found a tool for scraping these kind of lists:
Here it is (with some other cool tools) , have fun:
-
Hi Scott,
I'm going through older questions. Did you ever find a tool to do what you wanted to do here?
-
One thing to look at is Outwit Hub for Firefox. It might be able to help with that. It can scrape data from a page and do a lot with it. http://www.outwit.com/products/hub/. Don't know that it meets all of your needs, but I also haven't seen a response with anything better at the moment.
-
Hey Scott,
What a great question and <sigh>I don't have the answer. I am going to back to find out what people come up with here. Surely there is someone that lurks these parts that can throw something together?</sigh>
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Working with Open Site Explorer
Hi everyone, I'm new to keyword analysis, and am in the process of consuming a lot of SEOmoz articles and resources on the subject. I wanted to see if I'm correct in my analysis of two compared sites, and hope you can shed some light on the matter. I've been to the Google Keyword Tool and looked for my informational keywords for the project I'm working on, since the user intent is all about information. A not-so-great keyword phrase I've found with 12,100 local monthly searches is: "programa de inglês" (english programme) I'm just using this as a quick example. I have performed a Google search query for the above phrase from google.com.br (Brazil), and I'm comparing the #2 and #4 results from the 1st page of the SERPs which are: #2) www.programa-ingles.net and #4) http://www.baixaki.com.br/categorias/educacao-e-diversao.htm. What's confusing me is that in Open Site Explorer, the #4 result gets a much higher page authority compared to the #2 result, and beats #2 on every category except for internal–external link ratio and all the social categories. Here's an image attached of the comparison. Is it the fact that the external links of #2 account for 100% of the links pointing to it, or that the #2 position beats (rather pitifully) #5 on social sharing, or is it something that I've not stumbled across yet? Thanks in advance for helping out a n00b. 6HU6hBi.png
Moz Pro | | featherseo0 -
Open Site Explorer Link Metrics
I'm new to SEOMOZ. I have a Pro account and have been messing with the tools available. I ran a report in Open Site Explorer on one of my prospects. I also entered in several of their competitors to see how they rank. I would like to show the report to my prospective client but couldn't figure out how to download the information. Is that info available for download? Does anyone have a recommendation for showing prospects how they rank against their competitors. Any help appreciated.
Moz Pro | | MedGroupMedia1 -
Why does the Keyword Difficulty Tool shows no Search Volume?
I've run a full report in many keywords on the Keyword Difficulty Tool, but in the Search Volume field all of them have the message "not enough data". Screenshot attached. Why is that? I have keywords like "magnesium" that popular and have quite some traffic. Any help is appreciated, thanks! semoz-keywords-analytics.jpg
Moz Pro | | bernardovailati0 -
What happened to the SEOmoz Term Extractor Tool?
I am looking for the SEOmoz Term Extractor Tool and it's nowhere to be found. Does it exist anymore? If not why and what would be a good alternative tool to use? Thank you.
Moz Pro | | brianhughes0 -
Link Assistant SEO Tool - Any Others?
I have the most boring copier service website, clients cool, concept is really boring; Well I am looking to generate this client some link back juice, cause this guy is struggling for traffic, and I am am struggling to generate a buzz about copier services?? Fortunately, it's only local, so that helps me with testing my localization campaigning. 🙂 Any opinions on this software? http://www.link-assistant.com Do you know of some better "Back Linking Software? I was considering buying it. But once I go to purchase it, I second guess myself; "It just looks shady, and the perpetual up-sale, turns me off.". I really just value the MOZer communities opinions! and I want to see what the community thinks of the above company, and if ya heard anything good, or bad. The Demo seems legit as hell!
Moz Pro | | smstv0 -
How to force a recrawl of a site?
Hi, I made changes in my site. I would like to see the result of the crawl diagnostic. I know the crawl is happening every week, however, is there a way to force a re-crawl in order not to have to wait 5 days? Cheers,
Moz Pro | | nuxeo0 -
Which tools do not use backlinking and/or tagging?
I am new to SEOmoz and was wondering if any, which tools on SEOmoz does not use or require backlinking and/or tagging?
Moz Pro | | aschraegle0 -
Page Rank and offline sites
I have a domain with PR6 according to the Historical Pagerank Checker. But that last PR was calculated 2 years ago. I brought the site back online a few days ago and have checked that many/most of the backlinks are still valid. It is now in the Google index but the Historical Pagerank Checker shows PR0. Will it get back its previous rank or something close to it? How long will it take?
Moz Pro | | DomainOptions0