Any tools for scraping blogroll URLs from sites?
-
This question is entirely in the whitehat realm...
Let's say you've encountered a great blog - with a strong blogroll of 40 sites.
The 40-site blogroll is interesting to you for any number of reasons, from link building targets to simply subscribing in your feedreader. Right now, it's tedious to extract the URLs from the site. There are some "save all links" tools, but they are also messy.
Are there any good tools that will
a) allow you to grab the blogroll (only) of any site into a list of URLs (yeah, ok, it might not be perfect since some sites call it "sites I like" etc.)
b) same, but export as OPML so you can subscribe.
Thanks!
Scott
-
Not at all. I guess my feeling here is that there is a sort of untapped social graph defined by blogrolls. If it were simple to harvest them upon visiting a blog (e.g. this blogger recommends...) one could do a stumble-on-steroids approach to a niche.
-
I thought you might be able to use the outbound link scraper to grab the outbound link onto the page. Pop in your URLS of the pages you want to scrape and it will spit out our a list of those domaind and urls. You can take those urls and put them into the contact finder and it will return the contact details for those sites. Combine the two spreadsheets for an epiuc list of blogs to contact for your outreach.
This is obviously for link building rather than subscribing - sorry if I have misunderstood what you were trying to do
-
Hi Keri,
That is a very cool tool, but is overkill for this. It takes far too many steps to accomplish only part of the desired goal of grabbing all blogroll URLs (within the blogroll DIV tag) and exporting the list to a valid OMPL file or URL list.
thanks!
-
nothing I saw there would do this. It looks like it could manage to list all external links, and I suppose you could manually pick the blogroll out of it.
-
Hi there,
Well, Keris response reminded me of this question and the fact that I found a tool for scraping these kind of lists:
Here it is (with some other cool tools) , have fun:
-
Hi Scott,
I'm going through older questions. Did you ever find a tool to do what you wanted to do here?
-
One thing to look at is Outwit Hub for Firefox. It might be able to help with that. It can scrape data from a page and do a lot with it. http://www.outwit.com/products/hub/. Don't know that it meets all of your needs, but I also haven't seen a response with anything better at the moment.
-
Hey Scott,
What a great question and <sigh>I don't have the answer. I am going to back to find out what people come up with here. Surely there is someone that lurks these parts that can throw something together?</sigh>
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
MoZ & Other Keyword Tools
Hi There, 1- Does MoZ provide the data on “which keyword searchers are searching in a particular market”, so that I can make a better decision regarding my own keywords?If MoZ does not do this what other best tool is out there in market for this particular purpose? 2- Suppose I decide some keywords for my site and start doing optimization for them. "Does MoZ tell me “how much traffic my those keywords are receiving with the passage of time”. If MoZ does not do this what other best tool is out there in market for this particular purpose? 3- Does Moz give "data on competitors keyword activity also"? If MoZ does not do this what other best tool is out there in market for this particular purpose? Hope somebody with concrete knowledge and experience will enlighten me. Cheers Tanveer
Moz Pro | | Sequelmed0 -
Now our site is ready what's next
Site is all inxed in google, with analytics and webmaster tool. No error in all our 5000 products pages So where do i start my organic seo? What should I read first? Its an online store for retailler only but we still want people to learn about the product or find a store.
Moz Pro | | koloss
We have made 300 original video on youtube with a full description these video are also associated in our site in each respective product. I know our competitor. So now everything is clean and ready. How do I start seo, we have no external link at all. Wich would be the first best step to do. Regards, Bruce0 -
How can I download all the inbound links to my Site?
Hi, From Site Explorer, I want to download all the inbound links linking to my site: www.comm100.com/livechat/. But it seems that there is a limitation. I can only download a part of the inbound links? What should I do? Or is it because my account is free trial? Do I have to pay for it? Please help. Thanks,
Moz Pro | | Sophia_M0 -
Open Site explorer
I checked some of the results with OSE that ranked top on Goolge SERPS that have high Domain Authority,Page Authority 1 and content on the website's particular page is thin. How come they rank high when PA is very low?
Moz Pro | | Frost0 -
Adding LinkedIn to the new Social Media tool
I am loving the new Social Media data that SEOMoz recently added. I am sure more will come soon, but I wondering if they have plans of adding LinkedIn Company pages as apart of a campaign to track. Does anyone have the inside clue about this? Do you think it would be a good idea as well?
Moz Pro | | nextraq0 -
Is the Open Site Explorer Tool working right now?
Is the Open Site Explorer Tool working right now? I keep getting directed to a sign-up page.
Moz Pro | | LarryEngel0 -
How to set up SEOMOZ to track multilanguage sites?
Hi, I am managing a site in 5 different languages/regions. Our language structure is as follows: www.domain.com/en-us/ - For US www.domain.com/en-uk/ - For UK www.domain.com/fr/ - For France ... However, SEOMOZ does not allow to set up a domain with "/" as a campaign. How can I do it? Thanks
Moz Pro | | hockerty1