Since folks are angling a bit for some hardcore SEO knowledge, I thought I'd take a stab at the thorny and often unproductive process of researching keywords in the long tail. The tail of search queries in a given industry is typically not visible via any of the major KW research programs or search engine ad databases (Overture, Google, MSN). In these instances, there is a research method to find those terms that can carry value, but it requires either a good amount of legwork or a talented programmer.

Process for Finding Long Tail Terms:

1. Extract the top 10-50 most common search phrases at the head of the curve from your existing KW research in the industry
2. Search Google, Yahoo! and MSN for each term
3. For each page in the top 10-30 results, extract the usable text on the page
4. Remove stopwords and filter by phrase size
5. Remove instances of terms/phrases already in your KW research databse
6. Sort through the most common remnants first, and comb as far down as you feel is valuable

Through this process, you're basically text mining relevant documents on the subect of your industry/service/product for terms that, while they may not have high search volume, have a reasonable degree of relation. Obviously, it's imperative to have human eyes reviewing the extracted data, but it's hard to find better source material for long tail targeting. You may even find additional terms at the head of the keyword curve.

How can this method be expanded upon?

• Use Technorati or Del.icio.us for your results (MSN and Ask, too)
• Use documents purely from specific types of results - local, academic, etc.
• Mine forum threads on your subject matter - you could even use inurl:forum(s) in the searches to grab conversational keywords

There's nothing uber-secretive about this methodology, but it is highly effective. The return on this research has a direct relationship to the amount of effort you expend (and how far you dig).

Any other suggestions/ideas for long tail term mining?