(Authors: Andres Torrubia, Antonio Parraga and Hector Rodes)

The Problem with Classic Keyword Research

When tasked with defining a keyword strategy for a given site, most SEOs will likely choose just a handful of keywords based on keyword research.  

If you are in a niche market, this approach may be perfectly fine but when you have a site that plays in many different categories, you can quickly end with hundreds of individual long-tail keywords to rank for and you need a streamlined way of prioritizing which keywords are more attractive.

This post explains a method to prioritize which keywords you should focus your SEO efforts on by calculating its SEO Appeal

Assessing Keyword Difficulty

SEOmoz provides tools to help in SEO: One of them is the Keyword Difficulty Tool  (https://mza.seotoolninja.com/keyword_difficulty) (we are confessed lovers of this tool), which returns a numeric score from 0% to 100% expressing how difficult is to rank for a given keyword.

Difficulty is a good measure to bring us an idea about our capacity to rank for a keyword or the next one, but at the end of the day is not just what to rank for, but how much traffic a given keyword yields. It doesn’t make any sense to rank for keywords that nobody is searching for: How difficult is to rank for the keyword “estritukol kalkulati”? According to SEOmoz tool, this may be the easiest one: 0% of difficulty. 0 results in SERP. Now my reflection is: how many people are searching for “estritukol kalkulati”? 

Any SEO would rank this keyword at the first (and the only one) position. A single H1 “To kalkulati or not to kalkulati, this is the estritukol” in a free-hosting service may be enough, yes.

This is the reason because we need to introduce another variable into the equation: traffic. 

There are a myriad of services that would estimate SERP traffic on a per-keyword basis.  For the purposes of this article, the traffic value can be expressed in number of searches for a given time period.  As long as the traffic values are consistent and linear (e.g. searches per month) we do not care what they are expressed in.

Establishing a Relationship Between Difficulty and Traffic

Which is the best keyword to promote: an easy one with a few searches and no competition, or a heavily competed one driving million of searches? In an ideal world, a keyword with a 0% difficulty may give us the total number of searches in SERP traffic since we’re going to be alone in the SERP. Conversely, a keyword with a 100% difficulty rating will not give us a single click since we’re not going to rank at all for that keyword. For the purpose of this discussion, let’s assume a 100% difficulty means “impossible, no SERP traffic attainable from this keyword” (note that in practice no keyword is going to have 100% difficulty), and 0% means nobody ranks today for that given keyword (hence it should be easy to rank for).

Although difficulty is expressed as a percentage, its calculation includes non-linear elements (e.g. pagerank, page strength of the 10 top results, etc.), and as such a difference in difficulty between 10% and 20% does not represent the same effort to rank for as a difficulty gap between 20% and 30%. 

Rank Effort

We cannot use keyword difficulty to prioritize the keywords that we are going to rank, because it does not account for the actual strength that we currently have for that given keyword.

To illustrate this, imagine the keyword “obama president” which returns a difficulty of say 80%.  The effort that a relevant page at an authority domain (such as the New York Times or the White House ) would have to make to rank for that keyword is significantly less than the effort that a less authoritative domain would (assuming in-page optimization is equivalent in both cases).

We define rank effort as the relative effort (from 0% to 100%) that a given page would have to make to rank for a given keyword accounting for its actual strength.  

We have empirically tested various curves to model this effect, and we have found that a Gaussian curve with variable standard deviation (σ) based on your site’s (or target URL) strength maps keyword difficult into rank effort nicely:

The more value the σ has the more the curve fits into a line, so difficulty values and rank efforts will be more aligned (e.g. the blue curve above). The less value the σ2 has the more the curve fits into an squared “S”, so rank efforts will be strongly low near to 0% for values under the 50% of difficulty, and the opposite way. 

The σ value should be ideally proportional to the strength of the page to promote for the given keyword. Weak pages must use a curve like the red one to calculate the rank effort since they are not going to rank for keywords with high levels of difficulty while strong pages may feel free to use a curve like the blue one since they are in a position to rank for high difficulties. We should be able to tune the σ value according to the strength of each page individually that we want to rank each keyword for.

According to that, the rank effort could be described by a cumulative distribution function:

In this equation, x is the keyword difficulty. μ (the mean) is 50 since we are limiting our values from 0 up to 100%.

SEO Appeal

Finally, the SEO Appeal could be calculated as the traffic reduced by the rank effort percentage:

Now, let’s put all together in a practical example: 

After a keyword discovering session for say faucet plumbing and related keywords, we have realized that our most interesting keywords are 5: repair faucet, fix faucet, faucet repair, faucet plumbing and plumbing faucets.

After applying the rank effort formula, our results are as follows (assuming a σ2 3.1):

Should we focus on keyword difficulty exclusively, the easiest one would be fix faucet. However, from the traffic standpoint fix faucet is quickly superseded by repair faucet or faucet repair.  However, they are also more difficult to rank on. After applying the SEO Appeal formula, the perspective changes: the most interesting keyword is “repair faucet”. The next one in this order is “fix faucet” (surprise! “faucet plumbing” is the third one, not near to his terms brother). “fix faucet” has much less traffic than “faucet repair”, however the difficulty difference between both of them justify a greater SEO Appeal associated to “fix faucet”, which is intended to bring more traffic at the end of the day. 

For the last one, “plumbing faucets”, even the good amount of traffic it has, 0 for SEO Appeal means that we are not in a position to promote it by now.

All those values change according to σ. The more the σ value is the more the strength of our page is and, in consequence, the more we’ll be in a position to promote high difficulty keywords while low difficulty keywords will lost interest for us. And again: always guided by the traffic.

NOTE: This article does not take into account factors like landing page conversion, we know: not the same to sell private jets or to sell oranges. This article is just focused in terms of traffic.