Around two thirds of trackable web traffic comes from search engines, whether from paid listings (paid search or PPC) or organic/free listings (organic search or search engine optimisation - SEO). The chances are that search represents a very meaningful source of online traffic, leads and customers for your business – even if it is at the start of a long B2B purchase journey.
It follows that when you are setting your digital marketing strategy, you should be seeking to understand your potential opportunities for growth within paid and organic search, and then tracking changes on an ongoing basis.
Search engines helpfully provide a lot of information of advertisers on the performance of their paid search activities, but this is not the case in organic search meaning that marketers must rely on third party tools to track their SEO rankings.
In our experience, organic search is overlooked in terms of its commercial importance to most organisations, receiving much less Management time, fewer metrics in the board pack and insufficient investment than other marketing channels. Your SEO rankings can often account for 50% of new customer acquisition once you have clear picture of marketing attribution, at a very attractive Cost Per Acquisition CPA compared to other channels.
This lack of attention results from a combination of the difficulty of measuring/tracking the value of your SEO rankings and the (misplaced) idea that organic search traffic is both hard to influence and in terminal decline, so why spend time focusing on it. A Search Headroom analysis that is based on your SEO rankings can help change some of these perceptions and help to bridge the gap between senior managers and technical SEO practitioners.
What is a Search Headroom analysis?
A search headroom analysis highlights a business’s share of its potentially addressable organic search traffic at a specific point in time. The higher up your business appears in the organic search rankings for any given keyword, the greater your share of traffic will be.
You can think of this like a digital version of a traditional ‘market share’ analysis. The difference is that traditional market share is based on the ‘stock’ of customers in a market (e.g. a car manufacturer’s share of all the cars on the road today), where as a search headroom analysis considers the ‘flow’, the customers who are actively searching for a given product or service (e.g. a car manufacturers share of the new cars sold this year).
Businesses that are growing will often have a higher share of search traffic than their overall market share – hence this can be a valuable leading indicator of growth.
In principle there are a handful of steps to follow when creating a Search Headroom analysis:
Build a list of your own web domains and those of your competitors;
Use a third-party tool to produce a list of all the keywords where each domain appears in the search results, and their respective SEO rankings;
Aggregate the results together and remove duplicated keywords, irrelevant keywords, and those where the ranking is so far down the results, they are unlikely to generate any traffic;
Combine with data on the overall monthly searches for each keyword;
Translate the rankings into estimated traffic for each domain on each keyword (considering both the ranking and other search results page (SERP) features which could impact click-through rate (CTR));
Group the keywords into common sense segments for your product/service;
Explore the results at a segment, keyword and even landing page level; and
Review the search results for your most important keywords to check for any additional competitor domains to include the next time you update the analysis.
Figure 1 - example summary from a Search Headroom analysis showing 'market share' by domain and keyword segment.
A Search Headroom analysis is a very powerful tool as it can be built both ‘outside in’ using 3rd party providers of search results tracking, as well as being enhanced with more accurate internal data, for example when estimated CTRs. We have used this approach both as operators and investors as a result, to understand a market overall, dig into competitor strategy or track who is gaining/losing share. You can also apply this methodology in other channels e.g., Amazon.
A Search Headroom analysis is also very useful when you are planning big changes in your digital marketing strategy – launching a new website, re-platforming an existing website, or planning a domain consolidation. All these changes can cause significant and immediate change in your SEO ranking that you need to carefully monitor and mitigate.
What digital marketing insights can a Search Headroom analysis generate?
Understanding your business’s search headroom can yield many interesting insights about your business, your competitors, and your market. For example:
Your ‘market share’ of organic search traffic, and how this varies by segment and keyword
Trends in your ‘market share’ over time
Level of fragmentation/concentration of traffic in your market
Seeing where your competitors are winning traffic, but you are not
Growth YoY in terms of search volume (10 years ago it seemed like every keyword was going in volume terms, but overall search volumes are now relatively stable, so growth in searches tends to correlate with overall market growth))
The level of volatility in your market i.e., how often SEO ranking changes
How commercial/sophisticated the digital marketing strategies are in your market i.e., understanding the mix of organic vs paid traffic (search ads and shipping ads in some categories)
Where you are doing well in organic search but not paid search and vice versa, by comparing the Search Headroom analysis to your paid search data
How overlapped your market is with other markets that may have similar search terms – think about a market like cyber security where you will find many different overlapping niches as well as job seekers and students searching very similar keywords
Seeing your SEO rankings at keyword level (where do you rank vs. where you ‘should’ rank based on the product/service offering of your business)
Where you may have recently lost high volume SEO rankings
Whether your SEO/ content team are spending time in the right areas to both protect your most valuable SEO rankings and grow your visibility in the areas of biggest opportunity
Comparing the performance of your different landing pages (and those of your competitors)
Figure 2- example keyword level market share from a Search Headroom.
What makes this difficult?
Whilst the benefits of a Search Headroom analysis are hopefully clear, this is something that many businesses have never attempted. One of the challenges in the complex, technical nature of search marketing, and in particular SEO. In our experience, many talented, technical SEO professionals don’t think about top-down opportunity enough and for most management teams SEO is the ultimate ‘black box’ where cause and effect are very hard to understand. The closest teams often get to quantitative reporting of their search marketing is a page in the board pack listing their top 10 keyword rankings.
There are a few other factors that make producing a Search Headroom analysis hard:
The (very) long tail matters – previously businesses I’ve worked with have generated as much as 90% of their organic search conversions from keywords with fewer than 100 monthly searches. You will likely need a dataset with thousands or tens of thousands of keywords, potentially more if you are an established business in a large market e.g., online travel. This means that completing the Search Headroom analysis need more advanced analytical skills which your team may not have.
Most SEO tools – and there are lots – track a selection of SERPS and produce various metrics like ranking, perhaps even estimated traffic, but this is rarely out in the context of overall category and certainly doesn’t highlight opportunities and risks – so you need to do your own analysis to produce an overall view of your ‘market share’ as well as being able to drill down. To get a complete view of your SEO rankings you may need to combine data from multiple tools.
There are many features that can appear on the search results page and influence the click-through rate for any given ranking. The number of paid search ads, shopping ads, maps, and featured answers all play a role – you ended to look carefully at your own data to make sensible estimates for each potential scenario, again adding to the analytical complexity. The recent emergence of generative AI is going to lead to a lot of change in the SERPs over the next couple of years which will only add to this complexity.
Deciding how to segment your keywords can be subjective and requires some test & error – how many groupings to create and how to define them. In our experience, we find it helps to remember that not all traffic is equal in terms of its likelihood of converting, so we will create segments that differentiate by level of intent e.g., whether a search includes a high purchase intent word like ‘buy’, ‘compare’ or ‘reviews’. We will also always create a segment for branded terms as these behave very differently with very high click-through rates on your own brand terms.
The changing search engine landscape – this will vary by business and geography, but Google has c.85% share globally, and Bing has been gaining share and has reached 8%. For now, if you build your Search Headroom analysis based on Google you will get an accurate enough answer, unless you are focused on one of the handful of markets where Google is not the outright market leader (e.g. China, South Korea).
Novel products / services with limited directly relevant search traffic - I’ve worked with businesses at the vanguard of a new category where consumers are not yet searching explicitly for their product or service in high volumes. This means a Search Headroom analysis will typically show the business as having a very low share of some large, adjacent keyword categories – which isn’t especially actionable in the short term. In these cases, we narrow down the keyword focus to the handful of directly relevant keywords but monitor closely for new keywords which will be likely to appear every month.
How can I create my own Search Headroom analysis?
None of these difficulties should prevent you from undertaking a Search Headroom analysis – the insights you can generate will be truly insightful, helping you to both spot opportunities and manage risks.
At Coppett Hill, we've created our own tool to create Search Headroom analyses for our clients, "Searchscope". We've combined our experience of SEO across many different industries and geographies with proprietary AI to rapidly produce actionable insights that can be updated every month. This saves our clients considerable time and effort in understanding this critical area on an ongoing basis.
If you’d like to discuss how you can use your SEO rankings to use our Searchscope tool to create a Search Headroom analysis for your business, please Contact Us.
All views expressed in this post are the author's own and should not be relied upon for any reason. Clearly.