At the beginning of the Balanced Scorecard, a book on the new generation of performance metrics, authors Dr. Robert Kaplan and Dr. David Norton  present an analogy to drive home their case. They ask you to imagine entering an airline jet cockpit, and in front of the pilot, you see just one gauge.

You ask the pilot, "What's that gauge measure?"
"Altitude", you're told.
"What about the other gauges?"
"We won't be using them this flight. I'm just focusing on altitude."
"How about air speed?"
"No, that's the gauge I was using last flight. I wanted to try something different this one."
"Not this time"
"Fuel gauge?"

The idea is, of course, that you need a balanced set of measures to accurately monitor business performance. The analogy got me to thinking about the metrics we typically see companies enter their search marketing campaigns with. And I couldn't help thinking, "At least the airline pilot got one of the metrics right!"

If it's Not Measured, It's Not Managed

After working with search marketing for 8 years, I'm amazed at the number of otherwise intelligent marketers who enter their search campaigns without a clue of how to measure the success of the campaign. The entire motivation for looking at search optimization is because the CEO had a hissy fit because they didn't rank for a broad industry term that was, at best, only marginally relevant to their actual product and service offerings. They're pretty sure they want better rankings, but they really haven't given much thought as to why.

You do search marketing for a reason, and unless your company is run by a Fascist egomaniac, the reason shouldn't be so your CEO can search for his dog's name and see your company's site rank number one. Those reasons should come down to bottom line goals that align with both your corporate and marketing strategy. Is it lead gathering, on line sales, brand building, relationship building, extending marketing share in new areas or positioning your product against the competition? All are valid objectives, and all can be aided greatly by a well thought out search marketing strategy. But, until you know the reasons, you're flying blind.

Also, I've been in the boardrooms of more than one Fortune 500 company and asked to see the research done on their customers feelings about the website and how they interact with it. Please, if you ever get a chance, do this. At first, you're greeted by blank stares. Then the sidewise glances start between the IT department and the marketing people. "I thought you were going to do that!" Finally, you get the answer you knew was coming. "Uhmmwe've been meaning to do that. But we haven't really got any research of that kind".

The IT department probably has technical stats on user sessions, pages visited, length of session and enough assorted statistics to choke a CTO, but no one has thought to ask a customer how they feel about the site.

And, we've found, if they haven't done this research on their own site, it's a lead pipe cinch that they have no idea how their customers interact with search engines. 

Decide What to Measure

Search marketing metrics have improved over the past 2 years. Back then, the only metric was ranking. The problem with this is that rankings didn't really mean anything by themselves. A high ranking was useless unless it drove traffic. And, in turn, that traffic was useless unless it translated into something that added to the bottom line of your business. That bottom line impact could be long term, as in the building of brand equity, or short term, as in an online purchase or the capture of a potential sales lead. Because search marketing was primarily used by direct marketers, it was generally the second of these that was chosen as a performance metric.

That brings us to today, as increasingly the focus turns to ROI, as measured through conversion rates. This measurement is primarily done on the sponsored side of search. Tools that allow you to accurately measure ROI on organic search campaigns are still rare and usually involve gathering data from different sources, analyzing it and, in many cases, throwing in some scientific guess work.

While the move to ROI measurement is an improvement over just measuring rankings, it's a bit like the pilot using the altitude gauge and ignoring the other instrumentation in the cockpit. The problem is, as you move beyond conversion tracking, accurate measurement of other factors becomes more difficult.

Conversions are Good, But They're Not Everything

A conversion is generally defined as a visitor taking an action that allows the website owner to establish a relationship with them. It could mean capturing contact information, having them complete an online form, or actually make a purchase. Conversions are defined on a case by case basis.

Most of the better search engine marketing companies now offer some type of conversion tracking on sponsored search campaigns, and a few (like Enquiro) also offer it on organic search optimization campaigns. It allows the client to see how specific keywords and engines perform by monitoring if traffic being driven from these sources takes the actions defined as a conversion.

The problem comes when the conversion happens off line. Often, consumers use the web and search engines to conduct purchase research, creating a short list of the best products or models and shopping for the best prices. In most cases, this can be done without ever taking an action that could be defined as a conversion. The researcher may gather up their information, then take it and visit a local outlet to actually make the purchase. In this case, conversion tracking would have never recorded the visit as a successful one.

The Brand Value of the Online Experience

Another limitation with conversion tracking is that it can't capture the building of brand awareness online. And increasingly, websites are being recognized as an important channel to enhance to overall brand experience of a company.

I was recently meeting with a prospective client and he mentioned that his company (a multinational business to business service provider) didn't really monitor online metrics because "we're not Amazon". His reasoning what that because they didn't really sell items online, there wasn't anything to measure. He also mentioned that search marketing wasn't a high priority because of this lack of revenue generation from their site.

I was a little taken aback, so I probed a little deeper. I asked if they had given any value to the brand awareness they were building when visitors came to their site for information (of which there were thousands of pages, few of which could be found with a search engine). I asked if they had done any research on the types of users their site attracts, their reaction to the site, the frequency of repeat visits, and whether introduction to the firm through the website later translated into new clients. The answer was a long and continuous string of no's, and it was becoming clear; this company had no intention of beginning to monitor these metrics.

I hope this company is the exception, and that other marketing departments are beginning to recognize the importance of the online brand experience. Increasingly, more and more of a companies interaction with its clients is happening online, and the quality of that experience is vital to building a strong relationship with that customer.

Align Your Metrics with your Goals

To measure the effectiveness of any online campaign, you should come up with a set of metrics that give you a complete performance picture. The nature of your metrics will depend on what your corporate and online marketing strategic objectives are. What are your unique competitive advantages and how does your online presence contribute to them? What are the metrics that your customers measure you by? You have to ask yourself these questions before you can begin defining online metrics.

(Note: You might want to start with reevaluating how you use metrics throughout your organization before beginning this process. Sometimes, lack of metrics in marketing can be symptomatic of inadequate measurement of other areas of the business. If this is the path you want to go down, check out and invest in the book by Robert Kaplan and David Norton)

Return on Investment:
If you're an e-commerce site, the primary metric will likely be return on your advertising investment derived through online sales. Other metrics could include monitoring the frequency and nature of repeat visits and sales, average sale amounts, profits on sales (are you driving sales on higher margin items?), customer satisfaction levels, percentage of converted visits vs. non converted visits or percentage of sales in specific categories

Brand Awareness:
In this case, you will have to be a little more creative in defining your metrics. It will probably have to involve some type of user survey to determine brand awareness at various stages in the online relationship. A survey will also be able to determine awareness of products and services.

Building Relationship
As with brand awareness, measuring the development of stronger relationships with customers is more difficult to do. Again, you will probably have to use other research methods to get the full picture. You will also want to measure the duration of visits, frequency of repeat visits, sections of the site visited, and actions taken while on the site.

Measuring: The Bottom Line

The role of metrics in online marketing is quite new, and when it comes to search marketing, effective measurement is in its infancy.

Deciding on the right metrics is not an easy task to take on. It requires a fundamental understanding of your own goals and strategies, your customer's needs and behavior and the values and advantages that sets you apart from the competition. But if you're making a significant investment in your website and marketing that website, without metrics not only are you flying blind, you're also throwing money out the window.

March 9, 2004

Gord Hotchkiss is President and CEO of Enquiro, Canada's leading search engine marketing firm and one of the top firms in North America. His articles are regularly published in both on and off line newsletters, including Marketing Monitor, SEOToday, Marketing and many other trade journals. Enquiro's own information portal is

With an extensive 20 year background in the marketing and advertising business, Gord has been working to increase client's search engine visibility since 1996 and has specialized in search engine marketing since 1999.

Search Engine Guide > Gord Hotchkiss > Flying Blind? Search Marketing Metrics