Tuesday, July 3, 2012

Evaluating Media and Advertising Allocation

Use most, use at all, or value?

Working in media, I come across a lot of research studies rating media against one another in reach and utility, but are these the best way to draw media investment decisions?  There’s value to these assessments, but also some significant pitfalls, which begs the question: “Is there a better way?”
In my hypothetical example, as a “research consultant,” I’m looking to see where I would get my biggest bang for my buck in media and communication investments.
A search engine marketing firm, Guide Dog Marketing, brings me the following report from CEOs and CMO’s surveyed nationally in the sources they use “most” when choosing a research consultant (Exhibit A). 

By a measure of more than three to one, I should clearly be investing in SEO (search engine optimization) and SEM (search engine marketing).  Is it really that obvious? No.  Remember, the question only enabled decision makers to select one from a list of 12 options (single source multiple choice question).  Here are a few of things to consider about the “most” question and results:
  • Most isn’t defined.
  • Does it mean most often or does it mean I rely on it the most?
  • Either way it still brings additional questions to the table.
  • What isn’t measured is how much more or less each source is used compared to the others.  One CEO/CMO may only use one particular source, while another may use six different sources and only selected search web sites because she ultimately concluded that she only uses it slightly more often than the next source on her list (Translation: same answer, but two very different hidden behaviors).
Now, a sales rep from Research Geeks Monthly comes to me with a different report.  He shares the following report from CEOs and CMO’s surveyed nationally in the sources they “used” when choosing a research consultant (See Exhibit B). 

In this instance, respondents were allowed to choose multiple answers.  By a measure of more than two to one, Industry publications (like Research Geeks Monthly), are the most used traditional media source for helping making a research consultant decision.  It’s not far behind the leaders of referrals, research web sites, and search.  They use it, but how often and how valuable do they find each source.
Many would end up dismissing both sales representatives because they’re reports are so different.  Most would conclude that each report was spun to make their respective companies look like one of the best.  As a “research guy” I know this isn’t an accurate assessment.  Each question has truth, but each does not merit decision making on its own.  Industry conferences are not used most (hardly at all in this hypothetical), but nearly half said they used it in decision making.  It may be that most decision makers only attend one or two conferences a year, so frequency of attendance or abilities to attend conferences may impact their ability to evaluate its value.  The great part about the used questioned is its ability to demonstrates the number of different sources decision makers use without the forced single selection found in the “most” question (For an example on how to calculate, visit this post: Consumer impressions).  It’s more representative of source utility.  But I believe value combined with utility is the key to evaluating media.  The value metric largely goes unmeasured.  Just because search web sites are used by nearly everyone and is leagues ahead of the rest in being selected “most,” doesn’t necessarily mean it’s of significant value in the decision making process. It’s a means to an end, but not necessarily the ultimate reason in one’s decision.
Source Value
A multi-media sales rep from Fightin’ Analyst Inc. comes to me with the same sources used most and sources used report.  In this hypothetical situation the sales rep also has a report showing how CEOs and CMOs rate the value of each source in decision making (See Exhibit C). 

Now I have the tools to evaluate my investment.  My web site, conferences, and referral strategies sit front and center.  These should be the core of my investment.  The sources “used” help me evaluate my secondary investments in driving inquiries.  Search is used, but if I have a terrible web site, all is lost.  Search and traditional sources can help drive traffic and possible inquiries.
Ultimately source utility and value should take precedence in evaluating advertising and media allocation.  If a lot of people use it and that find it valuable, that’s where I want to be.  To me, the sources used most question has the least relevance.  It creates an environment that doesn’t exist, which is limited access to resources, and there’s no arguing that this is far from our modern day marketing realities.  Decision makers have nearly unlimited access to resources.
Geeking-Up the Evaluation
If you what to take this premise to a whole new level in evaluation, you can convert the value scale results into total percentage of value, the sources used percentages into total sources used percentages, and then average the two new percentages for a “one” metric evaluation for utility and value (see Exhibits D and E for examples of conversion and the final “one metric” results).  I’ve used this conversion method in analyses like The Consideration-Purchase Gap.

My evaluation method is far from perfect, but it sheds greater insight into consumer behavior and offers businesses and decision makers a systematic way to research or analyze media and advertising.