When Numbers Lie – The Convenience of Market Research

Posted: October 14, 2010 in Metrics & measurement, Research
Tags: , , , , ,

When Fast Company, a publication that I have read and respected for years, published a story based on faulty data, I had to call them out. The story is titled Twitter Crushing Facebook’s Click Through Rate, and is based on research from Social Twist. I think I threw up in my mouth a little when seeing these numbers.

First of all – enough with the click through rate already. It has always been a bad KPI that is not indicative of performance. But more importantly here, CTR is not even the actual metric they are reporting on, and the real value or insight in the data is sort of lost, albeit the lesson of consumers sharing in social platforms like Facebook and Twitter is not much of an insight. File under “DUH!”

Really a CTR rate of 1904% and 287%? This is what happens when you can’t track the denominator (reach/exposure) of your calculation. What they are actually calculating is the volume of responses to shared content and not a CTR, it is actually a more valuable metric and they should try to better define it.

I think that Social Twist’s Tell-a-Friend sharing widget is a great addition for many marketers, but guys, releasing misleading and incorrect stats like this removes some of the credibility and thought leadership from your quest. Research and stats are a great way to get press coverage. Kudos for pulling the wool over the eyes of Fast Company (and surely a number of others), but the industry doesn’t need more fuzzy math market research confusing marketers.

This is just one example, there are so many questionable stats floating around – even from credible companies who are in the business of producing research.

We all love stats and research. Good research does help refine our decision making. But it is no secret that market research is often self serving and misleading. Next time you get blown away by some market research or stats, take a moment to question the research methodologies, determine if there are actually insights provided, and even analyze the motive of the research.You might be surprised how often you  find the data useless, misleading or self serving.

 

If you enjoyed this post, please consider adding a comment, subscribing to post updates via email or subscribing to the RSS feed. Thanks.

Advertisements
Comments
  1. Kimberly says:

    Jason

    This is so true! The crazy debates continue but true metrics and benchmarks still are often overlooked or diluted. The media world is full of fuzzy math as it is and this type of research reinforces that. You are right on the mark that it is often misleading or self-serving – it’s all in how you spin it right? We continually point out to clients the source and methodology of research. This doesn’t make it bad research often just a bad analysis. Keep it in context!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s