Archive for the ‘Research’ Category

When Fast Company, a publication that I have read and respected for years, published a story based on faulty data, I had to call them out. The story is titled Twitter Crushing Facebook’s Click Through Rate, and is based on research from Social Twist. I think I threw up in my mouth a little when seeing these numbers.

First of all – enough with the click through rate already. It has always been a bad KPI that is not indicative of performance. But more importantly here, CTR is not even the actual metric they are reporting on, and the real value or insight in the data is sort of lost, albeit the lesson of consumers sharing in social platforms like Facebook and Twitter is not much of an insight. File under “DUH!”

Really a CTR rate of 1904% and 287%? This is what happens when you can’t track the denominator (reach/exposure) of your calculation. What they are actually calculating is the volume of responses to shared content and not a CTR, it is actually a more valuable metric and they should try to better define it.

I think that Social Twist’s Tell-a-Friend sharing widget is a great addition for many marketers, but guys, releasing misleading and incorrect stats like this removes some of the credibility and thought leadership from your quest. Research and stats are a great way to get press coverage. Kudos for pulling the wool over the eyes of Fast Company (and surely a number of others), but the industry doesn’t need more fuzzy math market research confusing marketers.

This is just one example, there are so many questionable stats floating around – even from credible companies who are in the business of producing research.

We all love stats and research. Good research does help refine our decision making. But it is no secret that market research is often self serving and misleading. Next time you get blown away by some market research or stats, take a moment to question the research methodologies, determine if there are actually insights provided, and even analyze the motive of the research.You might be surprised how often you  find the data useless, misleading or self serving.


If you enjoyed this post, please consider adding a comment, subscribing to post updates via email or subscribing to the RSS feed. Thanks.


Old Shcool Kool AidThis one’s been eating me up inside for a week or so. Did you catch the research report from the Pew Internet & American Life Project regarding the web’s influence on purchases?

Consumers were asked questions regarding their product research, shopping and purchasing habits and the results were analyzed and parsed based on four dimensions: 

  • Search:How much people rely on the internet in product research and to what extent
    online information smooths the path to a purchase decision.
  • Influence: Where the internet’s influence is great (or not) in consumer’s decisions.
  • Participation:Whether internet users get involved in online chatter or take other
    steps after purchase to engage with what they bought.
  • Disintermediation:Whether the internet serves as a way to go around traditional
    means (e.g., going to retail stores) to buy products.

I like these delineations, they are great indicators of overall impact of a medium and therefore by indirect correlation, the marketing within the medium.

Here is the one line take away from the report: Online resources are tactical tools in the online shopping experience, but online resources do not play a large role in the the products consumers choose or how they consume them.

To that I respectfully say: “You gotta be figgin’ kidding me!?!”

I ordinarily appreciate the research from Pew, it is objective and usually fairly meaningful.

This flies in the face of all online based research, which indicates the web has become a primary research and influence channel in purchases for many categories among online consumers.So this begs the question about methodological equalities between online based research and random telephone surveys. Is there a disparity between the type of person who willingly answers a telephone survey and those who take online surveys? Does a random telephone survey of 2,400 consumers accurately represent the online universe? Probably not (subjective, I know).

I clearly understand that Pew is trying to represent the American public versus the Online American public. But to marketers, we really want to understand how the internet influences the ripe digital media market of 219 million users in the US (Nielsen April 2008)…otherwise known as most civilized adults with actual buying power. 

Is the internet a major influence in the purchasing process and decision for music, cars, or travel? The answer to that starts with a resounding “DUH!”. For years we have been studying how the internet influences online and offline purchases, various methodologies have been employed to research this objectively. Transactions need not occur online for the influence to occur here. Likewise, the increasing scope of influence of the social media ecosystem, including all the tools from consumer reviews to blogs, forums and of course social networks also plays a major role in purchasing decisions across various categories (as in virtually all). Both the medium itself and the people participating online create influence. Do you know anyone who can’t claim this for their own behavior as a consumer? I try to never look at the ‘focus group of one’. Seriously, I was disturbed and borderline angry to see such data spewed into the market without further scrutiny.

Again, I ask – do random telephone surveys accurately represent the online consumer? The Pew report stated that the internet does not play a major role in influencing music purchases — excuse me???

Of course the internet is just a cog in an influence chain that includes many other factors. I am not a digital elitist by any stretch, but come on – does anyone believe this research?? Well the US Census counts 304 million Americans, so I guess that would make about 90 million possibilities – let’s give them a call and ask them.