In the delightful Marx brothers’ film, A Night at the Opera, Chico and Groucho discuss a contract.

Image via Wikipedia

Groucho: “That’s in every contract, that’s what you call a sanity clause.”

Chico: “You can’t a fool a me. There ain’t no sanity clause.”

In many organizations, market researchers are often the sanity clause or, more rightly, the sanity check on squishy information.

Take for example the recent headlines around the web that Facebook is responsible for one out of every five divorces. Goodness, what a great headline!

Step back and investigate even a little and one soon discovers that the headline plays fast and loose with the underlying data.

The headline was generated based on a press release from Divorce-Online.co.uk. Here is the salient portion of the release:

“Divorce-Online scanned their divorce petition database for the use of the word Facebook, and found 989 instances of the word in over 5,000 divorce petitions sampled.

This means that just under 20 per cent of all the petitions filed through the company had references to Facebook within the text of the divorce petitions.”

That alone is a far cry from one in five divorces were caused by Facebook. When did mention morph into causality?

Turns out as well that the sample of divorce petitions skewed to the young and was sourced in Great Britain which does not allow divorce without cause.

However, the juiciness of the headline overrode any sense of subtlety or just plain old fact-checking. Besides, it sounds so plausible.

It may not be accurate but it sure is salacious. So, let’s publish!

Part of our responsibility as market researchers is to do the fact-checking, to understand the limitations of our data and help our clients apply research conclusions to the highest advantage.

And in today’s voracious world where public relations and spin often govern, that is a tall order.

Are we up for the job?

Comments welcome.

Dr. Bob

Image via Wikipedia

Market Researchers generally occupy a world of perceptions, opinions, demographics, psychographics, and preferences, driven extensively by psychological or sociological thought paradigms. And sometimes we get caught up in the language of our specialization, particularly statistics.

We live in a world of probabilities, where findings and conclusions are caveated with statistical significance or likelihoods and where often the language reflects uncertainty.

Often, however, the clients we serve do not live in such a world. They have to make decisions, expend the company’s monies, take risks, and assume revenue responsibility. Imagine then their frustration when they hear researchers hem and haw about on the one hand, but then again on the other, or that this course of action might produce a certain results, but a different course might produce different results, which might be worse or better. We don’t know; we can only estimate the probabilities.

Please, please, don’t write me saying that issuing such warnings are part of our job and our responsibilities. I fully understand that.

What I am suggesting is that researchers would have more effectiveness if we better understood the nature of the decisions facing our clients. In other words, what is their bottom line?

Often that bottom line is a go-no go decision or a this-one-not-that-one decision on product development, marketing strategies and campaigns, investments, or resource allocation.

And here, we are armed and dangerous, so to speak, in understanding and using the fundamental statistical measures of Type I and Type II error.

From Wikipedia:

A table as follows can be useful in understanding the concepts.

Null Hypothesis (H0) is true   Alternative Hypothesis (H1) is true

Fail to Reject Null Hypothesis                                      Right decision                             Wrong decision

Type II Error

False Negative

Reject Null Hypothesis                                                 Wrong decision

Type I Error

False Positive

Right decision

Source: http://en.wikipedia.org/wiki/Type_I_and_type_II_errors

Thus armed, in our next discussion, I will explore the translation of Type I and Type II error into the language of the bottom line.

Comments welcome.

More to follow.

Dr. Bob

A truism in market research states that market research results are not the business decision, but one input, albeit usually an important input, to a business decision.

In this context, the primary purpose of market research is to provide relevant, timely, salient insights, usually about customers or prospective customers, to provide rational guidance to decision-makers about business choices and decisions.

So far, so ho-hum.  Of course, you might say. Is this a memo from the Department of Redundancy Department???

And here comes the big but. No, not the Wagnerian mother of opera fame. The but here concerns the volume of market research that is executed either as a feel-good exercise or as a check-off in a to-do list.

“Did you do the research?”

“Yes, and we got a green light.”

“Okay. Execute!”

In conversations with market researchers, and in surveys of client-side market researchers, one of our biggest complaints is lack of respect as evidenced by management’s seeming unwillingness to act of market research results. This may or may not be true. Either way, market researchers must accept some mea culpa in an organization’s unwillingness to properly utilize market research results.

One of the major disconnects I have observed across my career is that of the divide between the language of business decision-making and the language of market research.

It is to these disconnects that we will now turn our attention.

More to follow.

Comments welcome!

Dr. Bob

The market research industry, via forums, blogs, conferences and commentators, is busy declaring the death of market research.

Phone surveys are dead.

Online surveys are passe.

Traditional analytic techniques are moribund.

Why all the hand-wringing?

Social media and social networking are taking over the world, market research included.

True? Maybe. But maybe not.

I suggest that the declaration of the death of market research is premature.

Of course, it really depends on the purpose of market research. And the purpose is what is being lost in the discussion.

If you’re a regular reader of this blog, you know that I am a great admirer of The Economist as a solid, balanced news source. In a recent article titled The Status Seekers, Economist columnist Schumpeter discusses consumer-watchers, which he describes as an industry “as crowded and competitive as any other.” In what could be an indictment of modern market research he observes that “they hype passing fads as seismic shifts. And their propensity to be spectacularly wrong seems not to damage their business at all.”

Could it be that much of the lamentation about the death of market research is really about the tendency of some in the industry to embrace the ephemeral lure of consumer-watching rather than rationally and logically adapt traditional market researchers to electronic media?

I strongly suspect so.

Trendy is cool. It is way more fun and a lot more catchy to make absurd predictions about the future (who’s going to hold you accountable?) than it is to put the proverbial nose to the grindstone and do the hard work of solid  questionnaire writing/guide development, selection and execution of appropriate methodologies, and proven data analytics.

But some of us have to live with ourselves. Personally, I like to sleep well…and I cannot do that by adopting the flashy and trendy in market research simply to be hip, cool, and modern.

But I digress.

The real debate here is about the purpose of market research. From the perspective of a hard-nosed business owner, executive, or manager, why do market research?

Why indeed?

More to follow.

Comments welcome!

Dr. Bob

A few posts ago, I began a discussion of what is apparently a favorite pastime in modern media, lying with statistics.

Having just passed through an election, I watched with utter fascination as pols quoted statistics to “prove” the correctness of their positions; many of the statistics they quoted were ludicrous to any thinking person with a stats course behind them. But out the statistics came. Apparently innumeracy and a severe resistance to basic fact-checking obviate the need to base statistical “facts” in reality. If it appeals to voters, I’ll quote it!

Now you may be wondering what these musing have to do with market research. The connection, I will admit, is emphemeral but real. We are bombarded daily with so many stats that it is virtually impossible to check the validity of each and every one. We simply assume that if we judge the source of the statistic, that is, the person stating it, then we have to assume that they have done the fact-checking. It’s just too overwhelming otherwise.

Our simple inability to check it all means that we need to be cautious in passing on what may be false information–false conclusions based on bad statistical analysis or no statistical analysis at all. I would guess that a significant majority of the statistics I see quoted as fact in the media and on the web are figments of someone’s fertile imagination.

Consider the case explored recently about social media statistics: the claim that Facebook commands 41% of all social media traffic.

On the face of it this may seem plausible. And so utterly believable. Every day we hear more and more as marketers that we have to be on Facebook or we will be left in the dust of the marketing world. The AMA offers teleseminars about how to use social media; hundreds if not thousands of companies work to sell us their Facebook social media strategies. Everyone’s kid or grandma is whiling away all their free time chattering with their friends, posting photos and kibitzing on the site.

More to follow.

Comments welcome.

Dr. Bob

Visiting a local restaurant that serves wings and gives out hand wipes, I discovered the wipe packages had pithy little statements on each one, sort of like a fortune cookie. On one “58.3% of statistics are made up” was printed. I laughed. Not that this was my first exposure to the joke, but because of the prevalence of such boldly told postulations that dominate the media today.

I was reminded of perhaps the most important book in a professional market researcher’s library: How To Lie With Statistics by Darrell Huff. Published in 1954 it is as timely and relevant now as then, perhaps even more so.

Thankfully (in my mind at least) Darrell chose the word “lie” for his title; not “spin” or “convince” or “argue.” I think it is an important distinction.

As market researchers, our main responsibility is to provide our employers with thoughtful, timely and accurate depictions of markets and potential markets. In so doing, we analyze data, scrutinize it, poke and prod it, connection it with other data, compare it, and challenge it. We look to generate a robust, relevant analysis that is both compelling and accurate.

As market researchers, we pass findings first through the “is it plausible?” filter.  A finding simply has to make sense, or we won’t believe it and certainly, well, probably, our management won’t either. If an implausible finding is accurate (mathematically, based on the data set we are analyzing) we then check the context of the finding. Was the sampling frame accurate? Was the surveying done properly? We ask all the usual questions market researchers ask to validate findings.

And if an implausible finding is still in front of us, then we examine an even wider context. Does the finding fit with other findings in the study? What might be causing it?

In short, we seek and search for an answer that is both mathematically accurate and intellectually and logically plausible and satisfying.

Consequently, it is disappointing to encounter so much blatant manipulation of statistics and their reporting.

More to follow.

Comments welcome.

Dr. Bob

I read the article that follows on the Taipan Publishing Group website. While this blog usually focuses solely on market research news and issues, I found this article thought-provoking, especially given its focus on the linkage between quality customer service and macro financial performance. So, with permission of the publisher, the article is presented in its entirety. I hope you enjoy.

Comments welcome as always.

Dr. Bob

Companies That Ignore Quality Customer Service Are in Trouble

Todd M. Schoenberger, Managing Editor, Taipan’s Tipping Point Alert
Wednesday, July 07, 2010

Perfect customer service used to be the norm for American companies. Many can remember a time when full-service gas stations had a crew checking the oil, washing the windows and filling the tank – all without extra fees, excuses and pushback.

Or, companies recognized more for improving customer loyalty in hopes of receiving additional business in the future. You rarely hear stories about cultivating relationships with customers, and more tales about one-dimensional transaction-based interactions.

It’s discouraging, and many companies are overlooking the one true asset they can offer that costs virtually nothing: Excellent customer service.

During these tough times, consumers are more likely to remember how they are treated now and will reward those businesses when the economy fully improves. But it goes even further than this.

Have you noticed the companies with a reputation for great customer service are also the companies that have seen better performance out of their stocks during these difficult fiscal times? Think of Southwest Airlines (LUV:NYSE) and Apple (AAPL:NASDAQ) – companies known for going above and beyond for their customers all the while their stocks have appreciated 65% and 86%, respectively, over the past year.

Then, you look at hard-nosed companies like Comcast Cable (CMCSA:NASDAQ), Home Depot (HD:NYSE) and Bank of America (BAC:NYSE), companies that have barely beaten or lag the S&P 500 over the past year. Consumers have choices and are not stupid, yet these companies continue to disrespect the one group that helps keep the lights on in the building.

I’m sure there are countless examples of what you have personally experienced over the past two years that will probably have you returning – or ignoring – those places of business. It’s about how the company treats its customers that will set the tone for future revenues and higher equity valuations.

If you have a company you have had the pleasure, or displeasure, of dealing with, feel free to post a comment and let us know. We appreciate your thoughts.

At first glance, the numbers are both dismal and daunting.

Telephone surveying is undergoing a sea change.

According to the Pew Research Center (based on figures from the National Center for Health Statistics), 25% of U.S. households are cell phone only.

Households are jettisoning landline service at an accelerating pace.

Cell only households are more prevalent in various subgroups of the population: half (49% of adults aged 25 to 29) and 30% of Hispanics are cell only.

As Pew states, “survey researchers…face a difficult decision as to whether to include cell phones in their samples. Doing so adds significantly to the cost and complexity of surveys at a time when respondent cooperation is becoming increasingly difficult to obtain.”

Some in the market research community have opted for Web-only surveying as a replacement for telephone. But this presents an even larger issue. In spite of the shift to cell service, phone service overall is near ubiquity. Only 2% of U.S. households have neither landline or cell service. Some 38% of U.S. households do not have internet service at home.

The market research industry is arguing furiously about how to solve the problems with telephone surveying. Thus far, there is little consensus. But the effort must continue.

Meanwhile, market researchers in the corporate world can rest relatively easily. Surveying out of customer databases by phone continues to work and continues to be statistically robust. The challenge on the corporate side is in sampling and data collection outside their own customers. Hybrid phone surveys, combining both cell and landline, are still relatively untested and their reliability is still largely unknown with unanswered questions about the representativeness of surveying via cell in dual landline-cell households.

Market research will clearly become only more challenging in the near future, at least.

But, I guess that’s why market researchers get paid the big bucks, right?

With a smile,

Dr. Bob

In the prior post on this site, I discussed the challenges in market research surveying of measuring consumer behavior.

For years, consumer market research has found that consumers support Green, while their actions in the marketplace belie this support.

A recent study published in the Journal of Personality and Social Psychology highlights shifts in Green thinking and behavior among consumers, showing that the Green movement might benefit from encouraging consumers to be conspicuously Green.

Researchers developed a series of experiments in which college students were offered equally priced products in both green and conventional versions. Indeed, the green models were less feature-rich than the conventional products. The test group was then primed to increase their awareness of status while the control group remained neutral.

Both groups then made their product selections. A significantly higher percentage of the test group chose the green products.

The study suggests the motivation for this difference: “costly signaling theory says that although costly, altruistic acts may benefit the altruist indirectly, by establishing a ‘reputation’.”

In a telling follow-up study, researchers found that preference for Green lessened among the test group as the price of the Green products were lowered.

For companies marketing Green, market researchers may want to measure these possibilities.

For more details about the study, click here.

Dr. Bob

Market researchers tend to explore the realms of consumer and customer perceptions and attitudes. Depending on the topics being investigated, a long history of both qualitative and quantitative research methodologies have established norms for obtaining valid and relatively precise data from respondents.

However, caution is required in market research when dealing with consumer and customer behavior.

A recent report by the Economist newspaper on the state of television illustrates this continuing issue. The Economist reports on the findings of Sarah Pearson, a research from England, who has almost 100,000 hours of recording of consumers watching television. As the Economist says, “There turns out to be an enormous gap between how people say they watch television and how they actually do….In surveys they almost always underestimate how much television they watch, and greatly overstate the extent to which they watch video in any other form.”

The accompanying data from Nielsen shows that consumers understate their television consumption by some 40% and overstate their consumption of online video by more than 100%.

Market researchers do well to proceed with caution in assessing the validity of self-reported behavior, such as product and service usage.

Comments welcome.

Dr. Bob

[Note: A subscription to the Economist is required to access this report online. It is available for purchase on their website.]