Recently a fellow LinkedIn member posed set of questions in a market research forum about what criteria might be used to select market research suppliers. Having responded to these questions there, I have revised and refined nine criteria that could be used to choose quantitative market research firms.

First, full disclosure: My business is as a market research supplier. I conduct both qualitative and quantitative market research studies for my clients. No doubt, the criteria that follow are influenced significantly by having been a supplier for some fifteen years.

I also have almost fifteen years’ experience as a purchaser of market research services, in positions of both market research management and product management for my employers, mostly notably one of largest telecommunications firms in the country. So I have much experience dealing with market research vendors from a client perspective.

In these positions, I developed several criteria and areas of inquiry for selecting market researchers for quantitative projects. (As a manager for a very large company, you might imagine how many solicitations for business I received.) Several of these criteria I learned to employ based on years in the school of hard experience.

All the criteria will be discussed in the next few posts.

Comments welcome.

Dr. Bob

At first glance, the numbers are both dismal and daunting.

Telephone surveying is undergoing a sea change.

According to the Pew Research Center (based on figures from the National Center for Health Statistics), 25% of U.S. households are cell phone only.

Households are jettisoning landline service at an accelerating pace.

Cell only households are more prevalent in various subgroups of the population: half (49% of adults aged 25 to 29) and 30% of Hispanics are cell only.

As Pew states, “survey researchers…face a difficult decision as to whether to include cell phones in their samples. Doing so adds significantly to the cost and complexity of surveys at a time when respondent cooperation is becoming increasingly difficult to obtain.”

Some in the market research community have opted for Web-only surveying as a replacement for telephone. But this presents an even larger issue. In spite of the shift to cell service, phone service overall is near ubiquity. Only 2% of U.S. households have neither landline or cell service. Some 38% of U.S. households do not have internet service at home.

The market research industry is arguing furiously about how to solve the problems with telephone surveying. Thus far, there is little consensus. But the effort must continue.

Meanwhile, market researchers in the corporate world can rest relatively easily. Surveying out of customer databases by phone continues to work and continues to be statistically robust. The challenge on the corporate side is in sampling and data collection outside their own customers. Hybrid phone surveys, combining both cell and landline, are still relatively untested and their reliability is still largely unknown with unanswered questions about the representativeness of surveying via cell in dual landline-cell households.

Market research will clearly become only more challenging in the near future, at least.

But, I guess that’s why market researchers get paid the big bucks, right?

With a smile,

Dr. Bob

Market researchers tend to explore the realms of consumer and customer perceptions and attitudes. Depending on the topics being investigated, a long history of both qualitative and quantitative research methodologies have established norms for obtaining valid and relatively precise data from respondents.

However, caution is required in market research when dealing with consumer and customer behavior.

A recent report by the Economist newspaper on the state of television illustrates this continuing issue. The Economist reports on the findings of Sarah Pearson, a research from England, who has almost 100,000 hours of recording of consumers watching television. As the Economist says, “There turns out to be an enormous gap between how people say they watch television and how they actually do….In surveys they almost always underestimate how much television they watch, and greatly overstate the extent to which they watch video in any other form.”

The accompanying data from Nielsen shows that consumers understate their television consumption by some 40% and overstate their consumption of online video by more than 100%.

Market researchers do well to proceed with caution in assessing the validity of self-reported behavior, such as product and service usage.

Comments welcome.

Dr. Bob

[Note: A subscription to the Economist is required to access this report online. It is available for purchase on their website.]

Sometimes management is insistent on working to turn around very dissatisfied customers as an organizational imperative.

“Well,” I have heard, “they are satisfied and they are staying as customers. Why spend money to make them more satisfied?”

Good question.

Here is why: companies need to build positive satisfaction bank accounts, so to speak, with customers. With a positive account, a customer will be more likely to forgive (but not forget) the next time they encounter a curt customer service rep, or have to wait twenty minutes to talk to one (while listening to a recording telling me there is heavy call volume and they really value my business [wouldn’t valuing my business really translate into putting more resource into answering my call more rapidly?]).

Here’s a personal story to this effect. I was a customer of one of the satellite television providers. I was reasonably satisfied with them. If they had ever asked, I would have given them a four on the one to five scale.

But, their standing with me was constantly based on the next encounter. They had flunked the installation test at the beginning of our relationship by both failing to show up as promised (they did not show at all) and then I had to call them to complain and reschedule.

The service had been okay. The few times I had technical problems, their reps were helpful.

Then I received a flyer from them in the mail, pitching me an introductory offer to start service (I am already a customer!) at less than half the price I was paying after being subscribed for more than five years. That certainly made me feel warm and fuzzy toward them. Moreover, they would not make the same offer to the existing customers.

My conclusion: they valued a new customer more than they valued me, a long-standing customer. My satisfaction is decreasing……………………………..

The kicker came when I was moving. I called to transfer the service from myself to my significant other who did not have an account and was not on my account.

So I made a phone call to their customer service center. The rep I spoke with confrontational, rude, and insistent that I would have to pay to cancel my contract (which had expired after two years of service; I was on month to month).

What did I do? I called their competitor and established service with them. They were more than happy to help, as you might imagine (and are continuing to provide good service three months later—right now, I would say they are a five).

The first provider had no reservoir of good will with me.  And I left when the opportunity (and timing) presented itself.

What could they have done? In the case of the final straw, probably nothing short of free service for the grief they gave me. I am a realist: that was not (and did not) happen.

But if their management was less focused on shoveling new customers into the pipeline and more focused on retaining customers who had proved their value, the distasteful encounters I had with them might not have happened and I would still be their customer.

More to follow.

Comments welcome!

Dr. Bob

My issue with focusing on turning very dissatisfied customers into satisfied one is this: psychology tells us, and qualitative (and I would argue quantitative) market research confirms that there are some people, some small percentage of the customer/consumer base, that simply is never going to be satisfied. No matter what you do for them, they are never going to be happy with you. The damage is done.

Now I am not a psychologist (but I do play one on tv, ……er, no!). But as an experienced focus group moderator, I have observed over the course of thousands of groups that there are people who are simply unhappy. Why? It is not for me to say or speculate. But they are. You have probably observed this as well if you have set in enough focus groups or in-depth interviews.

This small group (the two to three percent) walk around with thunderclouds over their heads perpetually……and it’s raining!

Most of us have bad days where something happens and we are upset. We may snarl at the dog or get upset that the checker at the grocery counter is not enforcing the ten item rule (that guy has got eleven items!!!). But we get over it.

I suspect that the two to three percent do not.

That may be one reason why the dissatisfaction level in customer satisfaction surveys is so relentlessly stubborn.

But then what about the issue of motivating the organization to higher levels of achievement?

Your observer postulates that this lies in moving customers up in satisfaction level, from somewhat dissatisfied to neutral, from neutral to somewhat satisfied and from somewhat satisfied to very satisfied.

In my experience, many companies who have a solid customer satisfaction measurement (and improvement) system in place report satisfaction as top 2 box and bottom two box (assuming a five or seven point scale).  Maybe it is time to revisit this practice.

Rather than focus on eliminating the “1”’s (very dissatisfied), a more potentially productive path might be to focus on implementing change that will make the satisfied even more so.

More to follow.

Comments welcome!

Dr. Bob

I have yet to see any definitive research to indicate either the continuing validity or the abrogation of prior market research findings that for every customer with a problem another nine or ten have the same problem but are silent and that an unhappy customer will tell five other people while a happy customer is generally quiet.

But let’s speculate a little.

First, we might ask if consumers or customers are more unhappy than in yesterday. Probably not. Customer satisfaction market research suggests that consumers are generally getting happier with products and services as quality controls, technological advancements, and more rapid distribution networks have evolved.

But there is a lingering cloud of discontent that is consistently measured in customer satisfaction. For companies using one to five scales, very dissatisfied to very satisfied, many continue to see the very dissatisfied percentage of customers remaining stubbornly in the two or three percent range, with the somewhat dissatisfied range slightly higher, typically three to four percent.

What is going on here? If customers are that dissatisfied, why don’t they vote with their feet? (Or are they, and then are simply being replaced with equally dissatisfied customers.)

A few years ago, your observer ran a customer satisfaction measurement process for a major company. The client was the Quality Assurance group. The director of the group (a Board level position) and I would often speculate on what was so upsetting these very dissatisfied customers. He was pushing hard to continually up the bar.

This organization was one of the pioneers of driving a portion of employee bonus pay (all employees) based on the customer satisfaction results. The director wanted the employees to constantly work toward improving their performance and a logical way to measure their improvement was to look at increasing satisfaction scores (or decreasing dissatisfaction scores).

He and I had some interesting debates on this topic. My observation, then as now, is that it is a Sisyphean task to set a compensation-based goal of continually increasing customer satisfaction scores by reducing the percentage in the very dissatisfied category. Now, that might seem odd coming from a strong believer that businesses must measure and act on customer satisfaction market research to up their game and continue to be successful.

More to follow.

Comments welcome.

Dr. Bob

Ever heard the old saying in that for every disgruntled, dissatisfied customer who lets you know of their unhappiness with your product or service, that there are nine or ten other customers who are upset but don’t bother to express it? Moreover, these discontented customers would tell five or so other people. On the other hand, happy customers do not necessary tell anyone unless that are particularly happy or if they are asked directly about the business they are patrons of.

Common wisdom had it that several, if not most of the unsatisfied customers would vote with their feet, that is, they would move their business to one of the competitors and unless the company in question is  conducting attrition surveys, the organization would never know why they left. (Or worse, it might not even notice.)

Much of the above is supported by research conducted in the 1980s. But is it still valid today?

Probably not.

Yesterday, a consumer had to be ardent to express a complaint. The consumer would have to physically visit a location, or be willing to make multiple phone calls to find someone, anyone, who would listen, or write, address and mail letters. Moreover, an unhappy customer had little recourse if their complaint was not answered. They had little or no market power to express their discontent in a public forum.

A few hardy souls would, for example, paint their cars lemon-yellow as a expression of outrage against either a manufacturer who refused (or couldn’t fix a problem with a new vehicle) or a dealer who they felt ignored their problem(s). But except for an appeal to the consumer watchdog reporter from a local television news show, usually a complaint or an incident would not gain the attention of the public (and for those of us who remember those days, wasn’t there some sense that the complainers where just cranky kooks?).

Ah, how the worm has turned!

Today, a disgruntled customer has but to turn to the Internet to potentially create a firestorm of adverse publicity for the company that is the (alleged) source of their pain. A consumer can post a negative review of a product (or vendor) on a shopping site (Amazon?), or make a video to post on YouTube and a hundred other sites, or express their anger on a blog, or even put up a website dedicated to”I hate [such and such a company].” Maybe, with some cleverness and a little bit of luck, their Internet efforts will go viral and thousands, hundreds of thousands or even millions of people in the online community would view their missives. Examples are rife.

What to make of all of this?

Have the research findings of yesterday been displaced?

Maybe.

More to follow.

Comments welcome!

Dr. Bob

As powerful as the Internet can be, marketers and market researchers need to remember that consumer e-commerce activity is only a tiny fraction of total retail activity in the United States, still in single digits.

On the other hand, e-commerce on the b2b (business-to-business) side is booming, with some 35% of activity in the manufacturing sector conducted online (as of 2007). Note that this is close to double the percentage of just five years prior (18%).

In spite of the hoopla of the Internet, and all the spin, e-commerce retail sales are were only 3.2% of total retail sales in 2007.

Is this going to change? Absolutely. At some point, a critical mass will catapult a much higher percentage of consumer retail sales online. But how soon and by how much? We don’t know.

Until there is clarity in this regard, marketers, and market researchers, must continue to look at the entire market, the majority of which are not e-commerce users, yet.

As marketers and market researchers, we are all connected. We have to be. As is virtually all of corporate America. But just because we are, does not mean that everyone is (the connectivity rate among households remains stubbornly at around 75%) nor does it mean that the Internet is the center of connected consumers’ lives. It isn’t.

Want proof? Just look at the e-commerce retail numbers above.

Sure, the Internet is the Wild West. Lots of folks, especially marketers, are getting rich quick, seemingly popping up websites and selling hundreds of thousands or millions of dollars worth of stuff.

But right now, the circus is in town. And the bright lights and cotton candy and carnival barkers are enticing and distracting.

As marketers, we must have the sobriety to view the world, and the markets, not through our own lenses, but through those of the customers and markets we serve. That is what we do as market researchers. Report back to the insular corporation the realities facing the customers and markets outside.

It’s a job I relish.

Comments welcome.

Dr. Bob

The old cartoon of a dog tapping on a keyboard with the caption, on the Internet no one knows you’re a dog still applies. Market researchers have to take careful, considered steps to ensure that in Internet surveying they are really talking to the population they think they are talking to.

For businesses that use other communication and distribution channels, relying solely on the Internet for market research, despite “black box” solutions to the sampling challenge, is a recipe that must be carefully brewed.

Bottom line? Fundamentals are fundamental.

Some years ago, I attended the ART (Advanced Research Techniques) conference of the AMA (American Marketing Association). One of the presentations was about the analytic power of a new statistical modeling technique that one of the big guns in academic market research had developed. It happened that another big gun, and one who had a competing model, was in attendance. The two almost came to blows about which model was more accurate and relevant. I set in the session, fascinated with the altercation. I was very tempted to add fuel to the fire by pointing out a fundamental flaw in both of their models: they were developed and tested using N sizes of under 100 respondents. (I didn’t.)

Now I’m not a Ph.D. in statistics. I work exclusively in applied, or practical market research. But even a student in Statistics 101 knows that a sample of less than 100 yields qualitative data only (unless, of course, the population being sampled is less than 1000). End of story.

I’m sure these gurus had a jargon-laden explanation of why, in their cases, it was acceptable and reasonable to break the fundamental rules of quality market research.

But there are no excuses.

If the fundamentals are not met, the results are fallacious.

It’s the same for market research 2.0 as with market research 1.0. Do the fundamentals according to correct statistical practice.

Comments welcome!

Dr. Bob

Web 2.0 is changing everything about market research, or is it?

In some ways, the market research community (or industry if you will) is in crisis.

The bulwark of surveying, telephone interviews, is in a sea change: according to the Economist, about 25% of Americans rely solely on a cell phone for telecommunications. This is up from just 7% in 2005. At current rates of decline, the last lineline will be disconnected in 2025.

Meanwhile, only 62% of Americans have internet access at home as of 2007, according to the U.S. Census. While this number has undoubtedly increased as of 2010, it is still nowhere near the ubiquity that landline phones once held.

Both these trends are making reliable national probability sampling a major difficulty.

And the Internet and Internet marketing (and Internet market research) are all the rage as the answer to the problem.

Looking out over this landscape I am struck by one overwhelming observation: Web 2.0 market research (communities of users, Facebook fans, etc.) and even probability sampling studies on the Web are often convenience samples and therefore strictly qualitative in nature.

Useful? Certainly! Web 2.0 provides immediate feedback from highly motivated and involved customers and potential customers. You can quickly, easily and inexpensively float new ideas, new concepts, and trial balloons and get a quick brush on viability.

But in the rush to Web 2.0, market research 2.0 should tread carefully.

Market research is build upon the hard math of probability theory. And a fundamental truth about market research is that it is only as good as the sampling from which results are drawn.

Businesses that are exclusively or primarily Internet marketers certainly have an easier time insuring that they have solid customer (or population) samples. Easier, but caution is the watchword still.

More to follow.

Comments welcome!

Dr. Bob