Part 2, Part 3, Part 4, Part 5 and Part 6 of this series discussed the first eight criteria for selecting a quantitative market research firm. Part 7, the last in the series, introduces the ninth.

Ninth, ask about the last three projects the supplier did.

A reputable supplier will NOT reveal client names or project specifics, but will give you an overview of the projects.

The answers to these questions are important; how they are answered, with what speed, with what depth and with what openness are critical. You are also listening for what is not said.

Along with probing the areas above, I would also verify their credentials and qualifications. If anyone is insulted by your request for credentials and qualifications, run fast and far. If you ask them for their credentials, they should be happy to supply them and to give you the information to verify them. It is not insulting…if they have the credentials. If they are insulted by this request, either they don’t have the credentials they say they have, or they are neurotic. In either case you would not want to work with them.

In the next posts, I will explore the criteria I use to evaluate qualitative market research suppliers.

If you have additional criteria you use to evaluate quantitative market research suppliers and would like to share, please feel free to comment below.

Comments welcome.

Dr. Bob

Part 2, Part 3, Part 4 and Part 5 of this series discussed the first six criteria for selecting a quantitative market research firm. This part introduces the seventh and eighth.

Seventh, how clear is their pricing? What is included in their services and what is extra?

Pricing clarity is critical. You may or may not want to always seek the lowest price. In evaluating market research services, pricing is driven by the confluence of complexity of the project, the sampling costs, the timeframes, and most critically, length and quantities of the survey or interview, analytic requirements, and quality control.

That said, you do want pricing clarity.

I have worked with suppliers who would low-ball the initial project costs and then nickel-and-dime me to death on the back end. Oh, you want a frequency? Well, that’s extra. You want SPSS output? That’s extra. You want a daily status report? That’s extra.

Insist on knowing exactly what the deliverables are and how much they will cost.

Eighth, do their references check out?

It continually surprises me that otherwise rational, hard-headed business people, let alone market researchers, do not ask for or check references. I surmise that the rationale for not doing so is that if a potential supplier can give references, they must check out okay. What company would provide references that would not give the supplier a good review?

Checking out references is a matter of tone. When you check references you listen for what is said, how it is said and what is not said. Unless a supplier was truly abysmal, I would not give a negative review to any supplier if you called me for a reference on the ones I use, for example. But I would certainly choose my words very carefully and use intonation to convey my lack of enthusiasm. If a supplier is truly superior, I would let you know and I would let you know what they do that makes them superior.

More to follow.

Comments welcome.

Dr. Bob

Part 2, Part 3 and Part 4 of this series discussed the first four criteria for selecting a quantitative market research firm. This part discusses the fifth and sixth criteria.

Fifth, are they willing to take a small project as a test or do they only want the bigger pricier ones?

Market research suppliers often specialize in the types of projects they conduct. Before you contact suppliers, it is wise to determine if they have an industry specialty and then to get a sense of the types of customers they serve. Select those that fit your market research needs.

I may be headed into a sensitive area, but so be it.

Some firms position themselves as all things to all potential clients, at least in my experience as a client. In actuality, some firms specialize in what the industry calls “field and tab,” that is, relatively simple studies of attitudes and perceptions, basic customer satisfaction, direct mail tests, concept evaluation and the like. Such a study basically involves developing a questionnaire, locating an appropriate sample, fielding data collection, analyzing the data, generally with frequencies and/or cross-tabulations and report-writing.

Other firms specialize in (or are definitely prepared to handle) more sophisticated studies using conjoint analysis (which requires specific types of questions) or discrete modeling. Such projects require significantly more statistical analysis and therefore expertise and the requisite staff to handle this level of analysis.

Narrow your field of potential market research suppliers beforehand, based on the types of projects they tend to do.

Having done so, you will probably want to test your chosen supplier with a relatively small project to get a sense of how well you work together.

Is the supplier open to, and appreciative of, such an arrangement? Or do they push to either up-sell your project or want to hold off for a larger opportunity? If so, this supplier might not be a good fit for you.

Sixth, how responsive are they? How soon are my messages answered? Do I have to pull teeth to get answers? Are they proactive in keeping me informed about the status of the project or do I have to pester them for this information?

This aspect of a client-supplier relationship is difficult to ascertain short of conducting a project with them. In my experience, almost all suppliers are very responsive during the initial sales cycle.

One way to get a peek at potential communication issues is your conversations with both the person who will be your primary contact at the firm and the other significant team members (the data collection manager, the analyst to name two).

This is also a main reason to test new suppliers with a small project. If they don’t meet this test, I would decline to do business with them.

More to follow.

Comments welcome.

Dr. Bob

Part 2 and Part 3 of this series discussed the first three of nine criteria for selecting a quantitative market research firm. This post explore the fourth criterion.

Fourth, will I have a single point of contact at the firm that will take full responsibility for the development and execution of my project? If that person is not the person selling me their services, can I meet and talk with that person?

This criterion is crucial to me. Before I hire a market research supplier, I MUST meet, talk with and get to know the people who will actually be handling my project. I MUST have access to these folks. And one of them MUST take overall responsibility for the projects, for running interference, for problem-solving and for communicating regularly with me about the project. If the sales person does not have operational responsibility, I have to talk with the person who does.

Fundamentally, do we get along? Can we work together? If you really like and relate to the sales person, but he or she drops out of the project picture once you are on board, you could be left working with a person to whom you cannot relate. Don’t simply take the sales person word for it that the project manager is a great guy or gal and you will love him or her. Meet that person and ask that person some of the same questions you have asked the sales person.

Too many times I tried suppliers based on the likeability and assurances of the sales person, only to discover that the project manager or other key team member on the vendor side had the business and social skills of Frankenstein. Or worse, there was no one person who could or would take responsibility for the project, leaving me to call all over the firm to find someone to answer my questions or solve problems with the project.

Know who is responsible before you ink the deal.

More to follow.

Comments welcome.

Dr. Bob

Part 2 of this series discussed the first two of nine criteria for selecting quantitative market research firms. Part 3 discusses the third criteria.

Third, are they offering me a black-box solution? Is their solution something I (and colleagues) can understand or am I going on faith that they know what they are doing? Can what they are offering be validated and replicated?

Here is another criterion that if failed will send me running. Some hot-shot statistical guru has invented some proprietary analytic process that answers more questions, solves more problems, and is more accurate than any test, process, or model known to market researchers, statisticians or even God. And they will deploy it just for you. But, wait a minute, they cannot tell you how it works, they cannot give you enough information so you can understand it, and, of course, no one else has it so it cannot be validated or replicated. But, boy, does this snake oil get results!

Right, and I have a CDO for sale.

Run, and run fast.

The fact of the matter is that there is no black box solution, just as the Wizard of Oz ultimately proved to be a humble man from Kansas.

Don’t buy it. You are MUCH better off with tried-and-true market research methods and analytic techniques and tests. Hey, they might not be sexy, but they are proven to work and have been endlessly validated and replicated.

I am reasonably well educated, have market research experience and a good head on my shoulders. My inability to understand a black box solution does not lie with me. Instead, it is that the black box solution is a Rube Goldberg machine. The guys who invented the black box are so much smarter than I am, so smart that a mere mortal could not possibly understand their technique. It is that they are selling smoke and mirrors. Let them peddle their brilliance on Wall Street—the quants there will buy anything (and did).

There is a saying in Las Vegas that if I sit down at a poker table and I cannot identify the patsy, the patsy is me.

Just don’t buy into solutions you don’t understand. It’s not that you’re not smart enough to get it; it is that the sellers of black boxes either aren’t smart enough to explain their solution effectively, or that they don’t understand it either. Regardless of which is the case, you don’t want to be a buyer.

(Pardon me if I sound passionate about this topic—I am. I have seen this happen too many times. A black box seller persuades someone in the company to buy it and inevitably the project blows up. Lots of money wasted and lots of time expended for naught.

Show them the door and find a market research supplier who meets the criteria here. You will get solid research and the pride of a job well done.

More to follow.

Comments welcome.

Dr. Bob

Here is how I evaluate quantitative market research suppliers:

First, what is their market research philosophy? How do they approach projects? Are they focused on actionable, practical results?

What I am really seeking with this series of inquiry is suppliers’ willingness to understand my needs and assist me in reaching my goals.

Of course, my first requirement with any supplier is their adherence to market research standards, to ensure that the study is representative of the population(s) I am looking to assess, to collect the data in a professional manner, and will keep me apprised of any issues.

Understanding how they approach projects is my first clue to their level of competence in these areas.

Second, are they trying to sell me a solution in search of a problem or do they really attempt to listen to my needs (and limitations, particularly in the budget area) and offer me a solution to my problems?

This is absolutely a deal-breaker. If I get a sense that a potential supplier is flogging one single tool or solution as a solution to all problems in market research, the supplier is off my list. This happens with surprising regularity. The “solution” they propose is often proprietary (see criteria three below in the next post) and usually carries a high price tag.

I expect a supplier to listen carefully to my needs and the project specifications and act and bid accordingly. I am especially concerned that they demonstrate a clear understanding of the project objectives.

There are only so many solutions in a market researcher’s bag of tools. I do not expect a supplier to invent a new tool. I simply expect them to give thought and consideration and a solid rationale for their recommended approach, methodology and analysis plan for my project.

Part 3 to follow.

Comments welcome.

Dr. Bob

Market researchers tend to explore the realms of consumer and customer perceptions and attitudes. Depending on the topics being investigated, a long history of both qualitative and quantitative research methodologies have established norms for obtaining valid and relatively precise data from respondents.

However, caution is required in market research when dealing with consumer and customer behavior.

A recent report by the Economist newspaper on the state of television illustrates this continuing issue. The Economist reports on the findings of Sarah Pearson, a research from England, who has almost 100,000 hours of recording of consumers watching television. As the Economist says, “There turns out to be an enormous gap between how people say they watch television and how they actually do….In surveys they almost always underestimate how much television they watch, and greatly overstate the extent to which they watch video in any other form.”

The accompanying data from Nielsen shows that consumers understate their television consumption by some 40% and overstate their consumption of online video by more than 100%.

Market researchers do well to proceed with caution in assessing the validity of self-reported behavior, such as product and service usage.

Comments welcome.

Dr. Bob

[Note: A subscription to the Economist is required to access this report online. It is available for purchase on their website.]

Recently perusing posts on market research, I read a blog reporting recent data about YouTube viewership. The blog stated that “American internet users…watched an average of 200 videos a day in the month of May….According to comScore Inc., a globally-accredited Marketing Research company, about 183 million Americans are tuned in YouTube every month watching 34 billion movies online. This means that there is an average of 186 videos watched per person in the US, based on comScore’s Video Metrix service.”

Hmmmmm. Every American internet user watched an average of 200 videos a day. Let’s assume for a moment that each video is 3 minutes in length. That’s 600 minutes, or 10 hours per day, spent watching videos on YouTube. Even if we assume a length of 2 minutes, that’s 400 minutes or 6.67 hours per day per user.

Plausible? No.

A quick consult of the data originator reveals that “183 million U.S. Internet users watched online video during the month….U.S. Internet users watched nearly 34 billion videos in May” or 186 videos during the month. With an average length of 4.3 minutes, according to comScore, this equates to 0.45 hours (26 minutes) of viewing per day per user.

Plausible? Yes.

Errors happen, of course. As market researchers, we have a responsibility to both check our facts and to subject our results to a plausibility test. Our credibility, and our ability to influence, are at stake.

Dr. Bob

Dr. Bob: But what about the bureaucracy that seems to invariably slow everything down? Inspections, custom forms, clearances, shipping manifests and the like?

Chetan:  On the B2B side of e-commerce this is all coming online. It’s an issue of scale. With such large volumes moving literally around and across the world, all the systems are adapting. There is a real push out there in the B2B world to move online, not only for the speed improvements but also for the accuracy of the information and the ability to track the timing of movement within the supply chain.

E-commerce connects all the entities in the supply chain together. Everyone can know precisely what each other one is doing, and can intervene if and when problems arise. So now, as a manufacturer I know there is a problem and I help fix it immediately rather than have a shipment of parts arrive only for me to then find out there was a problem.

And transparency is key. Each entity is responsible for their jobs in the process flow so that every piece can work in tandem and in sequence.

Dr. Bob: Thanks for that explanation! One last question: if B2B e-commerce is exploding, what is holding back B2C?

Chetan: It’s a combination of factors. Consumers can shop for pretty much everything online now, but they still want to walk into a store and touch and feel. They may do their research online, but they are still enjoying the tactile experience of shopping. The trend is moving, but very slowly.

Dr. Bob: What needs to happen to improve the transition rate?

Chetan: Online retailers have to find ways to increase both security and trust for consumers. And they need to be better organized internally by being proactive and more dynamic. In B2B, e-commerce is moving large quantities of a relatively small number of different items. In B2C, everything is being sold in small quantities. More e-commerce in B2C will require systems and can manage and fulfill in that environment.

Dr. Bob: Thank you, Chetan. How can readers contact you?

Chetan: They can visit www.digitalsabre.net or if they email you, Dr. Bob, on your Contact form, I’m sure you will pass it on to me.

Dr. Bob: Yes, I will!

Comments welcome!

Dr. Bob

One of the fascinating facts about Web commerce is that while marketing and selling to consumers has become all the rage and is soaking up the spotlight, a quiet revolution of much more startling proportions has been happening, unheralded and unnoticed.

Here are the eye-popping statistics:

As of 2007 (the most recent reliable data), business-to-business (B2B) e-commerce is far outstripping business-to-consumer (B2C or Retail) e-commerce by a ten-fold factor in the case of the manufacturing sector.

In Manufacturing, e-commerce accounts for 35% of total B2B sales; in the Merchant Wholesalers category, the figure is 21%.

E-commerce in the B2C area accounts for only 3.2% of total B2C Retail sales in the United States.

What is going on here?

To find out, Dr. Bob had a conversation with Chetan Amalean, President of Digital Sabre Solutions, LLC out of Charlotte, N.C. Chetan is a customer software developer and helps clients, especially mid-tier companies, establish and grow their e-commerce businesses.

Dr. Bob: Chetan, what is driving an e-commerce explosion in the B2B market, while B2C is a laggard?

Chetan: What we are witnessing is a rapid move to efficiency. Before e-commerce, manufacturers and wholesalers had no choice but to stock supplies; communication was relatively slow and lead times were long. There were bottlenecks throughout the supply chain.

What e-commerce has allowed is a massive shrinking of lead times in the supply chains. E-commerce has allowed businesses to fully realize the potential of just-in-time ordering and lean practices, moving closer and closer to real-time, rather than having to actually have stock in the warehouse.

E-commerce has affected everything: document exchanges, sourcing agreements, purchase orders, shipping notices, inspections, billing and payments. Before, this all happened via fax or snail mail.  Now it’s all e-commerce.

It’s all driven by demand and scale and quantity. E-commerce allows for much greater efficiency, because every partner in the supply chain knows exactly what is expected of them and when on a coordinated basis. It’s all right there on the e-commerce systems.

Part 2 to follow.

Comments welcome!

Dr. Bob