The old cartoon of a dog tapping on a keyboard with the caption, on the Internet no one knows you’re a dog still applies. Market researchers have to take careful, considered steps to ensure that in Internet surveying they are really talking to the population they think they are talking to.
For businesses that use other communication and distribution channels, relying solely on the Internet for market research, despite “black box” solutions to the sampling challenge, is a recipe that must be carefully brewed.
Bottom line? Fundamentals are fundamental.
Some years ago, I attended the ART (Advanced Research Techniques) conference of the AMA (American Marketing Association). One of the presentations was about the analytic power of a new statistical modeling technique that one of the big guns in academic market research had developed. It happened that another big gun, and one who had a competing model, was in attendance. The two almost came to blows about which model was more accurate and relevant. I set in the session, fascinated with the altercation. I was very tempted to add fuel to the fire by pointing out a fundamental flaw in both of their models: they were developed and tested using N sizes of under 100 respondents. (I didn’t.)
Now I’m not a Ph.D. in statistics. I work exclusively in applied, or practical market research. But even a student in Statistics 101 knows that a sample of less than 100 yields qualitative data only (unless, of course, the population being sampled is less than 1000). End of story.
I’m sure these gurus had a jargon-laden explanation of why, in their cases, it was acceptable and reasonable to break the fundamental rules of quality market research.
But there are no excuses.
If the fundamentals are not met, the results are fallacious.
It’s the same for market research 2.0 as with market research 1.0. Do the fundamentals according to correct statistical practice.
Comments welcome!
Dr. Bob
{ 1 comment… read it below or add one }
Thanks for the interesting article. I’m gathering any information I can find on conducting market research with web 2.0. and from what I understand it can be only qualitative in nature (for now, unless someone comes up with a quantitative technique for this). But considering social media/web 2.0 are the rage these days, will companies even care about quantitative research and probability sampling anymore?