Five things everyone needs to know about online panels
Online surveys and research panels go together like…well, milk and cookies. If they are both high-quality, yummy snack. If either is poor quality, the experience is ruined.These days, there is a lot of awareness of online panel quality issues. So what does a market research buyer need to know?Here are 5 things you need to be aware of to find the best panel sources for your needs, and mitigate potential risks. Why is this so important? Because panel quality varies. A lot. You can’t assume all research panels are the same.
- Some panel companies focus on selling to market research agencies—not folks looking to do research in-house. So don’t be surprised if you find some agencies have a sales process that feel awkward to you.
- Panel suppliers have different processes for validating the authenticity of their members. Some do a better job of weeding out “incentive hunters” and “professional respondents” than others. A key concept here is “digital fingerprinting.” For an excellent introduction, I strongly recommend this very easy-to-read article, “Digital Fingerprinting and Sample Quality,” from Simon Chadwick as a guest blogger on the QuestionPro blog.
- Panel suppliers have varying policies about research participation for their panel members. In some panels, a member can take 10 surveys a day. In others, it is restricted to a few per month. If you want to know more about some of the brutal realities, read the new and controversial report, “The Dirty Little Secrets of Online Panels”, thanks to the research of Ron Sellers from Grey Matter Research. Read the article for instructions on how to get the full document.
- Just because a panel company allows you to use a very long, onerous questionnaire with their panelists, doesn’t mean it’s a good idea. They just make more money that way. For more on this and related points, read Jeffrey Henning’s excellent article, “Data Quality & Validation Lessons from Panel Professionals.”
- Some panel companies aggregate multiple panels. Some panel companies sell access to their own panels. But there are a few that aggregate, and de-duplicate, across multiple panels. These aggregators can be handy, especially if you are doing research with a hard-to-find population.
OK, those 5 points are key. But if you can handle a little more, I also recommend reading these 3, short articles for more tips and practical insights into online market research panels:
- From Forrester’s Reineke Reitsma, on Online Panel Quality.
- From ESOMAR, the wonderful “26 Questions” to ask before you buy online sample.
- From the QuestionPro blog, Steven H. Gittelman wrote a short blog about how Microsoft has dealt with buying online panel sources.
Look, I am not trying to scare you off. But there are risks, and by being informed you will be able to make better decisions about which panel companies to work with. There are plenty of them out here—but you do need to be vigilant to find one that will best meet your needs.