subject: Demystifying The Panel Creation Process [print this page] Panels used for sampling are not all created equal. When you plan a market research project, one of the most significant hurdles you must jump is developing a high quality sample. One of the most important questions you need to ask your sample provider about the panels they sell includes (but is certainly not limited to), How were panelists recruited?
Recruitment is typically conducted online in spite of the claims panel companies may make. However, the best panels use some form of verification (e.g., telephone interviews, IP address tracking, and always a double opt-in with CAPTCHA, which stands for Completely Automated Public Turing Test To Tell Computers and Humans Apart). The quality of the samples you extract from a panel will increase when one or more of these confirmation methods are actively managed. In addition, panelists profiles should be updated using a systematic and scheduled process.
Are Panels More than a Database of Emails?
All panels are housed in some form of database. However, the sophistication of the database structures and capabilities to manage the panel vary. The best of the panel management practices track panelists activity across studies and include a process for updating and expanding panelist profiles. The management process should be capable of de-duplication of panel members selected for a study when a panel provider uses partners (many panelists register with more than one panel provider). Without a de-duplication process the same person can respond twice (or more) without your knowledge.
Some panel providers use a community approach to supplying sample. The profiles and refresh cycles of names and emails is likely to vary from community to community, which presents a challenge for the providers and for you as a researcher. The Community Model is gaining momentum and reasonably sophisticated software is available to handle the duplication problem. However, the issue of how representative the communities are of a defined market remains a serious concern.
If your goal is to better understand a specific community this issue may be mitigated or not exist. If on the other hand you want to generalize beyond a specific community, it would be best to know the demographic parameters of the community and the target markets to which you want to generalize your results. Making comparisons between these populations to explore whether significant differences exist is an important part of the sample quality control process. If there are significant differences, you may need to be cautious and perhaps consider a weighting approach to compensate for differences.
Does the Panel Provider Profile?
The idea of profiling has become associated with terrorism and the geo-political arguments for and against practicing profiling. We are not talking about that form of profiling. Panel providers efforts to profile panelists have two major objectives:
1. Send panelist to complete questionnaires they are qualified to take
2. Pre-qualify panelists to improve the incidence rates for clients
These are clearly complementary efforts. Profiling is used to better understand each panelist. The more you know about a panelist or more precisely the panel as a whole the better you can target a specific group of panelists (potential respondents) to meet the objectives of a project.
Invitations to complete a questionnaire are sent to panelists who appear to qualify. Screening questions are then used to used to identify and qualify respondents with greater specificity. However, screening questions are not a panacea, but thats a topic for another time.
