Research On Research: Rigorous Panel Management Controls Heavy Responder Bias
TNS, Ipsos And The NPD Group Conduct Study To Address Market Research Industry Concern
Monday, October 23, 2006
New York, NY – A new study from three leading market research companies, TNS, Ipsos and The NPD Group, indicates that carefully managed access panels are the key to mitigating the impact of “heavy responders” on consumer studies conducted via online methodologies.
As online research has become increasingly popular, the industry has seen a new trend in survey participation, with a subset of consumers emerging as heavier responders. To address the issue, TNS, Ipsos and The NPD Group teamed up to conduct “Research on Research” and assess the prevalence of heavier responders and their true impact on research results. The results are published in a new report, Heavier Responders in Online Survey Research, which reveals that rigorous panel methodology successfully minimizes research biases in the final results by balancing the use of well-crafted incentive systems, survey invite frequency, and client survey demand.
The report examines the frequency of survey participation over several months, factors influencing that participation, and the variation in responses among approximately 1,000 panelists from each company’s managed access panel. Researchers fielded 14 concept tests across eight categories, the results of which were analyzed, along with additional information about panelists’ other research participation.
“Any online sample today will include frequent survey responders,” explained Efrain Ribeiro, Global Head of Access Panels for TNS. “However, this report clearly shows that by reducing reliance on those responders, it is possible to control the impact on results and thus ensure the quality of research.”
Lee Markowitz, Chief Research Officer with Ipsos in the U.S, concurs. “As experienced, professional researchers, we place the highest priority on controlling a number of factors that we know will influence the final product to ensure findings are valid,” he said.
The study found that approximately eight out of ten survey responders claimed to complete ten or fewer surveys during a 30-day period. The average number of surveys was 7.9. In contrast, the average number of surveys presented by the participating supplier (Ipsos, The NPD Group or TNS) and were completed by individual panelists was four in the past 30-day period. Analysis of the make-up of panels by survey participation revealed that the Ipsos, NPD or TNS panels contain far fewer heavy responders than other panel companies.
The study also found that while heavier responders did have lower purchase intent and value scores, the lower percentage of heavier responders on the Ipsos, NPD or TNS panels meant that survey results were not meaningfully impacted. Compared to the total sample results, when data from the heaviest responders were excluded, neither the mean nor top two box sub-sample concept scores differed. The same was true when results from respondents who participated in 10 or fewer surveys were compared to the total sample results.
Ipsos, NPD and TNS each rely on balanced recruiting techniques and ongoing panel management to ensure optimal panel composition. All three companies recruit panelists using a range of techniques and incentives so that no single incentive (such as financial compensation, often seen as the primary motivator for heavier responders) dominates. Once panelists are recruited, extensive efforts are taken to guard against opinion fatigue, including limiting number of surveys, targeting and varying survey topics, and sample balancing on multiple demographic targets.
“While in some respects, consumer research has become more accessible than ever, given the advantages of the Internet – lower costs in particular – we have new issues to contend with in the new environment,” said Steve Coffey, Chief Research Officer for The NPD Group. “Taking the appropriate steps to ensure quality are more important than ever. We are committed to making the investments necessary to ensure quality today and in the future.”
For more information on this press release, please contact:
Chief Research Officer
New York, NY
About Ipsos in North America
Ipsos is one of the fastest growing market research companies in the U.S., market leader in Canada, and among the most trusted research brands in North America. With more than 1,300 professionals and support staff in the U.S. and Canada, Ipsos offers a suite of survey-based services—guided by industry experts and bolstered by advanced analytics and methodologies—in advertising, customer loyalty, marketing, and public affairs research, as well as forecasting, modeling, and consulting. Ipsos companies offer a complete line of custom, syndicated, omnibus, panel, and online research products and services.
Ipsos conducts polling on behalf of the Associated Press, the world’s oldest and largest news organization.
In 2005, Ipsos generated €717.8 million ($853.8 million U.S.) in total revenues, of which 34% came from its North American operations. The Paris-based company was founded in 1975 and has been listed on the Paris Stock Exchange since 1999.
To learn more, visit: www.ipsos-na.com
Ipsos is a leading global survey-based market research company, owned and managed by research professionals. Ipsos helps interpret, simulate, and anticipate the needs and responses of consumers, customers, and citizens around the world.
Member companies assess market potential and interpret market trends. They develop and build brands. They help clients build long-term relationships with their customers. They test advertising and study audience responses to various media. They measure public opinion around the globe.
Ipsos member companies offer expertise in advertising, customer loyalty, marketing, media, and public affairs research, as well as forecasting, modeling, and consulting. Ipsos has a full line of custom, syndicated, omnibus, panel, and online research products and services, guided by industry experts and bolstered by advanced analytics and methodologies. The company was founded in 1975 and has been publicly traded since 1999. In 2005, Ipsos generated global revenues of €717.8 million ($853.8 million U.S.). Visit www.ipsos.com to learn more about Ipsos’ offerings and capabilities.
Ipsos, listed on the Eurolist of Euronext – Comp B, is part of SBF 120 and the Mid-100 Index, adheres to the Next Prime segment and is eligible to the Deferred Settlement System. Isin FR0000073298, Reuters ISOS.PA, Bloomberg IPS:FP