Our Brave New World: Ipsos Comment on AAPOR’s Statement on Credibility Intervals
Thursday, October 18, 2012
Washington, D.C. - The AAPOR statement on "Understanding a credibility interval" is fair, balanced, and agile in showing the complexity of the issues: in particular, that our changing survey research world demands new approaches and
ways of thinking, especially with respect to nonprobability designs. Ipsos, as one of the firms which conducts
online polling and employs credibility intervals, believes that a Bayesian approach is one possible way to address
the "nonprobability inference conundrum" (Roshwalb et al).
To further the discussion, we have several additional comments:
(1) We do not purport to have all the answers nor that a Bayesian approach is the only way forward. Here it is
important to remember that our field of survey research is in a period of rapid change where old benchmarks are
under increasing pressure as humans interact and communicate in new virtual ways. This puts a premium on
experimentation – ours is one such attempt. Ipsos does not believe in 'black boxes' nor 'special sauce' and as such
all learnings from this polling program which are scientifically relevant will be shared with the larger community in
conferences and white papers.
(2) Ipsos believes in complete transparency in the process and is a signatory to the AAPOR transparency initiative.
To date, our methods statements on our polls have included an explanation of Bayesian Credibility Intervals as well
as a statement that MOE cannot be used for online polls. We will also be including a link to the AAPOR statement
on our methods blurb for each poll. We believe it does a good job of educating the consumer.
(3) One of the central concerns raised by the AAPOR statement is with the biases associated with opt-in, nonprobability,
online surveys. We know that in general all surveys, including online, suffer from an assortment of
biases including self-selection, non-response, and non-coverage. Our own correction model attempts to control for
each of these types of bias at the design stage as well as at the post-survey weighting stage. Ultimately, we are still
refining our model, and these learnings will be one of the outputs of this year's initiative*.
(4) One important point we feel the AAPOR Statement does not fully address is the ever changing world of online
research itself: from one of 'single panel' online surveys to one of online surveys with multiple sample sources. In
practice, research firms, including Ipsos, no longer exclusively use single opt-in online panels (Young et al). Our
online poll, for instance, includes a 'blend' of multiple panel and non-panel sources, such as social media and
rewards sites. In our opinion, this is significant for two reasons. First, we minimize the bias associated with any one
panel or source. Consequently, our correction models become potentially more generalizable than panel-specific
ones. Second, the long-term possibilities are potentially manifold. Think about this: a research world where our
sample frames are virtual and where websites, or 'virtual nodes', are our sampling units. Yes, this is our brave new
world of survey research: one of much uncertainly but also one of much possibility!
If you have questions or comments, or would like to get in touch with Ipsos about our approach to online blended
sampling, please contact:
Ipsos Public Affairs
SVP and Managing Director, Polling and Public Sector
Ipsos Public Affairs
(*) The Reuters / Ipsos Online Election Polling Program:
Ipsos is undertaking an unprecedented polling engagement with our media partner Thomson Reuters which
includes daily online tracking polls. By the end of the electoral season, we will have conducted approximately
150,000 interviews which allow for micro-targeting and other subgroup analyses together with the daily horserace
numbers. Even now, users can access the micro data at http://elections.reuters.com/#poll
About Ipsos Public Affairs
Ipsos Public Affairs is a non-partisan, objective, survey-based research practice made up of seasoned professionals. We conduct strategic research initiatives for a diverse number of American and international organizations, based not only on public opinion research, but elite stakeholder, corporate, and media opinion research.
Ipsos has media partnerships with the most prestigious news organizations around the world. In the U.S., UK and internationally, Ipsos Public Affairs is the media polling supplier to Reuters News, the world's leading source of intelligent information for businesses and professionals, and the Hispanic polling partner of Telemundo Communications Group, a division of NBC Universal providing Spanish-language content to U.S. Hispanics and audiences around the world.
Ipsos Public Affairs is a member of the Ipsos Group, a leading global survey-based market research company. We provide boutique-style customer service and work closely with our clients, while also undertaking global research.
To learn more visit: www.ipsos-na.com
Ipsos is an independent market research company controlled and managed by research professionals. Founded in France in 1975, Ipsos has grown into a worldwide research group with a strong presence in all key markets. In October 2011 Ipsos completed the acquisition of Synovate. The combination forms the world’s third largest market research company.
With offices in 84 countries, Ipsos delivers insightful expertise across six research specializations: advertising, customer loyalty, marketing, media, public affairs research, and survey management.
Ipsos researchers assess market potential and interpret market trends. They develop and build brands. They help clients build long-term relationships with their customers. They test advertising and study audience responses to various media and they measure public opinion around the globe.
Ipsos has been listed on the Paris Stock Exchange since 1999 and generated global revenues of €1,363 billion (1.897 billion USD) in 2011.