About Our Surveys
KFF has a long history of conducting public opinion polls to amplify the public’s voice in policy debates, including groups that are under-researched and under-reported. For the past three decades, KFF’s polling work has combined both rigorous methodology and transparency. Every survey we publish is accompanied by a full methodology statement and topline results, so readers can see details about how the data was collected, read every question included in the survey in the order in which they were asked, and judge its quality for themselves.
While asking the public about political issues and candidates is an important part of our polling work, KFF does not conduct “horse race” polling with the goal of predicting election outcomes. Our election-related polling focuses on how the public thinks about health care issues when making voting decisions, not on forecasting who might win. While certain things about our survey methods change from project to project, the principles of our methodology are consistent and described below. To find more details for each survey, check out the “Topline & Methodology” link at the top right of each survey project on our website.
How We Reach People
KFF surveys rely almost exclusively on probability-based sampling, meaning respondents are selected at random rather than choosing to participate on their own. Most of our national population surveys use a sampling approach that combines two sample types: 1) a probability-based panel and 2) a random digit dial sample of pre-paid cell phone numbers.
Probability-based panel members are recruited through random sampling, using two main methods: mailed invitations sent to addresses randomly selected from U.S. Postal Service delivery records, and random digit dialing of cell phone numbers. This means that, in principle, every U.S. adult has a known chance of being invited to join the panel, whether or not they were looking to take a survey. Once someone joins the panel, they can complete our surveys online, by phone, or in some cases by text message invitation to a web survey. This hybrid approach helps us reach people who might otherwise be left out of an online-only survey, including older adults, people with lower incomes, and people without reliable internet access. KFF works with multiple probability-based panels including the SSRS Opinion Panel, Ipsos Knowledge Panels, and NORC AmeriSpeak.
For most projects, we supplement probability-based panels with a sample of random digit dialed pre-paid cell phone numbers. Pre-paid or “pay as you go” cell phone numbers are used disproportionately by population groups that respond at lower rates to traditional survey methods, including young Black and Hispanic men, and people who do not speak English. Combining these sample types helps our surveys be more representative of the diversity of the national population, not just those who are easiest to reach.
Our national survey projects are conducted in English and Spanish, usually fielded over 5-6 days so people have more time to respond, and provide small financial incentives for individuals who participate, to help bring in those who might otherwise skip a survey. Without these steps, surveys run the risk of overrepresenting the most engaged or opinionated members of the public.
In addition, KFF conducts deeper surveys on populations that general population polling cannot adequately capture. These projects start from the same principle as our general population surveys: probability-based sampling, whether through probability-based panels, address-based recruitment, voter registration lists, or purpose-built panels of previous respondents, adapted to fit the population being studied. Reaching a small, specific, or hard-to-reach group often requires fielding a survey in more languages, using more modes of contact, and longer field periods.
How We Work To Be Representative
Even with a well-designed sampling frame, most polls will not perfectly mirror the population on every characteristic, so we use statistical techniques to adjust, or weight, the results to align with known population benchmarks from sources like the Census Bureau on measures such as age, gender, race and ethnicity, education, and region. Weighting corrects for the fact that some groups are easier to reach than others and helps ensure that no single group is overrepresented or underrepresented in the final results.
All KFF survey reports include margins of sampling error for the total sample and key subgroups, which reflect the statistical uncertainty that comes with asking questions of a sample rather than every single member of the population. In addition to sampling error, all public opinion polls may be subject to other types of error such as question wording effects, nonresponse bias, or mode effects.
Our Polling Expertise and News Partnerships
KFF has a team of polling experts who design our surveys, write questionnaires, analyze data, and write reports. This team works closely with KFF health policy experts who contribute their knowledge on issues like health reform, ACA, Medicare, Medicaid, women’s health, global health, and other issues to the survey design and reporting process.
We don’t commission others to conduct polling on our behalf or undertake surveys on behalf of other organizations. We contract with outside organizations to perform survey fieldwork and statistical weighting, which is paid for through KFF’s endowment and sometimes using funds from outside funders. KFF maintains full editorial control over all its policy analysis, polling, and journalism activities, regardless of funding source.
Some of our surveys are conducted in partnership with major news organizations in order to help KFF magnify its polling by combining survey data with in-depth journalism and storytelling, an approach that KFF has used since 1995. KFF and its partners choose survey topics together and design the survey instrument with input from both organizations while retaining full editorial independence over what each organization publishes.
Artificial Intelligence in Survey Research
KFF uses Artificial Intelligence (AI) in specific ways to strengthen our work and we publicly disclose when we’ve used AI in a way that is central to our survey methods. For example, we often use AI tools (with human oversight) to assist with coding of open-ended survey responses into discrete categories. KFF does not use AI to create or model “synthetic” public opinion. Every survey we field is grounded in real interviews with human respondents, selected through probability-based sampling, never generated, simulated, or estimated by AI standing in for actual human answers.
Our Commitment to Methodological Transparency
KFF public opinion and survey research is a charter member of AAPOR’s Transparency Initiative. The American Association for Public Opinion Research (AAPOR) Code of Professional Ethics and Practices commits members to maintain the highest standards of scientific competence, integrity, accountability, and transparency in designing, conducting, analyzing, and reporting their work, and requires researchers to disclose sufficient information about how the research was conducted to allow for independent review and verification of research claims, regardless of the methodology used.
Excellent survey research is not just about getting people to answer questions, it is about making sure the estimates are representative, can hold up to scrutiny, and are collected in a way that respects the people who take the time to answer. As trust in institutions, including in survey research itself, continues to decline, KFF remains committed to the methods that have proven most reliable over decades of public opinion research: random selection, careful weighting, and full transparency about how the work gets done. Reach out if you have any questions about our polls or our methodology.
