Filling the need for trusted information on national health issues…

Survey Methodology

This is the fourth in a series of surveys by the Kaiser Family Foundation (KFF) tracking the views and experiences of a group of Californians who were uninsured in the summer of 2013, prior to implementation of the ACA’s insurance market reforms and coverage expansions through Covered California and Medi-Cal. The first survey (Wave 1) was conducted from July 11-August 29, 2013, with a randomly selected group of individuals who were uninsured at the time of the interview, and was paid for entirely by KFF. The second survey (Wave 2) was conducted from April 1-June 15, 2014. The third survey (Wave 3) was conducted from February 18-May 13, 2015, and the current survey (wave 4) was conducted from February 10-June 20, 2016,   with the same longitudinal panel of respondents, whether they obtained coverage or remained uninsured. All four surveys were designed and analyzed by researchers at KFF. Social Science Research Solutions collaborated with KFF researchers on sample design, weighting, and supervised fieldwork. Fieldwork costs associated with Waves 2, 3, and 4 of the survey were paid for by The California Endowment.

The Wave 1 survey was conducted among a representative random sample of 2,001 adults ages 19-64 living in California who reported having been without health insurance for at least two months at the time of the interview1 (NOTE: persons without a telephone could not be included in the random selection process). Computer-assisted telephone interviews conducted by landline (990) and cell phone (1,011, including 660 who had no landline telephone) were carried out in English and Spanish by SSRS. To increase efficiency in reaching this low-incidence, hard-to-reach group, both the landline and cell phone sampling frames oversampled areas with a lower-income population (since being uninsured is negatively correlated with income). The landline sample frame also oversampled households whose phone numbers were matched with directory listings indicating the presence of at least one person age 19-64 and a household income of less than $25,000. Additionally, 230 interviews (130 landline, 100 cell phone) were conducted with respondents who previously completed recent national SSRS omnibus surveys of the general public and indicated they were ages 19-64 and uninsured. These previous surveys were conducted with nationally representative, random-digit-dial landline and cell phone samples. Waves 2, 3, and 4 also consisted of computer-assisted telephone interviews conducted in English and Spanish by landline and cell phone, including those who had no landline phone.

 Table 4: Comparing Respondent Contact Mode Across Waves
Total Landline Cell phone (no landline) Web
Wave 1 (July 11-August 29, 2013) 2,001 990 1,011 (660) Not applicable
Wave 2 (April 1-June 15, 2014) 1,219 623 545 (327) 51
Wave 3 (February 18-May 13, 2015) 1,105 555 463 (317) 87
Wave 4 (February 10-June 20, 2016) 1001 484 404 (240) 113

In order to re-connect with respondents who may be more willing to complete the survey online, an abbreviated web version was introduced for Waves 2, 3, and 4 after attempts had been made to reach respondents by phone. The online questionnaire was offered in English and Spanish and was limited to key questions about insurance status, type of coverage, and reasons for obtaining coverage or remaining uninsured. A total of 51 respondents in Wave 2, 87 respondents in Wave 3, and 113 in Wave 4 completed the online version of the survey.

Screening for Waves 2, 3, and 4 involved verifying that the respondent had participated in Wave 1. Multiple attempts were made to reach every respondent from Wave 1 and encourage participation in later waves. Efforts included multiple dialing at various times of day and throughout the week, mailings and emails to those who provided such contact information, repeated dialing of non-working numbers, and attempts to find alternative phone numbers for non-working numbers.

A multi-stage weighting design was applied to ensure accurate representation of California’s nonelderly adult uninsured population prior to the ACA’s coverage expansions. The weighting process for Waves 2, 3, and 4 involved corrections for sample design, as well as sample weighting to match the weighted Wave 2 sample, weighted Wave 3 sample, and weighted Wave 4 sample to Wave 1 responses along demographic characteristics. As it did for Wave 1, the base weight for Waves 2, 3, and 4 accounted for the oversamples used in the sample design, as well as the likelihood of non-response for the sample from earlier omnibus surveys, number of eligible household members for the landline sample, and a correction to account for the fact that respondents with both a landline and cell phone have a higher probability of selection. Demographic weighting parameters for Waves 2, 3, and 4 were based on Wave 1 weighted demographics, which were adjusted for age, education, race/ethnicity, nativity (for Hispanics only), Hispanics by gender, presence of own child in household, marital status, California region, poverty level, and phone usage. For more information on weighting and data sources, see the Wave 1 methodology. All differences referred to in the report are statistically significant. Statistical tests of significance account for the effect of weighting, and, for trend analysis, testing takes into account the survey’s panel design.

A unique consideration for panel surveys such as the Kaiser Family Foundation California Longitudinal Panel Survey, is whether those who participate in subsequent waves are different in terms of their attitudes or demographics than those who refuse to participate again or were unable to be re-contacted. Of the total 2,001 respondents who completed Wave 1, 1,219 participated in Wave 2, 1,105 completed Wave 3, and 1001 completed Wave 4. These completion rates are within an expected range given that the uninsured are already an often a difficult to reach population since many are lower income, younger, undocumented immigrants, and members of racial/ethnic minority groups, and may change phone numbers or move more often than the public at large. After data collection was complete, data from Wave 1 and Wave 4 were compared to evaluate the impact of some respondents not completing Wave 4, referred to as attrition. While there are some differences in the unweighted demographics of those who completed Wave 4 and the full Wave 1 sample, these observable differences are corrected for by weighting. As shown in the table below the total weighted distributions are similar for Wave 1 and Wave 4 for age, gender, race/ethnicity, party identification, education and income. See the Wave 4 Attrition Appendix for more information on attrition.

 Table 5: Comparing Respondent Demographics
Unweighted Weighted
Wave 1

(n=2001)

Completed Wave 4

(n=1001)

Completed all 4 Waves

(n=764)

Percentage Point
Difference
(W1 – W4 Total)
Wave 1 Completed Wave 4 Completed all 4 Waves Percentage Point
Difference
(W1 – W4 Total)
Gender
Male 48% 43% 41% 5 54% 53% 50% 1
Female 52% 57% 59% -5 46% 47% 50% -1
Race/ethnicity
White 27% 33% 36% -6 26% 27% 28% -1
Black 7% 8% 8% -1 5% 5% 5% 0
Hispanic 58% 51% 50% 7 56% 55% 54% 1
Other Race 8% 8% 7% 0 13% 12% 13% 1
Age
18 to 29 23% 19% 17% 4 33% 31% 28% 2
30 to 39 21% 19% 19% 2 24% 23% 24% 1
40 to 49 22% 20% 19% 2 21% 22% 23% -1
50 to 64 35% 43% 45% -8 22% 24% 25% -2
Education
HS or less 57% 49% 47% 8 58% 57% 57% 1
Some college 28% 33% 34% -5 29% 29% 29% 0
College Grad+ 15% 17% 18% -2 12% 13% 14% -1
Phone status
Landline 49% 55% 56% -6 42% 45% 48% -3
Cell 51% 45% 44% 6 58% 55% 52% 3
Marital status
Married 33% 34% 33% -1 37% 38% 37% -1
Not Married 67% 65% 67% 2 62% 62% 63% 0
Family income
<138% FPL 60% 57% 57% 3 52% 53% 55% -1
138%-400% FPL 30% 33% 33% -3 36% 34% 33% 2
400%+ 5% 6% 6% -1 7% 7% 6% 0
Language of interview
English 63% 70% 72% -7 65% 66% 64% -1
Spanish 37% 30% 28% 7 35% 34% 34% 1
Resident Status
Citizen/ legal immigrant 79% 85% 86% -6 78% 81% 81% -3
Undocumented immigrant 20% 14% 13% 6 21% 18% 18% 3
Party Identification
Republican 11% 13% 13% -2 11% 11% 12% 0
Democrat 35% 39% 39% -4 32% 34% 34% -2
Independent 35% 32% 32% 3 37% 36% 35% 1
Other 9% 8% 8% 1 9% 9% 9% 0

Another consideration for panel surveys is the potential for “sensitization effects,” that is, what effect returning to the same people about the same topics has on their experiences or views. For example, after taking the baseline survey that covered many aspects of the coverage expansions under the ACA, were people more likely to seek out information about health insurance and enroll than they would have been otherwise? While there is no direct way to measure this effect on this survey, other analyses have found that these effects are minimal and short-lived,2 and we do not believe they would have had a substantial impact on results presented here, particularly given all the other media coverage, advertising, and outreach targeted at this population during the fall and winter of 2013, 2014, and 2015.

The margin of sampling error including the design effect for the full sample is plus or minus 5 percentage points. For the recently insured, it is plus or minus 5 percentage points and for the remaining uninsured it is plus or minus 9 percentage points. Numbers of respondents and margin of sampling error for key subgroups are shown in the table below.

 Table 6: Margin of Sampling Errors Across Subgroups
Group N MOSE
Total Wave 4 1001 +/- 5 percentage points
Recently Insured 781 +/- 5 percentage points
     Recently insured by Medi-Cal 393 +/- 7 percentage points
     Recently insured by non-group plan 147 +/- 11 percentage points
          Recently insured through Covered California 121 +/- 12 percentage points
     Recently insured through an employer 180 +/- 10 percentage points
     Recently insured Hispanics 366 +/- 7 percentage points
Remaining Uninsured 218 +/- 9 percentage points
     Remaining uninsured Hispanics 143 +/- 11 percentage points

For results based on other subgroups, the margin of sampling error may be higher. Sample sizes and margin of sampling errors for other subgroups are available by request. Note that sampling error is only one of many potential sources of error in this or any other public opinion poll.

Some demographic measures referred to in the report were only asked during the baseline survey, such as questions about educational attainment, debilitating chronic condition, length of time uninsured, resident status, and race/ethnicity. For more information on the first and second waves of the Kaiser Family Foundation California Longitudinal Panel Survey visit:

Wave 1: http://kff.org/health-reform/report/californias-uninsured-on-the-eve-of-aca-open-enrollment/

Wave 2: http://kff.org/health-reform/report/where-are-californias-uninsured-now-wave-2-of-the-kaiser-family-foundation-california-longitudinal-panel-survey/

Wave 3: http://kff.org/health-reform/report/californias-previously-uninsured-after-the-acas-second-open-enrollment-period/

About The Terms Used In This Report Appendices

The Henry J. Kaiser Family Foundation Headquarters: 2400 Sand Hill Road, Menlo Park, CA 94025 | Phone 650-854-9400
Washington Offices and Barbara Jordan Conference Center: 1330 G Street, NW, Washington, DC 20005 | Phone 202-347-5270

www.kff.org | Email Alerts: kff.org/email | facebook.com/KaiserFamilyFoundation | twitter.com/KaiserFamFound

Filling the need for trusted information on national health issues, the Kaiser Family Foundation is a nonprofit organization based in Menlo Park, California.