AAPOR 2016 - Eran Ben-Porath & Kaiser

HABITUAL & NON-HABITUAL RESPONDENTS IN A LONGITUDINAL TELEPHONE STUDY:

DIFFERENCES & IMPLICATIONSP R E S E N TAT I O N F O R A A P O R 2 0 1 6

1

ERAN BEN-PORATH, SSRS

BIANCA DIJULIO, KAISER FAMILY FOUNDATION

MOLLYANN BRODIE, KAISER FAMILY FOUNDATION

JAMIE FIRTH, KAISER FAMILY FOUNDATION

KAISER FAMILY FOUNDATION CALIFORNIA UNINSURED LONGITUDINAL PANEL SURVEY

• Baseline Survey: July-August 2013 (Prior to Open Enrollment)

• 2001 Randomly Selected California Residents:

− Ages 19-64

− Uninsured for at Least 2 Months at the Time of the Baseline Survey

• RDD Landline and Cell Phone, Oversampled Areas with Lower-income Populations

• English and Spanish

• Wave 2: April-June 2014 (After Open Enrollment Period Closed)

• Wave 3: Feb-May 2015 (After 2nd Open Enrollment Closed)

• Wave 4: Feb-June 2016 (After 2nd Open Enrollment Closed)

• Funded in Part by the California Endowment

• All Waves Fielded by SSRS

© SSRS & Kaiser Family Foundation 2016 2

EFFORT TO MAINTAIN COOPERATION RATES

Attempts to Recontact Respondents at Waves 2-4

• Collected Multiple Pieces of Contact Information, Emails, Alternate Phone #S• Incentives to All Respondents, Including Following Up with Those Who Didn’t Cash Their Checks to

Make Sure They Received it• Waves 2-4: Pre-notification Letters with $2 Pre-incentives to Anyone Who Gave Us Their Address

for Incentive• Up to 30 Callback Attempts• Reminders Via Email (if Provided An Email Address), Postcards, Voicemails• Look-up For Alternative Phone Numbers For Numbers That Came Up As Non-working • Continue Re-dialing Non-working• Allowing Respondents to Complete Interviews “On Demand” if They Call in Directly • Web Version of the Survey for People When Several Attempts to Contact Via Phone Were

Unsuccessful


FOUR TYPES OF RESPONDENTS (BY COOPERATIVENESS)

I II III IV

Type Dropouts Casual Committed Habitual

Definition Only Wave I Any 2 waves Any 3 waves All 4 waves

N



I II III IV



N 560 367 365 709



I II III IV



N 560 367 365 709


• Assumption: Non-random differences between types.• Systematic non-response

DEMOGRAPHIC DIFFERENCES BETWEEN RESPONDENTS TYPES

55.4%

50.7%

45.5%

42.0%

44.6%

49.3%

54.5%

58.0%

DROPOUTS

CASUAL

COMMITED

HABITUAL

GENDER

m f



31.4%

23.2%

20.3%

16.4%

23.9%

21.0%

22.2%

18.6%

19.5%

27.2%

22.2%

19.9%

25.2%

28.3%

35.3%

45.1%

DROPOUTS

CASUAL

COMMITED

HABITUAL

AGE

18-29 30-39 40-49 50+



32.9%

29.7%

32.9%

22.0%

31.8%

32.2%

26.6%

24.5%

21.1%

27.0%

25.2%

34.6%

12.5%

10.6%

14.2%

18.3%

DROPOUTS

CASUAL

COMMITED

HABITUAL

EDUCATION

<HS HS Grad Some Coll Coll+



OTHER SIGNIFICANT DIFFERENCES

HABITUALS: Significantly More Likely to Have Completed Original

Interview in English than Any Other Type (and Conversely: Least Likely in Spanish)

HABITUALS:Significantly More Likely to Have had a Landline in Their

Homes (At T1)


W1 ATTITUDINAL DIFFERENCES BETWEEN RESPONDENTS TYPES

52.0%

50.1%

56.7%

45.8%

21.1%

25.6%

24.9%

33.9%

27.0%

24.3%

18.4%

20.3%

DROPOUTS

CASUAL

COMMITED

HABITUAL

ACA FAVORABILITY

Favorable Unfavorable DK/Ref


BEHAVIORAL DIFFERENCES BETWEEN RESPONDENTS TYPES?

47.3%

49.6%

48.2%

48.7%

31.4%

33.0%

31.2%

32.6%

DROPOUTS

CASUAL

COMMITED

HABITUAL

INTENT TO OBTAIN COVERAGE

Will obtain health insurance Will remain uninsured



27.1%

35.1%

35.6%

41.2%

34.8%

29.7%

26.6%

21.2%

DROPOUTS

CASUAL

COMMITED

HABITUAL

PREVIOUS INSURANCE

Employer Never Insured



50.9%

47.9%

53.9%

47.8%

48.9%

41.6%

CASUAL

COMMITED

HABITUAL

W2 COVERAGE

Covered Not Covered



67.3%

65.1%

72.6%

31.7%

34.6%

25.8%

CASUAL

COMMITED

HABITUAL

W3 COVERAGE

Covered Not Covered


ASSESSING IMPACT OF HABITUAL RESPONSE/NON-RESPONSE

• The share of 4-wave habituals increases

• Wave 4: prevailing majority of respondents are Habitual

• Assessing impact of nonresponse

• Using W3 data


ASSESSING IMPACT OF HABITUAL RESPONSE/NON-RESPONSE

• The share of 4-wave habituals increases

• Wave 4: prevailing majority of respondents are Habitual

• Assessing impact of nonresponse

• Using W3 data

• Propensity Scoring:− Model includes

Demos

Past insurance pattern

− Controlling for demos:

Past insurance is significant


INCORPORATING PROPENSITY SCORE INTO WEIGHTING

Mean Propensity Weight

Male 1.137

18 to 29 1.331

50 to 64 0.802

Spanish 1.196

White 0.746

HS or Less 1.138

CPO 1.203

Always Uninsured 1.256

Once Employer 0.872

Inverse of propensity score included in base-weight


WEIGHTED OUTCOMES

W3 Habitual Adjustment

Without With

Uninsured 34.5% 35.0%

ESI 14.5% 14.2%

Medi-Cal 33.3% 32.9%

Self Purchased 12.9% 12.9%

With inverse of propensity score included in base-weight


CONCLUSION: NEED TO ACCOUNT FOR HABITUAL RESPONDENTS IN PANEL DATA

• Risk posed in these data seems addressed in weighting

• But: Indication of possible association between habitual respondent status and substantive outcome variables

• Weighting to original sample – an option for minimizing this risk

• Nonresponse reduces statistical power in panel research:

• Smaller N

• Greater Design Effect

• Restricted ability to analysis subgroups

• Findings indicate a concurrent need to take action to minimize bias


ERAN BEN-PORATH, SSRS@en_bp

BIANCA DIJULIO, KAISER FAMILY FOUNDATION@BeeDiJulio

MOLLYANN BRODIE, KAISER FAMILY FOUNDATION@Mollybrodie

JAMIE FIRTH, KAISER FAMILY FOUNDATION

Data & Analytics

AAPOR 2016 - Eran Ben-Porath & Kaiser