In-Brief Note

Amid concerns of rising anxiety and depression driven by the COVID-19 pandemic (Vahratian et al. 2021), reliable and validated screening instruments to track population mental health are crucial. The 2-item Patient Health Questionnaire (PHQ-2) and 2-item Generalized Anxiety Disorder Scale (GAD-2) are widely used short screeners for probable depression and anxiety, respectively. While developed for use in clinical settings, these instruments have been validated for use in population surveys (Löwe et al. 2010). They begin with the question: “Over the last 2 weeks, how often have you been bothered by the following problems?” Explicit response options typically include “not at all,” “several days,” “more than half the days,” and “nearly every day.” When used in clinical settings these questions typically do not include a “Don’t Know” (DK) response option (Sapra et al. 2020). However, when used in surveys, some instruments have included DK (for example, National Health Interview Survey Questionnaire [Centers for Disease Control and Prevention 2019]), while others have not (for example, the Household Pulse Survey [United States Census Bureau 2022]). When adapting clinical instruments for surveys, does the inclusion of an explicit DK option impact population mental health prevalence estimates? We aimed to explore this question, considering two conflicting concepts in survey practice: (1) DK options in sensitive questions reduce survey breakoff, and (2) DK options increase item nonresponse (Alwin and Krosnick 1991; Schuman and Presser 1996).

During the COVID-19 pandemic, the New York City Department of Health and Mental Hygiene (NYC DOHMH) has conducted Health Opinion Polls (HOPs) using probability-based samples to assess health and health-related opinions among NYC adults. HOPs are primarily self-administered online; some responses are collected over the phone with an interviewer. From March 2021 to November 2021, four HOPs were conducted that included the GAD-2 and PHQ-2 scales with a DK option to mirror the response options of the other questions on the largely opinion-based survey. DK responses were recoded as missing and excluded from analysis. In August 2022, a HOP was conducted without a DK option, and probable depression and probable anxiety decreased relative to previous waves. To investigate if this change was due to the altered response options or a true change over time, in the October 2022 HOP, respondents were randomized to receive (n=683) or not receive (n=687) an explicit DK option on the GAD-2 and PHQ-2 scales.

The two groups did not differ significantly (α = 0.05) on major demographic characteristics or mode of survey completion (phone vs. web). The mental health questions were located at the end of the survey, and the two groups had similarly low breakoff numbers. However, consistent with previous literature (Schuman and Presser 1996), nonresponse (including DK responses) was higher among the group with the explicit DK option (67 vs. 4). The unweighted prevalence of probable anxiety was higher among the group with the explicit DK option (18.94% vs. 14.72%, χ²=4.24, p-value=0.0395), but probable depression was not significantly different between the two groups (15.03% vs. 11.99%, χ²=2.61, p-value=0.1060). Among individual items, scores on one item on the GAD-2 (not being able to stop or control worrying) differed significantly between the two groups (χ²=8.25, p-value=0.0412).

While further research comparing with a “gold standard” measure is needed to establish whether including a DK option yields estimates closer to the true prevalence, these results have implications for survey practice. Including an explicit DK option can significantly alter prevalence estimates of some mental health outcomes. When adapting clinical instruments for surveys, researchers should consider whether including a DK option will yield the most accurate results.

Funding

This work was supported by the Centers for Disease Control and Prevention Grants: Epidemiology and Laboratory Capacity for infectious Diseases SUPPLEMENTAL COVID-19 CARES: NU50CK00517, Epidemiology and Laboratory Capacity for infectious Diseases SUPPLEMENTAL COVID-19 PPPHCE NU50CK00517, ELC COVID Enhancing Detection Expansion SUPPLEMENTAL: NU50CK00517, Immunization & VFC COVID-3 CYCLE VACCINATION SUPPLEMENTAL: 6 NH23IP922636, CDC COVID-19 Crisis Response Workforce supplemental grant: 6 NU90TP922148, and CDC Public Health Infrastructure grant: 1NE11OE000057.

Competing interests

The authors have no competing interests to report.