Vocal vs. typed responses in AI-Moderated Interviews: A Comparative Study of Data Quality, Disclosure, and Participant Experience

1. Introduction

The adoption of conversational AI has accelerated the use of AI-moderated interviews (AIMIs) in market research and user experience research. A key design decision concerns response modality: should respondents interact through voice, text, or both? Previous research suggests that speaking encourages longer and more detailed responses than typing, leading to richer qualitative insights (Melumad 2023; Melumad and Meyer 2020). Voice interactions are generally perceived as more natural and conversational, whereas text encourages greater reflection and control over responses (Joinson 2001). Consequently, response modality may influence not only the quantity of information collected but also the cognitive processes underlying respondents’ answers.

Despite growing interest in AI-mediated qualitative research, few studies have systematically compared how response modality affects both data quality and participant experience in structured AI-moderated interviews.

This study addresses two research questions:

• RQ1: How does response modality (voice, text, hybrid) affect data quality in AI-moderated interviews?

• RQ2: How does response modality influence participant experience and willingness to disclose personal information?

Drawing on previous literature and observations from market research projects on the Glaut platform (spanning pharmaceuticals to consumer goods, B2B and B2C) we test two hypotheses:

• H1: Voice responses will produce higher verbosity, lexical diversity, descriptiveness, and thematic richness than text responses.

• H2: Participant experience will not substantially differ across response modalities, since we expect it to be shaped primarily by the AI moderator’s dynamic, respondent-specific follow-up questions (interview personalization) rather than modality itself; personalization was held constant across conditions and not directly tested.

2. Methodology

We employed a between-subjects one-way factorial design with three experimental conditions:

• Voice: respondents answered exclusively through spoken input.

• Text: respondents answered exclusively through typed input.

• Hybrid: respondents could freely switch between voice and text throughout the interview.

Participants were randomly assigned to one of the three conditions and completed the study on desktop, tablet, or smartphone.

2.1. Sample

A total of 252 UK participants aged 18–65 were recruited through Prolific, an online platform that connects researchers with a large, vetted, and pre-screened pool of participants for paid research studies, to complete AI-moderated interviews about living alone in the UK. This topic was selected because it encourages rich qualitative responses, evokes diverse emotional reactions, and generates commercially relevant insights.

Participants were recruited via three independently published Prolific studies, one per condition (Voice: n = 78; Text: n = 89; Hybrid: n = 85), using Prolific’s cross-study exclusion criteria so that no participant could complete more than one condition and would be randomly assigned to one version. Participants received approximately $1.50 for completing the interview, consistent with standard Prolific compensation practices.

Device type also varied somewhat by condition (Appendix 3) and was not modeled as a covariate; we address this as a limitation in §4.1.

Completion rates also varied by condition (Voice 81%, Text 94%, Hybrid 92%), suggesting the richer data obtained via voice may come at some cost to respondent completion, a trade-off revisited in the Discussion.

Chi-square tests confirmed no significant demographic differences between conditions on gender, age, ethnicity, employment, income, or location (all p > .25; Table 1, Appendix 3).

The interview consisted of eight focal open-ended questions used for the data quality analyses, additional closed-ended questions reflecting a typical market research study, and four post-interview evaluation questions measuring participants’ experience. The full questionnaire is provided in Appendix 2.

2.2. Interview Procedure

All interviews were conducted using the Glaut AIMI platform (Glaut Inc.), an AI-native research platform that conducts AIMIs via voice and text and performs automated qualitative coding of open-ended responses. The AI moderator used identical wording, prompts, and a neutral conversational style across all experimental conditions to ensure methodological consistency. Participants first received standardized instructions explaining the study purpose, confidentiality, and that there were no right or wrong answers. Depending on condition, respondents were informed whether they would answer using voice, text, or could choose between both. Figure 1 includes images of the platform and message conveyed.

Figure 1.Glaut’s platform interface (welcome message, interview instructions and first question)

In the Text condition, the platform enforced a platform-level block on audio input, preventing respondents from using their device’s native voice-to-text or dictation functionality to generate typed responses.

The platform automatically transcribed spoken responses using the same speech-to-text engine for all participants, ensuring that all linguistic analyses were conducted on a standardized textual corpus. Participants completed the interview in their natural environment using their preferred device.

3. Results

3.1. Data quality comparison between voice, text and hybrid

To compare the three response modalities, we analyzed answers to the eight focal open-ended questions (reported in Appendix 2) across five complementary dimensions of response quality: verbosity, lexical diversity, descriptiveness, thematic variety, and psycholinguistic properties (concreteness-abstractness). Prior to group comparisons, data were assessed using Shapiro–Wilk tests for normality and Brown–Forsythe tests for homogeneity of variance. Because variances and group sizes differed, all comparisons were conducted using Welch’s ANOVA followed by Games–Howell post-hoc tests where appropriate.

3.1.1. Verbosity

Verbosity was measured as the total number of words produced across the eight focal questions. Welch’s ANOVA indicated a significant effect of response modality on overall word production, F(2, 9.69)=35.10, p<.001. Participants in the Voice condition (M=159.26, SD=96.31) produced significantly more words than those in the Text condition (M=46.20, SD=25.49; p<.001) and the Hybrid condition (M=93.15, SD=88.22; p<.001). Hybrid participants also produced significantly more words than Text participants (p<.001). Overall, voice responses contained approximately 236% more words than text responses (Chart 1).

3.1.2. Lexical diversity

Lexical diversity was assessed by counting the number of unique words after removing repetitions. Welch’s ANOVA showed significant group differences, F(2, 9.83)=38.94, p<.001. Voice responses (M=84.64, SD=36.95) contained significantly more unique words than Text (M=34.86, SD=16.17; p<.001) and Hybrid responses (M=56.07, SD=37.61; p<.001). Hybrid responses also exceeded Text (p<.001). Voice participants produced approximately 138% more unique words than Text participants (Chart 2).

3.1.3. Descriptiveness

Descriptiveness was measured as the number of content words (nouns, verbs, adjectives, and adverbs). Welch’s ANOVA revealed significant group differences, F(2, 9.71)=34.25, p<.001. Voice responses (M=70.67, SD=41.62) contained significantly more content words than Text (M=22.06, SD=11.84; p<.001) and Hybrid responses (M=42.60, SD=38.80; p<.001). Hybrid responses also exceeded Text (p<.001). Overall, Voice participants produced approximately 213% more descriptive content words than Text participants (Chart 3).

Chart 1.Verbosity

Chart 2.Lexical diversity

Chart 3.Descriptiveness

3.1.4. Thematic variety

Using Glaut’s thematic encoder (details provided in Appendix 1; see also AI Disclosure Appendix) with identical prompts across conditions, we calculated the average number of distinct themes identified in each response. Welch’s ANOVA indicated significant group differences, F(2, 10.39)=10.69, p=.002. Voice responses (M=3.44, SD=1.04) contained significantly more themes than both Text (M=2.65, SD=0.58; p<.001) and Hybrid responses (M=2.86, SD=0.81; p<.001), whereas the difference between Hybrid and Text was not significant (p=.18).

Across the eight focal questions, Voice responses generated an average of 27 themes, compared with 23 for Hybrid and 21 for Text, indicating broader thematic coverage.

3.1.5. Concreteness-Abstractness

To further characterize response quality, we examined the concreteness of respondents’ language using the psycholinguistic norms of Paetzold and Specia (2016). Each response received a concreteness score, from which an abstractness score was derived (see Appendix 3 for computational details). Because the two scores are perfectly inversely related, they represent a single concreteness–abstractness dimension and were analyzed together.

Concreteness reflects the extent to which language refers to tangible, specific concepts (e.g., table, car) rather than abstract ideas (e.g., freedom, strategy). More concrete language is generally easier to process and tends to convey specific, actionable insights, whereas more abstract language may signal general attitudes, evaluations, or high-level reflections (Kacewicz et al. 2014; Paetzold and Specia 2016; Pennebaker et al. 2007). This dimension therefore offers an additional perspective on response richness, capturing whether respondents describe their experiences in specific, grounded terms or through more general and reflective language.

Shapiro–Wilk tests indicated non-normal residuals (W = 0.963, p < .001). Given the relatively large group sizes (n = 78–89) and unequal variances across groups, we used Welch’s ANOVA, which does not assume homogeneity of variance, followed by Games–Howell post-hoc tests. Groups differed significantly on the concreteness–abstractness dimension, F(2, 159.6) = 40.07, p < .001. Text responses were the most concrete (M = 333.46, SD = 10.52), followed by Hybrid (M = 328.38, SD = 8.88) and Voice (M = 322.53, SD = 5.64). Abstractness scores showed the inverse pattern, with Voice responses scoring highest (M = 488.74, SD = 2.82), followed by Hybrid (M = 485.81, SD = 4.44) and Text (M = 483.27, SD = 5.26). Games–Howell tests confirmed significant differences between all pairs of groups.

These findings suggest that response modality shapes not only how much respondents say but also how they say it. Text responses, despite being shorter, contained more concrete and specific language, whereas voice responses leaned toward more abstract, reflective expression, and Hybrid responses consistently fell between the two. This cautions against equating higher verbosity with higher-quality data: longer voice responses may offer broader reflections, while shorter text responses may deliver more specific, actionable content. Table 2 illustrates this contrast with matched verbatim responses from two participants at opposite ends of the concreteness–abstractness spectrum.

3.2. Participant experience

Participants evaluated their interview experience on three dimensions: perceived ease of expressing their thoughts, perceived empathy of the AI moderator, and willingness to disclose personal information, each measured on a five-point Likert scale. As expected for ordinal data, Shapiro–Wilk tests indicated deviations from normality across all three measures (all p < .05). Consequently, group differences were analyzed using Welch’s ANOVA. Overall, respondents reported highly positive experiences regardless of response modality, supporting H2.

3.2.1. Perceived ease

Participants reported high levels of ease across all three conditions (Voice: M = 4.5; Text: M = 4.6; Hybrid: M = 4.6). Welch’s ANOVA revealed no significant group differences, F(2, 161.24)=0.59, p = .556, indicating that respondents found it equally easy to express themselves regardless of response modality.

3.2.2. Perceived empathy

Participants also perceived the AI moderator as highly empathic across all conditions (Voice: M = 4.2; Text: M = 4.0; Hybrid: M = 3.9). Welch’s ANOVA showed no significant differences between groups, F(2, 164.7) = 1.76, p = .175.

3.2.3. Willingness to disclose

Participants reported similarly high willingness to share personal thoughts with the AI moderator (Voice: M = 4.3; Text: M = 4.2; Hybrid: M = 4.2). Welch’s ANOVA indicated no significant group differences, F(2, 163.54) = 0.46, p = .632.

To better understand these ratings, respondents were asked to explain their disclosure score in an open-ended question.

Overall, 83% of respondents (209 of 251) reported high willingness to disclose (ratings 4–5), while 17% (42 of 251) reported lower willingness (ratings 1–3).

Qualitative verbatim analysis shows that, among participants reporting high willingness to disclose (ratings 4–5; N = 209), the most frequently cited reasons were the conversational and effortless nature of the interview (52.6%), particularly among Voice participants (56% vs. 43% in Text). Voice respondents also more often described the interaction as anonymous and free from judgment (61% vs. 47%), while Hybrid participants most frequently emphasized the smoothness of the overall experience (59%).

Among respondents reporting lower willingness to disclose (ratings 1–3; N = 42), the main barriers were discomfort interacting with AI (60%), lack of real-time feedback (38%), and concerns about data security (33%). Data security concerns were particularly common among Voice participants (50%), whereas Text participants more frequently mentioned the rigidity of typed interactions and the limited personalization of the conversation (20%).

3.2.4. Modality perceptions

When asked which modality they would prefer for a future interview, text was slightly favored overall (55% vs. 45% NS). This was strongly shaped by condition, suggesting a priming effect: 77% of Voice participants preferred voice, 76% of Text preferred text, and Hybrid split 62%/38% toward text.

Among Hybrid participants, modality switching was rare (80% did not switch; see Appendix 3 for details).

Qualitative analysis of follow-up responses clarified these preferences: Voice was chosen for quickness/ease (38%), natural/conversational feel (29%), and situational convenience (11%); Text was chosen for clarity/control (26%), familiarity/comfort (19%), privacy/anonymity (12%), and reduced anxiety (10%). Chart 4 summarizes preferences for each modality.

4. Discussion & Business Relevance

This study provides one of the first quantitative comparisons of response modalities in AI-moderated interviews. Consistent with prior research on conversational communication (Melumad 2023; Melumad and Meyer 2020), voice elicited richer qualitative data than text; longer, more lexically diverse, more descriptive, and thematically broader. Participant experience remained consistently positive across all three conditions, with no significant differences in ease, empathy, or willingness to disclose, supporting H2. This supports findings in the human–computer interaction (HCI) literature that perceived empathy and comfort are often shaped more by conversational design and personalization than by input modality alone (Cranshaw and Caine 2022; Zhou et al. 2020).

Beyond confirming that voice improves data richness, our findings highlight a trade-off between data quality and respondent preference: although voice generated the most detailed responses, a slight majority (55%) preferred text for future interviews. Qualitative feedback suggests these preferences reflect different respondent needs rather than differences in interview quality; voice was valued for speed and naturalness, text for control and privacy (§3.2.4). Hybrid interviews partially reconciled these advantages, producing richer responses than text while letting respondents choose their preferred modality.

From a managerial perspective, response modality should be aligned with research objectives rather than treated as a purely technological feature. Voice is particularly valuable during exploratory research, where maximizing insight richness is critical; text may remain preferable when respondent comfort, privacy, or scalability are primary considerations. Offering both provides greater flexibility while accommodating diverse respondent preferences.

4.1. Limitations and Future Research

Several limitations should be considered. First, the study was conducted with a UK sample and focused on a single research topic (living alone), which may limit the generalizability of the findings across populations and research contexts. Second, the between-subjects design does not allow direct observation of how individual respondents adapt their behavior across modalities. Finally, although participants completed interviews in naturalistic settings, environmental factors such as background noise, device quality, and privacy were not experimentally controlled and may have influenced modality choice and response behavior. In addition, the device and completion-rate differences noted in §2.1 were not modeled as covariates and warrant more controlled follow-up.

Future research should examine whether these findings generalize across cultures, languages, and more sensitive research topics. Longitudinal studies could also investigate whether repeated exposure to AI-moderated interviews changes modality preferences over time or increases respondents’ willingness to engage with voice-based interactions. Finally, future work should explore how contextual factors (including device type, environmental constraints, and respondent characteristics) influence modality choice and response quality.

4.2. Conclusion

Response modality is not a neutral design choice in AI-moderated interviews. While voice consistently produces richer and more diverse qualitative data, participants experience similarly high levels of ease, empathy, and willingness to disclose regardless of modality. The observed preference for text, despite the superior richness of voice responses, highlights a practical trade-off between maximizing research quality and accommodating respondent preferences. Rather than identifying a universally optimal modality, our findings suggest that researchers should strategically select, or offer, a response modality that best aligns with the objectives of the study and the needs of participants. Applied to our own topic of living alone, which often elicits emotionally nuanced narratives, a voice-forward or hybrid design would likely surface the richest insights, while retaining a text option for respondents prioritizing privacy or comfort.

Conflict of Interest and Funding

The author is Head of Research at Glaut Inc., the company that developed and provided the AI-moderated interview platform used to conduct this study.

Corresponding author contact information

E-mail address: veronica@glaut.com