Abstract
Objective:
Interaction with voice recognition systems, such as Siri™ and Alexa™, is an increasingly important part of everyday life. Patients with voice disorders may have difficulty with this technology, leading to frustration and reduction in quality of life. This study evaluates the ability of common voice recognition systems to transcribe dysphonic voices.
Methods:
Participants with (n=30) and without (n=23) voice disorders were recorded reading the “Rainbow Passage”. Recordings were played at standardized intensity and distance to dictation programs on Apple iPhone 6S™, Apple iPhone 11 Pro™, and Google Voice™. Word recognition scores were calculated as the proportion of correctly transcribed words. Word recognition scores were compared to auditory-perceptual and acoustic measures.
Results:
Mean word recognition scores for participants with and without voice disorders were 68.6% and 91.9% for Apple iPhone 6S™ (p<0.001), 71.2% and 93.7% for Apple iPhone 11 Pro™ (p<0.001) and 68.7% and 93.8% for Google Voice™ (p<0.001). There were strong, approximately linear associations between CAPE-V ratings of overall severity of dysphonia and word recognition score, with correlation coefficients (R2) of 0.609 (iPhone 6S™), 0.670 (iPhone 11 Pro™) and 0.619 (Google Voice™). These relationships persisted when controlling for diagnosis, age, sex, fundamental frequency, and speech rate (p<0.001 for all systems).
Conclusion:
Common voice recognition systems function well with non-dysphonic voices, but are poor at accurately transcribing dysphonic voices. There was a strong negative correlation with word recognition scores and perceptual voice evaluation. As our society increasingly interfaces with automated voice recognition technology, the needs of patients with voice disorders should be considered.
Keywords: dysphonia, voice recognition, hoarseness, mobile phone, technology
Introduction
There is increasing prevalence of voice disorders, with a 2012 National Health Interview Survey revealing 1 in 13 adults reported voice problems.1,2 Voice disorders impair communication and contribute substantially to decreased quality of life and increased healthcare costs. Dysphonia has been shown to negatively impact many aspects of health, including functional, social, emotional and physical well-being.3–5 In addition to decreased quality of life, patients with voice disorders often report poor overall health. Benninger et al. compared dysphonia to other chronic diseases and found that dysphonic individuals reported lower social functioning relative to those with sciatica, back pain, and angina.6 Another study found patients rated moderate dysphonia equivalent to monocular blindness using health state utility values.7
The decreased quality of life experienced by dysphonic speakers may be attributed to difficulty being understood, in addition to difficulty with voice production. Speakers with voice disorders have been repeatedly shown to be more difficult to understand relative to speakers without voice disorders.8–14 Disordered voice production is associated with a number of deviations that may contribute to decreased intelligibility, including decreased vocal intensity,8 and noisy turbulent airflow.9 In the setting of incomplete glottic closure, speakers have more errors in stop consonant production as well as increased turbulence during high-frequency sounds,11 which can impact the ability of listeners to differentiate between sounds.15
Voice recognition technology enables the translation of voice input to text or to programmed actions. This technology is intended to streamline communication by enhancing mobility, multitasking, and information gathering. Over the past decade, substantial advances in computational power via cloud computing and machine learning have enabled voice recognition technology to make unprecedented strides in accuracy and utility. Smartphones have become the primary everyday computational device for the vast majority of people. Furthermore, smart speakers and voice recognition assistants such as Apple’s Siri™, Google Assistant™, and Amazon’s Alexa™ have emerged to further integrate the experience of data acquisition and programmable tasks. According to Google, 29% of the global online community uses their voice to search on mobile devices, regardless of device type.16 Voice recognition technology has many applications beyond accessing information, including: driving with hands-free mobile access, routing customer service queries, and refilling pharmacy prescriptions. Voice recognition technology grants additional opportunities for individuals with physical impairments or poor literacy to have access to the internet and other assistive tasks.
Given that disordered voices result in impaired intelligibility among human listeners, and with the increased utilization of voice recognition technology, it is important to investigate the impact of dysphonia on voice recognition technology. We hypothesize that the accuracy of voice recognition technology will be reduced for individuals with voice disorders relative to individuals without voice disorders and that voice recognition technology accuracy will be decreased relative to the overall severity of dysphonia. Therefore, the objective of this study is to assess the accuracy of common voice recognition systems in transcribing dysphonic voices. Secondarily, we aim to evaluate which auditory-perceptual or acoustic measures impact the accuracy of voice recognition technology.
Materials and Methods
Study design, setting, and patient selection
After receiving approval from the Institutional Review Board at Boston Medical Center, the Otolaryngology-Head and Neck Surgery department voice recording catalogue was searched for English-speaking patients with dysphonic voices who had undergone comprehensive voice assessment and were recorded reading the Rainbow Passage. Additional recordings of both dysphonic and non-dysphonic voices were sourced from existing recordings of study participants at the Stepp Lab for Sensorimotor Rehabilitation Engineering at Boston University, all of whom completed informed consent in compliance with the Boston University Institutional Review Board. All available voice samples were considered for inclusion. Samples were excluded if they did not include a Rainbow Passage reading, if they did not complete the first 99 words of the Rainbow Passage, if the speaker had impaired articulation, or if the speaker spoke English with a non-native accent. One subject was excluded for mild ataxic dysarthria in the setting of Parkinson’s disease. The voice recordings utilized in the study represent a variety of dysphonia severity, diagnoses, and patient demographic characteristics. All voice disorder diagnoses were made by fellowship-trained laryngologists.
All microphone signals were digitally recorded and analysis occurred offline. Participants were recorded in one of two settings: the first was a quiet room at Boston Medical Center with an omnidirectional headset condenser microphone (model Beta 53; Shure, Niles, IL). These recordings took place between December 19, 2018, and September 4, 2019. The second setting was a quiet room at the Stepp Lab using a unidirectional headset condenser microphone (model SM35XLR; Shure, Niles, IL). These recordings were conducted between July 5, 2016, and August 20, 2019. All participants were recorded with the microphone at 7–10 cm from the lips and at a 45-degree angle from midline. Microphone signals were sampled at 44.1 kHz with 16-bit resolution, at both locations. In each setting, participants were instructed to read the “Rainbow Passage” in a comfortable speaking voice. As the recordings were accessed retrospectively, the speakers were not aware that their samples would be transcribed with voice recognition technology.
Demographic, treatment, and outcome variables
Demographic variables and potential contributors to intelligibility were documented for each case. These included age, sex, diagnosis, fundamental frequency (fo), standard deviation of fo, and cepstral peak prominence smoothed (CPPS). CPPS has been demonstrated as a marker of dysphonia, with lower CPPS values corresponding to more severe dysphonia.17 The fo, standard deviation of fo, and CPPS were calculated with Praat software.18 Speech rate, in words per minute, inclusive of pauses in speech, was also calculated.
Each recording of the “Rainbow Passage” was played at standardized intensity for each voice recognition software system. The recordings were played at intensity between 65 and 70 decibels through a Bose SoundLink Wireless speaker (Bose Corporation, Framingham, MA) to a device 12 inches from the source. The passages were played and processed one sentence or phrase at a time. After completion of the first paragraph (98 words) of the Rainbow Passage, the number of correct words was tallied and divided by the total to calculate the word recognition score. This process was repeated with Apple iPhone 6S speech dictation (Apple, Cupertino, CA), Apple iPhone 11 Pro speech dictation (Apple, Cupertino, CA), and Google Voice online dictation software (Google, Mountain View, CA) using an Apple MacBook Air laptop computer (Apple, Cupertino, CA). No software updates occurred during the study and all devices were cleared of data prior to study initiation. To control for the potential of machine learning, the accuracy of word recognition was compared throughout time for all three studied devices. There was no difference in scores throughout the duration of the study period for any device. A fourth system, the Microsoft Windows 10 talk-to-type dictation (Microsoft, Redmond WA), was also attempted. The system was unable to recognize and transcribe non-dysphonic “Rainbow Passage” recordings so further testing was not pursued.
Two trained listeners (authors MLR and DPB) conducted independent perceptual evaluation of voice for each voice sample using the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V).19 The CAPE-V provides a standardized and validated approach to evaluation of vocal quality, including six specific vocal attributes: overall severity, roughness, breathiness, strain, pitch, and loudness.19 For the purpose of this study, loudness was not included since volume was standardized. The listeners were blinded to the diagnosis and to the word recognition scores. Per described technique, a visual analog scale was used to rate the overall severity, roughness, strain, pitch, and loudness19. The scale was converted to a numeric value from 0–100 by measuring the relative position of the mark (i.e., a mark directly in the center of the line would be 50/100). The voice samples were reviewed as ordered by their study identifiers, which allowed a consistent mix of dysphonic and non-dysphonic speakers.
Statistical analysis
Statistical analyses were conducted using SPSS Version 23 (IBM, Armonk, NY). Descriptive statistics are shown in Table 1 (categorical variables) and Tables 2–4 (continuous variables). For the CAPE-V evaluations, inter-rater reliability (Cohen’s kappa for two raters) was calculated for ratings of the overall severity of dysphonia and for the ratings of roughness, breathiness, strain, and pitch. Given the high reliability with correlation coefficient 0.913 for CAPE-V (overall), the ratings were combined. The final combined rating is the average of the individual ratings. Since recordings were made in two settings under differing conditions, a pairwise analysis was conducted to assess for difference in voice recognition score outcomes. Ten cases from each site were matched based on overall CAPE-V ratings. There was high correlation (correlation coefficient 0.879, 95% CI 0.686 to 0.996, p=0.001) and no significant difference between means (p=0.227) in paired comparisons of iPhone 6s word recognition score. This was true for the Google system (correlation coefficient 0.965, 95% CI 0.883 to 0.996, p<0.001; no difference between means, p=0.294). This was also true for the iPhone 11 Pro (correlation coefficient 0.934, 95% CI 0.361 to 0.990, p<0.001; no difference between means, p=0.145). Given this, the recordings were pooled for all further analyses.
Table 1:
Word recognition scores (WRS) using iPhone and Google Voice dictation systems to transcribe the “Rainbow Passage”, according to categorical patient factors. Q1–4=quartiles,
| Patient factor | N (%) | Apple iPhone 6s, WRS (%) | Google Voice, WRS (%) | Apple iPhone 11 Pro, WRS (%) | |||
|---|---|---|---|---|---|---|---|
|
| |||||||
| Mean (SD) 80.3 (19.6) | p value | Mean (SD) 81.3 (19.8) | p value | Mean (SD) 82.2% (19.1) | p value | ||
| Gender | |||||||
| Male | 13 (25.0) | 79.7 (22.8) | 0.904 | 83.8 (17.6) | 0.578 | 81.5 (22.0) | 0.892 |
| Female | 39 (75.0) | 80.5 (18.7) | 80.5 (20.6) | 82.4 (18.4) | |||
| Age | |||||||
| Q1 (18–32) | 13 (25.0) | 84.8 (4.4) | 0.689 | 86.7 (5.9) | 0.570 | 86.5 (5.8) | 0.658 |
| Q2 (33–50) | 13 (25.0) | 81.1 (21.0) | 80.5 (25.0) | 82.8 (21.4) | |||
| Q3 (54–67) | 12 (23.1) | 80.0 (18.7) | 82.8 (17.5) | 82.8 (17.9) | |||
| Q4 (68+) | 14 (26.9) | 75.6 (27.1) | 75.9 (24.6) | 77.1 (25.7) | |||
Table 2:
Word recognition scores (WRS) using iPhone 6s dictation system to transcribe the “Rainbow Passage”, with associations to continuous patient factors calculated via simple linear regression analysis. CAPE-V=Concensus Audiotory-Perceptual Evaluation of Voice, fo =fundamental frequency, CPPS=cepstral peak prominence, smoothed.
| Patient factor | Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | R2 | p value |
|---|---|---|---|---|---|
|
| |||||
| Age | −0.203 | −0.476 | 0.070 | 0.043 | 0.142 |
| CAPE-V (Overall) | −0.648 | −0.794 | −0.501 | 0.612 | <0.001 |
| CAPE-V (Roughness) | −0.562 | −0.810 | −0.314 | 0.293 | <0.001 |
| CAPE-V (Breathiness) | −0.752 | −0.960 | −0.545 | 0.515 | <0.001 |
| CAPE-V (Strain) | −0.471 | −0.709 | −0.232 | 0.239 | <0.001 |
| CAPE-V (Pitch) | −0.433 | −0.813 | −0.054 | 0.095 | 0.026 |
| f o | −0.148 | −0.289 | 0.007 | 0.081 | 0.054 |
| Standard deviation of fo | −0.344 | −0.93 | −0.095 | 0.133 | 0.008 |
| CPPS | 6.46 | 4.59 | 8.33 | 0.491 | <0.001 |
| Speech rate | 0.252 | 0.090 | 0.415 | 0.162 | 0.003 |
Table 4:
Word recognition scores (WRS) using iPhone 11 Pro dictation system to transcribe the “Rainbow Passage”, with associations to continuous patient factors calculated via simple linear regression analysis. CAPE-V=Concensus Audiotory-Perceptual Evaluation of Voice, fo =fundamental frequency, CPPS=cepstral peak prominence, smoothed.
| Patient factor | Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | R2 | p value |
|---|---|---|---|---|---|
|
| |||||
| Age | −0.183 | −0.451 | 0.085 | 0.036 | 0.176 |
| CAPE-V (Overall) | −0.664 | −0.795 | −0.553 | 0.674 | <0.001 |
| CAPE-V (Roughness) | −0.559 | −0.800 | −0.319 | 0.304 | <0.001 |
| CAPE-V (Breathiness) | −0.723 | −0.919 | −0.526 | 0.498 | <0.001 |
| CAPE-V (Strain) | −0.506 | −0.731 | −0.281 | 0.290 | <0.001 |
| CAPE-V (Pitch) | −0.437 | −0.807 | −0.068 | 0.101 | 0.021 |
| f o | −0.143 | −0.281 | −0.006 | 0.080 | 0.042 |
| Standard deviation of fo | −0.300 | −0.547 | −0.053 | 0.106 | 0.018 |
| CPPS | 6.22 | 4.36 | 8.07 | 0.476 | <0.001 |
| Speech rate | 0.272 | 0.116 | 0.428 | 0.198 | 0.001 |
Comparative analysis was performed between binary patient groupings (i.e., voice disorder or no voice disorder) and also by considering degree of perceptual dysphonia (as represented by CAPE-V ratings of overall severity) as a continuous variable. The analysis treating dysphonia as a continuous variable is emphasized as the authors believe it provides a more comprehensive assessment across varying degrees of dysphonia. The comparative analysis was performed to assess the association between potential covariates and the outcome of interest, word recognition score. This was repeated for each voice recognition system. One-way ANOVA tests were used to compare word recognition score means for categorical variables, and linear regression was used to assess correlation between continuous variables and word recognition score. From these analyses, variables with p-values of 0.05 or lower were considered potential contributors and selected to be included in multivariate analysis.
Multivariate stepwise linear regression was used to assess the relationships between the selected variables and the word recognition score. Covariates selected from the univariate analyses were considered in the stepwise regression model. This was repeated for each voice recognition system. The final model is described in Table 5. Findings were considered statistically significant at p < 0.05.
Table 5:
Stepwise regression analysis to predict the word recognition scores (WRS) using the Apple iPhone and Google Voice systems to transcribe the “Rainbow Passage”, according to patient factors identified in univariate analysis. CAPE-V=Concensus Audiotory-Perceptual Evaluation of Voice
| Patient factor | Apple iPhone 6s, WRS (%) | ||||
|---|---|---|---|---|---|
| Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | p value | R2 | |
|
| |||||
| CAPE-V (overall) | −0.648 | −0.794 | −0.501 | <0.001 | 0.612 |
| Constant | 98.7 | 93.3 | 104.1 | - | |
|
|
|||||
| Google Voice, WRS (%) | |||||
| Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | p value | R2 | |
|
| |||||
| CAPE-V (overall) | −0.665 | −0.810 | −0.520 | <0.001 | 0.630 |
| Constant | 100.2 | 94.9 | 105.5 | - | |
|
|
|||||
| Apple iPhone 11 Pro, WRS (%) | |||||
| Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | p value | R2 | |
|
| |||||
| CAPE-V (overall) | −0.664 | −0.795 | −0.533 | <0.001 | 0.674 |
| Constant | 101.0 | 96.2 | 105.8 | - | |
Results
There were 52 voice recordings of “Rainbow Passage” analyzed. Of these participants, 29 (55.8%) had a diagnosis of voice disorder and 23 (44.2%) were without voice disorder. All 15 patients recorded in the Boston Medical Center setting had a diagnosis of voice disorder. Of the patients recorded at Boston University, 14 (37.8%) had a voice disorder. For the 23 participants without voice disorder, the mean age was 52.2 years (standard deviation 20.4 years, range 19 to 83 years). For the 29 participants with a voice disorder, the mean age was 48.9 years (standard deviation 19.8 years, range 19 to 82 years). There was no significant difference in mean age (p=0.561). The cohort without voice disorders was 21.7% male and 78.3% female; the cohort with voice disorders was 20.7% male and 79.3% female. There was no significant difference in sex distribution by Fisher’s exact test (p=0.595). The most common voice disorder diagnoses were muscle tension dysphonia (N=11, 21.2%), benign vocal fold lesion (N=5, 9.6%), and chronic laryngitis (N=4, 7.7%). Other diagnoses included paresis or paralysis (N=3, 5.8%), spasmodic dysphonia (N=3, 5.8%), vocal fold atrophy (N=2, 3.8%), and laryngeal trauma (N=1, 1.9%). Mean word recognition scores for participants with and without voice disorders were 71.1% and 91.9% for Apple iPhone 6S™ (p<0.001), 71.5% and 93.7% for Apple iPhone 11 Pro™ (p<0.001) and 73.0% and 93.8% Google Voice™ (p<0.001).
For the iPhone 6s, there was significant negative correlation between all attributes of CAPE-V and word recognition scores, with the strongest association for the CAPE-V rating of overall severity of dysphonia (R2=0.612, p<0.001) (Table 2, Figure 1). There was also a significant negative correlation for standard deviation of fo (R2=0.133, p=0.008) and significant positive correlation for CPPS (R2=0.491, p<0.001) and speech rate (R2=0.162, p=0.003). In stepwise regression analysis, only the CAPE-V overall severity retained significant correlation (p<0.001). All other potential covariates were excluded from the stepwise model as they did not generate a significant change in correlation.
Figure 1:
Correlation of word recognition score and CAPE-V ratings of overall severity of dysphonia for Apple iPhone 6s voice recognition software.
For the Google Voice system, there was significant negative correlation between all CAPE-V scores except pitch and word recognition scores, with strongest association for the overall CAPE-V (R2=0.630, p<0.001) (Table 2, Figure 2). There was also significant negative correlation for fo (R2=0.100, p=0.023), standard deviation of fo (R2=0.156, p=0.004) and significant positive correlation for CPPS (R2=0.446, p<0.001) and speech rate (R2=0.128, p=0.009). In stepwise regression analysis, CAPE-V overall severity (p<0.001) was the only factor that retained significant correlation. All other potential covariates were excluded from the stepwise model as they did not generate a significant change in correlation.
Figure 2:
Correlation of word recognition score and CAPE-V ratings of overall severity of dysphonia rating for Google Voice recognition software.
For the iPhone 11 Pro, there was significant negative correlation between CAPE-V scores and word recognition scores, with strongest association for the CAPE-V overall severity of dysphonia (R2=0.674, p<0.001) (Figure 3). There was also significant negative correlation for fo (R2=0.080, p=0.042), standard deviation of fo (R2=0.106, p=0.018) and significant positive correlation for CPPS (R2=0.476, p<0.001) and speech rate (R2=0.198, p=0.001). In stepwise regression analysis, the only factor that retained significant correlation was CAPE-V overall rating of severity (p=0.001). All other potential covariates were excluded from the stepwise model as they did not generate a significant change in correlation.
Figure 3:
Correlation of word recognition score and CAPE-V ratings of overall severity of dysphonia for Apple iPhone 11 Pro voice recognition software.
Discussion
The purpose of this study was to examine the impact of dysphonia on voice recognition technology accuracy as measured by word recognition scores. The results indicated that word recognition scores were significantly lower for speakers with voice disorders in comparison to those without voice disorders, and this finding was consistent across all three studied systems.
A secondary interest was investigating which measures of dysphonia had the highest correlation with voice recognition technology intelligibility. A number of factors were considered as potential contributors to word recognition scores. Participants’ fo demonstrated a weak but significant association with word recognition score in univariate analysis, with lower fo associated with higher word recognition scores. This suggests that the voice recognition technology may be better equipped to understand voices with lower fo. This finding is similar to previous work demonstrating improved listener accuracy in speech recognition for lower fo.20 This may be due to the more narrow spacing of harmonics in lower frequency domain which allows for better resolution. This then allows for improved resolution in resolving vocal tract resonances at lower frequency. In contrast, higher fo values have decreased resolution and formants become poorer estimates of underlying resonances. The lack of being able to track formants in higher fo which relate to vocal tract resonances could be related to decreased word recognition.
A higher standard deviation of fo was also associated with lower word recognition scores. A higher fo in this studied cohort correlated with higher standard deviation of fo, which has been demonstrated previously. 21 Because of the close relationship between fo and standard deviation of fo, it is difficult to conclude which variable had the more significant effect on the outcome. The authors speculate that perhaps a voice with more frequency variation could be more difficult for the technology to follow and recognize. The correlation coefficients (R2) were quite low for these variables; for fo, R2 was 0.100 (Google Voice) and 0.080 (iPhone 11), and iPhone 6s demonstrated nonsignificant correlation. For standard deviation of fo, R2 was 0.133 (iPhone 6s), 0.156 (Google Voice), and 0.106 (iPhone 11). So although there were significant correlations, fo and standard deviation of fo have limited impact on the variability in the outcome.
Speech rate was considered as a potential confounder and it was hypothesized that a slower speech rate would result in improved voice recognition technology accuracy. However, faster speech rate demonstrated a positive association with word recognition scores. The authors’ hypothesize that this discrepancy occurred because participants with more severe dysphonia spoke at a slower rate. The mean speech rate between individuals with a voice disorder was 152.6 words per minute (95% CI 141.1 to 163.9) in comparison to 175.6 words per minute for individuals without a voice disorder (95% CI 164.7 to 186.0; p=0.006). Thus, speech rate in this study is likely another proxy for dysphonia.
Cepstral peak prominence smoothed (CPPS) showed a significant correlation with the voice recognition technology performance in univariate analysis. Since CPPS is a correlate of the overall severity of dysphonia22, the authors hypothesized that lower CPPS values (which designate more severe dysphonia) would correlate with decreased word recognition. In this study, CPPS correlated significantly with voice recognition performance with coefficients of 0.491 (iPhone 6s), 0.446 (Google Voice), and 0.476 (iPhone 11), meaning that changes in CPPS can explain a moderate amount of the change in outcome. In the stepwise regression analysis, they did not generate significant improvement of the model and therefore were not included in the final model. This is likely a representation of the covariance between CPPS and CAPE-V. In linear regression, these two variables have a moderate, negative correlation (R2=0.605, p<0.001). Therefore, although CPPS did not change the regression model, it is a useful marker of dysphonia that did correlate with the performance of the technologies.
In multivariate analysis, the CAPE-V rating of the overall severity of dysphonia was the most consistent predictor of word recognition scores across all three systems. Therefore, increases in degree of perceived dysphonia – even when controlling for other factors – was associated with a significant decrease in accuracy of the voice recognition technology. Voice recognition technology accuracy was not impacted by any clinically perceived or patient-reported articulation disorder in the studied cohort, therefore it is possible that compensatory laryngeal actions and posturing in the setting of dysphonia decreased the clarity of the speech output. As dysphonic voices have been shown to have decreased intensity, this potential confounder was controlled by equalizing vocal intensity for each recording.17 Therefore, individuals with voice disorders associated with lower vocal intensity may have worse intelligibility then demonstrated in this study. The recordings were obtained in quiet rooms without background voice, given that background noise has been shown to limit listener intelligibility of dysphonic voices.23 However, voice recognition technology is often used in situations with surrounding noise and functionality may be additionally limited in this setting.
The accuracy of voice recognition systems may be improving with advances in technology. Improved software, independent of learning may contribute to improved accuracy as demonstrated by the improved word recognition scores for iPhone 11 Pro in comparison to iPhone 6s. However the absolute difference for dysphonic voices was small, with 69.1% for Apple iPhone 6S™ versus 71.1% for Apple iPhone 11 Pro™, and there remains substantial opportunity for improvement. Furthermore, similar to listeners using cognitive-perceptual techniques to improve comprehension,24 voice recognition technology may have the capacity to “learn” accuracy through increased experience with voice. It is possible that voice recognition technology can improve word recognition score with increased familiarity with a speaker’s vocal profile; additional investigation is warranted to evaluate this hypothesis. The adaptability of voice recognition technology to “learn” how to transcribe dysphonic voices through repetition was not investigated, and this would benefit from additional study.
To our knowledge, this is the first study to evaluate the ability of common voice recognition technology systems to transcribe English-speaking voices with variable degrees of dysphonia. A study of automatic speech recognition with Arabic digits on dysphonic patients similarly showed decreased speech recognition ranging from 56–84%, compared to 100% with normal voices.25 In this study, there was variable recognition based on pathology, with sulcus vocalis having least accurate in voice recognition and highest accuracy with vocal cord nodules.23 Other literature concerning voice recognition and dysphonia has focused on the identification of vocal pathology using automatic speech recognition technology or to evaluate intelligibility of speech in patients with head and neck cancer to guide rehabilitation after treatment.22,26
Of note, there is limited publically available information about the design of these proprietary technologies. This study evaluates ability to transcribe recordings of dysphonic and non-dysphonic voices and is not a comparison of overall quality of these technologies. This area represents an opportunity for additional investigation.
Limitations to this study include the heterogenous diagnoses contributing to the voice disorder cohort. Further investigation is warranted for subgroup analysis of how these diagnoses separately impact efficacy of voice recognition technology. The inclusion of a broader range of voice recognition technology systems would also improve generalizability. Also, the recordings were conducted at two different clinical sites, with different clinicians or researchers and slightly different protocols and equipment. A paired comparison is included that demonstrates no significant difference between sites as it relates to word recognition, but this should be considered a methodological limitation. Furthermore, the methodology would be strengthened by use of live speakers in lieu of or in addition to recorded voices however this was not possible due to restrictions associated with the COVID-19 pandemic. The authors are planning to perform follow up investigations with live speakers when restrictions are lifted in order better understand and optimize the interactions of dysphonic voices with these technologies.
As voice recognition technology becomes increasingly embedded in daily life, dysphonic patients remain potentially excluded from these innovations and interactions. Early models of voice recognition were based on recordings with ideal acoustic qualities and limited diversity. Future advancements in voice recognition technology should be directed towards adding more diverse voice models to enable recognition of nonnormative voices and speech patterns.
Conclusions:
Commonly used voice recognition technology systems are deficient in their ability to transcribe moderately and severely dysphonic voices. The results from auditory-perceptual ratings show increasing severity of dysphonia correlates with worse function of voice recognition technology. Daily interaction with voice recognition technology is increasing, therefore additional investigation with more diverse voice models is warranted to improve applicability and accessibility of this technology.
Table 3:
Word recognition scores (WRS) using Google Voice dictation system to transcribe the “Rainbow Passage”, with associations to continuous patient factors calculated via simple linear regression analysis. CAPE-V=Concensus Audiotory-Perceptual Evaluation of Voice, fo =fundamental frequency, CPPS=cepstral peak prominence, smoothed.
| Patient factor | Coefficient (β) | 95% CI for β, Low | 95% CI for β, High | R2 | p value |
|---|---|---|---|---|---|
|
| |||||
| Age | −0.214 | −0.490 | 0.062 | 0.046 | 0.126 |
| CAPE-V (Overall) | −0.665 | −0.810 | −0.520 | 0.630 | <0.001 |
| CAPE-V (Roughness) | −0.549 | −0.804 | −0.295 | 0.273 | <0.001 |
| CAPE-V (Breathiness) | −0.643 | −0.883 | −0.404 | 0.368 | <0.001 |
| CAPE-V (Strain) | −0.521 | −0.754 | −0.287 | 0.286 | <0.001 |
| CAPE-V (Pitch) | −0.167 | −0.568 | 0.234 | 0.014 | 0.407 |
| f o | −0.165 | −0.307 | −0.024 | 0.100 | 0.023 |
| Standard deviation of fo | −0.337 | −0.626 | −0.128 | 0.156 | 0.004 |
| CPPS | 6.23 | 4.26 | 8.20 | 0.446 | <0.001 |
| Speech rate | 0.227 | 0.059 | 0.395 | 0.128 | 0.009 |
Acknowledgments
Funding: This work was supported in part by the National Institutes of Health through grants R01DC015570 (C.E.S.)
Presentation: Accepted for podium presentation at American Laryngological Association Annual Meeting, Atlanta, GA; April 22, 2020.
Footnotes
Conflict of Interest Disclosures: No conflicts of interest exist for the authors in this study.
Conflict of Interest: None
Level of Evidence: Level IV
References
- 1.Benninger MS, Holy CE, Bryson PC, Milstein CF. Prevalence and Occupation of Patients Presenting With Dysphonia in the United States. J Voice. 2017;31(5):594–600. [DOI] [PubMed] [Google Scholar]
- 2.Bhattacharyya N. The prevalence of voice problems among adults in the U.S. Laryngoscope. 2014;124:2359–2362. [DOI] [PubMed] [Google Scholar]
- 3.Spina AL, Crespo AN. Assessment of Grade of Dysphonia and Correlation With Quality of Life Protocol. Journal of Voice. 2017;31(2):243.e21–243.e26. [DOI] [PubMed] [Google Scholar]
- 4.Cohen SM, Dupont WD, Courey MS. Quality-of-Life Impact of Non-Neoplastic Voice Disorders: A Meta-Analysis. Ann Otol Rhinol Laryngol. 2006;115(2):128–134. [DOI] [PubMed] [Google Scholar]
- 5.Wilson JA, Deary IJ, Millar A, Mackenzie K. The quality of life impact of dysphonia. Clin Otolaryngol Allied Sci. 2002;27(3):179–182. [DOI] [PubMed] [Google Scholar]
- 6.Benninger MS, Ahuja AS, Gardner G, Grywalski C. Assessing outcomes for dysphonic patients. J Voice. 1998;12(4):540–550. [DOI] [PubMed] [Google Scholar]
- 7.Naunheim MR, Goldberg L, Dai JB, Rubinstein BJ, Courey MS. Measuring the impact of dysphonia on quality of life using health state preferences. The Laryngoscope. 2019. 00:1–6. [DOI] [PubMed] [Google Scholar]
- 8.Ishikawa K, Boyce S, Kelchner L, et al. The effect of background noise on intelligibility of dysphonic speech J Speech Lang Hear Res, 60 (2017), pp. 1919–1929 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hillenbrand J, Houde RA Acoustic correlates of breathy vocal quality dysphonic voices and continuous speech. J Speech Lang Hear Res, 39 (1996), pp. 311–321 [DOI] [PubMed] [Google Scholar]
- 10.Hillenbrand J, Cleveland RA, & Erickson RL (1994). Acoustic correlates of breathy vocal quality. Journal of Speech, Language, and Hearing Research, 37, 769–778 [DOI] [PubMed] [Google Scholar]
- 11.Isshiki N, Kitajima K, Kojima H, & Harita Y. (1978). Turbulent noise in dysphonia. Folia Phoniatrica et Logopaedica, 30, 214–224. [DOI] [PubMed] [Google Scholar]
- 12.Casper JK, & Leonard R. (2006). Understanding voice problems: A physiological perspective for diagnosis and treatment. Baltimore, MD: Lippincott, Williams & Wilkins [Google Scholar]
- 13.Evitts PM, Starmer H, Teets K, et al. The Impact of dysphonic voices on healthy listeners: listener reaction times, speech intelligibility, and listener comprehension. Am J Speech Lang Pathol. 2016;25:561–575. [DOI] [PubMed] [Google Scholar]
- 14.Bender B, Cannito MP, Murry T, et al. Speech intelligibility in severe adductor spasmodic dysphonia. J Speech Lang Hear Res. 2004;47:21–33. [DOI] [PubMed] [Google Scholar]
- 15.Hartl DM, Crevier-Buchman L, Vaissiere J, & Brasnu D. (2005). Phonetic effects of paralytic dysphonia. Annals of Otology, Rhinology, and Laryngology, 114, 792–798. [DOI] [PubMed] [Google Scholar]
- 16.Voice search mobile use statistics. Think with Google. https://www.thinkwithgoogle.com/data/voice-search-mobile-use-statistics/. Accessed March 1, 2020.
- 17.Watts CR, Awan SN, Maryn Y. A comparison of cepstral peak prominence measures from two acoustic analysis programs. J Voice. 2017. May; 31(3):387. [DOI] [PubMed] [Google Scholar]
- 18.Boersma P. (2001). “ Praat, a system for doing phonetics by computer,” Glot Int. 5(9–10), 341–345. [Google Scholar]
- 19.Kempster GB, Gerratt BR, Verdolini Abbott K, Barkmeier-Kraemer J, Hillman RE. Consensus auditory-perceptual evaluation of voice: development of a standardized clinical protocol. Am J Speech Lang Pathol. 2009. May;18(2):124–32. [DOI] [PubMed] [Google Scholar]
- 20.Goldwater S, Jurafsky D, Manning CD. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Communication. 2010; 52(3):181–200. [Google Scholar]
- 21.Brown CA, Helms Tillery K, Apoux F, Doyle NM, Bacon SP. Shifting Fundamental Frequency in Simulated Electric-Acoustic Listening: Effects of F0 Variation. Ear Hear. 2016. Jan-Feb;37(1):e18–25 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sauder C, Bretl M, Eadie T. Predicting Voice Disorder Status From Smoothed Measures of Cepstral Peak Prominence Using Praat and Analysis of Dysphonia in Speech and Voice (ADSV). J Voice Off J Voice Found. 2017;31(5):557–566. doi: 10.1016/j.jvoice.2017.01.006\ [DOI] [PubMed] [Google Scholar]
- 23.Ishikawa K, Boyce S, Kelchner L, et al. The Effect of Background Noise on Intelligibility of Dysphonic Speech. J Speech Lang Hear Res. 2017. Jul;60(7): 1919–1929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lindblom B. On the communication process: speaker-listener interaction and the development of speech. Augment Altern Commun. 1990;6:220–230. [Google Scholar]
- 25.Muhammad G, Mesallam TA, Malki KH, Farahat M, Alsulaiman M, Bukhari M. Formant analysis in dysphonic patients and automatic Arabic digit speech recognition. Biomed Eng OnLine. 2011;10(1):41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Maier A, Haderlein T, Stelzle F, et al. Automatic Speech Recognition Systems for the Evaluation of Voice and Speech Disorders in Head and Neck Cancer. EURASIP J Audio Speech Music Process. 2009;2010(1):926951. [Google Scholar]