MonitaChatterjee,Ph.D.BoysTownNationalResearchHospital

Conflicts of Interest: None � AllworkwasfundedbytheNationalInstitutesofHealth

The focus of this presentation: Voice Emotion

Ø Perception of emotional prosody by listeners with CIs

Ø Production of emotion by listeners with CIs

Ø Neural plasticity and adaptation in children – age, device experience, linguistic environment

Ø Cognitive and linguistic development

COMPLEXPITCHCUES

Frequency

Am

plit

ude Spectral detail

Time

Am

plit

ude Temporal detail

EMOTION:STIMULI

Happy Angry Neutral Sad Scared

6

DanielleZionUMD/WalterReed

�  12sentences,5emotioneach:happy,angry,sad,neutral,scared(child-directedspeech)

�  1femaleand1maletalker(selectedfrompilotwith4talkers)

2030405060708090

100

4-ch 8-ch 16-ch Full

Female Talker

ANHcNHACIcCI

Perc

ent C

orre

ct

Degree of Spectral Resolution

Ø 10NHadultsØ 9CIadultsØ 31NHchildren(6.38–18.76years,meanage10.76years).

Ø 36CIchildren(6.83–18.44years,meanage12.15yrs,meandur.dev.8.76yrs.)

Chatterjeeetal.,2015(HearingRes)7

AgeeffectsinNHchildrenandadultswith8-channelnoise-vocodedspeech.

DevelopmentaleffectinNHchildrenlisteningto8-channelNBVspeech

20

30

40

50

60

70

80

90

5 10 15 20 25 30 35

cNHNHA

Perc

ent C

orre

ct

Age (years)

Chatterjeeetal.,2015(HearingRes)

r=0.73p<0.0001

ACloserLookAtIndividualData

Chatterjeeetal.,2015(HearingRes)9

10

VocalEmotionProductionByChildrenwithCochlearImplants

www.postersession.com

-1

0

1

ACI ANH CCI CNHStatus

MeanDiff_F0RangeRatio

factor(Status)

ACI

ANH

CCICNH

Summary

Voice Emotion Communication By Listeners With Cochlear ImplantsMonita Chatterjee1, Julie A Christensen1, Aditya M Kulkarni1, Mickael LD Deroche2, Sara A Damm1, Adam K Bosen1, Mohsen Hozan1, Charles J Limb3

1Boys Town National Research Hospital, Omaha, NE, USA; 2McGill University, Montreal, QC, CA; 3University of California, San Francisco, CA, USA

References

Introduction Results

1

2

3

ACI ANH CCI CNHStatus

Ratio_F0

factor(Status)

ACI

ANH

CCICNH

Recently published data on voice emotion recognition by children and adults with CIs (cCI and aCI respectively), compared to NH counterparts. CI listeners’ performance was equivalent to that of NH adults’ (aNH) performance with 8-channel noise-vocoded speech. NH children’s (cNH) performance reached ceiling with full-spectrum speech, but their performance with 8-channel vocoded speech was significantly poorer than NH adults’, and showed a strong developmental effect. (From Chatterjee et al., 2015). [Participants: 36 cCI age 6-18 years, mean age 12.15 years; 31 cNH age 6-18 years, mean age 10.76 years; 10 aNH, 9 aCI.]

Voice emotion processing, considered to be important in children’s quality of life and emotional development, is an area of deficit for children and adults with CIs (Luo et al, 2007; Schorr et al, 2009; Nakata et al, 2012; Most and Aviner, 2009; Wang et al., 2013).

Here, we present recent findings on voice emotion recognition, complex pitch discrimination and acoustic analyses of voice emotion production by school-aged children and adults with CIs, as well as their NH counterparts.

Methods1. Voice Emotion Recognition (Chatterjee et al., 2015)¾ Stimuli were 12 sentences from the HINT corpus,

spoken by a male and a female talker with each of five emotions (angry, happy, neutral, sad, scared) in a child-directed manner.

¾ Task was to hear each utterance and indicate which of the five emotions the participant thought they heard. No feedback was provided. Passive exposure to each talker’s style was provided prior to testing.

¾ NH children and adults performed the task both with full spectrum and noise-vocoded versions. CI children and adults heard only the full spectrum versions.

2. Complex Pitch Discrimination (Deroche et al., 2014)¾ Stimuli were broadband harmonic complexes, F0 =

100 Hz and F0 = 200 Hz.¾ Task was 3-interval forced-choice, method of

constant stimuli, child-friendly interface.¾ Psychometric functions were obtained and thresholds

and slopes were derived.

3. Voice Emotion Production¾ Participants read 20 sentences (e.g., This is it) in a

happy way and a sad way. Recordings were analyzed in Praat and Matlab..

1. Voice Emotion Recognition 2. F0 Discrimination and Voice Emotion Recognition

3. Voice Emotion Production

Mean F0 (Hz)LME analysis showed significant effects of Emotion (F(1,2107)=798, p<0.0001) and Group (F(3, 47)=4.38, p=0.009) on mean F0, and a significant interaction (F(3, 2107)=86.24, p<0.0001).

Paired t-tests (Bonferronicorrection) showed that cCIproduced smaller H/S contrasts than all other groups.

LME analysis showed effects of Emotion (F(1,2107)=386.4, p<0.0001), and a significant interaction between Emotion and Group (F(3, 2107)=13.10, p<0.0001).

Paired t-tests (Bonferroni correction) of the happy/sad differences in F0 range (Difference in ratio of F0max/F0min for H and S) showed that cCI and aNH produced smaller H/S contrasts than aCI.

0

5

10

15

20

ACI ANH CCI CNHStatus

MeanIntDiff

factor(Status)

ACI

ANH

CCICNH

F0 Range (F0max/F0min) Intensity (dB)

Spectral Centroid (Hz) Spectral Centroid Variation (s.d. in Hz)

1.0

1.5

2.0

2.5

3.0

ACI ANH CCI CNHStatus

MeanSpCsdRatio factor(Status)

ACI

ANH

CCICNH

1.0

1.2

1.4

1.6

ACI ANH CCI CNHStatus

MeanSpCentRatio

factor(Status)

ACI

ANH

CCICNH

LME analysis showed significant effects of Emotion (F(1,2106)=559.4, p<0.0001), a significant effect of Group (F(3, 47)=4.34, p=0.009) on Spectral Centroid, and a significant interaction (F(3, 2106)=43.82, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the spectral centroid showed that cCI produced smaller H/S contrasts than all other groups.

LME analysis showed significant effects of Emotion (F(1,2106)=342.7, p<0.0001), a significant effect of Group (F(3, 47)=5.92, p=0.002) on the s.d. of the Spectral Centroid, and a significant interaction (F(3, 2106)=29.84, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the s.d. of the Spectral Centroid showed that cCI produced smaller H/S contrasts than aCI and cNH.

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

1. School-aged children with CIs performed similarly to post-lingually deaf adult CI listeners on a voice emotion recognition task. Both showed significant deficits re: NH children and adults. NH children, however, showed poorer performance than NH adults in voice emotion recognition of 8-channel NV stimuli. (Chatterjee et al, 2015).

2. CI children’s voice emotion recognition scores were significantly correlated with their thresholds in an F0 discrimination task (Deroche et al, 2014; Deroche et al., submitted.)

3. Preliminary acoustic analyses of Happy/Sad productions of simple sentences show significant differences between groups, with children with CIs producing significantly smaller acoustic contrasts than others in most instances.

LME analysis showed significant effects of Emotion (F(1,2107)=807.7, p<0.0001), a significant effect of Group (F(3, 47)=3.66, p=0.019) on Intensity, and a significant interaction (F(3, 2107)=36.78, p<0.0001).

Paired t-tests (Bonferronicorrection) of the happy/sad difference in intensity (dB) showed that cCI produced smaller H/S contrasts than all other groups.

AcknowledgementsThis work was supported by R01 DC 00486-S08, R21 DC011905, R01 DC 014233, P30 DC004662, and T32 DC000013. We thank the Emily Shannon Fu Foundation and Dr Qian-Jie Fu for software used in noise vocoding and in experimental control of the emotion recognition task. We are grateful to the research participants for their support of our work.

1. Chatterjee M., Zion DJ, Deroche ML, Burianek BA, Limb CJ, Goren AP, Kulkarni AM, Christiansen JA, 2015. Voice emotion recognition by cochlear-implanted children and their normally-hearing peers. Hearing Res. 322, 151-162.

2. Deroche ML, Lu H, Limb CJ, Lin Y and Chatterjee M, 2014. Deficits in the pitch sensitivity of cochlear-implanted children speaking English or Mandarin. Front. Neurosci. 8:282. doi: 10.3389/fnins.2014.00282

3. Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., Papsin, B. C., 2009. Recognition of affective speech prosody and facial affect in deaf children with unilateral right cochlear implants. Child Neuropsychology 15, 136-146.

4. Luo, X., Fu, Q. J., & Galvin, J. J. , 2007. Vocal emotion recognition by normal-hearing listeners and cochlear implant users. Trends in Amplification, 11(4), 301-315.

5. Most, T., Aviner, C., 2009. Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing AIDS, and normal hearing. J. Deaf Stud. Deaf Educ. 14, 449-464.

6. Nakata T, Trehub SA, Kanda Y, 2012. Effect of cochlear implants on children’s perception and production of speech prosody. J. Acoust. Soc. Am. 131, 1307-1314.

7. Schorr, E. A., Roth, F. P., Fox, N. A., 2009. Quality of life for children with cochlear implants: perceived benefits and problems and the perception of single words and emotional sounds. J. of Sp., Lang., & Hear. Res. 52, 141-152.

8. Wang, D.J., Trehub, S.E., Volkova, A., Van Lieshout, P., 2013. Child implant users‘ imitation of happy-and sad-sounding speech. Front. Psychol. 4, 351.

Participants:

17 children with NH (cNH): 10 F, 7 M, mean age 10.77 years (s.d. 3.01 years).

13 children with CIs (cCI): 6 F, 7 M, mean age 13.44 years (s.d. 4.51 years), mean age at implantation 2.99 years (s.d. 4.23 years), mean duration of experience 10.45 years (s.d. 4.24 years).

10 adults with NH (aNH): 6F, 4 M

11 postlingually deaf adults with CIs (aCI): 7F, 4 M.

CI children’s voice emotion recognition scores (average of male and female talkers’ performance in RAUs) plotted against their mean F0 discrimination threshold (average of 100 and 200 Hz reference F0 conditions, in log semitones). A significant correlation was observed between the two (r=0.52, p = 0.006). The solid line shows the result of a Type II regression analysis.Chronological age was significantly related to the mean F0 thresholds but not to voice emotion recognition scores. Age at implantation was not related to either measure. [Participants: 27 children with CIs, age range 7-18 years, mean age 11.84 years; duration of device experience: 1-16 years, mean 8.59 years]

Diff

eren

ce in

F0

Ran

ge (H

-S)

Rat

io o

f F0

(Hap

py/S

ad)

Diff

eren

ce in

Mea

n In

tens

ity (

dB)

Rat

io o

f s.d

.of S

pect

ral C

entro

id

Rat

io o

f Spe

ctra

l Cen

troid

(H

appy

/Sad

)

**

************

********

****************

********

********

****

**

********

********

********

********

********

********

InitialAcousticAnalyses:Happy/SadContrastsinProduction

www.postersession.com

-1

0

1

ACI ANH CCI CNHStatus

MeanDiff_F0RangeRatio

factor(Status)

ACI

ANH

CCICNH

Summary

Voice Emotion Communication By Listeners With Cochlear ImplantsMonita Chatterjee1, Julie A Christensen1, Aditya M Kulkarni1, Mickael LD Deroche2, Sara A Damm1, Adam K Bosen1, Mohsen Hozan1, Charles J Limb3

1Boys Town National Research Hospital, Omaha, NE, USA; 2McGill University, Montreal, QC, CA; 3University of California, San Francisco, CA, USA

References

Introduction Results

1

2

3

ACI ANH CCI CNHStatus

Ratio_F0

factor(Status)

ACI

ANH

CCICNH

Recently published data on voice emotion recognition by children and adults with CIs (cCI and aCI respectively), compared to NH counterparts. CI listeners’ performance was equivalent to that of NH adults’ (aNH) performance with 8-channel noise-vocoded speech. NH children’s (cNH) performance reached ceiling with full-spectrum speech, but their performance with 8-channel vocoded speech was significantly poorer than NH adults’, and showed a strong developmental effect. (From Chatterjee et al., 2015). [Participants: 36 cCI age 6-18 years, mean age 12.15 years; 31 cNH age 6-18 years, mean age 10.76 years; 10 aNH, 9 aCI.]

Voice emotion processing, considered to be important in children’s quality of life and emotional development, is an area of deficit for children and adults with CIs (Luo et al, 2007; Schorr et al, 2009; Nakata et al, 2012; Most and Aviner, 2009; Wang et al., 2013).

Here, we present recent findings on voice emotion recognition, complex pitch discrimination and acoustic analyses of voice emotion production by school-aged children and adults with CIs, as well as their NH counterparts.

Methods1. Voice Emotion Recognition (Chatterjee et al., 2015)¾ Stimuli were 12 sentences from the HINT corpus,

spoken by a male and a female talker with each of five emotions (angry, happy, neutral, sad, scared) in a child-directed manner.

¾ Task was to hear each utterance and indicate which of the five emotions the participant thought they heard. No feedback was provided. Passive exposure to each talker’s style was provided prior to testing.

¾ NH children and adults performed the task both with full spectrum and noise-vocoded versions. CI children and adults heard only the full spectrum versions.

2. Complex Pitch Discrimination (Deroche et al., 2014)¾ Stimuli were broadband harmonic complexes, F0 =

100 Hz and F0 = 200 Hz.¾ Task was 3-interval forced-choice, method of

constant stimuli, child-friendly interface.¾ Psychometric functions were obtained and thresholds

and slopes were derived.

3. Voice Emotion Production¾ Participants read 20 sentences (e.g., This is it) in a

happy way and a sad way. Recordings were analyzed in Praat and Matlab..

1. Voice Emotion Recognition 2. F0 Discrimination and Voice Emotion Recognition

3. Voice Emotion Production

Mean F0 (Hz)LME analysis showed significant effects of Emotion (F(1,2107)=798, p<0.0001) and Group (F(3, 47)=4.38, p=0.009) on mean F0, and a significant interaction (F(3, 2107)=86.24, p<0.0001).

Paired t-tests (Bonferronicorrection) showed that cCIproduced smaller H/S contrasts than all other groups.

LME analysis showed effects of Emotion (F(1,2107)=386.4, p<0.0001), and a significant interaction between Emotion and Group (F(3, 2107)=13.10, p<0.0001).

Paired t-tests (Bonferroni correction) of the happy/sad differences in F0 range (Difference in ratio of F0max/F0min for H and S) showed that cCI and aNH produced smaller H/S contrasts than aCI.

0

5

10

15

20

ACI ANH CCI CNHStatus

MeanIntDiff

factor(Status)

ACI

ANH

CCICNH

F0 Range (F0max/F0min) Intensity (dB)

Spectral Centroid (Hz) Spectral Centroid Variation (s.d. in Hz)

1.0

1.5

2.0

2.5

3.0

ACI ANH CCI CNHStatus

MeanSpCsdRatio factor(Status)

ACI

ANH

CCICNH

1.0

1.2

1.4

1.6

ACI ANH CCI CNHStatus

MeanSpCentRatio

factor(Status)

ACI

ANH

CCICNH

LME analysis showed significant effects of Emotion (F(1,2106)=559.4, p<0.0001), a significant effect of Group (F(3, 47)=4.34, p=0.009) on Spectral Centroid, and a significant interaction (F(3, 2106)=43.82, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the spectral centroid showed that cCI produced smaller H/S contrasts than all other groups.

LME analysis showed significant effects of Emotion (F(1,2106)=342.7, p<0.0001), a significant effect of Group (F(3, 47)=5.92, p=0.002) on the s.d. of the Spectral Centroid, and a significant interaction (F(3, 2106)=29.84, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the s.d. of the Spectral Centroid showed that cCI produced smaller H/S contrasts than aCI and cNH.

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

1. School-aged children with CIs performed similarly to post-lingually deaf adult CI listeners on a voice emotion recognition task. Both showed significant deficits re: NH children and adults. NH children, however, showed poorer performance than NH adults in voice emotion recognition of 8-channel NV stimuli. (Chatterjee et al, 2015).

2. CI children’s voice emotion recognition scores were significantly correlated with their thresholds in an F0 discrimination task (Deroche et al, 2014; Deroche et al., submitted.)

3. Preliminary acoustic analyses of Happy/Sad productions of simple sentences show significant differences between groups, with children with CIs producing significantly smaller acoustic contrasts than others in most instances.

LME analysis showed significant effects of Emotion (F(1,2107)=807.7, p<0.0001), a significant effect of Group (F(3, 47)=3.66, p=0.019) on Intensity, and a significant interaction (F(3, 2107)=36.78, p<0.0001).

Paired t-tests (Bonferronicorrection) of the happy/sad difference in intensity (dB) showed that cCI produced smaller H/S contrasts than all other groups.

AcknowledgementsThis work was supported by R01 DC 00486-S08, R21 DC011905, R01 DC 014233, P30 DC004662, and T32 DC000013. We thank the Emily Shannon Fu Foundation and Dr Qian-Jie Fu for software used in noise vocoding and in experimental control of the emotion recognition task. We are grateful to the research participants for their support of our work.

1. Chatterjee M., Zion DJ, Deroche ML, Burianek BA, Limb CJ, Goren AP, Kulkarni AM, Christiansen JA, 2015. Voice emotion recognition by cochlear-implanted children and their normally-hearing peers. Hearing Res. 322, 151-162.

2. Deroche ML, Lu H, Limb CJ, Lin Y and Chatterjee M, 2014. Deficits in the pitch sensitivity of cochlear-implanted children speaking English or Mandarin. Front. Neurosci. 8:282. doi: 10.3389/fnins.2014.00282

3. Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., Papsin, B. C., 2009. Recognition of affective speech prosody and facial affect in deaf children with unilateral right cochlear implants. Child Neuropsychology 15, 136-146.

4. Luo, X., Fu, Q. J., & Galvin, J. J. , 2007. Vocal emotion recognition by normal-hearing listeners and cochlear implant users. Trends in Amplification, 11(4), 301-315.

5. Most, T., Aviner, C., 2009. Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing AIDS, and normal hearing. J. Deaf Stud. Deaf Educ. 14, 449-464.

6. Nakata T, Trehub SA, Kanda Y, 2012. Effect of cochlear implants on children’s perception and production of speech prosody. J. Acoust. Soc. Am. 131, 1307-1314.

7. Schorr, E. A., Roth, F. P., Fox, N. A., 2009. Quality of life for children with cochlear implants: perceived benefits and problems and the perception of single words and emotional sounds. J. of Sp., Lang., & Hear. Res. 52, 141-152.

8. Wang, D.J., Trehub, S.E., Volkova, A., Van Lieshout, P., 2013. Child implant users‘ imitation of happy-and sad-sounding speech. Front. Psychol. 4, 351.

Participants:

17 children with NH (cNH): 10 F, 7 M, mean age 10.77 years (s.d. 3.01 years).

13 children with CIs (cCI): 6 F, 7 M, mean age 13.44 years (s.d. 4.51 years), mean age at implantation 2.99 years (s.d. 4.23 years), mean duration of experience 10.45 years (s.d. 4.24 years).

10 adults with NH (aNH): 6F, 4 M

11 postlingually deaf adults with CIs (aCI): 7F, 4 M.

CI children’s voice emotion recognition scores (average of male and female talkers’ performance in RAUs) plotted against their mean F0 discrimination threshold (average of 100 and 200 Hz reference F0 conditions, in log semitones). A significant correlation was observed between the two (r=0.52, p = 0.006). The solid line shows the result of a Type II regression analysis.Chronological age was significantly related to the mean F0 thresholds but not to voice emotion recognition scores. Age at implantation was not related to either measure. [Participants: 27 children with CIs, age range 7-18 years, mean age 11.84 years; duration of device experience: 1-16 years, mean 8.59 years]

Diff

eren

ce in

F0

Ran

ge (H

-S)

Rat

io o

f F0

(Hap

py/S

ad)

Diff

eren

ce in

Mea

n In

tens

ity (

dB)

Rat

io o

f s.d

.of S

pect

ral C

entro

id

Rat

io o

f Spe

ctra

l Cen

troid

(H

appy

/Sad

)

**

************

********

****************

********

********

****

**

********

********

********

********

********

********

F0Range(F0max/

www.postersession.com

-1

0

1

ACI ANH CCI CNHStatus

MeanDiff_F0RangeRatio

factor(Status)

ACI

ANH

CCICNH

Summary

Voice Emotion Communication By Listeners With Cochlear ImplantsMonita Chatterjee1, Julie A Christensen1, Aditya M Kulkarni1, Mickael LD Deroche2, Sara A Damm1, Adam K Bosen1, Mohsen Hozan1, Charles J Limb3

1Boys Town National Research Hospital, Omaha, NE, USA; 2McGill University, Montreal, QC, CA; 3University of California, San Francisco, CA, USA

References

Introduction Results

1

2

3

ACI ANH CCI CNHStatus

Ratio_F0

factor(Status)

ACI

ANH

CCICNH

Recently published data on voice emotion recognition by children and adults with CIs (cCI and aCI respectively), compared to NH counterparts. CI listeners’ performance was equivalent to that of NH adults’ (aNH) performance with 8-channel noise-vocoded speech. NH children’s (cNH) performance reached ceiling with full-spectrum speech, but their performance with 8-channel vocoded speech was significantly poorer than NH adults’, and showed a strong developmental effect. (From Chatterjee et al., 2015). [Participants: 36 cCI age 6-18 years, mean age 12.15 years; 31 cNH age 6-18 years, mean age 10.76 years; 10 aNH, 9 aCI.]

Voice emotion processing, considered to be important in children’s quality of life and emotional development, is an area of deficit for children and adults with CIs (Luo et al, 2007; Schorr et al, 2009; Nakata et al, 2012; Most and Aviner, 2009; Wang et al., 2013).

Here, we present recent findings on voice emotion recognition, complex pitch discrimination and acoustic analyses of voice emotion production by school-aged children and adults with CIs, as well as their NH counterparts.

Methods1. Voice Emotion Recognition (Chatterjee et al., 2015)¾ Stimuli were 12 sentences from the HINT corpus,

spoken by a male and a female talker with each of five emotions (angry, happy, neutral, sad, scared) in a child-directed manner.

¾ Task was to hear each utterance and indicate which of the five emotions the participant thought they heard. No feedback was provided. Passive exposure to each talker’s style was provided prior to testing.

¾ NH children and adults performed the task both with full spectrum and noise-vocoded versions. CI children and adults heard only the full spectrum versions.

2. Complex Pitch Discrimination (Deroche et al., 2014)¾ Stimuli were broadband harmonic complexes, F0 =

100 Hz and F0 = 200 Hz.¾ Task was 3-interval forced-choice, method of

constant stimuli, child-friendly interface.¾ Psychometric functions were obtained and thresholds

and slopes were derived.

3. Voice Emotion Production¾ Participants read 20 sentences (e.g., This is it) in a

happy way and a sad way. Recordings were analyzed in Praat and Matlab..

1. Voice Emotion Recognition 2. F0 Discrimination and Voice Emotion Recognition

3. Voice Emotion Production

Mean F0 (Hz)LME analysis showed significant effects of Emotion (F(1,2107)=798, p<0.0001) and Group (F(3, 47)=4.38, p=0.009) on mean F0, and a significant interaction (F(3, 2107)=86.24, p<0.0001).

Paired t-tests (Bonferronicorrection) showed that cCIproduced smaller H/S contrasts than all other groups.

LME analysis showed effects of Emotion (F(1,2107)=386.4, p<0.0001), and a significant interaction between Emotion and Group (F(3, 2107)=13.10, p<0.0001).

Paired t-tests (Bonferroni correction) of the happy/sad differences in F0 range (Difference in ratio of F0max/F0min for H and S) showed that cCI and aNH produced smaller H/S contrasts than aCI.

0

5

10

15

20

ACI ANH CCI CNHStatus

MeanIntDiff

factor(Status)

ACI

ANH

CCICNH

F0 Range (F0max/F0min) Intensity (dB)

Spectral Centroid (Hz) Spectral Centroid Variation (s.d. in Hz)

1.0

1.5

2.0

2.5

3.0

ACI ANH CCI CNHStatus

MeanSpCsdRatio factor(Status)

ACI

ANH

CCICNH

1.0

1.2

1.4

1.6

ACI ANH CCI CNHStatus

MeanSpCentRatio

factor(Status)

ACI

ANH

CCICNH

LME analysis showed significant effects of Emotion (F(1,2106)=559.4, p<0.0001), a significant effect of Group (F(3, 47)=4.34, p=0.009) on Spectral Centroid, and a significant interaction (F(3, 2106)=43.82, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the spectral centroid showed that cCI produced smaller H/S contrasts than all other groups.

LME analysis showed significant effects of Emotion (F(1,2106)=342.7, p<0.0001), a significant effect of Group (F(3, 47)=5.92, p=0.002) on the s.d. of the Spectral Centroid, and a significant interaction (F(3, 2106)=29.84, p<0.0001). Paired t-tests (Bonferronicorrection) of the happy/sad ratio of the s.d. of the Spectral Centroid showed that cCI produced smaller H/S contrasts than aCI and cNH.

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

Group

aCI aNH cCI cNH

1. School-aged children with CIs performed similarly to post-lingually deaf adult CI listeners on a voice emotion recognition task. Both showed significant deficits re: NH children and adults. NH children, however, showed poorer performance than NH adults in voice emotion recognition of 8-channel NV stimuli. (Chatterjee et al, 2015).

2. CI children’s voice emotion recognition scores were significantly correlated with their thresholds in an F0 discrimination task (Deroche et al, 2014; Deroche et al., submitted.)

3. Preliminary acoustic analyses of Happy/Sad productions of simple sentences show significant differences between groups, with children with CIs producing significantly smaller acoustic contrasts than others in most instances.

LME analysis showed significant effects of Emotion (F(1,2107)=807.7, p<0.0001), a significant effect of Group (F(3, 47)=3.66, p=0.019) on Intensity, and a significant interaction (F(3, 2107)=36.78, p<0.0001).

Paired t-tests (Bonferronicorrection) of the happy/sad difference in intensity (dB) showed that cCI produced smaller H/S contrasts than all other groups.

AcknowledgementsThis work was supported by R01 DC 00486-S08, R21 DC011905, R01 DC 014233, P30 DC004662, and T32 DC000013. We thank the Emily Shannon Fu Foundation and Dr Qian-Jie Fu for software used in noise vocoding and in experimental control of the emotion recognition task. We are grateful to the research participants for their support of our work.

1. Chatterjee M., Zion DJ, Deroche ML, Burianek BA, Limb CJ, Goren AP, Kulkarni AM, Christiansen JA, 2015. Voice emotion recognition by cochlear-implanted children and their normally-hearing peers. Hearing Res. 322, 151-162.

2. Deroche ML, Lu H, Limb CJ, Lin Y and Chatterjee M, 2014. Deficits in the pitch sensitivity of cochlear-implanted children speaking English or Mandarin. Front. Neurosci. 8:282. doi: 10.3389/fnins.2014.00282

3. Hopyan-Misakyan, T. M., Gordon, K. A., Dennis, M., Papsin, B. C., 2009. Recognition of affective speech prosody and facial affect in deaf children with unilateral right cochlear implants. Child Neuropsychology 15, 136-146.

4. Luo, X., Fu, Q. J., & Galvin, J. J. , 2007. Vocal emotion recognition by normal-hearing listeners and cochlear implant users. Trends in Amplification, 11(4), 301-315.

5. Most, T., Aviner, C., 2009. Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing AIDS, and normal hearing. J. Deaf Stud. Deaf Educ. 14, 449-464.

6. Nakata T, Trehub SA, Kanda Y, 2012. Effect of cochlear implants on children’s perception and production of speech prosody. J. Acoust. Soc. Am. 131, 1307-1314.

7. Schorr, E. A., Roth, F. P., Fox, N. A., 2009. Quality of life for children with cochlear implants: perceived benefits and problems and the perception of single words and emotional sounds. J. of Sp., Lang., & Hear. Res. 52, 141-152.

8. Wang, D.J., Trehub, S.E., Volkova, A., Van Lieshout, P., 2013. Child implant users‘ imitation of happy-and sad-sounding speech. Front. Psychol. 4, 351.

Participants:

17 children with NH (cNH): 10 F, 7 M, mean age 10.77 years (s.d. 3.01 years).

13 children with CIs (cCI): 6 F, 7 M, mean age 13.44 years (s.d. 4.51 years), mean age at implantation 2.99 years (s.d. 4.23 years), mean duration of experience 10.45 years (s.d. 4.24 years).

10 adults with NH (aNH): 6F, 4 M

11 postlingually deaf adults with CIs (aCI): 7F, 4 M.

CI children’s voice emotion recognition scores (average of male and female talkers’ performance in RAUs) plotted against their mean F0 discrimination threshold (average of 100 and 200 Hz reference F0 conditions, in log semitones). A significant correlation was observed between the two (r=0.52, p = 0.006). The solid line shows the result of a Type II regression analysis.Chronological age was significantly related to the mean F0 thresholds but not to voice emotion recognition scores. Age at implantation was not related to either measure. [Participants: 27 children with CIs, age range 7-18 years, mean age 11.84 years; duration of device experience: 1-16 years, mean 8.59 years]

Diff

eren

ce in

F0

Ran

ge (H

-S)

Rat

io o

f F0

(Hap

py/S

ad)

Diff

eren

ce in

Mea

n In

tens

ity (

dB)

Rat

io o

f s.d

.of S

pect

ral C

entro

id

Rat

io o

f Spe

ctra

l Cen

troid

(H

appy

/Sad

)

**

************

********

****************

********

********

****

**

********

********

********

********

********

********

MeanF0(Hz)(H/SRatio)

VariationinF0(F0max/F0min)

Intensity(dB)(DifferenceinMeans)

Group

11

N=17N=13N=10N=11

NHListeners’PerceptionofProductionsbyCIChildren

� CIchildtalkers’phoneticproductionsofthesentenceswerehighlyintelligible.

� Howintelligibleweretheemotionsintheirproductions?

12

50

60

70

80

90

100

Percen

tcorrect

Medianscore:97%correctRange:91-100%correct

HowintelligiblearetheemotionsproducedbychildrenwithCIs?

13

Listeners:32participantswithNH(age6-21years)Talkers:7childrenwithCIs(age7–18years).Listeners:12participantswithNH(age7–22years)Talkers:9childrenwithNH(age6–18years)Task:2-alternative,forcedchoice[Happy/Sadidentification]

14

1.CIchildren’semotionproductionsarehardertoidentifythanNHchildren’sproductions.

14

PerformancewithCIchildtalkers’productionswassignificantlypoorerandmorevariable,thanperformancewithNHchildtalker’sproductions(p=0.011:independentsamplest-test)

0.5

0.6

0.7

0.8

0.9

1

NH CI

Scor

e (%

cor

rect

)

TalkerGroup

15

2.Predictors:Listener’sAge

ListenerAge(F(1,36)=17.10,p=0.0002)

ListenerAge(years)

Score(R

AUtr

ansformed

)

16

3.AdditionalPredictors:Talker’sAgeatImplantation,recencyofimplantation

à Talker’sAgeofImplantation:earlierisbetter(p<0.0001)

Summary:MainTake-HomeMessages� ChildrenwithCIshavesignificantdeficitsre:NHpeersinvoiceemotionrecognition.

� However,theirperformancewasverysimilartothatofpost-linguallydeafadultswithCIsàperipheral/devicedominated?

�  Inproductionofemotion,childrenwithCIsshowsmalleracousticcontraststhanpost-linguallydeafadultswithCIsàearlyauditoryexperiencematters.

� YoungerNHpeersofCIchildrenhavemoredifficultyidentifyingtheirvocalemotions.

17

Acknowledgements

JulieChristensen

AdityaKulkarniShauntelleCannon

JenniSis

CharlesLimb,UCSF

Yung-SongLin

MickaelDerocheShu-Chen

Peng

Hui-PingLu

Thanks to research participants and families, BTNRH CI Clinic and colleagues, and to NIH for

funding

MaryPatMoeller SophieAmbrose