the cocktail party effect. an inclusive vision of conversational interactions
Post on 21-Apr-2017
68 Views
Preview:
TRANSCRIPT
Isabella Loddo Università Iuav di Venezia
NTT Data Italia
Dario Martini Università degli Studi
della Repubblica di S.Marino
The cocktail party effect. An inclusive vision of conversational interactions
Myopia
Diversity
Different People
Different Devices
Interaction Model
Sullivan & Igoe, 2004Model Reality
Impairment
Microsoft, 2016
Captchas
Testing
Benchmarking Reaction
Testing + Feedback
Assistive devices
“For most of us, technology makes things easier. For a person with disability, it makes things possible.”
Judy Heumann
Assistive Technologies
Comparison
BulkyFragile
Expensive
EmbeddedConsistent
Cheap
Higher adoption BRAILLE SCREEN-READERS
Talking Machines
1889
Talking Machines
1939 >
synthesis recording
1927 >
1939 >
synthesis recording
1927 >
…
…
formant speech synthesis
concatenative speech synthesis
Talking Machines
Speech Synthesis
formant speech synthesis
concatenative speech synthesis
no database required
artificial sounding
more intelligible than human speech
large database
natural sounding
intelligible as human speech
Expression
Prosodyphrasing
pitch
loudness
tempo
rhythm Liu, 2006
phrasing
pitch
loudness
tempo
rhythm Hewlett et al, 2006
Prosody
phrasing
pitch
loudness
tempo
rhythm
Prosody
talking shouting
phrasing
pitch
loudness
tempo
rhythm
Oliveira, 2012
Prosodyenhance storytelling
express own speech style
Dlugan, 2012
phrasing
pitch
loudness
tempo
rhythm Hammen et al, 1994
Prosody
Intelligibility
Trouvain, 2007
concatenative
formant
Average 100
com
preh
ensio
n
rate (syllabes / second)
95
90
85
80
75
2 4 6 8 10 12 14 16 18
concatenative
formant
Blind
Speech Rates
Asakawa et al, 2003
50%80%50%80%sighted blind
350
wpm
300
400
450
500
250
200
150
100
50
0
software peak rate
software avg. rate
1.6 X
2.8 X
Relevant Scanning
On the web, scanning tasks are more frequent than linear readings.
75%50%
15%
Mobility
Mobile devices have become the main usage scenario.
100
75
50
25
02009 2010 2011 2012 2013 2014 2015 2016
mobile screen reader usage (%)
Situated listening
Users need to distinguish relevant information in real environments.
Cocktail party effect
1953
Guerreiro et al, 2015
Concurrent voices
10090
70
50
30
10
com
preh
ensio
n
information bandwidth
80
60
40
20
01 X 1,5 X 2 X 2,5 X 3 X 3,5 X 4 X 5 X4,5 X 5,5 X 6 X
1 voiceVoices
2 voices
3 voices
Concurrent voices
information bandwidth is increased equally in sighted and blind subjects
the maximum increase also depends on age
voice differences are preferred, but not essential
are very promising for web searches, news, assisted navigation, multi-touch simulations
Guerreiro et al, 2016
From Reading to Conversation
human level
Word Error Rate
-18% / year
thanks to artificial intelligence cloud computing
25
20
15
10
5
02008 2009 2010 2011 2012 2013 2014 2015 2016 2017
introduction of deep learning
Mutual Recognition
“Normal people, when they think about speech recognition, they want the whole thing. They want recognition, they want understanding and they want an action to be taken.”
Hsiao-Wuen Hon, Microsoft Research
Meaning
Mental models
screen reader: tool-like
vs
voice assistant: human-like
Qvarfordt, 2004
Black Mirror, 2013
Human-like
Human-like?
Affective Interaction
2013
Conclusions
Screen Readers are the most popular assistive technology
1
Conclusions
Naturalness has improved, but blind users still prefer formant synthesis
2
Conclusions
Screen Readers are slow in relevant scanning tasks
3
Conclusions
Screen Readers can be improved by applying the Cocktail Party Effect
4
Conclusions
Voice spatiality and formant synthesis can improve mobile experience
5
Scenario
Doctor
Crossing
Tickets
ParkingVegetablesBank
Thank youQuestions?
ISABELLA LODDO Università Iuav di Venezia NTT DATA Italia isabella.loddo@nttdata.com
DARIO MARTINI Università degli Studi della Repubblica di San Marino dariomartini@me.com
top related