speech-enabling web apps

Speech-Enabling Web Apps

CAN YOU SPEAK MAGIC?

!2


!2

Ben Klang


ADD SPEECH TO THE WEB

!3


ADD SPEECH TO THE WEB•Speech Input API

!3


ADD SPEECH TO THE WEB•Speech Input API•Text-To-Speech (<Audio/>)

!3


ADD SPEECH TO THE WEB•Speech Input API•Text-To-Speech (<Audio/>)•WebRTC

!3


ADD SPEECH TO THE WEB•Speech Input API•Text-To-Speech (<Audio/>)•WebRTC

!3

http://bit.ly/HTML5_Speech_Input_APIhttp://www.w3.org/TR/webrtc/

http://bit.ly/HTML5_Speech_Input_API

http://www.w3.org/TR/webrtc/


SPEECH INPUT API

!4


SPEECH INPUT API

!5


SPEECH INPUT API

!5

<input type="text" x-webkit-speech />


ANNYANG!

!6


!7


DEMO

!8


SPEECH INPUT API CAVEATS

!9


SPEECH INPUT API CAVEATS•Chrome Only :(

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)•Partial Firefox implementation from GSoC

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)•Partial Firefox implementation from GSoC•Requires ASR Server

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)•Partial Firefox implementation from GSoC•Requires ASR Server•Only Google runs one today

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)•Partial Firefox implementation from GSoC•Requires ASR Server•Only Google runs one today•serviceURI attribute not yet implemented

!9


SPEECH INPUT API CAVEATS•Chrome Only :(•Uses Google ASR(duh)•Partial Firefox implementation from GSoC•Requires ASR Server•Only Google runs one today•serviceURI attribute not yet implemented•Specification maturity seems slow

!9


TEXT-TO-SPEECH

!10


TTS API + <AUDIO/>

!11


TTS API OPTIONS

!12


TTS API OPTIONS•AT&T: http://developer.att.com

!12

http://developer.att.com

http://nuancemobiledeveloper.com/

http://translate.google.com/translate_tts?tl=en&q=TEXT



•Nuance NDEVhttp://nuancemobiledeveloper.com/

!12






•Nuance NDEVhttp://nuancemobiledeveloper.com/

•Google: http://translate.google.com/translate_tts?

tl=en&q=TEXT

!12





<AUDIO/> CAVEATS

!13


<AUDIO/> CAVEATS•You can’t pay for Google TTS

!13

http://bit.ly/Browser_Audio_Codecs


<AUDIO/> CAVEATS•You can’t pay for Google TTS•No specified Mandatory To Implement (MTI) codecs

!13



<AUDIO/> CAVEATS•You can’t pay for Google TTS•No specified Mandatory To Implement (MTI) codecs•Broad consensus

!13



<AUDIO/> CAVEATS•You can’t pay for Google TTS•No specified Mandatory To Implement (MTI) codecs•Broad consensus•Everyone: MP3 (+containers H.264, MP4)

!13



<AUDIO/> CAVEATS•You can’t pay for Google TTS•No specified Mandatory To Implement (MTI) codecs•Broad consensus•Everyone: MP3 (+containers H.264, MP4)•Except IE: Ogg/Vorbis, Opus, WebM

!13



<AUDIO/> CAVEATS•You can’t pay for Google TTS•No specified Mandatory To Implement (MTI) codecs•Broad consensus•Everyone: MP3 (+containers H.264, MP4)•Except IE: Ogg/Vorbis, Opus, WebM•http://bit.ly/Browser_Audio_Codecs

!13



!14

CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?

WHAT IS WEBRTC TO ME?

!15



!15

Telephones in Web Browsers!



!15

Telephones in Web Browsers!Telephones in Web Browsers!


How does WebRTC Work?

!16


!17

http://


!17

http://

Alice


!17

http://

Alice Bob


!17

http://Get

me B

ob p

leas

e!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

Alice Bob


!17

http://Get

me B

ob p

leas

e!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

SDP:v=0 o=bob 19915 0 IN IP4 0.0.0.0

s=- t=0 0 m=audio 61001 RTP/SAVPF 109

Alice Bob


!17

http://Get

me B

ob p

leas

e!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

SDP:v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

SRTP

SRTP


!17

http://Get

me B

ob p

leas

e!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

SDP:v=0 o=bob 19915 0 IN IP4 0.0.0.0


Alice Bob

SRTP

SRTP

X


!18

Alice Bob


!18

Alice Bob

Get m

e Bob

ple

ase!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09


!18

Alice Calling!

SDP:

v=0 o=freeswitch 19915 0 IN IP4 0.0.0.0

s=- t=0 0

m=audio 61001 RTP/SAVPF 109

Alice Bob

Get m

e Bob

ple

ase!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09


!18

Alice Calling!

SDP:

v=0 o=freeswitch 19915 0 IN IP4 0.0.0.0

s=- t=0 0

m=audio 61001 RTP/SAVPF 109

Alice Bob

SRTP

SRTP

Get m

e Bob

ple

ase!

SDP:

v=0

o=al

ice 2

0518 0

IN IP

4 0.0

.0.0

s=-

t=0 0

m

=audio

54609 R

TP/SAVPF 1

09

CAN YOU SPEAK MAGIC?CAN YOU SPEAK MAGIC?

Example RTC Apps

!19


Example RTC Apps

!19

2 Examples


“Communicating isn’t going to be what you’re doing - it’s what you’ll be doing

while you’re doing something else”

- Geoff Hollingworth Ericsson Head of AT&T Foundry

!20


1. Incident Response

!21


!22


INCIDENT RESPONSE

!23


INCIDENT RESPONSE•Timely, Contextual Information•Adapt for mobile vs. desktop users•Group-based communication•Inherit from existing organizational groups•Allow ad-hoc participants (“guest” parties)•Federate with external services•Incident recording/logging•“Lessons learned” and process improvement•Links from/to issue tracking systems

!23


2. Medical Records Management

!24


!25


MEDICAL RECORDS MGMT

!26


MEDICAL RECORDS MGMT•Automate Medical Claims•Secure Caller Authentication•Reuse primary auth via website•Verify with voice biometrics•Cross-check against caller location•Call recording/transcription•Medical advice given to patient automatically added to patient file•Auditing/Service Quality Assurance

!26


HTTPS://TALKY.IO/ATLRUG

!27

https://talky.io/atlrug


WEBRTC CAVEATS

!28


WEBRTC CAVEATS•Bleeding edge, developing standard

!28

http://iswebrtcreadyyet.com/


WEBRTC CAVEATS•Bleeding edge, developing standard•Only available on Chrome, Firefox

!28



WEBRTC CAVEATS•Bleeding edge, developing standard•Only available on Chrome, Firefox•Only available on Desktop

!28



WEBRTC CAVEATS•Bleeding edge, developing standard•Only available on Chrome, Firefox•Only available on Desktop•Well funded/backed development

!28



WEBRTC CAVEATS•Bleeding edge, developing standard•Only available on Chrome, Firefox•Only available on Desktop•Well funded/backed development•Expect to see it mainstream (Desktop + Mobile) as soon as 2014

!28



WEBRTC CAVEATS•Bleeding edge, developing standard•Only available on Chrome, Firefox•Only available on Desktop•Well funded/backed development•Expect to see it mainstream (Desktop + Mobile) as soon as 2014•http://iswebrtcreadyyet.com/

!28



!29


!30

adhearsionconf.comEarly Bird Discount: atlrug


!31

http://mojolingo.com @MojoLingo@bklang

[email protected]

http://bit.ly/HTML5_Speech_Input_APIhttp://www.w3.org/TR/webrtc/http://iswebrtcreadyyet.com/

Early Bird Discount: atlrug

http://mojolingo.com

http://twitter.com/bklang

mailto:[email protected]

http://bit.ly/HTML5_Speech_Input_API

http://www.w3.org/TR/webrtc/


speech-enabling web apps

Technology