speech mashups - dan burnett - voxeo - speechtek ny 2010
DESCRIPTION
At SpeechTEK New York in August 2010, Voxeo's Director of Speech Technologies, Dan Burnett, was part of a panel on "speech mashups". Dan explained what mashups are and demonstrated speech mashups written in Ruby that work on the Tropo service at http://www.tropo.com/ More information about Voxeo's activities at SpeechTEK NY can be found at: http://blogs.voxeo.com/events/speechtek-ny-2010/TRANSCRIPT
![Page 1: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/1.jpg)
Speech MashupsDan Burnett, Voxeo
SpeechTek 2010
![Page 2: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/2.jpg)
What are mashups?
<XML>JSON
HTTP
![Page 3: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/3.jpg)
Why VoiceXML?
![Page 4: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/4.jpg)
Why not VoiceXML?
![Page 5: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/5.jpg)
Tropo scripting
answer/reject/redirect/hangupcall/transfer/conference ask/say/record/log
startCallRecording/stopCallRecording
![Page 6: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/6.jpg)
Weather app
• Collect postal code from caller
• Fetch weather from Yahoo using YQL
• Speak weather to the caller
![Page 7: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/7.jpg)
Weather app
require 'open-uri'require 'json'
answer
hangup
![Page 8: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/8.jpg)
Weather app
require 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
say 'Thats all. Goodbye.'hangup
![Page 9: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/9.jpg)
Weather apprequire 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
ask 'Enter the ZIP code for a weather check'say 'Thats all. Goodbye.'hangup
![Page 10: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/10.jpg)
Weather apprequire 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI
#Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash
#Get the relevant weather channel details and throw them into a hash
#Speak back the results
} }
ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup
![Page 11: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/11.jpg)
Weather apprequire 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI yahoo_url = 'http://query.yahooapis.com/v1/public/yql?format=json&q=' query = "SELECT * FROM weather.forecast WHERE location = " + choice.value url = URI.encode(yahoo_url + query) #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash weather_data = JSON.parse(open(url).read) #Get the relevant weather channel details and throw them into a hash weather_results = weather_data["query"]["results"]["channel"]
#Speak back the results
} }
ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup
![Page 12: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/12.jpg)
Weather apprequire 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash #Get the relevant weather channel details and throw them into a hash
#Speak back the results say weather_results["description"] say "The wind chill is #{weather_results["wind"]["chill"]} degrees, " + "the wind speed is #{weather_results["wind"]["speed"]}" say "The forecast is #{weather_results["item"]["forecast"][0]["text"]}, " + "with a high of #{weather_results["item"]["forecast"][0]["high"]} degrees, " + "and a low of #{weather_results["item"]["forecast"][0]["low"]} degrees." } }
ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup
![Page 13: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/13.jpg)
Weather apprequire 'open-uri'require 'json'
answersay 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]", :repeat => 3, :timeout => 7, :onBadChoice => lambda { say 'Invalid entry, please try again.' }, :onTimeout => lambda { say 'Timeout, please try again.' }, :onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash #Get the relevant weather channel details and throw them into a hash
#Speak back the results
} }
ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup
![Page 14: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/14.jpg)
Demo
• Upload script
• Link to a telephony application name
• Add phone numbers/IM/Jabber/SMS
![Page 15: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/15.jpg)
Tropo Web API
answer/reject/redirect/hangupcall/session/transfer/conference ask/say/record/on/result/log
startCallRecording/stopCallRecording
JSON (JavaScript Object Notation)
tropo
![Page 16: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/16.jpg)
Weather app
{"tropo": [ {"say": [{"value": "Welcome to the Yahoo weather reader"}]}, {"ask": {"say": [{"value": "Enter the ZIP code for a weather check"}, {"value": "Invalid entry, please try again.", "event": "nomatch"}, {"value": "Timeout, please try again.", "event": "timeout"}]}, "repeat": 3, "timeout": 7, "choices": {"value": "[5 DIGITS]"}, }]}
----------------- Receive recognition result ------------------------
{"tropo": [ {"say": [{"value": "The wind chill is ..."}, {"value": "The forecast is ..."}, {"value": "Thats all. Goodbye."}]}}]}
![Page 17: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/17.jpg)
Other mashups• Yahoo Local over phone, SMS, and IM
https://www.tropo.com/docs/scripting/t_php-localsearch.htm
• SFO BART train arrival estimatehttps://www.tropo.com/docs/scripting/t_ruby-barteta.htm
• Find location of someone based on telephone numberhttps://www.tropo.com/docs/scripting/t_groovy-location-mashup.htm
• Look up political campaign contributionstext: [2-letter state id] [name] to +1.240.242.7944 .. ie "ut craig m smith"(also on jabber at [email protected])
• Look up bills in the NY Senate via IM or Twitterhttp://www.voiceingov.org/blog/?p=1005
• Volunteer firefighter arrival time at fire station
![Page 18: Speech Mashups - Dan Burnett - Voxeo - SpeechTEK NY 2010](https://reader033.vdocuments.mx/reader033/viewer/2022052900/555c037ad8b42a56448b53da/html5/thumbnails/18.jpg)
Summary