using the power of speech in your windows phone apps
DESCRIPTION
The Windows Phone SDK provides APIs which make it easy for developers to incorporate speech recognition and voice commands within a Windows Phone app. In this session, you will learn how to incorporate speech synthesis, voice commands, and simple speech recognition within your own Windows Phone app to provide a truly engaging user experience.TRANSCRIPT
![Page 1: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/1.jpg)
Using the Power of Speech in your Windows Phone Apps
Lori LalondeTwitter: @loriblalonde
Blog: geekswithblogs.net/lori-lalonde
![Page 2: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/2.jpg)
![Page 3: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/3.jpg)
About Me
Twitter: @loriblalondeEmail: [email protected]
Blog: geekswithblogs.net/lorilalondeLinkedIn:
http://ca.linkedin.com/in/lorilalonde
CTTDNUG
![Page 4: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/4.jpg)
Why should I integrate speech in my mobile
app?
![Page 5: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/5.jpg)
Minimum RequirementsWindows Phone 8.1 Development
Windows 8.1 (x64)
Visual Studio 2013 Update 2 or later (Pro, Premium, or Ultimate)OR
Visual Studio 2013 Express for Windows Update 2 or later
*includes Windows Phone 8.1 SDK
*Windows Phone Emulators – require Professional edition of Windows, and a processor that supports Client Hyper-V and Second Level Address
Translation (SLAT)
![Page 6: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/6.jpg)
Speech for Windows Phone
Voice Commands
Speech Recognition
Text-To-Speech
![Page 7: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/7.jpg)
Voice Commands
Simply say something to launch (or navigate into) the app…
… instead of:
tap, tap, scroll, select, dang…go back…
tap, scroll…
oh geez forget it!
![Page 8: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/8.jpg)
Add Capabilities
![Page 9: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/9.jpg)
Add a Voice Command Definition File<?xml version="1.0" encoding="utf-8"?>
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.0"> <CommandSet xml:lang="en-US"> <CommandPrefix>Contoso Rodeo</CommandPrefix> <Example> play a new game </Example>
<Command Name="PlayGame"> <Example> play a new game </Example> <ListenFor> [and] play [a] new game </ListenFor> <ListenFor> [and] start [a] new game </ListenFor> <Feedback> Starting a new game... </Feedback> <Navigate /> </Command>
... </CommandSet></VoiceCommands>
![Page 10: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/10.jpg)
Phrase Lists
<Command Name="PlayLevel"> <Example> replay level two </Example> <ListenFor> replay level {number} </ListenFor> <Feedback> Going to level {number}... </Feedback> <Navigate /> </Command>
<PhraseList Label="number"> <Item> one </Item> <Item> two </Item> <Item> three </Item> </PhraseList>
![Page 11: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/11.jpg)
Register the VCD using Windows.Media.SpeechRecognition; using Windows.Storage; ... private async Task InitializeVoiceCommands() {
Uri vcdUri = new Uri("ms-appx:///MyVoiceCommands.xml", UriKind.Absolute);
StorageFile file = await StorageFile.GetFileFromApplicationUriAsync(vcdUri);
await VoiceCommandManager.InstallCommandSetsFromStorageFileAsync(file); }
Installs the CommandSet whose specified language matches the device language setting
![Page 12: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/12.jpg)
Handle Navigation protected override void OnActivated(IActivatedEventArgs args) {
if (args.Kind == ActivationKind.VoiceCommand) {
Frame rootFrame = Window.Current.Content as Frame;VoiceCommandActivatedEventArgs vcArgs =
(VoiceCommandActivatedEventArgs)args;
//check for the command name that launched the appstring voiceCommandName = vcArgs.Result.RulePath.FirstOrDefault();
if (voiceCommandName == "ViewEntry") {
rootFrame.Navigate(typeof(ViewDiaryEntry), vcArgs.Result.Text);
} }
}
![Page 13: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/13.jpg)
DEMO!!!
![Page 14: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/14.jpg)
Speech Synthesis (Text-To-Speech)
Enable the app to read text to the user
The text can be:
A simple string
Speech Synthesis Markup Language (SSML) string
SSML file
![Page 15: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/15.jpg)
Speak! Part 1
using Windows.Media.SpeechSynthesis; ...
private SpeechSynthesizer synthesizer;
public async Task SpeakAsync(string textToSpeech) {
SpeechSynthesizer synthesizer = new SpeechSynthesizer();SpeechSynthesisStream synthesisStream =
await synthesizer.SynthesizeTextToStreamAsync(textToSpeech);
... }
![Page 16: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/16.jpg)
Speak! Part 2
public async Task SpeakAsync(string textToSpeech) {
SpeechSynthesizer synthesizer = new SpeechSynthesizer();SpeechSynthesisStream synthesisStream =
await synthesizer.SynthesizeTextToStreamAsync(textToSpeech);
if (synthesisStream != null) {
this.media.AutoPlay = true; this.media.SetSource(synthesisStream, synthesisStream.ContentType); this.media.Play();
} }
<MediaElement x:Name="media“ AutoPlay="False" />Add MediaElement to the view’s XAML
Set the media source to the stream and play
![Page 17: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/17.jpg)
DEMO!!!
![Page 18: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/18.jpg)
Enable the app to react when the user is speaking to it
Speech Recognition
![Page 19: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/19.jpg)
Built-In Speech Prompts/Confirmations
Heard You Say…
Unrecognized input
Did You Say…
Listening…
![Page 20: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/20.jpg)
using Windows.Media.SpeechRecognition; ... private SpeechRecognizer speechRecognizer; public async Task InitializeSpeechRecognizerAsync(string topicHint) {
speechRecognizer = new SpeechRecognizer();
//add web search grammar to the recognizer SpeechRecognitionTopicConstraint topicConstraint = new SpeechRecognitionTopicConstraint(SpeechRecognitionScenario.Dictation, topicHint);
speechRecognizer.Constraints.Add(topicConstraint);
await speechRecognizer.CompileConstraintsAsync(); }
Initialize SpeechRecognizer
A speech recognizer needs at least one constraint, and constraints must be compiled, before speech recognition can be performed.
![Page 21: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/21.jpg)
private SpeechRecognizer speechRecognizer; public async Task SpeakTextAsync() {
SpeechRecognitionResult recognitionResult = await speechRecognizer.RecognizeWithUIAsync();
... }
Using the Built-In UI
![Page 22: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/22.jpg)
private SpeechRecognizer speechRecognizer; public async Task SpeakTextAsync(string audiblePrompt, string exampleText) {
speechRecognizerUI.UIOptions.IsReadBackEnabled = true;speechRecognizerUI.UIOptions.ShowConfirmation = true;
speechRecognizer.UIOptions.AudiblePrompt = audiblePrompt;speechRecognizer.UIOptions.ExampleText = exampleText;
SpeechRecognitionResult recognitionResult = await speechRecognizer.RecognizeWithUIAsync();
... }
Setting UI Options
![Page 23: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/23.jpg)
DEMO!!!
![Page 24: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/24.jpg)
Questions
![Page 25: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/25.jpg)
Additional ResourcesMSDN - Speech Design Guidelines for Windows Phone Apps
MSDN - Quickstart: Speech Recognition (XAML)
MSDN - Responding to speech interactions (XAML)
Channel9 Video: Integrating Your App into the Windows Phone Speech Experience
W3C Speech Recognition Grammar Specification
W3C Speech Synthesis Markup Language
![Page 26: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/26.jpg)
apress.com
30% off Windows Phone 8 Recipes eBook
Discount Code: LORI14Expires: Dec 31, 2014
![Page 27: Using the Power of Speech in Your Windows Phone Apps](https://reader036.vdocuments.mx/reader036/viewer/2022062313/558e6a051a28abfa658b4659/html5/thumbnails/27.jpg)
Lori LalondeTwitter: @loriblalonde
Blog: geekswithblogs.net/lori-lalonde