217 wwdc16 – introducing sirikit final 05...“hey siri, can you please do me a favor and launch...

© 2016 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

“Hey Siri, say hello to apps”

App Frameworks #WWDC16

Session 217

Introducing SiriKit

Robby Walker SiriKit EngineeringBrandon Newendorp SiriKit EngineeringCorey Peterson SiriKit Design

Agenda

SiriKit

NEW

Agenda

SiriKitExtensions

NEW

Agenda

SiriKitExtensionsDesign

NEW

UnicornChat

“Hey Siri, send a UnicornChat message”UnicornChat

“Hey Siri, send a UnicornChat message”

To whom?

UnicornChat


“Celestra”

To whom?

UnicornChat

What do you want to say to Celestra?


“Celestra”

To whom?

UnicornChat



“Let’s add more sparkle transitions”

“Celestra”

To whom?

UnicornChat

“Send a message to Celestra on UnicornChat”

“UnicornChat Celestra let’s add more sparkle transitions”

“ ”

“Hey Siri, can you please do me a favor andlaunch the UnicornChat app to send a text withthe message let’s add more sparkle transitions”

To whom?

“Hey Siri, can you please do me a favor andlaunch the UnicornChat app to send a text withthe message let’s add more sparkle transitions”

“My very best friend Celestra”

SiriKit and Your App


Quality


QualityConsistency


QualityConsistencyEasy to adopt

Speech

Speech Intent

Speech Intent Action

Speech Intent ResponseAction

Vocabulary


Vocabulary AppLogic


Vocabulary AppLogic

UserInterface



“Hey Siri, send a UnicornChat message to Celestra”



Vocabulary AppLogic

UserInterface



Vocabulary AppLogic

UserInterface

Recipient





Vocabulary AppLogic

UserInterface

Recipient

Content


Vocabulary AppLogic

UserInterface

Domain: MessagesIntent: sendMessageRecipient: CelestraContent: Let’s add more sparkle transitions


Vocabulary AppLogic

UserInterface

SiriKit

SiriKit

Simple

SiriKit

SimplePowerful

Speaking Siri’s Language

Brandon Newendorp SiriKit Engineering


Siri works in domains


Siri works in domainsMessaging, VoIP calling, Payments, Workouts, Ride booking, Photo search


Siri works in domainsMessaging, VoIP calling, Payments, Workouts, Ride booking, Photo searchGroup of known actions, or intents


“Hey Siri, send Buttercup 85 bucks on UnicornPay for rainbow polish”



Domain: Payments



Domain: Payments Intent: sendPayment



Domain: Payments Intent: sendPayment App: UnicornPay



Domain: Payments Intent: sendPayment App: UnicornPayPayee: Buttercup



Domain: Payments Intent: sendPayment App: UnicornPayPayee: ButtercupAmount: 85



Domain: Payments Intent: sendPayment App: UnicornPayPayee: ButtercupAmount: 85Currency: USD



Domain: Payments Intent: sendPayment App: UnicornPayPayee: ButtercupAmount: 85Currency: USDNote: Rainbow polish


Common language


Common languageIntents describe actions


Common languageIntents describe actionsIntentResponses describe results

IntentSpeaking Siri’s Language


Action to be performed


Action to be performedZero-to-many parameters


Action to be performedZero-to-many parametersClassified into a domain

IntentResponseSpeaking Siri’s Language


Result of handling an intent


Result of handling an intentResponse code


Result of handling an intentResponse code NSUserActivity

Lifecycle of an Intent


Resolve


Resolve Confirm


Resolve Confirm Handle

ResolveLifecycle of an Intent


Help Siri understand the values the user provided


Help Siri understand the values the user providedOpportunity to influence Siri’s behavior


Help Siri understand the values the user providedOpportunity to influence Siri’s behaviorProvide resolution response



Domain: Payments Intent: sendPayment App: UnicornPayPayee: Buttercup Amount: 85Currency: USDNote: Rainbow polish

Your app matched the user’s request to a specific personsuccess(with:)



Your app would like the user to confirm the resolved person

confirmationRequired(with:)


Your app needs the user to select from a list of people

disambiguation(with:)


Your app had insufficient information to resolve the person, and would like Siri to ask the user to be more specific

needsMoreDetails(for:)


Your app requires a value be specified, but no value was provided

needsValue


Your app can’t use the specified valueunsupportedWithReason

Your app doesn’t require a value for this parameter


notRequired

ConfirmLifecycle of an Intent


Tell Siri the expected result of an intent


Tell Siri the expected result of an intentCheck required state


Tell Siri the expected result of an intentCheck required stateProvide an intent response


Tell Siri the expected result of an intentCheck required stateProvide an intent responseSiri prompts for confirmation as necessary

Service available?



Service available?


Signed in?


Service available?


Signed in?Sufficient balance?


IntentResponse

Service available?


Signed in?Sufficient balance?


ResultCode: Success Payee: Buttercup Amount: 85 Currency: USD Note: Rainbow polish

HandleLifecycle of an Intent

Perform the requested action


Perform the requested actionProvide as much information about the result as possible



Network calls can take time


Network calls can take timeSiri shows waiting UI


Network calls can take timeSiri shows waiting UIProvide response within a few seconds


Network calls can take timeSiri shows waiting UIProvide response within a few secondsOtherwise use InProgress response code



IntentResponse

ResultCode: Success Payee: Buttercup Amount: 85 Currency: USD Note: Rainbow polish

Adding SiriKit to Your App

Adding SiriKit

Adding SiriKit

Two new extension points

Adding SiriKit

Two new extension points• Intents extension

Adding SiriKit

Two new extension points• Intents extension• Intents UI extension

Adding SiriKit


Provide words and phrases to aid recognition

Adding SiriKit


Provide words and phrases to aid recognition• App vocabulary

Adding SiriKit


Provide words and phrases to aid recognition• App vocabulary• User vocabulary

Adding SiriKit

Intents Extension

Intents extensionAdding SiriKit


Basis of SiriKit


Basis of SiriKitSupports one or more intents


Basis of SiriKitSupports one or more intentsRuns in the background while Siri is active


Basis of SiriKitSupports one or more intentsRuns in the background while Siri is activeImplements resolve, confirm, and handle methods

SecurityAdding SiriKit


Restrict while locked


Restrict while lockedLocal Authentication framework

WWDC 2014Keychain and Authentication with Touch ID


Restrict while lockedLocal Authentication frameworkApple Pay support

NEW

WWDC 2014Keychain and Authentication with Touch ID

What’s New with Wallet and Apple Pay Mission Tuesday 3:00PM

Adding SiriKit

Intents Extension

Adding SiriKit

Intents Extension Intents UI Extension

Intents UI extensionAdding SiriKit


Bring your app’s interface into Siri


Bring your app’s interface into SiriProvide a UIViewController


Bring your app’s interface into SiriProvide a UIViewController Optional


Bring your app’s interface into SiriProvide a UIViewController OptionalAvailable for certain intents


Show additional information


Show additional informationUpdate status


Show additional informationUpdate statusDisplayed alongside Siri content


Show additional informationUpdate statusDisplayed alongside Siri contentCan replace select Siri interface elements


Show additional informationUpdate statusDisplayed alongside Siri contentCan replace select Siri interface elementsNot interactive

Adding SiriKit

Intents Extension Intents UI Extension

Adding SiriKit

Intents Extension AppIntents UI Extension

VocabularyAdding SiriKit


Many apps have unique ways of describing things


Many apps have unique ways of describing thingsSiri needs your assistance to understand words and phrases unique to your app


Many apps have unique ways of describing thingsSiri needs your assistance to understand words and phrases unique to your appSome phrases are part of your app


Many apps have unique ways of describing thingsSiri needs your assistance to understand words and phrases unique to your appSome phrases are part of your appOther phrases are unique to the user of your app

App-specific vocabularyAdding SiriKit


Words and phrases that are part of your app


Words and phrases that are part of your appKnown to all users of your app


Words and phrases that are part of your appKnown to all users of your appDefined in a plist


Words and phrases that are part of your appKnown to all users of your appDefined in a plistLocalized


Vocabulary plist


Start a workout with UnicornFit

Vocabulary plist

Example Uses for App



workoutName

Vocabulary plist


Parameter


Phrase


workoutName

Colorlicious

Vocabulary plist


Parameter


Pronunciation

Phrase


workoutName

Colorlicious

colorlishus

Vocabulary plist


Parameter


Pronunciation

Examples

Phrase


workoutName

Colorlicious

colorlishus

Do a Colorlicious in UnicornFit

Vocabulary plist


Parameter

User-specific vocabularyAdding SiriKit


Terminology that’s specific to an individual user


Terminology that’s specific to an individual userProvided by your app


Terminology that’s specific to an individual userProvided by your appOrderedSet of terms

Other Contacts


All Contacts

Other Contacts


Recent Contacts All Contacts

Other Contacts


Recent Contacts

Other Contacts


Recent ContactsFavorites


Contacts, workout names, photo albums, photo keywords


Contacts, workout names, photo albums, photo keywords• iOS address book is already available to Siri



Only provide necessary values



Only provide necessary valuesSend data on changes



Only provide necessary valuesSend data on changesDelete promptly



Only provide necessary valuesSend data on changesDelete promptlyRemove all vocabulary when needed

PermissionsAdding SiriKit


Request authorization


Request authorization Explain what data goes to Siri


Request authorization Explain what data goes to SiriRequest permission from your app

SiriKit and iOS

Siri is an integral part of iOS

SiriKit and iOS

Siri is an integral part of iOSIntents.framework

SiriKit and iOS


Contacts integration

SiriKit and iOS


Contacts integrationCallKit

SiriKit and iOS

For Free!

Selecting Your App

Selecting Your App

User must say the name of your app

Selecting Your App

User must say the name of your appBundle display name

Selecting Your App

User must say the name of your appBundle display nameCan be used in many parts of speech

Selecting Your App

User must say the name of your appBundle display nameCan be used in many parts of speechVerbing the app name

Selecting Your App

User must say the name of your appBundle display nameCan be used in many parts of speechVerbing the app name• “UnicornChat Celestra order five rainbow rolls please”

Get me a bicorn ride on UnicornRide to the forest

UnicornPay 18 dollars to Sparkle Sparkly

UnicornChat Celestra order five rainbow rolls please

Start a 15-minute trot with UnicornFit

Ask CJ to UnicornPay me 25 dollars to cover dinner at Noon’s Grill

Using UnicornChat tell Pinky Nose that I’m running late

I need a UnicornRide to get me home

End my UniDuel in UnicornFit

Call Lucky Moontrotter using UnicornChat

Designing a Great Siri Experience

Corey Peterson SiriKit Design

Design Topics

Design Topics

How Siri helps you

Design Topics

How Siri helps youDesign guidelines

Design Topics

How Siri helps youDesign guidelinesPolish the design

User Interface

Conversational Interface


Context through conversation


Context through conversationInformation in the user’s head


Context through conversationInformation in the user’s headShaped by questions and responses


Context through conversationInformation in the user’s headShaped by questions and responses• Important to think about how Siri asks and responds

Ready to Send It?

“Send it”

Ready to Send It?

“Send it”

“No”

Ready to Send It?

“Send it”

“No”

“Cancel”

Ready to Send It?

“Send it”

“No”

“Cancel”

“Read it to me again”

Ready to Send It?

“Send it”

“No”

“Cancel”


“Change it”

Ready to Send It?

“Send it”

“No”

“Cancel”


“Change it”

“Yes” “Yes please” “Send” “OK” “Yes thank you” “Do it” “Yes I would”

“No thanks” “Don’t do it” “Not yet” “I’m not ready” “Nope” “Nah”

“Discard the message” “Cancel it” “Cancel the message” “Discard”

“Read it” “What does it say” “Read it back to me”

No edit” “Send it to Eric instead” “Edit the message” “Change it to say…”

Ready to Send It?

And how you can help, tooHow Siri Helps You


Test with many different requests


Test with many different requestsTest in different situations


Test with many different requestsTest in different situationsProvide info to Siri

Design Guidelines

A conversation with your usersDesign Guidelines


Prepare


PrepareBe a good listener


PrepareBe a good listenerAsk questions


PrepareBe a good listenerAsk questionsRespond to the request

PrepareDesign Guidelines


Users aren’t exactly like you


Users aren’t exactly like youAsk real people—“What would you say to Siri?”

“Get a ride”

“Get a ride”

“Get me to SFO with UnicornRides”

“Get a ride”

“Get me to SFO with UnicornRides”

“Ask UnicornRides to get me a Pegasus for 6 people from 300 Post Street to Bill Graham Auditorium”

Be a good listenerDesign Guidelines


Use the original request


Use the original requestPick good defaults


Use the original requestPick good defaultsMake educated guesses


Use the original requestPick good defaultsMake educated guessesAvoid bad surprises

Ask questionsDesign Guidelines


Keep it simple!


Keep it simple! Only ask for things that are necessary


Keep it simple! Only ask for things that are necessaryHandle errors

ConfirmationDesign Guidelines


Presenting a single option


Presenting a single optionCheck with the user:


Presenting a single optionCheck with the user:• If you made an educated guess


Presenting a single optionCheck with the user:• If you made an educated guess• If there is a big impact

DisambiguationDesign Guidelines


Presenting multiple options


Presenting multiple options• Small number of possibilities


Presenting multiple options• Small number of possibilities• Filtered list from the user’s request


Presenting multiple options• Small number of possibilities• Filtered list from the user’s request

Siri can read the list

Final confirmationDesign Guidelines


Siri checks with your extension


Siri checks with your extension• Send info to Siri to present


Siri checks with your extension• Send info to Siri to present

Siri checks with the user

Siri gives the resultsRespond to the Request


Matches the request


Matches the requestWorks everywhere (CarPlay, “Hey Siri”)


Matches the requestWorks everywhere (CarPlay, “Hey Siri”)Siri provides visual UI

Custom UI (optional)Respond to the Request


Bring the feel of your app into Siri


Bring the feel of your app into SiriLimit what you show


Bring the feel of your app into SiriLimit what you showDesign for multiple device sizes


Use API to hide redundant info


Use API to hide redundant infoRepresent everything visually


Use API to hide redundant infoRepresent everything visuallyStill send all info to Siri

In your appRespond to the Request


App provides follow up actions


App provides follow up actionsDon’t surprise the user

Polish the Design

647 x 1150

GuidelinesGuide Examples


Include your app name


Include your app nameKeep your examples short


Include your app nameKeep your examples shortProvide multiple examples for each intent


Include your app nameKeep your examples shortProvide multiple examples for each intent• Prioritize intents and examples


Include your app nameKeep your examples shortProvide multiple examples for each intent• Prioritize intents and examples• Localized examples


Include your app nameKeep your examples shortProvide multiple examples for each intent• Prioritize intents and examples• Localized examples

Make sure they work

AvoidGuide Examples

Would you please tell meet a ride with UnicornRides”g

here is my UnicornRide?”w““Hey Siri,G

W

GoodGuide Examples

““ et a ride with UnicornRides”gG

here is my UnicornRide?”wW

Iterate on the Experience


Experiment


Experiment Try it with real people


Experiment Try it with real peopleTry it in different situations


Experiment Try it with real peopleTry it in different situationsDoes it feel right?

Summary

Summary

SiriKit

Summary

SiriKitExtensions, Intents, Vocabulary

Summary

SiriKitExtensions, Intents, VocabularyDesign the conversation

More Information

https://developer.apple.com/wwdc16/217

Related Sessions

What’s New with Wallet and Apple Pay Mission Tuesday 3:00PM

Engineering Privacy for Your Users Pacific Heights Wednesday 4:00PM

Enhancing VoIP Apps with CallKit Mission Thursday 5:00PM

Related Sessions

Extending Your Apps with SiriKit Nob Hill Thursday 1:40PM

Labs

SiriKit Lab FrameworksLab C Thursday 3:00PM

SiriKit Lab FrameworksLab B Friday 9:00AM

217 wwdc16 – introducing sirikit final 05...“hey siri, can you please do me a favor and launch...

Documents