computers have feelings too
TRANSCRIPT
And you are?Paul Glavich
Principal Consultant @ readify
Author
Former MVP of 13 years
International Speaker (if you count NZ)
Industry veteran > 20 years
AgendaCognitive services – Overview
Setting up
API Examination, Analytics, Emotion, Vision
Flailing while demo fails
Future?
Wrap up
Microsoft AI Overview
Cognitive Services
• Prebuilt/Pre-trained Models
• Dev / Human friendly
Machine learning,
Bots
• Cortana Intelligence Suite
• Bot framework
• Build/train your own models
Cognitive toolkit
• Deep learning
• Neural networks
• Script driven
Cognitive Services OverviewMicrosoft Cognitive Services - an API platform
That’s right – It is all about the “Apey Eyes”
Approx. 30 Artificial intelligence API’s
Current ListVision
◦ Computer Vision API, Emotion API PREVIEW, Video Indexer PREVIEW, Face API, Video API PREVIEW, Content moderator, Custom Vision Service PREVIEW
Speech◦ Custom Speech Service PREVIEW, Translator Speech API, Bing Speech API, Speaker Recognition API PREVIEW
Language◦ Language Understanding Intelligent Service PREVIEW, Bing Spell Check API, Web Language Model API PREVIEW, Text Analytics API
PREVIEW, Translator Text API, Linguistic Analysis API PREVIEW
Knowledge◦ Recommendations API PREVIEW, Academic Knowledge API PREVIEW, Knowledge Exploration Service PREVIEW, QnA Maker API
PREVIEW, Entity Linking Intelligence Service API PREVIEW, Custom Decision Service PREVIEW
Search◦ Bing Autosuggest API, Bing Video Search API, Bing Image Search API, Bing Web Search API, Bing News Search API, Bing Custom Search
Labs◦ Project Prague, Gesture-based controls, Project Johannesburg, Route logistics, Project Abu Dhabi, Distance matrix, Project Nanjing,
Isochrones calculations, Project Cuzco, Event associated with Wikipedia entries, Project Wollongong
Current ListVision
◦ Computer Vision API, Emotion API PREVIEW, Video Indexer PREVIEW, Face API, Video API PREVIEW, Content moderator, Custom Vision Service PREVIEW
Speech◦ Custom Speech Service PREVIEW, Translator Speech API, Bing Speech API, Speaker Recognition API PREVIEW
Language
◦ Language Understanding Intelligent Service PREVIEW, Bing Spell Check API, Web Language Model API PREVIEW, Text Analytics API PREVIEW, Translator Text API, Linguistic Analysis API PREVIEW
Knowledge◦ Recommendations API PREVIEW, Academic Knowledge API PREVIEW, Knowledge Exploration Service PREVIEW, QnA Maker API PREVIEW, Entity Linking
Intelligence Service API PREVIEW, Custom Decision Service PREVIEW
Search◦ Bing Autosuggest API, Bing Video Search API, Bing Image Search API, Bing Web Search API, Bing News Search API, Bing Custom Search
Labs◦ Project Prague, Gesture-based controls, Project Johannesburg, Route logistics, Project Abu Dhabi, Distance matrix, Project Nanjing, Isochrones calculations,
Project Cuzco, Event associated with Wikipedia entries, Project Wollongong
The favourite childLUIS
◦ Language Understanding Intelligent Service
Conversational API (Apey Eye )
https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/
We wont be looking at this
What we will be looking atTextAnalytic API
◦ Extract information such as sentiment, key phrases, topics and language from your text
Emotion API◦ Examines facial expression in an image as an input, and returns the confidence across a set of emotions
for each face in the image, as well as bounding box for the face.
But before we dive inWhat about some of the other players?
Google◦ Machine Learning, Cloud Natural Language API, Cloud Speech API, Cloud Translation API, Cloud Vision API,
Cloud Video Intelligence API BETA◦ Tensorflow, Kaggle, API.AI◦ Deepmind – acquired in 2014
AWS◦ Lex, Polly, and Rekognition, Alexa voice service◦ Speech recognition, natural language understanding, lifelike speech, image recognition, and image analysis
Salesforce?◦ Yep◦ Can distinguish customers from imposters by tracking action patterns
And others….
SetupAdd Cognitive services to your azure subscription
◦ Manually
◦ ARM Template
◦ Powershell
Access keys
Location
Rest Call formathttps://[location].api.cognitive.microsoft.com/[service]/[version]/[action]
Headers required◦ Content-type
◦ Ocp-Apim-Subscription-Key
SentimentSimple Json structure to submit text documents
ID used to correlate response
https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment
Request with synchronous response
Supports English, Spanish, French, and Portuguese with 12 more in preview.
DemoTextAnalytic Raw API usage
Is that positive or negative?
Key phrasesSame Json structure as Sentiment
https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/keyPhrases
Request with synchronous response
Currently supports English, German, Spanish and Japanese.
DemoKey phrases Raw API usage
We can determine whether it is positive or negative, but what are the key phrases in that?
LanguageSimilar Json structure as Sentiment
Request with synchronous response
120 Languages are supported
https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/languages
DemoLanguage Raw API usage
Watch your language
TopicsSame Json structure as Sentiment
Stop words – excluded from topic detection
Stop phrases – excluded from returned results
Minimum 100 Documents
Submit request, then check on operation status.
https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/languages
https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/operations/{operationId}
Topics responseInitial submit
◦ 202 Accepted
◦ “operation-Location” header also returned
◦ https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/operations/54a25882d7ef494ca779b31d46f842bb
OperationStatus
EmotionCan detect
◦ Happiness
◦ Sadness
◦ Surprise
◦ Anger
◦ Fear
◦ Contempt
◦ Disgust
◦ Neutral
Works with video as well (via async operation)
https://westus.api.cognitive.microsoft.com/emotion/v1.0/recognize
DemoEmotion API Raw API usage
VisionCategorise content of an image
Find faces
Determine if contains mature/adult content
Url or image upload
https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze?visualFeatures=...&details=...&language=...
Supports English as default and Simplified Chinese as language to return.
VisionSpecify what you want as query string parameters
visualFeatures◦ Categories - categorizes according to taxonomy defined in documentation.
◦ Tags – returns list of words related to the image content.
◦ Description - describes image with a complete English sentence.
◦ Faces – if faces are present, return coordinates, gender and age.
◦ ImageType - clipart or line drawing.
◦ Color - accent color, dominant color, or black&white.
◦ Adult – Adult in nature? Racy in nature?
details◦ Celebrities - identifies celebrities if detected in the image.
◦ Landmarks - identifies landmarks if detected in the image.
DemoVision API Raw Usage
A Fluent APIBest way to learn (IMHO) is do.
Fluent API on top of cognitive services
1. TextAnalytics
2. Emotion
3. Vision and more to come
Simple usage
DemoSentiment and keyphrase analysis of emails
Display sentiment result of emails.
http://dddsydney2017glavsdemo.azurewebsites.net
Demo - high level
Gmail
Scheduler Q
Collect Email
Text Analysis
CosmosDB
Website
Send
Collect
Analyse
Store
Analyse
Get Mail
Analysedresults
Analysed Email
Azure Sentiment
Analysis
Other fluent examplesShow me the tests
Fluent API
◦ https://github.com/glav/CognitiveServicesFluentApi
What I think the future holdsMore consumerisation of AI
Specialised services beyond “simple API’s”
More compute “on the device” with greater value augmented by the cloud
Google’s API.AI is a step towards easier consumption
More seamless application integration.
What now?Visual Studio
◦ File -> New -> Console Application (.Net core)
Install-Package Glav.CognitiveServices.FluentApi.TextAnalytic –Pre
var result = awaitTextAnalyticConfigurationSettings.CreateUsingConfigurationKeys(TestConfig.TextAnalyticsApiKey, LocationKeyIdentifier.WestUs)
.SetDiagnosticLoggingLevel(LoggingLevel.Everything)
.AddDebugDiagnosticLogging()
.UsingHttpCommunication()
.WithTextAnalyticAnalysisActions()
.AddSentimentAnalysis("I am having a fantastic time.")
.AnalyseAllSentimentsAsync();
Links and resourcesThe demo
◦ https://github.com/glav/DDDSydney2017SentimentDemo
Fluent API
◦ https://github.com/glav/CognitiveServicesFluentApi
Nuget Packages
◦ https://www.nuget.org/packages/Glav.CognitiveServices.FluentApi.TextAnalytic/0.5.0-alpha
◦ https://www.nuget.org/packages/Glav.CognitiveServices.FluentApi.Emotion/0.5.0-alpha