aws re:invent 2016: robots: the fading line between real and virtual worlds (iot403)
Post on 08-Jan-2017
60 Views
Preview:
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Markku Lepisto, Principal Technology Evangelist, AWS
Olivier Klein, Emerging Technologies SA, AWS
Glen Fields, VP GTM and Strategy, ThingWorx
November 2016
IOT403: ROBOTSThe Fading Line Between Real and Virtual Worlds
What to Expect from the Session
• Connect Bluetooth devices with AWS IoT and Lambda
• Integrate an Amazon Alexa Skills and have it
communicate with AWS IoT
• Get a glimpse of the Amazon Lumberyard game
engine and use Cloud Canvas to integrate
a virtual world with AWS IoT
• Peak into the world of Augmented Reality
• Have fun with robots!
Intel Edison - AWS IoT & Cylon SDKs, Bluetooth
AWS IoT Device SDK &CylonJS Robot SDK
for Node.JS
Bluetooth connections for Edison and Sphero
Intel Edison - AWS IoT Thing & Thing Shadow
Register Edison Thing & Thing
Shadow clients, with keys and
certificates
Intel Edison – MQTT Messages & Shadow Updates
Connect to AWS IoT & Publish
(robot) connected & (initial) language
MQTT message
Message received. If the Action is
‘roll’, command robot to <roll> in
<direction> at <speed>
Shadow update received.
Command robot to change its color to
the <newcolor>
Intel Edison – Driving the Robot
Read keypresses on Edison,
send driving commands
via Bluetooth to Sphero
and publish actions as
MQTT messages
Intel Edison – Detecting Robot Collisions
On Collision event from the robot,
publish <collision> MQTT message
Speaker
Raspberry Pi 2Intel Edison
AWS IoT Device SDK &CylonJS Robot SDK
for Node.JS
AWS IoT Device SDK for Node.JS
MQTT MQTT
AWS IoTPlatform
Bluetooth
Intel Edison
Roll:
<direction, speed>
Raspberry Pi 2
Action: roll Action: roll
AWS IoTPlatform
Speaker
Gogogo !
Intel Edison Raspberry Pi 2
Action: collision Action: collision
AWS IoTPlatform
Event:
<collision>
Speaker
Ouch !
Raspberry Pi - AWS IoT Device SDK & Raspbian Sound Player
Register AWS IoTclient
Subscribe to all MQTT topics for sphero
Function for executing audio player on
Raspbian OS
Raspberry Pi – Playing Sounds for Action Messages
On receiving a message, play corresponding sound files in <current language>
Example JSON Messages
{ 'Action': ‘connect’, ‘Type’: ‘sphero’} // connected to Sphero robot
{ 'Action': ‘connect’, ‘Type’: ‘lumberyard’} // connected to Lumberyard cloud canvas
{ 'Action': ‘disconnect’ , ‘Type’: ‘sphero’} // disconnected from Sphero robot
{ 'Action': ‘disconnect’ , ‘Type’: ‘lumberyard’} // disconnected from Lumberyard canvas
{ 'Action': roll’, ‘Speed’: 30, ‘Direction’: 0} // impulse to roll at speed 30 straight ahead
{ 'Action': ‘speed’, ‘Speed’: 50} // change robot speed to 50
{ 'Action': ‘collision’} // collision event detected by robot
{ 'Action': ‘language’, ‘Language’: ‘korean’} // change robot language to Korean
The Power of Speech: Alexa
• Alexa, the voice service that powers
Echo, provides capabilities, or skills,
that enable customers to interact with
devices using voice
• Alexa Skills Kit (ASK) allows everyone
to build and publish their own skills
• Skills can be powered by AWS
Lambda
AWS Lambda
Skill:
Sphero
Intel Edison
Color: <orange> Color: <orange>
AWS IoTPlatform
Amazon AlexaSDK
Alexa, tell Sphero to change color to orange
Color:
<orange>
I changed the color for you
AWS Lambda - Implement Alexa Skill for Color Change
Lamdba Response to Alexa – what should Alexa say to the user
Get the current shadow of Edison thing
Lumberyard - Cloud Canvas
• Tools integrated into Lumberyard to
directly access AWS resources
• Storage, Database, Compute,
Identity, Queuing, Notification
• Integrated into the FlowGraph of the
IDE for easy visual scripting
• Lumberyard handles the backend
communication with AWS
Let’s connect the Virtual and Physical World
• Idea is to send robot movements from
Lumberyard to AWS IoT
• Glue is the Cloud Canvas, which calls
and AWS Lambda function via the
Lumberyard FlowGraph
• Sphero robot works with “impulses”,
Lumberyard with vectors changing at
high frequency – sending every
movement to AWS IoT is not scalable!
Amazon Lumberyard
AWS Lambda
Cloud
Canvas
Solution: Calculate ∆ on Coordinate System
• Camera / Player position exposes X,
Y, Z coordinates in Lumberyard
• Send coordinates to AWS Lambda in
a certain frequency (e.g. 1 second)
and calculate the ∆ between each
function call
• ∆ can then be used to calculate
impulses and sent it to the MQTT
topic
Math to the help - Coordinates to Impulse?
• Impulse = Direction * Speed
• Trigonometry helps to calculate
direction (0-360º)
• Pythagorean Theorem helps to
calculate distance
• Speed is distance over time
(frequency of AWS Lambda
function call)
• Z left out, Sphero can’t jump
X1,Y1
0º
90º
180º
270º
X2,Y2
AWS Lambda – IoT MQTT Protocol Bridge
Import AWS SDK for Python
Publish Lambda event payload to MQTT topic
Amazon Lumberyard
AWS LambdaCalculates Impulse
based on delta of coordinates
Intel Edison
Action: roll<direction, speed>
Action: roll<direction, speed>
AWS IoT
Action: Roll<direction, speed>
Cloud Canvas <X, Y, Z>
Amazon
LumberyardAWS Lambda
TopicsThingIntel Edison
Alexa ASK
Sphero
Robot
Amazon
Echo
The Final Architecture
Raspberry
PiSpeaker
Go Go Go!
Thing
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Glen Fields, VP GTM and Strategy
November, 30th, 2016
ThingWorxBridging the Digital and Physical Worlds
BRIDGING THE DIGITAL AND PHYSICAL WORLDS
DIGITAL & PHYSICAL WORLDS COMBINED
INTERNET OF THINGS
DIGITAL WORLD PHYSICAL WORLD
Augmented Reality: Superimposing computer graphics onto a live view of the world. Also known as “AR”
Internet of Things
ThingHuman
Data
Collect Data
Send DataThe ChallengeHow does the human
process and interact
with all of that data?
Monitor & Control Thing
IoT : Ability to digitally talk &
listen to physical things to
monitor and control
45
1000101011000011110100010011110111111011101101011000011101101110111101101111010000
0000001111110101011010001110011000101001001101010101111101000011000101011000011110
1000100111101111110111011010110000111011011101111011011110100000000001111110101011
0100011100110001010010011010101011111010000110001010110000111101000100111101111110
1110110101100001110110111011110110111101000000000011111101010110100011100110001010
0100110101010111110100001100010101100001111010001001111011111101110110101100001110
1101110111101101111010000000000111111010101101000111001100010100100110101010111110
1000011000101011000011110100010011110111111011101101011000011101101110111101101111
0100000000001111110101011010001110011000101001001101010101111101000011000101011000
0111101000100111101111110111011010110000111011011101111011011110100000000001111110
1010110100011100110001010010011010101011111010000110001010110000111101000100111101
1111101110110101100001110110111011110110111101000000000011111101010110100011100110
0010100100110101010111110100001100010101100001111010001001111011111101110110101100
0011101101110111101101111010000000000111111010101101000111001100010100100110101010
1111101000011000101011000011110100010011110111111011101101011000011101101110111101
1011110100000000001111110101011010001110011000101001001101010101111101000011000101
01100001111010001001111011111101110110001111000
10001010110
00011110100
01001111011
11110111011
01011000011
10110111011
11011011110
10000000000
11111101010
11010001110
01100010100
10011010101
01111101000
01100010101
10000111101
00010011110
11111101110
11010110000
11101101110
11110110111
10100000000
00111111010
10110100011
10011000101
00100110101
01011111010
00011000101
01100001111
IoT Data Growth (ZETTABYTES)
2014
9.82016
18.52018
34.32020
70.0
Big Data
Analytics
Visual
Processing
via Augmented Reality
JOV-0386-2DA-2N
Internet of
ThingsAugmented
Reality
Ability to digitally talk to
physical things to
monitor and control
Ability to see and
experience the digital
attributes of physical things
144
215
303
FY14 FY15 FY16
App Installs (million)
Source: Aeon Mobile., August 2016. Includes CV AR apps found in the Apple App Store and Google Play;
Vuforia SDK is the de facto standard
Market Share
270K+ 300M+80%+
121k
181k
272k
FY14 FY15 FY16
Registered Developers
Meet Thingworx studio
Simple UI DesignEasy, codeless “drag and drop” user
interface for ease of authoring AR
experiences
Content IntegrationSeamlessly integrate existing 3D
content, SLM and other enterprise
systems data
IIoT ReadyQuickly integrate IIoT sensor data and
enterprise system data from
ThingWorx
AR AnimationsPresent step-by-step animations with
Creo Illustrate
Designed for the Industrial Enterprise
Unique IdentifierQuickly identifies the unique thing and its relevant experiences
Real World AlignmentAligns 3D information contextually and spatially
Ability to ScaleTracking vast numbers of products, dynamically updated in the Cloud
Unique Identifier
Real World Alignment
Ability to Scale
S/N 1005 S/N 1006 S/N 1007 S/N 1008
S/N 1001 S/N 1002 S/N 1003 S/N 1004
S/N 1009 S/N 1010 S/N 1011 S/N 1012
ThingMark
Create Publish Scan Experience
• Codeless UI
• Import 3D CAD
and external data
• Integrate IoT
• Extend with Java
Script and CSS
• Publish to AWS
Cloud
• Saves all
experiences
• Access control
• Unique marker
• Provides Real
World Alignment
• Scalable
• Connects ‘Thing’
to Experience
• Window to the
Experience
• One app = All
experiences
Architecture
Experience
Creator
experience
servicefoundation
Publish
ThingMark
external data
sources
&
enterprise systems
experiences
thing model & thing
Sensor
Data
PLM CRM Service
experience
service
foundation
ThingMark
experience
AWS IoT
Connector
<Domain ID> + <Instance ID>
Solves two fundamental IoT business problems
1. Collect and Connect
• AWS IoT collects data from the edge into the cloud securely, at scale, and at a low cost
• AWS Cloud Services provides compute, storage, and security of your data
2. Interact
• ThingWorx uses data to analyze, create, and experience the IoT in a meaningful way.
• Contextually see and experience the digital data in the physical world through the power of Augmented Reality
Why partner with AWS as IoT cloud provider
CONNECT
ANALYZE
EDGE
CREATE
EXPERIENCE
AWS IoT Amazon
EC2
Amazon
DynamoDBAmazon Kinesis
Streams
Thingworx - AWS connector
• AWS IoT – Ingestion Layer
• AWS IoT Rule that forwards
data to Kinesis
• Kinesis is buffer between
AWS IoT and ThingWorx
Connector
• ThingWorx Connector pulls
data from the stream and
ingest into ThingWorx
platform
AWS IoT
Amazon
Kinesis
Streams
IoT
action
ThingWorx
AWS IoT
Connector
ThingWorx
Core
AWS Iot Edge
Node.js
AWS IoT ThingWorx
ThingWorx AR
Experience
Service
Thing Shadow Format
Shadow REST API
Thing Model Format
Amazon Kinesis
Client Library
Thank you!
Markku Lepisto, Principal Technology Evangelist, AWS
Olivier Klein, Emerging Technologies SA, AWS
Glen Fields, VP GTM and Strategy, ThingWorx
tinyurl.com/thingworx-studio
top related