building open source iot cloud

BUILDING OPEN SOURCE IOT CLOUD

Dejan Bosanac

WHO AM I?

Dejan Bosanac

•  Senior Software Engineer at Red Hat •  Messaging and Integration background (ActiveMQ, Camel, Fabric8) •  Focused more on messaging and backend for IoT lately

AGENDA

•  Describe typical IoT application •  Describe all the components and common pitfalls •  Introduce Eclipse IoT projects to the rescue

•  Kura •  Hono •  Kapua

•  Future

IOT APPLICATIONS

Devices Cloud

Applications

IOT APPLICATIONS

Device

Device

Applica+on

Device Applica+on

Applica+onIOTCloud

DEVICES

•  Everything from simple sensors to small computers •  Connectivity

•  Some are IP enabled •  Some have just near-range connectivity, like Bluetooth •  Variety of network protocols used (MQTT, LWM2M,…)

•  Functions •  Sensors – send data – temperature sensor •  Actuators – control environment – A/C Unit

APPLICATIONS

•  Enterprise applications, micro-services and everything in between •  Want to use data generated by devices •  Want to control devices •  Want to control the cloud environment … more about that in a minute

CLOUD

•  Connects devices and applications •  Provides connectivity layer •  Provides security layer •  Provides device state •  Handle device data

PITFALLS

•  Create a silo application •  Hard to upgrade and maintain •  Solution don't scale •  Security is an afterthought

SOLUTION

•  Build well maintained open source stack •  Scalable, secure and maintainable •  Developers should focus on devices and applications

STACK

•  Device Gateway - Eclipse Kura - https://www.eclipse.org/kura/ •  IoT Connector – Eclipse Hono - https://projects.eclipse.org/projects/iot.hono •  IoT Cloud – Eclipse Kapua - https://projects.eclipse.org/projects/iot.kapua

GATEWAY

DEVICE GATEWAY

•  Device IP onboarding •  Data pre-processing •  Control devices •  Operational management

GATEWAY ARCHITECTURE

Data Center

Gateway

Application

Sensors

INSERT DESIGNATOR, IF NEEDED 15

KURA ARCHITECTURE

KURA SERVICES

•  I/O Services – serial, Bluetooth, GPS, … •  Data services - Store and forward data using MQTT, Apache Camel •  Cloud services – request/reply •  Configuration service – OSGi configuration •  Web administration interface

KURA APPLICATION

•  OSGi Bundle •  Deployed and managed by Kura •  Using Kura APIs to communicate with devices and cloud •  Apache Camel Integration

KURA APPLICATION public void setCloudService(CloudService cloudService) { cloudService = cloudService;}protected void activate(ComponentContext componentContext, Map<String, Object> properties) { ... // Acquire a Cloud Application Client for this Application cloudClient = cloudService.newCloudClient("greenhouse"); cloudClient.addCloudClientListener(this); ...}protected void doPublish() { ... KuraPayload kuraPayload = new KuraPayload(); kuraPayload.addMetric("temperature", temperature); cloudClient.publish("sensors", kuraPayload, DFLT_QOS, DFLT_RETAIN,DFLT_PRIORITY); ...}

KURA CAMEL

public class MyKuraRouter extends KuraRouter { @Override public void configure() throws Exception {

from("timer://heartbeat"). setBody(constant("Hello")). to("kura-cloud:myApplication/myTopic"); }}

CONNECTOR

CONNECTOR

•  Messaging infrastructure that connects gateways and applications •  Eclipse Kura uses MQTT •  Usually combined with message brokers like Apache ActiveMQ or Eclipse Paho

MQTT

•  OASIS standard (v3.1.1) •  Created by IBM and Eurotech •  Lightweight

•  Small network message •  Simple protocol

•  Pub / Sub •  Quality of Service •  Connection failures •  Very popular in IoT scenarios

LIMITATIONS

•  Scalability •  Connections •  Destinations

•  Security based on broker addresses •  General purpose messaging

•  No message format

ECLIPSE HONO – GOALS

•  Tailored general messaging for IoT solutions •  Solve recurring problems •  Provide messaging APIs for common operations •  Support multiple IoT protocols (MQTT, AMQP, LWM2M,…) •  Support any underlying messaging infrastructure

•  JMS •  Kafka

ECLIPSE HONO – FEATURES

•  Scalability •  Multi-tenancy •  Device-based security •  Multi-protocol support

ECLIPSE HONO – APIS

•  AMPQ 1.0 based •  Defines message formats coming in and out of Hono •  Defines message exchange patterns

INSERT DESIGNATOR, IF NEEDED 27

AMQP

•  International Standard (ISO/IEC ISO 19464) •  Binary Protocol •  Rich feature set:

•  conversation multiplexing •  advanced flow control •  Type system •  QoS Guarantees

•  Symmetrical message exchange •  No Broker required

INSERT DESIGNATOR, IF NEEDED 28

AMQP

Message(properties: {

correlation-id: 1,to: "$management",reply-to: "/myaddress"

},application-properties: {

"name" -> "newQueue","operation" -> "CREATE","type" -> "org.example.queue"

},application-data: AmqpValue(

Map("max_size" -> "2000Mb"

))

)

•  It is not a broker •  It never owns a message •  It propagates AMQP transfer, settlement and disposition frames between endpoints •  Message based or link based routing

AMQP ROUTER

Router

/device1

/device2

/device2

INSERT DESIGNATOR, IF NEEDED 30

AMQP ROUTER

•  It can be deployed in multiple router-broker-endpoint topology •  Redundant paths •  Benefits

•  Better scaling due to more focused tasks •  Smart routing can be used to partition the traffic •  Ideal candidate for gateway into the system

INSERT DESIGNATOR, IF NEEDED 31

SCALABLE MESSAGING

•  Combination of brokers and routers provides powerful tool box •  Brokers should focus on storing messages •  Routers should do the rest •  Allows for horizontal scaling topologies that can solve IoT challenges

ECLIPSE HONO- APIS

•  Telemetry •  Command and Control •  Device Registration •  Device Lifecycle

ECLIPSE HONO – ARCHITECTURE

MQTTDevice

LWM2M Device

AMQP Device

HONO

ProtocolAdapter

ProtocolAdapter

Device Management

DataCollec+on

AMQP

AMQP

AMQP

AMQPAMQP

ECLIPSE HONO – ARCHITECTURE

•  Protocol Adapters •  Stateless •  Provide conversion to common protocols used in IoT

•  MQTT •  LWM2M •  HTTP/Rest

•  Clients – devices and Cloud services •  Connects using AMQP using well defined APIs

ECLIPSE HONO – ARCHITECTURE

Client

Client

Client

Router Network

HonoServer

HonoServer

App

App

HonoServer

Brokers

ECLIPSE HONO – ARCHITECTURE

•  Hono server •  Stateless – can be scaled •  Validates message format •  Does device-based authentication

ECLIPSE HONO – ARCHITECTURE

•  Router Network •  Provide connection scalability •  Routes AMQP messages through the system

•  Brokers •  Any AMQP 1.0 compatible system •  Used to save persistent messages

ECLIPSE HONO – TECHNOLOGY

•  Hono server •  Vert.x + Qpid Proton •  Spring Boot •  Docker

• Scalable and Cloud Ready!

ECLIPSE HONO – TECHNOLOGY

•  Scalable messaging •  Apache Qpid Dispatch Router •  Apache ActiveMQ Artemis

• Scalable and Cloud Ready!

ECLIPSE KAPUA

ECLIPSE KAPUA – GOALS

•  Provide complete IoT Cloud solution •  Define and Implement needed services •  Ready to run

ECLIPSE KAPUA – ARCHITECTURE

ECLIPSE KAPUA – BACKHAND SERVICES

•  Data Management •  Device Registry •  Device Management

ECLIPSE KAPUA – FRONTEND SERVICES

•  Management Console •  API Gateway

ECLIPSE KAPUA - IMPLEMENTATION

•  Micro-services oriented •  Pluggable service locator

•  Single JVM •  OSGi •  Cloud Deployment

ECLIPSE KAPUA – 1.0

•  MQTT based •  Apache ActiveMQ in the messaging layer •  JDBC store for services •  Elasticsearch for data store •  Single VM deployment

FUTURE

INSERT DESIGNATOR, IF NEEDED 47

INSERT DESIGNATOR, IF NEEDED 48

FUTURE - KURA

•  More data pre-processing •  BPM integration

•  AMQP support •  Gateway support proxy

INSERT DESIGNATOR, IF NEEDED 49

KAPUA

•  Micorservice implementation •  Docker images •  REST/AMQP APIs

•  Hono support

KAPUA + HONO

CONCLUSION

•  Together Kura, Hono and Kapua will be able to answer even the most challenging IOT demands •  Lots of work ahead •  Everything open source •  Backed by big companies, like Red Hat, Eurotech, Bosch, … •  Join the effort

THANK YOU

•  https://www.eclipse.org/kura/

•  https://projects.eclipse.org/projects/iot.hono

•  https://projects.eclipse.org/projects/iot.kapua