iot data as service with hadoop

14
Internet Of Things & Big Data As Service

Upload: quantified-self-dublin

Post on 14-Apr-2017

193 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: IoT Data as Service with Hadoop

Internet Of Things & Big Data

As Service

Page 2: IoT Data as Service with Hadoop

About me

Sriram ReddyArchitectwww.Ammeon.com

Software Architect with a passion for

• Linked Devices• Device Multi Tenancy• Low Power Wide Area Networks• Distributed Device Management • Low Power Device Security

• Distributed Data Persistence• Distributed Data Processing• Analytics

Page 3: IoT Data as Service with Hadoop

Index

1. Introduction2. Internet Of Things

• Micro Controller & Processor• Power Management

3. IOT Networks• Long Range• Short Range

4. IOT Data Types• Binary (MQTT & Proto buff)• Jason

5. IOT Data Acquisition• MQTT• CoAP• Kafka

5. IOT Data processing• Spark• Nifi• StreamSets

6. Data Persistence• Hadoop• Cassandra• OrentDB• Titan• Mongo

7.Reporting• BIRT Reporting• Eclipse RCP

8. Analytics9. Market Vertices 10. Q&A

Page 4: IoT Data as Service with Hadoop

Introduction

Nothing is new. There are devices, there is network and there is data.

Challenge is Lot Of Devices, Network Everywhere, Lot Of Data

Page 5: IoT Data as Service with Hadoop

IOT Triad

Communications

AnalyticsDevice

EnablementHW-ClinetContainerModeling

Dashboard

EthernetGSM/3G/4G/MB-IOT BLE/LORA /WIFI

AVR/PIC/Psoc (MCU)SOC(MCP)Boards (Pis)

MQTTCoAPHTTPKAFKA

HadoopCassandraIgniteKafkaSpark/Flink/DrillNIFI/Streamsets/Camel

An IOT deployment architecture demands right device design, right communications and data management & analytics.

Page 6: IoT Data as Service with Hadoop

IOT Devices

Microprocessor Based Devices

Microcontroller Based Devices

Raspberry Pi, Orange Pi, Banana Pi, Nano Pi,Bagel Bone, Panda board, Onion, Chip etc.

• SOCs from Boradcom, Rock chip, All winner, Intel etc • Runs full Linux/Windows • Powerful • Power-hungry

Arduino, Wiring, Sypris etc • Micro-chip, Atmel, Nordic etc • SOC with memory and other peripherals• Low power• Less power consumption

Page 7: IoT Data as Service with Hadoop

IOT Communications

Long Range

Short Range

Networks to span full metros to countries

• Traditional operator provided 2G/3G/4G• LORA• SIGFOX

With in 100 meter range

• WIFI• BLE

Page 8: IoT Data as Service with Hadoop

IOT Data Types

Binary Data

JSON

Binary data provides size advantage but adds more processing.

• MQTT• Protobuffs

JSON or any markup payloads offer human readability and existing processing infrastructure.

Page 9: IoT Data as Service with Hadoop

IOT Data Acquisition

MQTT

CoAP

Kafka

• Lot of queues (millions)• Light Weight• Open source brokers• Well Integrated • Well supported

• HTTP like• Matured with telco industry• Human readable

• Light server • Not very heavy client• Client controlled• Fast• Easy to scale• Easy to replicate

Page 10: IoT Data as Service with Hadoop

IOT Data Processing

Spark Ignite

NifiStreamSets

Beam

Page 11: IoT Data as Service with Hadoop

IOT Data Persistence

Hadoop Cassandra

Oriented DB

Page 12: IoT Data as Service with Hadoop

IOT Market

Page 13: IoT Data as Service with Hadoop

IOT Market

Page 14: IoT Data as Service with Hadoop

Q&A