Kafka ProducerOpenDaylight Project Proposal

Xiaoyu Chen, williscc, williscc@cisco.comAndrew McLachlan, amclach@cisco.com

• Plugin Overview

– Provides an interface between OpenDaylight and a Kafka Cluster• Implemented as a Northbound plugin• Supports a Kafka Producer implementation

– Ability to publish to kafka topics– Supports compression (currently gzip/snappy)– Supports codec types (currentl raw/avro) but naturally extensible– Supports sync/async modes

• Currently acquires data from the Event Topic Broker (ETB) >=Lithium• Able to filter data by Topic ID (ETB) into a defined topic, using x-path• Supports multiple Kafka endpoints.

Kafka Cluster

Kafka Consumer

OpenDaylight

Event Topic Broker (ETB)

Kafka Consumer

Kafka Producer

NETCONF connector (supporting ETB SPI)

hweventsource generator (supporting

ETB SPI)

• Implementation Example

– Nothing out of the ordinary here• Current code has been tested in

this configuration– NETCONF Notifications as an

event source– hweventsource (see Core

Tutorials) is included in the project at time of writing

– Code tested against Lithium stable

Ope

nDay

Ligh

t odl-message-bus

Southbound Plugin(example is hweventsource

Network DevicesNetwork DevicesSome Device

Northbound ODL-Kafka Plugin

SB APIs from Models

Data models (YANG)

NB APIs from Models

Data models (YANG)

SAL Service Registry

Compile yang model and register

Compile yang model and register

RESTful API

Event Handling

Subscribe to a topic {pattern matching}

Events

Aggregated Events

Request:{ "event-aggregator:input": { "notification-pattern": “notification-pattern": "*sample*", "node-id-pattern":”*00" }}Response:{ "output": { "topic-id": "9f4e18cd-06f0-4ad6-9d16-ad9a34eb4031" }}

<?xml version="1.0" encoding="UTF-8"?><payload xmlns="urn:cisco:params:xml:ns:yang:messagebus

:eventaggregator"><SampleEventSourceNotification><Source>EventSourceSample01</

Source><Message>Hello World [Mon Oct 05 14:08:55 BST

2015]</Message></SampleEventSourceNotification></payload>

{u'timestamp': 1444050535820, u'rawdata': '<?xml version="1.0" encoding="UTF-8"?

><payload xmlns="urn:cisco:params:xml:ns:yang:messa

gebus:eventaggregator"><SampleEventSourceNotification><Source>EventSourceSample01</

Source><Message>Hello World [Mon Oct 05 14:08:55 BST

2015]</Message></SampleEventSourceNotification></payload>

', u'host_ip': u'0.0.0.0', u'src': u'EventSourceSample01'}

• Example REST Configuration

• PUT: http://localhost:8181/restconf/config/kafkaproducer:kafka-producer-config

• HEADERS: application/yang.data+json

• BODY:• {• "kafka-producer-config": {• "metadata-broker-list": "192.168.1.1:9092,192.168.1.2:9092",• "topic": “test-topic”,• "producer-type": "sync",• "compression-codec": "none",• "message-serialization": "raw", • "event-subscriptions": "278206ca-746e-45b9-9846-a51dfbb086f5"• }• }

• Project Status

– Working Code– Willing Committers– Looking to update for Beryllium and Boron– Examining next set of feature requirements– Examining other input methods

• Thank you