Trending Topics by Geo-Location

Insight Data Engineering Tigran Antonyan

New York

Demo!!! API/Web Form Works!

• Only the a simple API/Form works so far ‣ via Flask

• http://trendinghashtags.net:5000

Data Transformation

Map ReduceFilter Map Map Reduce

Data Flow / PipelineMessageBroker

Real-TimeStreaming

DBNo-SQL

UIWeb-based

Camus

Batch Processing

Some work remains

❌

❌

❌

Cluster Setup

HDFS Namenode

Camus (cron) / Kafka Consumer

Spark Master

Kafka Broker

HDFS Datanode #1

Stream Reader / Kafka Producer

Kafka BrokerSpark

HDFS Datanode #2

Kafka BrokerSpark

HDFS Datanode #2

Kafka BrokerSpark

Cassandra Seed

Cassandra

Cassandra

m4.large nodes m3.medium nodes

About MeCamping

• BS in CS at FAU

• Software Engineer

‣ Citrix

‣ UCONN VoTeR Center