hue architecture in the hadoop ecosystem and sql editor

18
SQL AND SEARCH WITH SPARK IN YOUR BROWSER Romain Rigaux

Upload: romain-rigaux

Post on 21-Apr-2017

920 views

Category:

Data & Analytics


2 download

TRANSCRIPT

SQL AND SEARCH WITH SPARKIN YOUR BROWSER

Romain Rigaux

100 000

WEB INTERFACE FOR ANALYZING DATA WITH APACHE HADOOPFREE AND OPEN SOURCE

—> USER TOOL

WHAT IS HUE?

CORE FEATURE: QUERY 1. Explore 2. Compose 3. Productionize

1. PREPARE, BROWSE

LIGHT ETL

http://livy.io/

INDEXER

morphline.conf

schema.xml

+

+

Collection

2. COMPOSE

EDITOR(S)

API

HS2

HS2

oozie

solr

Oracle, PostgreSQL, MySQL, JDBC, Phoenix, MR, ...

SEARCH DASHBOARDS

/select/admin/collections/get/luke...

/add_widget/zoom_in/select_facet/select_range...

REST AJAX

Templates+

JS Model

state:CA

3. PRODUCTIONIZE

WORKFLOWS

HDFS fileSaved query

SCHEDULER

/data/20160601/data/20160602/data/20160603...

year=2016,month=06,day=01year=2016,month=06,day=02year=2016,month=06,day=03...

Instance wf 1Instance wf 2Instance wf 3Instance wf 4

workflow.xmlcoordinator.xml

DEMO TIME

Bank auditing: > 10 000$

WHAT IS NEXT - Improved Query experience

- Richer SQL experience- Metadata search

- Single page layout UI

TWITTER

@gethue

USER GROUP

hue-user@

WEBSITE

http://gethue.com

LEARN

http://learn.gethue.com

THANKS!