sql for elasticsearch
DESCRIPTION
Slides of the Talk Jodok Batlogg held at the Elasticsearch Meetup in BerlinTRANSCRIPT
![Page 1: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/1.jpg)
SQL on Elasticsearch?
![Page 2: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/2.jpg)
How all started
![Page 3: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/3.jpg)
You know, for searchquerying 24 000 000 000 Records in 900ms
@jodok
![Page 4: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/4.jpg)
6 ES Master Nodesc1.xlarge
40 ES nodes per zonem1.large8 EBS Volumes
6 Node Hadoop Cluster+ Spot Instances
3 AP server / MCc1.xlarge
![Page 5: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/5.jpg)
![Page 6: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/6.jpg)
Elastic Search as Primary Storage?
NoSQL Roadshow 2013 Jodok Batlogg
![Page 7: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/7.jpg)
• Security Model? • Transactions? • Data security? • Toolsets? • Larger Computations? • Availability?
![Page 8: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/8.jpg)
D I S T R I B U T E D D A T A S T O R E W I T H S Q L . S I M P L E . R E L I A B L E . S C A L A B L E .
![Page 9: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/9.jpg)
Open Source (Apache 2.0)
shared nothing
is high available and cheap to operate.
not NOSQL but SQL
NOFS but distributed BLOBs
![Page 10: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/10.jpg)
![Page 11: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/11.jpg)
Storage
Data Aggregation
Query
Client
Network/Cluster
CRATE Dashboard Python JavaDB-API
SQLAlchemyCRATE Shell
ES native
Transport
FB Presto SQL Parser
Query planner
Bulk import/export
BLOB streaming
Distributed SQL
ES Transport protocol
ES Discovery and state
Lucene BLOB storageES
CRATE DATA – Module overview
3rd party Open
Source Module
s
CRATE
BLOB streaming support
Netty
ES Scatter/Gather
Distributed reduce Data transformation and reindex support
ES Sharding
Ruby
![Page 12: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/12.jpg)
S T A R T A C L U S T E R I N 1 M I N
H T T P S : / / C R A T E . I O
![Page 13: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/13.jpg)
BLOB Storage
Distributed Accurate Aggregations
Partitioned Tables
Import/Export
Update by Query
Insert by Query
Integrated Admin-UI
How is Crate Data different than Elasticsearch?
![Page 14: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/14.jpg)
![Page 16: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/16.jpg)
![Page 17: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/17.jpg)
Demo Video
![Page 18: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/18.jpg)
http://bigdatanerd.files.wordpress.com/2011/12/cap-theorem.jpg
• Basically Available - you always get an response
• Soft State - it’s not consistent all the time.
• Eventually Consistent - it becomes consistent at a later point in time
BASE & CAP
![Page 19: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/19.jpg)
![Page 20: SQL for Elasticsearch](https://reader033.vdocuments.mx/reader033/viewer/2022051209/547e47385806b5b35e8b46a8/html5/thumbnails/20.jpg)