hug bigtop
Post on 05-Apr-2018
243 Views
Preview:
TRANSCRIPT
-
7/31/2019 HUG Bigtop
1/37
What's stacking?
Or:How I Learned to Stop Worrying
and Love the BigTop
Dr. Konstantin BoudnikApache Hadoop committer since 2009
Co-author of iTest (BigTop's kernel)
Senior architect @Karmaspherecos@apache.org
-
7/31/2019 HUG Bigtop
2/37
Distributed under CC3.0https://creativecommons.org/licenses/by/3.0/
-
7/31/2019 HUG Bigtop
3/37
Disclaimer!
All characters appearing in this work are fictitious. Anyresemblance to real creatures, living or dead, is purely
coincidental
-
7/31/2019 HUG Bigtop
4/37
Can you see the forrest behind the trees?
-
7/31/2019 HUG Bigtop
5/37
Not a technical talk...
What's the real intrinsic value of BigTopWhat is our philosophical proposition
-
7/31/2019 HUG Bigtop
6/37
Why Hadoop data stack is important?
-
7/31/2019 HUG Bigtop
7/37
The elephant is yesterday's news...An article over at Forbes:
...it is not a database, it is not an analyticsenvironment, and it is not a visualization tool.
By itself, it is not a solution for helpingbusinesses make better decisions.Some self-proclaimed hot-shot dude
-
7/31/2019 HUG Bigtop
8/37
Sorry, I am allergic to bullshit...
-
7/31/2019 HUG Bigtop
9/37
There are limits, all right...
Maximum size of the file system High availability Recoverability
-
7/31/2019 HUG Bigtop
10/37
But there is a lot to pick from...
Do you want to experiment with CEPH
Cassandra Ganglia Puppet
-
7/31/2019 HUG Bigtop
11/37
Why I am wasting your time?
You are an ASP and need to ship a stack
-
7/31/2019 HUG Bigtop
12/37
Why I am wasting your time?
You are an ASP and need to ship a stack You need a higher level abstraction for MR
-
7/31/2019 HUG Bigtop
13/37
Why I am wasting your time?
You are an ASP and need to ship a stack You need a higher level abstraction for MR
You are in need of virtual deployment
-
7/31/2019 HUG Bigtop
14/37
Why I am wasting your time?
You are an ASP and need to ship a stack You need a higher level abstraction for MR
You are in need of virtual deployment
Hey!
I am gonna just pick up a distro from B or S 1)
1) B - Big Ego DistributorS - Small Ego Distributor
-
7/31/2019 HUG Bigtop
15/37
Yeah, right! (case study)
Commercial vendor BS packs Hive 0.7.1 intoa release
-
7/31/2019 HUG Bigtop
16/37
We know how it is full of it
-
7/31/2019 HUG Bigtop
17/37
Good news!
Our next release comes with Hive 0.8.1BS' VP of Product
-
7/31/2019 HUG Bigtop
18/37
But may be they just forgot this?..
public int executeUpdate(String sql, String[]columnNames) throws SQLException {
throw new SQLException("Method notsupported");
}
-
7/31/2019 HUG Bigtop
19/37
So, no worries! We can deliver!
-
7/31/2019 HUG Bigtop
20/37
But can they?
-
7/31/2019 HUG Bigtop
21/37
Do you REALLY have to be at mercy
of a distribution vendors?
-
7/31/2019 HUG Bigtop
22/37
I have no resource to do it myself
DIY: and there's always but...
-
7/31/2019 HUG Bigtop
23/37
Maybe ASP or ISV business isn't for you?
-
7/31/2019 HUG Bigtop
24/37
Or maybe BigTop stack: take it and go...
Modify a stack BOM Build Deploy Configure with Puppet (included) Test (scenarios are provided or easy to add) Grab a virtual appliance if short of hardware
-
7/31/2019 HUG Bigtop
25/37
Hit a problem?
Just ri nse and repeat (C)
-
7/31/2019 HUG Bigtop
26/37
Many thanks to the good folks from Hadoop community!
Many components Gazillions of versions
Lot of patches if you like it hot and dirty
-
7/31/2019 HUG Bigtop
27/37
Where is the philosophy?
WAIT!
-
7/31/2019 HUG Bigtop
28/37
Competition is great! Competition enabling open platform Diversity of stacks No more vendors lock-ups Don't be afraid to try Be your own Sprechstallmeister
-
7/31/2019 HUG Bigtop
29/37
May be something wrong with this?
-
7/31/2019 HUG Bigtop
30/37
News And Updates
-
7/31/2019 HUG Bigtop
31/37
5 releases of BigTop!
Three are outTwo more in the work
-
7/31/2019 HUG Bigtop
32/37
Strong community and growing!
20 committers and many contributorsWe are looking for more!
-
7/31/2019 HUG Bigtop
33/37
And sorry: no more binaries for you
BigTop won't be publishing conveniencebinary artifacts anymore
-
7/31/2019 HUG Bigtop
34/37
Might as well use this expertise:http://www.magnatempusgroup.net
-
7/31/2019 HUG Bigtop
35/37
Come and join us!
Powered By Bigtop:ClouderaEMC
HortonWorksMagnaTempusGroupTrend Micro
https://incubator.apache.org/bigtop/
https://blogs.apache.org/bigtop/
-
7/31/2019 HUG Bigtop
36/37
May be one day...
-
7/31/2019 HUG Bigtop
37/37
Q & A
top related