® © 2015 inmar®, inc. confidential not to be reproduced or distributed without written permission...
TRANSCRIPT
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Why We Chose Hadoop
1
5/26/2015
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Who Am I
I am the Director of Platform Engineering at Inmar. I have 19 years of IT experience; 17 years of which has been SQL Server focused starting with version 6.5 and the past 3 years have been focused on Big Data, Hadoop and NoSQL. I have worked in many industries including manufacturing, distribution and logistics, financial services, IT outsourcing and have worked for several fortune 500 companies.
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Agenda
• Who is Inmar• Big Data Decision• Evaluation Process• Current architecture• Regulated Data• The desire for Real Time• Lamda
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
• Privately-held company with Team Support Center in Winston-Salem, NC
• 4,200 employees• 29 facilities across North America• Clients in 11 countries• 34 years of experience working with retailers,
manufacturers and health systems• 25 clients in Fortune 100
We manage over $44 billion a year in commerce value
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
To Put Those Numbers in Perspective…
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
We have all this data, now what?
Analytics• Pick a platform• Hire the expertise
needed for insight• Build products
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The platform
• What we have• SQL Server• MySQL• PostgreSQL• iSeries• Informix• Ingres
• What we tried in the past• SQL Server• Analysis Services• Cognos
• What we looked at• Netezza• Greenplum• SQL Server PDW• Hadoop
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The choice
• Netezza• Really expensive
• Greenplum• Really expensive• Product roadmap was very murky
• SQL Server PDW• Really expensive• Sales guy really annoyed me
• Hadoop• Pricing looked really good• We could make a minimal investment and grow as needed
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The evaluation, engine
• Who we looked at• Cloudera
• Training was expensive with no breaks• Closed source manager
• Greenplum?• Good initial price for license• Closed source• All over the place on strategy
• MapR• Closed source drivers
• Hortonworks• Completely open sourced
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The evaluation, hardware
• We are a VMWare, Cisco UCS shop• Cisco thought we were a slam dunk
• The storage density was low• The price was high
• Dell• Pizza boxes were relatively cheap, ~$4500• Came with 3 year warranty • 64GB RAM• 2 socket, 4 cores per socket• 4 x 3TB drives
• We used VM Guests for primary and secondary name nodes• 10 physical data nodes, ~$45,000
• 640GB RAM• 80 cores• 120 TB of storage
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The evaluation, encryption
• Who we looked at• Gazzang
• Encrypted at a “Volume” level• Did not provide format preserving encryption
• DataGuise• Didn’t return most calls
• Voltage• Provided format preserving encryption
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Regulated Data
• ePHI• Any access to non-obfuscated data must be logged• Access to non-obfuscated data should be restricted to TPO• For Analytics
• Some scenarios need fully obfuscated data• Some scenarios require the ability to reverse the obfuscation
• PII• No real US laws dictating treatment of PII, yet• We believe there will be• We expect the laws to begin to closely mirror HIPAA• Might as well treat it that way now
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
The desire for RealTime
• Competitive Advantage• Faster response• $$$
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Lambda
®
© 2015 Inmar®, Inc. CONFIDENTIALNot to be reproduced or distributed without written permission from Inmar
Thank You