how to overcome data silos to achieve big data business transformation
TRANSCRIPT
Agenda
• 4 Vs of Big Data and their impact
• Typical ‘Big Data’ landscape
• Considerations for tools
• How to start talking Big?
• How can Ashnik help?
4 Vs of Big Data equation
• Velocity
• Volume
• Variety
• Veracity
Impact of 4 Vs on technology
• Database
• Scale out architecture
• Data integration and enrichment
• Different shades of grey
Typical Big Data Landscape
Which tools can help you?
• Data Store – Postgres, MongoDB, Elastic, Couchbase, Hadoop
• Data Integration – Pentaho, Foreign Data Wrapper, Logstash
• Data Enrichment – Pentaho, Beats, Logstash
• Data Accuracy – Natural Language Processing (Python, R etc)
Considerations for Data Store
• What data model?• Nature of queries - Correlation, Aggregation or more?• How Big is your Big Data?• Linking or embedding?• Real Time or Batch?• Cost per GB Vs Business Value?
Considerations for Data Integration tools
• What are your data sources? - REST, RDBMS, noSQL, RSS...
• What connectors are available? - API call, ODBC, JDBC...
• Do you need much custom routines? - cleansing, filtering, modelingetc...
• What Data formats? - XML, JSON, Relational etc
Considerations for Data Enrichment tools
• Final and intermediate data formats?
• Do you need to normalize/de-normalize?
• Do you need to add additional info? e.g. timestamp, user, sensor-id etc
• Do you need to change the format? e.g. XML to JSON
Learnings from Early Failures
• Choose the right data-store
• Don’t build another Data warehouse - only larger
• Don’t end up with more Data Silos - only put together in one store
• Data must correlate and make business sense
How to get you talking Big?
• Identifying business problem
• Overcoming data silos
• Integrating and co-relating the data from different sources
• Identify the queries - Analytics
How can Ashnik Help you?
• We partner with different technologies• Vendor-independent consulting
• Bring the experience to your team• Database• Data Modelling• Integration
• End-to-end engagements• Consulting• Services• Training