BIG DATABY : PUJA SINGH (801)SANGHAMITRA BAL (819)SURABHI SINHA(844)

ST. XAVIER’S COLLEGE,RANCHI

What is Big Data?• The buzz word.

• The misnomer .

• No Single Standard Definition

“Big Data” is data whose scale, diversity, and Complexity require new architecture, techniques, algorithms, and analytics to manage it and extract value and hidden knowledge from it...

Characteristics of Big Data : 3 V’s

Characteristics of Big Data : Volume

Facts : • IBM estimates 2.5 quintillion

bytes of data are generated each day.

• 90% of the data in the world is less than two years old.

Some more facts :

• 230 millions tweets a day.

• 24 petabytes processed by Google everyday.

• 29 petabytes used by National Security Agency.

• CERN’s Large Hydron Collider (LHC) generates 15 PB a year.

• NASA’s EOS program used 4.2 petabytes (2010)

Maximilien Brice, © CERN

Characteristics of Big Data : Variety

Various Formats of Data :• Black Box Data• Social Media Data• Stock Exchange

Data• Power Grid Data• Transport Data• Search Engine Data

The three types of Data :

Structured data: Relational data. Semi Structured data: XML data. Unstructured data: Word, PDF, Text, Media Logs.

Measure of value of Data

1. Timely2. Accessible3. Holistic4. Trustworthy5. Relevant6. Secure7. Authoritive8. Actionable

Characteristics of Big Data : Velocity

• Data is begin generated fast and need to be processed fast :

Online Data Analytics.• Performing analytics against volume and

variety of Data while it is still in motion.• Late decisions missing opportunities

12

Big Data: 3V’s

13

Some Make it 4V’s

Benefits of Big Data It creates transparency to increase efficiency. It allows for better analysis of employee and

systems performances. It can replace and support human decision

making with automated algorithms. It creates innovative business models, products

and services.

Key enablers for the growth of "Big Data" are:

◦ Increase of storage capacities ◦ Increase of processing power

◦ Availability of data

Big Data Challenges Capturing data Curation Storage Searching Sharing Transfer Analysis Presentation

Concerns and LimitationsPrivacy and Security Issues : Significant opportunity for malicious data input

and inadequate data validation. User authentication and access to data from

multiple locations may not be sufficiently controlled.

Big Data has the potential for significant negative impacts that may be impossible to avoid.

Concerns about the motives of government and corporations.

The rich will profit from big data and the poor will not.

Humans will still be most capable of making decisions using Big Data. Statistics can still lie.

Taming Big Data

Following practices must be adopted to work with Big Data :

i. Availabilityii. Managementiii. Disaster Recoveryiv. Provisioningv. Optimizationvi. Backup & Restorevii. Securityviii.Governanceix. Auditingx. Replicationxi. Virtualizationxii. Archiving

Big Data Technology

Big Data Solutions

Google’s Solution Google solved the problem of processing big data through a single database bottleneck, using an algorithm called MapReduce. This algorithm divides the task into small parts and assigns them to many computers, and collects the results from them which when integrated, form the result dataset.

Hadoop

It is an Open Source Project developed by Doug Cutting. Hadoop is used to develop applications that could perform complete statistical analysis on huge amounts of data.

THANK YOU