big data - university of babylonbig data: huge amount of data fb clicks, likes, comments...

24
Big Data 3/17/2019 1

Upload: others

Post on 13-Mar-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Big Data

3/17/2019

1

Page 2: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

3/17/2019

2

What is Data?

Page 3: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

What is Big Data?

3/17/2019

3

Page 4: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Another example

3/17/2019

4

Page 5: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

How does they use our Data?

3/17/2019

5

Page 6: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Example

3/17/2019

6

First problem is to get space 300TB on the disk

Page 7: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cont..

3/17/2019

7

Second Problem is processing this big data

Page 8: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cont..

3/17/2019

8

In single PC

Page 9: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cont..

3/17/2019

9

Page 10: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Hadoop

3/17/2019

10

Page 11: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

3/17/2019

11

Vertical and Horizontal Scaling

Page 12: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Vertical and Horizontal Scaling

3/17/2019

12

Page 13: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Difference between Hadoop & SQL

3/17/2019

13

Page 14: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cont…

3/17/2019

14

Page 15: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cont..

3/17/2019

15

Page 16: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

3/17/2019

16

Some basic Terminology

Page 17: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Cluster…

3/17/2019

17

Page 18: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Block … in Hadoop 64MB

3/17/2019

18

Page 19: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

3/17/2019

19

Page 20: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Block Size..

3/17/2019

20

Page 21: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Example…

3/17/2019

21

Page 22: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

Split Data…

3/17/2019

22

Complete Record

Page 23: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

3/17/2019

23

Hadoop

Page 24: Big Data - University of BabylonBig data: Huge amount of Data FB clicks, likes, comments Sensors(whether forcasting sensors, traffic sensors) Google(Rs 6) Search Engine enrique songs,

THANK YOU

3/17/2019

24