temporal data and real-time algorithms aj jicha - presenter ryan jicha - presenter ian kaufer -...

TEMPORAL DATA AND REAL-TIME ALGORITHMS

AJ Jicha - PresenterRyan Jicha - PresenterIan Kaufer - Slide MakerRoy Zacharias - Slide Maker

Frontiers in Massive Data Analysis Chapter 4, Pages 37-41

Group 3

Agenda

Topic Overview

Data Acquisition

Processing, Representation and Inference

System and Hardware

Challenges

Topic Overview

Temporal data - data which depends on time

Advertising

Google Maps: Imaging & mapping with real-time traffic

folding@home: Protein folding research

Cybersecurity (Security Information and Event Management Systems)

Shift in computing environment

Distributed computing

Data Acquisition

Various sources of data Different locations/destinations

Processing requirements based on types of data

Scheduling theories: Hard real-time

Firm real-time

Soft real-time

Bounded-tardiness

Processing High-speed data streams may exceed processing capacity

Algorithms can be used to guess the missed data

Representation Coding vs sketching

Inference

Algorithms used to guess answers based on real-time data

Processing, Representation, Inference

System and Hardware

Distributed file systems are necessary Google’s file system (GFS), which is proprietary

Large quantity of data-acquisition machines to funnel ingest to processors

Numerous engineers for system support

Major Challenges

Algorithm design for massively distributed data that can adapt over time

Algorithms that work on many platforms

Distributed real-time acquisition, storage, transmission

Consistency

Infrastructure – Systems, Hardware, & Software

Summary

Data acquisition Processing Representatio

n Inferencing