TEMPORAL DATA AND REAL-TIME ALGORITHMS
AJ Jicha - PresenterRyan Jicha - PresenterIan Kaufer - Slide MakerRoy Zacharias - Slide Maker
Frontiers in Massive Data Analysis Chapter 4, Pages 37-41
Group 3
Agenda
Topic Overview
Data Acquisition
Processing, Representation and Inference
System and Hardware
Challenges
Topic Overview
Temporal data - data which depends on time
Advertising
Google Maps: Imaging & mapping with real-time traffic
folding@home: Protein folding research
Cybersecurity (Security Information and Event Management Systems)
Shift in computing environment
Distributed computing
Data Acquisition
Various sources of data Different locations/destinations
Processing requirements based on types of data
Scheduling theories: Hard real-time
Firm real-time
Soft real-time
Bounded-tardiness
Processing High-speed data streams may exceed processing capacity
Algorithms can be used to guess the missed data
Representation Coding vs sketching
Inference
Algorithms used to guess answers based on real-time data
Processing, Representation, Inference
System and Hardware
Distributed file systems are necessary Google’s file system (GFS), which is proprietary
Large quantity of data-acquisition machines to funnel ingest to processors
Numerous engineers for system support
Major Challenges
Algorithm design for massively distributed data that can adapt over time
Algorithms that work on many platforms
Distributed real-time acquisition, storage, transmission
Consistency
Infrastructure – Systems, Hardware, & Software
Summary
Data acquisition Processing Representatio
n Inferencing
