Upload File

motivation mining big time-series data on the web …yasuko/talks/16-www...– multi-dimensional...

  • Home
  • Documents
  • Motivation Mining Big Time-series Data on the Web …yasuko/TALKS/16-WWW...– Multi-dimensional time-series data (e.g., IoT device data) – Complex time-stamped events (e.g., web-click
3
Tutorial@WWW'16 © 2016 Sakurai, Matsubara & Faloutsos 1 Mining Big Time-series Data on the Web Yasushi Sakurai (Kumamoto University) Yasuko Matsubara (Kumamoto University) Christos Faloutsos (Carnegie Mellon University) http://www.cs.kumamoto-u.ac.jp/ ~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 1 Kumamoto U CMU CS Roadmap http://www.cs.kumamoto-u.ac.jp/ ~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 2 Part 1 Part 2 Part 3 - Motivation - Similarity search, pattern discovery and summarization - Non-linear modeling and forecasting - Extension of time- series data: tensor analysis Kumamoto U CMU CS Big Data Big time-series data http://www.cs.kumamoto-u.ac.jp/~yasuko/ TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 3 climate Economy Web Social/natural phenomena IoT devices Epidemic Kumamoto U CMU CS Big Data Big time-series data http://www.cs.kumamoto-u.ac.jp/~yasuko/ TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 4 climate Economy Web Social/natural phenomena IoT devices Online user activities - Web clicks, - Online reviews Application - Market analysis Epidemic Kumamoto U CMU CS Big Data Big time-series data http://www.cs.kumamoto-u.ac.jp/~yasuko/ TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 5 climate Economy Web Social/natural phenomena IoT devices IoT data streams - Vibration sensors, acceleration, temperature, etc. Application - Self-driving car - Structural health monitoring - Manufacturing Epidemic Kumamoto U CMU CS Motivation • Given: Big time-series data • Goal: Find important patterns Forecast future social activities • At-work: – Online marketing – Sensor monitoring, anomaly detection – Forecasting future events http://www.cs.kumamoto-u.ac.jp/ ~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 6

Upload: others

Post on 10-Jun-2020

7 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Motivation Mining Big Time-series Data on the Web …yasuko/TALKS/16-WWW...– Multi-dimensional time-series data (e.g., IoT device data) – Complex time-stamped events (e.g., web-click

Tutorial@WWW'16

©2016Sakurai,Matsubara&Faloutsos 1

Kumamoto U CMU CS

Mining Big Time-series Data on the Web

Yasushi Sakurai (Kumamoto University) Yasuko Matsubara (Kumamoto University) Christos Faloutsos (Carnegie Mellon University)

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 1

Kumamoto U CMU CS

Roadmap

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 2

Part1

Part2

Part3

-  Motivation -  Similarity search,

pattern discovery and summarization -  Non-linear modeling

and forecasting -  Extension of time-

series data: tensor analysis

Kumamoto U CMU CS

Big Data

Big time-series data

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 3

climate

EconomyWeb

Social/natural phenomena

IoT devices

Epidemic

Kumamoto U CMU CS

Big Data

Big time-series data

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 4

climate

EconomyWeb

Social/natural phenomena

IoT devices

Online user activities -  Web clicks, -  Online reviews Application -  Market analysis

Epidemic

Kumamoto U CMU CS

Big Data

Big time-series data

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ © 2016 Sakurai, Matsubara & Faloutsos 5

climate

EconomyWeb

Social/natural phenomena

IoT devices

IoT data streams -  Vibration sensors, acceleration,

temperature, etc. Application -  Self-driving car -  Structural health monitoring -  Manufacturing

Epidemic

Kumamoto U CMU CS

Motivation•  Given: Big time-series data •  Goal:

Find important patterns Forecast future social activities •  At-work: – Online marketing – Sensor monitoring, anomaly detection – Forecasting future events

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 6

Page 2: Motivation Mining Big Time-series Data on the Web …yasuko/TALKS/16-WWW...– Multi-dimensional time-series data (e.g., IoT device data) – Complex time-stamped events (e.g., web-click

Tutorial@WWW'16

©2016Sakurai,Matsubara&Faloutsos 2

Kumamoto U CMU CS

Motivation•  Time-series analysis for big data – Web and social networks – IoT data streams – Medical and healthcare records

•  Digital universe growth – 4.4 zettabytes

(4.4 trillion gigabytes) – 44 zettabytes in 2020

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 7

4.4ZB

44ZB

2013 2020TheDIGITALUNIVERSEofOPPORTUNITIES(IDC2014)

Kumamoto U CMU CS

Big Time-series analysis •  Volume and Velocity – High-speed processing for large-scale data – Low memory consumption – Online processing for real-time data management

•  Variety of data types – Multi-dimensional time-series data (e.g., IoT device data) – Complex time-stamped events (e.g., web-click logs) – Time-evolving graph (e.g., social networks)

•  Advanced techniques for big data – Model estimation, summarization – Anomaly detection, forecasting

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 8

Time

Kumamoto U CMU CS

Big Time-series analysis•  Time-series data mining

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 9

Indexing,similaritysearch

Featureextraction

Streammining

Linearmodeling

PCA ICA Model(M)Data(X)

ED,DTWCorrela-on

AR,ARIMA,LDS

StatStreametc…

DFT,DWT,SVD,ICA

Y

X

DTW

Kumamoto U CMU CS

New research directions R1. Automatic mining (no magic numbers!) R2. Non-linear (gray-box) modeling R3. Tensor analysis

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 10

X

≈ + ...NOmagicnumbers!

Kumamoto U CMU CS

(R1) Automatic mining No magic numbers! ... because,

Big data mining: -> we cannot afford human intervention!!

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 11

Manual -  sensitive to the parameter tuning -  long tuning steps (hours, days, …)

Automatic (no magic numbers) - no expert tuning required

Kumamoto U CMU CS(R2) Non-linear (gray-box) modeling

•  Gray-box mining – If we know the equations

•  Non-linear (differential) equations – Epidemic – Biology – Physics, Economics, etc.,

•  Modeling non-linear phenomena – Non-linear analysis for big time-series data

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 12

ImagecourtesyofTinaPhillipsandamenic181atFreeDigitalPhotos.net.

Gray-box

Page 3: Motivation Mining Big Time-series Data on the Web …yasuko/TALKS/16-WWW...– Multi-dimensional time-series data (e.g., IoT device data) – Complex time-stamped events (e.g., web-click

Tutorial@WWW'16

©2016Sakurai,Matsubara&Faloutsos 3

Kumamoto U CMU CS(R3) Large-scale tensor analysis

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 13

Time URL User08-01-12:00 CNN.com Smith

08-02-15:00 YouTube.com Brown

08-02-19:00 CNET.com Smith

08-03-11:00 CNN.com Johnson

… … …

RepresentasMthordertensor(M=3)

URL

user

Time

x

u

v

n

Elementx:#ofevents

e.g.,‘Smith’,‘CNN.com’,‘Aug1,10pm’;21times

•  Time-stamped events – e.g., web clicks

X

Kumamoto U CMU CS

•  Time-series data analysis – Indexing and fast searching – Sequence matching – Clustering – etc.

•  New research directions R1. Automatic mining R2. Non-linear modeling R3. Large-scale tensor analysis

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 14

X ≈ + ...

Time

New research directions

NOmagic#

Kumamoto U CMU CS

•  Time-series data analysis – Indexing and fast searching – Sequence matching – Clustering – etc.

•  New research directions R1. Automatic mining R2. Non-linear modeling R3. Large-scale tensor analysis

http://www.cs.kumamoto-u.ac.jp/~yasuko/TALKS/16-WWW-tut/ ©2016Sakurai,Matsubara&Faloutsos 15

X ≈ + ...

Time

New research directions

NOmagic#

Part2Part3

Part1


Big Data & Security - tuwien.ac.at · Security Challenges • Confidentiality – Cloud storage (e.g. Dropbox) – Authentication (e.g. WhatsApp) – Open data vs. unintended data

Data base structure Reading and data process. (C/L, display) Communication Manipulating (e.g. selecting) Tracking (e.g. sidestick) zyxzyxzyx Direction

Getting the Big (Data) Picture - GOTO Conferencegotocon.com/dl/goto-aar-2014/slides/DeanWampler_and_Eva...• Real Time Search (e.g. SolrCloud) • I have unstructured data I want

Not Your Grandma’s Time-Series Analysisvcpsybxt/Time Series Analysis.pdfWhat is Time Series Analysis? Data from a single individual over at least 50 time periods E.g.: weight before

A vision of the future of food safety...provides modern solutions (e.g. in data capture, real-time data approach, AI etc.). DATA IN A FUTURE 5 Increasing speed of data generation

ABRA Abra Cloud · 2020-01-06 · abra cloud real-time api integrations (e.g., ccc one mitchell) batch data sources (e.g., abse, gts) data lake (amazon s3) message que (amazon sqs)

Recent Trends in Substation Automation and Enterprise Data ... · the Substation Automation System (e.g., PC with ... RTU Real-time SCADA data and controls (any IED’s data) Fault

Data Stream Management Systems · • Real-time delivery, Quality of Service – Regular databases also have QoS requirements, but slower! • E.g. guaranteed average response time

Inextricably Linked: Reproducibility and Productivity in ......Challenges with git in data science ⇢ In data science, it's not natural to commit every time you change anything, e.g

Infoday on 2020 Partnership Calls - GPPQ€¦ · •Showcase real-time data validation within an actual production line & incorporate data integrity strategies based on, e.g., distributed

 · DAVID S, WISE NOVEMBER, ... (e.g. operating and data base systems) which involve real time and concurrency, ... saoaJJê K Tuo pua

CS 425 / ECE 428 Distributed Systems Fall 2020 · 2020. 10. 18. · •Large amounts of data => Need for real-time views of data • Social network trends, e.g., Twitter real-time

Generalized Discrete-Time Equivalent Model for the …motor.ece.iit.edu/data/Generalized Discrete-Time Equivalent Model.pdf · dynamic equivalence (e.g., [8]–[13]). Therefore, the

ITS323: Introduction to Data Communications Contents Data … · time, e.g. police radio Full duplex: both directions at the same time, e.g. telephone 3 ITS323 Data Transmission Terminology

PowerPoint Presentation · To Markets Financial Crime, Horizon Scanning Data And Security ... e.g. real time presentation or ... International Payments — e.g

Rolando Hernandez – Becoming A Knowledge Engineer · Who will build the Apps ... Why Data (Thing) Process (Action) Person Place Time ... e.g. Logical Data Model Ent = Data Entity

Quality Review Record BookQuality Review Record Book 5 Pre-review Preparation (continued) Document Reviewed E.g., SSEF Data Trail E.g., Teachers’ use common planning time to analyze

Overview GOLD Science Data Processing · 2017-02-16 · UCF GOLD Science Data Center Primary science data archive, genera-tion of science data products L1D-L5 Real-time Users* (e.g

Ran TAO [email protected] Missing Spatial Data. Examples Places cannot be reached E.g. Mountainous area Sample points E.g. Air pollution Damage of data E.g

HPE Apollo Systems - Router-Switch.com · 2018. 7. 13. · Different data assets have different value • Mission-critical data must be stored for a long time—e.g., customer data

Duration of measurement, h:min...Downloading Logger Data When should I download logger data? – At least once per week, at the same time each week (e.g., Monday morning) – Any time

Streamsheets - smart applications€¦ · Streamsheets send the data in real-time to the standardized data collection instance (e.g. Apache Kafka Stream or SQL/NOSQL-Database). Real-Time

Temporal data - University of Pittsburghpeople.cs.pitt.edu/~iyad/temp.pdfTemporal data • Sequential data (no explicit time) vs. time series data –Sequential data e.g. : Gene sequences

1 Decomposition Method. 2 Types of Data Time series data: a sequence of observations measured over time (usually at equally spaced intervals, e.g.,

The Role of Geospatial Data and Technologies · Sophisticated geospatial data sources (e.g. real-time weather data) are coming online on a ... Two primary functions of government

Graphing Paired Data Sets Time Series Data set is composed of quantitative entries taken at regular intervals over a period of time. – e.g., The amount

GIS STEERING COMMITTEE - | ddot...• Capture Big Data to identify new correlations (e.g. enable data analytics using aggregated tax info, incidents, permit locations, real-time data,

Modeling Historical AIS Data For Vessel Path Prediction: A ...each AIS trajectory contains both time-varying features (e.g. position, speed, whose values change with time) and static

Explainable Machine Learning Models for Structured Data · Overview •Structured Data •Symbolic Sequences (e.g., DNA, malware) •Numeric Sequences (e.g., time series) •Explainable

Multilevel Discrete-Time Event History AnalysisMultilevel Event History Data Multilevel event history data arise when events are repeatable (e.g. births, partnership dissolution) or

Predicting Travel Time Reliability using Mobile Phone GPS Data€¦ · travel time reliability to a user (e.g. the travel time is predicted to be in the range 10-15 minutes), and

Genomic Medicine Implementation in a High Value, Learning ... · • Clinical laboratory data – Financial data (e.g., billing, payment) – Operational data (e.g., events, scheduling)

Industry Paper: Classification of vessel activity in ... · challenges when applied in streaming data (e.g., limited execution time, memory consumption, data cleaning, etc.).

Varying-Coefficient Panel Data Models with Partially ... · For panel data models, in terms of time dimension, one often encounters three types of regressors: (1) stationary (e.g.,

Chapter 19 Big Longitudinal Data Analysis · 2018-08-27 · The basic concepts in time series analysis include: † The characteristics of (second-order) stationary time series (e.g.,