f inding patterns in temporal data

FINDING PATTERNS IN TEMPORAL DATAKRIST WONGSUPHASAWATTAOWEI DAVID WANGCATHERINE PLAISANTBEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND

27th HCIL Symposium

May 27, 2010

FINDING PATTERNS IN TEMPORAL DATAKRIST WONGSUPHASAWATTAOWEI DAVID WANGCATHERINE PLAISANTBEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND

27th HCIL Symposium

May 27, 2010

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:36 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/14/2008 06:19 Exit

Time

EmergencyICU

FloorExit

TEMPORAL CATEGORICAL DATA• A type of time series

04/26/2010 10:00 31.0304/26/2010 10:15 31.0104/26/2010 10:30 31.0204/26/2010 10:45 31.0804/26/2010 11:00 31.16

EventCategory

Stock: MicrosoftNumerical

Arrival

Event

TEMPORAL CATEGORICAL DATA

Electronic Health Records: symptoms, treatment, lab testTraffic incident logs: arrival/departure time of each unitStudent records: course, paper, proposal, defense, etc.

Others: web logs, usability study logs, etc.

10+ years work on temporal visualization(mostly on Electronic Health Records)

LIFELINES

SINGLE RECORD

[Plaisant et al. 1998]http://www.cs.umd.edu/hcil/lifelines

LifeLines – Single Patient

working with physicians at WASHINGTON HOSPITAL CENTER

EXAMPLE DATA• Patient transfers

ARRIVAL Arrive the hospitalEMERGENCY Emergency roomICU Intensive Care UnitINTERMEDIATE Intermediate Medical

CareFLOOR Normal roomEXIT-ALIVE Leave the hospital aliveEXIT-DEAD Leave the hospital dead

TASKS

within 2 days

ICU Floor ICU

• Example: Finding “Bounce backs”

LIFELINES 2

RECORDRECORDRECORD

RECORD

RECORD

[Wang et al. 2008, 2009]http://www.cs.umd.edu/hcil/lifelines2

LifeLines2 – Search and Visualize

ARF (Align-Rank-Filter)Framework

Temporal Summary

Multiple Records

ALIGNMENT• Sentinel events as reference points

Time

Patient #45851737 ArrivalEmergency

ICUFloor

ExitPatient #43244997 Arrival

EmergencyICU

FloorExit

June July August

ALIGNMENT (2)• Time shifting

Time

Patient #45851737 AdmitEmergency

ICUFloor

ExitPatient #43244997 Admit

EmergencyICU

FloorExit

0 1 M 2 M

SIMILAN

RECORDRECORDRECORD

RECORD

RECORD

[Wongsuphasawat & Shneiderman 2009]http://www.cs.umd.edu/hcil/similan

Similan – Search by Similarity

Similan – Search by Similarity

FINDING “BOUNCE BACKS”Before After

• Much faster to specify new query• Visualizing the results gives better

understanding

USER STUDIES: SEARCH

ExactMUST have A, B, C

Record#1

Record#2

Record#3

moresimilar

Similarity-basedSHOULD have A, B, C

SimilanLifeLines2

Query

Record#2

Record#1

Record#3

Query

USER STUDIES: SEARCH

ExactMUST have A, B, C

Similarity-basedSHOULD have A, B, C

Query

Record#1

Record#2

Record#3

moresimilar

Query

Record#2

Record#1

Record#3

SimilanLifeLines2

1

NEW STUFFNeeds for an overview -> LifeFlow!

TASKS

within 2 days

ICU Floor ICU

• Example: Finding “Bounce backs”

• Other questionsArrival

ICU

?

??

LIFEFLOW

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

AGGREGATEMerge multiple records into

tree

VISUALIZEDisplay the aggregation

AGGREGATE• Aggregate by prefix

#1

#2#3

#4

Example with 4 records

AGGREGATE• Aggregate by prefix

#1

#2#3

#4

VISUALIZE• Inspired by the Icicle tree [Fekete 2004]

Number of files

VISUALIZE (2)• Use horizontal axis to represent time• Video

DEMO – LIFEFLOW When the lines are combined into flow

FUTURE WORK• Comparison

Jan-Mar 2008 April-June 2008

IntermediateICUIntermediateICU

Floor

TAKE-AWAY MESSAGEInformation visualization is a powerful way to explore temporal patterns.

You can work with us on new case studies.

TEMPORAL CATEGORICAL DATA

Electronic Health Records: symptoms, treatment, lab testTraffic incident logs: arrival/departure time of each unitStudent records: course, paper, proposal, defense, etc.

Others: web logs, usability study logs, etc.

EXAMPLE – TRAFFIC INCIDENTS

ACKNOWLEDGEMENTDR. PHUONG HO, DR. MARK SMITH, DAVID

ROSEMANWASHINGTON HOSPITAL CENTER

http://www.whcenter.org

NATIONAL INSTITUTES OF HEALTH (NIH) - GRANT CA147489 http://www.nih.gov

MICHAEL PACK, MICHAEL VANDANIKERCENTER FOR ADVANCED TRANPORTATION

TECHNOLOGY LAB(CATT LAB)

http://www.cattlab.umd.edu

TAKE-AWAY MESSAGEInformation visualization is a powerful way to explore temporal patterns.

You can work with us on new case studies.

More demos this afternoon{kristw, tw7, plaisant, ben}@cs.umd.eduhttp://www.cs.umd.edu/hcil/temporalviz

Q&AQuestions?

{kristw, tw7, plaisant, ben}@cs.umd.edu

http://www.cs.umd.edu/hcil/temporalviz

THANK YOUThank you

BACKUP SLIDESJunkyard...

LIFELINES2• 8 case studies

– Bounce backs– Step ups– BIPAP– Etc.

DR. P

Does not help exploring sequential patternsNeeds a new overview

LifeLines2’s Temporal Summary [Wang et al. 2009]

Continuum’s Histogram [Andre 2007]

USER STUDIES• 8 Extensive case studies• Compared LifeLines2 with Similan

– Learn advantages & disadvantages• Drawing is preferred• Clear cut off points is needed

• Working on improvements– Flexible temporal search

SIMILAN• Compared with LifeLines2 in an

experiment– Learn advantages & disadvantages– Drawing is preferred– No clear cut off points

• Working on improvements– Flexible temporal search

LIFEFLOWAGGREGATE

VISUALIZE

Merge multiple records into tree

Display the tree

APPROACHESExact SearchMUST have A, B, C

Similarity-based SearchSHOULD have A, B, C

Query

Record#1

Record#2

Record#3

moresimilar

Query

Record#1

Record#2

Record#3

MOTIVATION

RESEARCH QUESTIONS

RESEARCHQUESTION#1

PRELIM. + PROPOSED WORK CONCLUSION

RESEARCHQUESTION#2

PRELIM. + PROPOSED WORK

EXPECTED CONTRIBUTIONS1. Design of visual representations, user

interfaces and interaction techniques2. Algorithms for flexible temporal

search3. Evaluation results4. Open new directions for exploring

temporal categorical data

NEEDS FOR AN OVERVIEW• We learn

NEEDS Visualize overviewor show summary

Where should I start?

TEMPORAL VISUALIZATIONSBackground and related work

RELATED WORK• Single record

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Visualization

• E.g. LifeLines, MIDGAARD, etc.

RELATED WORK (2)• Multiple records

VisualizationVisualizationVisualizationVisualization

• E.g. LifeLines2, Continuum, ActiviTree, etc.

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

More space please....

INFORMATION VISUALIZATION MANTRAOVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND

RELATED WORK (3)• Multiple records

Visualization

• E.g. LifeLines2, Continuum

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

SEQUENTIAL PATTERNS

within 2 days

ICU Floor ICU

• Examples: “Bounce backs”

Patient #1

Patient #2

Patient #3

Patient #4

DESIGN AN OVERVIEW• Sequential patterns• Scalability vs. Loss of information