adaptive cleaning for rfid data streams
DESCRIPTION
Shawn Jeffery Minos Garofalakis Michael Franklin UC Berkeley Intel Research Berkeley UC Berkeley . Adaptive Cleaning for RFID Data Streams. Presented by Willie and Abhishek. Disclaimer: The slides are taken from Jeffrey’s talk at VLDB ‘06. - PowerPoint PPT PresentationTRANSCRIPT
Adaptive Cleaning for Adaptive Cleaning for RFID Data StreamsRFID Data Streams
Presented by Willie and Abhishek
Shawn Jeffery Minos Garofalakis Michael Franklin UC Berkeley Intel Research Berkeley UC Berkeley
Disclaimer: The slides are taken from Jeffrey’s talk at VLDB ‘06
04/22/23
RFID: Radio Frequency RFID: Radio Frequency IDentificationIDentification
04/22/23
RFID data is dirtyRFID data is dirtyShelf 0 Shelf 1
RFIDReaders
StaticTags
Mobile Tags
15ft1.5ft
3ft9ft
3ft
3ft
3ft
A simple experiment:•2 RFID-enabled shelves•10 static tags•5 mobile tags
04/22/23
RFID Data CleaningRFID Data Cleaning
Time
Raw readings
Smoothed output
• RFID data has many dropped readings• Typically, use a smoothing filter to
interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_idBut, how to set the size
of the window?
Smoothing Filter
04/22/23
Window Size for RFID Window Size for RFID SmoothingSmoothing
Fido moving Fido resting
Small windowRealityRaw readings
Large window
Need to balance completeness vs. capturing tag movement
04/22/23
Truly Declarative Truly Declarative SmoothingSmoothing
• Problem: window size non-declarative• Application wants a clean stream
of data• Window size is how to get it
• Solution: adapt the window size in response to data
04/22/23
ItineraryItinerary
• Introduction: RFID data cleaning• A statistical sampling perspective• SMURF
• Per-tag cleaning• Multi-tag cleaning
• Ongoing work• Conclusions
04/22/23
A Statistical Sampling A Statistical Sampling PerspectivePerspective
• Key Insight: RFID data random sample of present tags
• Map RFID smoothing to a sampling experiment
04/22/23
RFID’s Gory DetailsRFID’s Gory Details
Epoch TagID ReadRate0 1 .90 2 .60 3 .3
Tag 1
Tag 2
Tag 3
Tag 4
Antenna & readerTags
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
Read Cycle (Epoch)
(For Alien readers)
Tag List
04/22/23
RFID Smoothing to SamplingRFID Smoothing to Sampling
RFID SamplingRead cycle (epoch) Sample trialReading Single sampleSmoothing window Repeated trialsRead rate Probability of inclusion
(pi)
Now use sampling theory to drive adaptation!
04/22/23
SMURFSMURF• Statistical Smoothing for Unreliable RFID
Data• Adapts window based on statistical
properties• Mechanisms for:
• Per-tag and multi-tag cleaning
Multi-tagCleaning
SMURFPer-tag
Cleaning
raw RFID streams
cleanedcount readings
cleanedper-tag readings
Application(s) Application(s)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: Model and BackgroundModel and Background
• Use a binomial sampling model
Time (epochs)
pi
1
0
Smoothing Window
wi Bernoulli trials
piavg
Si
(Read rate of tag i)
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
• If the tag is there, read it with high probability
Want a large window
pi
1
0
Reading with a low pi
Expand the window
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
Expected epochs needed to read
With probability 1-
Desired window size for tag i
1ln*1
avgi
ip
w
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions• Detect transitions as statistically
significant changes in the data
pi
1
0
Statistically significant difference Flag a transition and
shrink the window
The tag has likely left by this point
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions
# expected readings Is the difference
“statistically significant”?# observed
readings
)1(**2|*||| avgi
avgii
avgiii ppwpwS
04/22/23
SMURF in ActionSMURF in ActionFido moving Fido resting
SMURF
Experiments with real and simulated data show similar results
04/22/23
Multi-tag CleaningMulti-tag Cleaning
• Some applications only need aggregates• E.g., count of items on each shelf Don’t need to track each tag!
• Use statistical mechanisms for both:• Aggregate computation • Window adaptation
04/22/23
Aggregate Aggregate ComputationComputation
• –estimators (Horvitz-Thompson) • Count:
• P[tag i seen in a window of size w]:
Use small windows to capture movementUse the estimator to compensate for lost
readings
wSiwN
1
wavgii p )1(1
04/22/23
Window AdaptationWindow Adaptation
• Upper bound window similar to per-tag
• “Transition” based on variance within subwindows
1ln*1
avgpw
CountNw
Nw’
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
'VarVar2ww NN
04/22/23
Multi-tag ScenarioMulti-tag Scenario
04/22/23
Ongoing Work: Spatial Ongoing Work: Spatial SmoothingSmoothing
• With multiple readers, more complicated
Reinforcement
A? B? A U B? A B?Arbitration
A? C? All are addressed by statistical framework!
U
A
B
C
D
Two rooms, two readers per room
04/22/23
Beyond RFIDBeyond RFID
• -estimator for other aggregates Use SMURF for sensor networks
• Use SMURF in general streaming systems (e.g., TelegraphCQ)
Remove RANGE clause from CQL
Other sensor data
Other streaming data
04/22/23
Related WorkRelated Work
• Commercial RFID middleware• Smoothing filters: need to set smoothing
window• RFID-related work
• Rao et al., StreamClean: complementary• Intel Seattle, HiFi, ESP: static window size
• BBQ, MauveDB• Heavyweight, model-based• SMURF is non-parametric, sampling-based
• Statistical filters (digital signal processing)• Non-linear digital filters inspired SMURF design
04/22/23
ConclusionsConclusions
• Current smoothing filters not adequate• Not declarative!
• SMURF: Declarative smoothing filter• Uses statistical sampling to adapt window size
04/22/23
Thanks!Thanks!
Questions?