active learning and the importance of feedback in sampling
DESCRIPTION
Active Learning and the Importance of Feedback in Sampling. Rui Castro Rebecca Willett and Robert Nowak. Motivation – “twenty questions”. Goal: Accurately “learn” a concept, as fast as possible, by strategically focusing in regions of interest. Active Sampling in Regression. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/1.jpg)
Active Learningand the Importance of Feedback in Sampling
Rui CastroRebecca Willett and Robert Nowak
![Page 2: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/2.jpg)
Motivation – “twenty questions”
Goal: Accurately “learn” a concept, as fast as possible, by strategically focusing in regions of interest
![Page 3: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/3.jpg)
Learning by asking carefully chosen questions, constructed using information gleaned from previous observations
Active Sampling in Regression
![Page 4: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/4.jpg)
Sample locations are chosen a priori, before any observations are made
Passive Sampling
![Page 5: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/5.jpg)
Sample locations are chosen as a function of previous observations
Active Sampling
![Page 6: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/6.jpg)
Problem Formulation
![Page 7: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/7.jpg)
Passive vs. Active
Passive Sampling:
Active Sampling:
![Page 8: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/8.jpg)
Estimation and Sampling Strategies
Goal:
The estimator :
The sampling strategy :
![Page 9: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/9.jpg)
Classical Smoothness Spaces
Functions with homogeneous complexity over the entire domain
- Hölder smooth function class
![Page 10: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/10.jpg)
Smooth Functions – minimax lower bound
Theorem (Castro, RW, Nowak ’05)
The performance one can achieve with active learning is the same achievable with passive learning!!!
![Page 11: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/11.jpg)
Inhomogeneous Functions
Homogenous functionsspread-out complexity
Inhomogeneous functionslocalized complexity
The relevant features of inhomogeneous functions are very localized in space, making active sampling promising
![Page 12: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/12.jpg)
Piecewise Constant Functions – d≥2
best possible rate
![Page 13: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/13.jpg)
Passive Learning in the PC Class
Estimation using Recursive Dyadic Partitions (RDP)
Prune the partition, adapting to the dataRecursively divide the domain into hypercubesDecorate each partition set with a constantDistribute sample points uniformly over [0,1]d
![Page 14: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/14.jpg)
RDP-based Algorithm
Choose an RDP that fits the data well, but it is not overly complicated
empirical riskmeasures fit of the data Complexity penalty
This estimator can be computed efficiently using a tree-pruning algorithm.
![Page 15: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/15.jpg)
Error BoundsOracle bounding techniques, akin to the work of Barron’91, can be used to upper bound the performance of our estimator
approximation error complexity penalty
balancing the two terms
![Page 16: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/16.jpg)
Active Sampling in the PC class
Active Sampling Key: learn the location of the boundary
Use Recursive Dyadic Partitions to find the boundary
![Page 17: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/17.jpg)
Active Sampling in the PC Class
Stage 1: “Oversample” at coarse resolution
• n/2 samples uniformly distributed
• Limit the resolution: many more samples than cells
• biased, but very low variance result(high approximation error, but low estimation error)
“boundary zone” is reliably detected
![Page 18: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/18.jpg)
Active Sampling in the PC Class
Stage 2: Critically sample in boundary zone
• n/2 samples uniformly distributed within boundary zone
• construct fine partition around boundary
• prune partition according to standard multiscale methods
high resolution estimate of boundary
![Page 19: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/19.jpg)
Main Theorem
* Cusp-free boundaries cannot behave like the graph of |x|1/2 at the origin, but milder “kinks” like |x| at 0 are allowable.
Main Theorem (Castro ’05):
*
![Page 20: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/20.jpg)
Sketch of the Proof - Approach
![Page 21: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/21.jpg)
Controlling the Bias
Not a problem after shift
Potential Problem Area
Cells intersecting the boundary may be pruned if ‘aligned’ with cell edge
Solution:• Repeat Stage 1 d-times,
using d slightly offset partitions
• Small cells remaining in any of the d+1 partitions are passed on to Stage 2
![Page 22: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/22.jpg)
Iterating the approach yields a L-step method
Compare with minimax lower bound:
Multi-Stage Approach
![Page 23: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/23.jpg)
Passive Sampling:
Learning PC Functions - Summary
Active Sampling:
This rates are nearly achieved using RDP-based estimators, that are easily implemented and have low computational complexity.
![Page 24: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/24.jpg)
Spatially adaptive estimators based on “sparse” model selection (e.g., wavelet thresholding) may provide automatic mechanisms for guiding active learning processes
Instead of choosing “where-to-sample” one can also choose “where-to-compute” to actively reduce computation.
Can active learning provably work in even more realistic situations and under little or no prior assumptions ?
Spatial Adaptivity and Active Learning
![Page 25: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/25.jpg)
![Page 26: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/26.jpg)
![Page 27: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/27.jpg)
Piecewise Constant Functions – d =1
Consider first the simplest non-homogenous function class
step function
This is a parametric class
![Page 28: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/28.jpg)
Passive Sampling
Distribute sample points uniformly over [0,1] and use a maximum likelihood estimator
![Page 29: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/29.jpg)
Active Sampling
![Page 30: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/30.jpg)
Learning Rates – d =1
Passive Sampling:
Active Sampling:
(Burnashev & Zigangirov ’74)
![Page 31: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/31.jpg)
Sketch of the Proof - Stage 1
Intuition tells us that this should be the error we experience away from the boundary
Error due to approximationof the boundary regions estimation error
![Page 32: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/32.jpg)
Sketch of the Proof - Stage 1
Key: Limit the resolution of the RDPs
1/k
This is the performance away from the boundary
1/k
![Page 33: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/33.jpg)
Sketch of the Proof - Stage 1
Are we finding more than the boundary?
Lemma:
At least we are not detecting too many areas outside the boundary.
![Page 34: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/34.jpg)
Sketch of the Proof - Stage 2
n/2 more samples distributed uniformly over the boundary
Total error contribution from boundary zone:
![Page 35: Active Learning and the Importance of Feedback in Sampling](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5681584e550346895dc5ab32/html5/thumbnails/35.jpg)
Sketch of the Proof – Overall ErrorError away from the boundary
Balancing the two errors yields
Error in the boundary region