Poisson Processes and Maximum Likelihood Estimator for Cache Replacement

A Preliminary AttemptECEn 670 Semester ProjectWei DangJacob FroggetPoisson Processes and Maximum Likelihood Estimator for Cache ReplacementOutlineMotivationCache replacement algorithmsPoisson processes in webserver-based cachePoisson processes in microprocessor-based cacheMaximum Likelihood estimatorPredicting proceduresSimulation and evaluationResults compared to LRUFuture workMotivationMemory-processor speed gap is getting larger

One of the SolutionsMemory hierarchyL1/L2/L3closer to chip, less latency, but smaller sizeCache replacementCache is fast but limited in sizeconflictGood replacement policies needed on contentionCache Replacement AlgorithmsCache conflict in a 4-way associative cache

Set indexCommon Cache Replacement AlgorithmsRandomLeast Recently Used (LRU, most widely used)Least Frequently UsedFIFOPoisson Processes for webserver cacheArrival time of queries to a webserver can be modeled as a Poisson Process

Interpretation: the probability of having k queries up to some point of timeAssumptionArrivals of queries are independent of each otherNot always true but valid for most cases

Inspired by this7Poisson Processes for microprocessor cacheIndependence assumption invalidReferences to cache are highly correlated (especially to data cache)Temporal localitySpatial localityPoisson Processes for microprocessor cache cont.One Poisson process for each block within a set

Set indexAn example set from a 4-way associative cache

Poisson Processes for microprocessor cache cont.Correlation between these four random processes

Set index

Local counters to each blockOne global counter n for each set

Maximum Likelihood Estimator

The estimate is the arithmetic mean of

Predicting ProceduresInitially each block has Given a previous calculated for block , the estimated is calculated as:Probabilities for replacement is then

Replace block with lowest probabilityChoose randomly on equal probabilities

is used instead of because we wanted to calculate the probability of block being accessed again given current access count of . Having computed the probabilities for all blocks within the set, we compare them and replace the block with the lowest probability if there is conflict.12Simulation and EvaluationSimulator: MyDLX cache simulator from EE628Metrics: miss rate for Instruction-cache and Data-CacheVarious associativitiesFive benchmarksCompared to LRUResults

Results

Results

Results

Results

Results

ResultsResults not encouragingSometimes 0% miss rate for both algorithms (might be due the inherent characteristics of benchmarks)Statistical approach worse than LRU for most casesGetting worse for higher associativity (more blocks to predict)Analysis of deficiencies of our modelIndependence model may be inaccurate (even accesses to the same block within a set may not be independent)Local counter is reset to 0 on eviction (history eliminated)Future work and challengesMore accurate model with more correlation parameters for each Poisson processImplementation complexity (hardware expensive; LRU is already expensive at high associativity)May be implemented as a software cache as a supplement to hardware cache.The EndThank you!Questions?