E81 CSE 532S: Advanced Multi-Paradigm Software Development

Chris GillDepartment of Computer Science and Engineering

Washington University in St. Louiscdgill@cs.wustl.edu

C++11 Lock-free Data Structures

Lock-Free and Wait-Free Semantics

• Lock-free behavior never blocks (but may live-lock)– Suspension of one thread doesn’t impede others’ progress– Tries to do something, if cannot just tries again– E.g., while(head.compare_exchange_weak(n->next,n));

• Wait-free behavior never starves a thread– Progress of each is guaranteed (bounded number of retries)

• Lock-free data structures try for maximum concurrency– E.g., ensuring some thread makes progress at every step– May not be strictly wait-free but that’s something to aim for

• Watch out for performance costs in practice– E.g., atomic operations are slower, may not be worth it– Some platforms may not relax memory consistency well

Lock-Free Stack Case Study

• Even simplest version of push requires careful design– Allocate/initialize, then swap pointers via atomic operations

• Need to deal with memory reclamation– Three strategies: thread counts, hazard pointers, ref counts

• Memory models offer potential performance gains– E.g., if relaxed or acquire/release consistency is in fact

weaker on the particular platform for which you’re developing– Need to profile that, e.g., as we did in the previous studio

• Resulting lock free stack design is a good approach– E.g., Listing 7.12 in [Williams]– Please feel free to use (with appropriate citation comments)

in your labs (we’ll code this up in the studio exercises)

Lock-Free Queue Case Study

• Contention differs in lock-free queue vs. stack– Enqueue/dequeue contention depends on how many nodes

are in the queue, whereas push/pop contend unless empty– Synchronization needs (and thus design) are different

• Application use cases also come into play– E.g., single-producer single-consumer queue is much

simpler and may be all that is needed in some cases– Service configuration, template meta-programming, other

approaches can enforce necessary properties of its use

• Multi-thread-safe enqueue and dequeue operations– Modifications (to e.g., reference-counting) may be needed– May need to use work-stealing to be lock free (!)

Lock Free Design Guidelines

• Prototype data structures using sequential consistency– Then analyze and test thread-safety thoroughly– Then look for meaningful opportunities to relax consistency

• Use a lock-free memory reclamation scheme– Count threads and then delete when quiescent– Use hazard pointers to track threads accesses to an object– Reference count and delete in a thread-safe way– Detach garbage and delegate deletion to another thread

• Watch out for the ABA problem– E.g., with coupled variables, pop-push-pop issues

• Identify busy waiting, then steal or delegate work– E.g., if thread would be blocked, “help it over the fence”