tpl dataflow

of 19/19
Dmitri Nesteruk [email protected]

Post on 24-May-2015




7 download

Embed Size (px)


Slides from my talk on TPL DataFlow given at the 33rd meeting of the user group.


2. A publishes an event B subscribes to the event If lifetime of B < lifetime of A We can get a memory leakWeak event pattern 3. Library for composing async/event-based programs Event subscription wrapped as IDisposable Lets you manually unsubscribe Unsubscription automatic when source runs out of data to supply 4. Declarative subscriptions Pub/sub mechanismObjects not aware of each other Typically built on top of a DI container Each component references the broker 5. Erlang F# MailboxProcessor Axum (dead) Task Parallel Library (TPL) DataFlow A library approach 6. Fine-grained Data-level (SIMD) not in .NET AutoparallelizationCoarse-grained Using threads Weird synchronization primitives (e.g. Pulse&Wait) TPL DataFlow 7. A directed graph of data Message passing Buffering Parallel execution Dependency management 8. Pipeline readprocessNetwork readprocesswriteprocess collateprepare processwrite 9. InputPropagateCalculate indicesOutput Render to screen Save to DBRead stock market data FilterSave to DB 10. ISourceBlock Source of data; readITargetBlock Where you write the data toIPropagatorBlock Propagator; performs data processing/filteringAs with Rx, ready-made implementations provided 11. TransformBlock Transforms data Also: TransformManyBlockActionBlock Processes the data 12. Post() Sends a piece of data to the block for processingLinkTo () Links a block with the next block in the graph Takes an optional predicate Returns an IdisposableCancelationTokenSource if you change your mind 13. Basic block processing 14. BatchBlock Combines sets of blocks into an arrayJoinBlock and Collects 2 or 3 inputs into a TupleBatchJoinBlock Merger of the above Collects Tuple,> 15. Provides a data storage buffer Can be used synchronously block.Post() block.Receive()or asynchronously await block.SendAsync() Await block.ReceiveAsync() 16. Grouping Blocks 17. Not a competitor to Rx or TPL itself Course-grained parallelization/sequencing mechanism Cheap multithreading 18. Questions? 19. .NET 4 2014,