async and parallel patterns and application design - techdays2013 nl
DESCRIPTION
TechDays2013 NL session on async and parallel programming. Gives an overview of todays relevant .net technologies, examples and tips and tricks. This session will help you to understand and select and use the right async/parallel technology to use in your .net application. ([email protected])TRANSCRIPT
Async and parallel patterns and application design
Arie LeeuwesteijnTechnology Specialist [email protected] (@arieleeuw)
In this Session…• Threads and Tasks• Task Parallel Library• C# 5.0 async await• Concurrent Collection Classes• Reactive Extensions• TPL Dataflow• ASP.NET, MVC, Win8
Objectives• Understanding Async and Parallelism
• Patterns, Frameworks and tools– Purpose– Scenarios– Combinations and Integrations– Real world examples
Why Async and Parallel?• Multicore is the rule
– Keep your cores busy– Used them in an efficient way
• Client– User experience
• Server– Scalability– Handling requests
• Real life is async and parallel
Question• What’s the difference between
Async and Parallel?
• Parallel : Having the work done by multiple workers at the same time
• Async: Use workers efficiently, don’t let them wait for slow actions to complete
History• .NET 1.0– Asynchronous Programming Model, APM
• .NET 2.0– Event-Based Programming Model, EPM
• .NET 4.0– Task-based Programming Model, TAP
• .NET 4.5 – C# 5.0, async, await(*)
(*) Async Targeting Pack for Visual Studio 2012 to target .NET 4.0
Asynchronous Programming Model
WebRequest client = WebRequest.CreateHttp(url);
client.BeginGetResponse(Callback, client); private void Callback(IAsyncResult ar) { WebRequest client =(WebRequest)ar.AsyncState; WebResponse resp = client.EndGetResponse(ar);}
Asynchronous Programming Model
• BeginMethod, EndMethod pattern
• Complex code– Error handling– Object lifetime– Nesting async calls
• Synchronization context
Event-based Programming Modelvar client = new WebClient();
client.DownloadStringCompleted += DownloadCompleted;
private void DownloadCompleted( object sender, DownloadStringCompletedEventArgs e) { string resp = e.Result;}
client.DownloadStringAsync(uri);
Event-based Programming Model• Events and EventHandlers
• Complex code– Error handling– Inconsistent synchronization context
• Possible Memory leaks– += handler keeps reference– Dangling handlers
Tasks• Task and Task<T>– A promise to do something or return a value– Does not say when or how
• Task != Thread– A task is something you want to be done– A thread is a possible worker to perform that task– A task may not even need a thread
Why use Tasks• Simplifies code• One object to access to get the– Result, Status and possible Errors
• Composition– ContinueWith, WhenAll, WhenAny
• Task Cancelation
Creating a Task// Using the Task constructorvar task1 = new Task<int>(() => { Thread.Sleep(1000); return 42; });
task1.Start(); var res = task1.Result;
// Using the TaskFactoryvar task2 = new TaskFactory<int>().StartNew(() => { Thread.Sleep(1000); return 42; });
This is the work to be done
TaskScheduler• Handles low level queuing of Tasks to
Threads• Uses lightweight TreadPool– Number of cores– Default scheduler proviced by .NET– TaskScheduler.FromCurrentSynchronizationConte
xt()
• Can specify alternative scheduler for task to run on– Task.Start()– TaskFactory.Constructor
TaskCreationOptions• Task creation options to hint
TaskScheduler– AttachedToParent– HideScheduler– LongRunning– PreferFairness– A few more..
Task Scheduler
WorkerThread 1
WorkerThread 2
WorkerThread 3
WorkerThread n
Local QueueTasks created by running task in Local Queue (LIFO)
= Queued Task
Global Queue (FIFO)Work StealingIdle threads steal work if no tasks inlocal or global Queue
PreferFairness Task optionTask created by tasks placed in global queue
= Running Task
Threadpool
LongRunning Task optionRun on standalone task outside Instead of threadpool
Using PreferFairnessTask<int> outer = Task<int>.Factory.StartNew( () => {
var cts = new CancellationTokenSource(); var token = cts.Token; var tasks = new Task<int>[]{ Task.Factory.StartNew(() => { methodA(token); }, token), Task.Factory.StartNew(() => { methodB(token); }, token) }; var winnerIndex = Task.WaitAny(tasks); cts.Cancel(); return tasks[winnerIndex].Result; });return outer.Result;
Outer task is executed by thread from threadpool
Creates 2 more inner tasks, queued in local
task LIFO queue
On a busy system one of the tasks might never
execute (no workstealing by other thread)
Task.Factory.StartNew(() => { methodA(token); }, token, TaskCreationOptions.PreferFairness), Task.Factory.StartNew(() => { methodB(token); }, token), TaskCreationOptions.PreferFairness),,
ConcurrentExclusiveSchedulerPair• Provides task schedulers that coordinate
to execute tasks• Scheduler properties– ConcurrentScheduler
• Schedule tasks to this pair that may run concurrently with other tasks on this pair
– ExclusiveScheduler• Schedule tasks to this pair that must run
exclusively with regards to other tasks on this pair
SchedulerPairsvar schedPair =
new ConcurrentExclusiveSchedulerPair(); // Tasks on this scheduler may run concurrentlyreaderTask.Start(schedPair.ConcurrentScheduler); // Tasks on this scheduler run exclusivlywriterTask.Start(schedPair.ExclusiveScheduler);
How to Cancel a Task
• Tasks does not have Stop Method• CancelationTokenSource• -Create CancelationToken and pass to Task
– Signal a Task to Stop– Task should check token for cancelation– Return or Throw Exception
• Additional options for timed cancellations etc.• Can also be used for (timed) Wait actions
Cancelling a Taskvar tokenSource2 = new CancellationTokenSource();var ct = tokenSource2.Token;
var task = Task.Factory.StartNew(() => { while (true) { if (ct.IsCancellationRequested) ct.ThrowIfCancellationRequested(); // Do Work } }, ct );
tokenSource2.Cancel();
Error and Exception Handling• Exceptions occurred in execution
of task thrown when accessing the Result
• System.AggregateException– InnerExceptions
try { var result = task.Result;} catch( AggregateException ex ){foreach (var e in ex.InnerExceptions){ // Exception handling here }
Task Parallel Library• Task• Parallel– Parallel.For, Parallel.ForEach, Invoke
• PLINQ– Parallel extensions for LINQ– AsParallel()
• TPL Dataflow
Stopping Parallel Loopsvar cancellationSource = new CancellationTokenSource();var options = new ParallelOptions();options.CancellationToken = cancellationSource.Token;
Parallel.For(0, 10, options, (a, loopState) => { . . . . // cancellationToken.Cancel can be called externally // or Break can be called directly if condition is true if (cancellationToken.IsCancellationRequested) loopState.Break(); // loopState.Stop(); . . . .});
Throws OperationCanceledException
Using C# Break
Using loopstate
Parallel Options for Loops• CancelationToken– Stopping all Threads in the loop
• MaxDegreeOfParallelism– If you need more or less Threads
• TaskScheduler
PLINQvar query = (from d in data where … select d).AsParallel();
query.ForAll((d) => DoWork(d));
Concurent collections• Set of thread safe collection classes• System.Collections.Concurrent– ConcurrentBag<T>– ConcurrentDictionary<TKey, Tvalue> – ConcurrentQueue<T>– ConcurrentStack<T>
ConcurrentDictionaryvar dict = new ConcurrentDictionary<string, string>();
var key = "key01";var value = "value01";
// Get a valuedict.GetOrAdd(key, k => GetValue(key));
// Update or add a valuevalue = "newvalue";dict.AddOrUpdate(key, value, (k, o) => value);
C# 5.0• New keywords for async
async marks method or lambda expression as async await suspends execution of method until the awaited tasks returns
• Compiler does generate all the code to make this happen
C# 5.0public int LongRunningMethod(){ Task.Delay(1000); var result = 21 * 2; return result;}public async Task<int> LongRunningMethodAsync(){ await Task.Delay(1000); var result = 21 * 2; return result;}
Wait for Task to returnControl is returned to caller
This code runs when Task returns
The story of two Tasks (1)await DoAsync("Task-1");await DoAsync("Task-2");
Async but not parallel
Run
The story of two Tasks (2)var t1 = DoAsync("Task-1");var t2 = DoAsync("Task-2");
Parallel but returns immediately
Run
The story of two Tasks (3)var t1 = DoAsync("Task-1");var t2 = DoAsync("Task-2");await t1;await t2;
Parallel and wait for both tasks to return
Run
The story of two Tasks (4)var t1 = DoAsync("Task-1");var t2 = DoAsync("Task-2");
await Task.WhenAll(t1,t2)
Parallel and wait for both tasks to returnBut more efficient
Where await can’t be used• catch and finally blocks• lock blocks• LINQ query expressions– Allowed in extension methods (.
syntax)
• Unsafe code
How does await work• Does the C# compiler depends on .NET Task
class for await keyword ?• Compiler needs GetAwaiter() method• Method must return a class that implements
– GetResult()– IsCompleted()– INotifyCompleted.OnCompleted
• Implemented in Task
Demo• Implementing an awaitable class– See what the compiler needs for
await – Just for demo purposes
Make something async using Task
• Simply create a new Task to do the work
public static async Task<int> MyAsync(){ var result = await new Task<int>(() => { // your long running code code here... var calculatedResult = LongCalculation(); return calculatedResult; });
return result;}
Tasks without Threads• Task construction always take
code to execute on threadpool• But what if you…– Already have a thread– Don’t need a thread– Waiting for async IO, network etc– Handle events
TaskCompletionSource• Create Task objects that don't
execute code.
var tcs = new TaskCompletionSource<int>(); return tcs.Task;
Demo await anything• What you’ll see..
– await oneSecond;
– await techDays;
– await DayOfWeek.Saturday;
– return await Process.Start("HelloWorld.exe");
Task Interop patterns• Why
– Unified Task-based programming model– Composition
• Asynchronous to Task-based application model– Factory.TaskFromAsync
• Event-basedto Task-based application model– TaskCompletionSource
Async to Task-basedWebRequest wr = WebRequest.CreateHttp(url);
var t = Task<Stream>.Factory.FromAsync( wr.BeginGetRequestStream, wr.EndGetRequestStream, null );
Event to Tasks-based (1)public static Task<string> DownloadStringAsync(Uri url){
var wc = new WebClient(); var tcs = new TaskCompletionSource<string>(); wc.DownloadStringCompleted += (s,e) =>{ if (e.Error != null ) tcs.TrySetException(e.Error); else if (e.Cancelled) tcs.TrySetCanceled(); else tcs.TrySetResult(e.Result) }
wc.DownloadStringAsync(url); return tcs.Task;}
Task complete
Event to Task-based (2)private Task<string> LoadDocumentAsync( WebBrowser browser, string url){ var tcs = new TaskCompletionSource<string>(); WebBrowserDocumentCompletedEventHandler handler = null;
handler = (sender, e) => { tcs.SetResult(browser.DocumentText); browser.DocumentCompleted -= handler; };
browser.DocumentCompleted += handler; browser.Url = new Uri(url); return tcs.Task;}
Event source lifetime managed externally
Need to unregister handler
Using Lazy<T> with async• Lazy<T> is thread safe and supports async
// sync versionvar lazy = new Lazy<string>(() => { Thread.Sleep(5000); return DateTime.Now.ToString();}); var value = lazy.Value;
Factory code
// async versionvar lazyAsync = new Lazy<Task<string>>(async () => { await Task.Delay(5000); return DateTime.Now.ToString();
}); var value = await lazyAsync.Value;
Using Lazy<T> with asyncvar lazyAsync = new Lazy<Task<string>>(async () => { Console.WriteLine("I'm only called once..."); await Task.Delay(5000); return DateTime.Now.ToString();});
// Starts and wait for task var t1 = lazyAsync.Value; // Waits for same task var t2 = lazyAsync.Value;
Building an async cachepublic class AsyncCache<TKey, TValue>{ private Func<TKey,Task<TValue>> factory; private ConcurrentDictionary<TKey,Lazy<Task<TValue>>> dict; public AsyncCache(Func<TKey, Task<TValue>> valueFactory){ factory = valueFactory; dict = new ConcurrentDictionary<TKey, Lazy<Task<TValue>>>(); }
public Task<TValue> this[TKey key] { get { return dict.GetOrAdd(key, toAdd => new Lazy<Task<TValue>>(() => factory(toAdd))).Value; } } }
var cache = new AsyncCache<string, string>( async k => { wait Task.Delay(1000); return DateTime.Now.ToString(); });
var t = await cache["key1"];
SynchronizationContext
Ensures one at a time Dispatcher.BeginInvoke Control.BeginInvoke
• Used when code needs specific context– e.g. updating UI– SynchronizationContext class is abstract– Static Current property returns relevant
instance (Post is specific)
SynchronizationContextprivate void Button1Click(object sender,EventArgs e) { var ctx = SynchronizationContext.Current; new TaskFactory().StartNew(() => { ctx.Post(state => { label1.Text = string.Format("Update from Task") }, null}); }
• await Task continues automatically on Current context
Windows8• WinRT is based on COM• Async based on interfaces like this– IAsyncAction– IAsyncOperation<T>
• Task nor CLR do not implement these interfaces
Win8• How can we call these new asynch methods
from .NET or use Tasks in Win8 applications?• WindowsRuntimeSystemExtensions methods
take care of conversions public static Task AsTask<TResult>( this IAsyncOperation<TResult> source); public static IAsyncOperation<TResult> AsAsyncOperation<TResult>(this Task<TResult> source);
Async and ASP.NET• Async controller methods– Don’t block your IIS worker threads– Avoid 503 (Server too busy) errors
• MVC4/.NET4.5 supports async/await– Possible in MVC3, but more complex– WebForms also have support for async
• Always use async proxies when calling external services
Async in MVC3public class MVC3Controller : AsyncController{
public void IndexAsync(){ AsyncManager.OutstandingOperations.Increment(1); Task.Factory.StartNew(() =>{ var client = new WebClient(); string reply = client.DownloadString(uri); AsyncManager.Parameters["data"] = reply; AsyncManager.OutstandingOperations.Decrement();}); }
public ActionResult IndexCompleted(string data){ this.ViewBag.Result = data; return View(); } }
Async in MVC4public class MVC4Controller : Controller{ public async Task<ActionResult> Index() { var client = new WebClient(); this.ViewBag.Result = await client.DownloadStringTaskAsync(uri); return View(); }}
Async service proxies
Parallel programming models• .NET 4.0– Here’s the data, now setup computing– Primitives for Tasks and Data parallelism
• The Inverse model– Setup the Computing, now here’s the
data– Primitives for Dataflow parallelism
Enumerables and Observables
IEnumerator<T> IEnumerable<T>
Current{get;}
MoveNext()
Reset()
GetEnumerator()
IObserver<T> IObservable<T>
OnNext(T)
OnError(Ex)
OnCompled()
IDisposableSubscribe(IObserver<T>)
Iterations, pull model Subscription, pushmodel
Reactive Extensions and TPL Dataflow• Reactive Extension (Rx)
– Coordination and composition of event streams– LINQ-based API
• Dataflow (TDF)– Building blocks for message passing and parallelizing – Explicit control over how data is buffered and moved
• Many similarities, but each address distinct needs
Reactive Extensions Rx• Library for composing asynchronous
and event-based programs using observable sequences and LINQ-style query operators. – Rx for .NET, Rx.NET– Rx for JavaScript, RxJS– Rx for Windows Phone
Reactive Extensionsvar observable1 = Observable.Range(1, 20);var subscription1 = observable1.Subscribe<int>(Console.WriteLine);
var oneSecond = TimeSpan.FromSeconds(1);var observable2 = Observable.Timer(oneSecond,oneSecond);var subscription2 = observable2.Subscribe<int>(Console.WriteLine);
var observer3 = observable1 .Select((i) => i) .Skip(2) // skip first two values .Where(i => (i % 2 == 0)) // only get the even values .Zip(observable2, (i, t) => i) // one value per second .Subscribe((i) => Console.WriteLine(i));
Reactive Extensionsvar mouseMove = Observable .FromEventPattern<MouseEventHandler, MouseEventArgs>( h => this.MouseMove += h, h => this.MouseMove -= h);
var mouseUp = Observable .FromEvent<MouseEventHandler, MouseEventArgs>( h => this.MouseUp += h, h => this.MouseUp -= h); var mouseDown = Observable .FromEvent<MouseEventHandler, MouseEventArgs>( h => this.MouseDown += h, h => this.MouseDown -= h);
var observable = mouseMove // Get mousemove positions .SkipUntil(mouseDown) // Skip until mouse button down .TakeUntil(mouseUp) // Take until mouse button is up .Select(a => a.EventArgs.Location);
Reactive Extensions• Many operators– Skip, Take, Zip, Throttle, Buffer, Repeat…
• ObserveOn() and SubscribeOn() methods– Optional Scheduler and Context
parameters– Specify which thread/context the observer
and subscribers run on
TPL Dataflow• Primitives for in-process message/data
passing– Blocks for buffering and processing data
• Linkable to form a network– Data automatically propagated from sources to
linked targets– Enables building powerful parallel and
asynchronous pipelinesBased
• Integrates with Task, IObservable,…
Executor Blocks
Buffering Blocks
Join Blocks
Building a Dataflow network
input.LinkTo(action1);input.LinkTo(transform1);input.LinkTo(transform2);
transform1.LinkTo(join.Target1);transform2.LinkTo(join.Target2);
join.LinkTo(action2);
ActionBlock in action
for (var i = 0; i < 10; i++) actionBlock.Post(i);
var actionBlock = new ActionBlock<int>((i) => { Console.WriteLine("[{0}]\t{1}", Thread.CurrentThread.ManagedThreadId,i); });
ActionBlock in action
for (var i = 0; i < 10; i++) actionBlock.Post(i);
var actionBlock = new ActionBlock<int>((i) => { Console.WriteLine("[{0}]\t{1}", Thread.CurrentThread.ManagedThreadId,i); }, new ExecutionDataflowBlockOptions() { MaxDegreeOfParallelism = 4 } );
Max 4 instances running
Linking Blocks
var actionBlock = new ActionBlock<int>((i) => Console.WriteLine(i));
var transformBlock = new TransformBlock<int, int>((i) => i * i);
transformBlock.LinkTo(actionBlock);
for (var i = 0; i < 10; i++) transformBlock.Post(i);
Buffering
var actionBlock = new ActionBlock<int>((i) => Console.WriteLine(i));
var bufferBlock = new BufferBlock<int>(new DataflowBlockOptions( { BoundedCapacity = 10 });
bufferBlock.LinkTo(actionBlock); Post blocks if buffer is full
How about JavaScript?• Use async in the browser– Reactive Extensions for JavaScript– jQuery Defered and Promises
Thank you…
Arie Leeuwesteijn [email protected]@arieleeuwhttp://tinyurl.com/c78tn5j