python-csp: bringing OCCAM to Python

Download python-csp: bringing OCCAM to Python

Post on 19-May-2015

3.395 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

Bringing OCAAM

TRANSCRIPT

<ul><li> 1. Ancient history python-csp: features and idioms Using built-in processes Future challenges python-csp: bringing OCCAM to PythonSarah Mount Europython 2010Sarah Mount python-csp: bringing OCCAM to Python </li></ul><p> 2. Ancient history python-csp: features and idioms Using built-in processes Future challenges 1 Ancient history2 python-csp: features and idiomsProcess creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models3 Using built-in processes4 Future challenges Sarah Mount python-csp: bringing OCCAM to Python 3. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges This talk. . . is all about the python-csp project. There is a similar project called PyCSP, these will merge over the next few months. Current code base is here: http://code.google.com/p/python-csp Please do contribute bug xes / issues / code / docs / rants / . . . Sarah Mount python-csp: bringing OCCAM to Python 4. Ancient history python-csp: features and idioms Using built-in processes Future challenges INMOS B0042 Board c David May The Transputer was the original multicore machine and was built to run the OCCAM language a realisation of CSP. http://www.cs.bris.ac.uk/~dave/transputer.htmlSarah Mount python-csp: bringing OCCAM to Python 5. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Tony Hoare Invented CSP, OCCAM, Quicksort and other baddass stu. You may remember him from such talks as his Europython 2009 Keynote. Sarah Mount python-csp: bringing OCCAM to Python 6. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges OCCAM OCCAM lives on in its current implementation as OCCAM-http://www.cs.kent.ac.uk/projects/ofa/kroc/It can run on Lego bricks, Arduino boards and other things. LikePython it uses indentation to demark scope. Unlike PythonOCCAM is MOSTLY IN UPPERCASE. Sarah Mount python-csp: bringing OCCAM to Python 7. Ancient history python-csp: features and idioms Using built-in processes Future challenges CSP and message passing This next bit will be revision for many of you!Sarah Mount python-csp: bringing OCCAM to Python 8. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Most of us think of programs like thishttp://xkcd.com/205/Sarah Mount python-csp: bringing OCCAM to Python 9. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges This is the old standard model!Multicore will soon take over the world (if it hasnt already). Shared memory concurrency / parallelism is hard: Race hazards Deadlock Livelock Tracking which lock goes with which data and what order locks should be used. Operating Systems are old news! Sarah Mount python-csp: bringing OCCAM to Python 10. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Message passingDenitionIn message passing concurrency (or parallelism) everything is aprocess and no data is shared between processes. Instead, data ispassed between channels which connect processes together.Think: OS processes and UNIX pipes. The actor model (similar to Kamaelia) uses asynchronouschannels. Processes write data to a channel and continueexecution.CSP (and -calculus) use synchronous channels, orrendezvous, where a process writing to a channel has to waituntil the value has been read by another process beforeproceeding. Sarah Mount python-csp: bringing OCCAM to Python 11. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Process diagram Sarah Mount python-csp: bringing OCCAM to Python 12. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Why synchronous channels?Leslie Lamport (1978). Time, clocks, and the ordering of events ina distributed system. Communications of the ACM 21 (7)Sarah Mount python-csp: bringing OCCAM to Python 13. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Hello World!from c s p . c s p p r o c e s s i m p o r t @process def h e l l o () : p r i n t H e l l o CSP w o r l d ! hello () . start () Sarah Mount python-csp: bringing OCCAM to Python 14. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Example of process creation: web server@process def s e r v e r ( host , port ) : sock = socket . socket ( . . . ) # S e t up s o c k e t s w h i l e True : conn , a d d r = s o c k . a c c e p t ( ) r e q u e s t = conn . r e c v ( 4 0 9 6 ) . s t r i p ( ) i f r e q u e s t . s t a r t s w i t h ( GET ) :ok ( r e q u e s t , conn ) . s t a r t ( ) else : n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python 15. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models Example of process creation: web server @process d e f n o t f o u n d ( r e q u e s t , conn ) : page = F i l e n o t found conn . s e n d ( r e s p o n s e ( 4 0 4 , Not Found ,page ) ) conn . shutdown ( s o c k e t . SHUT RDWR) conn . c l o s e ( ) return Sarah Mount python-csp: bringing OCCAM to Python 16. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Combining processes: in parallel # With a Par o b j e c t :Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( )# With s y n t a c t i c s u g a r :p1 // ( p2 , p3 , . . . pn )# RHS must be a s e q u e n c e t y p e . Sarah Mount python-csp: bringing OCCAM to Python 17. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Combining processes: repeating a process sequentially @processdef h e l l o () :p r i n t H e l l o CSP w o r l d ! if name== m a i n :hello () 32 hello () Sarah Mount python-csp: bringing OCCAM to Python 18. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models Channels @process d e f c l i e n t ( chan ) : p r i n t chan . r e a d ( ) @process d e f s e r v e r ( chan ) : chan . w r i t e ( H e l l o CSP w o r l d ! )if name == m a i n : ch = C h a n n e l ( ) Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( )Sarah Mount python-csp: bringing OCCAM to Python 19. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models What you need to know about channels Channels are currently UNIX pipes This will likely change Channel rendezvous is slow compared with JCSP: 200 s vs 16 s This will be xed by having channels written in C Because channels are, at the OS level, open les, there can only be a limited number of them (around 300 on my machine) This makes me sad :-( Sarah Mount python-csp: bringing OCCAM to Python 20. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Message passing an responsiveness This isnt about speed.Sometimes, you have several channels and reading from themall round-robin may reduce responsiveness if one channel readblocks for a long time.This is BAD: @processdef foo ( channels ) :f o r chan i n c h a n n e l s :p r i n t chan . r e a d ( ) Sarah Mount python-csp: bringing OCCAM to Python 21. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models ALTing and choice DenitionALTing, or non-deterministic choice, selects a channel to read fromfrom those channels which are ready to read from. You can do this with (only) two channels: @processd e f f o o ( c1 , c2 ) :p r i n t c1 | c2p r i n t c1 | c2Sarah Mount python-csp: bringing OCCAM to Python 22. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models ALTing proper (many channels) @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : a l t = A l t ( c1 , c2 , c3 , c4 , c5 ) # Choose random a v a i l a b l e o f f e r print alt . se le ct () # Avoid l a s t s e l e c t e d channel print alt . f a i r s e l e c t () # Choose c h a n n e l w i t h l o w e s t i n d e x print alt . p r i s e l e c t () Sarah Mount python-csp: bringing OCCAM to Python 23. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Syntactic sugar for ALTing @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : gen = A l t ( c1 , c2 , c3 , c4 , c5 ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) Sarah Mount python-csp: bringing OCCAM to Python 24. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models ALTing: when you really, really must return right now @processd e f f o o ( c1 , c2 , c3 ) :# Skip () w i l l / always / complete# a read immediatelygen = A l t ( c1 , c2 , c3 , S k i p ( ) )p r i n t gen . n e x t ( )...Sarah Mount python-csp: bringing OCCAM to Python 25. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models Channel poisoningDenitionIdiomatically in this style of programming most processes containinnite loops. Poisoning a channel asks all processes connected tothat channel to shut down automatically. Each process which hasa poisoned channel will automatically poison any other channels itis connected to. This way, a whole graph of connected processescan be terminated, avoiding race conditions. @processdef foo ( channel ) :...channel . poison ()Sarah Mount python-csp: bringing OCCAM to Python 26. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models A bigger example: Mandelbrot generatorSarah Mount python-csp: bringing OCCAM to Python 27. Process creationAncient historyParallel, sequential and repeated processespython-csp: features and idiomsCommunication via channel read / writesUsing built-in processes More channels: ALTing and choiceFuture challengesMore channels: poisoning Producer / consumer or worker / farmer models @process d e f main ( IMSIZE ) : channels = [ ] # One p r o d u c e r + c h a n n e l f o r e a c h image column . f o r x i n r a n g e ( IMSIZE [ 0 ] ) : c h a n n e l s . append ( C h a n n e l ( ) ) p r o c e s s e s . append ( m a n d e l b r o t ( x , . . . channels [ x ]) ) p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE , channels ) ) mandel = Par ( p r o c e s s e s ) mandel . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python 28. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models @process def mandelbrot ( xcoord , cout ) : # Do some maths h e r e cout . w r i t e ( xcoord , column data )Sarah Mount python-csp: bringing OCCAM to Python 29. Process creation Ancient historyParallel, sequential and repeated processes python-csp: features and idiomsCommunication via channel read / writes Using built-in processes More channels: ALTing and choice Future challengesMore channels: poisoningProducer / consumer or worker / farmer models @process d e f consumer ( c i n s ) : # I n i t i a l i s e PyGame . . . gen = l e n ( c i n s ) A l t ( c i n s ) f o r i in range ( len ( c i n s ) ) : x c o o r d , column = gen . n e x t ( ) # B l i t t h a t column t o s c r e e n f o r e v e n t i n pygame . e v e n t . g e t ( ) : i f e v e n t . t y p e == pygame . QUIT : for channel in cins :channel . poison () pygame . q u i t ( )Sarah Mount python-csp: bringing OCCAM to Python 30. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges Oscilloscope Sarah Mount python-csp: bringing OCCAM to Python 31. Ancient historypython-csp: features and idiomsUsing built-in processesFuture challenges @process def O s c i ll o s c o p e ( inchan ) : # I n i t i a l i s e PyGame ydata = [ ] w h i l e n o t QUIT : y d a t a . append ( i n c h a n . r e a d ( ) ) y d a t a . pop ( 0 )# B l i t data t...</p>