python asynchronous programming with salt stack (tornado ... · python asynchronous programming...
TRANSCRIPT
![Page 1: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/1.jpg)
Python Asynchronous Programmingwith
Salt Stack (tornado, asyncio) and RxPYPyCon Korea 2017
![Page 2: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/2.jpg)
Python Asynchronous Programming with
Salt Stack (tornado, asyncio) and RxPY
![Page 3: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/3.jpg)
Interest
The safeness of scalable—code and performance—programs.
![Page 4: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/4.jpg)
Interest
Methods to Build Safe and Scalable Programs
1. Born to be Googler
2. Software Analysis, Checking errors before execution Static Analysis, Abstract Interpretation, Sparrow, FB Infer
3. Software Verification, Implementing programs with robust mathematical basis Coq, Machine-checked Proofs, Type Inference, Robust Type Systems, …
4. Frameworks, Using safeness & productivity of frameworks Asynchronous Frameworks, Reactive Programming, …
![Page 5: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/5.jpg)
Interest
1. Born to be Googler
2. Software Analysis, Checking errors before execution Static Analysis, Abstract Interpretation, Sparrow, FB Infer
3. Software Verification, Implementing programs with robust mathematical basis Coq, Machine-checked Proofs, Type Inference, Robust Type Systems, …
4. Frameworks, Using safeness & productivity of frameworks Asynchronous Frameworks, Reactive Programming, …
What I have been doing in NHN Ent.
Methods to Build Safe and Scalable Programs
![Page 6: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/6.jpg)
Interest
1. Born to be Googler
2. Software Analysis, Checking errors before execution Static Analysis, Abstract Interpretation, Sparrow, FB Infer
3. Software Verification, Implementing programs with robust mathematical basis Coq, Machine-checked Proofs, Type Inference, Robust Type Systems, …
4. Frameworks, Using safeness & productivity of frameworks Asynchronous Frameworks, Reactive Programming, …
What I will discuss in this talk
👉
Methods to Build Safe and Scalable Programs
![Page 7: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/7.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 8: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/8.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 9: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/9.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 10: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/10.jpg)
Parallelism - Concurrency Blocking I/O - Non-blocking I/O
Synchronous Programming (?) - Asynchronous Programming
https://kstreee.github.io/techmemo/async_and_webframework.pdf
Preliminary
![Page 11: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/11.jpg)
JobsJobsJobsJobsJobsJobsJobsJobs
JobsJobsJobsJobs
Parallelism - ConcurrencyPreliminary
WorkerWorker
JobsJobsJobsJobsJobsJobsJobsJobs
JobsJobsJobsJobsJobsJobsJobsJobs
JobsJobsJobsJobs
Worker
JobsJobsJobsJobsJobsJobsJobsJobs
![Page 12: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/12.jpg)
Worker Pool
Parallelism + ConcurrencyPreliminary
JobsJobsJobsJobsJobsJobsJobsJobs
JobsJobsJobsJobs
Worker
JobsJobsJobsJobsJobsJobsJobsJobs
![Page 13: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/13.jpg)
PreliminaryBlocking - Non-blocking I/O
while not_finished: data = socket.recv(buf_size) do_something(data)
![Page 14: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/14.jpg)
PreliminaryBlocking - Non-blocking I/O
while not_finished: try: data = socket.recv(buf_size) do_something(data) except socket.error as e: if e.args[0] in _ERRNO_WOULDBLOCK: # DO SOMETHING ELSE
![Page 15: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/15.jpg)
Asynchronous ProgrammingPreliminary
data = yield tornado.iostream.read_until('\r\n')
![Page 16: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/16.jpg)
Asynchronous ProgrammingPreliminary
Implementation Details
while True: try: data = socket.recv(buf_size) except socket.error as e: if e.args[0] in _ERRNO_WOULDBLOCK: # DO SOMETHING ELSE
Non-blocking socket I/O
data = yield tornado.iostream.read_until('\r\n')
![Page 17: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/17.jpg)
Asynchronous ProgrammingPreliminary
Could use any other methods, even, those don't need to be non-blocking I/O.
Implementation Details
data = yield tornado.iostream.read_until('\r\n')
![Page 18: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/18.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 19: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/19.jpg)
Providing Concurrency by Scheduling Events
Asynchronous Programming
![Page 20: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/20.jpg)
Providing Concurrency by Scheduling Events
Asynchronous Programming
data = yield tornado.iostream.write(data)
![Page 21: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/21.jpg)
How to communicate with a scheduler ?
ex) Callback, Future, Promise, Await…
Asynchronous Programming
![Page 22: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/22.jpg)
Callback, Future, Await, …
Trigger async jobs
SchedulerUser Program
Scheduling & Executinghttp.get(uri, function(response) { ..})
asyncio ioloop …
Asynchronous Programming
![Page 23: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/23.jpg)
Callback, Future, Await, …
Trigger async jobs
Scheduler
Scheduling & Executing
response_future = http.get(uri) response = response_future.get()
User Program asyncio ioloop …
Asynchronous Programming
![Page 24: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/24.jpg)
Callback, Future, Await, …
Trigger async jobs
Scheduler
Scheduling & Executing
response = yield http.get(uri)
User Program asyncio ioloop …
Asynchronous Programming
![Page 25: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/25.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 26: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/26.jpg)
Massive I/O
When do we have to use Async?
![Page 27: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/27.jpg)
Salt MasterSalt API Channel (ZeroMQ, TCP, …)
Salt Minion
10K+ Servers
Clients
I/O
I/O I/O
When do we have to use Async?Salt Stack with Tornado in NHN Ent.
Toast Cloud Products, (TC Deploy, RDS, …),Other NHN Ent. Internal Systems, … Total : 10+ Systems
![Page 28: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/28.jpg)
API Gateway
ServicesClientsI/O
When do we have to use Async?
I/O
Services with Massive I/O
REST API Server
…
![Page 29: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/29.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 30: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/30.jpg)
The safeness of scalable—code and performance—programs.
Why Async Frameworks Matters?
![Page 31: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/31.jpg)
👉
Why Async Frameworks Matters?
1. Born to be Googler
2. Software Analysis, Checking errors before execution Static Analysis, Abstract Interpretation, Sparrow, FB Infer
3. Software Verification, Implementing programs with robust mathematical basis Coq, Machine-checked Proofs, Type Inference, Robust Type Systems, …
4. Frameworks, Using safeness & productivity of frameworks Asynchronous Frameworks, Reactive Programming, …
Methods to Build Safe and Scalable Programs
![Page 32: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/32.jpg)
1. Born to be Googler … ?
😵 😇 👶
Why Async Frameworks Matters?
![Page 33: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/33.jpg)
2. Software Analysis, Checking errors before execution Static Analysis, Abstract Interpretation, Sparrow, FB Infer
1. Abstractize
3. Concretize & Report Alarms
Real WorldMathematical World
2. Analysis Implement
Why Async Frameworks Matters?
![Page 34: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/34.jpg)
3. Software Verification, Implementing programs with robust mathematical basis
Coq, Machine-checked Proofs, Type Inference, Robust Type Systems, …
Real WorldMathematical World
2. Export
1. Implement
Why Async Frameworks Matters?
![Page 35: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/35.jpg)
4. Frameworks, Using safeness & productivity of frameworks Asynchronous Frameworks, Reactive Programming, …
Mathematical World
Real World
1. Implement
Why Async Frameworks Matters?
![Page 36: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/36.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 37: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/37.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 38: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/38.jpg)
Everything must be compositional
Code Scalability
![Page 39: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/39.jpg)
RemindCode Scalability
data = yield tornado.iostream.read_until(‘\r\n’)
Asynchronous Frameworks (tornado, asyncio)
while True: try: data = socket.recv(buf_size) except socket.error as e: if e.args[0] in _ERRNO_WOULDBLOCK: # DO SOMETHING ELSE
non-blocking I/O
much more compositional
![Page 40: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/40.jpg)
Code Scalability
Is it enough?
Not enough for us
Asynchronous Frameworks (tornado, asyncio)
NO!
data = yield tornado.iostream.read_until(‘\r\n’)
![Page 41: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/41.jpg)
Code ScalabilityNot enough for us
![Page 42: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/42.jpg)
Code ScalabilityComparison between (tornado, asyncio) and RxPY
Multiple Async HTTP Calls Multiple Async HTTP Calls, Take Fastest Response
Sleep between Jobs & Global Timeout
![Page 43: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/43.jpg)
res1 = yield service1_api_call() res2 = yield service2_api_call() res3 = yield service3_api_call() response = res1 + res2 + res3
Multiple Async HTTP Calls, Torando, asyncio
Must be refactored
Code Scalability
![Page 44: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/44.jpg)
futures = [service1_api_call(), service2_api_call(), service3_api_call()]
response = [] for future in futures: response.append((yield future))
Multiple Async HTTP Calls, Torando, asyncioCode Scalability
![Page 45: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/45.jpg)
Multiple Async HTTP Calls, RxPY
Observable.merge(service1_api_call(), service2_api_call(), service3_api_call()) .map(lambda response: ...)
Code Scalability
![Page 46: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/46.jpg)
Multiple Async HTTP Calls, Take Fastest Response, Torando, asyncio
futures = [server1_api_call(), server2_api_call(), server3_api_call()]
response = None for future in futures: response = yield future break
Code Scalability
…?
Must be refactored
![Page 47: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/47.jpg)
Multiple Async HTTP Calls, Take Fastest Response, Torando, asyncioCode Scalability
class Any(Future): def __init__(self, futures): super(Any, self).__init__() for future in futures: future.add_done_callback(self.done_callback)
def done_callback(self, future): if not self.done(): self.set_result(future)
futures = Any(system1_api_call(), system2_api_call(), system3_api_call()) response = yield futures
https://github.com/saltstack/salt/blob/b7cd30d3ee919fcf3f03a1afe349fb68b357cd99/salt/netapi/rest_tornado/saltnado.py#L269-L281
JUGGLING FUTURES
![Page 48: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/48.jpg)
Observable.merge(server1_api_call(), server2_api_call(), server3_api_call()) .take(1) .map(lambda response: ...)
Multiple Async HTTP Calls, Take Fastest Response, RxPYCode Scalability
![Page 49: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/49.jpg)
More Complex Examples, RxPY
res1 = Observable.merge(service1_server1_api_call(), service1_server2_api_call(), service1_server3_api_call()) .take(1)
res2 = Observable.merge(service2_server1_api_call(), service2_server2_api_call(), service2_server3_api_call()) .take(1)
Observable.merge(res1, res2) .map(lambda response: ...)
Code Scalability
![Page 50: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/50.jpg)
More Complex Examples, Torando, asyncio
…?
Code Scalability
![Page 51: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/51.jpg)
Sleep between Jobs & Global Timeout, tornado, asyncio
elapsed_time = 0 for item in many_items: begin_time = time.time() yield gen.sleep(1000) yield insert_to_db(item) elapsed_time += time.time() - begin_time if elapsed_time > many_items.length * 1000 + MARGIN: raise Exception(...)
Code Scalability
![Page 52: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/52.jpg)
Sleep between Jobs & Global Timeout, RxPY
Observable.from(many_items) .zip(Observable.interval(1000), lambda (data, interval): data) .flat_map(insert_to_db) .timeout(many_items.length * 1000 + MARGIN) .subscribe(success, exception, completion)
Code Scalability
![Page 53: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/53.jpg)
Code Scalability
Asynchronous Frameworks (tornado, asyncio)
non-blocking I/O
much more compositional
much more compositional
Reactive Programming (RxPY)
data = yield tornado.iostream.read_until(‘\r\n’)
while True: try: data = socket.recv(buf_size) except socket.error as e: if e.args[0] in _ERRNO_WOULDBLOCK: # DO SOMETHING ELSE
![Page 54: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/54.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 55: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/55.jpg)
+
Providing Concurrency by Scheduling Events
Providing Operators by Calling Functions
Reduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 56: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/56.jpg)
Rx Operators
timer, defer, interval, repeat, just, map, flat_map, buffer, filter, debounce, last, skip, zip, merge, catch_exception, retry, delay, timeout, reduce, average, max, min, count, …
+
Observable<Data>
Operators
Reactive Programming
![Page 57: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/57.jpg)
Essence of Observable
Stream<Optional<Async<Data>>>
ALREADY, Fork-Join, asyncio, ioloop, …
ALREADYALREADY, ROP
Reactive Programming
![Page 58: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/58.jpg)
Stream<Optional<Async<Data>>>(Data Stream<Optional<Async<Data>>>)Stream<Optional<Async<Data>>>
Rx :
Essence of Observable
Reactive Programming
![Page 59: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/59.jpg)
Stream<Optional<Async<Data>>>(Data Stream<Optional<Async<Data>>>)
Stream<Optional<Async<Data>>>
Rx :
(Data Stream<Optional<Async<Data>>>)(Data Stream<Optional<Async<Data>>>)(Data Stream<Optional<Async<Data>>>)
Essence of Observable
Reactive Programming
![Page 60: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/60.jpg)
Stream<Optional<Async<Data>>>(Data Stream<Optional<Async<Data>>>)
Stream<Optional<Async<Data>>>
Rx :
(Data Stream<Optional<Async<Data>>>)(Data Stream<Optional<Async<Data>>>)(Data Stream<Optional<Async<Data>>>)
Scheduler User
Essence of Observable
Reactive Programming
![Page 61: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/61.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
Reduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 62: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/62.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
1. Stream (infinite data structure based on Co-induction)Reduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 63: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/63.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
2. Some other obvious examples of compositionalityReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 64: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/64.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_teacher) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
2. Some other obvious examples of compositionalityReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 65: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/65.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(super_super_complex_logic) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
2. Some other obvious examples of compositionalityReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 66: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/66.jpg)
3. Compositionality of Operators
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .some_other_operator(. . .)
operator = some_other_operator : Observable<a> -> * -> Observable<b>
Observable.from(get_people_by_async_task) . (. . .) .flat_map(feature_x) . (. . .) .some_other_operator(. . .) .subscription(success, error, complete)
feature_x =
feature_y =
Reduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 67: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/67.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
https://fsharpforfunandprofit.com/rop/
Reduce Complexity using functional programming patterns, disciplines
Reactive Programming4. Functional Programming Patterns (ROP), First Class Effect
![Page 68: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/68.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
5. Abstracted Data Type based on Type Theory - First Class Effect
https://www.cl.cam.ac.uk/teaching/1415/L28/monads.pdf
Reduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 69: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/69.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
from : a -> Observable<a>
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 70: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/70.jpg)
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
filter : Observable<a> -> (a -> bool) -> Observable<a>
flat_map : Observable<a> -> (a -> Observable<b>) -> Observable<b> map : Observable<a> -> (a -> b) -> Observable<b>
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 71: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/71.jpg)
operators : Observable<a> -> * -> Observable<b> ≃ flat_map
Observable.from(get_people_by_async_task) .filter(non_empty) .filter(is_student) .map(extract_student_id) .flat_map(get_person_detail_by_async_task) .map(extract_name) .subscription(success, error, complete)
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 72: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/72.jpg)
Observable.return(get_people_by_async_task) .flat_map(filter_non_empty) .flat_map(filter_is_student) .flat_map(extract_student_id) .flat_map(get_person_detail_by_async_task) .flat_map(extract_name) .subscription(success, error, complete)
return : a -> Observable<a>flat_map : Observable<a> -> (a -> Observable<b>) -> Observable<b>
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 73: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/73.jpg)
Stream.return(Optional.return(Async.return(get_people_by_async_task))) .flat_map(Optional.flat_map(Async.flat_map(filter_non_empty))) .flat_map(Optional.flat_map(Async.flat_map(filter_is_student))) .flat_map(Optional.flat_map(Async.flat_map(extract_student_id))) .flat_map(Optional.flat_map(Async.flat_map(get_person_detail_by_async_task))) .flat_map(Optional.flat_map(Async.flat_map(extract_name)))
return : a -> Stream<Optional<Async<<<a>>>flat_map : Stream<Optional<Async<<<a>>> -> (a -> Stream<Optional<Async<<<b>>>) -> Stream<Optional<Async<<<b>>>
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 74: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/74.jpg)
Observable.return(get_people_by_async_task) .flat_map(filter_non_empty) .flat_map(filter_is_student) .flat_map(extract_student_id) .flat_map(get_person_detail_by_async_task) .flat_map(extract_name)
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 75: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/75.jpg)
for { data <- get_people_by_async_task() data <- filter_non_empty(data) data <- filter_is_student(data) data <- extract_student_id(data) data <- get_person_detail_by_async_task(data) data <- extract_name(data) } yield { . . . }
In Scala
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 76: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/76.jpg)
do data <- get_people_by_async_task data <- filter_non_empty data data <- filter_is_student data data <- extract_student_id data data <- get_person_detail_by_async_task data data <- extract_name data data . . .
In Haskell
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 77: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/77.jpg)
let%lwt data = get_people_by_async_task in let%lwt data = filter_non_empty data in let%lwt data = filter_is_student data in let%lwt data = extract_student_id data in let%lwt data = get_person_detail_by_async_task data in let%lwt data = extract_name data in . . .
In OCaml
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 78: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/78.jpg)
Observable.return(get_people_by_async_task) .flat_map(SomeObj::filter_non_empty) .flat_map(SomeObj::filter_is_student) .flat_map(SomeObj::extract_student_id) .flat_map(SomeObj::get_person_detail_by_async_task) .flat_map(SomeObj::extract_name)
In Java
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 79: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/79.jpg)
Observable.return(get_people_by_async_task) .flat_map(filter_non_empty) .flat_map(filter_is_student) .flat_map(extract_student_id) .flat_map(get_person_detail_by_async_task) .flat_map(extract_name)
In Python
5. Abstracted Data Type based on Type Theory - First Class EffectReduce Complexity using functional programming patterns, disciplines
Reactive Programming
![Page 80: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/80.jpg)
Essence of Observable
Stream<Optional<Async<Data>>>
Reactive Programming
ALREADY, Fork-Join, asyncio, ioloop, …
ALREADYALREADY, ROP
![Page 81: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/81.jpg)
First-Class Effect
Co-induction
Stream<Optional<Async<Data>>>
First Class Effect
Patterns & Details
Essence
Essence of Observable
Reactive Programming
ALREADY, Fork-Join, asyncio, ioloop, …
ALREADYALREADY, ROP
![Page 82: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/82.jpg)
Interface
Implementation
Value First-Class Effect
Co-induction
Essence
Stream<Optional<Async<Data>>>
ALREADY, Fork-Join, asyncio, ioloop, …
ALREADYALREADY, ROP
First Class Effect
Patterns & Details
Essence of Observable
Reactive Programming
What we can obtain by following the principles can be explained based on mathematics.
![Page 83: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/83.jpg)
Preliminary Asynchronous Programming
When do we have to use Async? Why Async Frameworks Matters?
1. Preliminary of Asynchronous Programming
2. Async Frameworks Details
INDEX
Code Scalability Reactive Programming (RxPY)
Why Reactive Programming Matters?
![Page 84: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/84.jpg)
Mathematical World
Real World
1. Implement
Interface Syntax Type Systems Syntactic Sugars Usage of Frameworks Usage of Libraries Programming Patterns . . .
Implementation Compiler Type Checkers Frameworks Libraries Implementation Details Engineerings . . .
Essence Curry-Howard Isomorphism Type Theory Category Theory Monads, First Class Side Effect Propositional Logics . . .
👆What we can obtain by following the principles can be explained based on mathematics.
Why Reactive Programming Matters?4. Frameworks, Using safeness & productivity of frameworks
Asynchronous Frameworks, Reactive Programming, …
![Page 85: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/85.jpg)
Mathematical World
Real World
1. Implement
Interesting!👇
Why Reactive Programming Matters?
![Page 86: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/86.jpg)
Just Use! Dependency Injection, Flux, Redux (Redux-React, Revue, …), …
Mathematical World
Real World
1. Implement
Engineeringly
Why Reactive Programming Matters?
![Page 87: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/87.jpg)
Advanced Type System (Scala DOT, …),GADT, Dependent Types, Ur/Web, …
Mathematical World
Real World
1. Implement
Mathematically
Why Reactive Programming Matters?
Interesting!👇
![Page 88: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/88.jpg)
Reactive Programming (RxPY, RxJava, …)
Mathematical World
Real World
1. Implement
Mathematically
Engineeringly+
Why Reactive Programming Matters?
Interesting!👇
![Page 89: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/89.jpg)
SUMMARY IN 3 SLIDES
![Page 90: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/90.jpg)
Code Scalability
data = yield tornado.iostream.read_until(‘\r\n’)
Asynchronous Frameworks (tornado, asyncio)
while True: try: data = socket.recv(buf_size) except socket.error as e: if e.args[0] in _ERRNO_WOULDBLOCK: # DO SOMETHING ELSE
non-blocking I/O
much more compositional
much more compositional
Reactive Programming (RxPY)
![Page 91: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/91.jpg)
Interface
Implementation
Value First-Class Effect
Co-induction
Essence
Stream<Optional<Async<Data>>>
ALREADY, Fork-Join, asyncio, ioloop, …
ALREADYALREADY, ROP
First Class Effect
Patterns & Details
Essence of Observable
Reactive Programming
What we can obtain by following the principles can be explained based on mathematics.
![Page 92: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/92.jpg)
Mathematical World
Real World
1. Implement
Interesting!👇
Why Reactive Programming Matters?
![Page 93: Python Asynchronous Programming with Salt Stack (tornado ... · Python Asynchronous Programming with Salt Stack (tornado, asyncio) and RxPY PyCon Korea 2017. Python Asynchronous Programming](https://reader031.vdocuments.mx/reader031/viewer/2022012319/5d67246f88c99340518b481f/html5/thumbnails/93.jpg)
References & Further More
Salt Stack & RxPY • Salt Stack : https://github.com/saltstack/salt• RxPY : https://github.com/reactivex/rxpy
Asynchronous Programming & Reactive Programming • The introduction to Reactive Programming you’ve been missing : https://gist.github.com/staltz/868e7e9bc2a7b8c1f754• Your mouse is a database : http://queue.acm.org/detail.cfm?id=2169076• Lwt: a Cooperative Thread Library : https://www.irif.fr/~Vouillon/publi/lwt.pdf• Optimizing the Netflix API : https://medium.com/netflix-techblog/optimizing-the-netflix-api-5c9ac715cf19• There is no Fork: an Abstraction for Efficient, Concurrent, and Concise Data Access :
https://research.fb.com/publications/there-is-no-fork-an-abstraction-for-efficient-concurrent-and-concise-data-access/
Advanced Materials • Functional Program Design in Scala : https://www.coursera.org/learn/progfun2• Advanced Functional Programming : https://www.cl.cam.ac.uk/teaching/1415/L28/materials.html• “Mostly functional” programming does not work : http://queue.acm.org/detail.cfm?ref=rss&id=2611829• Railway Oriented Programming : https://fsharpforfunandprofit.com/rop/• First-Class Effect : https://www.cl.cam.ac.uk/teaching/1415/L28/monads.pdf• Moving fast with software verification : https://research.fb.com/wp-content/uploads/2016/11/
publication00124_download0001.pdf?