async programming and python

Async Programming and

PythonPyCon India, 2014

Chetan Giridhar

Basics• Programming tasks:

o I/O bound

o CPU bound

• Say, you’re doing I/O o Will it complete immediately? When will it be done?

o Wont they block you?

Blocking I/O: Exampleimport requests

r = requests.get(‘http://google.co.in’)

r.status_code

• What if the request takes a long time?• Operation blocks until all the data is recieved from the server

Can we do something in the meanwhile?

Can we run another task, concurrently?

Non Blocking / Async• Non-blocking means the ability to make continuous

progress at all times

• Resources needed for a response must not be

monopolized

• As such it can enable both lower latency, higher

throughput

Programming ModelsSynchronous model

time

time

time

Threaded model

Asynchronous model

Task 1 Task 2 Task 3

Math• Task = make a call to http://ip.jsontest.com

• Say, Task = Task1 = Task2 = Task 3 = 400ms

• Sync Modelo Time taken = Task1+ Task2 + Task3 = 1.2 sec

• Threaded Modelo Time taken = 510 ms

• Async Modelo Time taken = 460 ms

What’s the magic here?

Async Paradigm• Clients requests the event driven web server;

• requests are processed by event loop;

• event handlers cater to events with callbacks

Client Event driven server I/O loop

Event driven I/O loop

RequestIO loop handles request

Event Handlers

Reactor Pattern• Typical non blocking frameworks work on a

philosophy of single threaded event loop o keeps polling for events

Reactor Pattern

Waiting for Events

Handling Events

More Details!• Framework typically maintains a list of file

descriptors(fd), events to monitor and

corresponding event handlers for each of the fd

• Listening to events on a fd is a kernel space tasko epoll, [kqueue/select] – libraries provide event notifications in a non-

blocking way

• Epoll watches file descriptors (sockets) and returns

needed (READ, WRITE & ERROR) events

Async way• Async strategy aims for:

o Making I/O tasks non blocking

o I/O tasks would run independently

o generate an event when tasks are complete

o with help of callbacks

• Benefitso No need to wait till blocking I/O tasks are complete

o More responsive real time applications

o Thread safety isn't an issue

• Can we solve any other Python problems with this mechanism?o Eliminating GIL?

Async in Python• Frameworks

o Tornado

o Twisted

o Gevent

• Moduleso Tulip

o Asyncio

Asyncio• Part of Python library

o The latest module for async application development

• Only for Python > 3.4o Incompatible with prior versions

• A whole new way to developmento Let’s you write self contained, synchronous looking tasks

o Run two infinite loops at the same time on the same thread

• Works with other frameworko Tornado, Twisted, GEvent

Asyncio…• Write single threaded concurrent code

• Principle of Interleaved execution of subroutines

• Co-operative schedulingo Only one task at a time

• Based on libevento Select, kpoll, kqueue

Asyncio: Components

Event loop

Co-routines, Futures, Tasks

Transports, Protocols

Asyncio: Components• Event loop

o Register, executing and cancelling calls

o Schedule execution of a task (co-routine)

o Creates transport (async client and server)

o Runs I/O callbacks (Watches file descriptors)

o Thread interface

o [BaseEventLoop.create_task()] or async()

o [asyncio.get_event_loop()]

Asyncio: Components

• Co-routineo Generator (“yield from”)

o suspended at preset execution points, and

o resumed later by keeping track of local state

o @coroutine decorator

Asyncio: Components• Task

o responsible for executing a coroutine

o If coroutine yields from a future, the task suspends the execution of the

coroutine and waits for the future

o coroutine restarts when future is done

o Subclass of class Future

o [async(coroutine)]

o BaseEventLoop.create_task(coro)

Asyncio: Components• Future

o A class

o for results that are

available later

import asyncio

@asyncio.coroutine

def slow_operation(future):

yield from asyncio.sleep(1) <- Co-routine suspend

future.set_result('Future is done!')

def got_result(future):

print(future.result())

loop.stop()

loop = asyncio.get_event_loop() <- Event loop

future = asyncio.Future() <- Future object

asyncio.async(slow_operation(future)) <- Task

future.add_done_callback(got_result)

try:

loop.run_forever()

finally:

loop.close()

Asyncio: Components• transport

o represent connections such as sockets, SSL connection and pipes

o Async socket operations

• Usually frameworks implement e.g. Tornado

• protocolso represent applications such as HTTP client/server, SMTP, and FTP

o Async http operation

o [loop.create_connection()]

Example: Asyncio Redisimport asyncio

import asyncio_redis

@asyncio.coroutine

def my_subscriber(channels):

connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379)

subscriber = yield from connection.start_subscribe()

yield from subscriber.subscribe(channels)

while True:

reply = yield from subscriber.next_published()

print('Received: ', repr(reply.value), 'on channel', reply.channel)

loop = asyncio.get_event_loop()

asyncio.async(my_subscriber('channel-1'))

asyncio.async(my_subscriber('channel-2'))

loop.run_forever()

Example: Asyncio ‘Tasks’import asyncio

@asyncio.coroutine

def factorial(name, number):

f = 1

for i in range(2, number+1):

print("Task %s: Compute factorial(%s)..." % (name, i))

yield from asyncio.sleep(1)

f *= i

print("Task %s: factorial(%s) = %s" % (name, number, f))


tasks = [

asyncio.async(factorial("A", 2)),

asyncio.async(factorial("B", 3)),

asyncio.async(factorial("C", 4))]

loop.run_until_complete(asyncio.wait(tasks))

loop.close()

Async in Python• Frameworks

o Tornado

o Twisted

o Gevent

• Moduleso Tulip

o Asyncio

Tornado Async• Event Loop => tornado.ioloop

• Coroutine => tornado.gen.coroutine

• Future => tornado.concurrent.future

• Transport/Protocol => tornado.iostream

• Bridge the gap => tornado.platform.asyncio –

Combines asyncio and tornado in same event loop

Tornado Async Httpimport tornado.ioloop

from tornado.httpclient import AsyncHTTPClient

def handle_request(response):

'''callback needed when a response arrive'''

if response.error:

print("Error:", response.error)

else:

print(’Success')

print(response.body)

http_client = AsyncHTTPClient() # initialize http client

http_client.fetch(” http://ip.jsontest.com/", handle_request)

print("Before Event Loop Starts!")

tornado.ioloop.IOLoop.instance().start() # start the tornado ioloop

Callback

Before Event Loop Starts!Successb'{"ip": "117.192.252.80"}\n'

Tornado Coroutineimport tornado.web

import tornado.gen


class GenAsyncHandler(tornado.web.RequestHandler):

@tornado.gen.coroutine

def get(self):

http_client = AsyncHTTPClient()

response = yield http_client.fetch("http://google.com")

print(response)

application = tornado.web.Application([ (r"/",

GenAsyncHandler), ])

if __name__ == "__main__":

application.listen(8888)

tornado.ioloop.IOLoop.instance().start()

gen.coroutine schedules the generator

to be resumed when the Future is

resolved

‘yield’ makes the function a generator

The generator in turn returns a Future

instance

In this case, response, will resolve with

response from fetch or an exception

Tornado Engineclass MainHandlerAsync(tornado.web.RequestHandler):

@tornado.web.asynchronous

@tornado.gen.engine

def get(self):

req = tornado.httpclient.HTTPRequest("http://127.0.0.1:8888/",)

client = tornado.httpclient.AsyncHTTPClient()

response = yield tornado.gen.Task(client.fetch, req)

self.finish()

application = tornado.web.Application([

(r"/async", MainHandlerAsync),

])

if __name__ == "__main__":

http_server = tornado.httpserver.HTTPServer(application)

http_server.listen(8888)


Let’s create our Future!import time

import datetime

from tornado.concurrent import return_future

class AsyncHTTPClient(object):

@return_future

def fetch(self, url, callback=None):

print("In my fetch")

time.sleep(0.02)

result = str(datetime.datetime.utcnow())

callback(result)

import tornado.web

import tornado.gen

from myfuture import AsyncHTTPClient

def test(arg):

print('In test:' + arg)

class GenAsync(tornado.web.RequestHandler):

@tornado.gen.coroutine

def get(self):

http_client = AsyncHTTPClient()

r = yield http_client.fetch(“http://google.com”,test)

print(r)

application = tornado.web.Application([

(r"/", GenAsync),])

if __name__ == "__main__":

application.listen(8888)


myfuture.py server.py

Performance:Blocking vs Async

Work to be achieved

Performance Results

ab -n 500 -c 10 http://localhost:8888/blockingab -n 500 -c 10 http://localhost:8888/async

0

1000

2000

3000

4000

5000

6000

Async Blocking

Time per request

Async

Blocking

0

50

100

150

200

Async Blocking

Requests per second

Async

Blocking

http://localhost:8888/blocking

http://localhost:8888/async

Learnings• Async programming is an efficient, easy to

understand design and code

• Python asyncio module is comprehensive

• Has generic use cases for vast variety of

applicationso Responsive web applications

o Networking applications

• Requires a new way to program and design

Recommendations• Async programming is not a holistic solution

• It has its own pros and conso Suitable for primarily I/O bound applications

o Needs enough tasks available to run

• asyncio module is only available for Python 3

applications

• Also explore other methods of concurrency:o Eventlets

o STM

o Multiprocessing/threads

o Special languages e.g. GO, Scala

• Understand and use

References• asyncio – http://python.org

• Python asyncio –

o http://www.buzzcapture.com

o www.slideshare.net/saghul

• Tornado – http://tornadoweb.org

• Multithreading – www.drdobbs.com

• Event loop: https://docs.python.org/3/library/asyncio-eventloop.html

http://python.org

http://www.buzzcapture.com

http://www.slideshare.net/saghul

http://tornadoweb.org

http://www.drdobbs.com/

https://docs.python.org/3/library/asyncio-eventloop.html

Contact Us

• Chetan Giridharo www.technobeans.com

o https://github.com/cjgiridhar

• Vishal Kanaujiao www.freethreads.wordpress.com

o https://github.com/vishalkanaujia

http://www.technobeans.com/

https://github.com/cjgiridhar

http://www.freethreads.wordpress.com

https://github.com/cjgiridhar

Backup

Asyncio: exampleimport asyncio

@asyncio.coroutine

def create():

yield from asyncio.sleep(3.0)

print("(1) create file")

@asyncio.coroutine

def write():


print("(2) write into file")

@asyncio.coroutine

def close():

print("(3) close file")

@asyncio.coroutine

def test():

asyncio.async(create())

asyncio.async(write())

asyncio.async(close())


loop.stop()


asyncio.async(test())

loop.run_forever()

print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop))

loop.close()

Python Async Modules• Asynccore

• Asyncchat

• Gevent

• Twisted

• Eventlets

Concurrency Techniques• Multithreading/processing

• Green Threads

• STM

Tornado + AsyncIOfrom tornado.platform.asyncio import AsyncIOMainLoop


import asyncio

AsyncIOMainLoop().install() -- # Tell Tornado to use the asyncio eventloop

loop = asyncio.get_event_loop() -- # get the loop

http_client = AsyncHTTPClient() -- # the Tornado HTTP client

def aio_fetch(client, url, **kwargs):

fut = asyncio.Future()

client.fetch(url, callback=fut.set_result, **kwargs)

return fut

@asyncio.coroutine

def main():

print("fetching my site")

mysite = yield from aio_fetch(http_client, "http://google.com/")

print("hello httpbin")

httpbin = yield from aio_fetch(http_client, "http://httpbin.org?q=%d" % mysite.code)

print(httpbin.body)

loop.run_until_complete(main())

Co-routine v/s Callbackimport asyncio

def just_print_messages(loop):

print('Just print')

loop.call_later(1, just_print_messages, loop)

def main():


try:

loop.call_soon(just_print_messages, loop)

loop.run_forever()

finally:

loop.close()

if __name__ == '__main__':

main()

import asyncio

@asyncio.coroutine

def just_print_messages():

while True:

print('Just print')

yield from asyncio.sleep(1)

def main():


try:

loop.run_until_complete(just_print_messages())

finally:

loop.close()

if __name__ == '__main__':

main()

Async in NodeJS

var http = require(‘http’);

var server = http.createServer;

server.on(‘request’, function(request,response) {

response.writeHead(200);

response.end(‘Hello World’);

}).listen(8001);

console.log(‘Server running on port 8001’);

Callback

request is an event

async programming and python

Engineering