async python

Download Async Python

Post on 25-May-2015

245 views

Category:

Data & Analytics

8 download

Embed Size (px)

TRANSCRIPT

  • 1. Python

2. C10K 10 . 3. C10K 1 2 4. , nginx, which relies on an event-driven (asynchronous) architecture, instead of threads, to handle requests Lighttpd, which relies on an asynchronous architecture to handle requests Tornado, a non-blocking web server and web application framework written in Python (used by Facebook's FriendFeed) Apache AWF (retired, formerly Apache Deft), asynchronous, non-blocking web server running on the JVM JBoss Netty, a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients Node.js, asynchronous, non-blocking web server running on Google's V8 JavaScript engine 5. , EventMachine, an asynchronous, non-blocking web server running on Ruby EventMachine Cowboy (web server), another very lightweight web server written in Erlang asyncore (in the standard Python library), a non- blocking web server library. It is based on Medusa, which is no longer maintained. Django, some research is being done using the asynchronous IO support in Python 3.3 6. sync VS async 7. Sync . 8. Sync: 1 thread . , . 9. Sync: N threads " " , . . 10. Async . ( ) . 11. sync VS async , : 1. , , . 2. -, , , , . 3. . 12. Event-driven programming - : . 13. Reactor , , - . , . . , , 14. Async-servers in Python Name Lic. Doc Ex. Prod. Com. Act. Blog Twt Rep. Pool Wsgi Scket Cmet Epoll Test Style Twisted! MIT! Yes! Yes! Yes! Huge! Yes! Lots! No! Trac! Yes! Yes! Yes! No! Yes! Yes! Callback! Tornado! Apache! Yes! Yes! F.Feed!Yes! Yes! FB! Yes! GHub! No! Lim.! Yes! No! Yes! No! Async! Orbited MIT Yes Yes Yes Yes Yes Yes No Trac No No Yes Yes Yes Yes Callback DieselWeb BSD Yes Yes STalk Yes Yes Yes Yes BitB. No Lim. Yes Yes Yes No Generator MultiTask MIT Some No No No No Yes No Bzr No No No No No No Generator Chiral GPL2 API No No IRC No No No Trac No Yes Yes Yes Yes Yes Coroutine Eventlet! MIT! Yes! Yes! S. Life! Yes! Yes! Yes! No! BitB.! Yes! Yes! Yes! No! Yes! Yes! Greenlet! FriendlyFlow GPL2 Some One No No No No Yes Ggle No No Yes No No Yes Generator Weightless GPL2 Yes No Yes No No No Yes SF No No Yes No No Yes Generator Fibra MIT No No No No No Yes No Ggle No No Yes No No No Generator Concurrence! MIT! Yes! Yes! hyves!Yes! Yes! No! No! GHub! No! Yes! Yes! No! Yes! Yes! Tasklet! Circuits MIT Yes Yes Yes Yes Yes Yes Yes Trac No Yes Yes No No Yes Async http://nichol.as/asynchronous-servers-in-python 15. Async-servers in Python What License does the framework have? Does it provide documentation? Does the documentation contain examples? Is it used in production somewhere? Does it have some sort of community (mailinglist, irc, etc..)? Is there any recent activity? Does it have a blog (from the owner)? Does it have a twitter account? Where can i nd the repository? Does it have a Thread Pool? Does it provide access to a TCP Socket? Does it have any Comet features? Is it using EPOLL? What kind of server is it? (greenlets, callbacks, generators etc..) http://nichol.as/asynchronous-servers-in-python 16. Tornado , - , Python. FriendFeed; Facebook 2009 Tornado. 17. Tornado import tornado.httpserver import tornado.ioloop import tornado.options import tornado.web import logging from tornado.options import define, options ! define("port", default=8888, help="run on the given port", type=int) ! class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world!") ! def main(): tornado.options.parse_command_line() application = tornado.web.Application([ (r"/", MainHandler), ]) http_server = tornado.httpserver.HTTPServer(application) http_server.listen(options.port) tornado.ioloop.IOLoop.instance().start() ! if __name__ == "__main__": main() http://programmingzen.com/2009/09/13/benchmarking-tornado-vs-twisted-web-vs-tornado-on-twisted/ 18. Twisted Twisted TCP, UDP, SSL/TLS, IP Multicast, Unix domain sockets, , HTTP, XMPP, NNTP, IMAP, SSH, IRC, FTP . - , , Twisted , . 19. Twisted from twisted.internet import epollreactor epollreactor.install() from twisted.internet import reactor from twisted.web import server, resource ! ! class Simple(resource.Resource): isLeaf = True def render_GET(self, request): return "Hello, world!" ! ! site = server.Site(Simple()) reactor.listenTCP(8888, site) reactor.run() http://programmingzen.com/2009/09/13/benchmarking-tornado-vs-twisted-web-vs-tornado-on-twisted/ 20. Twisted 21. Twisted - . Twisted , . , . , . , . callback , , . callback', 22. ? 23. Errbacks , . -- , . 24. Sync VS Async 1. :! try:! text = run_print()! except Exception, err:! print err! sys.exit()! else:! print text! sys.exit()! ! ! 2. :! def print_ok(text):! print text! reactor.stop()! ! def print_fail(err):! print >>sys.stderr, err! reactor.stop()! ! run_print(print_ok, print_err)! ! reactor.run() 25. Deferred from twisted.internet.defer import Deferred! ! def print_ok(text):! print text! ! def print_fail(err):! print >>sys.stderr, err! ! d = Deferred()! d.addCallbacks(print_ok, print_fail)! d.callback("Text is correct") ! # d.errback(Exception('I have failed.'))! print "End."! ! reactor.run() 26. Deferred 27. Twisted from twisted.internet import reactor! from twisted.internet.protocol import ServerFactory ! from twisted.protocols.basic import LineOnlyReceiver ! ! class ChatProtocol(LineOnlyReceiver): ! ! name = "" ! def getName(self): ! if self.name!="": ! return self.name ! return self.transport.getPeer().host ! ! def connectionMade(self): ! print "New connection from "+self.getName() ! self.sendLine("Welcome to my my chat server.") ! self.sendLine("Send '/NAME [new name]' to change your name.") ! self.sendLine("Send '/EXIT' to quit.") ! self.factory.sendMessageToAllClients(self.getName()+" has joined the party.") ! self.factory.clientProtocols.append(self) 28. Twisted ! ! def connectionLost(self, reason): ! print "Lost connection from "+self.getName() ! self.factory.clientProtocols.remove(self) ! self.factory.sendMessageToAllClients(self.getName()+" has disconnected.") ! ! def lineReceived(self, line): ! print self.getName()+" said "+line ! if line[:5]=="/NAME": ! oldName = self.getName() ! self.name = line[5:].strip() ! self.factory.sendMessageToAllClients(oldName+" changed name ! ! ! ! ! to "+self.getName()) ! elif line=="/EXIT": ! self.transport.loseConnection() ! else: ! self.factory.sendMessageToAllClients(self.getName()+" says "+line) ! ! def sendLine(self, line): ! self.transport.write(line+"rn") ! ! 29. Twisted class ChatProtocolFactory(ServerFactory): ! ! protocol = ChatProtocol ! ! def __init__(self): ! self.clientProtocols = [] ! ! def sendMessageToAllClients(self, mesg): ! for client in self.clientProtocols:! client.sendLine(mesg) ! ! print "Starting Server"! factory = ChatProtocolFactory()! reactor.listenTCP(12345, factory)! reactor.run()! : telnet localhost 12345 30. Async I/O for Python 3 (. coroutine) , , ( ) . 31. Yield from ! ! class BinaryTree_ForLoop:! def __init__(self, left=None, us=None, right=None):! self.left = left! self.us = us! self.right = right! ! def __iter__(self):! if self.left:! for node in self.left:! yield node! if self.us:! yield self.us! if self.right:! for node in self.right:! yield node! class BinaryTree:! def __init__(self, ! left=None, us=None, right=None):! self.left = left! self.us = us! self.right = right! ! def __iter__(self):! if self.left:! yield from self.left! if self.us:! yield self.us! if self.right:! yield from self.right! ! # For comparison, here is the same thing using for-loops! # instead of yield-from. 32. Tulip VS Twisted import tulip! from tulip import http! ! @tulip.coroutine! def download(url):! response = yield from http.request('GET', url)! for k, v in response.items():! print('{}: {}'.format(k, v[:80]))! ! data = yield from response.read()! print('nReceived {} bytes.n'.format(len(data)))! ! if __name__ == '__main__':! loop = tulip.get_event_loop()! coroutine = download('http://omegafeihong.tumblr.com')! loop.run_until_complete(coroutine) 33. Tulip VS Twisted from twisted.internet import reactor! from twisted.internet.defer import Deferred, succeed! from twisted.internet.protocol import Protocol! from twisted.web.client import Agent! ! def print_headers(response):! for k, v in response.headers.getAllRawHeaders():! print('{}: {}'.format(k, v[0][:80]))! ! return get_response_body(response)! ! def get_response_body(response):! class BodyReceiver(Protocol):! def dataReceived(self, data):! chunks.append(data)! def connectionLost(self, reason):! finished.callback(''.join(chunks))! ! finished = Deferred()! chunks = []! response.deliverBody(BodyReceiver())! return finished! ! def print_body(data):! print('nReceived {} bytes.n'.format(len(data)))! return succeed(None)! ! if __name__ == '__main__':! agent = Agent(reactor)! d = agent.request('GET', 'http://megafei d.addCallback(print_headers)! d.addCallback(print_body)! d.addCallback(lambda x: reactor.stop())! reactor.run() 34. http://ninaevseenko.github.io/async_twisted_ru/async_twisted_ru.pdf http://legacy.python.org/dev/peps/pep-3156/#abstract http://moscowdjango.ru/meetup/18/tulip/ http://neednourishment.blogspot.ru http://www.ibm.com/developerworks/ru/library/l-python_part_10/index.html http://www.slideshare.net/Smirnov.Andrey/twisted-framework-python-2211313 http://moscowdjango.ru/meetup/14/gil-and-python-why/ http://legacy.python.org/dev/peps/pep-0380/ http://legacy.python.org/dev/peps/pep-3156/#event-loop-classes http://ru.wikipedia.org/wiki/%D1%EE%E1%F