building an inflight entertainment system controller in twisted
TRANSCRIPT
TWISTED AS AN IOTCONTROLLER.
BUILDING A 300+ SEAT IFE CONTROLLER IN TWISTED. / / David P. Novakovic @dpn dpn.name
QUICK OVERVIEWWTF is Twisted?IFE System DevicesIFE System requirementsCode examples
TWISTED THE SLAYER OF NOOBSSUPER POWERFUL
ALSO CONTROVERSIALSeems to have a bad rep with people who don't use it.
TWISTEDevent driven framework in pythonparticular focus on networking related thingsfrom web down to serial ports, tun/tap and udevasync io - single process can handle thousands of concurrentopen connections on a low end machineClean APIs for both callbacks and deferredsthread pool with deferred interfacelocking "primitives" - DeferredSemaphore, DeferredLock,DeferredQueue
MORE TWISTEDApplication frameworkAsync unit testingIntegration with most major event loops(wx/gtk/qt/gevent/etc)third party libsklein - sinatra/flask style web frameworkcyclone - tornado port to twistedtreq - "requests" style libampoule - nice enough (if slightly neglected) process poolimplementation
IFE STUFF.. YAY
TRADITIONAL"dumb" screens heavy use of streaming contentvery heavy requirements on serversvery expensive to upgrade
GLIDEHeavier clientssystem can be upgraded by replacing playersplane doesnt need rewiring etcserver can remain fairly unchangedmore importantly don't need to be upgraded in lockstep
GLIDE SOLUTIONplayers - embedded debianseat disconnect - custom embedded devicetriple redundant power supplies - switch + powercm controller - dual core atom running ubuntu LTS + lots ofembedded things attached
SYSTEM
SOME RELEVANT REQUIREMENTSmulticast commands out to embedded devices300+ seat updates over HTTP every secondlisten to audio stream over multicast (PA, pilot etc)low latency control of players ie. if pilot talks/decomptelneting into a streaming VLC process.some legacy - sync code needed to run in threadsrespond to and control hardware in the plane (overheadscreens etc)cabin crew inserting HDD with content (lock down usb)downloading content from the web (at gate)kiosk (lock down control keys/usb ports)manhole for debugging a running processssh reverse tunnel for remote access - conchtftp - firmware updates to players
MULTICASTfrom twisted.internet import reactor, task, protocol
class MulticastSeatControl(protocol.DatagramProtocol):
def startProtocol(self): self.transport.joinGroup("228.0.0.5")
def sendSeatControl(self, bytes): self.transport.write(bytes, ("228.0.0.5", 8005))
def datagramReceived(self, datagram, address): print "Datagram %s from %s" % (repr(datagram), repr(address))
HTTP INTERFACEimport jsonfrom klein import Klein
class DeviceAPI(object): app = Klein()
def __init__(self, cmc): self._cmc = {}
@app.route('/stat/<string:name>') def stat(self, request): body = json.loads(request.content.read()) self._cmc.logstat(body['something']) request.setHeader('Content-Type', 'application/json') return json.dumps({"status": "success"})</string:name>
SERIAL PORTfrom twisted.internet.serialport import SerialPortfrom twisted.internet import reactorfrom twisted.protocols.basic import LineReceiver
class SensorProtocol(LineReceiver): def connectionMade(self): print "Connected to serial port."
def lineReceived(self, line): print "Received line:", line
def send_our_command(self, command): self.sendLine("command:%s" % command)
def connect_serial_port(): return SerialPort(SensorProtocol(), "/dev/ttyACM0", reactor, 115200, rtscts=False, xonxoff=False, timeout=1)
DJANGO
WAIT.. WHAT?Yep, ORM is used a fair bit, as is admin.
DJANGOfrom django.core.handlers.wsgi import WSGIHandlerfrom twisted.python import threadpoolfrom twisted.internet import reactorfrom somewhere import Root
def gimme_some_django(): pool = threadpool.ThreadPool() wsgi_resource = wsgi.WSGIResource(reactor, pool, WSGIHandler()) r = Root(wsgi_resource) s = server.Site(r) pool.start() return internet.TCPServer(8082, s)
eg. django admin available at http://localhost:8082/admin/
https://github.com/clemesha/twisted-wsgi-django
YET MOREclass VLCRemoteControlProtocol(LineReceiver): ...
class UDevMonitor(abstract.FileDescriptor): ...
AND TELNET INTO YOUR OWN PROCESS..import twisted.manhole.telnetfrom twisted.internet.endpoints import TCP4ServerEndpoint
def start_manhole(service): f = twisted.manhole.telnet.ShellFactory() f.username = "b" f.password = "b" f.namespace["var1"] = service endpoint = TCP4ServerEndpoint(reactor, 7777) endpoint.listen(f)
TIE IT ALL TOGETHERApplication frameworkUnit Testing
APPLICATION FRAMEWORKTWISTD
Daemonises optionallypidfilelog filesError catchallhttp://twistedmatrix.com/documents/13.2.0/core/howto/basics.html
CREATE SERVICESclass IOTCServer(Service): def startService(self): self.serialport = connect_serial_port() p = MulticastSeatControl() reactor.listenMulticast(8005, p) start_manhole(self) self.mc_looper = task.LoopingCall(p.sendSeatControl, "s1:on,s2:on") self.mc_looper.start(1)
def stopService(self): self.serialport.transport.loseConnection() self.mc_looper.stop()
def get_http_service(iots_instance): device_api = DeviceAPI(iots_instance) server.Site(device.api.app.resource()) return internet.TCPServer(8082)
TWISTD PLUGINclass Options(usage.Options): optParameters = [["serialport", "s", "/dev/ttyACM0", "Serial port."]
class ServiceMaker(object): tapname = "iotc" description = "The IOTC Server" options = Options
def makeService(self, options): service_parent = service.MultiService() iots = IOTCServer() iots.setServiceParent(service_parent) http_service = get_http_service() http_service.setServiceParent(service_parent) django_service = gimme_some_django() django_service.setServiceParent(service_parent) return service_parentserviceMaker = ServiceMaker()
yourproject/twisted/plugins/iotc_plugin.py$ twistd -r epoll -n iotc --serialport=/dev/ttyACM0
UNIT TESTINGTwisted TrialWrapper around pyunit "unittest"Allows tests to return deferred responsessubclass twisted.trial.unittest.TestCaserun tests: trial yourpackageSome helpers to let you mock wire-level connectionshttp://twistedmatrix.com/documents/11.1.0/core/howto/trial.html
BUILD AWESOME THINGS!DIGECOR IS HIRING
If you enjoy working on this kind of stuff, let me know and I'llforward your details onto digEcor.
@dpn
THE ENDDAVID NOVAKOVIC - DPN.NAME