ECE477 Project MinotaurSoftware Narrative

Jon Roose

Project Overview

• Home Security Robot• Interaction through a Webserver• Robot driven manually through house and

spots intruders• Relays video to web interface using kinnect

Hardware Hierarchy

Software Hierarchy

Benefits of Model

• Reliance upon open source projects• Ease of access, available to HTML5 browsers• Emphasis on x86 architecture• Modularity• Scalability

Software Hierarchy - Web Interface

Software Hierarchy - Web Interface

• Apache Web Server– Serves interface to user’s web browser

• HTML5 and JavaScript WebSockets– Opens efficient communication with C&C Server– Real time commands, removes HTTP reliance

• RDP Video Stream– Implemented and configured through FFServer– Connects via an embedded video object

Software Hierarchy - C&C Server

Software Hierarchy - C&C Gateway

• C&C Server (Written in Python)– Mediator for command and control signals– Communicates on internal LAN with robot– Simple websocket interface with User’s browser– Leverages TCP/IP LAN interface to communicate

with robot via WiFi• FFServer (Open source application)– Accepts video from Atom board and serves to

user, manages RDP video stream

Software Hierarchy - Atom Board

Software Hierarchy – Atom Board

• Intelligent Drive System (C++)– Navigates room and avoids obstacles– Accepts commands on a TCP/IP socket from C&C– Interprets sensor data from Kinect and PIC24– Converts video into YUV encoding for FFMpeg

• FFMpeg (Open source application)– Accepts data via stdin from the Intelligent Drive

system and packages it in an MPEG video format

IDS Software Architecture

• Thread 1 - 30 FPS event loop– Transcodes and Outputs video to FFMpeg– Polls Kinect Video and Depth Sensor– Polls network socket for C&C Commands

• Thread 2 - Queue driven event loop– Implements room mapping and human detection– Timers add standard events to queue– Flexible queue design allows adjusting task frequency– Communicates with PIC24 over RS232

Software Hierarchy - Microcontroller

Software Hierarchy - MicroController

• Low Level Peripheral Controller (C)– Implements fail-safe obstacle avoidance– Accepts commands via 5 byte protocol over RS232– Controls motors and polls IR sensors– Flag driven event loop

• Libfreenect & OpenNI (Open source)– Library for communicating with Kinect– Retrieves depth sensor and video data– Recognizes human form

Debugging

• Hardware/MicroController– Standard Reset pushbutton and debug LED– RJ11 allows for in-circuit debugging via ICD3– Headers for port pins broken out– Printing strings over SCI

• Atom Board– SSH directly into board– Sending metrics and debugging data to C&C– Sending bitmaps of room map