S

High 6 CDR

Group Members

• Kirk Chan• Brian Troili• Ali Mizan• Laura Rubio-Perez

Project Introduction

Motivation

• Fresh idea to the UCF community• This project has the potential to help the speech impaired• Based on the research, technologies necessary were

interesting– Machine learning– Personal taste

Goals

We want the following key factors:

HARDWARE SOFTWARE

● Low-power ● Android

● Lightweight ● Accurate & fast translations

● Wireless ● Support text & audio translations

Specifications

Component Design Specification

Hand gesture translation Less than 1s

Glove weight Less than 1.5lbs

Battery life (from one charge) At least 5hr

Application startup Within 3s

Setup Bluetooth connection Within 10s

Bluetooth connection range Up to 50m

Default gesture library American 26 letters, 10 numbers

Design Approach

Design Overview

Hardware Components

• Flex Sensors▪ Able to detect changes in bend/flex▪ Changes its resistance at several points

along the device▪ When a current is applied, it creates a

voltage divider

Hardware Components

Hardware Components

• Pressure Sensors▪ Acts as a force sensing resistor▪ When the sensor is unloaded, its

resistance is very high▪ When pressure is applied, its

resistance decreases

Hardware Components

Analog/Digital Converter (ADC)

• Serial communication preferred.• Large number of input channels.• Avoid serial address conflict.

- ADS7828➔ I2C compatible➔ 8 Channel ADC➔ variable I2C address

Hardware Components

Analog/Digital Converter (ADC)

• Serial communication preferred.• Large number of input channels.• Avoid serial address conflict.

- ADS7828➔ I2C compatible➔ 8 Channel ADC➔ variable I2C address

Hardware Components

Accelerometer and Gyroscope

– Inertial Measurement Unit (IMU)– Speed demand allow for serial buses.

- ITG3200/ADXL345 combo board➔ 3.3V input➔ I2C compatible➔ 3 axis each➔ calibrate to 2, 4, 8, and 16g

Hardware Components

Wireless Communication

Wifi Bluetooth NFCSecure connection Portable connection Low power

Approx. 100m range Approx. 100m range Approx. 7-8in

Hardware Components

Wireless Communication

Wifi Bluetooth NFCSecure connection Portable connection Low power

Approx. 100m range Approx. 100m range Approx. 7-8in

Bluetooth Low Energy (BLE)

● Low power consumption● Approx. 50m range

Hardware Components

BLE TTL Transceiver

• Bluetooth v4.0

• 3.3V input voltage

• Approximately $6

• Customizable Baud Rate

Hardware Components• Microcontroller

MSP430 ATmega328p ATmega32u4Operating Voltage 3.3V 5V 5.5V

Digital I/O Pins 12 14 12

Analog Input Pins 12 6 12

Clock Speed 16MHz 16MHz 16MHz

Architecture RISC RISC RISC

Hardware Components

• Microcontroller

MSP430 ATmega328p ATmega32u4Operating Voltage 3.3V 5V 5.5V

Digital I/O Pins 12 14 12

Analog Input Pins 12 6 12

Clock Speed 16MHz 16MHz 16MHz

Architecture RISC RISC RISC

Hardware Components

• Development Environment

Arduino IDE Code Composer

Free Free

Open source (Unlimited support ) Limited support

Extensive libraries Direct access to control registers

avr-gcc compiler C/C++ compilers and assembly

Hardware Components• Development Environment

Arduino IDE Code Composer

Free Free

Open source (Unlimited support ) Limited support

Extensive libraries Direct access to control registers

avr-gcc compiler C/C++ compilers and assembly

Hardware components

• Li-ion Batteries

▪ Small size and lightweight▪ High energy density

▪ Capacity gradually declines▪ Can drop below regulated voltage

Hardware components

• Switching Regulator

Advantages:❏ Efficiency❏ Minimal power dissipated❏ Minimal switch duty-cycle

Disadvantages:❏ Voltage ripple❏ Complexity of external passive

components on board

Hardware components

• LM2576 Switching Regulator:

● Step down switching regulator● Vin range: 4V - 40V● Vout range: 3.3V - 37V

PCB

Software Components

Android vs iPhone

• Android– Can be developed on Windows,

Mac, and Linux– Apps written in Java

• iPhone– Can only be developed in Mac– Apps written in objective C– Apple development software

only works with other apple development software

Android IDEs

IntelliJ (free version)• Advantages:

– Less buggy– More intuitive– Faster– Better GUI

• Disadvantages:– Java, Groovy, or Scala are only 3

languages supported in free version

Eclipse• Advantages:

– More plug-ins available– More commonly used

• Disadvantages:– Has bugs and crashes a lot

Application Features

Application Features

Class Diagram

Software Components

• Two main components: – Android Application

• is the interface between the user and the machine learning algorithm– Takes in raw data from glove– Displays letter on screen

– Translator• There is no way to learn every single sign language gesture with 100%

accuracy. • Machine learning gives ~95% accuracy. • Uses learning algorithm to learn from examples

Machine Learning

• General Overview– Uses data in order to approximate

target function– Uses examples to determine

which hypothesis is closest approximation of unknown target function

• 3 popular types – Regression– Classification– Clustering

Machine Learning Algorithm chosen

• Advantage of Hidden Markov Model– No need for segmentation– Very robust towards small changes in

motion– History of being used for language

recognition

Facts and image taken from IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 12, DECEMBER 1998

BudgetDESCRIPTION QTY PRICE

Flex sensors 11 $88.00

3-Axis Gyro/Accelerometer 1 $13.00

Pressure sensors 4 $75.00

Glove 1 $20.00

Bluetooth module 1 $12.00

Li-Ion Batteries 2 $12.00

Recharging Battery Dock 1 $15.00

2-Layer PCB 1 $33.00

PCB 1 $200.00

Microcontroller 1 $50.00

Boeing sponsorship - - $518.00

TOTAL - $0.00

Financing up to date • Boeing Sponsorship

DESCRIPTION QTY PRICE

Flex sensors 2.2” 11 $79.50

IMU Digital Combo Board (3-Axis Gyro/Accelerometer) 1 $51.96

Flexiforce Pressure Sensors 4 $79.80

Bluetooth module (Free) 1 $0.00

Microcontroller (Atmega 328P) 1 $37.97

TOTAL - $249.23

Boeing sponsorship - - $518.00

Total Available from Sponsorship - $268.77

Progress

Issues

• The Hidden Markov Model is very complicated to both understand and implement

• Training the algorithm• Varying hand sizes

• BLE is relatively new and requires more research

Approaching the issues

• Implementation of Hidden Markov• Reading and researching

• Varying Hand size issue• Smooth the trajectory, hand shape, and orientation• Creating tolerances for hand gestures (for flexion)

• BLE• Research or switching to classic Bluetooth (version 3.0)

Questions?