department of electrical and computer engineering true …wayneliu/milestones and worklogs and...

Department of Electrical Engineering

Department of Electrical and Computer Engineering

True 3D Image Manipulator

ELEC/CENG 499 – Design Project

Final Report

Dept. Electrical and Computer Engineering

University of Victoria

All rights reserved. This report may not be reproduced in whole or in part, by photocopy or other

means, without the permission of the author.

Prepared By: Fangzheng Song

Nelson Lee

Candace Chen

Yang Liu

Tristan Sturgess

V00700925

V00688530

V00723040

V00735027

V00684498

[email protected]

[email protected]

[email protected]

[email protected]

[email protected]

Supervisors: Dr. Kin Fun Li

Group No.: 16

Due Date: April 4, 2014

ELEC/CENG 499 – Final Report


Abstract

The use of LEDs to create images has been widely used in modern society. Nowadays, large flat

screen advertisement can be seen everywhere, in terms of public information display and

business advertising, playing a crucial role of disseminating messages. However, ordinary LED

flat screen has limitation on dealing with 3D images due to the planar structure. With the

development of technology and the needs of advertising market, people can expect an application

of displaying 3D image so advertising companies can bring about more creative ad design. In

this case, the LED cube can easily meet the criteria of a 3D display. Compared with a LED

screen, the LED cube has a lower resolution, but its true 3D image creation ability is not

something a normal LED flat screen can achieve.

For the ELEC/CENG 499 project design course, our team will implement a continuation of our

ELEC 399 project. The 3D image manipulator is a system that displays a 3D image which can be

manipulated by intuitive hand gestures. Due to time and budget limitations, a very high

resolution LED cube would be very difficult to construct. However, this project will focus on

providing a proof of concept for the True 3D image manipulation technology. During the

demonstration day, many basic manipulations were shown to be done through an 8x8x8 LED

Cube. Other than the LED Cube the Leap Motion and Arduino are two of the other major

components in this project. The Leap motion is responsible for detecting hand motion and the

Arduino is responsible for transmitting the control signals to the LED cube. Overall, this report

will explain each part of the project, including project research, implementation plan, project

milestones, alternatives and detail project design. The team hopes readers could gain a deep

understanding on this true 3D image manipulator project and raise interest on true 3D image

technology.


TABLE OF CONTENTS

1 Introduction ............................................................................................................................. 1

1.1 TIM-3D Team Members ................................................................................................. 2

1.2 Supervisor ....................................................................................................................... 2

2 Project Goals ........................................................................................................................... 2

2.1 3D Image Manipulator .................................................................................................... 3

2.2 3D Game ......................................................................................................................... 3

3 Project Research...................................................................................................................... 4

3.1 LED Cube Research ........................................................................................................ 4

3.2 Alternative 3D Displays .................................................................................................. 5

3.3 Arduino Research............................................................................................................ 5

3.4 Leap Motion Research .................................................................................................... 6

4 Early Project Plan Description ................................................................................................ 7

4.1 Project Implementation Strategy..................................................................................... 7

4.2 Milestones ....................................................................................................................... 8

4.3 Project Cost List .............................................................................................................. 9

4.4 Design Alternatives Comparison (Pros and Cons) ....................................................... 10

4.4.1 Cube – UI Connector – Java – Leap Motion ............................................................ 11

4.4.2 Cube – Arduino – Processing – Leap Motion ........................................................... 11

4.4.3 Cube – Arduino – Java – Leap Motion ..................................................................... 11

5 Project Detail Designs........................................................................................................... 12

5.1 LED Cube Initialization ................................................................................................ 12

5.2 Image Creation on Arduino .......................................................................................... 14

5.3 Arduino Interface .......................................................................................................... 17

5.4 Image Management ....................................................................................................... 18

5.5 Leap Motion Detection ................................................................................................. 18

5.6 3D Game Design ........................................................................................................... 18

5.6.1 Concepts of Pong 3D ................................................................................................ 19

5.6.2 Pong 3D Program ...................................................................................................... 20

6 Workload Distribution & Achievements .............................................................................. 21


7 Website Design ..................................................................................................................... 22

8 Project Discussion ................................................................................................................. 23

9 Conclusion ............................................................................................................................ 24

10 APPENDIX A (Java Program) ............................................................................................. 25

11 APPENDIX B (Game Code) ................................................................................................ 15

12 APPENDIX C (Cube Construction) ....................................................................................... 8

13 REFFERNCES ........................................................................................................................ 5

Page : 1 of 25


1 INTRODUCTION

True 3D effect has always been the ultimate goal of display technology. With the growing

popularity of pseudo-3D screens that can be found in movie theatres, TV’s, and mobile devices

there is growing pressure to make displays that are truly 3 dimensional. It is necessary to point

out that there are still some significant problems with this pseudo-3D technology. For example,

when the angle is changed between a person's eyes and the pseudo-3D screen the three

dimensional image will disappear. A truly 3D image will not be constrained by the same issues.

As the technology for 3D displays improve a new window of opportunity for 3D manipulation is

also emerging in the market.

It is our hope that we can create a cheap and effective way of initial prototyping that does not

require the user to spend lots of money on a solid prototype. Our solution to this problem is to

create a 3D display that will present the model and allows the user to interact with it using simple

hand gestures. We would also like to showcase the potential of this 3D image manipulator

project by rebooting some of the classic games such as Pong, Snakes, and/or Space Invaders, but

adding a third dimension.

The idea for this project came from the movie Iron Man 3, where Tony Stark was able to create a

3D projection of his suit and interact with that model. By using a similar concept, we propose to

create a True 3D Image Manipulator (TIM-3D) that can be used to manipulate a model by using

simple hand gestures. This would be done by displaying the model on an 8x8x8 LED Cube that

would be controlled by hand gestures detected with a Leap Motion. The LED cube will be

constructed by using 512 LED’s that are positioned in an 8x8x8 grid. This LED cube then

communicates to a computer that is programmed with a 3D image. By using the Leap Motion,

which is able to detect motion in its spatial grid, we will be able to manipulate the 3D image by

using intuitive hand gestures.

Page : 2 of 25


1.1 TIM-3D Team Members

Nelson Lee - is a 4th year Electrical Engineering student taking a Biomedical option at the

University of Victoria. Nelson has done many coop’s from a number of companies such as the

Genome Science Center and Power Grid Specialist Corporation. Using his past experiences

Nelson has contributed his unique programing and organization skills to this project.

Fangzhen Song - is a 4rd year electrical engineering student. He is interested in electrical power

systems. He believes that the 3D image manipulator will be widely used for business

presentations, teaching, and advertisements. Working at AIM and BC Hydro provides him a

solid experience on project management and programming skills. This experience is very helpful

for his involvement in the design of the project. In his spare time, he enjoys watching soccer and

playing DOTA.

Wayne Liu - is a 4th year Computer Engineering student who has done two co-op terms as a

system administrator assistant. Wayne is good at system network, research, debug and analysis.

Wayne also creates the contribute ideas on the project.

Tristan Sturgess - is a 4th year Electrical Engineering student specializing in digital signal

processing. He has done a co-op as a software developer and a co-op developing VOIP

components for routers. He brought his programming skills and experience to the project.

Candace Chen - is a 4th year Electrical Engineering student who has done one co-op as a junior

process engineer at a research, development, and manufacturing establishment “Silian Sapphire

Corp”. Candace used her research and design specialties in contribution to the completion of

TIM-3D.

1.2 Supervisor

Dr. Kin Fun Li - the TIM-3D would like to thank Kin Fun Li, the Director of Computer

Engineering Program, for his supervision and patient.

2 PROJECT GOALS

It is our hope that we can create cheap and cost effective way of initial prototyping that does not

require the user to spend lots of money on a solid prototype. Our solution to this problem is to

create a 3D display that will present the model and allows the user to interact with it by using

Page : 3 of 25


simple and intuitive hand gestures. We would also like to showcase the potential of this 3D

image manipulator by rebooting some classic games such as Pong, Snakes, and/or Space

Invaders, but adding a third dimension.

2.1 3D Image Manipulator

Our primary goal for the TIM-3D project is to manipulate a 3D image that is displayed in an

LED cube. This would be done by using an 8x8x8 LED Cube that would interface with a

computer that uses a product called Leap Motion. Using the Leap Motion, which is able to detect

motion in its spatial grid, we will be able to manipulate the 3D image using intuitive hand

gestures. This will enable the user to examine a model before they have it prototyped.

For this goal there are a large number of tasks that would need to be completed. The major

components of this goal will require the team to interface the LED cube to an Arduino

microcontroller, the Arduino microcontroller to a computer, and the computer to Leap Motion.

Once all of these connections have been made care will have to be taken to design the methods

of control and display of the model.

2.2 3D Game

Our secondary goal will be to make a 3D game that can be played on LED cube and controlled

by the Leap Motion hardware. Due to the limitations of the LED Cube and time constraints, it

will be very unlikely to produce a commercially viable product able to display high resolution

models. The 3D image manipulator is still a viable goal; however, the hardware required for high

resolution models is beyond our budget. Therefore, our goal of making some 3D games will

allow us to create a fun way of showcasing the potential of the 3D manipulation on the LED

cube. Since many old retro games use low resolution art already we would like to create one or

two games, such as Pong, Space Invaders, or Snake in a three dimensional grid.

Page : 4 of 25


3 PROJECT RESEARCH

3.1 LED Cube Research

According to 399 project design, the initial plan is to build an 8x8x8 LED cube. However, when

the team started working on the cube, every team members realized that the workload of making

a cube was higher than anticipated. The detailed cube construction plans are explained in the 399

project report which has been included in Appendix B. It should be noted that the cube was

purchased instead of built by the team so that more time could be spent on the interface

development.

There are many LED cube products available in the market, especially in China. Multi-color

cubes also can be found in online stores. A multi-color cube has a similar size and structure as a

regular LED cube, but it uses multi-color LEDs that can display a full range of colors. On the

other hand, a multi-color LED cube is much more expensive than a single color one. Due to the

limited budget, TIM-3D still chooses a single color cube as the main 3D-image display. A

multicolor LED is also larger than a single color LED so the LEDs in the front of a cube

sometimes cover the LEDs behind them, which could affect the view of a 3D image. The figure

below is a photo of a multi-color cube; it’s fancy, but not feasible for this project.

Figure 3.1.1 – Multi-color LED Cube

Page : 5 of 25


3.2 Alternative 3D Displays

We had a few options to choose from when deciding on a method of display. The two main

preferences were the 3D LED cube and a 3D image-reflecting prism. The cube provided a true

3D image representation, but places a limitation on the resolution based on the number of LED’s

used. A much sharper image could be produced within a prism using four reflective surfaces to

project a 3D image display. With a cleaner and a better resolution, we were hesitant in choosing

our method of display. In the end the LED cube was chosen for its lower cost and simpler

interface.

3.3 Arduino Research

The easiest way to send commands to the cube is using a microcontroller to send commands with

the I2C protocol. The microcontroller can then be used to translate commands from the

computer’s USB port to commands for the cube. However, not all types of micro-controller are

able to satisfy the LED cube. There are three major considerations when choosing a proper

microcontroller: compatibility with the LED cube, an open-source development platform, and

expansibility.

Figure 3.3.1 – Arduino Microcontroller

Page : 6 of 25


The first requirement is the micro-controller support the I2C protocol to control the cube. The

cube purchased for the project has a mini-USB port, allowing users to design their own 3D

default animation through this port. Therefore, it is required that the microcontroller can get the

control of the cube through the mini-USB port so that the CPU keep sending commands to

manipulate the 3D image instantaneously. Another important requirement of the microcontroller

is programming. It is our hope that the micro-controller has its individual development platform.

Such a platform can make the programming work a lot more efficient. Open source platform

allows users research and edit their code and libraries a lot easier. It also provides an opportunity

of communicating with other developers, which is quite helpful when the programming work

encounter difficulties. Decent expansibility does not only stands for available open-source

libraries to simplify the code, but also illustrate the chip is able to interface with some other

graphic design software. The main reason of this feature so important is that the graphic

manipulation design work would be done easier by using professional design programs.

Based on the criteria shown above, Arduino is one of the most famous microcontrollers available.

The first advantage of the Arduino is that it has a built in bootloader which eliminates the need

for an expensive chip programmer. This feature makes the Arduino very convenient to setup and

program. Another important aspect of the Arduino is the CPU board can be connected to a

variety of interchangeable add-on modules. In this project, the LED cube is actually an add-on

module since the cube supports the I2C serial protocol, allowing the cube to receive data and

timer information through a single USB line from the Arduino.

The Arduino comes with a built in USB to serial converter and natively supports serial

communication. The 3D image manipulator design can be achieved using any PC software that

supports serial communication. These options will be explained in chapter 4.

3.4 Leap Motion Research

Leap Motion is a hand motion detector and a data collection device which sends hand motion

and hand position information to the computer. Based on commands from the Leap Motion the

Page : 7 of 25


computer makes changes to the 3D image on the LED cube. Hand gestures such as swiping

forward, backward, up, down, left or right translate the image shown on the LED cube and

rotating the hand causes the image to rotate in kind. Unlike the Microsoft Kinect, which focuses

on body movements and actions, the Leap Motion focuses only on hand gestures which are all

that is needed for this project. In addition, the cost of Leap Motion is 65% lower than the

Microsoft Kinect. Another significant advantage of the Leap Motion is the free and open source

developer library it comes with which greatly simplifies the interface implementation.

4 EARLY PROJECT PLAN DESCRIPTION

4.1 Project Implementation Strategy

The project was implemented in pursuance of the ELEC 499 course schedule, ELEC 499 course

information and UVIC ECE Department Lab Safety Regulations, and abides by general

university policy and standards of professional behavior. The project cost will be included in this

report and will be reported to the project supervisor. The project strategy is composed of the

following components and sub-components, and their numerical order associated with time.

Component -1: Project Team Selection and Organizing

Sub-Component 1.1: Team meeting

Sub-Component 1.2: Workload Distribution

Component-2: Project Preparation

Sub-Component 2.1: Project equipment purchase

Sub-Component 2.2: Equipment assembly and testing

Sub-Component 2.3: Project research

Component-3: Project Design and Testing

Sub-Component 3.1: Design Alternatives

Sub-Component 3.2: LED Cube Interface

Sub-Component 3.3: Leap Motion Interface

Sub-Component 3.4: Game Design

Page : 8 of 25


Sub-Component 3.5: Final Testing

Component-4: Other works and Documentation

Sub-Component 4.1: Monthly work-logs and milestones

Sub-Component 4.2: Presentation and Pamphlet design

Sub-Component 4.3: Final Report

From above, it is clearly seen that the entire project consists of 4 components, which are

according to the four milestones in the following project timeline figure. The first three

components include the major works required for this project. The design team spent

approximately one month for each of three components due to the three month duration of 499.

The last component was started on March 21 and was completed and will be submitted on April

4, 2014.

The third component in this implementation plan is the most important and involved the most

man-hours. Based on this component, every team member got their own assignments to do. Most

of the project work was programming which lead to each individual focusing on specific parts of

the project rather than working together to get one part done at a time. This working style was

decided on in the first team meeting. More importantly, the project plan was intended to

encourage everyone contribute to the project and avoid unbalanced workload distribution.

4.2 Milestones

The graphic below visualises the core milestones of the project and when they were

scheduled to take place.

Figure 4.2.1 – TIM-3D Milestones

Page : 9 of 25


Comparing with the milestones and work logs submitted to the project supervisor, there are some

places with slight differences. The monthly milestones show the milestones and workloads for

each team member due to the working style the team decided on the reason actually is associated

with the team working style. Since each group member was doing their own parts for long

periods of time, it was more efficient to record the milestones for each person than it would have

been to record milestones for the whole team. Moreover, during the team meetings, every group

members could quickly know what other people did for the project.

4.3 Project Cost List

The table below shows the detail information of the equipment and material used and purchased in this 3D image manipulator design.

Table 4.3.1: Project Budget

Resources Required Prices

Leap-Motion(Donated by Team Member) $80.00

First LED Cube Kit (purchased during ELEC 399):

1. 3MM blue LED x550

2. 10MM LED x4

3. Mini USB slot x1

4. USB cable x1

5. DC power slot x1

6. 1 base circuit board

7. 2.54MM pin slot x1

8. 1k ohm resister x2

9. 10k ohm resister x2

10. TM1818 chip x4

11. 74HC138 chip x1

12. 2MM pin slot x1

13. 10UF capacitance x1

14. 0.1UF capacitance x1

15. 10 ohm resister x1

16. 4953 chip x4

$120.00

Second LED Cube kit (purchased during ELEC 399) 1. programmable circuit board

$120.00

Page : 10 of 25


2. LED cube case

3. 8 8x8 LED slices

4. cube base

Arduino $30.00

USB to serial converter $20.00

USB to TTL converter $25.00

Breadboard and jumpers $15.00

Total: $410.00

4.4 Design Alternatives Comparison (Pros and Cons)

The previous section explained the details of the project implementation plan and schedule as

well as the project cost. In this alternative analysis we are examining the methods of

implementation that were considered and why each was or was not used. After completing the

project research it was decided that the Arduino and Java code method would be the best for the

TIM-3D project. Two alternative implementations were also considered for this project. The

following table shows advantages and disadvantages for each of these three implementations.

Table 4.4.1: Advantages and Disadvantages of Implementations

Implementation Advantages Disadvantages

Cube-USB to TTL

Converter – Java - Leap

Motion

-Cheapest Design

-Eliminates need to

program Arduino

-Cannot implement I2C

protocol on USB to TTL

connector

Cube-Arduino-Processing-

Leap Motion

-Fewer interfacing issues

-Better graphic design

-Open source library

-Require knowledge of

Arduino

-Require knowledge of

Processing

Cube – Arduino – Java –

Leap Motion

-Fewer interfacing issue

-Team familiar with Java

-Open source library

-Delay Problem

Page : 11 of 25


4.4.1 Cube – UI Connector – Java – Leap Motion

Ideally the cube could be controlled directly from the computer. This would eliminate the

Arduino and the problems that come with it such as increased latency and complexity. It was

thought that a USB to TTL converter would be able to relay commands from a Java program to

the cube. The investigation into this alternative revealed that the USB to TTL converter could not

implement the I2C protocol required to control the cube. The lack of I2C protocol support made

this alternative impractical.

4.4.2 Cube – Arduino – Processing – Leap Motion

This alternative is very similar to the design that was chosen. The only difference in this

alternative is that the Processing program environment is used for 3D image manipulation

instead of using Java code. One of the nice things of developing 3D images through the

Processing environment is that the Processing code interfaces with the Arduino perfectly. There

are also several Leap Motion libraries available. Additionally, the Processing IDE is designed for

media art, electronic arts,c and visual design. This functionality applies directly to the

requirements of the TIM-3D project.

The Processing programming environment has the significant limitation that none of the team

members have solid experience with it. This situation results in the team having to spend time

learning the new programming environment which would be difficult in this 3-month-long

project. Team members Song and Wayne attempted to design and code 3D image manipulation

using the Processing IDE. Even though the final product does not use this alternative both Song

and Wayne believe that they could make this alternative work if given more time.

4.4.3 Cube – Arduino – Java – Leap Motion

This design alternative is the one selected for implementation. From the table above, it is

apparent that this design satisfies the most design criteria. The Arduino is able to control the

LED Cube and the Java programming environment allows us to establish a 3D coordinate system

and also design the 3D pong game. Comparing with UI connector method, this design reduces

the work on making up a solution of transmitting LED data to control the Cube. The Arduino can

Page : 12 of 25


handle this part of work using open source libraries. Everyone on the team has also used Java

before, so it brings an easier developing environment for this project.

5 PROJECT DETAIL DESIGNS

As mentioned before in the main design milestones the chosen implementation is based on a Java

program and an Arduino. This Java program has the function of translating the hand position

data from the Leap Motion into commands and creating the 3D images accordingly. Then the

Arduino board is used to receive the image data and send it to the cube. The block diagram

below shows the entire system. Next, this report will concretely explain the design and function

of each block and how they interact with each other.

5.1 LED Cube Initialization

The 8x8x8 LED Cube used in this project is one of the products from Doyoung Studio. It mainly

includes two black mirror surface PCBs, 512 LEDs with pins, one microcontroller chip

(STC12C5A60S2), one mini USB port and a plastic cover. Overall, the cube is simple, cheap,

and reliable. The pictures below shows all the items included in the cube and the assembled one.

Figure 5.1.1 – LED Cube Components

Page : 13 of 25


Figure 5.1.2 – Completed LED Cube

The cube assembling process didn’t cost a lot of time to team, but it played a crucial role in the

project. As the main display in 3D image manipulator, all the 3D effects and performance were

presented though this Cube. Therefore, in order to design and improve the performance, the first

step is to discover how the cube works and how to control the cube. Actually, Doyoung Studio

designs this cube to be able to work in two different modes: default mode and user mode. When

the cube is set up in default mode, it will keep displaying the animation which is saved in its

memory. In this mode, the cube won’t be interrupted unless the mode switch button is pressed.

The user mode requires the user to send 3D images to the cube; otherwise the cube won’t display

anything. Obviously the user mode is the mode chosen to be used in the project. By using this

mode, the LED cube can be used to display any kind of 3D image, such as rotation and

translation, by interfacing with the connected Arduino microcontroller.

The communication protocol of the LED cube is based on the I2C serial protocol. I2C uses only

two bidirectional open-drain lines, Serial Data Line (SDA) and Serial Clock Line (SCL). The

voltages used here are 5V. The I2C reference designed to use two bytes to finish one command.

One byte is address and the other byte is for data. The graph below is the I2C serial bus timing

diagram.

Page : 14 of 25


Figure 5.1.3 – I2C Serial Bus Timing Diagram

More importantly, the LED cube provides the instruction sets for the user mode development.

There are three main instructions that were used in the project: select LED, lighten LED and

clean cube. The value of each instruction is in hexadecimal. For example, 0xC000 is the

instruction to clean the cube according to the instruction table below.

Figure 5.1.4 – Instruction Set for LED Cube

5.2 Image Creation on Arduino

If the entire project is viewed as a human body, the Arduino microcontroller would be the

nervous system. Similar to the nervous system, the Arduino plays the role of information

transmitter. The circuit connection between the Arduino and the LED cube is simple. Since the

Arduino fully support I2C serial bus, one single mini USB line enough for the connection to LED

cube. On the Arduino side, the USB line has to be modified in order to connect to Arduino.

Page : 15 of 25


Basically, there are four slender lines inside an USB line: SDA line, SCL line, power line and

ground line. Depending on the Arduino program code, port 2 and port 3 are separately assigned

to SDA line and SCL line. The ground line of course connects to the grounding port on the

Arduino. The finished connection is shown below. The yellow line is SDA line and the green one

is SDL.

Figure 5.2.1 – Arduino Setup

The Arduino IDE is the software that the team used to program the Arduino chip. The IDE is

written in Java, and actually is derived from the IDE for the Processing programming language.

Arduino programs are written in C or C++; C++ was used for this project. The main function of

the Arduino code is to establish a scenario to receive a string of data from the Java program and

translate it to the cube's instruction set, and finally send these instructions to the cube. The code

is shown below.

#include "CUBE8.h"

int _SDA = 2;

int _SCL = 3;

Page : 16 of 25


CUBE8 Cube(_SDA,_SCL);

unsigned int c;

boolean update_go;

unsigned int matrix[64];

void setup(){

Serial.begin(115200); // initialize serial communication

Cube.GetControl(); // initialize the cube

Cube.Display(0xc000);

update_go = true;

}

void loop(){

// check for new data

if(Serial.available() >= 2) {

unsigned int inAddr = Serial.read();

unsigned int inByte = Serial.read();

if(inAddr==64){ update_go = true;

}

if(inAddr>=0 && inAddr<63){

// add new data to the array

matrix[inAddr] = inByte;

}

}

if(update_go){

// update columns

for(int addr = 0; addr < 64 ; addr++){

Cube.Display(addr*(0x100) + matrix[addr]);

}

update_go = false;

}

Cube.Display(0xE006);

Page : 17 of 25


}

From the code, it is can be seen that the library cube8.h is included in the code. This library code

is attached in Appendix D. In the cube8.h library, functions like Cube.Display(), Cube.

GetControl(), and Cube8 Cube(SDA, SDL) are defined. Just as with any other Arduino Code,

this code only needs to define two functions to make it run able. Setup() is the function run once

at the start of a program that initializes settings. Also, the loop() is the function called repeatedly

until the board powers off.

In the setup() function, two things are initialized: the serial read speed and the cube mode. The

serial speed is set to be 115200 which is the highest speed for Arduino. The reason behind is that

when the team was doing the final testing, approximately 1 second delay occurred between the

hand movement and image display. The highest transmitting speed (115200) was set up to

reduce the delay period. The function Cube.GetControl() is implemented to push the cube into

user mode so that the cube will display the Image created in the main loop.

In the Loop(), first of all the program is checking is there any data coming from the serial.read()

function. At least two bytes must be received at a time; the first byte indicates the column that is

to be updated and the second byte indicates the state of each LED in that column. This data is

then stored in an array so that it can be sent to the cube. The size of the array is 64 and it saves

the data for one frame of an animation. The last step is to tell the cube which LEDs should be

turned on and the last command, Cube.Display(0xE006), is to actually turn on these LEDs.

5.3 Arduino Interface

The Arduino interface is implemented using an open source library called RXTX. This library

defines a Java class that handles serial input and output using a standard stream API. Data is sent

to the Arduino in pairs of bytes with the first byte indicating the column that should be updated

and the second byte indicating the data that should be written to that column.

Page : 18 of 25


5.4 Image Management

The image is stored in the Java program as a 3 dimensional binary matrix. This form makes it

simple to translate or rotate the image by moving each individual point. When the cube needs to

be updated the matrix can be read out and sent to the Arduino. Each binary column can be

treated as a byte and sent to the Arduino with its corresponding column number.

5.5 Leap Motion Detection

The Leap Motion offers development libraries that provide built in support for detecting several

gestures. Of interest for this project are the circle and swipe gestures. The original plan was to

use the swipe gesture to translate the image and the circle gesture to rotate the image. The

problem with this approach is that the Leap Motion only detects circle gestures in one plane. An

alternative approach was devised which used only the position of a hand above the Leap Motion

to manipulate the image.

The first step is to define a starting point for the hand. A fraction of a second after the Leap

Motion detects a hand the position of that hand is defined as the origin point. If the hand moves a

certain distance from the origin point then the image moves in the direction of the hand. Once the

image moves the hand must return to the origin point before the image can be moved again.

Rotation is handled is a similar manner by recording the rotation of the hand along each axis.

5.6 3D Game Design

For the project our team only had enough time to create one 3D game, which was a simple game

of pong. Pong is the classic table tennis sport game that is featured using simple two-dimensional

graphics. Players would use paddles to hit a ball back and forth in a playing field, and aim to

score points against their opponent. A point is earned when an opponent is unable to successfully

return the ball to that player. Pong 3D is your typical game of pong; however it utilizes the third

dimension. In Pong 3D the player is essentially playing against a wall, to keep the programming

simple. For this game the ball is allowed to bounce around in the three-dimensional confines of

the LED cube and a paddle controlled by the player reflects the ball away from the player

controlled end plane. For this section of the report we will describe the concepts needed to

Page : 19 of 25


govern the game, discuss the major components of the code, and any difficulties encountered

during the programming.

5.6.1 Concepts of Pong 3D

Pong 3D consists of two major components which are the ball and the paddle. The ball will be

confined to bounce within the 8x8x8 grid of the LED cube, and will report its position to be

displayed in the cube. The paddle is confined to one of the outside planes and will receive its

position from the Leap Motion interface. In the most general sense this is the logic used in the

Pong 3D script.

Figure 5.6.1.1 – Pong 3D Logic

The ball is allowed to bounce around in the confines of the LED cube which is defined by the

limits of 0≤X≤8, 0≤Y≤8, Z≤8, with the origin at the corner near the axis legend (in the figure

above). Now if the ball’s Z direction component is less than zero, the player will lose the game

because it refers that a point has been scored against the player. The ball contains an X, Y, and Z

positional and directional components of its movement. When the ball hits one of the boundary

planes it will reverse its corresponding direction. For example, if the ball hits the YZ-plane it

Page : 20 of 25


reverses the X components direction of the ball. It should also be noted that the ball is of size of

1x1x1 and will not be affected by the laws of gravity, to keep it simple.

The paddle is confined to the XY-plane where Z=0 and is allowed to move freely along that

plane ( 0≤X≤8, 0≤Y≤8, Z=0). The size of the paddle is a 2x2x1 rectangle that will reflect the ball

in the inverse Z direction. However, if the ball does not intersect with the paddle it will continue

outside the boundary and the player will lose the game. The paddle’s position will be defined by

the position data from the Leap Motion hardware.

5.6.2 Pong 3D Program

Pong 3D game was created using the Java programming language, and built with a number of

classes. The main classes used in this program are the shown in the figure below and it outlines

how each of those classes interacts with one another. The GameEngine.java class contains the

Leap Motion handlers that control the paddle in the game. This class also communicates with the

Paddle.java and Grid.java classes. The Paddle.java class updates the position of paddle to the

LED cube, and the size of that paddle. The Grid.java class completes a number of tasks that

checks if the ball has collided with one of the planes or paddle and continues to increment the

position of the ball. This class also checks to see if the ball has passed the paddle plane and a

point has been scored, so the game will be shut down. The Ball.java class takes the current

position of the ball and increments its direction according the path of travel. Please see Appendix

B for Pong 3D code.

Figure 5.6.2.1 - Major Classes Interactions in Pong 3D

Page : 21 of 25


6 WORKLOAD DISTRIBUTION & ACHIEVEMENTS

In order to complete this project in an efficient way, the team built a work log to record each

period involved with the task. The whole project is divided into four stages.

First stage is initiating a group with five group members and continues pursuing the topic which

has been chosen in ELEC/CENG 399 during the first two weeks.

Second stage, preparation of the project including: Leap Motion open source coding

investigation, LED cube full functionality test, Arduino investigation, and Arduino and Cube

communication theory. When team finished planning our project in this stage, a brief

introduction was presented to our advisor and ELEC/CENG 499 classmates during the proposal

presentation. The topic contains the motivation of project, materials need, procedures and the

total project price cost.

Third stage, the whole group was divided by two small teams. Team A began to work on the new

LED cube construction, as well as installing the eight 8x8 LED slices on the cube base. In

addition, the team tested each slice of LED cube in order to make sure the LED pin connection,

soldering joints and LEDs functionality were working properly. Another task of team A was to

code the Arduino program to design an image and send image signals to the LED

cube. Meanwhile, team B worked on the game coding and communications configuration

between the Leap Motion and the Arduino.

The fourth stage was the last 3 weeks, two teams worked together to test the whole project's

functionality and work on the demonstration and final report.

The group clearly defined jobs to each group members and accomplished the cube as planned.

Most of the primary functionality was achieved; specifically the Leap Motion to Arduino

communication, the Arduino to LED cube communication, and the image manipulation.

However, the 3D game program still had some bugs that could not be fixed by the date of

demonstration. However, the primary goal of the project was achieved.

Page : 22 of 25


Fangzheng Song: My contribution to the project mainly focus on the LED Cube interfacing with

Arduino, which includes researching LED cube documentation, researching Arduino library and

programming Arduino code. A portion of time was spent on the alternative of connecting

Arduino to Processing. Moreover, the last two week of the semester was spent on project report

and demo preparation.

Nelson Lee: My contributions for this project have been to oversee many of the different tasks

throughout this project. I was also primarily responsible for the Pong 3D code and aiding the

other team members with any of the other tasks (such as the Arduino to cube interface and

coding, webpage design). I have also spent many hours on the PR side of this project as well,

such as the Final Report, Milestones, Worklogs, and the many presentations.

Tristan: My primary focus was interfacing the Leap Motion with our program and interfacing

our program with the Arduino. I was also involved in interfacing the Arduino and the cube.

Candace Chen: My contribution to the project consisted of researching on 3D LED Cube and

alternative 3D image displays. We worked cooperatively weekly to design and talk about the

motivation of this project. A great deal of the time was spent designing and creating project

webpage and pamphlet. A portion of time was spent on LED cube functionality testing.

Remaining week of the semester was spent on completing the final project report.

Yang Liu: My contributions to the project consisted of researching on new LED cube, Arduino

circuit board, processing coding, website design and Leap Motion. About 50 hours were spent on

LED cube, Leap Motion and Arduino communication, 10 hours were spent on website design

and the website post, and 10 hours were spent on LED cube functionality test and construction.

7 WEBSITE DESIGN

The main theme of the group project website is for demonstrating the TIM-3D project; this

website includes “Home”, “Team”, “Gallery” and “Documents” main links. Dreamweaver 8 was

used for designing the website. Previously, team tried to use raw html to design the website,

however, none of team members had solid html coding experience. Dreamweaver 8 also has an

Page : 23 of 25


html coding environment that made it easy to implement the website on different web browsers

and different operating systems. The built in web page templates were so that the web

components could be added or moved based on changes to the project during development.

On the main page, there was the project description part to introduce the project motivations,

basic theory and components. On the team page, there were general introductions of team

members and team photos. On the Gallery page, in this webpage, the photos of the Arduino Unit

Connected to Cube and Computer, Arduino Circuit Connection and Demonstration Day were

shown to help visitors to get a visual experience on the project. The Documents page listed all

worklogs, progress reports, milestones from the beginning of the project, and the final report.

These offered the details of how the project get start, process, develop and demonstrate.

The most useful function that was used to complete the webpage was the split screen function

which allowed us to view the webpage and the web page's code as well as being able to edit code

on one side, and see changes on the website on the other side immediately. The webpage zip

folder was then transferred to a domain to upload.

8 PROJECT DISCUSSION

The main concern with TIM-3D is the limitation on the resolution of the cube. Also, the

processing time of the cube created many restraints between the Arduino and Leap Motion

interfaces. The capabilities of Leap Motion exceeded the capability of the LED cube. Thus,

during demonstration day there was a delay in response to hand gestures being implemented on

the cube. It was not able to simulate real time hand movements. Instead, the demonstration had

to be implemented in steps, increments of motion up, down, left, right…etc. Expanding the size,

which will intensify the resolution, would do improvements to the cube. As well as upgrading

the cube’s controller to respond quicker to commands, it will need to compatible to the

capabilities of Leap Motion and the Arduino interface.

Good teamwork was achieved towards completion of this project. Due to the each member’s

distinctive schedules and commitments it was difficult to have many scheduled constructing

Page : 24 of 25


conventions. Improvements could have been made to reduce the amount of work done with

limited amount of time. Generally, the project and each milestone were completed on schedule.

Work was being accomplished cooperatively and individually. All compositions were

documented and up to date as planned. Overall, the project goal of displayed the theory and idea

of a 3D image manipulator was achieved.

9 CONCLUSION

The project includes this final report as well as a webpage which includes all milestone, work log

and progress reports that can be found on www.ece.uvic.ca/~wayneliu. The entire project

consists of 4 components: project team selection and organizing, project preparation, project

design and testing, other works and documentations. Additionally, a simple 3D pong game was

created to demonstrate the functionality of TIM-3D using java programming language.

This project has successfully allowed us to create and understand the technology of 3D image

manipulation. It started with the researching of methods to display 3D images. Through the pros

and cons, the 3D LED cube was chosen. Further on, an Arduino (the most famous single-broad

microcontrollers in market) was chosen to act as the microcontroller in controlling the image

production of the LED cube. The development language of the Arduino, C, C++, and Java are

most convenient and compatible with software running on a computer. This device allowed

connection between the cube and a computer. Also connected to the computer is the hand motion

detector, Leap Motion. Leap Motion transfers the commands from the user’s hands to the

computer. TIM-3D was able to read front, back, up, down, pitch, and roll hand gestures. All

components chosen were based on size, functionality but mostly price due to low budget

restraints.

This project completed in an efficient and creative way. The whole group was divided into two

small teams that worked collectively with clearly defined jobs. The cube was accomplished as

planned, as well as pamphlets, web page and poster before the deadline.

Page : 25 of 25


10 APPENDIX A (JAVA PROGRAM)

#include "CUBE8.h"

CUBE8::CUBE8(byte SDA,byte

SCL){

// TODO Auto-generated

constructor stub

_SDA = SDA;

_SCL = SCL;

pinMode(_SDA,OUTPUT);

pinMode(_SCL,OUTPUT);

ack = false;

}

void CUBE8::DELAY_U(unsigned

int a){

bool x = false;

while(a-- != 0)

{

delay(1);

x = ! x;

digitalWrite(_SCL,x);

}

}

void CUBE8::Display(unsigned

int data){

SendDeviceByte(data /

0x100,data % 0x100);

}

void CUBE8::I2cDelay(unsigned

int a){

while(a-- != 0)

asm("Nop");

}

void CUBE8::StartI2c(void){

I2cDelay(18);

pinMode(_SCL,INPUT);

while(!digitalRead(_SCL));

I2cDelay(9);

digitalWrite(_SDA,HIGH);

I2cDelay(9);

pinMode(_SCL,OUTPUT);

digitalWrite(_SCL,HIGH);

I2cDelay(9);

digitalWrite(_SDA,LOW);

I2cDelay(9);

digitalWrite(_SCL,LOW);

I2cDelay(4);

}

void CUBE8::I2cSendByte(byte

data){

byte i;

for(i=0;i<8;i++)

{

digitalWrite(_SDA,((data << i)

& 0x80));

I2cDelay(6);

Page : 2 of 15



I2cDelay(8);


}

I2cDelay(8);


I2cDelay(8);


I2cDelay(8);

pinMode(_SDA,INPUT);

ack = digitalRead(_SDA);


I2cDelay(8);

pinMode(_SDA,OUTPUT);

}

void CUBE8::GetControl(void){

DELAY_U(1000);


}

void CUBE8::SendDeviceByte(byte

address,byte data){

StartI2c();

I2cSendByte(address);

I2cSendByte(data);

StopI2c();

}

void CUBE8::StopI2c(void){

digitalWrite(_SDA,LOW);

I2cDelay(9);


I2cDelay(9);


I2cDelay(9);

}

void CUBE8::SetAll(unsigned int

data){

return;

}

void CUBE8::Update(unsigned int

data[]){

return;

}

CUBE8::~CUBE8(){

}

//---------------------------//

#ifndef CUBE8_H_

#define CUBE8_H_

#include "Arduino.h"

class CUBE8 {

public:

CUBE8(byte SDA,byte SCL);

virtual ~CUBE8();

void GetControl(void);

void Display(unsigned int

data);

void SetAll(unsigned int

Page : 3 of 15


data);

void Update(unsigned int

data[]);

private:

byte _SDA,_SCL;

bool ack;

void DELAY_U(unsigned int a);

void StartI2c(void);

void I2cDelay(unsigned int a);

void StopI2c(void);

void I2cSendByte(byte data);

void SendDeviceByte(byte

address,byte data);

};

#endif

/----------------------------//

import java.io.IOException;

import com.leapmotion.leap.*;

import

com.leapmotion.leap.Gesture.Sta

te;

import java.io.OutputStream;

import java.util.Scanner;

class Application {

public static void

main(String[] args) {

// Create a sample

listener and controller

DefListener listener =

new DefListener();

Controller controller =

new Controller();

// Have the sample

listener receive events from

the controller

controller.addListener(listener

);

// Keep this process

running until Enter is pressed

System.out.println("Press Enter

to quit...");

try {

System.in.read();

} catch (IOException e)

{

e.printStackTrace();

}

// Remove the sample

listener when done

controller.removeListener(liste

ner);

Page : 4 of 15


SerialOutput

serPortUSB = new

SerialOutput();

serPortUSB.initialize();

Scanner in = new

Scanner(System.in);

int inVar = 0;

String inStr = "";

int[] output_buffer;

Matrix mat = new

Matrix();

inStr =

in.nextLine();

mat.makeTestBox();

output_buffer =

mat.getMatrix();

serPortUSB.updateAll(outpu

t_buffer);

do {

inStr =

in.nextLine();

mat.translate(1,-1);

output_buffer =

mat.getMatrix();

serPortUSB.updateAll(outpu

t_buffer);

} while

(inStr.compareTo("")==0);

serPortUSB.close();

}

}

//---------------------------//

import java.math.*;

public class Matrix {

private boolean[][][]

data;

private int size_x;

private int size_y;

private int size_z;

public Matrix (int x, int

y, int z) {

size_x = x;

size_y = y;

size_z = z;

Page : 5 of 15


data = new

boolean[x][y][z];

for( int a=0; a<x;

a++ ){

for( int b=0;

b<y; b++ ){

for( int

c=0; c<z; c++ ){

data[a][b][c] = false;

}

}

}

}

public Matrix (int s) {

this(s, s, s);

}

public Matrix () {

this(8);

}

public int getColumn(int

x, int y) {

int sum = 0;

for( int z=0;

z<size_z; z++ ) {

if(

data[x][y][z] ) {

sum +=

(int) Math.pow(2, z);

}

}

return sum;

}

public int[] getMatrix() {

int[] data_out = new

int[64];

for( int a=0;

a<size_x; a++) {

for( int b=0;

b<size_y; b++) {

data_out[b+(a*size_y)] =

getColumn(a, b);

}

}

return data_out;

}

public void setAll(boolean

x) {

for( int a=0;

a<size_x; a++ ){

for( int b=0;

b<size_y; b++ ){

Page : 6 of 15


for( int

c=0; c<size_z; c++ ){

data[a][b][c] = x;

}

}

}

}

public void updateBall(int

x, int y, int z) {

}

public void

updatePaddle(int x, int y) {

}

/* x-axis: 1

* y-axis: 2

* z-axis: 3

*

* n is the number of 45

degrees increments

*/

public void rotate(int

dir, int n) {

}

/* x-axis: 1

* y-axis: 2

* z-axis: 3

*

* n is the number of

increments

*/

public void translate(int

axis, int n) {

boolean[][][]

new_data = new

boolean[size_x][size_y][size_z]

;

switch(axis) {

case 1: // x-axis

for(int a=0;

a<size_y; a++) {

for(int

b=0; b<size_z; b++) {

for(int c=0; c<size_x;

c++) {

new_data[c][a][b] = false;

if((c-n)>=0 && (c-

n)<size_x) {

new_data[c][a][b] =

data[c-n][a][b];

Page : 7 of 15


}

}

}

}

break;

case 2: // y-axis

break;

case 3: // z-axis

break;

default:

return;

}

data = new_data;

}

public void makeTestBox()

{

int[][][] box = {

{

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,1,1,1,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,1,1,1,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,1,0,0,1,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,1,0,0,1,0,0,0},

Page : 8 of 15


{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,1,1,1,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,0,0,1,0,0,0},

{0,1,1,1,1,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

Page : 9 of 15


{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}},

{

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0},

{0,0,0,0,0,0,0,0}}

};

for(int a=0; a<8;

a++) {

for(int b=0;

b<8; b++) {

for(int

c=0; c<8; c++) {

if(box[a][b][c]==1) {

data[a][b][c] = true;

}else{

data[a][b][c] = false;

}

}

}

}

}

}

//---------------------------//

import java.io.IOException;

import java.io.OutputStream;

import

gnu.io.CommPortIdentifier;

import gnu.io.SerialPort;

import gnu.io.SerialPortEvent;

import

gnu.io.SerialPortEventListener;

import java.util.Enumeration;

public class SerialOutput

implements

SerialPortEventListener {

SerialPort serialPort;

Page : 10 of 15


/** The port we're normally

going to use. */

private static final

String PORT_NAME = "COM5";

public OutputStream

output;

/** Milliseconds to block

while waiting for port open */

private static final int

TIME_OUT = 2000;

/** Default bits per

second for COM port. */

private static final int

DATA_RATE = 115200;

public void initialize() {

CommPortIdentifier

portId = null;

Enumeration portEnum

=

CommPortIdentifier.getPortIdent

ifiers();

//First, Find an

instance of serial port as set

in PORT_NAME.

while

(portEnum.hasMoreElements()) {

CommPortIdentifier

currPortId =

(CommPortIdentifier)

portEnum.nextElement();

System.out.println(currPor

tId.getName());

if

(currPortId.getName().equals(PO

RT_NAME)) {

portId =

currPortId;

break;

}

}

if (portId == null)

{

System.out.println("Could

not find COM port.");

return;

}

try {

// open serial

port, and use class name for

the appName.

serialPort =

(SerialPort)

portId.open(this.getClass().get

Name(), TIME_OUT);

Page : 11 of 15


// set port

parameters

serialPort.setSerialPortPa

rams(DATA_RATE,

SerialPort.DATABITS_8,

SerialPort.STOPBITS_1,

SerialPort.PARITY_NONE);

// open the

stream

output =

serialPort.getOutputStream();

} catch (Exception

e) {

System.err.println(e.toStr

ing());

}

}

/**

* This should be called

when you stop using the port.

* This will prevent port

locking on platforms like

Linux.

*/

public synchronized void

close() {

if (serialPort !=

null) {

serialPort.removeEventList

ener();

serialPort.close();

}

}

public synchronized void

serialEvent(SerialPortEvent

oEvent) {

}

public void updateAll(

int[] data ) {

try {

for( int i=0;

i<64; i++ ){

output.write(i);

output.write(data[i]);

}

output.write(64);

Page : 12 of 15


output.write(0);

} catch (IOException

e) {

e.printStackTrace();

}

}

}

//---------------------------//

import java.lang.Math;

import com.leapmotion.leap.*;

import com.leapmotion.leap.Gesture.State;

class DefListener extends Listener {

public void onInit(Controller controller) {

System.out.println("Initialized");

}

public void onConnect(Controller

controller) {

System.out.println("Connected");

controller.enableGesture(Gesture.Type.TYP

E_SWIPE);


E_CIRCLE);


E_SCREEN_TAP);


E_KEY_TAP);

}

public void onDisconnect(Controller

controller) {

//Note: not dispatched when running in

a debugger.

System.out.println("Disconnected");

}

public void onExit(Controller controller)

{

System.out.println("Exited");

}

public void onFrame(Controller controller)

{

// Get the most recent frame and report

some basic information

Frame frame = controller.frame();

System.out.println("Frame id: " +

frame.id()

Page : 13 of 15


+ ", timestamp: " +

frame.timestamp()

+ ", hands: " +

frame.hands().count()

+ ", fingers: " +

frame.fingers().count()

+ ", tools: " +

frame.tools().count()

+ ", gestures " +

frame.gestures().count());

if (!frame.hands().isEmpty()) {

// Get the first hand

Hand hand = frame.hands().get(0);

// Check if the hand has any fingers

FingerList fingers = hand.fingers();

if (!fingers.isEmpty()) {

// Calculate the hand's average

finger tip position

Vector avgPos = Vector.zero();

for (Finger finger : fingers) {

avgPos =

avgPos.plus(finger.tipPosition());

}

avgPos =

avgPos.divide(fingers.count());

System.out.println("Hand has " +

fingers.count()

+ " fingers, average

finger tip position: " + avgPos);

}

// Get the hand's sphere radius and

palm position

System.out.println("Hand sphere

radius: " + hand.sphereRadius()

+ " mm, palm position: " +

hand.palmPosition());

// Get the hand's normal vector and

direction

Vector normal = hand.palmNormal();

Vector direction = hand.direction();

// Calculate the hand's pitch, roll, and

yaw angles

System.out.println("Hand pitch: " +

Math.toDegrees(direction.pitch()) + "

degrees, "

+ "roll: " +

Math.toDegrees(normal.roll()) + " degrees, "

+ "yaw: " +

Math.toDegrees(direction.yaw()) + "

degrees");

}

Page : 14 of 15


GestureList gestures = frame.gestures();

for (int i = 0; i < gestures.count(); i++)

{

Gesture gesture = gestures.get(i);

switch (gesture.type()) {

case TYPE_CIRCLE:

CircleGesture circle = new

CircleGesture(gesture);

// Calculate clock direction

using the angle between circle normal and

pointable

String clockwiseness;

if

(circle.pointable().direction().angleTo(circle.

normal()) <= Math.PI/4) {

// Clockwise if angle is less

than 90 degrees

clockwiseness = "clockwise";

} else {

clockwiseness =

"counterclockwise";

}

// Calculate angle swept since

last frame

double sweptAngle = 0;

if (circle.state() !=

State.STATE_START) {

CircleGesture

previousUpdate = new

CircleGesture(controller.frame(1).gesture(ci

rcle.id()));

sweptAngle =

(circle.progress() -

previousUpdate.progress()) * 2 * Math.PI;

}

System.out.println("Circle id: "

+ circle.id()

+ ", " + circle.state()

+ ", progress: " +

circle.progress()

+ ", radius: " +

circle.radius()

+ ", angle: " +

Math.toDegrees(sweptAngle)

+ ", " + clockwiseness);

break;

case TYPE_SWIPE:

SwipeGesture swipe = new

SwipeGesture(gesture);

System.out.println("Swipe id: "

+ swipe.id()

+ ", " + swipe.state()

+ ", position: " +

swipe.position()

+ ", direction: " +

swipe.direction()

Page : 15 of 15


+ ", speed: " +

swipe.speed());

break;

case TYPE_SCREEN_TAP:

ScreenTapGesture screenTap =

new ScreenTapGesture(gesture);

System.out.println("Screen Tap

id: " + screenTap.id()

+ ", " + screenTap.state()

+ ", position: " +

screenTap.position()

+ ", direction: " +

screenTap.direction());

break;

case TYPE_KEY_TAP:

KeyTapGesture keyTap = new

KeyTapGesture(gesture);

System.out.println("Key Tap id:

" + keyTap.id()

+ ", " + keyTap.state()

+ ", position: " +

keyTap.position()

+ ", direction: " +

keyTap.direction());

break;

default:

System.out.println("Unknown

gesture type.");

break;

}

}

if (!frame.hands().isEmpty()

|| !gestures.isEmpty()) {

System.out.println();

}

}

}

11 APPENDIX B (GAME CODE)

package pong; import pong.frontEnd.GameEngine; public class Main { public static void main(String[] args) { GameEngine engine = new GameEngine(); engine.run(); } }

//------------------------------------------------//

package pong.frontEnd; //import java.awt.Dimension; import javax.swing.JPanel; import pong.components.Component; import pong.components.Paddle; import pong.constants.Constants; import pong.util.Point2D; import java.awt.event.MouseEvent; import java.awt.event.MouseMotionListener; import java.util.ArrayList; /**

Page : 2 of 8


* Gets the ball and paddle 3D location from the grid Class in the

* pong.frontEnd Package

*

* **Need to replace mouse listener with Leap Motion**

*/ public class GameEngine extends JPanel implements MouseMotionListener { protected Grid grid; protected ArrayList<Component> list; public GameEngine() { initInput(); initGame(); } //Starts the game and terminated loop when ball passes the paddle

public void run() { while (true) { //System.out.println("something"); stepBalls(); //ball=stepBalls(); try { Thread.sleep(Constants.DELAY); } catch (InterruptedException ex) {} } } protected void initGame() { this.grid = Grid.getInstance(); for (int i = 0; i < Constants.NUM_BALLS; ++i) grid.addBall(); } protected void initInput(){ this.addMouseMotionListener(this); } protected void stepBalls(){ grid.stepBalls(); }

public void mouseEntered(MouseEvent e) { } public void mouseExited(MouseEvent e) { } public void mouseDragged(MouseEvent e) { } public void mouseMoved(MouseEvent e) { Point2D point = new Point2D((int)e.getPoint().getX(), (int)e.getPoint().getY()); Paddle.getInstance().setPoint(point); } }

//------------------------------------------------//

package pong.frontEnd; import java.util.ArrayList; import pong.components.Ball; import pong.components.Component; import pong.components.Paddle; import pong.constants.Constants; import pong.util.CollisionDetector; import pong.util.Point2D; import pong.util.Point3D; public class Grid { private static Grid INSTANCE = new Grid();

protected ArrayList<Component> list; private Grid () { this.list = new ArrayList<Component>(); } public static Grid getInstance(){ return Grid.INSTANCE; }

Page : 3 of 8


public void addBall() { list.add((Component) new Ball()); }

// Iterate through list stepping only the balls

public void stepBalls(){ Point3D currentPoint;

for(Component item: this.list){ if(item instanceof Ball) { ((Ball)item).step(); currentPoint = ((Ball) item).getPoint(); //Matrix.updateBall(currentPoint.getX(),currentPoint.getY(),currentPoint.getZ()) }

} //System.out.print(list); checkCollisions(); //return(list); } protected void checkCollisions(){ for(Component item: this.list){ if(item instanceof Ball){ Point3D currentPoint = ((Ball) item).getPoint();

System.out.printf("x: %d y: %d z: %d\n", currentPoint.getX(), currentPoint.getY(), currentPoint.getZ()); //System.out.print(currentPoint);

if(currentPoint.getZ() < 0){ //System.out.printf("Point<0 z:

%d\nGame Over\n",currentPoint.getZ()); throw new IllegalStateException("Game Over"); }

//System.out.print(CollisionDetector.distanceCompare(currentPoint.getZ(), Constants.SCREEN_DEPTH, 1)); if(CollisionDetector.distanceCompare(currentPoint.getZ(), Constants.SCREEN_DEPTH, 1)){ ((Ball) item).bounce(Constants.PLANE_Z); //System.out.printf("Collision at +Z=%d\n",currentPoint.getZ()); }

//Check -Z plane (Plane 1)(Paddle Plane) else if(CollisionDetector.distanceCompare(currentPoint.getZ(), 0, 0)){ //System.out.printf("Collision at -Z=%d\n",currentPoint.getZ());

//Check if intercepted by paddle

Point2D paddlePoint = Paddle.getInstance().getPoint(); //System.out.printf("Paddle X:%d | Y:%d\n",paddlePoint.getX(),paddlePoint.getY()); if (CollisionDetector.paddleDistanceCompare(currentPoint.getX(), paddlePoint.getX(), Constants.PADDLE_SIZE/2) && CollisionDetector.paddleDistanceCompare(currentPoint.getY(),

Page : 4 of 8


paddlePoint.getY(), Constants.PADDLE_SIZE / 2)) { ((Ball) item).bounce(Constants.PLANE_Z); //System.out.println("Paddle Bounce"); }

} //Check +-Y plane (+Y=Plane 6)(-Y=Plane 5) if(CollisionDetector.distanceCompare(currentPoint.getY(), Constants.SCREEN_HEIGHT, 1) || CollisionDetector.distanceCompare(currentPoint.getY(), 0, 0)){ ((Ball) item).bounce(Constants.PLANE_Y); //System.out.printf("Collision at Y=%d\n",currentPoint.getY()); } //Check +-X plane (+X=Plane 4)(-X=Plane 3) if(CollisionDetector.distanceCompare(currentPoint.getX(), Constants.SCREEN_WIDTH, 1) || CollisionDetector.distanceCompare(currentPoint.getX(), 0, 0)){ ((Ball) item).bounce(Constants.PLANE_X); //System.out.printf("Collision at X=%d\n",currentPoint.getX()); } } //return(currentPoint); } } }

//------------------------------------------------// package pong.components; //import java.awt.Color; import java.awt.Graphics; import pong.constants.Constants; import pong.util.*; public class Ball extends Component{ protected Direction direction; protected Point3D point; public static int SIZE = Constants.MAX_BALL_RADIUS; public Ball (){ this.direction = new Direction(Constants.X_BALL_DIRECTION, Constants.Y_BALL_DIRECTION, Constants.Z_BALL_DIRECTION); this.point = new Point3D(Constants.MAX_BALL_RADIUS, Constants.MAX_BALL_RADIUS, Constants.MAX_BALL_RADIUS); }

//**Note: Returns a copy of the balls location**

public Point3D getPoint(){ return this.point.clone(); }

//**Note: Increments directional vectors**

public void step(){

Page : 5 of 8


int x = point.getX(); //gets current x-direction

int y = point.getY(); //gets current y-direction

int z = point.getZ(); //gets current z-direction

point = new Point3D(x + this.direction.getA(), //determines new 3D point y + this.direction.getB(), z + this.direction.getC()); }

//**Note: x direction changes only if it hits a paddle**

public void bounce(int planeID){ if(planeID == Constants.PLANE_Z){ this.direction = new Direction(this.direction.getA(), this.direction.getB(), -this.direction.getC()); } else if(planeID == Constants.PLANE_X){ this.direction = new Direction(-this.direction.getA(), this.direction.getB(), this.direction.getC()); } else if(planeID == Constants.PLANE_Y){ this.direction = new Direction(this.direction.getA(), -this.direction.getB(), this.direction.getC()); } } @Override

public void draw(Graphics g) {

// TODO Auto-generated method stub } }

//------------------------------------------------// package pong.components; //import java.awt.Color; //import java.awt.Graphics; import pong.constants.Constants; import pong.util.Point2D; /**

* Components used to define the paddle

*/ public class Paddle { protected Point2D point; protected static Paddle thePaddle = new Paddle(); private Paddle() { this.point = new Point2D(Constants.SCREEN_WIDTH/2, Constants.SCREEN_HEIGHT/2); } public void setPoint(Point2D point){ Point2D p = this.point; //Matrix.updatePaddle(p.getX(), p.getY()); } public Point2D getPoint(){ return this.point; } public static Paddle getInstance(){ return Paddle.thePaddle; } }

//------------------------------------------------//

Page : 6 of 8


package pong.util; /**

* Checks to see if the ball is out of bounds of the 8x8x8 grid and returns true if the

* ball is out of bound, and returns false if it is not. *

* distanceCompare() = Checks if ball is out of the +y,y0,+x,x0,+z planes

* paddleDistanceCompare() = Checks if ball is out of paddle plane (z0 plane) */ public class CollisionDetector { private CollisionDetector(){}

//Check ball position in respect to the boundary of the grid and return true if ball // is out of bounds and false if it is not //point - Position of ball //wall - Distance of wall //dir - Direction 0 = neg direction, 1 = pos direction

public static boolean distanceCompare(int point, int wall, int dir){

//System.out.println("Point : " + point + " | Wall: " + wall + "| dir:"+dir+"\n"); if((point>=wall) && (dir==1)){ //System.out.println("True 1"); return true; } if((point<=wall) && (dir==0)){ //System.out.println("True 2"); return true; }

//System.out.println("False"); return false; }

public static boolean paddleDistanceCompare(int point, int padPoint, int padSize){

//System.out.println("Point : " + point + " | PadPoint: " + padPoint + "| Pad:" + padSize + "\n"); if((point<=(padPoint+padSize)) && (point>=(padPoint-(padSize-1)))){ //System.out.println("True"); return true; } //System.out.println("False"); return false; } }

//------------------------------------------------// package pong.components; import java.awt.Graphics; public abstract class Component { public abstract void draw(Graphics g); }

//------------------------------------------------// package pong.util; /**

* Find and return the direction of the ball */ public class Direction { protected int a, b, c;

//Find current ball co-ordinates

Page : 7 of 8


public Direction(int a, int b, int c){ this.a = a; this.b = b; this.c = c; } //Determing magnitude

public int getMagnitude(){ return (int)(Math.sqrt(a * a + b * b + c * c)); } public Direction clone(){ return new Direction(this.a, this.b, this.c); } //New direction

public String toString(){ return "3D-Vector: a = " + this.a

+ " | b = " + this.b

+ " | c = " + this.c; }

public int getA() { return a; }

public int getB() { return b; }

public int getC() { return c; } }

//------------------------------------------------//

package pong.util; /**

* Find and return a 3D point */ public class Point2D { protected int x;

protected int y; public Point2D(int x, int y){ this.x = x; this.y = y; } public int getX(){ return this.x; } public int getY(){ return this.y; } public Point2D clone(){ return new Point2D(this.x , this.y); } }

//------------------------------------------------// package pong.util; /**

* Find and return a 3D point */ public class Point3D {

private int x, y, z; public Point3D (int x, int y, int z) { this.x = x; this.y = y; this.z = z; } public int getX() { return x; } public int getY() { return y; } public int getZ() { return z; } public Point3D clone(){

Page : 8 of 8


return new Point3D(this.x, this.y, this.z); } }

//------------------------------------------------// /**

* Config file for most constant values used in the game

*

* Constants: * -Delay

* -NumBalls

* -LED cube max parameters

* -Plane ID's

* -3D Directions

* -Ball Radius

* -Paddle size

*/ package pong.constants; public class Constants {

public static int DELAY = 100; public static int NUM_BALLS = 1;

//LED cube max parameters

public static int SCREEN_WIDTH = 10; public static int SCREEN_HEIGHT = 10; public static int SCREEN_DEPTH = 12; //Plane ID

public static int PLANE_Z = 0; public static int PLANE_X = 1; public static int PLANE_Y = 2;

//3D Directions

public static int X_BALL_DIRECTION = 1; public static int Y_BALL_DIRECTION = 1; public static int Z_BALL_DIRECTION = 1;

//Ball Radius

public static int MAX_BALL_RADIUS = 1;

//Paddle Constants

public static int PADDLE_SIZE = 2;

private Constants(){} }

12 APPENDIX C (CUBE CONSTRUCTION)

The 8x8x8 LED cube is the main display component of the whole project. The LED cube is

constructed by the 512 blue LEDs. Each LED has two leads, the longer is the positive lead, and

the shorter one the negative lead. The first steps was to bended all negative leads to 90o which

shows in the figure below.

Page : 2 of 6


Figure AC.1 – Bent Negative Leads

Then, the first slice was built by 8x8 LEDs, in order to keep all LEDs in line. A template board

was used during construction. The template board had 64 equal space holes which 3 cm

separated apart. The diameter of each LED holes was 3 mm which was perfect to fit LED head

inside. After that, the first row of eight holes on template board was filled with LEDs. All

negative leads had to solder on next LED’s negative lead. As shown in Figure AC.1. Then,

another six rows were soldered. After the total eight rows LED done, all positive leads soldered

to the positive leads of following rows. The bended position was at approximately 1.5 mm away

from LED head, as shown in Figure AC.3.

Figure AC.2 – Soldering of an LED Row

Page : 3 of 6


Figure AC.3 – Positive Lead Soldering

After the other seven slides were completed, all slides had to do on board test to ensure all LEDs

work as shown in Figure AC.4. The working voltage of LED is between 3.3~3.6V and working

current is between 0.02~0.03A. The purpose of testing after the slides completed is to find out

the faulty LEDs before the whole cube construction completed. Finally, all LED slides were

inserted into circuit board slots, and then the LED positive leads on last row of each LED slide

were inserted into the slots at the back circuit board. Due to the weak joints between LEDs,

faulty LEDs and nonprogrammable chip, the team decided to purchase a new programmable

LED cube materials. The new cube had 512 blue LEDs and one programmable external chip.

Comparing with the old cube, the new cube’s LED joints were stronger and the built on board

chip leads were perfect soldered. Basically, the team got a stable platform to design and test the

code. The completed cube can be seen in the figures below.

Page : 4 of 6


Figure AC.4 – Testing First Slide of LED’s

Figure AC.5 – Completed LED Cube

Page : 5 of 6


13 REFFERNCES

[1] What Arduino Can do. [online] [Cited: April 3, 2014.] http://arduino.cc/en/Main/Homepage

[2]STC12C5A60S2 series MCU Data Sheet. STCMCU. [online] [Cited: April 3, 2014] http://www.stcmcu.com/datasheet/stc/STC-AD-PDF/STC12C5A60S2-english.pdf

[3]LED Cube 8x8x8. Instructables. [Online]. [Cited: April 4, 2014] http://www.instructables.com/id/Led-Cube-8x8x8/

[4] John Boxall, Arduino Workshop, No Starch Press, May 2013.

[5] J.Eric and Robert Rothery, Design for Electrical and Computer Engineers, Wiley, 2002.

[6] Cube8. DoYoung Studio. [Online] [Cited: April 3, 2014] http://www.doyoung.net/works/CUBE8/index.html

[7] Cube8 User mode information sheet. DoYoung Studio. [online] [Cited: April 3, 2014] http://www.doyoung.net/works/CUBE8/CUBE8_U.pdf

[8] ATmega328. Atmel. [Online] [Cited: Dec 2, 2013.] http://www.atmel.com/devices/atmega328.aspx.

[9] Assembling Cube8. DoYoung Studio. [Online] [Cited: April 3, 2014] http://www.doyoung.net/articles/CUBE8/CUBE8_DYS.pdf

[10] Casey Reas and Ben Fry, Getting Started with Processing, Maker Media, 2010.

[11] LeapMotion Introduction [Online] [Cited: April 3, 2014] https://www.leapmotion.com/product

[12] Overview. A short introduction to the Processing software and projects from the communication [Online] [Cited: April 3, 2014] http://www.processing.org/overview/

http://arduino.cc/en/Main/Homepage

http://www.stcmcu.com/datasheet/stc/STC-AD-PDF/STC12C5A60S2-english.pdf

http://www.instructables.com/id/Led-Cube-8x8x8/

http://www.doyoung.net/works/CUBE8/index.html

http://www.doyoung.net/works/CUBE8/CUBE8_U.pdf

http://www.atmel.com/devices/atmega328.aspx

http://www.doyoung.net/articles/CUBE8/CUBE8_DYS.pdf

https://www.leapmotion.com/product

http://www.processing.org/overview/

Page : 6 of 6


[13] Dreamweaver 8 introduction [Online] [Cited: April 3, 2014] http://www.adobe.com/support/documentation/en/dreamweaver/dw8/releasenotes.html

http://www.adobe.com/support/documentation/en/dreamweaver/dw8/releasenotes.html

http://www.adobe.com/support/documentation/en/dreamweaver/dw8/releasenotes.html