Designing Kinect based games for children with Autism

These games are a fun interactions with one and twoplayer modes to encourage social and motor skills inchildren with Autism.

Designing: The games are prototyped based on usercentered design process with the teachers at the LionheartSchool, Alpharetta. The prototypes are iterated based ontheir requirements and feedback. There are two modes ofrepresentation of the child - 2 D Skeleton figures, GreenScreen Image.

Aug ‘13 – May’15, Georgia Institute of TechnologyFaculty Mentors: Dr. Agata Rozga, Dr. Brian Magerko, Dr. Gregory AbowdStudent Collaborators: Mirko Gelsomini, Patricia Perez

Evaluation: The requirement data is qualitatively analyzed using affinity modelling and the impactof the games on social and motor behavior is quantitatively and qualitatively analyzed by video-coding the childrens’ game play sessions.

Other technology probes included: Afreeform interactions that also mapped theKinect’s skeleton joints to 2D characterslike iron man, princess and Mr. Bean.

A story telling application with simple 2Danimations for various movements ofanimals like flying, jumping and hoppingand also explored gesture-detection withKinect .

Technology: The front end application was a web application designed using Html, css,javascript and jquery. The Kinect joints were detected by a C# application which sent theskeleton information to the web application using socket communication.

Video Game Design

I am working on Unity-3D to createmini games as a part of theassignments I am working on in myVideo Game Design Class. This is alink to the blog page that hosts. Myblog alias is “wolverine”. You willneed to install Unity Web Player toplay these games.

http://ael.gatech.edu/cs4455f14/author/wolverine/

I am currently working on creating more games for children with Autism on the Unityplatform with Kinect plugins .

Aug ‘14– Dec’14, Georgia Institute of Technology

The Worker HelperFeb ‘14– May’14, Georgia Institute of TechnologyHuman Computer InteractionStudent Collaborators: Varun Gatne, Keerthi Amurgam, David StoneCypher

The Worker helper is a prototype of a smart phone application designed for custodians (janitors) to facilitate communication, eliminate extensive paper work required to file complaints and order supplies.

User Study: We approached 10 custodians in 5 buildings at Georgia Tech, out of which 6 agreed to try our prototype and take our survey. 66% of them were positive that the app will be an useful in their work.

Low Fidelity Prototyping: Designed 4 main features into our app, text and voice chat, complaint and order. Android phones were taken to the janitors and a survey was taken after the usability testing.

Survey Design: Analyzed the demography and the response of the surveys. We also had qualitative input on changing certain names and features as per their daily usage and requirement for further iteration .

Posture Sensing Garment - NASA

Jan ‘14– May’14, Georgia Institute of TechnologyMobile and Ubiquitous ComputingStudent Collaborators: Shailesh Lohia, LouisFischer, Parth Saigal

• We used flex sensors on toa stretchable unitard garment andprogrammed the out put on to ateensy board to map the voltagechange due to stretching of thesensor to the angular displacementof the joint angle.

• Nine sensors were sewed on andconnected using conductive fiberfor both the upper and the lowerbody.

This was presented at the Wearable ComputingSymposium, NASA in February 2014.

Kinect based peer to peer Yoga TrainerSept ‘13– Nov’13, Georgia Institute of TechnologyPrinciples of User Interface DesignStudent Collaborators: Meghna Mehta

Desktop application: Kinectposture detection was doneusing Microsoft SDK forskeleton tracking in C#.

3D modelling: and mappingof skeleton joint data to theapplication UI for feedback.

Data transmission: Joint data transferred over remote network using php request response andtemporary storage in Mysql database.

Low Fidelity Prototyping

Figure shows a prototypedredesign of the coffee vendingmachine in Axure.

Other methods used in variousprojects include story-boarding,task analysis, paper-prototyping, balsamiq, html-cssprototyping, presentations andsequence modelling.

A 3 by 3 Matrix game for Eye-tracking technique using Artificial Neural Networks

• Image processing: done in Matlab to detect pupil and track eye-movement.

• Game UI: Created simple matrix games in flash to detect words.

• Artificial neural networks: used feed forward back propagation method to do training and mapping the eye coordinates to the desired gaze location on the screen.

July ‘12– April’13, Mumbai, IndiaB.E. ProjectStudent Collaborators: Pranita Desai, Aarti GuptaFaculty Mentor: Sharmila Sengupta

Automated Lip Reading

Image Processing: simulated simple words like "hello", "yellow", "yes","no" for lip contour detection and pattern storage in the form of image sets of 4 by 4 frames.Verified test cases for false negatives and false positives.Extended its application to speech for the disabled and automated password detection.

July ‘11– April’12, Mumbai, IndiaStudent Collaborators: Pranita Desai, Aarti GuptaFaculty Mentor: Sharmila Sengupta

Mobile App for predicting Home-Budget

Android Application :programmed in eclipse.Used php graph and Mysqldatabase to store the user'sdata and retrieve it in graphinterface.Prediction algorithm: allowedthe user to predict the amountof time that the user wouldtake to save up a desiredamount depending on theuser's past expenses andmonthly income.

Mumbai, India

CraYon: A Campaign Website against Child Abuse

Web Application:Front end designing: Used Html, css and javascript.Created forms and form validation using javascript.

Back end designing: Used JSP and JDBC connectivity to store and retrieve data from SQL server.

Pattern recognition and matching using sql queries enabled for site admin to flag and delete spam or abusive comments.

Mumbai, IndiaStudent Collaborator: Jayati Ambekar