american sign language alphabet app (project #2) (version 2 of...

American Sign Language Alphabet App (Project #2) (Version 2 of App Inventor) Description: This App will take text input and then translate each letter of the text to the corresponding American Sign Language Image. We will also use the TextToSpeech object in App Inventor to speak each letter as it is translated by the App. Part 1: User Interface Design

Process: 1. Go to http://ai2.appinventor.mit.edu/ and log in with your Google Account if necessary. 2. Select “My Projects -> Start new project”

3. Name the project “ASLAlphabet”. Click OK.

4. You will be presented with a blank App Inventor Project:

5. Now it is time to design the User interface. First, Click on ‘User Interface’ on the Palette and drag an Image object to the screen.

6. Note that the Image1 object is small and in the upper left corner. This is acceptable for now. The Image object will grow once we attach an image resource. We will change the name of the object. Click on the “Image1” in ‘Components’ and select ‘Rename’. Change the name to ‘imageView’ and click OK.

7. Now drag a ‘Label’ object from the Palette.

8. We need to adjust the properties of the Label1. First, click on ‘Label1’ on the Components and set the ‘Width’ property to ‘fill_parent.’ Click OK.

9. Change the Label1 Text property to no text. (Delete ‘Text for Label1’ in the Text field in Properties).

10. Change the name of ‘Label1’ to ‘textPhrase’ by using the ‘Rename’ button.

11. Next, drag a TextBox object to the Phone screen area:

12. Set the following properties for the TextBox1: a. Set the width to “fill_parent” b. Set the Hint to ‘Enter Phrase” c. Rename to ‘editPhrase’

13. We will now add a Button that the User will press to start the translation. Drag a Button to the phone screen.

14. Change the following properties on the Button: a. Change the width to “fill_parent” b. Change the text to “Translate” c. Rename to “buttonTranslate”

15. We will now add a button for the User to press and have the phone speak the phrase. Drag another button to the screen area.

16. Change the properties of the Button to: a. Change the width to “fill_parent” b. Change the text to “Speak” c. Rename to “buttonSpeak”

17. We need two ‘non-visible’ objects for the App functions. Drag a Clock object to the phone screen area and rename it to ‘timer’. Note that the timer shows up as a non-visible component. Uncheck the ‘TimerEnabled’ Property.

Uncheck this box!

18. Click on ‘Media’ on the Palette and drag a ‘TextToSpeech’ object to the phone screen. Rename the object to ‘textToSpeech’. Again, note that it is a non-visible component.

19. We are finished with the User Interface design. In the next section we will upload the ASL Alphabet images used in the Application.

Part 2: Image Resource Upload Image and sounds play a vital role in Assistive Technology Applications. For students who cannot read, the use of images and icons empowers the user to use the Application. In App Inventor, images and sounds are stored in the Media section. We will upload gif images of the 26 letters and space for the American Sign Language Alphabet. Process: 1. Download the following .zip file and extract: (For this example, I will extract to the downloads folder) http://nebomusic.net/appinventorlessons/aslapp/aslalphabet.zip 2. Go back to your ASL App project and find the Media section. Click ‘Upload File’

3. Select “Choose File”

4. Navigate to where you saved the aslalphabet images folder and select the ‘a.gif’ image. Click “Open”

5. Click “OK”

6. Note that now the ‘a.gif’ file is located in the Media resources:

7. We will now map the a.gif to the imageView object. Click on imageView on the Components section. Then click on ‘Picture’ in the Properties and select ‘a.gif’ and click OK.

8. Now upload the remaining alphabet pictures to the Media area. (‘b.gif’, ‘c.gif’ . . .) (Yes, you have to do this one image at a time . .sorry . . .).

9. You are done with the image uploads. We will now begin to program the App!

Part 3: Programming the ASLAlphabet App. Description: The algorithm for this project depends on the concept of the App moving letter by letter from the input phrase and then showing the corresponding picture:

In order to do this, we will use the Computer Science idea of an ‘Array’ to hold both a list of the picture names and individual letters: Array: aslImages Array: aslLetters

Note that the position of the image and the letter correspond (a is 1, b is 2, c is 3 . . .)

Process: 1. Click on the ‘Blocks’ button to go to the block programmer.

2. We need to create some variables to help control the flow of the App. Click on ‘Variables’ and drag an ‘initialize global’ block to the programming area.

3. Change the name of the variable to ‘letterIndex’

4. Drag a number value 1 to the socket in letterIndex. This is like saying letterIndex = 1. (You find the number blocks in “Math”)

5. We now need to make a Text variable that will store the text as we translate the world. Drag a new ‘initialize global’ block into the programming area and set the name of the variable to ‘displayText’.

6. Drag a Text block and connect it to the socket in displayText. (We want the text to be empty – this the empty quotation marks).

7. Let’s run a quick experiment. We want the App to Speak when the ‘buttonSpeak’ is clicked. This is called an ‘Event’ in programming (when the user clicks a button . . ). Click on ‘buttonSpeak’ on the Blocks and drag a ‘when buttonSpeak Click’ block to the programming area.

8. Now click on the ‘textToSpeech’ object and drag a ‘call texttoSpeech.Speak’ block and place it inside the buttonSpeak Click block.

9. The ‘textToSpeech.Speak’ block has a socket for a message. We want to speak the text the User types into the editPhrase box. So, click on ‘editPhrase’ and drag an ‘editPhrase.text’ block to the message socket.

10. We are ready to test part of the App! See http://appinventor.mit.edu/explore/ai2/setup.html for detailed instructions on connecting to an emulator or a phone. For this tutorial, I will describe how to connect to an emulator. Make sure you have ‘aiStarter’ running on your computer. On Windows you will see a screen that looks like:

11. Click on “Connect” and Select “Emulator”

12. You will see a message like this:

Be patient, it will take the Emulator a few minutes to start on the first try. 13. You will now see messages informing you that the emulator (or phone) is loading AiCompanion.

14. When the Emulator (or device) loads the App – it should look like:

15. Type a phrase into the Enter Phrase box and click the Speak button. The App should talk!

16. The fun ensues . . . (Students love to make their phones talk) Now we will continue with the translation part of the project.

Hello Mr. Michaud!

17. We now define and build the Array that holds the references to the images. Drag a new ‘initialize global’ block to the program area and change the name to aslImages.

18. Click on ‘Lists’ on the Blocks and connect a ‘make a list’ block to the socket of the ‘aslImages’ block.

19. Place a text block in the first socket of the ‘make a list’ block.

20. Type ‘a.gif’ in the text block.

21. Place an additional text block with ‘b.gif’ in the next socket:

22. We are now out of sockets. We need to add additional sockets by clicking on the blue squares in the ‘make a list’ block.

23. Drag the ‘item’ block into the ‘list’ bracket in the dialog. (You will see three ‘item’ blocks after the drag).

24. Notice that after you drag the ‘item’ block – an additional socket appears on the ‘make a list’ block. Drag another ‘item’ to create another socket.

25. Place a ‘c.gif’ and a ‘d.gif’ in the ‘make a list’ block.

27. Now create a new list called “aslLetters” (drag an ‘initialize global’ block and a make a list block.)

28. Repeat the previous procedures and add “a”, “b”, “c”, and “d” text blocks to the aslLetters array.

29. We are going to leave these Arrays at “a” through “d” for now. Eventually, all the letters have to defined in each Array. The directions will continue with the script to identify and display the ASL images.

30. Click on the ‘timer’ object and drag out a ‘when timer.Timer’ block:

31. The ‘timer.Timer’ script contains the algorithm that will: -identify the current letter in the word -add the current letter to the displayText variable -display the displayText in the textPhrase object -Find the matching picture from the aslPictures array. -Display the matching picture -increase the letterIndex count -check to see if the count is greater than the phrase.

32. To start, create a new variable named “currentLetter” and set it to blank text:

33. In the ‘timer.Timer’ block, drag a ‘set currentLetter to’ block inside the ‘timer.Timer’ block.

34. In the Text blocks, drag a ‘segment’ block into the socket of the ‘set currentLetter’ block:

35. Place an ‘editPhrase.Text’ block in the text socket of the ‘segment’ block.

36. Place the ‘get letterIndex’ variable value block into the ‘start’ socket.

37. Place the number 1 in the ‘length’ socket of the ‘segment’ block.

38. Place a ‘set global displayText’ block in the ‘timer.Timer’ block.

39. Go to Text and place a ‘join’ block in the socket of the ‘set global displayText’ block.

40. Connect a ‘get global displayText’ block to the first socket of the ‘join’ block.

41. Connect a ‘get global currentLetter’ in the second socket of the ‘join’ block.

42. We now want to speak the current letter. Place a textToSpeech.Speak block and a ‘get currentLetter’ block in the timer.Timer block.

43. We now want to display the ASL image of the currentLetter. Drag a ‘set imageView.Picture To’ block into the timer.Timer block. (Look in the imageView object).

44. Go to ‘Lists’ and place the ‘select list item’ block into the imageView.Picture block.

45. Place a ‘get global ‘aslImages’ block into the ‘list’ socket.

46. Drag a ‘index in list’ block from ‘Lists’ and place it in the ‘index’ socket.

47. Place a ‘get global currentLetter’ block in the ‘thing’ socket.

48. Place a ‘get global aslLetters’ in the ‘list’ socket.

49. Now we need to advance the letterIndex to the next letter. Place a ‘set letterIndex’ block in the timer.Timer block.

50. To add 1, place an addition block in the ‘set global letterIndex’ socket. Place a ‘get letterIndex’ in the fist space and a 1 in the second.

51. Now we need to check to make sure we stop the timer when the letterIndex is greater than the length of the phrase. Place an ‘if ‘ block in the timer block. (‘if’ blocks are found in the ‘Control’ category).

52. Place a ‘>’ block in the ‘if’ socket.

53. Place a ‘get letterIndex’ in the first socket of the > block.

54. We are comparing to the length of the editPhrase text. Click on ‘Text’ and place a length block in the second socket of the > block.

55. Now place an ‘editPhrase.Text’ block in the length socket.

56. We now need to program the ‘then’ part of the if statement. First, we will set the timer.TimerEnabled to false to stop the timer.

57. Now set the value of letterIndex to 1 so we can start over.

58. We need to add a block in the middle of the sequence to display the ‘displayText’ in the textPhrase object. Add the following blocks (indicated by the arrow).

59. Congratulations – you finished the timer.Timer block! We need to set up one more event with the buttonTranslate to finish the App. Drag a when buttonTranslate clicked to the program area.

60. Place the following block expressions into buttonTranslate.Click: a. displayText = “” (Blank) b. letterIndex = 1 c. set timer.Enabled to true

61. Congratulations! You finished the App. Test on the emulator to see if it works!