REU 2007 presents AVARI Animated Virtual Agent Retrieving Information Your Name2 2 Vision, Communication Shell, & Conversational Flow Lauren Cairco, primary Toni Bloodworth, secondary Your Name3 3 Background & Motivation [Thorrison, 1998] [Cassell, 2001] [Babu, 2005] [Simmons, 2002] Your Name4 4 Who is AVARI? Your Name5 Who is AVARI? Animated Virtual Agent Retrieving Information Answers questions about members of the computer science faculty of UNC Charlotte 5 Your Name6 6 General Architecture Animated Agent (Haptek) Information Display (HTML/Javascript) Vision (Matlab) Voice Recognition (SALT) Database (MySQL) Communication Shell (Javascript) Question Resolution Algorithm (PHP) Your Name7 7 My Contributions Vision Communication Shell Conversational Flow Your Name8 8 Vision Background subtraction using Matlab Your Name9 9 Vision Camera looks at the floor to see if anyone is standing in front of Avari Begins conversation when she sees someone Ends conversation when person leaves Your Name10 Vision Your Name11 Communication Shell Writing code to hold all the pieces together PHP (PHP Hypertext Processor) SALT (Speech Application Language Tags) Haptek (for virtual human) Matlab (for vision) Communicating with group members about needs from other pieces Version control Your Name12 Conversational Flow Greeting Get Professor Name Get Category Get Question Goodbye Say Answer Your Name13 Greeting Get Professor Name Get Category Get Question Goodbye Say Answer Conversational Flow Get Professor Name Get Category Keep professor? Keep talking? YesNo YesNo Prompt for professor User says professor name Keep category? YesNo Prompt for category User says category Your Name14 Conversational Flow Avaris phrases that prompt users are randomly selected from text files An additional screen displays information that is relevant to the conversation Avari gives users clearer cues if she does not recognize their speech multiple times Keywords on the other screen Example questions Your Name15 Sample Conversation Your Name16 Challenges in Design Predicting what people will say to Avari Deciding how to best direct conversation while keeping it natural Your Name17 Challenges in Implementation Communication problems Server side and client side languages Integrating vision software Security restrictions in Internet Explorer Timing and efficiency considerations Your Name18 Skills I Learned PHP, SALT, MySQL, Matlab, Java, Javascript, Haptek How to work with other peoples code Debugging without a compiler Your Name19 Vicky Fowler Animated Agent Morris Leblanc Speech Recognition Louis Fletcher Database Toni Bloodworth Question Resolution Algorithm Your Name20