Applied COIN LabAnatole Gershman, Alan Black, Louis Van Ahn

TA: Mehrbod SharifiAugust 26, 2008

Who are we?

Alan W Black

Luis von Ahn

Anatole Gershman

Mehrbod Sharifi

Why are we teaching this class?

This class is about inventing Applications

People don’t use computers – they use applications

In your future jobs you’ll be building applications

What kinds of applications will we see in the future?

Future applications will:

Sense the world through networks of sensors;

Think by making sense out of massive amounts of information; and help us

Act in the real world

Example:

Your smart phone senses that you are at the airport; it checks your schedule and concludes that you are there to take flight 123 to Chicago; it acts by displaying the departure gate

Is a smart phone a product or a service?

The distinction between physical products and services will largely disappear:

most services will be provided through physical products

most physical products will become conduits for services

Applications will combine physical products and servicesThey have to be designed together

What will you be doing in this class?

You will be inventing and building applications that combine physical products and services and can sense think and act

You will work in small groups or individually

We will provide project ideas, tools and advice

We expect you to submit:

a project proposal

a project design and mock-up

a working demo

a project report

We expect to hear from you on a weekly basis

Class Schedule

8/28/08 First Class

9/4/08 Proposal + Groups (1-3)

9/11/08 Mockup (Specs and PPT)

9/18/08

9/25/08 Final Plan and starting work

10/2/08

10/9/08

10/16/08

10/23/08

10/30/08 Mid Point Milestone

11/6/08

11/13/08

11/20/08

11/27/08 No class - Thanksgiving

12/4/08 Project presentations

12/11/08 Project presentation + final report

Deliverables

1. Project proposal (1-2 pages) due next Thursday

What will the prototype do?Why is this interesting and important?What kinds of tools and equipment will be necessary?Who will be the team members?

2. Mockup and preliminary specs of the system due 9/11

A sequence of Power Point slides depicting a demo scenarioPreliminary architecture and specsPreliminary selection of tools and equipmentPreliminary assignment of roles and responsibilities

3. System specs and work plan due 9/18

Architecture and design decision justificationsTools and equipmentRoles, responsibilities, schedule

Deliverables (cont.)

4. Weekly progress reports

One page summary of the last week accomplishments and goals for the next week

5. Final working prototype

A 10min demonstration, explaining what it does, why it is important and how it does it

6. Final report due 12/11

A 5 page paper describing:Project goalsSystem designUse scenarioPotential applicationsLessons learned

Project ideas: Smart Online Cabinet

How would the cabinet know what’s inside?

RFID tags and reader

How would it recognize the user?

camera, voice?

What other sensors should it have?

What services should it provide?

How do we communicate with it?

When and what should it say and when should it remain silent?

Project ideas: Smart Mobile Assistant

Your smart phone or PDA knows where you are and can record and play back snippets of information in the right context

For example, while meeting someone new, you might record a snippet of the introduction when you say:

“Hi John, nice to meet you”Later, you would be able to ask:

“Wednesday, noon, cafeteria”Your assistant will play the snippets recorded at that time in that location

Project ideas: Smart Objects

If this plant could talk, what should it say?

What services might it provide?

What about the door to your house or your office?

Project ideas: Interactive Picture

Remember pictures on the walls of Hogwarts School of Magic?

We already have the software and the hardware to track your position in front of the picture

What applications would be interesting and useful?

Project ideas: Tricorder

Or the remote control to the world

Scan the environment to see what’s controllable

Provide standard control interfaces

Example:

Point the tricorder at an interesting car in the street, click the “copy” button then find the nearest big screen, get a search engine and “paste” the car into it.

Project ideas: Speech Eliza

Joseph Weizenbaum, MIT 1966

20Q A.L.I.C.E. and AIML Chatterbot Dr. Sbaitso ELIZA effect Jabberwacky List of Chatterbots Loebner prize PARRY Racter Simulated consciousness Turing test Virtual Woman

Project ideas: Smarter Virtual Worlds

A school yard in Second Life

Robotic kids have internal states and behaviors governed by rules

They can sense each other location and behavior and engage in conversations

Verbal combat between characters

Tools and equipment

RFID kitsPDAsSmart Phones

Bluetooth beacons

Laptops Webcams Headsets and speech kits

Sensor kits

Questions?

Coin.scribblewiki.com

All course information will be on Scribble Wiki

1. Project proposal (1-2 pages) due next Thursday

What will the prototype do?Why is this interesting and important?What kinds of tools and equipment will be necessary?Who will be the team members?

2. Mockup and preliminary specs of the system due 9/11

A sequence of Power Point slides depicting a demo scenarioPreliminary architecture and specsPreliminary selection of tools and equipmentPreliminary assignment of roles and responsibilities