MoMEMEX

Evolution of the PAN & The Personal Relational DB.

Freeband Communicationknowledge for personalised and unrestricted information

• Aiming at establishing a knowledge base for a new generation of transparent communication and information exchange between individual people as a new paradigm for communication. It also indicates its economic and societal impact.

• The vision for 2010 is to consider communication and information transfer from the user, not the provider. In contrast to its present fragmented nature, the communication infrastructure will become transparent and abundant in all layers.

• Freeband ambition is to create a knowledge base for the construction and application of the upcoming ambient intelligent environment.

Shift of information

• We are on the edge of a ‘paradigm change’, which will move the centre of information control to the individual. Right under his personal guidance will be a ‘Wearable Digital Assistant’ that will serve as a direct extension of his intelligence.

MEMEX

• “a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility“ – Vannevar Bush in his 1945 essay “As We May Think”.

Bush’s camera on the head

My vision for 2010: MoMEMEX

• A PAN will contain a large number of different sensors that when combined will allow the user to track and store his actions, creating a large personal relational database of the individuals life that can be searched for information, relations, context and meaning.

Personal Area Network (PAN)

• A personal area network (PAN) is the interconnection of information technology devices within the range of an individual person, typically within a range of 10 meters.

• “As electronic devices become smaller, lower in power requirements, and less expensive, we have begun to adorn our bodies with personal information and communication appliances. Such devices include cellular phones, personal digital assistants (PDAs), pocket video games, and pagers. Networking these devices can reduce functional I/O redundancies and allow new conveniences and services”.

– Thomas Zimmerman.

PAN in the Network hierarchy

Internet

WANatm

MANcableadsl

LAN802.x

PANIrDA (3m)Bluetooth (10m)Wifi (100m)Bodylan (2m)

PAN Devices & Trends

INTEGRATION

Classes of PAN devicesFunctional integration will lead to 3 classes based on characteristics

Type: SmallCPU: 1MIPSMEM: 1MBBATT: 1YDISP: <QCIFUSE: HIGHI/O Functions:•Time•Date•Altitude•Direction•Temperature•Pressure•Heartbeat

Type: MediumCPU: 1000 MIPSMEM: 1GBBATT: 1WDISP: VGAUSE: MEDI/O Functions:Location (GPS)•Image (PNG/MP4)•Audio (MP3)•Agenda•Message(IDENT,SMS,MSN,email)•Network (3G,4G)

Type: LargeCPU: 50K MIPSMEM: 1TBBATT: 1DDISP: XGA>USE: LOWI/O Functions:•Datamining•Internet services•Network services•Various programs•Documents & Files

The body is the network

w

PDA

PC

• The natural salinity of the human body makes it an excellent conductor of electrical current.

• PAN technology takes advantage of this conductivity by creating an external electric field that passes an incredibly tiny current (one nanoamp) through the body, over which data is carried.

• Data can be transmitted at 10 Mb/s

• A handshake can exchange electronic business cards between users PDAs.

• Information at your fingertips

Always connected everywhere

802.11g

(W)CDMA

(W)CDMA

internet

GPS / GIS

Mon

5JAN

12

6

39

Bla bla

bar

Personal RelationalDatabase

Temp in C

Air pressure

(mov.) images

Sound, voice, chatHeartbeat

Altitude

Location

Direction

Date & Agenda

Time

Mail

Personal event generation

The storage issueA regular day could generate:

• 8 calls @ 7 min - recorded in mono MP3 22khz 32Kb/s: 13.2 MB• 20 mails @ 3KB each: 60KB• 5 attached documents @ 200KB: 1MB • IM Chatlog: 8KB• 6 Images taken in VGA format JPG (64KB/pic): 384KB• 1 Movie CIF format 3 minutes MPEG4 @200Kb/s: 4.4MB• 24x60 GPS longitude and latitude coordinates 10KB• Other messages (SMS, CallerID): 5KB• Met 2 new people and received e-Business cards: 400KB• 24x60 Temperature, Direction and Pressure. 20K• Received context information from buildings I passed – Train station,

University, café : 100KB

TOTAL storage required: 19.6 MB/day (or 7.13 GB/year)

Digital Life Layer (DLL)

Personal RelationalDatabase

Search EngineIndexing

Data CorrelationDatamining

Pattern recognitionAutomatic context retrieval

Internet

News Server

Weather server

Location server

SOHO Server

w

PDA

PC

Sensory Input

PLL DLL

Digital Life Layer (2)

Physical Life Layer

Digital Life Layer

Internet Layer

Physical Life Layer

Digital Life LayerAction

s&

Events

Actions&

Events

action

Sample questions• Give me a chronological overview of all images I took

during my last stay in Japan.

• Give me a list of all names including latest email address and telephone number of the people I met this year.

• What were the 5 most exiting events in my lifetime?

• What is the total time I spent in France?

• Who has called me most often last two weeks?

• What and where is the furthest place I’ve been from location x? the highest place?

• Show me all the messages I received last Monday

• Give me a list of messages related to “jane’s wedding”

• Show me the location and time when I received the last call from bob.

• How many times have I been to café x? are there any images I took there in the past?

?

Automated media annotation

Date : WED, 12 FEB 2003, Time: 11:34Loc. : Switzerland, VillarsAct. : Skiing holiday

Related media: phone call 12:00email 12/02/03images 11 - 12 - 13

“intelligent” intervention• Repetetive (life) patterns can be recognized by the system and acted upon.

Example: Every 2 weeks I spend a weekend at my parents home in Middelburg. I take the train to Brussel and transfer in Roosendaal. This time I fall asleep. The system notices a deviation from the pattern in location, heatbeat and body temperature. It can conclude that while normally my patterns match awake, they now match a sleeping pattern. It can therefore decide to wake me and ask me if I am still on the right track.

• Lidecker’s Man-Computer Symbiosis (1960):

Man-computer symbiosis is an expected development in cooperative interaction between men and electronic computers. It will involve very close coupling between the human and the electronic members of the partnership. The main aims are 1) to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and 2) to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs.

Social Implications

• Privacy & Security• Reliability• If the calculator made people forget how to calculate,

will this make people forget to remember?• Because the database is incomplete It might help

people remember things they otherwise would have forgotten.

• With IPv6, objects you interact with can communicate to and through you.

Related PojectsMicrosoft Lifebits Project:

• Aim: to produce a database that will allow people to store all their pictures, video, e-mails, letters, texts, and any other information you can think of, all in one place on a PC.

• Reason: solve the giantshoebox problem. We all end up with boxes full of photos, scraps of paper, and general memorabilia from our pasts. What MyLifeBits attempts to do is categorize this information and provide a permanent record.

• The main concern is the storage space required for such a mass of information. Microsoft states that 1,000 gigabyte hard drives will be common place in the next 5 years, and will sell for under US$300.

How to make this a reality

• Index all sensory inputs and their meaningful relations (I.e. location and time = speed)

• Create or use an open standard that can be used by PAN devices to share sensory data. (JXTA, MoMEMEXML?)

• Figure out what the database would look like (SQL, XML?), how it would operate and on what data.

• Create a GUI frontend (PHP?) for the user to make searching easy.

• Wait.

Project planning

? Mar Apr May Jun Jul Aug

Vooronderzoek:•Literatuurstudie•Veldonderzoek•Visie•Probleemstelling

Hoofdonderzoek•Framework•DB & GUI mockup•…

Paper•Visie•Framework•Resultaten•Conclusies

MoMEMEX

sensors&

searching

XML container for sensor data<Sensor>

<SensorName> GPS </SensorName><SensorID> 016 </SensorID><SensorTime> 22:45 </SensorTime><SensorInfo>

<Latitude> 32’48.12 </Latitude><Longitude> 114’53.75 </Longitude>

</SensorInfo> <SensorMeta>

<SensorTime> 22:45 </SensorTime><SensorDate> 09/03/2003 </SensorDate>

</SensorMeta> <SensorDevice>

<Brand> Casio </Brand><Type> CT10x </Type><IP6> fe80::202:b3ff:fe1e:8329 </IP6>

</SensorDevice></Sensor>

System structure

MySQL PHPbob

SPD

MPD

LPD

s

s

s

s s

s

s

sPAN Backend PAN Front-end

JXTAover bodylan

JXTAover WiFi

XML as a database• Use XML as a DB instead of SQL?

– JXTA as the glue between JAVA (VM) and MoMeMeXML.

– Use Xquery to extract information and relations on the XML.

– JXTA is a set of open, generalized peer-to-peer protocols that allow any connected device (cell phone, to PDA, PC to server) on the network to communicate and collaborate.

– Advantages:• Small and elegant

• Portable and scaleable

• Secure