Personal Multimedia Content Management Adrian Hornsby

… … from Sensing to Experiencingfrom Sensing to Experiencing

Outline

… learn from people… a world of data… the notion of sensing… raw sensing… life sensing

… behaviorial information

… social judgment

… contextual awarness… continuum experience… living metadata… ontology… personal content management

Learning Experience

Kolb (1984) model inspired by the work of Kurt Lewin.

Concrete Experience

AbstractConceptualisation

Reflective Observation

Active Experimentation

Learn from people

Source: Sanders (2002)

SAY

THINK

DO

USE

KNOW

FEEL

DREAM

EXPLICIT

OBSERVABLE

TACIT AND LATENT

A world of data

Data are often viewed as a lowest level of abstraction from which information and knowledge are derived.Data is usually collected as the result of experience, observation, experiment or processes within a computer system and may consist of numbers, words, or images.

The notion of sensing

Aristotle: Traditional five senses: sight, hearing, touch, smell, taste. Humans have six additional senses: pain (nociception), balance (equilibrioception), joint motion and acceleration (proprioception & kinesthesia), sense of time, sense of temperature differences (thermoception), and the sense of direction (magnetoception).

Raw sensing

Localization Accelerometers Proximity Touch Light Temperature Speed Humidity Etc.

Life Sensing

What if we could sense people's life ?? SMS Emails Call logging Phonebook Blogs Forums Social Website Galleries ...

Life Logging + Raw SensingLife Logging + Raw Sensing

Life SensingLife Sensing

Behavior, Social Judgment, ContextBehavior, Social Judgment, Context

PatternExtraction

Life Sensing

Pattern Extraction Who?? What ?? When ?? Where ?? With Whom ??

: Every Sunday, when with my family, at my grandma´s cottage, we take a traditional family picture and I send it to all the members of my family.

Recursive ExperienceRecursive Experience

Source: SeeSaw Adv. 2008

Life Sensing

Behaviorial Information

Exhibited by human beings and influenced by culture, attitudes, emotions, values, ethics, authority, rapport, hypnosis, persuasion, coercion and/or genetics.

Geographic Historic Associative Emotive etc...

Life Sensing

Social Judgment

Source: Sherif, Sherif, & Nebergall (1965)

The idea of Social Judgement is that attitude change is mediated by judgmental processes and effects used to persuade people.

Personality Mood Intelligence Education Sincerity Trustworthiness

Life Sensing

Contextual Awareness

Any information that can be used to characterize the situation of an entity, be it a person, place, object that is relevant to the interaction between a user and an application, including themselves.

Self- Awareness Group- Awareness World- Awareness

Experience

Raw

Sensing

Behavior

Contextual Awareness

Social

Judgement

Continuum Experience

Raw Sensing

Social Judgement

Contextual Awareness

Continuum Experience

Behavior

Before, Now and AfterBefore, Now and After

Living Metadata

Extract Combine Propagate Utilize Refresh

Ontology

“An ontology is an explicit specification of a conceptualization” (Gruber, 1995)

“A conceptualization is a set of conceptual relations defined on a domain space”

Provides ways to define new relations when needed and defined in a detailed manner.

Experience- based

Ontology

Upper: Raw Data, Life Patterns, Time, Location

Domain: Family, Sport, Tourism, Work...

Task: Voice, Conversation, Video, Audio, Photo

Application Ontology

LivingMetadata

Experience-basedOntology

PCM

MultimediaContent

Continuum Experience

Sensing

Conclusion

Full Sensing potential to enrich multimedia content Raw sensing Life sensing

Close the loop Continuum experience Living metadata Experience-based Ontology

Experience-basedExperience-basedPCMPCM