N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Isaac Maya, Ph.D., P.E., DirectorIndustry & Technology Transfer Programs

University of Southern California3740 McClintock Avenue, EEB 131Los Angeles, California 90089-2561

Tel: (213) 740-2592Fax: (213) 740-8931

May 2002

The Integrated Media Systems Center:Technology Transfer, Licensing, and

Commercialization ofImmersive Technologies

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC HeadquartersIMSC Headquarters

Downtown LADowntown LA

NSF

Integrated Media Systems Center

An $11 million/yr partnershipamong:

National Science FoundationUniversity of Southern California

School of EngineeringAnnenberg Center for Communication

State of California andthe City of Los Angeles

IndustryComputerHardware and SoftwareEntertainment and BroadcastingTelecommunicationsPublishingAerospace

Other Government AgenciesDARPA, NASA, JPL, U.S. Army

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Video FilesMac: vision_2001.movPC: immersipresence_vision_2001.avi

IMSC Vision:Immersipresence

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Immersipresence:Enabling Technologies and Applications

Manufacturing

EducationArts

Media Communications

Information Management

Sensory Interfaces

Applications

Communication

Immersipresence : Real worldcombined with virtual world

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC Immersipresence Vision: Applications in Entertainment

Animation File Goes Here:PC – panoramic_animation.aviMAC - panoramic_animation.mov

360º Camera

USC Homecoming Game at the Los Angeles Coliseum

Panoramic Visual Immersion: Malibu Hills

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Video FilesMac: ed_far.movPC: education_far term.avi

Video FilesMac: ed_mid.movPC: education_near term.avi

IMSC Immersipresence Vision: Applications in Education

Far-term ViewMid-term View

Collect real world data to informiterative design process for User

Centered Science issues andthe IMSC Research Program

Address technical and intuitiveinteraction issues to put students inthe picture of what they are studying

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Video FilesMac: face_1.movPC: avatars_doug.avi

Animation and Modelingof People for Immersipresence

3D Visual Environment

Avatar simulates live video

Avatar: Digital Cloning

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Hair Modeling for Avatars

� Grow hair on the avatar then you style it

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Animation and Modeling:Expression Cloning

Real Expression can be “cloned” to any model

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Video FilesMac: audio_10.2_channel.movPC: audio_10.2_channel.avi

Immersive Audio 10.2

3D Virtual audio field at thelocation of the ears at all times

Immersive Audio Environment

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Haptics Research

� Force control for haptics rendering� Collision detection� Compression of haptic data� Haptic playback and analysis� Application Research Project: Haptic Museum

CyberGrasp PHANToM (3D Mouse)

Movie File:PC: haptic_2.aviMAC: haptic_2.mov

Movie File:PC: haptic_1.aviMAC: haptic_1.mov

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Media Communications

High Bandwidth Networks: synchronization, latency, bandwidthHigh Bandwidth Networks: synchronization, latency, bandwidth

Wireless SystemsWireless SystemsOptical CommunicationOptical Communication

CompressionCompression

Photonic Photonic NetworksNetworks

QoS Management of IP NetworksQoS Management of IP Networks

Encryption and Network SecurityEncryption and Network Security

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Managing Immersidata

Haptic ApplicationControl

Immersive Audio Text3D Model

Avatar

Video

Immersidata

� Real-time acquisition and playback� Efficient query and analysis� Integration of disparate information sources

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Geospatial Data Integration

� Integration of imagery,maps, vector data,features, databases, andWeb sources

� Research problemsaddressed:� Extraction from

semistructured sources� Support for moving

object queries� Efficient support for

distributed spatial joins� Efficient integration of

heterogeneous data

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Video FilesPC: concert hall_music.aviMac: hall_mus.mov

Immersipresence Environment:RMI Experiment

� Synchronization challenge at three levels:� Storage, network, and display

� First experiment: acquisition and offline rendering from local servers� Second experiment: acquisition and rendering using Internet-2

Acquisition• 16 microphones• 360 º camera

Compression

Immersidata storageand access

Transmission

Rendering

Synthesis of Immersive

Environment

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC Industry Program Categories

� Senior Research Member ($100,000 for Directed Research)� Exclusive access to specific IP and licensing

� Research Member ($50,000 for Unrestricted Research)� General access to IP and licensing

Research Members access IP firstResearch Members access IP first

� Licensing Company� Negotiate license agreements

Licensing Companies avoid research costsLicensing Companies avoid research costs

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e rN a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC Industrial Members andCollaborators

IIIITRI

Taiwan

Microsoft WA

See California Map

Collaborations withGovernment Agenciesand Foundations:

US ArmyDARPANASA

NCR OH

Eastman Kodak NY

DRS TechnologiesPrentice Hall PTR

NJ

NTT DoCoMo Japan

NavTech IL

Intel OR

Time Domain AL

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Silicon Valley

Hollywood

Los Angeles

Silicon Valley

• ConceptLabs• F-X Palo Alto Laboratory• Geometrix• Hewlett-Packard• InterVideo• Lockheed Martin• Sun Microsystems

USCIMSC

IMSC Industrial Members andCollaborators

Collaborations withFoundations:

Toyota USA Foundation

Los Angeles/San Diego

• AboveNet Communications• Boeing• HRL• JPL• Los Angeles Times• NMUA• Panoram Technologies• TMH Corporation• TRW

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Commercial Applications, Products & Services

ConsumerProducts

Companies

IMSC MemberCompanies

Spin-offs andStart-up

Companies

Technology Transfer & Licensing Strategy

IMSC Technologies and ARPs

End Users

ConceptLabs

Internet-basedTechnologies

NASA/SoERTTCUSC/MSB

USC/OTL

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer by the Numbers

� 83 Invention disclosures� 46 Patents filed (6 Awarded)

� 24 Individual Patents Licensed� 38 Software Licenses� 77 Total commercial licenses & technology

transfers

� 7 Spin Off Companies ( 113 employees)

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC Research/Technology Transfer/LicensingSample Accomplishments

� 3D facial modeling andanimation

� Modeling from uncalibratedcameras

� Tracking for augmentedreality

� Head tracking and adaptivesoundfield correction

� Immersive 10.2-channelaudio

� Information integration,storage, retrieval, delivery(spatial querying)

� MPEG-4-based MultimediaDelivery Integration System

� Wavelength divisionmultiplex (WDM) fiber optics

� Compression algorithms� Wireless radio algorithms� Yima, RedHi

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

IMSC Research/Technology TransferRecent Success Story -- End-to-End

� Invention Disclosure� Prototype Development� Licensing� Product Development� Commercial/Strategic

Partner� Commercialization� Deployment

� IMSC� Geometrix� Visionix� Hoya� Optical Shops

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 1 - Recruiting Companies

� Objective: Agree on IP Issues

InterestedParty

NDA/CDA Discussions

SeniorMember

Member

Licensees

NoJoint IP

Joint IP

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 1 - Complication - IP Issues Cloudy

� Goal: Review agreements of behalf of Faculty

InterestedParty

NDA/CDA Discussions

SeniorMember

Member

Licensees

NoJoint IP

Joint IP

ConsultingAgreements

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 2 - Invention and Disclosure

� Objective: Protect IP

Research Discovery InternalDisclosure

Patent

Copyright

Trademark

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 2 - Complication -

Premature Public Disclosure

� Goal: Present benefits to inventors of providing adequate advancenotice of external disclosure

Research Discovery InternalDisclosure

Patent

Copyright

TrademarkPublicationsPresentations

Discussion

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 3 - Licensing

� Objective: Market and license technologies

$’s andRecognition

TechTransferPartners

SeniorMember

Member

ThirdParties

Spin-offsInternalDisclosure

LicensingAgreement

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Technology Transfer/Licensing Process:Step 3 - Negotiations

� Goal: Separate business from legal issues

Fees

Royalties

Equity

Performance

Patenting/Costs

On-going support

Improvements

Commercialization

Infringement

Liability

Indemnification

Warranty

Disclaimers

TechTransferPartners

SeniorMember

Member

ThirdParties

Spin-offsInternalDisclosure

LicensingAgreement

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Commercialization Issues� Technology transfer/licensing process� Non-disclosure/confidential disclosure agreements� Faculty consulting agreements, conflicts-of-interest� Business issues versus legal issues ( boilerplate agreements )� Ownership of IP rights versus licensing ( directed research, joint IP )� Patent costs and infringement/liability defense costs� Licensing fees, royalties, and equity participation

� Faculty spin-offs versus external companies� Licensee performance� Ongoing support of research versus licensees� Improvements on licensed technologies

$ Marketing/Identification of Quality Licensees/Strategic Partners$ Business plan/commercialization activities (Spin-offs as a special case of tech transfer/licensing)

$ Business Feasibility/Plan Preparation$ Raising Funds/Phases to Match Tech & Market Development

$ Business Incubation

N a t i o n a l S c i e n c e F o u n d a t i o n E n g i n e e r i n g R e s e a r c h C e n t e r

Concluding Remarks

� IMSC’s research program has yielded potentiallyvaluable technologies for tech transfer, licensingand company spin-offs

� Managing the process requires the cooperationof IMSC, the SoE, University Counsel and theOffices of Technology Licensing andTrademarks, and the Marshall School of Business

� Spin-off formation and commercialization oftechnologies constitute an on-going process

� Need to Focus on making Business case forTechnologies - key in developing opportunities