1

Integrating Real-time Collaboration into your Applications

Stephanie LindseyProgram ManagerMicrosoft Corporationslindsey@windows.microsoft.com

Session Code: CLI380

2

AgendaAgenda

OverviewExample SolutionsLonghorn Design Goals Real-time Collaboration APIsSummary

OverviewExample SolutionsLonghorn Design Goals Real-time Collaboration APIsSummary

3

Tools

Client Application Model

Avalon Windows Forms

Web & Service Application Model

ASP.NET / Indigo Win FSCompact

FrameworkYukon Mobile PC Optimized

System.HelpSystem.Help

System.DrawingSystem.Drawing

System.NaturalLanguageServicesSystem.NaturalLanguageServices

Data Systems Application Model

Presentation Data

Mobile PC & Devices Application Model

Communication

Command Line

NT Service

DataSetDataSet

MappingMapping

ObjectSpacesObjectSpaces

ObjectSpaceObjectSpace

QueryQuery

SchemaSchema

ItemItem

RelationshipRelationship

MediaMedia

AudioAudio

VideoVideo

ImagesImages

System.MessagingSystem.Messaging System.DiscoverySystem.Discovery

System.DirectoryServicesSystem.DirectoryServices

System.RemotingSystem.Remoting

System.Runtime.RemotingSystem.Runtime.Remoting

ActiveDirectoryActiveDirectory

UddiUddi

System.Web.ServicesSystem.Web.Services

Web.ServiceWeb.Service

DescriptionDescription

DiscoveryDiscovery

ProtocolsProtocols

System.MessageBusSystem.MessageBus

TransportTransport

PortPort

ChannelChannel

ServiceService

QueueQueue

PubSubPubSub

RouterRouter

System.TimersSystem.Timers

System.GlobalizationSystem.Globalization

System.SerializationSystem.Serialization

System.ThreadingSystem.Threading

System.TextSystem.Text

System.DesignSystem.Design

Base & Application Services

Fundamentals

System.ComponentModelSystem.ComponentModel

System.CodeDomSystem.CodeDom

System.ReflectionSystem.Reflection

System.EnterpriseServicesSystem.EnterpriseServices

System.TransactionsSystem.Transactions

Security

System.Windows.TrustManagementSystem.Windows.TrustManagement

System.Web.SecuritySystem.Web.Security

System.MessageBus.SecuritySystem.MessageBus.Security

AuthorizationAuthorization

AccessControlAccessControl

CredentialsCredentials

CryptographyCryptography

System.Web.ConfigurationSystem.Web.Configuration

System.MessageBus.ConfigurationSystem.MessageBus.Configuration

System.ConfigurationSystem.Configuration

System.ResourcesSystem.ResourcesSystem.ManagementSystem.Management

System.DeploymentSystem.Deployment

System.DiagnosticsSystem.Diagnostics

Configuration Deployment/Management

System.WindowsSystem.Windows System.WindowsSystem.WindowsSystem.Windows.FormsSystem.Windows.Forms

System.ConsoleSystem.Console

System.ServiceProcessSystem.ServiceProcess

System.Windows.FormsSystem.Windows.Forms System.WebSystem.Web System.StorageSystem.Storage System.Data.SqlServ

erSystem.Data.SqlServer

AnimationAnimation

ControlsControls

ControlControl

DesignDesign

PanelPanel

ControlsControls

DialogsDialogs

SideBarSideBar

NotificationNotification

System.WindowsSystem.Windows

DocumentsDocuments

Text ElementText Element

ShapesShapes

ShapeShape

InkInk

UI ElementUI Element ExplorerExplorer MediaMedia

System.Windows.FormsSystem.Windows.Forms

FormsForms

ControlControl

Print DialogPrint Dialog

DesignDesign

System.Web.UISystem.Web.UI

PagePage

ControlControl

HtmlControlsHtmlControls

MobileControlsMobileControls

WebControlsWebControls

AdaptorsAdaptors

DesignDesign

PortsPorts

InteropServicesInteropServices

System.RuntimeSystem.Runtime

System.IOSystem.IO

System.CollectionsSystem.Collections

GenericGeneric

System.SearchSystem.Search

AnnotationsAnnotations

MonitoringMonitoring

LoggingLogging

RelevanceRelevance

System.DataSystem.Data

SqlClientSqlClient

SqlTypesSqlTypes

SqlXMLSqlXML

OdbcClientOdbcClient

OleDbClientOleDbClient

OracleClientOracleClient

CoreCore

ContactContact

LocationLocation

MessageMessage

DocumentDocument

EventEvent

System.StorageSystem.Storage

System.WebSystem.Web

PersonalizationPersonalization

CachingCaching

SessionStateSessionState

System.XmlSystem.Xml

SchemaSchema

SerializationSerialization

XpathXpath

QueryQuery

PermissionsPermissions

PolicyPolicy

PrincipalPrincipal

TokenToken

System.SecuritySystem.Security

System.CollaborationSystem.Collaboration

RealTimeEndpointRealTimeEndpoint

PresencePresence

SignalingSessionSignalingSession

MediaMedia

ActivitiesActivities

HttpWebRequestHttpWebRequest

FtpWebListenerFtpWebListener

SslClientStreamSslClientStream

WebClientWebClient

System.NetSystem.Net

NetworkInformationNetworkInformation

SocketsSockets

CacheCache

System.WebSystem.Web

AdministrationAdministration

ManagementManagement

NavigationNavigation

Peer GroupPeer Group

PolicyPolicy

SerializationSerialization

CompilerServicesCompilerServices

RecognitionRecognition

System.SpeechSystem.Speech

SynthesisSynthesis

4

OverviewOverview

What is Real-time Collaboration?Windows COM API Features Exposed

Presence Contact List, GroupsAudio/Video (point to point)Multiparty Instant MessagingApplication Sharing

What is Real-time Collaboration?Windows COM API Features Exposed

Presence Contact List, GroupsAudio/Video (point to point)Multiparty Instant MessagingApplication Sharing

5

Desktop AppsMessenger ApplicationsCustomer Relationship ManagementIntegration with productivity and LOB applications (e.g. Office)Integrating Audio/Video

Server-Side AppsNotificationsBotsVoice (Telephony and VoIP)

Desktop AppsMessenger ApplicationsCustomer Relationship ManagementIntegration with productivity and LOB applications (e.g. Office)Integrating Audio/Video

Server-Side AppsNotificationsBotsVoice (Telephony and VoIP)

Example SolutionsExample Solutions

6

Helpdesk SolutionHelpdesk Solution

Help Personnel

Area Experts

Bot Employee

7

Why is this hard today?Why is this hard today?

No common platform to do collaborationDevelopers have to understand different APIs and protocols

SIP, SIMPLE, PINT, T.120, RTP, G.711, G.722.1, G.723.1, GSM, DVI4, SIREN, H.263, Acoustic Echo Cancellation, DirectX®, etc.Developers can’t pick and choose the different building blocks

No multipartyAudio/Video is not lifelikeLack of application sharing integrationHard to resolve an identity to a network addressCurrent implementation does not scale well for server-based environments

No common platform to do collaborationDevelopers have to understand different APIs and protocols

SIP, SIMPLE, PINT, T.120, RTP, G.711, G.722.1, G.723.1, GSM, DVI4, SIREN, H.263, Acoustic Echo Cancellation, DirectX®, etc.Developers can’t pick and choose the different building blocks

No multipartyAudio/Video is not lifelikeLack of application sharing integrationHard to resolve an identity to a network addressCurrent implementation does not scale well for server-based environments

8

Real-Time Collaboration Design GoalsReal-Time Collaboration Design Goals

Provide an extensible platform so developers can pick and choose building blocks

Provide common platform so developers don’t have to understand the different APIs

Provide libraries for common collaborative activities

UI controls so ISVs don’t have to rewrite UI

Multiparty is built into all collaboration modes Improve our audio/video stackImprove app sharing experienceAPIs will be fully managed codeSupport both IPv4 and IPv6

Provide an extensible platform so developers can pick and choose building blocks

Provide common platform so developers don’t have to understand the different APIs

Provide libraries for common collaborative activities

UI controls so ISVs don’t have to rewrite UI

Multiparty is built into all collaboration modes Improve our audio/video stackImprove app sharing experienceAPIs will be fully managed codeSupport both IPv4 and IPv6

9

Longhorn Client API ArchitectureLonghorn Client API Architecture

Signaling Plane

A+V

RDP

GXA

Other

Presence Roaming Signalin

g

SIP Protocol Stack

RTC Provider

UI Controls (presence, contacts, IM, a/v, app sharing, file transfer, etc.)

Activities (IM, a/v, app sharing, file transfer, etc.)

Shared Provider

Host

Media Plane

MSN Messenger 3rd Party App

10

Object ModelObject Model

RealTimeProfileProvider

RealTimeProfile

ProfilePresence

ProfilePersistedData

ProfileSignaling SignalingSession

ContactsSubscription

ContactGroupsSubscription

PresenceSubscription

LocalPresence

:

:

11

Profile LogonProfile Logonpublic RealTimeProfile LogOnSample(){

// Create an instance of the RTC SIP provider and profileRtcProvider provider = new RtcProvider();

RtcProfile  profile =         provider.CreateRealTimeProfile("sip:user@microsoft.com");profile.LogOnStateChanged +=        new LogOnStateChangedEventHandler(StateChanged);profile.LogOn();return profile;

}private void StateChanged(

object sender, LogOnStateChangedEventArgs e){    RtcProfile profile = (RtcProfile) sender;

if (profile.LogOnState == LogOnState.LoggedOn)    SetOnlineStateSample(profile);  // next slide  }

public RealTimeProfile LogOnSample(){

// Create an instance of the RTC SIP provider and profileRtcProvider provider = new RtcProvider();

RtcProfile  profile =         provider.CreateRealTimeProfile("sip:user@microsoft.com");profile.LogOnStateChanged +=        new LogOnStateChangedEventHandler(StateChanged);profile.LogOn();return profile;

}private void StateChanged(

object sender, LogOnStateChangedEventArgs e){    RtcProfile profile = (RtcProfile) sender;

if (profile.LogOnState == LogOnState.LoggedOn)    SetOnlineStateSample(profile);  // next slide  }

12

Set Online Presence StateSet Online Presence Stateprivate void SetOnlineStateSample(

RealTimeProfile profile){

// Set the local presence state to Online   

profile.presence.LocalPresence.PresenceState = PresenceState.Online;

   profile.presence.UpdateLocalPresence();

}

private void SetOnlineStateSample(RealTimeProfile profile)

{// Set the local presence state to Online

    profile.presence.LocalPresence.PresenceState = PresenceState.Online;

   profile.presence.UpdateLocalPresence();

}

13

Longhorn Client API ArchitectureLonghorn Client API Architecture

Signaling Plane

A+V

RDP

GXA

Other

Presence Roaming Signalin

g

SIP Protocol Stack

RTC Provider

UI Controls (presence, contacts, IM, a/v, app sharing, file transfer, etc.)

Activities (IM, a/v, app sharing, file transfer, etc.)

Shared Provider

Host

Media Plane

MSN Messenger 3rd Party App

14

Activity ObjectsActivity Objects

High-level task-oriented API on top of the signaling and media APIs

Audio/Video ActivityFlexible multi-party conferencing capabilitiesHandles media negotiationRich set of call control operations

App Sharing Activity Flexible multi-party conferencing capabilitiesUses RDP as well as T.120 for NetMeeting interoperability

File Transfer ActivityInstant Messaging Activity

High-level task-oriented API on top of the signaling and media APIs

Audio/Video ActivityFlexible multi-party conferencing capabilitiesHandles media negotiationRich set of call control operations

App Sharing Activity Flexible multi-party conferencing capabilitiesUses RDP as well as T.120 for NetMeeting interoperability

File Transfer ActivityInstant Messaging Activity

15

Audio/VideoAudio/Video

Easy to use Audio/video media API that provides rich control Improved audio/video quality

New, improved CODECs (RT Voice/RT Video)Jitter controlForward Error CorrectionImproved error-resilience techniques

Improved AEC module in LonghornSecure RTPImproved performance for server-side applications

Easy to use Audio/video media API that provides rich control Improved audio/video quality

New, improved CODECs (RT Voice/RT Video)Jitter controlForward Error CorrectionImproved error-resilience techniques

Improved AEC module in LonghornSecure RTPImproved performance for server-side applications

16

Audio/Video ArchitectureAudio/Video Architecture

RTC A/V EngineSystem audio

capture source

System audio

renderer

Tra

nsp

ort

RT

P

RTC Media API

AEC

AGC

RTC quality controller

System video

renderer

Tra

nsp

ort

RT

P

Ra

w d

ata

inp

ut

Ra

w d

ata

ou

tpu

t

RTC video render engine

RTC video capture engine RTC video encoder

System video

capture source

RTC video decoder

RTC audio render engine

RTC audio encoder

RTC audio decoder

RTC lip-sync controller

RTC audio capture engine

17

Longhorn Client API ArchitectureLonghorn Client API Architecture

Signaling Plane

A+V

RDP

GXA

Other

Presence Roaming Signalin

g

SIP Protocol Stack

RTC Provider

UI Controls (presence, contacts, IM, a/v, app sharing, file transfer, etc.)

Activities (IM, a/v, app sharing, file transfer, etc.)

Shared Provider

Host

Media Plane

MSN Messenger 3rd Party App

18

App Sharing ArchitectureApp Sharing Architecture

KernelKernel

User-ModeUser-Mode

App SharingActivity

T.120User-Mode

Generic Encoder

LH ClientWinSock

WM using NM

Console Display Driver

Mirror DisplayDriver

Driver Miniport

19

Longhorn Client API ArchitectureLonghorn Client API Architecture

Signaling Plane

A+V

RDP

GXA

Other

Presence Roaming Signalin

g

SIP Protocol Stack

RTC Provider

UI Controls (presence, contacts, IM, a/v, app sharing, file transfer, etc.)

Activities (IM, a/v, app sharing, file transfer, etc.)

Shared Provider

Host

Media Plane

MSN Messenger 3rd Party App

20

Instant MessagingInstant MessagingDemo the Instant Messaging ActivityBasic Steps

Create a Provider/ProfileCreate a SessionCreate an ActivitySend MessagesReceive Messages

Demo the Instant Messaging ActivityBasic Steps

Create a Provider/ProfileCreate a SessionCreate an ActivitySend MessagesReceive Messages

21

Instant Messaging Activity

Niko SchuesslerSoftware Design Engineer

22

Longhorn Client API ArchitectureLonghorn Client API Architecture

Signaling Plane

A+V

RDP

GXA

Other

Presence Roaming Signalin

g

SIP Protocol Stack

RTC Provider

UI Controls (presence, contacts, IM, a/v, app sharing, file transfer, etc.)

Activities (IM, a/v, app sharing, file transfer, etc.)

Shared Provider

Host

Media Plane

MSN Messenger 3rd Party App

23

UI ControlsUI ControlsUI controls are on top of the activity APIs to further simplify the developer’s workDevelopers can write very little code to make their collaboration applications UI-enabled right out of the box There will be other people centric controls

(people picker, previous conversations, contact control, etc. from people & groups) CLI310: People and Groups Controls in Windows “Longhorn”

UI controls are on top of the activity APIs to further simplify the developer’s workDevelopers can write very little code to make their collaboration applications UI-enabled right out of the box There will be other people centric controls

(people picker, previous conversations, contact control, etc. from people & groups) CLI310: People and Groups Controls in Windows “Longhorn”

Talk

24

Features Exposed For PDCFeatures Exposed For PDCFeatures Exposed

Basic Presence (Online State)Rich PresenceRoaming (Contact List, Groups, Permissions)Audio/Video (Point to Point)

Activity API (only)

Multiparty Instant Messaging

Features ExposedBasic Presence (Online State)Rich PresenceRoaming (Contact List, Groups, Permissions)Audio/Video (Point to Point)

Activity API (only)

Multiparty Instant Messaging

25

Features Exposed For RTMFeatures Exposed For RTM

Additional Features ExposedMultiparty Audio/Video Improved Audio/Video Audio/Video media APIs for more advanced apps Secure RTPFile Transfer Multiparty Application SharingUI Controls

Additional Features ExposedMultiparty Audio/Video Improved Audio/Video Audio/Video media APIs for more advanced apps Secure RTPFile Transfer Multiparty Application SharingUI Controls

26

SummarySummary

Provide an extensible platform so developers can pick and choose building blocksYou don’t need to understand

SIP, SIMPLE, PINT, T.120, RTP, G.711, G.722.1, G.723.1, GSM, DVI4, SIREN, H.263, Acoustic Echo Cancellation, DirectX®, etc.

to write great presence-enabled collaboration applicationsWe want you to start developing solutions on top of our platformWe welcome feedback

Provide an extensible platform so developers can pick and choose building blocksYou don’t need to understand

SIP, SIMPLE, PINT, T.120, RTP, G.711, G.722.1, G.723.1, GSM, DVI4, SIREN, H.263, Acoustic Echo Cancellation, DirectX®, etc.

to write great presence-enabled collaboration applicationsWe want you to start developing solutions on top of our platformWe welcome feedback

27

For More InformationFor More Information

Hands-on labs 402/403Ask the Experts

Tuesday 7 pm – 9 pm in Hall G,H

MSDNmsdn.microsoft.com/longhorn

Newsgroupmicrosoft.public.winfx.collaboration

Contact infoE-mail: slindsey@windows.microsoft.com

Hands-on labs 402/403Ask the Experts

Tuesday 7 pm – 9 pm in Hall G,H

MSDNmsdn.microsoft.com/longhorn

Newsgroupmicrosoft.public.winfx.collaboration

Contact infoE-mail: slindsey@windows.microsoft.com

28

Related TalksRelated Talks

ARC382: Make Your Software Cooler and Your Life Easier the Next-Generation of Microsoft Networking TechnologiesWSV306: “Indigo”: Building Peer-to-Peer ApplicationsARC343: Introducing the “Longhorn” Identity SystemCLI310: People and Group Controls in Windows “Longhorn”: Enabling Applications for Collaboration and Data Sharing

ARC382: Make Your Software Cooler and Your Life Easier the Next-Generation of Microsoft Networking TechnologiesWSV306: “Indigo”: Building Peer-to-Peer ApplicationsARC343: Introducing the “Longhorn” Identity SystemCLI310: People and Group Controls in Windows “Longhorn”: Enabling Applications for Collaboration and Data Sharing

29© 2003-2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.