say what you will say and listen to what you will listen to …

Say what you will say Say what you will say and listen to what you and listen to what you will listen to …will listen to …

About the importance of contracts About the importance of contracts and metadata…and metadata…

Clemens Vasters, newtelligence AGClemens Vasters, newtelligence [email protected]@newtelligence.com

© 2003 newtelligence Aktiengesellschaft, Microsoft Corporation – No unauthorized reproduction or distribution permitted. All rights reserved

This talkThis talk

Thinking about dataThinking about data Thinking about typesThinking about types Thinking about passing typed dataThinking about passing typed data Thinking about exchanging dataThinking about exchanging data Thinking about behaviors around Thinking about behaviors around

datadata

How is this relevant to today’s How is this relevant to today’s launch event?launch event?

FF FE 43 00 6C 00 65 00 6D 00 65 00 6E 00 73 0020 00 56 00 61 00 73 00 74 00 65 00 72 00 73 002C 00 20 00 47 00 69 00 6C 00 6C 00 65 00 73 0068 00 FC 00 74 00 74 00 65 00 20 00 39 00 39 002C 00 20 00 32 00 31 00 33 00 35 00 32 00 20 004B 00 6F 00 72 00 73 00 63 00 68 00 65 00 6E 0062 00 72 00 6F 00 69 00 63 00 68 00 2C 00 20 0047 00 65 00 72 00 6D 00 61 00 6E 00 79 00

There …There …

Clemens Vasters, Gilleshütte 99, 41352 Korschenbroich, Germany

… … are many …are many …

NAM,Clemens VastersSTR,Gilleshütte 99CIT,Korschenbroich*41352CTY,Germany

… … ways to …ways to …

<Address> <Name>Clemens Vasters</Name> <Street>Gilleshütte 99</Street> <City>Korschenbroich</City> <Postcode>41352</Postcode> <Country>Germany</Country></Address>

… … represent …represent …

Address address = new Address( “Clemens Vasters“, “Gilleshütte 99“, “Korschenbroich“, “41352“, “Germany“);

… … information and …information and …

DataSet ds = GetDataSet();DataTable dt = ds.Tables[“Address“];dt.Rows.Add( new object[]{ “Clemens Vasters“, “Gilleshütte 99“, “Korschenbroich“, “41352“, “Germany“});

…….none is wrong ….none is wrong …

[email protected]

…….or right.or right


A Billion € Problem…A Billion € Problem…

Many ways to represent informationMany ways to represent information Many ways to turn information into Many ways to turn information into

“data”“data” Many ways to represent dataMany ways to represent data

The difficulty is not choice, it’s The difficulty is not choice, it’s agreement.agreement.

Agreement requires eliminating Agreement requires eliminating ambiguity.ambiguity.

Agreement != DemocracyAgreement != Democracy

var FirstName;

Attempting to agree:Attempting to agree:Name and LocationName and Location

wchar_t * FirstName;

Attempting to agree:Attempting to agree:Name and typed LocationName and typed Location

string FirstName;

Attempting to agree:Attempting to agree:Name, Location, Type and Implemented Name, Location, Type and Implemented BehaviorBehavior

<item>Clemens</item>

Attempting to agree:Attempting to agree:Location and DataLocation and Data

<FirstName> Clemens</FirstName>

Attempting to agree:Attempting to agree:Name, Location and DataName, Location and Data

<FirstName xsi:type="xsd:string"> Clemens</FirstName>

Attempting to agree:Attempting to agree:Name, Location, Type and DataName, Location, Type and Data


Elements of Agreement Elements of Agreement (Step 1)(Step 1) NameName: Common understanding of : Common understanding of

names for data elements and their names for data elements and their interpretationinterpretation

LocationLocation: Common understanding of : Common understanding of (relative) location of data inside a (relative) location of data inside a contextcontext

Type:Type: Common understanding of Common understanding of persistent or transient persistent or transient representation of data elementsrepresentation of data elements

Data:Data: Representation of information Representation of information under agreement on name, location under agreement on name, location and type.and type.


How about How about implementation?implementation? Sorry, we can't reach agreement on Sorry, we can't reach agreement on

that ...that ... Implementation realizes requirements Implementation realizes requirements

in the scope of a (sub-)system contextin the scope of a (sub-)system context How do you agree on implementation How do you agree on implementation

across applications, frameworks and across applications, frameworks and languages; organizations and software languages; organizations and software suppliers?suppliers?

Impl. agreement contradicts Impl. agreement contradicts modularitymodularity Same behavior of data for all tiers and Same behavior of data for all tiers and

layers?layers? Same behavior of data for all modules?Same behavior of data for all modules? Same behavior of data for all Same behavior of data for all

applications?applications?


Where does that lead us?Where does that lead us?

We can agree on data, types and We can agree on data, types and location.location.

We can't agree on implementation.We can't agree on implementation.

Data can be exchangedData can be exchanged Implementation cannotImplementation cannot

Conclusion: Conclusion: Trying to agree on data and Trying to agree on data and implemen-tation (aka. "objects") implemen-tation (aka. "objects") might be a problem.might be a problem.

How good are your types?How good are your types?

string FirstName = "Q?C&89/&%$$";string FirstName = "Q?C&89/&%$$";


string FirstName = null;string FirstName = null;


string FirstName =string FirstName ="This is a very long string "This is a very long string which will likely not fit into which will likely not fit into your database, because the your database, because the database field for this element database field for this element does only allow 40 characters does only allow 40 characters to be stored and this is easily to be stored and this is easily longer";longer";


Agreeing on types is hard Agreeing on types is hard (1/2)(1/2) Data types describe storage and Data types describe storage and

behavioral rules.behavioral rules. Agreeable storage requirements:Agreeable storage requirements:

Strings: Bounded sequences of Strings: Bounded sequences of characters,characters,Integers: Bounded, integral numbers,Integers: Bounded, integral numbers,Floats: Bounded, floating point Floats: Bounded, floating point numbersnumbers

Agreement on storage rules is still Agreement on storage rules is still scoped: CLR, XML, JVM, SPARC, IA64, scoped: CLR, XML, JVM, SPARC, IA64, IA32IA32


Agreeing on types is hard Agreeing on types is hard (2/2)(2/2) Can't agree on implemented Can't agree on implemented

behavior, but on rules for behaviorbehavior, but on rules for behavior 1 + 1 = 21 + 1 = 2 "1"+"1"="11""1"+"1"="11" 2003-02-29 bad, 2004-02-29 good2003-02-29 bad, 2004-02-29 good FirstName: minLength=1, FirstName: minLength=1,

maxLength=40, pattern="\p{L}+"maxLength=40, pattern="\p{L}+" Particular record may only appear once Particular record may only appear once

in a sequence (uniqueness)in a sequence (uniqueness)


Common groundCommon ground

XML 1.0 provides storage rulesXML 1.0 provides storage rules XML Schema provides extensible XML Schema provides extensible

type settype set XML Schema provides ability to XML Schema provides ability to

restrictrestrict

XML Schema is a way to express XML Schema is a way to express (and impose) agreement on data!(and impose) agreement on data!


RestrictingRestricting<xsd:simpleType name="nameType"><xsd:simpleType name="nameType"> <xsd:restriction base="xsd:string"> <xsd:restriction base="xsd:string"> <xsd:pattern value="\p{L}[\p{L}\p{P}0-9\s]+"/> <xsd:pattern value="\p{L}[\p{L}\p{P}0-9\s]+"/> </xsd:restriction> </xsd:restriction></xsd:simpleType></xsd:simpleType>

<xsd:simpleType name="phoneNumberType"> <xsd:restriction base="xsd:string"> <xsd:pattern value="[0-9\+][0-9\-\s]*" /> <xsd:maxLength value="26" /> </xsd:restriction></xsd:simpleType>


ComposingComposing<xsd:complexType name="addressType"> <xsd:sequence> <xsd:element name="City"> <xsd:simpleType> <xsd:restriction base="nameType"> <xsd:maxLength value="80" />

</xsd:restriction> </xsd:simpleType> </xsd:element> <xsd:element name="Country" type="countryNameType" /> <xsd:element name="CountryCode" type="countryCodeType" /> <xsd:element name="PostalCode">

<xsd:simpleType> <xsd:restriction base="xsd:string"> <xsd:maxLength value="10" /> </xsd:restriction> </xsd:simpleType> </xsd:element> <xsd:element name="AddressLine"> <xsd:simpleType> <xsd:restriction base="xsd:string"> <xsd:maxLength value="160" /> </xsd:restriction> </xsd:simpleType> </xsd:element> </xsd:sequence></xsd:complexType>


VersioningVersioning

There's versioning built into XML There's versioning built into XML SchemaSchema It's so useful that you shouldn't use It's so useful that you shouldn't use

it.it. How can users tell looking at How can users tell looking at

instances?instances? Proper use is:Proper use is:

Import into new namespace and Import into new namespace and extension/restrictionextension/restriction

Replacement with new schema with Replacement with new schema with different namespacedifferent namespace

Design to extend using <any/>Design to extend using <any/>


DOM, DataSets and DOM, DataSets and SerializationSerialization XmlDocument: Hierarchical prog. XmlDocument: Hierarchical prog.

modelmodel DataSet: Relational prog. modelDataSet: Relational prog. model Serialization: OO prog. ModelSerialization: OO prog. Model

All those map from and to the All those map from and to the InfoSetInfoSet

Type system for InfoSet is SchemaType system for InfoSet is Schema Allows validation of data for correctnessAllows validation of data for correctness Allow generation of code enforcing rulesAllow generation of code enforcing rules


XSD.EXEXSD.EXE Generates classes from SchemaGenerates classes from Schema

Turns complexType into classes Turns complexType into classes Translates enumeration restrictions into Translates enumeration restrictions into

enumsenums Doesn't enforce other restriction facetsDoesn't enforce other restriction facets

Generates DataSets from SchemaGenerates DataSets from Schema Turns complexType into tablesTurns complexType into tables Turns "complexTypes in complexTypes" into Turns "complexTypes in complexTypes" into

relations between tablesrelations between tables Enforces length restriction facets Enforces length restriction facets

Also generates schema from classesAlso generates schema from classes Good start point, but that's itGood start point, but that's it

Contract


Contracts: Promises and Contracts: Promises and TrustTrust Contracts between endpoints …Contracts between endpoints …

Are promises and guarantees to deliver stuff Are promises and guarantees to deliver stuff (and how)(and how)

Are promises and guarantees to accept stuff Are promises and guarantees to accept stuff (and how)(and how)

Contract parts in Web ServicesContract parts in Web Services XSD: Data typesXSD: Data types WSDL/XSD: MessagesWSDL/XSD: Messages WSDL/BPEL4WS: Message exchange patternsWSDL/BPEL4WS: Message exchange patterns WS-Policy: Quality of service rules and req’sWS-Policy: Quality of service rules and req’s

Contract parts in Enterprise ServicesContract parts in Enterprise Services Metadata/IDL: Data types, messages, exchange Metadata/IDL: Data types, messages, exchange

patternspatterns Config/Negotiation: Quality of service rules and Config/Negotiation: Quality of service rules and

req’sreq’s


Contract StylesContract StylesRPC Contract Schema Contract

Dialog Contract

•“Method Calls”•Request/Response•Typically synchronous•Typically map 1:1

•Interface ↔ Impl.

•“Method Calls”•Request/Response•Typically synchronous•Typically map 1:1

•Interface ↔ Impl.

•“Messages”•Messaging•Synchronous & asynchronous•Multiple schema variants

one endpoint

•“Messages”•Messaging•Synchronous & asynchronous•Multiple schema variants

one endpoint

•“Messages”•Messaging Conversations•Asynchronous•Multiple schema variants

one endpoint

“Come back when you’re ready” instead of request/response

•“Messages”•Messaging Conversations•Asynchronous•Multiple schema variants

one endpoint

“Come back when you’re ready” instead of request/response


Contracts at workContracts at workRegistry(UDDI)

CustomerModule

Router

OrdersModule

EndpointAddress

ServicePolicy

ServicePolicy

XSDWSDLMEP

XSDWSDLMEP

ShippingModule

EndpointAddress

ServicePolicy

ServicePolicy

XSDWSDLMEP

XSDWSDLMEP

Taxo-nomyTaxo-nomy

ShippingModule

EndpointAddress

EndpointAddress

ServicePolicy

ServicePolicy

XSDWSDLMEP

XSDWSDLMEP

ShippingModule

EndpointAddress

EndpointAddress

ServicePolicy

ServicePolicy

XSDWSDLMEP

XSDWSDLMEP

EndpointAddress

ServicePolicy

ServicePolicy

XSDWSDLMEP

XSDWSDLMEP

XSDWSDLMEP

XSDWSDLMEP


Arrogance as design patternArrogance as design pattern

Coordinator Coordinator

Queue RM

Queue RM

Queue

If I send you a message that

complies to your contract, you better

take care of the message in a proper

way!

Errors? You better deal with them

yourself!


Cooperation as design Cooperation as design patternpattern

Coordinator Coordinator

If I send you a message that

complies to your contract, but it still fails the operation,

you can let me know and I'll try to fix

what I can.


OOA & OOD in a Services OOA & OOD in a Services WorldWorld Analyze data and behavior Analyze data and behavior

requirementsrequirements OOA is a OOA is a Good ThingGood Thing

Design data, not "objects"Design data, not "objects" OOD design that yields "business OOD design that yields "business

objects" is a questionable thing to do.objects" is a questionable thing to do. Designing data and context-appropriate Designing data and context-appropriate

behavior ("services") is a behavior ("services") is a Good ThingGood Thing There is no easy path from OOA to There is no easy path from OOA to

OOD if you want to build a scalable, OOD if you want to build a scalable, distributed system of services.distributed system of services.

Just do it.


XmlValidatingReaderXmlValidatingReader XmlValidatingReader is a good friendXmlValidatingReader is a good friend

Direct or as reader for XmlDocumentDirect or as reader for XmlDocument Validates instances using Schema Validates instances using Schema

collectionscollections Enforces all facets of type restrictionsEnforces all facets of type restrictions Enforces contractsEnforces contracts

Validation is an "edge service" like Validation is an "edge service" like authenticationauthentication Can't trust incoming data, must verify Can't trust incoming data, must verify

outbound dataoutbound data You pay performance tax, but it's worth the You pay performance tax, but it's worth the

efforteffort


From Schema to MetadataFrom Schema to Metadata

[Match(@"\p{L}[\p{L}\p{P}0-9\s]+"), MaxLength(80)]public string FirstName;

[Match(@"\p{L}[\p{L}\p{P}0-9\s]+"), MaxLength(80)]public string FirstName;

<xsd:simpleType name="nameType"><xsd:simpleType name="nameType"> <xsd:restriction base="xsd:string"> <xsd:restriction base="xsd:string"> <xsd:pattern value="\p{L}[\p{L}\p{P}0-9\s]+"/> <xsd:pattern value="\p{L}[\p{L}\p{P}0-9\s]+"/> </xsd:restriction> </xsd:restriction></xsd:simpleType></xsd:simpleType><xsd:element name="FirstName"><xsd:element name="FirstName"> <xsd:simpleType> <xsd:simpleType> <xsd:restriction base="nameType"> <xsd:restriction base="nameType"> <maxLength value="80"/> <maxLength value="80"/> </xsd:restriction> </xsd:restriction> </xsd:simpleType> </xsd:simpleType></xsd:element> </xsd:element>


"It'll absolutely get there""It'll absolutely get there"

MSMQ 3.0MSMQ 3.0 Reliable messaging infrastructureReliable messaging infrastructure

TCP, SOAP/HTTP TransportsTCP, SOAP/HTTP Transports Exposed through Exposed through System.MessagingSystem.Messaging

and Enterprise Services "Queued and Enterprise Services "Queued Components"Components"

Guarantees delivery, supports Guarantees delivery, supports transactionstransactions

Using Queues gives you …Using Queues gives you … Peace of mind. You know that it'll get Peace of mind. You know that it'll get

there.there. Fairness. Every machine runs at it's own Fairness. Every machine runs at it's own

pace.pace. Scalability. No need to be synchronous.Scalability. No need to be synchronous.


DataData

PresentationPresentation

Implementing Services Implementing Services

SQLSQL MQMQ WSWS XSDXSD RPCRPC

PublicInterface

Services &ResourceAccess

InternalImplemen-

tation

XMLXML HTMLHTML GUIGUI EDIEDI COMCOM

DataData

PresentationPresentation

LooserLooserCouplingCoupling

TighterTighterCouplingCoupling


Services as a Services as a recursive recursive patternpattern

DataServices

BusinessServices

Presentation

Services


On the edge:On the edge:ASMX & Enterprise ServicesASMX & Enterprise Services IIS 6.0 and ASP.NET Web Services IIS 6.0 and ASP.NET Web Services

(ASMX)(ASMX) Application infrastructure for Application infrastructure for

implementing public interfaces for "far implementing public interfaces for "far tiers"tiers"

Highly optimized for HTTP traffic, Highly optimized for HTTP traffic, appropriate process model for Web appropriate process model for Web ServicesServices

Windows Enterprise ServicesWindows Enterprise Services Application infrastructure for Application infrastructure for

implementing services for "near tiers"implementing services for "near tiers" Most appropriate process model for Most appropriate process model for

implementing robust, available implementing robust, available backendsbackends

ASMX and Enterprise Services are a ASMX and Enterprise Services are a teamteam


ASMX & ES in concertASMX & ES in concertBrowserBrowserBrowserBrowser

ASP.NETASP.NETASP.NETASP.NET

Enterprise Services ApplicationEnterprise Services ApplicationEnterprise Services ApplicationEnterprise Services Application

SQL ServerSQL ServerSQL ServerSQL Server

Anony-Anony-mousmous

NetwoNetwork-rk-ServicServicee

ServiceServiceAccounAccountt

RFC2617 RFC2617 AuthAuth

RFC2617 RFC2617 AuthAuth

Windows Windows SSPISSPI




ADO.NETADO.NET

ComponentComponentComponentComponentComponentComponentComponentComponent

ComponentComponentComponentComponent ComponentComponentComponentComponentComponentComponentComponentComponent

ComponentComponentComponentComponent

ServicedServicedComponentComponent




Identity:Identity:Domain/Domain/AccountAccount

Identity:Identity:Domain/Domain/AccountAccount

ACLACL PrincipalPermissionPrincipalPermission

RollRoll

RollRollGroupGroup

““Integrated Integrated Security=SSPI”Security=SSPI”

““Integrated Integrated Security=SSPI”Security=SSPI”


SummarySummary

For scalable systems to work, For scalable systems to work, contract is neededcontract is needed

Data type definitions need to be Data type definitions need to be stronger in distributed systemsstronger in distributed systems

Strong types enable better Strong types enable better modularizationmodularization

The .NET Framework enables you to The .NET Framework enables you to make that work for you ….make that work for you ….

say what you will say and listen to what you will listen to …

Documents

microsoft corporation

unauthorized reproduction

rights reserved ways

rights reserved information

clemens vasters gilleshtte

rights reserveda

new address clemens

clemens vasters str