Exchange Server 2013 Architecture

Peter O’DowdDatacom Systems (Wellington) Ltd

OFC306

AgendaFundamentalsClient ConnectivityNamespace Planning & PrinciplesMicrosoft’s Preferred Deployment Architecture

Exchange Server 2013 Fundamentals

Exchange 2013 Server Role Architecture

AD

Web

browserOutlook

(remote user)

Mobile

phone

Line of business applicationOutlook (local

user)

External

SMTPservers

Exchange Online

Protection

Enterprise Network

Phone system (PBX

or VOIP)

Edge TransportRouting and AS

2 building blocks:• Client Access

Array• Database

Availability Group

Edge Transport Role

Loosely coupled• Functionality• Versioning• User partitioning• Geo affinity

Layer

4 o

r Layer

7 L

B CAS

CAS

CAS

CAS

CAS

CAS Array

MBX

MBX

MBX

MBX

MBX

DAG

E2010Banned

Server1 (Vn) Server2 (Vn+1)

Protocols, Server Agents

EWS

RPC CA

Transport

Assistants

MRS MRSProx

y

Transport

Assistants

EWS

RPC CA

MRS MRSProx

y

Business Logic

XSOMail ItemOther API

CTS

XSOMail ItemOther API

CTS

StorageStore

Content indexFile

systemESE

StoreContent indexFile

systemESE

SMTP

MRS proxy protocol

EWS protocol

Custom WS

Every Server is an Island

CAS

The protocol stack used to access a mailbox is always on the Mailbox server that hosts the active database copy

Each CAS determines the right endpoint for the traffic, so all sessions – regardless of where they start – end up in the same place

Rendering for clients like OWA, and Transport transcoding, occurs on the Mailbox server

User

DAG1

MBX-A MBX-B

The key to enlightenment…

What is the Client Access server role?Domain-joined machine in the corporate forestThin, stateless protocol proxy serverComprised of three components:Client Access Front End aka CAFÉ (HTTP, IMAP, and POP protocol proxy)Front End Transport service (SMTP protocol stack and proxy)UM Call Router

Provides unified namespace and authenticationContains logic to route protocol requests to the appropriate destination endpointCapable of supporting legacy servers with redirect or proxy logic

What is the Mailbox server role?Server that hosts all of the components that process, render and store Exchange data Connectivity for mailbox access is via CAS to Mailbox*Exchange 2013 evolves the DAGDAG includes new repair and recovery featuresDAG includes networking enhancementsDAG leverages Windows Server 2012 R2 features

Exchange IOPS Trend

DB IOPS/Mailbox

Exchange 2003 Exchange 2007 Exchange 2010 Exchange 2013

1

0.8

0.6

0.4

0.2

0

>93%reduction!

8KB Page Size

STM Removed

Store Quaranti

ne

10GB Mailboxe

s

Elimination of Partial B+

Merges

Cache Warming on

Passive

Log Roll

32KB Page Size

Message properties stored as

blobs

Per-Database Process

Fast Failover

TBA Store Maintenance

Lost Write Detection

Cache Maintained after

Recovery

100GB Mailboxes

Database Compressi

on

1 Million Items / Folder

Managed Store

Lost Log Resilience

Page Dependency

Removal

Online Page Zeroing

Per-Mailbox Tables

100MB Checkpoint Depth on

Passive Copies

Lagged Copy Enhancements

OS Upgrade Support

128MB Extent Size

Optimized for 7.2K RPM Disks

100 Databases / Server

Hung IO and Bluescreen

Support

Gap Coalescing

Smooth IO Writes

Incremental Resync

Improved Async Read Capability

Support for 231 log

generations

1GB Mailboxes

64-bit architecture

Standby Continuous Replication

20,000 Items / Folder

ESE & Store ImprovementsLog checksum recovery from

single-bit errors

50 Databases /

server

Database Cache Compression

Improved IO

Coalescing

Continuous Replication

Parallel Mounting

Database Space Allocation Hints

Multiple Databases / JBOD Disk

Lazy View Update

Changes

Lazy Indexes

Online Database Checksum

1MB Log Files

100MB Checkpoint Depth on

Active Copies

Version Store Improvements

1:1 Read:Write Ratio

Physical Contiguity Store Schema Changes

Tuned Maintenanc

e Writes

Single Page Restore

100,000 Items / Folder

JBOD Support

Database Cache Priority

B+ Tree Defrag

BDM for Active and Passives

Pre-read Keys

2010No more

deferred content conversion

2007

AutoReseed

2013

What is the Edge Transport server role?Handles all Internet-facing mail flowDesigned to run in a perimeter networkDoes not have to be joined to a domainUses EdgeSync process to provide one-way replication of recipient and configuration informationCommunicates with FET when roles are co-locatedIncludes anti-spam, but no antivirusPowerShell management only

External

SMTPservers

EOP

Edge Transport

Servers

MailboxServers

AD

EdgeSync

TCP 50636

Mail flow

Client AccessServers

MDB

Transport

MBX Transport

Front-End Transport

2 Recipients

DAG

CAS

MBX

Transport Architecture

MDB

Transport

MBX Transport

Front-End Transport

CAS

MBX

Transport componentsTransport ships as part of 3 major componentsFront End Transport – Stateless SMTP service on client access roleTransport – Stateful SMTP service on mailbox roleMailbox Transport – Stateless SMTP service on mailbox role

Transport responsibilitiesReceive and deliver all inbound mail to the organization Submit and deliver all outbound mail from the organizationPerform all message processing within the pipelineSupport extensibility within pipelineKeep messages redundant until successfully delivered

Front End Transport

AD

Web browser

Outlook (remote

user)

Mobile phone

Outlook (local user)

ExternalSMTP

servers

Exchange Online

Protection

Enterprise Network

Layer

4LB

CAS Array

CAS

CAS

CAS

CAS

DAG2

MBX

MBX

MBX

…

DAG3

MBX

MBX

MBX

…

DAG1

MBX

MBX

MBX

…

Front End Transport

Client Access Server (CAS)• Evolution of E2010

CAS Array• Now includes SMTP Frontend Transport• Primary function is to

get the client to the right MBX server

Mailbox Server• Now includes all core

messaging protocols• Now includes

Transport and Mailbox Transport (Delivery & Submission)

Edge Transport Server• Perimeter network

SMTP gateway

Front End TransportHandles inbound and outbound external SMTP traffic (does not replace Edge Transport Server)Listens on TCP25 and TCP587 and TCP717Handles authenticated client submissions Functions as a layer 7 proxy and has full access to protocol conversation (inbound)Does not queue or bifurcate mail locallyAll outbound traffic to next hop appears to come from the CAS2013

Frontend Transport

SMTP Receive Protocol

Agents

SMTP from MBX 2013

Authenticated

SMTP

SMTP Send

SMTP to MBX 2013

External SMTP

Mailbox Selector

:25

:717

MSExchangeFrontendTransport.exe

:587

AnonymousSMTP

Front End Transport FeaturesNetwork protection – centralized, load balanced egress/ingress point for the organizationMailbox locator – avoids unnecessary hops by determining the best Mailbox to deliver the messageProvides unified namespace, for authenticated and anonymous mailflow scenarios

Transport*

AD

Web browser

Outlook (remote

user)

Mobile phone

Outlook (local user)

ExternalSMTP

servers

Exchange Online

Protection

Enterprise Network

Layer

4LB

CAS Array

CAS

CAS

CAS

CAS

CAS

DAG2

MBX

MBX

…

DAG3

MBX

MBX

…

DAG1

MBX

MBX

…

Transport

Transport

Transport

*previously known as Hub Transport

Client Access Server• Now includes SMTP

Frontend Transport

Mailbox Server• Now includes all core

messaging protocols• Now includes Transport and Mailbox Transport (Delivery & Submission)

Edge Transport Server• Perimeter network

SMTP gateway

Processes all SMTP mail flow for the organization

Will queue and route messages in and out of the organization

Performs content inspection

Supports extensibility in SMTP and categorizer

Listens on TCP 25 (or TCP2525 when co-located with CAS)

Transport*

Transport

SMTP to MBX-Transport

Delivery

SMTP from MBX-Transport Submission

SMTP from CAS

SMTP to CAS, MBX, HUB

Delivery Agents

*other protocols

Delivery Queue

Delivery Queue

Pickup/Replay

Categorizer

Routing Agents

SMTP Send

SMTP Receive

Protocol Agents

:25 or :2525

:25

Edgetransport.exe

Mail.que

Submission Queue

*previously known as Hub Transport

Transport PipelineAll incoming mail is stored in the mail.que databaseAll mail passes through the various stages of the categorizer There is exactly one submission queue but multiple delivery queues (one per destination)Agents subscribe to various events along the pipeline – Transport rules agent; Journaling agent; Malware agent; 3rd party agents

Categorizer

ResolveRecipients

SMTP Send

SMTP Receive

Protocol Agents

:25 or :2525

:25

Mail.que

Submission Queue

Find Route for Recipient

Content Conversion

& Bifurcation

On Submitted On Resolved

On Routed On Categorized

External Delivery Queue

Internal Delivery Queue

Mailbox Delivery Queue

Transport FeaturesPerforms all routing decisions for internal and external messagesProvides an extensibility platform for third-party agents to operate within the pipelineAllows messages to be routed in or out through connectors for special handlingProtects messages by making messages highly available on ‘shadow’ servers

Mailbox Transport

AD

Web browser

Outlook (remote

user)

Mobile phone

Outlook (local user)

ExternalSMTP

servers

Exchange Online

Protection

Enterprise Network

Layer

4LB

CAS Array

CAS

CAS

CAS

CAS

CAS

DAG2

MBX

MBX

…

DAG3

MBX

MBX

…

DAG1

MBX

MBX

…

Mailbox Transport

MailboxTranspor

t

Mailbox Transport

Client Access Server• Now includes SMTP

Frontend Transport• Primary function is to

get the client to the right MBX server

Mailbox Server• Now includes all core

messaging protocols• Now includes

Transport and Mailbox Transport (Delivery & Submission)

Edge Transport Server• Perimeter network

SMTP gateway

Mailbox TransportHandles mail submission and delivery from/to Store using two separate processesPerforms MIME to MAPI conversion (and vice versa)Combines Mailbox Assistant and Store Driver functionalityUses local MAPI/RPC for delivery to and submission from StoreDoes not have persistent storageDoes not support any extensibility

SMTP from Transport

Mailbox Transport

SMTP SendSMTP

Receive

Submission

Mailbox Assistant

s

MAPI MAPI

Store

SMTP to Transport

Submit Agents

:475

MSExchangeDelivery.exe MSExchangeSubmission.exe

SMTP Send

Deliver Agents

Delivery

SMTP to Transport

Mailbox Transport FeaturesBrings together all transport scenarios that access mailbox store under one componentEliminates the three-party mail submission hand-shakeHelps realize the “every server is an island” vision by ensuring MAPI is not used across the serverSimplifies handling of mailbox database *overs

Integrated monitoring and recovery infrastructure that detects and recovers from issues as they are discovered

Managed Availability

—OWA send—OWA failure—OWA failure detected —OWA recycle AppPool —OWA recycle complete —OWA verified as healthy —OWA send—OWA failure—OWA failure detected —OWA recycle AppPool —OWA recycle AppPool failed—Failover server’s databases—OWA service restarts—OWA verified as healthy —Server becomes “good” failover target (again)

LB CAS1

CAS2

DAG

MBX1

DB1 DB2

MBX2

OWA

DB1 DB2

MBX3

OWA DB1 DB2

OWA

OWA

OWA

OWA

DB1

DB1

Managed AvailabilityStuff breaks but the User Experience does not

Client Connectivity

CAS2013

MBX2013

RPC CA

IIS

RPSOWA, EAS, EWS, ECP,

OAB

POP IMAP

Transport

UM

RpcProxy

MDB MailQ

HTTP Proxy

IIS POPIMAP

SMTP

UM

TelephonyPOP | IMAP SMTP

OWA EAS EACOutlook PowerShell

Load Balancer

HTTPPOPIMAP SMTP

Redirect

SIP +

RTP

Client Protocol Architecture

Outlook Connectivity – RPC over HTTPExchange 2013 does not support RPC/TCPWhy not?RPC session is always on the MBX2013 server hosting the active database copyDoes not require a “RPC CAS array namespace” for the DAG

What changes?RPC end point for Outlook client is now a GUID (and SMTP suffix)Support for internal and external Outlook Anywhere namespacesNo longer have to worry about “The Exchange administrator has made a change that requires you to quit and restart Outlook” during mailbox moves or *over events

Outlook RPC over HTTP Connections

CAS2013

MBX2013

RPC CA

IIS

HTTP Proxy

IIS

LB

HTTP

MDB

HTTPSRPC_DATA_IN

HTTPSRPC_DATA_OUT

HTTPSRPC_DATA_IN

HTTPSRPC_DATA_OUT

HTTPSRPC_DATA_IN

HTTPSRPC_DATA_OUT

RpcProxy

HTTP

RPC

MAPI

Outlook

Outlook Connectivity – MAPI over HTTPWhat is it?New connectivity mechanismNo longer uses intermediary RPC components (on client or server)ROPs are still used, just sent to Exchange directly over HTTP

Advertised via AutodiscoverClient advertises support and server returns configuration settings

Disabled by defaultRequiresExchange 2013 SP1 (or later)Exchange 2013 SP1 mailboxOutlook 2013 SP1 (or later)Client restart

Why?Provides more reliable connection

80% connect in 5s or less82% resume from hibernate sync times of 30s or less73% take 30s or less to start sync from bootStandard HTTP pattern instead of two long-lived HTTP connections

Removes RPC stack dependencyBetter diagnostics

Header information

Common authentication scheme across protocol stack

Outlook MAPI over HTTP Connections

CAS2013

MBX2013

HTTP ProxyIIS

LB

HTTP

MDB

HTTPSReq/Response

HTTPSReq/Response

HTTPSReq/Response

HTTPSHanging Notification

HTTP

MAPI

Outlook

IIS

MAPI HTTP Handler

MBX2013

CAS2013

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

HTTP

MBX2007

CAS2007

Load Balancer

IIS

DB

Middle Tier Layer

OWA Legacy Redirect Request

MBX2007

DB

Cross-Site OWA Proxy Request

CAS2007

IIS

Middle Tier Layer

CAS2013 Client Protocol Connectivity FlowExchange 2007 Coexistence

Outlook Anywhere Proxy RequestActiveSync Proxy Request

Site

B

ou

nd

ary

MBX2013

CAS2013

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

HTTP

Legacy Proxy Request

MBX2010

DB

Cross-Site Legacy Proxy Request

CAS2010

IIS

Middle Tier Layer

CAS2013 Client Protocol Connectivity FlowExchange 2010 Coexistence

Load Balancer

Cross-Site OWA Redirect Request

Site

B

ou

nd

ary

MBX2010

CAS2010

Load Balancer

DB

Middle Tier Layer

IIS

MBX

CAS

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

Local Proxy Request

HTTP

HTTP

Site

B

ou

nd

ary

MBX

CAS

Load Balancer

HTTP Proxy

IIS

DB

Protocol Head

HTTP

OWA Cross-Site Redirect Request

HTTP

MBX

DB

Protocol Head

HTTP

Cross-Site Proxy Request

HTTP

Site

B

ou

nd

ary

CAS

HTTP Proxy

IIS

CAS2013 Client Protocol Connectivity FlowEnd State

Namespace Planning & Principles

Namespace PlanningNo need for namespaces required by Exchange 2010Can still deploy regional namespaces to control trafficCan still have specific namespaces for protocols

Two namespace modelsBound ModelUnbound Model

Leverage split-DNS to minimize namespaces and control connectivityDeploy separate namespaces for internal and external Outlook Anywhere host names

Sue (somewhere in

NA) DNS Resolution

DAG1

mail VIP mail2 VIP

mail.contoso.com

mail2.contoso.com

DAG2

Jane(somewhere in

NA)DNS Resolution

Passive

Active

Active

Passive

Bound Model

Round-Robin between # of VIPs

Sue (somewhere in

NA) DNS Resolution

DAG

VIP #1 VIP #2

mail.contoso.com

Unbound Model

Load BalancingExchange 2013 no longer requires session affinity to be maintained on the load balancerFor each protocol session, CAS now maintains a 1:1 relationship with the Mailbox server hosting the user’s data

Load balancer configuration and health probes will factor into namespace designRemember to configure health probes to monitor healthcheck.htm, otherwise LB and MA will be out of sync

CAS

OWA

ECP

EWS

EAS

OAB

MAPI

RPC

AutoD

Single Namespace / Layer 4

autodiscover.contoso.com

User

Layer

4LB

mail.contoso.com

health check

CAS

OWA

ECP

EWS

EAS

OAB

MAPI

RPC

AutoD

Single Namespace / Layer 7

autodiscover.contoso.com

User

Layer

7LB

mail.contoso.com

health check

Health check executes against each virtual directory

mapi.contoso.com

User

Layer

4LB

mail.contoso.com

ecp.contoso.com

ews.contoso.com

eas.contoso.com

oab.contoso.com

oa.contoso.com

CAS

OWA

ECP

EWS

EAS

OAB

MAPI

RPC

AutoD

autodiscover.contoso.com

Multiple Namespaces / Layer 4

Generalist IT admin

Those with increased network flexibility

Exchange Load Balancing Options Those who want

to maximize server

availability

+ Simple, fast, no affinity LB+ Single, unified namespace+ Minimal networking skillset

- Per Server Availability

+ Per protocol availability+ Single, unified namespace

- SSL termination @ LB- Requires increase networking skillset

+ Simple, fast, no affinity LB+ Per protocol availability

- One namespace per app protocol- One VIP per protocol

SimplicityFunctionality

Wh

o’s

it

for?

Trad

e-O

ffs

The Preferred Architecture

Preferred ArchitectureNamespace Design

For a site resilient datacenter pair, a single namespace / protocol is deployed across both datacenters

autodiscover.contoso.comHTTP: mail.contoso.comIMAP: imap.contoso.comSMTP: smtp.contoso.com

Load balancers are configured without session affinity, one VIP / datacenter

Round-robin, geo-DNS, or other solutions are used to distribute traffic equally across both datacenters

mail VIP

mail VIP

Preferred ArchitectureDAG Design

Each datacenter should be its own Active Directory siteDeploy unbound DAG model spanning each DAG across two datacentersDistribute active copies across all servers in the DAGDeploy 4 copies, 2 copies in each datacenterOne copy will be a lagged copy (7 days) with automatic play down enabled Native Data Protection is usedSingle network is used for MAPI and replication trafficThird datacenter used for Witness server, if possibleIncrease DAG size density before creating new DAGs

DAG

mail VIP

mail VIP

Witness Server

Preferred ArchitectureServer Design

Multi-role servers deployed on commodity hardwareJBOD storage utilizing large capacity 7.2K SAS disksMultiple databases / volumeAutoReseed with hot spare

DAG

mail VIP

Larger Mailboxes are BetterLarge Mailbox Size 100 GB+Aggregate Mailbox = Primary Mailbox + Archive Mailbox + Recoverable Items1-2 years of mail (minimum)1 million items / folder

Increased knowledge worker productivityEliminate or reduce PST relianceEliminate or reduce third-party archive solutions

Outlook 2013 can control OST size

Time ItemsMailbox

Size

1 Day 150 11 MB

1 Month 3300 242 MB

1 Year 39000 2.8 GB

2 Years 78000 5.6 GB

4 Years 156000 11.2 GB

Selina(somewhere in

NA)DNS Resolution

DAG

na VIP na VIP

Batman(somewhere in Europe)

DNS Resolution

DAG

eur VIP

eur VIP

Preferred Architecture

na.contoso.comeur.contoso.com

Summary

SummaryNew building block architecture provides flexibility in load balancing, namespace planning and high availabilityTake advantage of large, low-cost mailboxes by utilizing large capacity 7.2K RPM disksSimpler is better!

Questions?

Resources

TechNet & MSDN FlashSubscribe to our fortnightly newsletter

http://aka.ms/technetnz http://aka.ms/msdnnz

TechNet Virtual LabsFree Virtual Hands-on Labs

http://aka.ms/ch9nz

Microsoft Virtual AcademyFree Online Learning

http://aka.ms/mva http://aka.ms/technetlabs

Sessions on Demand

Complete your session evaluation now and win!

© 2014 Microsoft Corporation. All rights reserved.Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.