Exchange Server 2013 Architecture

Ross Smith IV Principal Program Manager, Exchange Server

Disclaimer
2012 Microsoft Corporation. All rights reserved. Microsoft, Office 365, and other product and service names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Agenda

Discuss the new server architecture in Exchange 2013

Discuss the Client Access server role

Discuss the Mailbox server role

E2007/E2010 Server Role Architecture

Enterprise Network 5 major roles Tightly coupled
Functionality

Forefront Online Protection for Exchange

Phone system (PBX or VOIP)

Edge Transport Routing and AV/AS

Hub Transport Routing & policy

Geo affinity
Versioning User partitioning

External SMTP servers Mobile phone Web browser Outlook (remote user)

Mailbox Storage of mailbox items

Unified Messaging Voice mail and voice access

Layer 7 LB

Client Access Client connectivity Web services

Outlook (local user)

Line of business application AD

Whats wrong with the existing model?

Exchange deployments are overly complicated Doing Exchange load balancing right is hard and often requires expensive solutions
Requiring session affinity at the load balancer significantly impacts scalability Many hardware load balancing solutions are expensive and thus, are a luxury many of our customers cant afford or dont have the expertise to deploy

Customers deploy based on dedicated server roles

This means that in many cases, hardware is deployed that is unutilized (e.g., DIMM slots and disk slots, etc.) or under-utilized

Too many namespaces are required (especially in site resilient designs)

Exchange: The Evolution

LB L7 LB CAS Ex HT Ex

MBX Ex

SAN

MBX Ex

Exchange: The Evolution

2000/2003
Role differentiation through manual configuration Hardware solutions for reliability ($$$$)

2007
Separate roles for ease of deployment and mgmt. segmentation Support cheaper storage

2010
Separate HA solutions for each role Introduced the DAG Rich management experience using RBAC Leaves resources on the ground in each role

2013
Simplify for scale, balanced utilization, isolation Integrate HA for all roles Simplify network architecture

LB

Ex
SAN

Ex Ex

CAS MBX

HT MBX

L7 LB

Ex

The new server role architecture

Exchange 2013 Architecture Theme

Use Building Blocks to facilitate deployments at all scales from self-hosted small organizations to Office 365 Server role evolution Network layer improvements Versioning and inter-op principles

Benefits
Hardware efficiency Deployment simplicity Low friction cross-version inter-op Failure isolation

Exchange 2013 Server Role Architecture

2 Building Blocks
Client Access Array Evolution of E2010 CAS Array SMTP Front-End Database Availability Group Evolution of E2010 DAG Includes core server protocols

Enterprise Network
Forefront Online Protection for Exchange
Edge Transport Routing and AV/AS

CAS Array
CAS CAS CAS CAS CAS

DAG
MBX MBX MBX MBX MBX

AD

External SMTP servers Mobile phone Web browser Outlook (remote user)

Loosely coupled

Layer 4LB

Functionality Versioning User partitioning Geo affinity

Outlook (local user)

Line of business Phone system application (PBX or VOIP)

Every Server is an Island

EWS protocol MRS proxy protocol SMTP

Protocols, Server Agents

EWS

MRS MRSProxy RPC CA Assistants

Transport

Custom WS

Transport

MRS MRSProxy RPC CA

EWS

Assistants

Business Logic

XSO

Mail Item

Banned E2010

XSO

Mail Item

CTS

Other API

CTS

Other API

Storage

Store

Content index

Store ESE

Content index File system

ESE

File system

Server1 (Vn)

Server2 (Vn+1)

Functional Layering
E2010 Architecture
Hardware LB

E2013 Architecture
CAS2013

L7LB

Load Balancer AuthN, Proxy, Re-direct

CAS, HT, UM

AuthN, Proxy, Re-direct Protocols, API, Biz-logic

MBX2013

Protocols, API, Biz-logic

MBX

Assistants, Store, CI Store, CI

The key to enlightenment

User For a given mailboxs connectivity, the protocol being used is always served by the protocol instance that is local to the active database copy This means that the rendering for clients like OWA occurs on the Mailbox server

CAS

This means that Transport transcoding is occurring on the Mailbox server etc

DAG1

MBX-A

MBX-B

Client Access Protocol Proxying

What is the CAS2013 role?

CAS2013 is comprised of three components:
Client protocols (HTTP, IMAP, POP) SMTP UM Call Router

Thin, stateless (protocol session) servers organized in a load balanced configuration

Provides a unified namespace and authentication for clients Where the logic lives to route a specific protocol request to the correct destination end point Is a domain-joined machine in the corporate forest
Capable of supporting legacy servers with redirect or proxy logic Session affinity NOT required at the load balancer

CAS2013 Client Protocol Architecture

OWA Outlook EAS EAC PowerShell IMAP SMTP Telephony
SIP + RTP

Load Balancer CAS2013

HTTP IIS IIS HTTP Proxy POP IMAP POP IMAP OWA, EAS, EWS, ECP, OAB RPC CA POP IMAP SMTP

Redirect

UM

SMTP Transport UM

MBX2013

RpcProxy RPS MDB MailQ

Outlook Connectivity
What are the benefits?

RPC/HTTP and the death of RPC/TCP

RPCProxy.dll

Does not require a RPC CAS array namespace for the DAG No longer have to worry about The Exchange administrator has made a change that requires you to quit and restart Outlook during mailbox moves or *over events Extremely reliable and stable connectivity model the RPC session is always on the MBX2013 server hosting the active database copy

What changes?
RPC end point for Outlook client is now a GUID (and SMTP suffix) Support for internal and external Outlook Anywhere namespaces

Third-Party MAPI Products

Third-party MAPI products will need to use RPC/HTTP to connect to CAS2013 Exchange 2013 will be the last release that supports a MAPI/CDO download
Third-parties must move to EWS in the future

The MAPI/CDO download will be updated to include support for RPC/HTTP connectivity
Will require third-party application configuration; either by programmatically editing a dynamic MAPI profile or setting registry keys Legacy environments can continue to use RPC/TCP

CAS2013 Client Protocol Connectivity Flow

HTTP

HTTP

Load Balancer
Site Boundary
CAS
IIS HTTP Proxy
HTTP

Load Balancer
Site Boundary
CAS
IIS HTTP Proxy
HTTP HTTP

MBX
Protocol Head DB Local Proxy Request

MBX
Protocol Head DB

MBX
Protocol Head DB

OWA Cross-Site Redirect Request

Cross-Site Proxy Request

Exchange Load Balancing Options

Whos it for?
Generalist IT admin Those with increased network flexibility Functionality Simplicity
+ Simple, fast, no affinity LB + Single, unified namespace + Minimal networking skillset - Per Server Availability + Simple, fast, no affinity LB + Per protocol availability - One namespace per protocol + Per protocol availability + Single, unified namespace - SSL termination @ LB - Requires increase networking skillset

Those who want to maximize server availability

Trade-Offs

A Single Common Namespace Example

Geographical DNS Solution
(somewhere in NA)

Sue

DNS Resolution Round-Robin between # of VIPs

mail.contoso.com

Sue

DNS Resolution via Geo-DNS Round-Robin between # of VIPs

(traveling in APAC)

VIP #1

VIP #2

VIP #3

VIP #4

DAG

DAG

CAS2013 Client Protocol Benefits

Simplifies the network layer
No longer requires session affinity at the load balancer Just get the traffic to CAS2013 and let it handle the affinity CAS2013 can be farther away from MBX2013 and still offer good client performance (because it is a 1:1 proxy) Removes the need for RPC Client Access arrays CAS2013 provides more deployment flexibility; for example, consolidate to fewer sites Can deploy a single world-wide namespace Designed to proxy to multiple Mailbox server versions, up and down level DAGs can be replaced with E2013 at any desired pace

Deployment flexibility

Simplifies upgrade and inter-op

The Mailbox Server Role

The Mailbox Server Role

A server that hosts all the components that process, render and store the data Clients do not connect directly to MBX2013 servers; connectivity is through CAS2013 Evolution of E2010 DAG
Collection of servers that form a HA unit Databases are replicated between servers in a given DAG Servers can be in different locations, for site resiliency Maximum of 16 Mailbox servers 50 database copies / server

MBX1

MBX2

MBX16

The New Store Process

Store is effectively made up of three processes
Replication service Store service process/controller Store worker process

Replication service initiates failovers and is responsible for issuing mount/dismount operations Store service process/controller manages the store worker processes Each database has its own Store worker process

Exchange IOPS Trend

1 0.8 0.6 0.4 0.2

DB IOPS/Mailbox

+99%

reduction!

0
Exchange 2003 Exchange 2007 Exchange 2010 Exchange 2013

Large Mailboxes for the win!

Large Mailbox Size 100GB+
Aggregate Mailbox = Primary Mailbox + Archive Mailbox + Recoverable Items 1-2 years of mail (minimum) 1 Day 1 Month 1 Year 2 Years 150 3300 39000 78000 11 MB 242 MB 2.8 GB 5.6 GB

Increased knowledge worker productivity

Eliminate or reduce PST reliance

4 Years

156000

11.2 GB

Eliminate or reduce third-party archive solutions

Outlook 2013 allows you to control OST size!
Gives more options around mailbox deployments

New Exchange Search Infrastructure

Leverages Search Foundation
Common, actively developed search platform used across Office server products Does consume more memory (1/6 available memory) to improve query performance

Provides
Significantly improved query performance compared to E2010 Significantly improved indexing performance compared to E2010

Feature parity with E2010 search Leverages the same cmdlets like Get-MailboxDatabaseCopyStatus

Exchange Indexing
MBX2013
Transport Transport

Reduced Processing of Body and Attachments

MBX2013
Content Transformation Service Mailbox Store Local Delivery ExSearch CTS Index Node Mailbox Passive

Log

DB

Reliable Event

Read Content

Log

DB

Idx

Idx

Public Folders
Dawn of a New Age
Architectural bet Details
Private logon Public Logon Public logon

Public folders are based on the mailbox architecture

CAS 2013
Hierarchy Mailbox Content Mailbox

MBX2013

MBX2013

MBX2013

Hierarchy is stored in PF mailboxes (one writeable) Content can be broken up and placed in multiple mailboxes The hierarchy folder points to the target content mailbox Uses same HA mechanism as mailboxes No separate replication mechanism Single-master model Similar administrative features to current PFs (setting quota, expiry, etc.) No end-user changes (looks just like todays PFs)

Not all public folder usage scenarios are best served by public folders

Transport Architecture

Transport Components on Client Access

Front-End Transport Service
Handles all inbound and outbound external SMTP traffic for the organization, as well as client endpoint for SMTP traffic
Does not replace the Edge Transport Server Role

Functions as a layer 7 proxy and has full access to protocol conversation Will not queue mail locally and will be completely stateless All outbound traffic appears to come from the CAS2013 Listens on TCP25 and TCP587 (two receive connectors)

Processing Inbound Messages

Externa l Server CAS MBX

1. New SMTP Connection 2. CAS performs envelope filtering 3. CAS determines route to best MBX server 4. Message delivery begins 1. If successful, CAS returns 250 OK acknowledgement to external server 2. If unsuccessful, CAS returns 421 response

Benefits of SMTP Front-End Service

The SMTP Front-End Service provides:
Protocol level filtering performs connection, recipient, sender and protocol filtering Network protection centralized, load balanced egress/ingress point for the organization Mailbox locator avoids unnecessary hops by determining the best MBX2013 to deliver the message Load balanced solution for client/application SMTP submissions

Scales based on number of connections just add more servers

Transport Components on Mailbox

Transport in MBX2013 has been broken down into three components
Transport Service - Stateful and handles SMTP mail flow for the organization and performs content inspection (Was previously referred to as Hub Transport) Mailbox Transport Delivery Service - Receives mail from the Transport service and deliveries to the Mailbox Database Mailbox Transport Submission Service - Takes mail from the Mailbox Databases and submits to the Transport service

Mailbox Transport is stateless and does not have a persistent storage mechanism Mailbox Transport performs content conversion

Transport Components on Mailbox

Responsibilities
Receives all inbound mail to the organization (Proxied through CAS or direct) Submits all outbound mail from the organization (Proxied through CAS or direct) Handles all internal message processing such as Transport Rules, Content Filtering, and Anti-Virus Performs mail flow routing Queue messages Supports SMTP extensibility

Routing Optimizations
Next hop selection is broken down into distinct delivery groups:
Routable DAG Mailbox Delivery Group Connector Source Servers AD Site (Hub Sites; Edge Subscriptions) Server list (DG expansion servers)

Queuing is per delivery group, connector, or mailbox Once message is received at final destination, Transport will deliver the message via SMTP to Mailbox Transport on the server hosting the active database copy Send/Delivery-Agent Connectors can have source servers from multiple DAGs or AD Sites, and can be proxied through CAS

Mail Delivery
CAS / MBX DAG MBX-1
SMTP

MBX-2

Transport
Mailbox Transport
MAPI
SMTP

SMTP

Transport
Mailbox Transport DB1 DB2

DB1

DB2

Service Availability Improvements

Service Availability Improvements

Best copy selection now includes health of services when selecting best copy Failover time reductions Lagged copy enhancements Simpler site resilience strategy Managed Availability

Managed Availability
Monitoring and recovery infrastructure is integrated with Exchanges high availability solution Detects and recovers from problems as they occur and are discovered Is user focused if you cant measure it, you cannot monitor it

Managed Availability
LB CAS-1 DAG
MBX-1

Managed Availability + Retriesstuff breaks and the Experience does not

OWA send OWA failure OWA failure detected OWA restart service OWA restart complete OWA verified as healthy OWA send OWA failure OWA failure detected OWA restart service OWA restart service failed Failover servers databases OWA service restarts OWA verified as healthy Server becomes good failover target (again)

OWA

DB1

DB2

MBX-2

CAS-2

OWA

DB1

DB2

MBX-3 OWA

DB1

DB2

Transport High Availability Improvements

Every message is redundantly persisted before its receipt is acknowledged to the sender Delivered messages are kept redundant in transport similar to active messages Every DAG represents a transport HA boundary and owns its HA implementation
If you have a stretched DAG, you also have transport site resilience

Resubmits due to transport DB loss or MDB *over are fully automatic and do not require any manual involvement

SMTP 250 OK
R1, R2, R3

CAS2013 or MBX2013

Transport HA
250 OK

1. Maintain a copy of the message in the queue database but dont acknowledge the DATA verb 2. Generate a shadow copy on another MBX2013 server in the DAG (remote site preferred) 3. Wait for acknowledgement from the shadow server 4. Send acknowledgement to SMTP client 5. Delete message from queue after SafetyNet threshold has expired

Transport
R1 R1,R3 R2, R3 R2

Transport
Mail.que Mail.que

Transport
Mail.que

Transport
Mail.que

MBX Transport
250 OK

MBX Transport

MBX Transport

MBX Transport

Store
DB 1 DB 2
Log

Store
Site Boundary
DB 1
Log

Store
DB 1
Log

Store
DB 1
Log

MBX1

DB 2

MBX2
250 OK

DB 2

Log

MBX3

DB 2

Log

MBX4

Transport MBX Transport Store

DB 3 DB 4

R3

Transport
Mail.que

Transport
Mail.que

Transport
Mail.que

Mail.que

MBX Transport Store

DB 3
Log

MBX Transport Store

DB 3
Log

MBX Transport Store

DB 3
Log

MBX5

DB 4

MBX6

DB 4

MBX7

DB 4

MBX8

Facilitates deployments at all scales from self-hosted small organizations to Office 365 Provides more flexibility in namespace management

All core Exchange functionality for a given mailbox is served by the MBX2013 server where that mailboxs database is currently activated Simplifies the network layer Transport protection is built-in

All components in a given server upgraded together No need to juggle with CAS <-> MBX versions separately

Utilize CPU core increase, cheaper RAM Utilize capacity effectively Fewer disks/server => simpler server SKUs

Thank you!