Large-scale SharePoint Architecture Eli Robillard

eli@erobillard.comTwitter: @erobillard

November 2011

THANKS TO OUR SPONSORS!

Agenda for this hour• The process, language and patterns

of architectural design. • Principals of sound architecture • Perspectives of SharePoint

architecture • Large-scale considerations

The Design Process

“The usual problem with process improvement is not having one to

begin with.”

The Design Process1. Define the elements. 2. Organise the elements into

entities. 3. Among the entities, identify the

relationships.4. Describe the processes that give

the system life.

Shared Language

Design Patterns• Christopher Alexander

– A Pattern Language, 1979

• Gang of Four– Design Patterns: Elements of Reusable

Object-Oriented Software, 1994

Change“The only constant is change.” –

Heraclitus

Good architects plan for day 1. Great architects plan for every day after that.

Principals of Sound Architecture

Have a process and get crisp on scope.

Express solutions in a shared language.

Identify and reuse design patterns.

Plan for change.

Introducing the worst acronym ever: SCLAPACH

Scope Language Patterns Change

Out-of-scopeBusiness value

“Feature <x> will be delivered in release <y>.”

Architectural StackGovernance :

Logical :

Physical :

Information :

Solution :

Team

Platform, farms, services and integration

Servers, networks and communications

Taxonomies and navigation

Applications

Architectural Stack

GovernanceGovernance is the architecture of the

team.

For every what, have a who and be sure who knows why.

Scope Language Patterns Change

Governance

Development : Integration :

Deployment :Test :

Operations :Training:

Marketing:

Write and compile code Assemble and document the package Install and configure the release UAT design, QA and load testing Monitor and maintain the platform Provide training and manuals Explain benefits, drive adoption

Teams and Tasks

Governance

Demo

Scope Language Patterns Change

Governance• Recommendations:

– Plan teams like server farms; think of roles as capabilities or services; decompose them until they can scale.

– People can wear multiple hats when it presents no conflict of interest.

Logical ArchitectureScope: Farms, zones, authentication,

application pools, web applications, site collections, service applications, external services.

Scope Language Patterns Change

Logical Architecture

Demo

Scope Language Patterns Change

Logical ArchitectureNew recommended limits: • 200 GB per Content DB • 4 TB supported limit for all use

cases• >4TB supported by Record Center

and Document Center

Scope Language Patterns Change

Physical ArchitectureScope: Servers (farm composition),

load-balancing, firewalls and port configuration, authentication providers, availability and failover.

Scope Language Patterns Change

Physical Architecture

Demo

Scope Language Patterns Change

Physical ArchitectureRecommendations:• Plan for full automation• Cookie-cutter provisioning

everywhere

Scope Language Patterns Change

Information Architecture

Scope Language Patterns Change

Information Architecture

Demo

Scope Language Patterns Change

Information ArchitectureRecommendations:• Create empty top-level content types

that inherit from Item and Document • Build for discovery • Keep taxonomies simple

Solution ArchitectureCommon problems:• How are solutions versioned?

– How do I deploy globally? – How is content upgraded?

• What is a SharePoint application? – Solution package payload – Features or sets of features – Site Template

Solution Architecture• Understand the solutions being

hosted– Make knowledge transfer part of change management

• Know:– What patch levels are expected? – Where is the payload going?

• GAC, BIN, 14 hive, SP Sandbox, Galleries, elsewhere? – Does it use elevated privileges? – How do I monitor and trouble-shoot?

Solution Architecture• Profile your content and applications

– Co-locate compatible solutions – Profile according to:

• SLA: Service windows and usage patterns • Users: zones and authentication • Corpus: Size, shape and growth of content • Physical isolation requirements • Feature availability vs. isolation

Client API: May be Javascript or JQuery clients for web services.

Service Layer: Web services, a service bus, or other protocols used to communicate across application domains.

Business Logic: Organizes raw data and relationships into concrete entities, with methods to allow interop among entities.

Data Access: Stored procedures or primitive classes to implement business rules, validation and security.

Data store: Relational, hierarchical or flat-file data.

Data Access ModelClient API or SharePoint BCS

Service Layer

Business Logic and/or Data

Access

Data Store

Solution Architecture• Insist on an n-tier architecture

– Data: .NET Provider model – Logic: .NET Define common business

entities – Presentation: Consider as an SDK

• Take a layered approach to building and provisioning application components

Where should these be documented?• Physical vs. virtual servers • My Sites • Service accounts • Disaster recovery environment

Rules of thumb“If it’s worth doing, it’s worth overdoing.” -

ER

“The problem with doing something right the first time is that no one will appreciate how difficult it was.” –Walt West

“Whatever you are, be a good one.” – Abe Lincoln

Thank-you! Please complete your evaluations to enter in the prize drawing.

References• Software Release Management Best Practices,

Kevin A. Lee, [Link].

Large-scale SharePoint Infrastructure

Eli Robillard eli@erobillard.com

Large-scale considerations• Patch management• Database scaling • Caching• Latency • Multi-lingual

Patch management issues• When content farms are patched

before service farms • Applications rely on patches • Manual vs. scripted processes

Patch management• Take a top-down approach to

upgrades • Decide between shared and central

ownership of the process • Track all regional upgrades and

associated tickets centrally • Consider your DB upgrades

(PSConfig)

Database scaling issues• Effects of long running operations

– Indexing, backup, profile imports, etc.

• Inability to scale onto new hardware – The single instance effect

Database scaling• Separate spindles for:

– OS, server logs (RAID 10 or 1), tempdb (RAID 10), SharePoint databases (RAID 10 or 5)

• Separate tempdb data files per CPU• Separate instances for:

– Content DBs, SP Analytics and Config DBs

• Change the default autogrowth setting

Caching• ASP.NET Page output cache• SharePoint object cache• BLOB cache• Content distribution networks (CDNs)

When you don’t use the object cache. . .

When you don’t use the object cache. . .

• What went wrong?– The hero control (which was implemented as a

Sandboxed Solution) was using the non-cached SharePoint API, instead of the cached API.

– The site was launched, and as soon as it got real load, the Sandboxed Solution throttling system (correctly) stopped executing the control.Good Publishing.CrossListQueryCache.GetSiteData (SPWeb, SPSiteDataQuery)

Bad SPList.GetItems(SPQuery)

Dealing with Latency• IIS Compression • WAN Acceleration • Remote terminals

Multi-lingual considerations• Configure variations sooner than

later and consider on-demand page propagation.

• If you baseline at SP1 plus either the August or October CU, language packs no longer need to be installed separately.

References• SPC382: Scaling SharePoint Records Centers to

Terabytes and Beyond, Russ Houberg, SharePoint Conference 2011.

• SPC 311: Best Practices for Building your Website for Scale with SharePoint 2010, Ethan Gur-esh , SharePoint Conference 2011.