Effective Search Deployment and Operations in SharePoint 2013Kristopher Loranger Mert SangarSr. PFE Sr. PFEMicrosoft Microsoft

SPC360

Search Topology Review

SharePoint 2013 Search Architecture

SearchAdmin

Content UXCrawl

ContentProcessing Index

QueryProcessing WFE

API

AnalyticsProcessing

Crawl

Search Admin

DB

Link

Analytics Reporting

FAST Search Index

SharePointSP AppsDevicesNon-SP UX

HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS

Public API

Search topology components

Content Query

New-SPEnterpriseSearchTopology New-SPEnterpriseSearchCrawlComponent New-SPEnterpriseSearchIndexComponent … Set-SPEnterpriseSearchTopology

SharePoint Search TopologyVital (and large) part of your farm!

Scalable – content and query load

Fault-tolerant\HA

Appears as a SharePoint Service Application (SSA)

Manage using PowerShell

Status from UI/PowerShell

Main changes from SharePoint 2010… from SharePoint Server 2010 Search

Crawl Component

Query Component

SharePoint 2010 Search Service Application

Crawl Indexing Engine

Query Engine

Search Admin

Property Store (SQL)

Content

UXWFE

Main changes from SharePoint 2010… from FAST Search Server 2010 for SharePoint

FAST Content SSA

FAST Query SSA

FAST back-end components(managed separately)

Extensibility:• Sandbox• Entity

Extraction Indexin

g Engine

Query Engine

Content Pipeline

Analysis Engine

Query Pipeline

Search Admin

UXWFE

Content

Crawl

Main changes from SharePoint 2010… to SharePoint Server 2013 Search

Entire index on local disk

UXWFE

More details: SPC336 - Search Architecture and Sizing in SharePoint 2013

Link/query analysis &

recommendations

Separate crawl and indexing

Content

SearchAdmin

Crawl

ContentProcessing Index

QueryProcessing

AnalyticsProcessing

FAST Search Index

Property Store (SQL)

SharePoint 2013 Search Service Application

Where does Search live in the farm?Windows servicesSharePoint Search Host Controller service

Runtime/lifecycle control of search components (except crawler) hostcontrollerservice.exe

SharePoint Server Search serviceCrawl Component mssearch.exe mssdmn.exe

ProcessesNoderunner.exe

Runtime environment for search components (except crawler)

msseearch.exe

mssdmn.exe

CrawlComponen

t

noderunner.exe

Search Runtime Environment

hostcontrollerservice.exe

Host Controller

Sh

are

Poin

t A

pp

Serv

er

Admin entitiesSearch Service Instance: Provisioning of the search service on each boxSearch Service Application: SharePoint Configuration entity

Still there, but only Crawl Component

noderunner.exe

noderunner.exe

noderunner.exe

noderunner.exe

Admin Component

Query Processing Component

Content Processing Component

IndexComponent

Analytics ProcessingComponent

Search High-AvailabilityComponent HA mode of operation

Crawl Component Load-sharing, stateless in DB

Index ComponentPer partition Primary/Sec/Ter (dynamic selection)

Query load-sharing

Content/Query Processing components Load-sharing, stateless

Analytics Processing Component Distributed map/reduce, stateless in DB

Admin Component Primary/Secondary (dynamic selection)Hot standby

Host Controller (for search dictionary repository)

Primary/Secondary (cmdlet controlled)Lost primary –no newdictionaries deployed

> New-SPEnterpriseSearchTopology> New-SPEnterpriseSearchIndexComponent, New-

SPEnterpriseSearchQueryProcessingComponent, … > Set-SPEnterpriseSearchTopology

Set up your initial search topology

Prepare servers• HW/SW reqs!• Install SharePoint• Add servers to

farm• Start Search

Service Instance• Create SSA

Create a new topology object for the SSA

Add search components to the topology

Assign components to servers

Activate the new topology

This will replace the single-server topology with a scaled-out topology

Need more performance or capacity?Scale out your search farm

Create an edit copy of your active topologyNew-SPEnterpriseSearchTopology -CloneYou never modify the active topology object

Add before removeAdd componentsActivateGreen stateRemove components

Small search topologyOne index PartitionPrepared for 10 Mill indexed documents

Fault tolerance 2 physical servers – 4 VMs

Which components on same server?

Search admin db Link db

Analytics dbCrawl db

All SharePoint databases All SharePoint databases

Analytics

Replica ReplicaIndex partition 0

Query Processing Query Processing

Crawl

Admin

Content processing

Analytics

Crawl

Admin

Content processing

Application Server Application Server

Application Server Application Server

Redundant copies of all databases

VM01

VM03

VM02

VM04

Host server A

Host server D

Host server C

Host server B

Crawl

Content Processing

Index

Analytics Processing

Admin

Query Processing

More details: SPC336 - Search Architecture and Sizing in SharePoint 2013

Index PartitionReplica Replica

Query ProcessingQuery Processing

Demo

Deploying a Multinode Search Topology

Location of files on disk matters!Local disk for index files!Index operations are I/O demanding

At install timeYou can specify the data folder for all search components (including index location)

Per Index ComponentSeparate data folder for the index filesWhen creating a new index componentNew-SPEnterpriseSearchIndexComponent

-RootDirectory E:\Index

ULS logsPrevent disk contention/floodingSet-SPDiagnosticConfig -LogLocation F:\Logs

Note change from SP 2010 Set index location per SSI not supportedSet-SPEnterpriseSearchServiceInstance

-DefaultIndexLocation

Scale out by re-partitioning the indexWhen each partition exceeds 10 million itemsStateless partitioning using consistent hashingPause query and feedingStart with a symmetric index layout

Index

Index

Index

Index

Content Processing

Partitions

Index

Index

Replicas

Regional farm 2Regional farm 1

Index

Regional farm 3

Crawler

Crawler

Crawler Index

Index

Regional farm 2Regional farm 1

Index

Office 365 farm

Crawler

Crawler

Crawler Index

Index

Federation

Regional farm 2

Central Search Farm

Regional farm 1

Search service

CrawlerCrawler

Crawler

Crawler

Regional farm 3

Large enterprise: Multi-farm deploymentsCentral search farm Serving multiple content farmsEasier administrationBetter relevancyBetter end-user experience

Distributed deploymentSearch index on each farmSearch results using federationResult blocks in the search result

HybridOn premise and O365 – Federation Stepwise migration to the cloud

When to choose federation?Slow WAN between remote farmsRegulatory restrictionsMost users want search results from local sourcesHybrid (On premise and O365)

Keeping your search topology healthy

Search Diagnostic SourcesSystem/Component state

Central Admin SSA Search Administration (UI)PowerShell: Get-SPEnterpriseSearchStatus

Search health reports (UI)Content feeding performance, query rate/latency, 0 queries

Monitoring / telemetrySCOM management pack ULS logs and performance counters

Understanding component issuesGreen = Good!Yellow (degraded)

Transient component lifecycle states – Restarts and topology changesIndex replica not in sync

Red (failed / unknown)Manual intervention neededIndicates that server, or search component, is unreachablePossible HW/Network issues

Get-SPEnterpriseSearchStatus gives more details

Get-SPEnterpriseSearchStatusComponent state for each search component

Active / degraded / failed / unknown (not reachable)Extended info with additional data when yellow/red

Primary/secondary role for stateful HA components

Admin component, index component (per partition)

Status for analytics background jobsUsage analytics / Search analytics

Detailed diagnostic data for indexerIndex housekeeping (seeding, master merge, catch up)Diagnostic data for issue resolution

Get-SPSearchTopologyState.ps1One stop shop for search status

Consolidates data from multiple cmdlets and sources

All data from Get-SPEnterpriseSearchStatusCrawl status (crawling/idle/paused)# of indexed documentsHA status for topologyIndex size alertHost controller dictionary repository

Download: See TechNet Wiki

Demo

Health Status and Reports

Monitoring using SCOM

Content UXCrawl

ContentProcessing Index

QueryProcessing WFE

API

AnalyticsProcessing

CrawlSearch Admin

DBLink

FAST Search Index

Query Service: Service availability query processing

QueryProcessing

Index: Missing partition

Index

FAST Search Index

Content Processing: Gatherer Content Processing connector

ContentProcessing

Search Gatherer: Disk Full Crawler

Crawl

Analytics analysis: failed to start warning – search analytics

AnalyticsProcessing

TechNet SCOM knowledge article - http://technet.microsoft.com/en-us/library/ee513082(v=office.15).aspx

QueryContent

WFE

API

Key day-to-day search operations

Search admin and Business users jobsDaily – AdminsCheck search topology UI, any component not green there is a problemMonitor performance reports, crawler health and query health

Weekly – AdminsVerify # of documents per partition and disk space on index, crawler and analytics componentsDR backupSecurity updates + Cumulative updates

Daily – Business userReview usage reports for trends

No Result Queries aka 0 hits

Top Queries by DayAbandoned Queries

Weekly – Business userFor a better search experience create

Query RulesBest betsAuthoritative Pages

Patching and updatesSequence is important!Define your update groups

1st half of components2nd half of componentsAdmin Component

Search admin db Link db

Analytics dbCrawl db

All SharePoint databases All SharePoint databases

Analytics

Replica ReplicaIndex partition 0

Query Processing Query Processing

Crawl

Admin

Content processing

Analytics

Crawl

Admin

Content processing

Application Server Application Server

Application Server Application Server

Group 1 Group 2 Group 3

SP1 and Cumulative UpdatesOffice SP1! Current best practice is to install

At the very least it is recommended to install March PU + Oct CU

Limit Type Old Value New Value

Crawl components Supported 2 / SSA; 1 / server Unlimited*

Crawl databases Boundary 5 per SSA 15 per SSA

Crawl log entries Supported 100 million Unlimited

Index components Supported 60 / SSA; 1 / server 60 / SSA; 4 / server

Index partitions Supported 20 / SSA 25 / SSA

Indexed items Supported 100 M / SSA10 M / indexer

10 M / indexer (250 M / SSA implied)

Search components Supported Not Documented 64 / SSA

  

Limit Type Value

Content processing components

Supported 1 / server

Content sources Supported 50 / SSA

Index replicas Supported 3 / index partition

Very important unchanged limits

What is your SLA? Cold – Hours to Days

Warm – Minutes to Hours

Hot – Minutes to Seconds

Pros: Cheap…. Maintenance and operational

Cons: Slow to recover but this may be ok for your business

Pros: Fairly inexpensive to recover Cons: expensive and time-consuming to maintain

Cons: Very expensive and time-consuming to maintain

Pros: Fastest recovery for critical apps

IaaS and Disaster Recovery

Cloud ServiceVirtual Network

Windows Azure

SQL DR1(A6)

SP DR1

(Large)

AD1(X-Small)

On Premises

Web servers

Application servers

SQL DR2(A6)

SP DR2

(Large)

SP DR4

(Large)

SP DR5

(Large)

SP DR3

(Large)

Directory servers

VPN Tunel

Queries, Crawler/Content

Replace a failed server (with HA)

Add replacement server to topology

Clone active topologyAdd componentsActivate topology

Wait until added components are green

For index replica: Seeding must be completed

Remove old server from topology

Clone, remove, activate

Before you remove server from the farm!

Verify Host Controller repository status

Needs action if primary host controller was on the failed server

Verify status of primary host controller – in case it was on the failed server> Get-SPEnterpriseSearchHostController

If primary host controller was on the failed server:> Set-SPEnterpriseSearchPrimaryHostController

Demo

Replace a failed server (with HA)

Backup and Restore

Everything but the index is in databases(just about)

Restore possible to a different deploymentMay use B/R for standby DR locationRestore location must have the same basic topology

Ballpark figuresMin: 8 minutes to backup 10K index: 3 nodes, 2GB of data, 6 minutes restoreMax: 8 hours to backup 80M index: 12 nodes, 2TB of data, 6 hours restoreYMMV - test

Backup and RestoreSharePoint CA UI

PowerShell Backup-SPFarm -Directory <BackupFolder> -BackupMethod {Full | Differential} -Item “<SearchServiceApplicationName>”

How does the standard backup work?Consistent across all DBs and indexIndex backup phase 1

Backup the index – indexing paused – feeding continues – content updates persisted to the journal

Index backup phase 2Backup the journal – content updates since phase 1 started – feeding paused

Continued operations during backupQueries continue to work throughout the backupCrawl is paused for ~10% of the overall backup time

Demo

Backup

Survival Guide (TechNet Wiki)http://social.technet.microsoft.com/wiki/contents/articles/14382.survival-guide-sharepoint-2013-and-enterprise-search-sp2013es.aspx

TechNet Documentationhttp://technet.microsoft.com/en-us/library/cc303422(v=office.15).aspx

Related Resources

TechNet contentSearch AdministrationAdminister search in SharePoint Server 2013

Search TopologyScale search for performance and availability in SharePoint Server 2013Technical diagrams for SharePoint 2013Manage the search topology in SharePoint Server 2013Get-SPEnterpriseSearchStatus

Backup/RestoreBack up Search service applications in SharePoint 2013Overview of SharePoint Foundation and the Volume Shadow Copy Service

MySPCSponsored by

connect. reimagine. transform.

Evaluate sessionson MySPC using yourlaptop or mobile device:myspc.sharepointconference.com

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.