dat353 analysis service: server internals

DAT353DAT353

Analysis Service: Server Analysis Service: Server InternalsInternals

Tom ConlonTom ConlonProgram ManagerProgram ManagerSQL Server Business Intelligence UnitSQL Server Business Intelligence UnitMicrosoft CorporationMicrosoft Corporation

Purpose of this SessionPurpose of this Session

• Remove some of the mysteryRemove some of the mystery• Explain how it is that we do some things Explain how it is that we do some things

so much better than our competitorsso much better than our competitors• Things are easier to understand when Things are easier to understand when

the internals are understoodthe internals are understood• Requirements:Requirements:

– You already know the basics – this is for You already know the basics – this is for the experiencedthe experienced

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Architecture – Single ServerArchitecture – Single Server

OLAPOLAPStoreStore

Ap

plicatio

nA

pp

lication

AD

O

AD

O

MD

MD

Pivo

tTab

le P

ivotT

able

Service

Service

OL

ED

B fo

r O

LE

DB

for

OL

AP

OL

AP

AnalysisAnalysisServerServer

ProcessingProcessingQueryingQuerying

AnalysisAnalysisManagerManager

DSODSO

SQL ServerSQL ServerDataData

WarehouseWarehouse

OtherOtherOLE DBOLE DB

ProvidersProviders

OLEDOLEDBB

Component Architecture - Query

MMSMDSRV.EXE

CACHE

Serv

er S

tora

ge E

ng

ine

METADATA MANAGER

MSOLAP80.DLL

CACHE

FO

RM

ULA

EN

GIN

E

METADATA MANAGER

AGENT FORMUL

A ENGINE MDXMDX

Component Architecture - Management

MMSMDSRV.EXE

CACHE

Serv

er S

tora

ge E

ng

ine

METADATA MANAGER

MSOLAP80.DLL

CACHE

FO

RM

ULA

EN

GIN

E

METADATA MANAGER

AGENT FORMUL

A ENGINE

METADATA MANAGER

MSMDGD80.DLL

DCubePARSER

MSMDCB80.DLL

DC

ub

e S

tora

ge

En

gin

e

MDXMDX

DDLDDL

Component Architecture - Distributed

MMSMDSRV.EXE

CACHE

Serv

er S

tora

ge E

ng

ine

METADATA MANAGER

MSOLAP80.DLL

CACHE

FO

RM

ULA

EN

GIN

E

METADATA MANAGER

AGENT FORMUL

A ENGINE

METADATA MANAGER

MSMDGD80.DLL

MMSMDSRV.EXE

CACHE

Serv

er S

tora

ge E

ng

ine

METADATA MANAGER

DCubePARSER

MSMDCB80.DLL

DC

ub

e S

tora

ge

En

gin

e

MDXMDX

AgendaAgenda

• Architecture ReviewArchitecture Review

• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Why Aggregations?Why Aggregations?

• Aggregations can result in orders of magnitude Aggregations can result in orders of magnitude improvement in performanceimprovement in performance– Don’t have to access every fact table record to Don’t have to access every fact table record to

determine query resultdetermine query result– Further savings with data compressionFurther savings with data compression– Biggest savings: reduce disk scanBiggest savings: reduce disk scan

Aggregations - OverviewAggregations - Overview

CustomersCustomersAll CustomersAll CustomersCountryCountryStateStateCityCityNameName

ProductProductAll ProductsAll ProductsCategoryCategoryBrandBrandNameNameSKUSKU

FactsFactscustIDcustID SKUSKU Units SoldUnits Sold SalesSales

345-23345-23 135123135123 22 $45.67$45.67

563-01563-01 451236451236 3434 $67.32$67.32

……

Highest Level AggregationHighest Level AggregationCustomerCustomer ProductProduct Units SoldUnits Sold SalesSales

AllAll AllAll 347814123347814123 $345,212,301.3$345,212,301.3

Intermediate AggregationIntermediate AggregationcountryCodecountryCode productIDproductID Units SoldUnits Sold SalesSales

CanCan sd452sd452 94569456 $23,914.30$23,914.30

USUS yu678yu678 46234623 $57,931.45$57,931.45

……

Partial AggregationPartial Aggregation

• Don’t want to create all possible aggregationsDon’t want to create all possible aggregations– Data explosion!Data explosion!

• What if a query is made to a combination of levels What if a query is made to a combination of levels where no aggregation exists?where no aggregation exists?– Can compute from lower level aggregationsCan compute from lower level aggregations– Don’t need to compute every possible aggregationDon’t need to compute every possible aggregation

CustomersCustomersAll CustomersAll CustomersCountryCountryStateStateCityCityNameName

ProductProductAll ProductsAll ProductsCategoryCategoryBrandBrandNameNameSKUSKU

Queries including a Queries including a combination of Country and combination of Country and Brand can be answered if Brand can be answered if aggregation Country by aggregation Country by Name exists. Name exists.

Fact TableFact Table

Highest level of aggregation (1,1,1,1,…)Highest level of aggregation (1,1,1,1,…)

Most detailed Most detailed AggregationsAggregations(m,m,m,…)(m,m,m,…)

Show me all sales for all products for all . . .Show me all sales for all products for all . . .

AggregationsAggregations


Show me all sales for all products for all . . .Show me all sales for all products for all . . .

Most detailed Most detailed AggregationsAggregations

Highest level of aggregationHighest level of aggregation

Partial AggregationPartial Aggregation

Aggregation DesignAggregation Design


Mo

nth

Mo

nth

ProductsProducts

Qu

arterQ

uarter

Pro.Pro. Family Family

Qu

arterQ

uarter

ProductProductMo

nth

Mo

nth

Pro. Pro. Family Family


Aggregation DesignAggregation DesignResultsResults

• Result: aggregations designed in waves Result: aggregations designed in waves from the top of the pyramidfrom the top of the pyramid

• At 100% aggregations, ‘waves’ all touch: At 100% aggregations, ‘waves’ all touch: overkilloverkill

• 20-30% Generally adequate (0% for the 20-30% Generally adequate (0% for the smaller cubes)smaller cubes)

Aggregation DesignAggregation Design

• Which aggregations are more important than Which aggregations are more important than others?others?– All are equalAll are equal– From design perspective, select the ones that From design perspective, select the ones that

result in overall improved query performanceresult in overall improved query performance

• Usage Based Optimization: Weightings on Usage Based Optimization: Weightings on each aggregation based on usage frequencyeach aggregation based on usage frequency

Flexible and Rigid AggregationsFlexible and Rigid Aggregations

• ‘‘Flexible’ aggregations deleted when a changing dimension is Flexible’ aggregations deleted when a changing dimension is incrementally processed.incrementally processed.

• ‘‘Rigid’ aggregations remain validRigid’ aggregations remain valid

• Changing dimensions allow members to be moved, added Changing dimensions allow members to be moved, added and deleted.and deleted.

• After members move, only incremental process of dimension After members move, only incremental process of dimension is requiredis required

AA BB CC

When member X is moved from When member X is moved from a child of A to a child of C, all a child of A to a child of C, all aggregations involving A or C aggregations involving A or C are invalidedare invalided

XX XX

Aggregation Data StorageAggregation Data Storage

• No impact on fact data or rigid aggregation data when changing No impact on fact data or rigid aggregation data when changing dimension incrementally processeddimension incrementally processed

• Flexible aggregations are invalidated when changing dimension Flexible aggregations are invalidated when changing dimension incrementally processedincrementally processed

• Data is in three files:Data is in three files:– partitionNamepartitionName.fact.data.fact.data– partitionNamepartitionName.agg.rigid.data.agg.rigid.data– [[partitionNamepartitionName.agg.flex.data].agg.flex.data]

Aggregations including (All) level are Aggregations including (All) level are rigid (if all other levels in the agg are rigid (if all other levels in the agg are rigid)rigid)

Aggregations with this level are always Aggregations with this level are always flexibleflexible

Aggregations with this Aggregations with this level are rigid (if all other level are rigid (if all other levels in the agg are rigid)levels in the agg are rigid)

AA BB CC

XX XX

Incremental Dimension Processing Incremental Dimension Processing (Chg Dimension)(Chg Dimension)

Query and processQuery and processdimension data:dimension data:

Keys, member names,Keys, member names,member propertiesmember properties

For each cubeFor each cubeusing this dimensionusing this dimension

Delete flexibleDelete flexibleAggs and indexesAggs and indexes

Start lazyStart lazyindexingindexing

Start lazyStart lazyaggregatingaggregating

• Potential Resource Potential Resource Competition during lazy Competition during lazy processing after changing processing after changing dimension incrementally dimension incrementally processedprocessed

• Fewer Aggregations!Fewer Aggregations!

• Result: Query performance Result: Query performance degradationdegradation

Flexible Aggregation Flexible Aggregation DemoDemo

demodemo

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations

• Data and Dimension Data and Dimension StorageStorage

• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Data StorageData Storage• No data stored for empty member combinationsNo data stored for empty member combinations• With compression – data storage approx 1/3 of space With compression – data storage approx 1/3 of space

required in RDBMS sourcerequired in RDBMS source• Data is stored by record in pagesData is stored by record in pages• Each record contains all measures at an intersection of Each record contains all measures at an intersection of

dimension membersdimension members

Record 1Record 1: mbr: mbrd1d1, mbr, mbrd2d2,…mbr,…mbrdndn mm11, m, m22,…m,…mnn


……


PagePage

Data StructuresData Structures

• Partition data stored in a file divided into SegmentsPartition data stored in a file divided into Segments• Each Segment contains 256 pages (each with 256 Each Segment contains 256 pages (each with 256

records) = 64K recordsrecords) = 64K records

Segment 1Segment 1

……

Page 1Page 1

Page 2Page 2

Page 3Page 3

Page 256Page 256

……

Segment 2Segment 2 Segment nSegment n

……

Data FileData File

Only last segment has fewer than 256 pages

ClusteringClustering• Physical order of the records in each page and Physical order of the records in each page and

segment is organized to improve performancesegment is organized to improve performance– Keeps records with same or close members together Keeps records with same or close members together – Similar in concept to SQL clustered index where data Similar in concept to SQL clustered index where data

sorted by key valuessorted by key values• Try to minimize distribution of records with the same Try to minimize distribution of records with the same

member across segments and pagesmember across segments and pages– Optimized, but no algorithm can keep records for the Optimized, but no algorithm can keep records for the

same member (unless the cube contains a single same member (unless the cube contains a single dimension)dimension)

– Similarly – SQL can only have a single clustered index Similarly – SQL can only have a single clustered index • Records with Records with identical dimension membersidentical dimension members can be can be

in multiple segmentsin multiple segments– Data is read and processed in chunks (more on this Data is read and processed in chunks (more on this

later…) later…)

IndexingIndexing

• How is the data retrieved?How is the data retrieved?

– Cubes can be in the terabyte rangeCubes can be in the terabyte range

– Scanning data files not an optionScanning data files not an option

• Need an index by dimension memberNeed an index by dimension member

– Answers question “Where is the data associated Answers question “Where is the data associated with this combination of dimension members?”with this combination of dimension members?”

• Map files provide thisMap files provide this

Map FilesMap FilesThere is a map for each dimension which There is a map for each dimension which indicates the page where the member is indicates the page where the member is included in a data recordincluded in a data record

MemberMember MapMap

…… ……

12131213

…… ……

……

Page 1Page 1

Page 2Page 2

Page 3Page 3

Page 256Page 256

Page 4Page 4

Page 5Page 5

Page 6Page 6

Segment 1Segment 1

Dimension 1 MapDimension 1 Map

MemberMember MapMap

…… ……

13241324

…… ……

Dimension 2 MapDimension 2 Map

To resolve a query containing a To resolve a query containing a member from each dimension, get list member from each dimension, get list of pages containing all membersof pages containing all members

Other ApproachesOther Approaches

• Array BasedArray Based– Normally allocates a cell for every Normally allocates a cell for every

combination.combination.– Result: Data explosion - much more Result: Data explosion - much more

disk space and longer processing disk space and longer processing timestimes

• Mix of Record and ArrayMix of Record and Array– ‘‘Dense’ dimensions are record likeDense’ dimensions are record like– Sparse are array likeSparse are array like

• Bit used per empty cell – sparsity Bit used per empty cell – sparsity explodes db sizesexplodes db sizes

– User chooses decides whether a User chooses decides whether a dimension is dense or sparsedimension is dense or sparse

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage

• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Processing Buffer Memory SettingsProcessing Buffer Memory Settings

• ‘‘Read-Ahead Buffer SizeRead-Ahead Buffer Size’ is the buffer ’ is the buffer containing data read from source db containing data read from source db – Defined in Server Property Dialog. Default: 4MegDefined in Server Property Dialog. Default: 4Meg– Rarely important – little effect when changedRarely important – little effect when changed

• Data is processed in chunks of ‘Data is processed in chunks of ‘Process Process Buffer SizeBuffer Size’’– Defined in Server Property DialogDefined in Server Property Dialog– Data is clustered within Process Buffer SizeData is clustered within Process Buffer Size

• Bigger Bigger Process Buffer SizeProcess Buffer Size the better – make the better – make as big as possibleas big as possible– Data for dimension members is clustered to keep Data for dimension members is clustered to keep

data for ‘close’ members close together data for ‘close’ members close together – The larger these memory settings are, the more The larger these memory settings are, the more

effective clusteringeffective clustering

Incremental ProcessingIncremental Processing

…………

……

……

Original Original partitionpartition

Incremental Incremental PartitionPartition

++

Two Step ProcessTwo Step Process• First, a partition is First, a partition is

created with the created with the incremental dataincremental data

• Second, the partition is Second, the partition is merged with the merged with the originaloriginal

• Complete Segments of Complete Segments of both partitions left both partitions left intact – incomplete intact – incomplete ones are mergedones are merged

• After many After many incremental incremental processes, data processes, data distributed: degraded distributed: degraded performance performance

• Reprocess (if you have Reprocess (if you have a large Process Buffer a large Process Buffer size) can provide size) can provide improved performanceimproved performance

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing

• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Querying a CubeQuerying a Cube

• CLIENT CLIENT – Select {[North America],[USA],[Canada]} on rows, Select {[North America],[USA],[Canada]} on rows,

Measures.members on columns from myCubeMeasures.members on columns from myCube

• Need two things:Need two things:– getting the dimension members – the axesgetting the dimension members – the axes– getting the datagetting the data

Resolve AxisResolve Axis

‘‘Christmas trees’Christmas trees’

• Dimension members cached on client in Dimension members cached on client in ‘Client Member Cache’‘Client Member Cache’– Levels with #members < Large Level Levels with #members < Large Level

Threshold sent in groupThreshold sent in group– Levels with #members > Large Level Levels with #members > Large Level

Threshold retrieved as neededThreshold retrieved as needed– Large Level Threshold default value:1000, Large Level Threshold default value:1000,

can be changed in server property and in can be changed in server property and in connection stringconnection string

• Where members not cached, members and Where members not cached, members and descendents retrieved to client until needed descendents retrieved to client until needed member retrievedmember retrieved

• Levels with members with 1000s of siblings Levels with members with 1000s of siblings result in degraded performanceresult in degraded performance

• Member cache not cleaned except for Member cache not cleaned except for disconnect or when cube structure changes.disconnect or when cube structure changes.

Cached Cached membersmembers

Non-cached Non-cached membersmembers

Requested Requested membermember

Client Data CacheClient Data Cache

• Client retains data of previous queries in client data Client retains data of previous queries in client data cachecache

• Client Cache Size property controls how much data Client Cache Size property controls how much data is in the client cacheis in the client cache– When 0: unlimitedWhen 0: unlimited– 1-99 (inclusive), percent of physical memory1-99 (inclusive), percent of physical memory– >99 use up to the value in KB>99 use up to the value in KB

• Default value: 25Default value: 25• When exceeded, client cache is cleaned at cube When exceeded, client cache is cleaned at cube

granularitygranularity

How Cubes Are QueriedHow Cubes Are Queried

Segment 6Segment 6Segment 5Segment 5Segment 4Segment 4


Partition: CanadaPartition: Canada



Partition: MexicoPartition: Mexico



Partition: USAPartition: USA

DimensionDimensionMemoryMemory

CacheCacheMemoryMemory

Client DataClient DataCacheCache

Data on diskData on diskServiceService ClientClient

QueryQueryProcessorProcessor

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing• QueryingQuerying

• Server Memory Server Memory ManagementManagement

• Distinct CountDistinct Count

Service Start UpService Start Up• Minimum Allocated Memory Minimum Allocated Memory

defines the amount of memory defines the amount of memory completely dedicated to the completely dedicated to the serverserver

• All dimensions in the database All dimensions in the database are retained in memoryare retained in memory– Tip: invalidate a dimension if not Tip: invalidate a dimension if not

used in a cubeused in a cube• Dimension requirements: ~125 Dimension requirements: ~125

bytes per member bytes per member plus member plus member propertiesproperties– 1M members: 125M1M members: 125M– With 25 char member property (eg, With 25 char member property (eg,

Address): 175MAddress): 175M• Large dimensions can migrate to Large dimensions can migrate to

separate process spaceseparate process space


Min

imu

m allo

cated m

emo

ryM

inim

um

allocated

mem

ory

During ProcessingDuring Processing

• Shadow dimensionsShadow dimensions– 2 copies of dimensions stored in 2 copies of dimensions stored in

memory while processingmemory while processing

• Processing BuffersProcessing Buffers– Read Ahead Buffer sizeRead Ahead Buffer size– Process Buffer SizeProcess Buffer Size

• If dimension and processing buffers If dimension and processing buffers memory requirements exceed memory requirements exceed Memory Conservation Threshold - Memory Conservation Threshold - no room for data cacheno room for data cache


ShadowShadowDimensionsDimensions

AvailableAvailableCacheCache

ProcessingProcessingBuffersBuffers

Min

imu

m allo

cated m

emo

ryM

inim

um

allocated

mem

ory

Mem

ory co

nservatio

n th

resho

ldM

emo

ry con

servation

thresh

old

During QueryingDuring Querying• Data cache stores query data for reuseData cache stores query data for reuse

– Faster than retrieving from storageFaster than retrieving from storage

• If Dimension Memory requirements > If Dimension Memory requirements > Memory Conservation Threshold, no Data Memory Conservation Threshold, no Data CacheCache

• ‘‘Cleaner’ wakes up periodically to reclaim Cleaner’ wakes up periodically to reclaim memory from data cachememory from data cache– BackgroundInterval registry setting. Default BackgroundInterval registry setting. Default

value: 30 secondsvalue: 30 seconds


Min

imu

m allo

cated m

emo

ryM

inim

um

allocated

mem

ory

Mem

ory co

nservatio

n th

resho

ldM

emo

ry con

servation

thresh

old

AvailableAvailableCacheCache

<= 0.5 * (Minimum Allocated Memory+ <= 0.5 * (Minimum Allocated Memory+ Memory Conservation Threshold): No Memory Conservation Threshold): No cleaningcleaning

0.5 * (Minimum Allocated Memory + 0.5 * (Minimum Allocated Memory + Memory Conservation Threshold) and < Memory Conservation Threshold) and < Memory Conservation Threshold: mild Memory Conservation Threshold: mild cleaningcleaning Memory Conservation Threshold: Memory Conservation Threshold: aggressive cleaningaggressive cleaning

Setting Server Setting Server PropertiesProperties

demodemo

AgendaAgenda

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management

• Distinct CountDistinct Count

Distinct CountDistinct Count

• Business Problem: Sales Manager wants to know: Business Problem: Sales Manager wants to know: – ““How many customers are buying Computers?”How many customers are buying Computers?”– ““How many active customers do I have?”How many active customers do I have?”

500 Printers

700 Games2500 Business1500 Home4700 Software

800 Monitors2000 Computers3300 Hardware8000All products

Sales

30

80100100150

607080

200

Number of Customers

Distinct Count: Changes to Data StructureDistinct Count: Changes to Data Structure

• DC Measure stored with each fact and aggregation recordDC Measure stored with each fact and aggregation record– Just like a new dimensionJust like a new dimension

• Data ordered by DC measureData ordered by DC measure– ““Order by” included in SQL statement during processingOrder by” included in SQL statement during processing

• Number of records can be increased by orders of magnitudeNumber of records can be increased by orders of magnitude– Dependant on number of distinct values per recordDependant on number of distinct values per record

Sample aggregate record without Sample aggregate record without Distinct Count…Distinct Count…

countryCodecountryCode productIDproductID Units SoldUnits Sold SalesSales

CanCan sd452sd452 94569456 $23,914.30$23,914.30

……# records increases with distinct count on customers# records increases with distinct count on customers

CustIDCustID countryCodecountryCode productIDproductID Units SoldUnits Sold SalesSales

132-45132-45 CanCan sd452sd452 13051305 4,453.014,453.01

432-39432-39 CanCan sd452sd452 23252325 7,212.237,212.23

639-53639-53 CanCan sd452sd452 14061406 4,890.014,890.01

430-30430-30 CanCan sd452sd452 31103110 8,476.548,476.54

964-90964-90 CanCan sd452sd452 13101310 4,490.234,490.23

Distinct Count: Changes to QueryDistinct Count: Changes to Query

• Single thread per partition instead of per Single thread per partition instead of per segmentsegment– Unlike regular cubes, cannot do a single Unlike regular cubes, cannot do a single

aggregation of results from each segment as a aggregation of results from each segment as a single value of the DC measure can cross segmentssingle value of the DC measure can cross segments

– Consequently – performance impactConsequently – performance impact

• Dimension slice requires much more disk scan Dimension slice requires much more disk scan than beforethan before– Segments clustered by DC measureSegments clustered by DC measure– ExpensiveExpensive

Distinct Count TipsDistinct Count Tips

• Keep DC measures in their own cubeKeep DC measures in their own cube– All measures are retrieved on query – All measures are retrieved on query –

even if some are not asked foreven if some are not asked for– Create virtual cube to merge DC with Create virtual cube to merge DC with

other measuresother measures

• Incremental processing DC cubes is Incremental processing DC cubes is very expensivevery expensive– Segments restructured and reordered to Segments restructured and reordered to

keep records ordered by DC measurekeep records ordered by DC measure– Time and memory intensiveTime and memory intensive

Distinct Count TipsDistinct Count Tips

• Unlike regular cubes, best to distribute DC Unlike regular cubes, best to distribute DC values evenly across each partitionvalues evenly across each partition– Most effective use of multiple threads for query Most effective use of multiple threads for query

processing processing

• If you have a dimenion that corresponds to If you have a dimenion that corresponds to DistinctCount MeasureDistinctCount Measure– Aggregations recommended only on lowest Aggregations recommended only on lowest

levellevel– (Example, Customer dimension in cube, (Example, Customer dimension in cube,

Customer as distinct count measure)Customer as distinct count measure)

SummarySummary

• Architecture ReviewArchitecture Review• AggregationsAggregations• Data and Dimension StorageData and Dimension Storage• ProcessingProcessing• QueryingQuerying• Server Memory ManagementServer Memory Management• Distinct CountDistinct Count

Don’t forget to complete the Don’t forget to complete the on-line Session Feedback form on-line Session Feedback form on the Attendee Web siteon the Attendee Web site

https://web.mseventseurope.com/teched/https://web.mseventseurope.com/teched/

https://web.mseventseurope.com/teched/

dat353 analysis service: server internals

Documents