hfm application design for performance

Hyperion Financial Management

Application Design for PerformanceApplication Design for PerformanceApplication Design for PerformanceApplication Design for PerformanceNovember 5, 2012

Chris BarbieriChris BarbieriChris BarbieriChris BarbieriPractice Director

Property of Edgewater Ranzal

Copyright 2012 Edgewater Ranzal

Personal Background

• Established HFM performance tuning techniques and

statistics widely used today

• 4+ years as Sr. Product Issues Manager at Hyperion

– HFM, Smart View, Shared Services, MDM

• 2001 HFM launch team 2001

• Certified HFM, Hyperion Enterprise

• B.S. Finance & Accounting, Boston College

• MBA, Babson College

• Established HFM Performance Tuning Lab at Ranzal

•



Foundation of Performance

• Hyperion Financial Management

• Metadata design as it impacts performance– Volume of members

– Impact of structures

• Data– Content

– Density

• Rules

• Environment



MetadataMetadataMetadataMetadata



12 Standard Dimensions

Application Profile

1. Year

2. Period

3. View

System

4. Value dimension,

includes

currencies

User controlled5. Entity6. Account7. ICP8. Scenario

User defined9. Custom 110. Custom 211. Custom 312. Custom 4



Application Profile

Year– No inherent impact on performance

– Can be increased after the application is built

– Impacts database table volume

Period– Base periods comprise column structure of everyeveryeveryevery table, whether you use them or not

– Avoid weekly profiles unless it is key to your entireentireentireentireapplication’s design

– Yearly is inadvisable

View– No impact, but only YTD is stored

– Other views are on-the-fly derivations• Consider the UI click volume



System DimensionValue Dimension

– Can not directly modify this

– “<Entity Currency>” points to entity’s default currency

– “<Parent Currency>” points to default currency of the entity’s parent• Anything above “<Entity Curr Total>” must be Parent.ChildParent.ChildParent.ChildParent.Child format

Currencies– Don’t add currencies you aren’t using

• Sets of calc status records for (every entity * every currency)

• Impact of loading metadata with entity or currency changes

– Normally translate from the entity’s currency only into it’s parent’s currency

– Beware of non-default translations• Impacted calc status

• Data explosion

• Adds to cycle time



User Controlled Dimensions

Entity

– Sum of the data of the children

– Avoid Consolidate All or All With Data on each hierarchy

– Assign Adj flags sparingly•Don’t disable if you ever had journals on entity

ICP

– “Hidden” dimension

Scenario

– Number of tables



Impact of Account Depth

� Effect is multiplied when you consider the custom dimensions

� Parent accounts don’t lock

4- Net Income

3- Optg Income

2- Gross Margin

1- Sales

4- Optg Income

3- Gross Profit

2- Gross Margin

1- Sales

5- EBIT

6- Net Income



User Defined Dimensions

Custom 1..4

– Think dozens or hundreds, but not thousands• If Thousands are necessary, 64 bit makes this possible

•Rules remain a major factor in performance

– Avoid:•Employees

•Products

•Anything that is very dynamic, changing greatly from year to year

•One to one relationship with the entities

Configurable dimensions in 11.1.2.2???



Metadata Efficiency Ratio

What does the average entity have in common with the top entity?– Density measurement of re-use of the accounts and customs across all entities

top entity

base



Metadata Volumes (Americas)80 Applications Median +1 Std Deviation High

Accounts 1,383 2,814 7,491

ICP Accounts With Plug 17 291 2,273

Accounts With Data Audit 26 1,356 7,490

Consolidation Rules 45%

OrgBy Period 16%

Phased Submission 19%

Consolidation Methods - 3 16

Currencies 25 57 150

Custom1 177 3,248 23,897

Custom2 67 2,397 20,484

Custom3 46 919 5,681

Custom4 19 184 1,199

Entity Hierarchies 4 12 44

Entities (unique) 672 4,242 21,199

ICP Members 200 1,161 7,770

Scenarios 10 27 81

Scenarios Using Process Management - 6 37

Scenarios Using Data Audit - 11 78



DataDataDataData



What’s a Subcube?

• HFM data storage unit

• Database tables stored by

– Each record contains allallallall periods for the [Year]

– All records for a subcube are loaded into memory togethertogethertogethertogether

Parent subcube,

stored in DCN tables

Currency subcubes,

stored in DCE tablesProperty of Edgewater R

anzal


Take it to the Limit

Reports, Grids, or Forms that:

– Pull lots of entities

– Lots of years

– Lots of scenarios

Not so problematic:

– Lots of accounts

– Or Custom dimension members

Smart View

– Cell volume impacts bandwidth

– Subcubes impact server performance



Data Design

• Density

• Content– Specifically: zeros

– Tiny numbers

– Invalid Records

“Metadata volume is interesting, but it’s

how you it that matters most”



Data Volume Measurement

•No perfect method

MethodMethodMethodMethod HowHowHowHow----ToToToTo ProsProsProsPros ConsConsConsCons

Data ExtractData ExtractData ExtractData Extract Extract all data,

count per entity

Simple, easy to see

input from calculated

Can only extract

<Entity Currency>

FreeLRUFreeLRUFreeLRUFreeLRU Parse HFM event

logs

Good sense of average

cube, easy to monitor

monthly growth

Can’t identify

individual cubes,

harder to understand

Database Database Database Database

AnalysisAnalysisAnalysisAnalysis

Query DCE, DCN

tables and count

Easy for a DBA, see all

subcubes

Doesn’t count

dynamic members,

includes invalid

records



Data Density Using FreeLRU

•Survey of data density using FreeLRU

methodNumber of applications reviewed:

44

Median Min Max +1 Std Dev

NumCubesInRAM 1,369 72 15,152 5,068

NumDataRecordsInRAM 1,170,908 247,900 23,019,754 4,574,074

NumRecordsInLargestCube 53,089 2,508 593,924 169,272

Records per cube 1,352 24 91,418 15,832

Metadata efficiency 3.4% 0.3% 39.7% 12.3%



Loaded vs. Consolidated Data

• What percent of the loaded data is a zero value?– <5% is reasonable

– No zeros are best

– Watch ZeroView settings on scenarios

• Watch out for tiny values, from allocations

• How much does the data expand from Sub Calculate?

• How many zeros are generated by the consolidation process?– Intercompany eliminations

– Allocations

– Empty variables

Loaded 0.9%

Calculated 9.4%

Consolidated 19.6%Property of Edgewater Ranzal


Growth Up the Entity Chain

LevelLevelLevelLevel NumberNumberNumberNumber RecordsRecordsRecordsRecords

Top EntityTop EntityTop EntityTop Entity 1 261,593

Average SubcubeAverage SubcubeAverage SubcubeAverage Subcube 814 5,193

Base entities including calculated dataBase entities including calculated dataBase entities including calculated dataBase entities including calculated data 516 680

Base entities input data onlyBase entities input data onlyBase entities input data onlyBase entities input data only 443 421

Base 421

Average 5,193

Top 261,593



Loaded, Calculated, and Consolidated

•Rough stats: median from 10101010 applications

MonthlyMonthlyMonthlyMonthly

Zeros Zeros Zeros Zeros

%%%%

Monthly Monthly Monthly Monthly

GrowthGrowthGrowthGrowth

Rules Rules Rules Rules

Growth Growth Growth Growth

Loaded RecordsLoaded RecordsLoaded RecordsLoaded Records 153,826 4.1% 3.3%

Loaded + Calculated Loaded + Calculated Loaded + Calculated Loaded + Calculated

RecordsRecordsRecordsRecords 353,122 19.7% 2.7% 2.0

Consolidated RecordsConsolidated RecordsConsolidated RecordsConsolidated Records 63,432 6.9% 3.2%

� Total data for all base (or top) entities

� Can be easily managed by better rule writing!



Invalid Records

• Type 1: Orphaned records from metadata that has been deleted– Member is removed from dimension_Item table, but not from the data tables

– These can be removed by Database > Delete Invalid Records

• Type 2: the member still exists, but is no longer in a valid intersection– Most often from changing CustomX Top Member on an account

– These cannot be removed by HFM, but are filtered out in memory



How Much Memory Do I Need?

Plan APlan APlan APlan A Plan BPlan BPlan BPlan B

Number of entitiesNumber of entitiesNumber of entitiesNumber of entities 814 814

* 2 cubes: entity currency + contribution* 2 cubes: entity currency + contribution* 2 cubes: entity currency + contribution* 2 cubes: entity currency + contribution 1,628 1,628

NonNonNonNon----USD entitiesUSD entitiesUSD entitiesUSD entities 483 483

add another cube for parent currencyadd another cube for parent currencyadd another cube for parent currencyadd another cube for parent currency******** 483 483

Entity_value cubesEntity_value cubesEntity_value cubesEntity_value cubes 2,111 2,111

Actual 2011, 2012Actual 2011, 2012Actual 2011, 2012Actual 2011, 2012 2 2

4 Currency scenarios, 3 Estimate, 3 Forecast scenarios4 Currency scenarios, 3 Estimate, 3 Forecast scenarios4 Currency scenarios, 3 Estimate, 3 Forecast scenarios4 Currency scenarios, 3 Estimate, 3 Forecast scenarios 3 10

Total Year_scenariosTotal Year_scenariosTotal Year_scenariosTotal Year_scenarios 5 12

Total cubesTotal cubesTotal cubesTotal cubes 10,555 25,332

Average records per cubeAverage records per cubeAverage records per cubeAverage records per cube 5,193 5,193

Optimal MaxRecordsInRAM settingOptimal MaxRecordsInRAM settingOptimal MaxRecordsInRAM settingOptimal MaxRecordsInRAM setting 54,812,115 131,549,076

bytes per recordbytes per recordbytes per recordbytes per record 120 120

Records * bytes converted to MB = Records * bytes converted to MB = Records * bytes converted to MB = Records * bytes converted to MB =

MaxDataCacheSizeInMBMaxDataCacheSizeInMBMaxDataCacheSizeInMBMaxDataCacheSizeInMB 6,273 15,055

** Many entities are translated into other currencies as well, making this value low** Many entities are translated into other currencies as well, making this value low** Many entities are translated into other currencies as well, making this value low** Many entities are translated into other currencies as well, making this value low....



Rules TimingRules TimingRules TimingRules Timing



Data Density <> Calc Time

correlation between density and calc times

• Most applications are rules bound

-

0.500

1.000

1.500

2.000

2.500

-

100

200

300

400

500

600

700

800

900

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Seconds

Seconds

Seconds

Seconds

Records

Records

Records

Records

Average Rule Execution Time in Contrast with Data VolumeAverage Rule Execution Time in Contrast with Data VolumeAverage Rule Execution Time in Contrast with Data VolumeAverage Rule Execution Time in Contrast with Data Volume



But Some are I/O Bound

Time vs. Volume

050

100150200250300350400450

8382

0.83

820_

DF

R.F

R_N

BM

TO

TA

L_R

EG

ION

S.U

SU

SC

A.U

SE

ME

A.D

EA

P.C

NC

Z.C

Z_N

BM

DE

_NB

M.8

3704

FR

_NB

M.8

3519

TH

.838

99U

S.U

SG

OU

S.8

0808

BR

.835

4583

820.

8382

0_18

01O

TH

AP

.828

2883

820.

8382

0_18

51E

ME

A.B

ELA

.BR

US

.808

20A

R.8

3856

Sec

on

ds

0

10,000

20,000

30,000

40,000

50,000

60,000

elapsed

totalrecords



How Long Should Rules Take?

• Total consolidation time for all entities, 12 periods

• Divide by 12 periods and total number of entities

0 0.25 2.0 4.0 10.0

Seconds Per EntitySeconds Per EntitySeconds Per EntitySeconds Per Entity



Rules Impact Ratio

• Total consolidation time

with rules

• Divided by time with Blank

Rules

• Typically 2- 5 times

• More than that is an

opportunity for improvementProperty of Edgewater R

anzal


Reference Reference Reference Reference

ApplicationApplicationApplicationApplicationProperty of Edgewater R

anzal


Small but Constant Application

•Applied across multiple environments

0:00:00

0:00:43

0:01:26

0:02:10

0:02:53

0:03:36

0:04:19

physical physical virtual virtual virtual virtual virtual virtual

HFM lab Cust E Ranzal dev T-61 laptop Cust A Cust B Cust C Cust D

Full Rules Blank Rules



Chris BarbieriChris BarbieriChris BarbieriChris [email protected]@[email protected]@ranzal.com

NeedhamNeedhamNeedhamNeedham, MA, MA, MA, MA

USAUSAUSAUSA

+1.617.480.6173+1.617.480.6173+1.617.480.6173+1.617.480.6173

www.ranzal.comwww.ranzal.comwww.ranzal.comwww.ranzal.comProperty of Edgewater R

anzal

hfm application design for performance

Technology

performanceapplication

gross margin1

cubeaverage

parent currencyadd

comcbarbieriranzal

currency

data

subcube