formula status update new usage patterns for finrep
DESCRIPTION
Formula Status Update New Usage Patterns for FINREP. Herm Fischer Formula Working Group 2008-04-20. Formula Status. Status = “Proposed Recommendation” 45 day wait before Recommendation Four “known” implementations Fujitsu & UBmatrix conformant, in production use - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/1.jpg)
1
Formula Status UpdateNew Usage Patterns for FINREP
Herm FischerFormula Working Group
2008-04-20
![Page 2: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/2.jpg)
2
Formula Status
• Status = “Proposed Recommendation”– 45 day wait before Recommendation
• Four “known” implementations– Fujitsu & UBmatrix conformant, in production use– CompSci Resources & New Lido implementations
• Major stake holders– BdE, BdF, BoJ, SEC deployed– FDIC has early-IWD formulas
![Page 3: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/3.jpg)
3
Specifications are Extendable
• Basic usage patterns in PR spec– Producing fact items (output instance document)– Assertions for
• Consistency (produced fact vs. reported fact)
• Existence
• Value
– Single input, single output of instances
• Extension usage patterns
![Page 4: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/4.jpg)
4
Extension usage patterns
Implementation prototypes• Message composition• Formula chaining• Tuple generation• Multi-instance processing• Linkbase & footnotes functions• Custom functions implemented in XPath• Assertion sets
Interesting, not implemented yet• Very Large Instances processing
![Page 5: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/5.jpg)
5
Existing prototypes
• Chaining and tuple generation– XSB required prototype to assure PR supports it
• Multi-instance processing – Became core part of chaining proposal– Prototyped (partially)
• Linkbase tree walks– Prototype allows linkbase to control formulas of
• Calc linkbase & dimension aggregation roll ups
• Movements & totalling by presentation linkbase
![Page 6: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/6.jpg)
6
Discussing by history vs. Discussing by FINREP urgency
• Message generation (needed)• Chaining (cool, but getting by without this)• Tuple generation (just for GL people?)• Multi-instance (it is the chaining solution)• Custom functions with XPath (need this)• Linkbase functions (top priority)
– Patterns in linkbase eliminate most formulas– Moves formula semantics from code to linkbase– Critical for success
• Assertion sets (in use)
![Page 7: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/7.jpg)
7
Chaining
• Most frequently request usage case
• Least needed (based on experience)
• Two solutions implemented – Multi-instance approach most flexible– Explicity dependencies useful for tuple output
• Facilitates modularization
• Helpful to manage large formula projects
![Page 8: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/8.jpg)
8
Chaining with explicit dependencyPrior result passed as factVariable
chained variableSet
chained variableSet e.g., nested tuple contents facts
factVariablefactVariablefilter(s)
V-F
factVariablefilter(s)
V-F
formula evaluation
factVariable
fact & gen’l variables by dependency, precondition, result, then varSet arcs to nested varSets (e.g., consis or tuple)
factVariablefilter(s)
factVariablefilter(s)
V-F
for consisAsser, parameters; for nested formula(s), factVars & genVars
location aspectfor nested results
precondition
source instance fact variables
precondition
consistencyAssertion
consisAssertion
![Page 9: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/9.jpg)
9
Dependency is explicit
• The arc assigns prior result to a factVariable
• The \author explicitly specifies this dependency
• Important for tuple output
• Difficult to maintain in large formula sets
• Difficult to factor formulae to separate files
![Page 10: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/10.jpg)
10
Tuple chaining must be explicit
Formula for deeper nested tuplefactVariableconceptName filter F-V
factVariableconceptName filter
V-F
factVariableconceptName filter
V-F
Formula for nested tuplefactVariableconceptName filter F-V
factVariableconceptName filter
V-F
factVariableconceptName filter
V-F
Formula for outer tuplefactVariable
conceptName filter
F-V
fact variables by their order dependency, then formula-formula arcs to nested tuple
factVariableconceptName filter
V-F
factVariableconceptName filter
V-F
![Page 11: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/11.jpg)
11
Muti-instance chaining solution
• Common solution to multi-instance and chaining
• Let’s first look at multi-instanceand then come back to chaining that uses it
![Page 12: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/12.jpg)
12
Other multi-instance solutions
• Complete DTS with linkbases needed– For tree walks: movements, totaling, and dimensions
• fn:doc() not useful to load instances– Does not discover DTS– Does not load linkbases– Can not handle shared DTS components
• Some referenced taxonomies common
• Some linkbases evolve
![Page 13: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/13.jpg)
13
Prior: xfi:inst() was prototyped
• Provides a fn:doc() counterpart to load DTS
• generalVariables can access these instances– No filtering on multi-instances– factVar, filter on primary instance,
genVar, XPath on additional instances
• Requires formula execution code to load instances (instead of formula processor infrastructure)
![Page 14: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/14.jpg)
14
New multi-instance features
• All instances loaded by infrastructure(doesn’t have to be coded)
• Filters, functions, sequence, covering work
• Should share formulas & filters - all instances
![Page 15: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/15.jpg)
15
New multi-instance solution
• Instances are represented by instance resource
• instance-variable arc to factVariable– If present, specifies non-default source instance
• formula-instance arc from formula– If present specifies the instance to receive fact
• Instance resources are files or temporary
![Page 16: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/16.jpg)
16
Instance resources
• Could be loaded by processor– E.g., java code in a server loads primary instance and
some prior-period or other-company instances– Or user of GUI adds ‘additional’ instances, such as
loading prior-period or other-company instances
• Default implied source and result instances
• Can be temporary in memory only– Used for chaining and modularization
![Page 17: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/17.jpg)
17
Multi-instance solution
• A better approach to chaining
• Implements multiple instance documents
• Applies to very large instance solution
![Page 18: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/18.jpg)
18
Multi-source and result instances
formula a = $b + $c
$d
fact
to result instance
factVariable
$c
filter(s)
V-F
factVariablefilter(s)
V-Fformula c = $d + $e
$e
fact
to result instance
fact & gen’l variables by dependency, precondition, result, then varSet arcs to nested varSets (e.g., consis or tuple)
factVariable
$d
filter(s)
V-F
factVariablefilter(s)
V-F
consisassertion
consisAsser
location aspectfor nested results
precondition
precondition
consistencyAssertion
consisAssertion
source instance (default)
temporary result
source instance (default)
result instance
source instance (default)
specifiedoutput
instance
defaultoutputinstance
output result(default)
![Page 19: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/19.jpg)
19
Aspect sources, implicit filtering
• Formula aspects come from its variables
• Variables from different instances contribute aspects– Aspects independent of the instances they come from– Aspect “covering” is by-aspect, not by-instance
![Page 20: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/20.jpg)
20
A=B+C; C=D+E use case (Explicit dependency chaining)
• Formula 1 (C=D+E)– Result is C, factVariables D & E– factVariables D & E are from the source instance
• Formula 2 (A=B+C)– Arc from formula 1, name $r given to Formula 1 result– Result is A, factVariables B & C– factVariable B is from source instance– factVariable $r is from result of formula 1
![Page 21: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/21.jpg)
21
A=B+C; C=D+E (Example 0027 v-01)Explicit dependency chaining
chained formulaA = B + C
fact AfactVariablefilter(s)
V-F
factVariable$r from above
formula evaluationC = D + E
fact C
fact & gen’l variables by dependency, precondition, result, then varSet arcs to nested varSets (e.g., consis or tuple)
factVariablefilter(s)
factVariablefilter(s)
V-F
for consisAsser, parameters; for nested formula(s), factVars & genVars
location aspectfor nested results
source instance fact variables
consistencyAssertion
consisAssertion
![Page 22: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/22.jpg)
22
A=B+C; C=D+E use case (Multi-instance chaining)
• Formula 1 (A=B+C)– Result is A, factVariables B & C– factVariable B is from source instance (default)– factVariable C is from result instance (has an arc)
• Formula 2 (C=D+E)– Result is C, factVariables D & E– factVariables D & E are from the source instance
![Page 23: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/23.jpg)
23
A=B+C; C=D+E (Example 0026 v-01) Multi-instance chaining
Formula evaluationA = B + C (from above)
$b
fact A
to result instance
factVariable
$c
filter(s)
V-F
factVariablefilter(s)
V-F
formula evaluationC = D + E
$e
fact C
to result instance
fact & gen’l variables by dependency, precondition, result, then varSet arcs to nested varSets (e.g., consis or tuple)
factVariable
$d
filter(s)
V-F
factVariablefilter(s)V-F
location aspectfor nested results
source instance (default)
result instance
source instance (default)
source is the result instance
source instance (default)
result instance (default)
result instance (default)
![Page 24: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/24.jpg)
24
COREP Use case 18: Weighted average of member children • Weighted average of its dimensional children
by another primary item
ii
iii
dp
dpdpdp
)(
)()()(
.12
.12.11
11
Primary items ==>PD Assigned to the obligor grade Exposure value
Capital requirements
Total Exposures (dimension) 60% 26.750,00 € 1 Assigned to obligor grade 27% 1.250,00 € Obligor grade 1 10% 100,00 € Obligor grade 2 20% 150,00 € Obligor grade 3 30% 1.000,00 € 2 Specialized lending slotting 62% 25.500,00 € Risk weight 0% 50% 500,00 € Risk weight 10% 75% 20.000,00 € Risk weight 150% 10% 5.000,00 €
![Page 25: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/25.jpg)
25
Current single-formula solution
• Excel formulas:
• Make PD controlling fact, get PD and EV of dimensional children
• General variable for PDxEV member matching
Primary items ==>PD Assigned to the obligor grade Exposure value Capital requirements
Total Exposures (dimension) =(B3*C3+B7*C7)/C2 =C3+C7 1 Assigned to obligor grade =(B4*C4+B5*C5+B6*C6)/C3 =SUMA(C4:C6) Obligor grade 1 0,1 100 Obligor grade 2 0,2 150 Obligor grade 3 0,3 1000 2 Specialized lending slotting =(B8*C8+B9*C9+B10*C10)/C7 =SUMA(C8:C10) Risk weight 0% 0,5 500 Risk weight 10% 0,75 20000 Risk weight 150% 0,1 5000
![Page 26: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/26.jpg)
26
<f:formula ...
value="$v:sumPDxEV div sum($v:EVkids)"
source="v:PD" />
Tests PD of input instance is within ½% of weighted average value
E-L
factVariable
PD
factVariablePdkids
(sequence)
Binds to any PD fact
Sequence of PD values for dimensional children of $PD
E-L
E-L
conceptName filterqname c:PD
label resources
explicitDimension filter, axis=childdimension qname c:ExposuresDimensionvariable c:PD
Pdkids & Evkids depend on PD, PDxEV
on both Pdkids & EVkids
factVariableEVkids
(sequence)
conceptName filterqname c:EV
Sequence of PD values for dimensional children of $PD
E-L
preconditiontest=”count($PDkids) eq count($EVkids)
and sum($EVkids) gt 0"
general variable sumPDxEVsum( for $pd in $PDkids, $ev in $EVkids[xfi:dimension-value(.,QName(‘ExposuresDimension’) = xfi:dimension-value($pd,QName(‘ExposuresDimension’)] return $pd * $ev )
Sequence of ev x pd for matching dimensions
consistencyAssertion strict="false" acceptRadius=".005"
Single formula (Example 0017 v-01)
difficultto explain
![Page 27: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/27.jpg)
27
Exposure value formula
• Each PD x EV produced by one formula– Result factItem PDxEV is the product for each
dimension value
• Second formula binds PDxEV’s of dim-children to sequence and EV’s of dim-children to second sequence, value assertion checks result
![Page 28: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/28.jpg)
28
New idea: multiple result instances
• The PDxEV result fact items aren’t needed for a real result instance
• Only a value assertion is really needed
• A temporary-results instance might be useful
• Also a temporary facts DTS would be needed (to define the PDxEV result fact item)
![Page 29: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/29.jpg)
29
Chained formulas (0026 v-20)
<f:formula ... value="sum($v:PDxEVkids) div sum($v:EVkids)" source="v:PD" />
Tests PD of input instance is within ½% of weighted average value
factVariablePD
factVariableEVkids
(sequence)
V-F
Binds to any PD fact
Sequence of EV values for dimensional children of $PD
E-LconceptName filter
qname c:PD
label resources
explicitDimension filter, axis=childdimension qname c:ExposuresDimensionvariable $PD
EVkids & PDxEvkids depend on PD,
PDxEV depends on above formula
factVariablePDxEVkids(sequence)
conceptName filterqname c:PDxEV
Sequence of PDxEV values for dimensional children of $PDfrom temp instance source
preconditiontest=”count($EVkids) eq count($PDxEVkids)
and sum($EVkids) gt 0"
consistencyAssertion strict="false" acceptRadius=".005"
<f:formula ... value="$v:PD * $v:EV" source="v:PD" conceptName=v:PDxEV/> (syntax abstracted)
Produces PDxEV for each dimension
factVariablePD
conceptName filterqname c:PD
factVariableEV
conceptName filterqname c:EV
conceptName filterqname c:EV
source instance (default)
result instance
fact PDxEV
![Page 30: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/30.jpg)
30
Implementation issues
• Multi-instance term binding– Variables can be bound to different source instances– (This already exists in xfi:inst() based solution.)– Each term in XPath ‘knows’ its instance/DTS (in the
internal model or DOM of implementation)
• Function binding– A function with item results must keep the instance/DTS
of the function result (based on the input terms)
![Page 31: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/31.jpg)
31
Tree walking
• Current implementation– Navigation returns concepts and attributes
( (c1, c2, c3), (a11, a12, a13), (a21, a22, a23) )– take subsequences with XPath for-loops– working (geeky)
• Change idea (not prototyped yet)– Navigation returns fully resolved relationship nodes
• A relationship has reference to arc node attributes
– Attributes: rel/@weight, rel/@preferredLabel– Concepts: maybe xfi:from/xfi:to( rel-node )
![Page 32: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/32.jpg)
32
Use of tree walking
• Calculation linkbase checking by formula– Uses xfi function for linkbase tree walk– Roll ups compared
• By threshold value
• By rounded values same as ordinary calc validation
– Extended links managed by formula• EDInet consolidated vs nonConsolidated conflicts
• Dimension aggregation by formula– Uses dimension filter child/descendant feature
![Page 33: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/33.jpg)
33
FINREP formulas
• Most current formulas can be custom tree walk– Consider optional/required attribute– Consider fall back values by arc attribute– Consider dimension filter by arc attribute– Other attributes as needed
• Replace 72± (BdF count) with few tree walks
![Page 34: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/34.jpg)
34
Very Large Instances Use Case
• Sizes > ½ million facts, > ½ GB DOMs– Census, Tax office, Security exchanges, etc.
• Multi-GB heaps not feasible with Java VMs– moribund at couple GB (incl code & data)
• Data almost always from Relational DBMSes
![Page 35: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/35.jpg)
35
Very Large Instances approach
• Basic PR formula solution– All facts, all filters, all variable sets in parallel– Not feasible with very large single- or multi-instances
• Multi-instance approach– Allows modularizing processing– Stage formulas to work on parts of very large instance– Cooperative filters & (stored) SQL DB interfaces– Intermediate result instances pass between stages
![Page 36: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/36.jpg)
36
Staged multi-instance strategy
SQL
SQL
SQL
Very LargeRelational DB
Formula Linkbase(s)
lazy loadearly GCinstance
filtersformula &variables
interiminstance
filtersformula &variables
filtersformula &variables
result
facts
facts
facts
interiminstance
![Page 37: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/37.jpg)
37
Custom functions with XPath
• Custom functions in PR require Java code
• Examples of custom functions– Taxonomy and linkbase access– Math with exponentials and recursion (loan value calc)
• Prototype adds XPath implementation
![Page 38: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/38.jpg)
38
a(b,c) = $b + $c (Example 0030 v01)
• <formula:formula value="my-fn:a($b, $c)“ …>• <function-impl:function xlink:type="resource"
xlink:label="cust-fn-a“ name="my-fn:a“ output="xs:decimal“ value="$b + $c" > <function-impl:input name="b“ type="schema-element(xbrli:item)" /> <function-impl:input name="c“ type="schema-element(xbrli:item)" /> </function-impl:function>
![Page 39: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/39.jpg)
39
Precision by unit (Example 0030 v-03)
<formula:decimals>my-fn:decimals($b) </formula:decimals>
value=" for $unit in local-name-from-QName( xfi:measure-name( xfi:unit-numerator( xfi:unit( $item ))[1] )) return ( if ($unit eq 'JPY') then -5 else -2 ) " >
![Page 40: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/40.jpg)
40
Recursion (Example 0030 v-04)
<function-impl:function xlink:type="resource" xlink:label="cust-fn-a" name="my-fn:power" output="xs:decimal" value=" if ($exp lt 0) then ( 1 div my-fn:power($y, - $exp) ) else ( if ($exp lt 1) then 1 else ($y * my-fn:power($y,$exp - 1)) ) " > <function-impl:input name="y" type="xs:decimal" /> <function-impl:input name="exp" type="xs:decimal" /> </function-impl:function>
![Page 41: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/41.jpg)
41
Present value (Example 0030 v-05)
<formula:formula xlink:type="resource" xlink:label="formula1"
value="$amountDue *
my-fn:power((1 + $interestRate), $numYears)"
![Page 42: Formula Status Update New Usage Patterns for FINREP](https://reader035.vdocuments.mx/reader035/viewer/2022081515/568145ef550346895db2f684/html5/thumbnails/42.jpg)
42
Back to FINREP
• Segue to – Use of tree walks to consolidate many formula