beat liver c-aise-2013_v1-0(final)
TRANSCRIPT
Integrity in Very Large Information
Systems
Dealing with Information Risk Black Swans
Public, Presentation for CAiSE 2013
June 21, 2013
Beat Liver and Helmut Kaufmann
About Credit Suisse
One of the world’s leading financial services providers
Offers to clients its expertise in
– Private Banking
– Investment Banking
– Asset Management
Operates in over 50 countries
– Around 550 locations 46,900 employees
June 21, 2013 Beat Liver and Helmut Kaufmann 2/19
Agenda
Introduction − Business-critical failures − What is common? How to identify and prevent such failures? − Why is this challenging?
Information risk − Integrity risk vs. integrity criticality − Levels of integrity criticality
Integrity controls − Minimum bar integrity design standards
Integrity controls enhancements − Minimum bar standards limitations − Independent controls (Proof-of-Concept)
Experience
Conclusions
June 21, 2013 Beat Liver and Helmut Kaufmann
Risk Controls
Rating Drivers
3/19
Business-critical Failures
A trading software bug
− generated wrong market orders
− resulting in a loss of 440 million USD within
30 minutes
After a software change
− a payment order processing batch failed.
− Around 7 million account holders were
impacted.
− Sorting out and restoring operations took
several weeks
A trader inadvertently entered
− an order to sell 610'000 shares at 16 Yen a
piece instead of 16 shares at 610'000 Yen.
− resulting in a loss of up to 100 million USD
Source: Risks Digest (see paper)
June 21, 2013 Beat Liver and Helmut Kaufmann
Risk Controls
Rating Drivers
4/19
Source WikiMedia, Uwe Kils and Wiska Bodo under Creative Commons license.
What is Common? How to identify and prevent business-critical integrity failures?
Integrity failure – incorrect data processing
− Correct modifications
Business expectation
− Authorized modifications
Integrity understanding mostly used
Business-critical impact
− Huge financial loss
− Enterprise at risk (sometimes)
Black Swans characteristics
− Unexpected events
− Rationalized in hindsight
− Hard to foresee
June 21, 2013 Beat Liver and Helmut Kaufmann
Risk Controls
Rating Drivers
5/19
Why is integrity challenging? Very Large Information Systems in the Financial Services Industry
Size − Such as, for instance, more than
6’000 Applications (e.g., red dot)
100’000’000 lines of code
10’000 employees
− globally distributed
Complexity − Multiple business lines and entities − Functional dependences (e.g., blue lines)
− Evolving requirements 24/7, low-latency and volume
Regulation (Basle III)
− Evolving technology
− Economic factors Value-chain / vertical integration Resource constraints
− Technical debts
Tailor-made IT systems − Custom components
June 21, 2013 Beat Liver and Helmut Kaufmann
Application landscape domain model with an Foreign Exchange
(FX) client order application with selected up- and downstream dependencies (i.e., data flow).
Risk Controls
Rating Drivers
6/19
Integrity Risk vs. Integrity Criticality
Risk equation assumptions
− Statistical basis
− History
− Number of instances
Airplanes, cars, etc.
but, very large information systems
− Unique,
− Diverse − Rapidly evolving
Risk assessment implications
− Can you assess the probability?
− Can you assess the impact?
June 21, 2013 Beat Liver and Helmut Kaufmann
Which scenario is business-critical?
a) 10 erroneous payments over 100 Million CHF each to banks
b) 10 million erroneous payments over 100 CHF each to individuals on accounts with other banks
Risks in above examples
• In (a), bank’s return money but a counterparty might default
• In (b), a recovery is possible but it costs too much
Risk Controls
Rating Drivers
Probability Impact [CHF] Risk [P x I] Criticality
0.01 1’000.00 10 low
0.001 1’000’000.00 1’000 medium
0.000001 1’000’000’000.00 1’000 high
7/19
Medium vs. High Integrity-Criticality
Probability (unsuitable parameter)
− Rare events
Impact defines integrity-criticality
− Negative black swans (concave losses)
− Possible Losses
Recoverable (I-2, normal critical)
– Cap on sum of residual possible loss
and recovery costs
Irrecoverable (I-1, business critical)
– Cap on absorbable possible loss
given
Business Controls
Assets at Risk
– Business objects
– Populations
June 21, 2013 Beat Liver and Helmut Kaufmann
See also [Results from the 2008 Loss Data Collection Exercise, Bank for International Settlements (BIS), July 2009,Table ILD6 - Distribution of Loss Amount by Severity of Loss
Risk Controls
Rating Drivers
Rare Events
Possible Loss
0.00
0.10
0.20
0.30
0.40
0.50
0.60
I-2
I-1
Likelihood
-6000.00
-5000.00
-4000.00
-3000.00
-2000.00
-1000.00
0.00
8/19
Integrity Design Standards Design to ensure that critical data is correctly modified
Audience
− Solution architects
− Application owners
Scope
− Individual applications rated as normal and business critical
differentiation in development, testing
and operation
In comparison
− to industry standards
ISO/IEC 17799:2005 practice guide
− our standards are
Concrete and specific*
– Standards infrastructure
– Compliance criteria
Coherent and complete**
Technology agnostic
June 21, 2013 Beat Liver and Helmut Kaufmann
Integrity Design Standard Summary Data aspect, where critical data
− Must be identified and defined **
− Must be uniquely identifiable, golden-sourced via services**
− Sole identifiers in user interfaces must support validation *
Processing aspects, where critical processing must
− Log critical steps using standard infrastructure*
− Perform a timely reconciliation for exchanged critical data
− Use patterns of the standard consistency model*
− Use idempotent operations, services and
batches** Validation aspects, where applications must on critical data
− Automatically validate the input/output plausibility
− Use second validation according to the four-eye principle
− Specify in service contracts authoritatively-validations**
− Resolve plausibility exceptions by sign-off,
degraded modes of operation or failure* Recovery aspects, where application must implement
− Use backup procedure supporting a timely recovery
− Idempotent and restart-able recovery procedures ensuring timely recovery including a integrity validation
Risk Controls
Rating Drivers
Similar to IT Auditing and Controls - A look at Application Controls, Kenneth Magee (InfoSec Resources)
9/19
Integrity Controls Enhancements Are the controls effective and efficient?
Integrity controls
− Controls limitations Devil is in the details (post mortems) – No safety critical-systems methods
Costs due to criticality propagation
− 2nd version independent controls Abstraction – Critical data attributes only – Approximations are sufficient – Process boundaries only
Independent
Application landscape
− Order business processes External process boundaries – Source (of external commitment) – Interface (to outside)
Account booking External payment
Internal process boundaries – Aggregation (e.g., position keeping)
− Audit trail (design standard)
Integrity Controls 2nd Controls
I/O Validations
Application Landscape
Interface
Source
Order
Management
Settlement
Messaging
Gateway
Payment
Audit Trail
Logging
Infrastructure
Controls Risk Controls
Rating Drivers
June 21, 2013 Beat Liver and Helmut Kaufmann 10/19
Application Landscape
Independent Controls Proof-of-Concept
Modeling
− Application Finite State Machines (FSM) Business objects life-cycle state
Business rules define transition conditions
− Communications among FSMs
Business rules define conditions
− What are the benefits? Abstraction for a class of systems
Reusability and modularity
Automata facilitates criticality rating
Validations Engine
− Big Data analytics tool
− Modular correlation rule sets
FSM with its business rules
− Tracking life-cycle state in data base Views based on deadlines
Reduce log retention duration
I/O Validation
Interface
Source
Order
Management
Settlemen
t
Messaging
Gateway
Payment
Audit Trail
Logging
Infrastructure
Communicating FSM Business Rules
Controls Risk Controls
Rating Drivers
June 21, 2013 Beat Liver and Helmut Kaufmann 11/19
Experience – Lessons Learned
Business/IT Alignment
+ Understanding (non-functional vs. functional)
+ Business controls and IT systems
Rating and minimum bar standards
+ Clearer directives and narrower discretion
- In-depth interdisciplinary understanding necessary
- Challenging institutionalization (comfort zone)
Independent controls Proof-of-Concept
+ Audit trails are suitable, but
- Heterogeneous format, representations, etc.
- Correlation identifiers segmented and directional
+ Standards infrastructure suitable with
moderate response time requirements (~ 10 s)
without automatic intervention (integrity gate) o But, a reliance on independent controls is undesirable!
- Manual modeling costly and brittle – the killer criteria
Large number of business rules
Frequent modifications across the landscape
Deliverables
Information Risk Assessment
Methodology
Minimal Bar Design
Standards
Minimum Bar Standards compliance assessment
of around 200 applications world-wide
Independent Controls Proof-of-Concept with standard
infrastructure and production audit trails
Controls Risk Controls
Rating Drivers
June 21, 2013 Beat Liver and Helmut Kaufmann 12/19
Possible Loss
I-1
Conclusions In ship fleet operations, watch icebergs. In banking IT, keep an eye on integrity.
Very-large banking information systems
− What is business-critical?
Do business and IT mean the same?
− What is integrity?
Authorized vs. correct modifications
− How to rate the integrity?
What is the impact?
What are the business controls?
How to mange dependencies?
− What integrity controls are necessary?
How to reduce the effort and increase the
assurance level?
Are independent controls an option?
Outlook
− Institutionalization and revision
− Research independent controls
Independent
Risk Controls
Rating Drivers
June 21, 2013 Beat Liver and Helmut Kaufmann 13/19
Source WikiMedia, U. Kils and W. Bodo under Creative Commons license.
Appendix
June 21, 2013 Beat Liver and Helmut Kaufmann 14/19
Integrity Criticality Rating
1) Information assets in scope − Financial perspective
− Compliance perspective
− IT Risk Management perspective
2) Assets at Risk (financial values) − Population of data objects
− Small and large attribute-value errors
3) Business controls (using other apps) − Possible financial losses
Control check-points (time bounds)
− Recoverability
Capped residual loss + recovery costs
4) Application Business and IT criticality rating (alignment of understanding)
5) Manage decencies using services − Services offered/required integrity-criticality
− Consume services meeting integrity-criticality
− Application sub-systems differentiation
June 21, 2013 Beat Liver and Helmut Kaufmann
Drivers
Financial
Compliance Information Risk
Confidentiality
Availability
Integrity
Operation Scope
Modifications
"Correct"
Authorized
Data Scope
IT Risk Mgmt
Compliance
Financial
Application, Service, ...
Business vs. IT Rating
Criticality Rating (Risk Assessment)
Integrity-criticality
Assets at Risk
Recoverabilty
Business Controls
Application, Service, ...
Business vs. IT Rating
Risk Controls
Rating Drivers
15/19
Risk-Adjusted Services In a SOA, direct dependencies are sufficient
Service functionality
− Data service
EVENT publisher
Read-only ACTION
− Data Processing service
EVENT consuming demon
Write ACTION Note: EVENT/ACTION refer to semantics and not the transport!
Service integrity-criticality rating
− Determined by application sub-system
Adequate service consumption − Service rating equal (or higher) to
consumer's
− Compensations
Service-based dependency management
June 21, 2013 Beat Liver and Helmut Kaufmann
getMarketData
updatePosition
FX Order
Management
CRUD Order
createSettlement
Risk Controls
Rating Drivers
16/19
Order Example
Foreign Exchange Spot
Joe Smith buys 1'000'000.00 USD against CHF
at a spot exchange rate of 0.9401 USD/CHF
on 2013-04-05 07:45 UTC
Business object with critical attributes − Order Type: FX Spot
− State {new, modified, canceled, matured}: new
− Counterparty: Joe Smith
− Traded Amount: 1'000'0000.00 USD
− Exchange rate: 0.9401 USD/CHF
− Trade date: 2013-04-05 07:45 UTC
FX Order Management Application
− Generates quotes given market data
− Order capture, modification and cancellation
Create, Read, Update, Delta SOA Service
Sends order life-cycle events down-stream
− Settlement application
− Position management application
June 21, 2013 Beat Liver and Helmut Kaufmann
FX Order
Management
FX Position Keeping
Market Data
FX Hedging
FX
Settlement
Payments
Messaging
Gateway
Risk Controls
Rating Drivers
17/19
Rating Example
Foreign Exchange Spot Joe Smith buys 1'000'000.00 USD against CHF at a spot exchange rate of 0.9401 USD/CHF on 2013-04-05 07:45 UTC
Business object with critical attributes − Order Type: FX Spot − State {new, modified, canceled, matured}: new − Counterparty: Joe Smith − Traded Amount: 1'000'0000.00 USD − Exchange rate: 0.9401 USD/CHF − Trade date: 2013-04-05 07:45 UTC
Asset at Risk: Spot Order population − 1'000 new orders over 1'000'000 € per day − 1 pip markup, i.e., 100'000 € markup per day
Data error scenarios - small vs. a few large − Mispricing exchange rate − Duplicate/ missing orders − Duplicate cash settlement payments
Business Controls − Are they detective and corrective? − Are possible losses recoverable?
June 21, 2013 Beat Liver and Helmut Kaufmann
Business Control
FX Order
Management
population
FX Position Keeping
Market Data
FX Hedging
CRUD Order
Volume
Profit/Loss
FX
Settlement
Payments
Messaging
Gateway
What is required from
consumed service?
Risk Controls
Rating Drivers
18/19
Disclaimer
This document was produced for information purposes and is for the exclusive use of the recipient. No guarantee is made regarding reliability or completeness of this document, nor will any liability be accepted for losses that may arise from its use. This document may not be distributed in the United States or given to any US persons (within the meaning of Regulation S under the US Securities Act of 1933, as amended). The same applies in any other jurisdiction except where compliant with the applicable laws. Copyright © 2013 Credit Suisse Group AG and/or its affiliated companies. All rights reserved.
June 21, 2013 Beat Liver and Helmut Kaufmann 19/19