Enterprise Information Management and Business Intelligence Initiatives at the Federal Reserve

XXXIV Meeting on Central Bank Systematization

7-9 September 2011

Kenneth Buckley

Associate Director

Division of Reserve Bank Operations and Payment Systems

Agenda

• Background and challenges

• Enterprise Information Management (EIM)

responses and strategies

• EIM at the Federal Reserve Bank of New

York (FRB NY)

• Business Intelligence initiatives

• General outlook and next steps

2

Background and challenges - The Federal Reserve System is comprised of 12 geographically dispersed banks

3

Regionally Located Business Lines:

•Research Departments at all Banks and the Board

•Markets Group at FRB New York

Centrally operated -

national mainframe, server,

network data centers

Regional IT Operations:

•Application

development

•Local server operations

•Local IS operationsTo varying degrees, these separate organizations all

had largely separate data efforts and supporting

infrastructure in place or under consideration

• The Federal Reserve collects and uses a wide variety of information from internal and external

sources to support its central bank mission to:

evaluate the condition of financial institutions;

forecast economic conditions

execute monetary policy; and

operate domestic payment and US Treasury services.

• Data supports monitoring of market risks

banking failures

business cycles & market conditions

liquidity and credit constraints

• In the mid-1990s, significant changes occurred in financial markets

products and institutions became more complex

markets consolidated and cross-border economies became interdependent

repeal of the Glass-Steagall Act removed separation between investment and depository banks

• 2008 Financial crisis

broad investor distrust of virtually all forms of private credit - especially complex credit products

largely unforecasted by standard market indicators

increased risk from decline in asset prices and sharp and sudden intensification and rapid global

spread

international pressure on central banks to act quickly

Background and challenges - Structural changes in market and banking environments

4

Background and challenges - Gaps in data and knowledge management tools

• Central bank response

provision of short-term liquidity to banks and other financial institutions

currency swap agreements with other central banks to assist in providing liquidity in their jurisdictions

• Operational challenges resulting from the financial crisis

expanding the balance sheet to account for new asset classes from new lending facilities

understanding complex financial instruments

managing the level of lending demand and complex collateral instruments

need for enhanced risk analytics

diverse information sources for asset valuation and risk management

business processes requiring external partners

• Problems with data management in the Federal Reserve became more visible:

data sources and repositories were disparately organized and managed

Individuals could create, acquire, and store data without much standardization

ownership blurred and silos emerged leading to constraints on sharing and controlling data

data not managed consistently and overall quality suffered or was hard to determine – often no “single

point of truth”

inadequate and inconsistent enterprise guidelines, policies, governance, and controls

Many efforts local in scope and focus, resulting in duplication of data, limited availability, and lack of

clear responsibility /ownership for data integrity, consistency, quality, availability

lots of data available, but that did not commensurately improve intelligence or insight

5

Data

Background and challenges – inability to meet the service demands from knowledge workers

Information KnowledgeActions & Outcomes

Integrate Access Report Analyze Understand Decide Act Produce Measure

Learn

Ability to Create Business Intelligence

Ability to leverage Information Assets

6

EIM responses and strategies

1. Create and evolve Federal Reserve

Enterprise Information Management

(FedEIM) framework

• Comprehensive data management strategy

comprised of integrated data management

disciplines

• Led by centralized Enterprise Architecture

function

• Primary objectives:

1. Improve operational efficiency, promote

transparency, and (most importantly)

help enable better business insight and

decision making

2. Be responsive to changing regulatory

priorities, business conditions, and

industry directions

• Need to structure, describe, access, and

govern data as assets across functional

areas, regardless of organizational or

technical boundaries

• Establish a business view of the technology

framework

7

EIM responses and strategies

2. Proceed with data governance and organizational alignment efforts

• Evolve organizational thinking and approach – effective data management is an

organizational discipline, and not just a technical issue

• Establishing roles, responsibilities, and organizational structures that can adapt to changing

conditions

3. Rationalize and coordinate data procurement and tools acquisition

• Processes and organization to support:

efficient / shared acquisition of data

data management tools

• Ways to better identify and manage user demands

8

EIM responses and strategies

• General strategy is to rely on and foster a combined regional and local approach

evolve over time to more appropriate mix

realize benefits and economies of national approach without losing value of allowing local

flexibility and innovation

• Numerous regional efforts proceeding across Federal Reserve System today under the

guidance of national FedEIM framework

• Business functions at FRB New York and Board of Governors are two of largest

“consumers” of data today, and both have taken significant steps to advance data

management capabilities and coordination

• FRB NY’s efforts currently focused on

establishing a formal Data Management Office (DMO);

enhancing and institutionalizing standard processes and related governance;

improving flexibility and responsiveness to changing business needs;

participating (with rest of FRS) in efforts to rationalize tools and data sources

9

EIM at the FRB NY - Strategy

• Situational Analysis

assessed the state of data assets and corporate data management in Markets function to

determine future structures needed to support strategic data management needs

concluded that a dedicated organization with a strong governance structure was needed to

execute the future state plan and support ongoing operations

• Established the Data Management Office (DMO) as the “organizational glue” bonding

business, operations, and technology

drive the „Data Management strategy‟ (long-term while supporting day-to-day deliverables)

establish data governance; identify data ownership; engage data stewardship

establish “authoritative sources” of analytics, reference and market data

define policies to govern data acquisition, storage and access

• Key tactical DMO objectives

10

improve collaboration between Market

Operations, Bank Supervision, and

Economic Research functions

rationalize relationships with market data

vendors and service providers

collaborate with Markets Technology to

implement the necessary applications

provide ongoing support for data

acquisition, data storage and data delivery

systems

EIM at the FRB NY - DMO Functional Organization Structure

11

• The DMO is a BUSINESS function liaising between “the consumers” and the “suppliers”

• The DMO is made up of Market Data subject matter experts, Data Management Business Analysts, Project Managers and QA specialists dedicated to the delivery of data management solutions

Consumers

Data Management Office (DMO)

Data Providers

Services

DMO• Market Data Specialists

• Data Management BAs

• Project Managers

• QA Specialists

• Market Data Vendor

Admin Specialists

(contracts)

RM’s

Domestic Markets

Treasury Markets

Domestic MM & Reserve

Disc Window & Collateral

Market Analysis

Foreign Exchange

Facilities Market Data Terminal Providers

Internal sourcing

Market Data Feed Providers

Markets Business Technology

Automation Group Technology

Automation Group QA

Market Data Service Providers

Business driven

Technology enabled

Operations supported

• Acquisition of

“golden” copy

data from

external sources

• Acquisition of

“golden copy”

data from

INTERNAL

operations

• Control and

governance over

purchases

• Require “Data

Review” for all

new systems to

ensure proper

use of shared

repositories

• Extract and

transform data

from internal and

external sources

that fit business

needs

• Understanding

of system data

flows

• Perform quality

checks,

exception

handling and

data remediation

• Identify existing

“authoritative

sources” of data

• Define future

data repositories

• Define the

strategic

repository

infrastructure

• Determine data

ownership -

stewardship

• Implement

proactive

processes to

maintain quality,

timeliness and

completeness

data (Data

Quality)

• Provide

Centralized

“Customer

Service” to

consuming

systems

• Define Data

Standards (Data

Models, Data

Dictionary,

Metadata, etc.)

• Define the

methods for

quick and

organized data

access (i.e.:

Portal; Excel; BI

tools; SAS;

Access)

• Partner with

consuming

applications to

ensure data is

“fit for purpose”

• Ensure ease of

access to data

(provide end-to-

end solutions)

Data Creation

& AcquisitionProcessing

Data

“Factories”Maintenance

Data

DistributionConsumption

Governance

Data Management is a multi-faceted function composed of a series of “component disciplines” that track to the Data Management Lifecycle

EIM at the FRB NY - DMO Scope

12

EIM at the FRB NY– Data/Tools Acquisition

Rationalizing and coordinating data procurement and tools acquisition

• Established a cross-bank committee with representatives from Markets, Supervision,

Research, and others to monitor and track all market data purchases

• Developed and now maintain a complete inventory of all purchased data into an

industry standard market data procurement tool (FITS)

• Worked with market data vendors to strategically identify and acquire data,

collaborating across functions and with other Reserve Banks and the Board of

Governors

• Participating in enterprise-wide efforts to pilot and standardize data management and

business intelligence tools (e.g., Informatica and FITS)

13

Business intelligence initiatives

• As a result of its experience during the financial crisis, the Federal Reserve is committed to

improving the responsiveness and flexibility of its BI tools for analysis and decision making.

• Experience with BI tools manipulating large data sets has yielded the following lessons learned:

traditional software products do not maximize the performance of the hardware

traditional methods for statistical and computational analysis of large datasets can be expensive and

challenging

transferring large amounts of data from one location to another can be slow

issues related to managing large data are analytical and not transactional (parallel vs. serial)

large data issues are a hardware/software problem

• Numerous BI initiatives underway today are evaluating practical applications of several different

tool sets, including,

GreenPlum DW appliance for the RADAR platform (provides large DB consumer loan analytical capabilities

using massively parallel processing technology)

TIBCO Spotfire, intended as a general purpose and provides powerful self service tools for data access and

manipulation, reporting and analysis, and collaboration

14

BI initiatives - RADAR

RADAR (Risk Assessment, Data Analysis, & Research)

• RADAR is a data warehouse (as well as separate securities evaluation service) used by Bank

Supervision as well as Research functions

broad array of US consumer credit data - credit cards, auto loans, student loans, mortgages

currently over 9 TB and 350 million records

• Generate and view data by various parameters, maps, charts Geography, various demographics, time periods, or loan status.

Options for customizing the views of data, including the ability to develop maps or charts

• Developed by staff in Philadelphia and Kansas City Reserve Banks and provides both preset and

ad-hoc query capabilities

• Platform is high performance clustered Linux on Apache/Tomcat web servers, coupled with

Greenplum database and 15 commercial/open source software analysis tools (SAS, Matlab,

C/C++, etc.), as well as various in-house developed query and reporting tools

• While the data warehouse is mainly used for bank surveillance purposes, it has also proven useful

in the Fed's community development initiatives

15

BI initiatives - RADAR

16

Mortgage

Delinquencies

by County: 90+

Past Due,

Foreclosure, All

50 States,

201009

BI initiatives – National Cash Data Warehouse

• Provides ability to aggregate and view depository institution and Reserve Bank

currency activity

• The NCDW, along with the associated data analysis and reporting tool, provides

important functionality to the Cash business to:

monitor and enforce certain business rules,

support billing for customer activity that exceeds rule-based parameters, and

support analyses that serve as a loss control function for the business.

• Migrating to a Microsoft SQL 2008 platform with Business Objects reporting tools

17

BI initiatives – National Cash Data Warehouse

18

General outlook and next steps

• EIM and BI will continue to be high priority objectives in our IT and business line

strategies. These objectives will influence other strategic initiatives such as

collaboration, mobility, and analytic tools for knowledge workers.

• Tactical next steps include:

building on successes and applying lessons learned, continuing to support locally driven DM

efforts, and evolving the enterprise wide programs that fit needs and realities

further improve analytic capabilities

creating standard directory-based interfaces for analysts using tools like Spotfire that

can free them from concern with physical DB and technologies used

more complex and larger volumes of data

further defining policies and standards for data governance, and supporting their broad

implementation and acceptance as a fundamental (and typical) aspect of how we conduct

our central bank activities

rationalizing toolsets and technologies and, by applying standard reference architectures,

transition over time to platforms that optimize performance and reduce operating costs

current initial efforts underway include moving to consolidated operating environments

for Informatica

establishing enterprise wide standard taxonomies or metadata that cross current

organizational and/or business area boundaries

improving specialized skill sets associated with data roles and BI technology that can reduce

time to market for delivering solutions

19

Questions

20