controlling excel chaos

Controlling Excel ChaosWith Business Intelligence

A White Paper by Diane Sklar

Executive Summary

The Excel Challenge

Spreadsheet Chaos

Manual Labor Begets Errors

On the Spreadsheet Audit Trail

WebFOCUS Excel Sourcebooks

Setting up WebFOCUS Excel Sourcebooks

There Is a Solution

A Spreadsheet in Every Mailbox

The Corporate Excel Sourcebook

Deliver Advanced Functionality

Answering the Ad Hoc Question

Conclusion

1

2

2

2

2

4

13

4

4

5

5

6

11

Table of Contents

Business analysts develop Excel spreadsheets to assist with the day-to-day operational

decisions their jobs demand. Pleased with the autonomy and sophisticated analysis that Excel

supports, they share these innovations with colleagues who modify spreadsheet logic and

manually tack on data from their own information silos. Over time, rogue spreadsheets with data

from dubious sources is propogated throughout the organization. Executives find themselves

making decisions based on untraceable, questionable data. The enterprise is at a loss to audit

these numbers for themselves and for regulatory agencies.

Meanwhile, the IT division has complete, auditable, and backed-up operational system data that

is untapped by the Excel user community. IT executives are frustrated by the rampant spread of

unreliable information and their inability to leverage corporate data stores.

Information Builders has both push and pull strategies to serve up corporate data for use in

Excel spreadsheets. These strategies buttress Excel’s weaknesses as an enterprise business

analysis tool while allowing users to continue to operate in their preferred analysis environment.

1 Information Builders

Executive Summary

2 Controlling Excel Chaos With Business Intelligence

Spreadsheet Chaos

Spreadsheets were the standard model for doing financial analysis long before enterprises

automated accounting and finance functions. When Microsoft launched Excel, it mimicked the

spreadsheet layout and made the business analysts’ transition from paper to computer a

seamless one. Moreover, the easy-to-use, point-and-click interface running on the Windows

platform gave Excel a market-share victory over competitors like Lotus and VisiCalc.

Manual Labor Begets Errors

Excel was designed for the individual analyst working alone on a computer with a limited amount

of data. The original Excel model had a business analyst typing a small subset of corporate data

into a spreadsheet, cell by cell, and then manually manipulating that data to solve a business

problem. But manual data entry is an expensive, time-consuming misuse of the business

analyst’s time and error prone as well. Cell-by-cell data entry leads to simple mechanical errors

such as mistyping a number or pointing to the wrong cell. Graver errors can also occur when the

analyst makes errors in formulas because of bad keystrokes or bad logic.

When data is available in an existing spreadsheet, it can be cut and pasted via the clipboard or

merged via Excel functionality. This works best when the source and target ranges are formatted

identically. If they are not identical, data and formulas can inadvertently be corrupted.

Alternatively, ranges of data from an external Excel workbook can be referenced in place. The

effectiveness of this technique is dependent on the source file remaining unchanged.

Maintenance to spreadsheets chained in this way can be horrendous when there is no

centralized change management policy.

Ideally, data from corporate stores should be transferable directly to Excel spreadsheets, thereby

insuring that all business analysts start their spreadsheet ventures with the one true version of

enterprise data. But Microsoft’s vision of interoperability has remained focused on Excel and

other Microsoft desktop products. Outside connections must be established through ODBC or

DDE connections. Beyond these options, one must transform data to the text-based CSV format

and stage it on the Windows platform.

Figure 1: Manual spreadsheet generation is a breeding ground for errors.

The Excel Challenge

CorporateData

Manual Data/Formula Entry

ReportOutput

StyledAggregatedSpreadsheet


On the Spreadsheet Audit Trail

For many companies Excel has grown beyond a small-scale analysis tool. It is instead the main

vehicle for financial analysis in mission-critical departments, pushing Excel functionality beyond its

original intent. Excel has no built-in mechanism for separating the presentation, calculation, analy-

sis, and storage layers of an application. The onus of solid spreadsheet design is still left to the

spreadsheet developer. Of course, skill levels in Excel and in programming methodology vary widely

among analysts. In non-IT departments untrained spreadsheet programmers can apply resource-

intensive calculations at the wrong level, creating a performance bottleneck. Rigorous testing with

sufficiently varied data samples is rarely done at the departmental level. These are just some

examples of possible fallout from not applying IT methodology to the spreadsheet community.

Accurate and traceable financial information is critical to making sound business decisions that

move the enterprise forward. Auditable information is required outside the organization as well.

The 2002 Sarbanes-Oxley Act requires disclosure in annual reports of internal controls over

financial reporting. Spreadsheet errors can lead to expensive and embarrassing problems. The

European Spreadsheet Risk Interest Group analyzes and quantifies the cost of spreadsheet

errors worldwide. In the last six months alone, they have reported various situations, including:

■ A public auditor’s office in Minneapolis mistakenly reported the percentage change in unreserved

fund balances; an analyst set up the formula for the column in the spreadsheet, dividing the

difference between 2003 and 2004 balances by the 2004 total, instead of the 2003 figure

■ A Housing and Urban Development (HUD) audit revealed that a local housing authority had to

pay over $200,000 to cover expenses incurred when the authority overpaid some landlords

due to a data-entry error

■ In Nevada, a 2006 municipal budget was developed from a copy of the city’s 2005 budgeting

spreadsheet; in late 2005, a problem was revealed causing a $5 million deficit in the water

and sewer fund; while fixing the problem with the water and sewer budget, other errors were

uncovered and fixed

■ A well-known medical and consumer imaging company had to amend its third-quarter loss by

$9 million, announcing that the adjustment was needed because too many zeros were added

to an employee’s accrued severance on a spreadsheet; the company’s CFO characterized the

situation as “an internal control deficiency”

True robust error protection and auditing for Excel is generally left to third-party products. The

flourishing market of add-on tools bears witness to Excel’s weakness in this area. Auditing with

add-on tools becomes another training and standardization dilemma. Once again the enterprise

is dependent on the skill, prudence, and discipline of the individual analyst to properly deploy

audit tools.

Despite these problems, usage of Excel is not diminishing. It is still the undisputed giant among

business analysis tools. It can be found on every desktop in the enterprise. Its cost per seat is

negligible, keeping it the first choice for on-going enterprise application needs. How can the

organization protect itself from the rampant spread of uncontrolled and unsanctioned numbers

in Excel spreadsheets?


Setting Up WebFOCUS Excel Sourcebooks

Since Excel usage shows no sign of waning, IT’s best approach is to eliminate the resource-

intensive, error-prone data entry process in traditional Excel deployment. In most enterprises, IT

has already invested resources in creating a protected and scrubbed warehouse of summarized

financial results. When necessary, IT can also provide links to extract relevant transaction-level data.

A comprehensive study of all Excel user requirements can feed the design of a set of master

Excel workbooks supplying content to all Excel users. Workbook structure should be flexible,

allowing users to exploit any subset of workbooks, worksheets, or cell ranges. Ideally, data

should be delivered to these master workbooks already summarized and styled. It should also

be formatted for the specific version of Excel that users have on their desktops. Complex

formulas should be pre-set to avoid errors. Excel structures that are difficult to develop, such as

graphs and pivot tables, should be delivered analysis-ready.

Users will have different data latency requirements as well. Some workbooks may need to be

refreshed quarterly, other workbooks may require a monthly, weekly, daily or hourly feed.

There Is a Solution

WebFOCUS, Information Builders’ business intelligence suite of products, generates Excel

spreadsheets across all generations of Excel. This includes XML-based formats with

comprehensive Excel functionality as well as binary formats that optimize transfer speed.

All WebFOCUS Excel spreadsheets have their data summarized and formatted as a complete

spreadsheet on the WebFOCUS Reporting Server. The Reporting Server, typically located on the

same platform as the target data, summarizes data as per the request, minimizing the volume of

data transmitted and formatted on the desktop. By the time the spreadsheet arrives on the

user’s desktop it is ready to go, without wait-time for additional formatting.

Delivery of corporate spreadsheet data to Excel users can be accomplished by several

automated distribution strategies. For each distribution scenario, WebFOCUS ReportCaster

schedules jobs and extracts data from known, auditable corporate sources. It makes data

available at any interval, from every n years to every n minutes. The resulting extract can be

formatted in several ways:

■ As Excel spreadsheets, which are pushed to a distribution list of Excel users via e-mail

■ As a Corporate Excel Sourcebook(s) on a central server – users connect to the corporate

Excel sourcebook and pull relevant data into their own Excel workbooks by referencing

named ranges of cells

■ As database extracts on a central server with a parameter-driven user interface; users supply

parameter values that determine all content and stylistic characteristics for a target spread-

sheet to answer an ad hoc question

WebFOCUS Excel Sourcebooks


A Spreadsheet in Every Mailbox

When spreadsheet contents and format are uniform across a large population of users, a

ReportCaster job can e-mail Excel spreadsheets to a distribution list of user addresses. The job

is programmed once and tested rigorously by IT professional staff familiar with the corporate

files. The job is run once and output is collected for all interested parties.

E-mails with spreadsheet attachments are created for all entries on the distribution list. The

distribution list can be developed manually or derived from Microsoft Active Directory or an

LDAP directory. Each recipient on the distribution list can receive the entire report. Or, if the

report contents are private, the report can be burst so that recipients receive content only for the

sort group(s) relevant to them. For example, a company-wide profit and loss statement can be

burst by cost center so that managers receive the statements only for their own cost centers.

Figure 2: Bursting Excel spreadsheets saves system resources by running a job once for all users, but ensuresprivacy by distributing only the appropriate sections to each user.

The Corporate Excel Sourcebook

Sometimes spreadsheet content and format are highly customized and frequently change. In

such an environment, a suitable strategy is to provide foundation spreadsheet data in a generic

workbook or Corporate Excel Sourcebook. This workbook standardizes detail, summary, and

calculated amounts. It is created and stored on a universally accessible server by a ReportCaster

job that refreshes it as often as necessary.

Organization of WebFOCUS Corporate Excel Sourcebooks is extremely flexible. Data can be

structured on a single worksheet or multiple worksheets. If there are multiple worksheets, all

sheets can have identical format but different ranges of data. Using the profit and loss statement

example, a workbook could be organized with one sheet per cost center.

Corporate Directory(LDAP, Active Directory)

Global ExcelSpreadsheet

France

Spain

UK

Brazil

SpreadsheetDistribution

Schedule

E-mail

Another multiple worksheet solution is to consolidate unrelated reports in a single workbook.

For example, one worksheet might contain inventory reports, the next sheet might contain

human resources reports, and so on. No connection between the worksheets is necessary.

To populate personal spreadsheets, users connect to the central server and reference the central

workbook in their local spreadsheets. References can pluck data from the entire workbook,

worksheet, or from a range of cells within a report. To facilitate selective data gathering, the

Corporate Excel Sourcebook can be designed with named cell ranges. Named ranges are put in

place when the WebFOCUS Excel job is set up. They shield users from needing to change their

personal spreadsheets when report size and position change in the Corporate Excel Source-

book. Named ranges can be set up for a sheet, report, column, row, or any rectangle of cells.

With Corporate Excel Sourcebooks, users each take exactly the data they want, no more and no

less. They can then style and manipulate that data in any way they please and the audit trail to

controlled corporate data is preserved.

Figure 3: A Corporate Excel Sourcebook contains corporate Excel data that all enterprise users can link to withtheir own spreadsheets.

Deliver Advanced Functionality

It is worth discussing the scope and variety of pre-formatted Excel features supported by

WebFOCUS. Maximum automated data manipulation means minimum chance for human error.


Employees

Purchasing Range

Orders

Customers

Sales

Inventory

123456789

A B C D E F

123456789

A B C D E F

Corporate Data

Payroll Sheet

Insurance Cost Range


WebFOCUS supports all Excel functionality, either directly or through the use of spreadsheet

templates. The specific mix of Excel capabilities pre-set by WebFOCUS extract and distribution

jobs is limited only by the standard versions of Excel in use. All features of every Excel release

are supported. Following is a summary of stylistic and analytic features available.

Summary LinesWebFOCUS generates report headings and footings, page headings and footings, and recap

lines on sort breaks, all of which can have a combination of text and numeric totals. Worksheet

and workbook names can also be automatically generated.

Styling and Conditional StylingWebFOCUS controls the font family, size, color, and style for each and every element of a

spreadsheet. Summary lines, column titles, column content, etc. can be styled separately.

Conditional styling is also supported so that data meeting defined business criteria is

highlighted by style or color.

Drill DownHypertext links to other spreadsheets, images, or HTML pages can be automatically embedded in

distributed Excel output. Links can be positioned in summary lines or within the data of the report.

Formulas and Calculation LogicComplex formulas can be implemented in the WebFOCUS extract job and passed to Excel. The

benefit is that these complex calculations are written and tested centrally by IT professionals

familiar with corporate data sources and with structured programming practices. The WebFOCUS

self-documenting code becomes the audit trail for Sarbanes Oxley and other compliance regulations.

Even relatively simple calculations such as growth rate and profitability can be incorrect when

factors are reversed. Consider the multiplicity of factors in some calculations typically imple-

mented in spreadsheets. In analyzing pay grades for example, human resources departments

often rate performance based on manager performance evaluation, absenteeism, years of

service, current salary within pay grade, and corporate guidelines for pay increases. If these

formulas are incorrectly calculated, legal issues could arise.

Insurance companies might base profitability projections for corporate health insurance

contracts on rates charged, historic dollar amount of claims, employee population age, and

numbers of dependents. All factors must be weighted and if a new factor is introduced, all

weights must be re-calculated.


Calculating the cost of goods sold and the value of inventory on hand can also involve an array

of factors. Inventory valuations typically involve a range of prices and costs for raw material,

labor, shipping, etc. at different points in time, periodic production volumes, and LIFO or FIFO

assumptions about the rotation of goods. A spreadsheet capturing these rules would quickly

become a dangerous breeding ground for error. The potential for error is dramatically reduced

when such calculations are done by a single, thoroughly tested and auditable module with

access to the correct data sources.

Figure 4: A typical WebFOCUS Excel spreadsheet has predefined calculations, styling, and drill-downs.

PivotTablesPivotTables make it possible to reveal trends in a data sample by sorting, filtering, adding key

elements, and charting. Despite a built-in Excel PivotTable Wizard, there are pitfalls to

developing PivotTables. While most report consumers manipulate PivotTables effectively, far

fewer users can readily produce sophisticated, styled PivotTables.

Pre-calculated formulas

Formatted headings and footings

Conditional styling when salesare greater than $100k

Drill down to other reports,spreadsheets, or graphs


With WebFOCUS, PivotTables can be developed by IT professionals and built into the extract job.

WebFOCUS PivotTable Developers work in a graphical user interface, building the initial view of

the table as well as the lists of cached and page control fields.

Figure 5: Most Excel users can manipulate PivotTables, but far fewer can design them.

TemplatesFor all spreadsheet functionality other than the features mentioned above, WebFOCUS can imple-

ment Excel functionality by leveraging templates. Additional functions might include print preview

functions or graphing. Undoubtedly, spreadsheets exist that represent a great deal of effort spent

in development of macros. If these macros have been tested and certified, they can be leveraged

in place without re-writing. They can be applied to WebFOCUS formatted data by use of templates.

Templates are skeleton workbooks containing any Excel functionality. During spreadsheet

creation, the WebFOCUS Reporting Server locates the template, merges data into a designated

sheet, and saves the results to a separate workbook file. The final spreadsheet has corporate

data inserted into a pre-existing user approved format. Templates not only leverage all existing


functionality of Excel, they also position WebFOCUS to leverage future functionality that

Microsoft will add to Excel.

Figure 6: WebFOCUS merges an Excel layout (template) with corporate data to provide the exact formattingand data manipulation required.

Figure 7: An Excel template before and after it is populated with data by WebFOCUS.

12%14%

5%

30%25%

CorporateData Excel

Templates

WebFOCUSPopulated

Spreadsheets

x x x x x xx x x x x xx x x x x xx x x x x xx x x x x xx x x x x xx x x

xxxxxxx xxxxxxxxxxxx xxxxxxxxxxxx xx

xxxxxxx xxxx


Answering the Ad Hoc Question

There will always be a need for new spreadsheets to address the latest business challenge. Late-

breaking situations that require new subsets of data can be tackled with an interactive ad hoc

reporting system. Such an application has a series of parameter selection screens that front-end

appropriate corporate data stores. All characteristics of a target Excel spreadsheet are defined

via the parameter selection screens. Which database fields and calculated fields are included,

what are the summarization rules of the report, the sort criteria, the formatting, and which

cascading stylesheet – these are all criteria picked by users from menus. The resulting output is

always a spreadsheet in the user’s preferred Excel format.

Figure 8: Answering the ad hoc question is handled with a nightly parameterized application that formatscustomized spreadsheet output.

Inventory

Sales

Orders

Corporate Data

A

B

C

▼

▼

▼

D

E

F

▼

▼

▼A

B

C

▼

▼

▼

D

E

F

▼

▼

▼A

B

C

▼

▼

▼

D

E

F

▼

▼

▼

Forms

Parameters:Sort

FilterAggregation

Styling


Figure 9: Detail of a parameterized form and the report it generates.

For example, the insurance company that historically computed health insurance rates based on

historic claims, population age, and number of dependents could add salary to the rate

calculation. Analysts want to model the effect of the new calculation on profitability of the

contract. Analysts can multi-select salary, along with the other rate calculation factors and derive

a new spreadsheet with factors including individual or aggregated salaries. This spreadsheet

could then be manipulated off-line in various what-if scenarios but the foundation data is

enterprise-certified.


Enterprises can take control of the burgeoning number of unsanctioned spreadsheets by

adopting a policy of centralized master spreadsheet generation. Excel is a powerful and

ubiquitous analytical tool that can make a business analyst self-sufficient, but it lacks strong

audit and error-checking capabilities. However, master spreadsheets drawn from the

company’s primary operational and warehoused data sources can provide an audit trail back

to elemental transactions.

WebFOCUS creates fully summarized, organized, and styled Excel spreadsheets with complex

formulas already in place, lowering the potential for keystroke and logic error. WebFOCUS-

generated spreadsheets can be pushed out to business analysts in e-mail or staged on a central

server where analysts fetch target data by hitching their own linked spreadsheets to the master

corporate spreadsheet. Furthermore, on the corporate Excel form, analysts can screen, sort, and

summarize samples of data to derive spreadsheet answers to ad hoc questions. With a little

analysis and the right WebFOCUS business intelligence tools, you can finally have a corporate

Excel spreadsheet strategy that works.

Conclusion

North AmericaUnited States■ Atlanta,* GA (770) 395-9913

■ Baltimore, MD Consulting: (703) 247-5565

■ Boston,* MA (781) 224-7660

■ Channels (800) 969-4636

■ Charlotte,* NC Consulting: (704) 494-2680

■ Chicago,* IL (630) 971-6700

■ Cincinnati,* OH (513) 891-2338

■ Cleveland,* OH (216) 520-1333

■ Dallas,* TX (972) 490-1300

■ Denver,* CO (303) 770-4440

■ Detroit,* MI (248) 743-3030

■ Federal Systems,* DC (703) 276-9006

■ Hartford, CT (860) 249-7229

■ Houston,* TX (713) 952-4800

■ Los Angeles,* CA (310) 615-0735

■ Metropolitan,* NY Sales: (212) 736-7928

Consulting: (212) 736-4433, ext. 4443

■ Minneapolis,* MN (651) 602-9100

■ New Jersey* (973) 593-0022

■ Orlando,* FL (407) 804-8000

■ Philadelphia,* PA (610) 940-0790

■ Pittsburgh, PA (412) 494-9699

■ St. Louis,* MO (636) 519-1411

■ San Jose,* CA (408) 453-7600

■ Seattle,* WA (206) 624-9055

■ Washington,* DC Sales: (703) 276-9006

Consulting: (703) 247-5565

CanadaInformation Builders (Canada) Inc.

■ Calgary (403) 538-5415

■ Ottawa (613) 233-0865

■ Montreal* (514) 334-0404

■ Toronto* (416) 364-2760

■ Vancouver* (604) 688-2499

MexicoInformation Builders Mexico

■ Mexico City 52-55-5062-0660

AustraliaInformation Builders Pty. Ltd.

■ Melbourne 61-3-9631-7900

■ Sydney 61-2-8223-0600

Europe■ Belgium Information Builders Belgium

Brussels 32-2-7430240

■ France Information Builders France S.A.

Paris 33-14-507-6600

■ Germany Information Builders (Deutschland)

Dusseldorf 49-211-522877-0

Eschborn 49-6196-77576-0

Munich 49-89-35489-0

Stuttgart 49-711-7287288-0

■ Netherlands Information Builders

(Netherlands) B.V.

Amsterdam 31-20-4563333

■ Portugal Information Builders Portugal

Lisbon 351-217-230-720

■ Spain Information Builders Iberica S.A.

Barcelona 34-93-344-32-70

Bilbao 34-94-425-72-24

Madrid 34-91-710-22-75

■ Switzerland Information Builders

Switzerland AG

Dietlikon 41-44-839-49-49

■ United Kingdom Information Builders

(UK) Ltd.

London 44-845-658-8484

Representatives■ Austria Raiffeisen Informatik Consulting GmbH

Vienna 43-12-1136-3870

■ Brazil InfoBuild Brazil Ltda.

São Paulo 55-11-3285-1050

■ China InfoBuild China, Inc.

Shanghai 86-21-50805432

■ Finland InfoBuild Oy

Espoo 358-207-580-843

■ Greece Applied Science

Athens 30-210-699-8225

■ Guatemala IDS de Centroamerica

Guatemala City 502-2361-0506

■ Gulf States Nesma Advanced Technologies

■ Bahrain ■ Kuwait ■ Oman

■ Qatar ■ Yemen ■ United Arab Emirates

Riyadh 96-1-465-6767

■ India Divas Offshore Software

Technologies Private Limited

Gurgaon 91-124-501-8801

■ Israel NESS A.T. Ltd.

Tel Aviv 972-3-5483638

■ Italy Selesta G C Applications S.P.A.

Genova 39-010-64201-224

Milan 39-02-2515181

Torino 39-011-5513-211

■ Japan K.K. Ashisuto

Osaka 81-6-6373-7113

Tokyo 81-3-5276-5863

■ Malaysia Elite Software Technology Sdn Bhd

Kuala Lumpur 60-3-21165682

■ Norway InfoBuild Norway

Oslo 47-23-10-02-80

■ Philippines

Beacon Frontline Solutions, Inc.

63-2-750-1972

■ Saudi Arabia Nesma Advanced Technology Co.

Riyadh 996-1-4656767

■ Singapore

Automatic Identification Technology Ltd.

65-6286-2922

■ South Africa Fujitsu Services (Pty.) Ltd.

Johannesburg 27-11-2335911

■ South Korea Unitech Infocom Co. Ltd.

Seoul 82-2-2026-3100

■ Sweden Cybernetics Business Solutions AB

Solna 46-7539900

■ Taiwan Galaxy Software Services

Taipei 886-2-2586-7890

■ Thailand Datapro Computer Systems Co. Ltd.

Bangkok 662-679-1927, ext. 200

■ Turkey Istanbul

Veripark 90-212-283-9123**

■ Venezuela InfoServices Consulting

Caracas 58-212-763-1653

Toll-Free Numbers■ Sales and Information

(800) 969-INFO

■ ISV, VAR, and SI Partner Information

(800) 969-4636

**Training facilities are located at these branches;additional locations are available.

**Authorized to sell iWay Software only.

Corporate Headquarters Two Penn Plaza, New York, NY 10121-2898 (212) 736-4433 Fax (212) 967-6406 DN4601444.0706

www.informationbuilders.com [email protected]

Canadian Headquarters 150 York St., Suite 1000, Toronto, ON M5H 3S5 (416) 364-2760 Fax (416) 364-6552 For International Inquiries +1(212) 736-4433

Copyright © 2006 by Information Builders, Inc. All rights reserved. [51]

All products and product names mentioned in this publication are trademarks or registered trademarks of their respective companies. Printed in the U.S.A.on recycled paper

Sales and Consulting Offices

controlling excel chaos

Technology

excel spreadsheets

errors excel

excel chaos

excel functionality

companies excel

corporate excel sourcebook

ranges of data

questionable data