management case study building a strategy map to drive … · management case study building a...

Management Case Study

Building a Strategy Map to Drive

Performance Accountability and

Improvement: The Case of Audit Scotland

For more information please visit: www.ap-institute.com

http://www.ap-institute.com/

© 2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com

Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study

Building a Strategy Map to Drive Performance Accountability and


By

Bernard Marr* and James Creelman

Abstract: This case study describes the best-in-class implementation of a

performance management framework in form of a strategy map with supporting Key

Performance Questions and Key Performance Indicators. It shows how Audit

Scotland, the nation‟s independent auditor of public bodies, was able to distill the

long stakeholder wish-list down to a simple and focused strategy map. Outlining the

strategic priorities on just one page using simple language has completely

transformed the way Audit Scotland is managing its performance to ensure external

accountability and internal performance improvements.

Version: 18th May 2010 * corresponding author

Bernard Marr is the Chief Executive and Director of Research at the Advanced Performance Institute. E-mail:

[email protected]

James Creelman is a Fellow of the Advanced Performance Institute

The Advanced Performance Institute (API) is a world-leading independent research and advisory

organisation specialising in organisational performance. It provides expert knowledge, research, consulting

and training to performance orientated companies, governments and not-for-profit organisations across the

globe. For more reading material or information on how the API might be able to help your organisation please

visit: www.ap-institute.com

How to reference this case study:

Marr, B. and Creelman, J. (2009) Building a Strategy Map to Drive Performance Accountability and

Improvement: The Case of Audit Scotland, Management Case Study, The Advanced Performance Institute

(www.ap-institute.com).

mailto:[email protected]

http://www.ap-institute.com/



Building a Strategy Map to Drive Performance Accountability and


Introduction

Having amassed a wealth of useful data and

information regarding stakeholder priorities over the

years, Audit Scotland faced the not uncommon

challenge of distilling this down into something that

was focused and manageable.

To achieve this, Audit Scotland crafted a strategy

map that comprises just two core activities and nine

supporting performance enablers. Each of these,

which are collectively aligned to the organization‟s

vision, is supported by Key Performance Questions

(KPQs) and Key Performance Indicators (KPIs) –

which together provide rich insights into how the

organization is progressing toward its strategic

goals.

The strategy map is designed to help Audit Scotland

make step-change improvements in performance

and to provide a more streamlined and focused

corporate plan.

An Impact framework is also being used to assess

and report the longer term benefits of Audit

Scotland‟s work, which will serve to provide a more

complete view of the organization‟s performance.

About Audit Scotland

Headquartered in Edinburgh, Audit Scotland was

created in 2000 to provide the Auditor General and

the Accounts Commission with the services that

they need to check that public money is spent

properly, efficiently and effectively. With about 300

full-time equivalent employees and an operating

budget of £27.5 million, the organization arranges

the audits of about 200 public bodies, such as the

Scottish Government, 32 councils and 44 joint

boards and committees (including police and fire,

health boards and rescue services). Importantly,

Audit Scotland is independent of the bodies that it

audits.

Three types of audit

Audit Scotland conducts three types of audits:

1. An annual audit provides an independent

opinion of the public bodies‟ financial

arrangements and governance practices.

2. Best value audits provide an independent

opinion of an organization‟s outcomes focus as

well as the effectiveness of its corporate

performance management and use of

resources.

3. Performance audits provide an independent

assessment of key areas of parliamentary or

public interest, often spanning traditional sector

boundaries. Performance audit reports include

“Scotland Public Finances,” „The role of Boards,‟

„Efficient Justice‟ and „Public Sector Pension

Schemes‟.

Together, these three audit types provide a detailed

and systematic evaluation of the efficiency and



effectiveness of how the nation‟s public

organizations work and are being managed.

Identifying Stakeholder Requirements

As well as helping to improve how others work,

Audit Scotland is firmly committed to continuously

improving its own performance. As part of this

ongoing process, during 2008 the organization

conducted consultation with a broad range of

stakeholders, including staff, partners and clients, in

order to better understand how to meet their needs

going forward. “We knew that the business and

operating environment of our stakeholders was

rapidly changing,” says Diane McGiffen, director of

corporate service. “And we wanted to ensure that

we captured these changing circumstances in our

corporate plan for 2009-2012.”

Using an external research company Audit Scotland

generated substantial quantitative and qualitative

information as to how its various stakeholder groups

thought it was performing and what they thought its

priorities should be over the following five years.

“We also extended this exercise to our staff which

gave excellent additional insights into how we

should develop and improve,” says McGiffen,

whose wide ranging remit includes being in charge

of business planning and performance reporting and

heading up human resources and training,

information technology, finance, administration and

communications.

The data challenge

Although such widespread consultation provided a

wealth of valuable data, Audit Scotland found itself

in a position that is common across both public and

commercial organizations. “The more data and

information that was generated from surveys,

discussions and other formats, the more challenging

it became to identify and focus on the critical few

things that will really make a difference to our

performance,” comments McGiffen. “We knew we

had very good information, but we were having

difficulty in distilling it into something that was

focused and manageable.”

A Powerful Corporate Vision

To support this distillation, Audit Scotland engaged

the services of the Advanced Performance Institute

(API). During the winter of 2008, API worked with

Audit Scotland to create a corporate strategy map

(shown on Figure 1) which identified the key

strategic objectives and supporting activities that

would be required to deliver to the following vision:

“On behalf of the Auditor General and the Accounts

Commission, we will provide assurance to the

people of Scotland that their money is spent

appropriately and we will help public sector

organizations in Scotland to improve and perform

better.”

This vision speaks directly to the efficiency (value

for money) and effectiveness (performance

improvement) strands of Audit Scotland‟s

responsibilities. As with any useful vision, this

became the anchor of the subsequent internal

discussions that focused on crafting the strategy

map. However, it should be noted that in the

process of creating the strategy map the vision was

slightly amended and updated. “This was a useful

outcome of the workshops conducted to create the

map,” says McGiffen. “The vision we have now

captures the essence of the priorities that our

stakeholders and clients have identified for the next

five years.”



Stakeholder priorities

The priorities identified by the stakeholders groups,

which have therefore become the key priorities of

Audit Scotland, are:

• Deliver more streamlined audits in partnership

with other scrutiny bodies.

• Maximize our contribution to the improvement of

public services.

• Increase the impact of our work.

• Become a centre of excellence for public audit.

• Improve the transparency of our costs and

governance arrangements.

The priorities are based on the following issues that

stakeholders are grappling with, that emerged

through the consultation process:

• increasing pressure on public spending

• Increasing emphasis on more efficient working

Figure 1: Audit Scotland‟s Corporate Strategy Map (source: Audit Scotland Corporate Plan 2009-12)



• The continuing need for high standards of

corporate governance, financial management

and control

• The need to deliver more efficient and effective

scrutiny and accountability.

Creating the Corporate Strategy Map

The first step in creating the strategy map that

would deliver to the vision and stakeholder priorities

involved API conducting focused and individual

interviews with each member of the executive

management team and the assistant directors as

well as the organization‟s chair John Baillie and

another board member.

A core benefit of one-to-one interviews is that it

enabled the external facilitator to gain a clear

picture as to how individual senior managers viewed

the drivers of organizational strategic success. Any

differences were debated within the workshops

conducted to create the strategy map, with the

ultimate goal of reaching consensual agreement as

to the critical few drivers of success.

Strategy mapping workshops

Audit Scotland‟s management group created a draft

strategy map in two dedicated workshops. A key

role of the external facilitator was to ensure the

selection of truly strategic objectives, rather than

goals that are essentially operational.

A further role was to facilitate the creation of cross-

organizational objectives instead of ones that could

be siloed – that are of relevance and interest to just

one part of the organization. Audit Scotland‟s

management team wanted to ensure that the

strategy map would galvanize the whole of the

organization in unison toward step change

performance improvement.

In-house consultation

Wide-ranging staff engagement was seen as crucial

for securing buy-in to the strategy map concept.

“Through focus groups and discussions with small

groups of staff we gained feedback on what they

thought of the map, what they thought were its

strengths and weaknesses and where they saw

areas for improvement,” recalls McGiffen. “This

showed us how closely the management team‟s

views were aligned with those of their colleagues

and the staff feedback was used to inform the final

map.”

According to McGiffen, the clarity and simplicity of

the map helped most staff to understand and talk

about what it means to them “Within the group

discussions it became evident that colleagues could

see how it all fits together, what Audit Scotland is

trying to achieve, how it will be achieved and what

they must do in their own jobs to make it happen.”

This, she adds, was largely due to the map‟s

simplicity and accessibility.

Ensuring the map is simple and focused

As an illustration of simplicity, the map comprises

just eleven elements: two core activities (which

Audit Scotland calls objectives) and nine

performance enablers (which Audit Scotland calls

supporting activities). A prime driver of formulating

the strategy map was to distil and make

manageable a mass of data and information, Audit

Scotland rightly realized that there would be little

benefit in creating an elaborate and complex map.

Indeed the experience of other companies shows



that complicated maps do not provide sufficient

focus onto the key drivers of success and rarely

survive over the longer term due to being too

unwieldy and unmanageable – a fate Audit Scotland

was determined to avoid.

Although simple, the map is far from simplistic.

Rather, each objective and supporting activity

serves as a doorway to a host of performance

management and measurement focal points –

primarily captured through Key Performance

Questions (KPQs) and Key Performance Indicators

(KPIs) - that work together to provide an overall

picture of how well Audit Scotland is performing

and, ultimately, to steer the organization toward the

delivery of its vision and priorities (For more

information on KPQs and KPIs see the API white

paper „What are KPQs‟ and the API management

overview „Key Performance Indicators‟1).

Although KPIs are commonly used within

organizations, KPQs are substantially less so. So

as an explanation, a KPQ focuses on and highlights

what the organization needs to know in terms of

executing existing strategic objectives. KPQs

enable a full and focused discussion on how well

the organization is delivering to these objectives

and serve as an important bridge between

organizational goals and KPIs. Indeed KPQs are

used to provide a performance context to KPIs and

help to more effectively prioritize the indicators

chosen, and this was certainly true of Audit

Scotland.

1 http://www.ap-institute.com/resources.htm

“People in most organizations, and we are no

exception, have quite rightly developed strong

disciplines in KPIs as a way to monitor and improve

performance,” says McGiffen. “KPQs do not replace

KPIs but are an enhancement. KPQs help us to

capture a much richer set of information than cannot

be gained solely from performance data.”

Today, KPQs are used to discuss performance at

many organizational levels within Audit Scotland

and Audit Scotland is refining and developing its

collection and reporting of data to address the

KPQs. A key initiative was a Performance

Management Festival held over two days to bring

focus and attention from across the business to the

performance reports.

We will now explore how the objectives/activities on

the map, KPQs and KPIs work together with Audit

Scotland‟s strategy management framework.

Two strategic objectives

As cited, Audit Scotland‟s strategy map includes just

two core activities (called objectives in their case),

these being: “we will conduct excellent risk based

audits of the public sector and report them in

public,” and “we will systematically identify and

promote good practice to help public bodies to

improve.”

Although both objectives are equally important, a

key message that emerged from the extensive

stakeholder consultation was that they were looking

to Audit Scotland to do more to help them to

improve performance. The KPQs to support the

„improve‟ objective (and note that KPQs and indeed

KPIs are still work in progress within Audit Scotland,

while they test which are the most powerful

http://www.ap-institute.com/resources.htm



questions and indicators to focus on) presently

include: „to what extent has our work led to

improvement‟ and „to what extent are we identifying

and actively promoting good practice (or better

ways of doing things) to help public bodies improve.

Note that the KPQs focus on both „lagging‟ (that is

how we have performed) and „leading‟ (how we will

improve performance) dimensions, enabling a fuller

overview of past, present and likely future

performance.

KPIs for these KPQs include „impact reports‟ (see

later in this case study for details of Audit Scotland‟s

impact framework); „client feedback‟; „number of

presentation at meetings, events and conferences

which include best practice guidance,‟ and

„performance audit reports with good practice

recommendations‟ (see Figure 2 for examples).

Supporting Activities

How Audit Scotland achieves its objectives is

largely through the delivery of enabling objectives

(called supporting activities in their case). In

creating the map, the senior team strived to ensure

that there was a balance of focus on what matters

to stakeholders, what matters to staff and improving

some of the key business support processes to

support that work.

Stakeholder-facing performance improvement

For a stakeholder-facing improvement, consider the

activity: „We will closely engage and communicate

with our key stakeholders, clients and partners, and

other scrutiny bodies.‟

Audit Scotland recognizes that it is only through

close engagement and communication that it can

effectively identify appropriate topics for its audit

work. Closely engaging and partnering with other

scrutiny bodies and improvement agencies also

helps to avoid duplication of effort and maximizes

Audit Scotland‟s contribution to the improvement of

public services.

To support the improvement objective it is also

recognized that it must clearly communicate its

findings and recommendations to ensure that its

messages are heard and its recommendations are

followed up.

KPQs used for this activity are „to what extent are

we keeping our stakeholders informed about our

programmes of work and their role/implications‟ and

„how well do we engage the people we work with.‟

KPIs include „positive stakeholder feedback‟

(engagement levels, clarity about and acceptance of

roles), „website statistics (downloads, click

throughs) and the „number of shared risk

assessment frameworks in place.‟

Internal process improvement

Step-change internal business process

improvements are largely captured by the

supporting activity: „We will deliver our work and

manage our resources efficiently and effectively and

will be clear where we need to improve. We will

have excellent governance procedures and will

monitor and report on the impact of our work.‟

This activity sets out to ensure that Audit Scotland

has strong internal governance processes in place

and that it is deploying best practice public

management principles and procedures. To this

end, Audit Scotland carries out a programme of



internal best value reviews to improve its efficiency

and effectiveness and manage its risks.

The KPQs for this activity include „to what extent

have we effectively and efficiently managed our

resources to deliver our outputs‟ and „how effective

are our internal governance arrangements for

delivering our objectives.‟ KPIs include: „percentage

of audit reports delivered on-time and to budget,‟

„reducing our carbon footprint by one per cent,‟

„percentage of invoices paid within 30 days and

„internal best value reports published.‟

Staff performance improvement

Moving onto improving staff performance this is

captured by several supporting activities, including

„We will maintain and develop our professional skills

and competencies‟.

As with any high-performing organization, Audit

Figure 2: Sample KPIs and Evidence of Performance (source: Audit Scotland Corporate Plan 2009-12)



Scotland recognizes that it can only deliver high-

quality work if its staff have the right skills and

competencies to do their job well. To achieve this,

the organization strives to embrace an ethos of

continuous improvement and continues to invest in

the development its people. The organization has

recently revised its competency framework and has

embedded this into its Performance Development

System. Regular individual performance reviews are

used to assess and follow-up on training and

development needs.

The KPQs for this activity include „to what extent are

our employees engaged and motivated‟ and „to

what extent do we have strong and effective

leadership in the organization‟. KPIs include „staff

survey results,‟ „staff focus group feedback‟ and the

development of „360 degree feedback (through

which employees are appraised by a range of

people, including their managers, colleagues and

direct reports)‟.

Knowledge management

A particularly interesting supporting activity, in that it

clearly demonstrates how Audit Scotland is working

to improve its own internal capabilities in order to

help its stakeholders improve is „We will gather,

organize and share knowledge and intelligence.‟

The organization recognizes that the basis of its

work is having the right knowledge and intelligence.

Therefore it is working diligently to ensure that the

right information is systematically gathered, that it is

organized in the most appropriate way, and shared

among staff and externally with stakeholders.

KPQs for this activity are „to what extent do we hold

the information that we need‟ ' to what extent are we

sharing our key findings internally‟, and „to what

extent are we able to find and share the information

that we find‟. KPIs include „focus group feedback‟,

„intelligence reports‟, „publication of technical

bulletins‟, „number of briefing reports produced and

shared internally‟.

Creating Strategy Maps for Specific Activities

Audit Scotland is presently reviewing and

developing how it gathers, organizes and shares

knowledge and intelligence. This is leading to some

interesting applications of the strategy map concept,

as McGiffen explains. “Our strategy map identifies

the key support activities where we need to focus

attention and commit resources,” she says. “What

we are doing now is creating strategy maps for

specific areas where we want to prioritize major

performance improvements, such as information

and knowledge management and shared learning.”

She continues: “Our stakeholders are increasingly

looking to Audit Scotland to share knowledge and

information quickly from the full range of our work

and advanced knowledge sharing and collaborative

capabilities. However, our technologies and

methodologies don‟t enable us to do what we need

to do to meet stakeholder requirements. We have

started to invest in this area and a strategy map

should prove a powerful mechanism for prioritizing

our investments against identified stakeholder

needs.” Environmental performance is another area

for which a dedicated strategy map will be created.

Performance Reporting

With the strategy map now the core tool for driving

performance improvement, Audit Scotland ensures

that it is closely managed on a day-to-day basis,

and this is the responsibility of McGiffen and a small



team from corporate reporting that is skilled in data

analysis. The team reports performance to the

management team and board on a quarterly basis,

who particularly look to see how objectives and

activities are improving and to focus on areas that

are proving challenging and where corrective action

might be required.

From a reporting standpoint, the strategy map and

related KPIs have also enabled McGiffen and her

team to cut the corporate plan down from a lengthy

document to something concise and focused and

that is more accessible to external stakeholders and

internal staff respectively.

Performance Management Festival

One innovative approach to engage people in

performance reporting was to stage a „Performance

Management Festival‟. In late 2009 Audit Scotland

invited key staff for a two-day festival of

performance. The aim was to bring together KPI

owners, performance analysts, and directors and

board members to agree performance reporting

formats. Using an off-site location with music, guest

speakers and performances this innovative event,

facilitated by the API, was designed to make the

topic of measurement and performance reporting

interesting, engaging and fun.

During the two-day event staff re-designed

performance reports using the latest techniques of

information communication and data visualization.

Within these reports, headlines and narratives

(essentially a performance story) were used to

capture the key messages, using ideas and

graphic/textual formatting from magazines and

newspapers. These new performance reports were

then „trialed‟ and improved during a staged mock

performance review meeting with the directors.

Impact Framework

Performance improvement is also being galvanized

through the deployment of an impact framework to

assess and report on the longer term impact of

Audit Scotland‟s work - in particular its performance

audits - and to capture evidence of that impact. The

organization has identified four areas where it

expects its work to have an impact and contribute to

improvements:

• Assurance and accountability

• Planning and management

• Economy and efficiency

• Effectiveness and quality.

The impact framework describes the types of

measurable change expected to result from Audit

Scotland‟s work within the different categories of

impact and includes examples of how individual

reports may contribute to these changes.

The impact framework is expected to dovetail well

with the strategy map in providing a complete view

of how Audit Scotland is performing now and is

likely to perform in the future.

Conclusion

In conclusion, McGiffen summarizes the role that

the strategy map has played in improving the

performance of Audit Scotland. “From the beginning

we knew that we wanted our Corporate Plan to be

simpler and more relevant and meaningful for

colleagues.” she says, “From an external

perspective we also knew that as an organization



we wanted a framework that would help our

stakeholders understand what we do and what we

have to be best at.”

“The strategy map is helping us to meet our external

and internal goals,” she continues. “But in the final

analysis, it is perhaps most important that the map

has better positioned Audit Scotland to be able to

demonstrate to stakeholders and clients that we are

doing what we promised that we would do.”

Endnotes, References & Further Reading

Marr, Bernard (2010), “The Intelligent Company: Five Steps to Success with Evidence-Based

Management”, Wiley, Oxford.

Marr, Bernard (2009), Managing and Delivering Performance: How Government, Public Sector

and Not-for-profit Organizations can Measure and Manage what Really Matters, Butterworth-

Heinemann, Oxford.

Marr, Bernard (2006), “Strategic Performance Management”, Butterworth-Heinemann, Oxford

www.ap-institute.com For more case studies, reports and articles visit www.ap-institute.com

“Working with the Advanced Performance Institute to develop a performance framework in form of a

strategic map, KPQs and KPIs has exceeded my expectations. It has helped us clarify our strategic

objectives quickly and easily in a way we couldn‟t have done on our own. The strategy map has

completely transformed the way we now communicate about our goals and deliver our strategy and it

has been an extremely powerful tool to engage everybody in managing and delivering performance.”

Diane McGiffen, Director of Corporate Services, Audit Scotland

committed to investing in the future of itself and the country by building on the best

management techniques they can find. Their programme is as good as anything we

have seen.”

Dr David Norton – Co-creator of the Balanced Scorecard



Highlights from the API resources library:

Our Resource and Research Library offers a selection of relevant downloads and links to books, articles and

case studies. These have been selected as useful information sources for further reading and to illustrate best

practice and leading thinking.

Some key ideas explained in simple terms:

What is Performance Management?

What is the Balanced Scorecard?

What is a Key Performance Indicator?

To read more just click: http://www.ap-institute.com/resources.htm

http://www.ap-institute.com/Performance%20Management.html

http://www.ap-institute.com/Balanced%20Scorecard.html

http://www.ap-institute.com/Key%20Performance%20Indicators.html

http://www.ap-institute.com/resources.htm

management case study building a strategy map to drive … · management case study building a...

Documents