PhD Research Project:An Improved Methodology for IS

Planning and Development

Based on IS Strategic Planning and Enterprise Architectural Practice

David WiltonIIMS, Massey University, Albany, NZ

Sequence:

• Introduction/motivation• IS Strategic Planning (ISSP) & Enterprise

Architectural Practice (EAP) – brief intro• Comparison• Outline of research task• Results of survey• Summary/ way ahead

Introduction:

Personal background and motivation:

• IS strategic planning study and practice in early 1990s

• Studied enterprise architecture in 2000-01 in preparation for DSTO Task

• Paper comparing approaches in JBT, presented to DIE seminar, both March 2001

• Initial proposal for doctoral thesis agreed by UNSW, 2001.

• Enrolled Massey 2004.

• Full registration May 2005.

Introduction (contd):

IT represents a significant investment for most organisations (Australian Defence Organisation: multi- $Bn in sunk investments, $1.3B per annum in operating costs)

Key issue: How to manage information and IT assets, and plan investment in an effective, yet affordable, manner?

Introduction (contd):

• ISSP and EAP are two approaches that have emerged – similarities apparent

• Scope of ISSP and EAP depends on individual perception: to meaningfully compare them one must choose specific instances or approaches.

• Theoretical comparison (2001 paper & lit review): – ISSP [CCTA - Central Computer & Telecomms Agency of UK Treasury; now

called Office of Government Commerce (OGC)]

– Enterprise Architectural Practice [ US DoD C4ISR AF (cf Zachman etc)]

IS Strategic Planning - Brief Overview:

CCTA denotes the following key objectives of IS strategic planning:

• understanding the aims and objectives of the business

• establishing the information requirements of the business

• outlining the systems to provide the information, and determining the role of technology in supporting the information systems

• agreeing policies and plans to develop and implement the information systems

• determining the role and use of resources to achieve the information systems required

• managing, reviewing and evolving the strategy

IS Strategic Planning (Contd):

The CCTA process is a sequence of actions, grouped into the common-sense phases of:• Where are we now?

• Where do we want to be?

• How do we get there?

The methodology includes definition of an IS vision, and the presentation of costed options, to realise the vision, to senior management

Methods ISSP ResearchFocus of ISSP

CSF1979

Competitive Advantage1984

IT InfrastructureBroadbent & Weill 1999

IS CapabilityPeppard & Ward 2004

Post-Net Era StrategyGupta et al 2004

BSP1988

Business Alignment1979

CompetitiveForces1980

ValueChain1985

Scenarios1994

BPR1990

CCTA1988

CCTA1999

Info Engin-eering 1989

Boar2001

Methodologies

Methods and Approaches:Galliers (1987)Lederer & Sethi (1988)Flynn & Goleniwska (1993)Earl (1993)Segars & Grover (1999)Min et al (1999)Doherty et al (1999)Levy & Powell (2000)Salmela & Spil (2002)

ISSP Theory and Assessment:Chan & Huff (1992)Lederer & Sethi (1992)Doukidis et al (1996)Lederer & Salmela (1996)Chan et al (1997b)Dufner et al (2002)Newkirk et al (2003)Wang & Tai (2003)

ISSP Success:Galliers (1991)Fitzgerald (1993)Segars & Grover (1998)

Automated Support for ISSP:Wagner (2004)

Proprietary

Sense-and-respond

1999= evolved

= influenced

= incorporates

ISSP for SMEs 2000

ISSP MethodsFocus of ISSP ISSP Methodologies

CSF1979

Competitive Advantagec. 1980

IT InfrastructureBroadbent & Weill 1999

IS CapabilityPeppard & Ward 2004

Post-Net Era StrategyGupta et al 2004

BSP1975

Business Alignmentc. 1975

CompetitiveForces1980

ValueChain1985

Scenarios1991

BPR1990

CCTA1988

CCTA1999

Info Engin-eering 1989

Boar2001

Proprietary

Sense-and-respond

1999ISSP for

SMEs 2000

LinkageAnalysis

1991

Evolution ofTechnology

DP Era1

c. 1960

1980

Micro Era

1990

NetworkEra

2010

Desktop PCs: c. 1982

IEEE/ISO 802.3 (LAN)standard: 1985

CCITT X.25 PacketSwitching (WAN) standard: 1976

Internete-Commerce:c. 1990

CASE tools: c. 1988

OSF DistributedComputingEnvironment standard: 1990

Relational OLTPSdatabases: c. 1980

First commercial hierarchical-model DBMS: 1966

(1) “Eras” defined by Nolan (2000)

m-Commerce: c. 2000

Architectural Practice - Brief Overview:

Architecture: The structure of components, their interrelationships, and the principles and guidelines governing their design and evolution over time (a blueprint of the enterprise & its IT)

Architectures are developed to portray the evolution of an IT environment over various points in time, beginning with the baseline, or as-is architecture.

The architecture envisioned to meet all [future] operational and business requirements is the objective or to-be architecture.

May go through a series of intermediate architectures.

(US DoD, Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance (C4ISR) Architecture Framework, Version 2, 1997)

Enterprise Architecture Framework Evolution:

IS Strategic Planning and Enterprise AP - Similarities:

• Basic Intent/Vision: Both are high-level approaches, intended to realise a rational, affordable IT infrastructure which is consistent with business strategy and goals

• Both include a baseline summary of existing IT infrastructure (“where are we now?”), and an objective architecture (“where do we want to be?”)

• Both establish the information requirements of the business and determine the systems required, to provide and manage the information

• Both produce plans/architectures that are dynamic, and need to be reviewed regularly

• Both can be used by, or adapted to, any type or size of organisation that uses IT.

IS Strategic Planning and Enterprise AP - Differences:

IT Strategic Planning EA Practice Scalability Tends to be targeted at

a single enterprise entity Can be adapted to fit a multi-level or multi-organisation enterprise (intended to produce nested architectures, or “systems of systems”)

Financial base Well defined Stated as a component but not really defined Process Well defined.

Methodical and analytical. Tightly coupled to business strategy and cost effectiveness.

Not defined

Time window for objective architecture/ strategy

3-5 years (limited by rapid advances in IT)

Not specified

Interoperability focus

Not specifically emphasized

Inter - and intra-organisational interoperability is a key focus

Summary of overall approach

Process-oriented Product oriented

Theoretical Conclusions:

• There are distinct similarities in the objectives and scope of both approaches

• The main difference is in the process-orientation of IT strategic planning, compared with the product-orientation of EA practice

• Both have been demonstrated in practice to produce valid results, assisting enterprises to optimise their IT infrastructures

• The two approaches can be viewed as complementary, rather than mutually exclusive, and there could well be significant benefits in combining elements of both, to produce a new paradigm in IT planning and management.

Outline of Research:

Intention is to study the problem space with the view to developing an improved approach, or preferably a practical, usable methodology, for IT planning and development

Bulluss and Chen (2001) suggest incorporation of other related planning and development approaches into EAP; eg systems analysis, systems engineering, software development methodologies

Other IS strategic planning methods, methodologies and architectural frameworks need to be evaluated (eg Zachman, TEAF, TOGAF).

Research Questions:

RQ 1. From both theoretical and empirical views, what is the relationship between ISSP and EAP? (e.g. unrelated, partially overlapping, synonymous)

RQ 2. To what extent are different ISSP and EAP methods used in NZ, how successful are they, and how have the methods used and success levels varied over time?

RQ 3. Are the methods used and success obtained related to organisational factors? (e,g. organisation type, size, level of IT maturity, senior management commitment, allocation of adequate resources).

RQ 4. Can ISSP and EAP methods be combined to produce an improved IS planning methodology?

Research Methodology:• Literature reviews of candidate methods, methodologies and

approaches including (but not limited to) ISSP and EAP. (Addresses RQ 1, from a theoretical viewpoint.)

• A survey of existing ISSP and EAP, to identify usage, success rates, “best-of-breed” methods and tools. (Addresses RQ 1 from an empirical viewpoint, also RQ 2 and 3.)

• Case studies of ISSP and EA in selected organisations, to provide in-depth information on ISSP and EAP, and to explore the feasibility and desirability of an improved methodology. (Addresses research questions 2, 3 and 4.)

• Investigation of the feasibility of an improved methodology for planning and ongoing management of IT resources in an enterprise. (Addresses research question 4.) It is anticipated that this will be accomplished by considering “best-of-breed” methods from ISSP, EAP and other relevant domains discovered during the study. [This approach to development of an improved planning methodology is supported in the literature, eg (Levy et al., 1999, Levy and Powell, 2000)]

• Any proposed improved methodology would require validation – initially intended as an action research phase within the project, but advised at ACIS that that would be too large in scope (extra 1-2 years) for a doctoral thesis – to be conducted as subsequent work

Progress-to-Date:

• Preliminary work in period 2001-2003 (paper published March 2001)

• Provisional enrollment PhD Feb 2004

• Initial literature review completed Aug 2004 (25,000 words)• IIMS postgrad seminar Sept 2004• First draft proposal completed Oct 2004• ACIS Doctoral Consortium Dec 2004 • Full registration May 2005• Survey planning, administration S1-2 2005• Paper presented ACIS 2005 –introduces notion of an IS meta-

strategy (a strategy for doing strategies) which could be a possible outcome from the main project

• Survey results analysed S1 2006

Survey Overview:

First draft survey questionnaire completed Mar 2005 –faculty review, pilot administration (some changes)

Stratified sample (defined in collaboration with Barry McDonald):

50 “large” organisations randomly chosen from NZ MIS Top 100 directory

50 “small” or “medium” organisations randomly chosen from Telecom Yellow Pages

Sub-stratified into industry groups as per survey

[Established contact with three major NZ corporates undertaking ISSP/EAP via NZ Computer Society – using as “real world sanity check”]

Survey administered S2 2005 – poor response rateFollow-up and re-administration Feb-March 2006

Research Model – ISSP and EAP(Adapted from Turban and Aronson, 1998)

Realised Enterprise InfoManagementInfrastructure V15

Key:

Decision variable Fixed variableIntermediate or finaloutcome variable

Certainty UncertaintyRandom (risk) variable

Year V1

ISSP Methods and Methodologies V5

EnterpriseArchitectureFramework(s) V6

OrganisationSize (small, medium, large) V2

OrganisationType V3

IS Maturity V4

(Chan et al,1997b)

(Nolan and Gibson, 1974,Galliers, 1991,Cerpa and Verner, 1998)

(Fitzgerald, 1993,Segars and Grover, 1998,Doherty et al, 1999,Newkirk et al, 2003Wang and Tai, 2003)

IS Strategic Plan V9

EnterpriseArchitecture V10

Resources V13

(Newkirk et al, 2003,Kearns and Lederer, 2004,Premkumar and King, 1994)

(Levy et al, 1999, Premkumar and King, 1994)

(Galliers, 1993,Gupta, 2004)

ManagementCommitmentV14

(Lederer and Sethi,1992, Earl 1993, Premkumarand King, 1994)

(Lederer and Sethi,1992, Earl, 1993)

Success? V7

Initial Hypotheses:H1: V9 is influenced by V1, V2, V3, and V4.

H2: V10 is influenced by V1, V2, V3, and V4.

H3: V7 is influenced by V5, V13 and V14.

H4: V7 is influenced by V6, V13 and V14.

H5: V15 is influenced by V13, V14 and (V9 and/or V10).

H6: V9∩V10 ≠ 0.

H7: V5 and V6 are influenced by V1

- We also need to measure (V7 ∀ V5) and (V7 ∀ V6) to determine “best of breed” methods.

Survey Results• 50 valid responses (another 8 in pipeline?)• Represents a limitation on this study that must be taken into

account when interpreting the results• However:

– < 2000 “large” organisations within NZ (MED, 2005) – a sample of 20 represents >1% of the population

– ISSP research literature: papers published (eg MISQ) with, e.g., samples of 18, 27, 80 organisations

• Another limitation: majority of respondents in large enterprises were IT staff (eg CIO) – over whole sample, majority were business staff

Effect of organisational size on existence of ISSP and EA

LargeSmall or medium (<20)

Small/medium or large organisation

30

25

20

15

10

5

0

Co

un

t

Under development

Yes

NoIS Strategic plan

LargeSmall or medium (<20)

Small/medium or large organisation

30

25

20

15

10

5

0C

ou

nt

Under development

Yes

NoEnterprise architecture

If no ISSP or EA, reasons why not

ISSP and EA (identical):

2. Didn’t consider we needed one.

3. Low benefit/cost ratio.

4. Insufficient management commitment.

5. Other options (rated by respondents as of relatively minor importance).

ISSP & EA Processes:

Maximum Minimum Mean Median "Window" of EA (years) 10.0 2.0 4.2 3.0

Staff effort (person-months)

420.0 0.5 44.0 12.0

Cost of EA ($) $130,000.00 $.00 $29,583.33 $.00

Duration of EA development exercise (weeks)

52.00 5.00 24.58 21.00

EA:

ISSP:

10.0 3.0 4.4 3.0

36.00 2.00 12.03 9.00

$600,000 $0 $75,188 $36,500

52.0 5.0 21.9 22.0

"Window" of plan (years)

Staff effort(person-months)

Cost of plan ($)

Duration of planningexercise (weeks)

Maximum Minimum Mean Median

Assessed “success” of proceses:

3.14 2.80 5.60 1.00Averaged success scoreMean Median Maximum MinimumISSP:

(Likert scale: 1 = Totally successful, 4 = neutral, 7 = totally unsuccessful)

EA:

Mean Median Maximum Minimum Avg. success of EA process 3.40 3.20 5.80 2.00

•Both rated successful (on average)•ISSP slightly higher success rating on all measures

Perceived resource allocation and management commitment

2121Minimum

6666

Maximum

4343Mean

Sufficient resources were available

during EA process

Sufficient resources were available during [IS] planning process

Management commitment

satisfactory during EA process

Management commitment satisfactory during [IS]

planning process

ISSP rated higher for resource levels & management commitment (neutral, on average, for EA)

(Likert scale: 1 = Totally satisfactory, 4 = neutral, 7 = totally unsatisfactory)

Techniques used:

EA:ISSP:

Count % Critical success factors 12 75.0% SWOT 12 75.0% Proprietary technique 8 50.0% In-house technique 6 37.5% Business systems planning 5 31.3%

Net present value or other financial modelling technique

4 25.0%

Business process re-engineering 4 25.0%

Porter's value chain 3 18.8% Scenarios 3 18.8% CCTA strategic planning methodology 2 12.5%

Boar's strategic planning methodology 0%

Info Engineering 0% Other technique 0%

Count % In-house AF or method 9 69.2% The Open Group AF 3 23.1% Zachman AF 2 15.4% US Federal EAF 1 7.7% Proprietary AF or method 1 7.7% US DoD AF 0% US Treasury EAF 0% Other AF 0%

Analysis of techniques used:

Little useful data re usability & success of techniques

Little data pre-Y2000, so no opportunity to study evolution of techniques over time

ISSP: most organisations used a combination of tools or methods (few used a comprehensive methodology)

EA: majority used in-house technique

“Realised” ISSP and/or EA:(“The ISSP/EA has been fully implemented”

ISSP:–no IS strategic plan has been fully implemented, –most organisations (73.3%) are in the mid-range between “mildly agree” and “mildly disagree”. –two organisations (13.3%) strongly disagree, indicating little or no progress towards implementing their plan.

EA:–only 18.2% of organisations have shown any level of agreement to the statement reflecting successful implementation of their EA (and then only “mild” agreement). –i.e. 81.8% of organisations have made little or no progress towards implementing their EA.

Reasons for incomplete realisation:1. An incremental (phased) approach has been

adopted but we expect to get there eventually.

2. Lack of management commitment.

3. Lack of funding.

4. Lack of stakeholder commitment or acceptance of the need.

Hypothesis TestingThe following hypotheses have been demonstrated :

H1b: V9 (the existence of an IS strategic plan) is influenced by V2 (organisational size).

H2b: V10 (the existence of an EA) is influenced by V2 (organisational size).

H3b: V7 (ISSP success) is influenced by V13 (resource allocation).

H3c: V7 (ISSP success) is influenced by V14 (management commitment).

H4b: V7 (EA success) is influenced by V13 (resource allocation).

H4c: V7 (EA success) is influenced by V14 (management commitment).

H5a: V15 (realisation of enterprise information management infrastructure) is influenced by V7 (“successful” ISSP).

H6: V9∩V10 ≠ 0 (there is a significant overlap between the scope of ISSP and EA).

The following hypotheses have not been demonstrated:H1a: V9 (the existence of an IS strategic plan) is influenced by V1

(age of organisation)

H1c: V9 is influenced byV3 (organisation type).

H1d: V9 is influenced by V4 (level of IS maturity).

H2a: V10 (the existence of an EA) is influenced by V1 (age of organisation).

H2c: V10 is influenced byV3 (organisation type).

H2d: V10 is influenced by V4 (level of IS maturity).

H3a: V7 (ISSP success) is influenced by V5 (methodology used).

H4a: V7 (EA success) is influenced by V6 (framework used).

H5b: V15 (realised IS infrastructure) is influenced by V7 (“successful” EA).

Not tested: H7:V5 and V6 are influenced by V1 (the choice of ISSP and/or EA techniques are influenced by the year the exercise was undertaken)

Relationship between ISSP and EAP

•H6: V9∩V10 ≠ 0 (there is a significant overlap between the scope of ISSP and of EA).•Objectives:

Key objectives - ISSP Key objectives - EA

1. Align IT with business needs. 1. Align IT with business needs.

2. Forecast IT requirements. 2. Establish technology path and policies.

3. Gain senior management commitment.

3. Forecast IT requirements.

4. Establish technology path and policies.

4. Gain senior management commitment.

5. Seek competitive advantage from IT.

5. Seek competitive advantage from IT.

6. Revamp the IT function. 6. Revamp the IT function.

7. Other reasons 7. Other reasons

(Identical, except “establish technology path & policies” promoted in EA)

Key findings:

There is a strong overlap between the objectives & scope of ISSP and EAP in NZ organisations (within the limitations of the survey)- Organisations may be wasting time & resources developing both

(management commitment lower towards EA)- Advantages in combining “best-of-breed approaches to create a

comprehensive methodology?

NZ SMEs have a very low incidence of ISSP and/or EA (~20%)– SMEs may not be taking advantage of the opportunities that IT affords

– Simplified “DIY” methodology?

Summary:

Low incidence of ISSP &/or EA in SMEs (in NZ)

IS strategic planning could be combined with EA Practice to create a new paradigm for planning and management of IT

My proposed research task is to realize that paradigm per medium of a methodology which combines the best of all relevant approaches and methods

Questions/comments?

Additional Terminology:

Method: an individual technique used to achieve some purpose, eg cost-benefit analysis, NPV

Methodology: an integrated collection of methods (to form some sort of process), eg CCTA IS strategic planning

Approach: a high-level, abstract description of a means of solving a problem, eg Rockart’s Critical Success Factors (CSF) approach to IS strategic planning