using performance measurement framework in portfolio management… · 2016-08-11 · using...

15
USING PERFORMANCE MEASUREMENT FRAMEWORK IN PORTFOLIO MANAGEMENT: A CONSTRUCTIVIST CASE Rogerio Tadeu de Oliveira Lacerda (UFSC) [email protected] Leonardo Ensslin (UFSC) [email protected] Sandra Rolim Ensslin (UFSC) [email protected] The main focus of this article is to present a framework to create a understanding of the business strategy context and aid the portfolio management process in a software company. The research method is a quali-quantitative, presented in a study case and the intervention instrument adopted is the Multicriteria Decision Aiding Methodology - Constructivist (MCDA-C). The framework enables to visualize the criteria that must be taken into account according to the decision- makers’ values in the project selection and sorting processes. The framework supports the ordinal and cardinal measurement of the project performance, making it possible to compare and rank proposals, as well providing a process to improve project proposals. This process has helped in negotiations between stakeholders in a portfolio context and, consequently, has helped the chief project officer to select and prioritize strategic projects within the demand for new products. Palavras-chaves: Performance Management, Portfolio Management, Decision, Technology Management, Multicriteria, Measurement XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October 2010.

Upload: dodieu

Post on 30-Jun-2018

216 views

Category:

Documents


0 download

TRANSCRIPT

USING PERFORMANCE

MEASUREMENT FRAMEWORK IN

PORTFOLIO MANAGEMENT: A

CONSTRUCTIVIST CASE

Rogerio Tadeu de Oliveira Lacerda (UFSC)

[email protected]

Leonardo Ensslin (UFSC)

[email protected]

Sandra Rolim Ensslin (UFSC)

[email protected]

The main focus of this article is to present a framework to create a

understanding of the business strategy context and aid the portfolio

management process in a software company. The research method is a

quali-quantitative, presented in a study case and the intervention

instrument adopted is the Multicriteria Decision Aiding Methodology -

Constructivist (MCDA-C). The framework enables to visualize the

criteria that must be taken into account according to the decision-

makers’ values in the project selection and sorting processes. The

framework supports the ordinal and cardinal measurement of the

project performance, making it possible to compare and rank

proposals, as well providing a process to improve project proposals.

This process has helped in negotiations between stakeholders in a

portfolio context and, consequently, has helped the chief project officer

to select and prioritize strategic projects within the demand for new

products.

Palavras-chaves: Performance Management, Portfolio Management,

Decision, Technology Management, Multicriteria, Measurement

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

2

A.

1. INTRODUCTION

An alternative to aid the strategic decision processes in context that involves

complexity is presented by portfolio management, by using problem structuring techniques

that improve the understanding of the consequences on the operational focus of the business.

The new competitive dimensions as agility and innovation have brought with them the

fact that the managers responsible for the portfolio management need to decide in an context

where they do not know the criteria to be met, nor do they develop mechanisms to improve

knowledge of the consequences of the portfolio management in the strategic objectives of the

organization (GANN & SALTER, 2000).

Nowadays, the decision to build understanding about the decision contexts, explaining

the objectives to be achieved and their association with the operational activities, becomes a

competitive key for organizations.

On this issue emerge research question of this article: How to measure a project

charter in light of the strategic objectives of a company in order to aid decisions on portfolio

management?

In order to answer the research question, the main objective in this study is propose a

methodology that allows identifying, measure and integrating dimensions judged by decision-

maker as necessary and sufficient for evaluating projects and thus enable the creation and

ordering of proposals in a cycle of selection of projects in an software development company.

Thus, the specific objectives of the research are: (i) To present a performance

measurement framework and to generate a better understanding of the portfolio context and

(ii) to present an application in order to illustrate the proposed framework for identifying and

evaluating a project in a portfolio.

To achieve the objectives set out above, the methodology selected by authors was the

Multicriteria Decision Aid Methodology – Constructivist (MCDA-C) by its characteristics of

constructivism, specifically regarding: (i) the ability to promote awareness of the actors, as

the context that we intend to improve, (ii) the ability to structure and evaluate the dimensions

considered relevant for these actors, giving more reliable results; (iii) the ability to spread

generated knowledge, (iv) capability to support the decision-making process.

The relevance of the theme is given by (i) the current organizations context with

respect to its business objectives, which are increasingly using project management as a tool

for competitive advantage (PWC, 2004); (ii) the feasibility to solve problems using the

methodology selected for this study (BANA E COSTA, ENSSLIN et al., 1999); (iii) to extend

the scientific publications of Englund and Graham (1999), Coombs et al (1998) and Cooper

(2000), suggesting the creation of criteria for selecting projects, but do not focusing their

studies on how to perform this activity.

Thus, beyond this introduction, this article is arranged in five sections. In the

following section is presented the theoretical framework for the portfolio management; in the

third section is presented the research methodology used; in the fourth section is presented the

built model, analysis and discussion; in the fifth section is for the final considerations of the

authors, and the last section the references used in this article are listed.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

3

2. THEORETICAL FOUNDATIONS

2.1 SUCCESS PROJECTS

Given its history had been developed in operational research, the discipline of project

management was coined within a positivist paradigm (WILLIAMS, 2005; POLLACK, 2007).

Thus, it is necessary to consider if such a paradigm is appropriate for a discipline that deals

with a complex reality (DE MEYER, LOCH et al., 2002; PICH, LOCH et al., 2002;

SOMMER & LOCH, 2004).

It is important to note that some hard thinking about project management should not

be regarded as wrong or should be replaced, but one must bear in mind that this view is only

one point of view to examine this field of knowledge (WINTER & CHECKLAND, 2003).

On the rationalist view, Roy (1994) consider that:

This is a reductionist conception of OR which I would call

unproductive. It is largely responsible for what has been called the OR

crisis. First of all, this conception of OR tends to cut it off from the milieu

which nourishes it and legitimates it as something other than a branch of

mathematics. Cutting off OR in this way encourages researchers to work in

isolation. This results in naive or impoverished references to managerial

reality and decision-making processes. Those responsible for solving

concrete problems are thus inevitably disappointed by the gap between

their own expectations and the results they receive.

Among the recent published studies on the effects of these two paradigms in theory

and practice of project management, we can highlight the text of Williams (2005). In this

systemic perspective, the project management has an attitude focused on stakeholders rather

than in pre-established requirements (TUKEL & ROM, 2001), a precedent in the "iron

triangle" (ATKINSON, 1999) in project management for a view more strategic (JUGDEV &

MÜLLER, 2006).

This evidence is relevant, since the end result of the project is assessed differently by

various stakeholders in the project and the success criteria should reflect diverse interests and

viewpoints (LIPOVETSKY, TISHLER et al., 1997; BACCARINI, 1999; SHENHAR,

TISHLER et al., 2002). Neglecting any point of view can mean the failure of the project

(DVIR, LIPOVETSKY et al., 1998), being then a great opportunity for multi-dimensional

studies, enabling us to analyze the mutual interactions of all variables and managerial success

metrics (DVIR, LIPOVETSKY et al., 1998).

Such characteristics in question, some authors have understood that the soft approach

has positive impacts on the management aspect when (i) the technological uncertainty is high

(DE MEYER, LOCH et al., 2002) and long-term consequences are diffuse, (ii) the project is

susceptible to external factors (WINTER & CHECKLAND, 2003) and (iii) the complexity of

the scope and context of the project are high (ATKINSON, CRAWFORD et al., 2006).

2.2 PORTFOLIO MANAGEMENT AND BUSINESS STRATEGY

The corporative strategy constitutes in an organizational process, in which Andrews

(1980) shows this process two important aspects: the formulation and implementation

(ANDREWS, 1980; MINTZBERG, 1994). However, to achieve success in the

implementation of strategy is needed its alignment with organizational interests (SHENHAR,

2004; MINARRO-VISERAS, BAINES et al., 2005), emerging the concept of projects

portfolio management.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

4

According to Cooper et al. (2000), portfolio management is the formulation, selection

and implementation of projects and the operationalization of organizational strategy.

Despite the growing interest and relevance to portfolio management by organizations,

these are faced with some difficulties (ELONEN & ARTTO, 2003), such as decisions on

project prioritization (COOPER, EDGETT et al., 2000), making a critical point the of project

selection.

Bearing this mind, Englund and Graham (1999) explore a process model for portfolio

management based on four stages, operated in the theoretical framework.

2.2.1 Establishing evaluation criteria

As a first step, it is necessary to define what should be done, focusing on the strategic

organization objectives (BOURNE, MILLS et al., 2000). This step represents a very crucial

point that determines whether the rest of the process will be successful (ENGLUND &

GRAHAM, 1999; COOPER, 2007).

At this point it is appropriate to increase the understanding of what are the dimensions

to be taken into account by decision-makers, especially in contexts that work with incomplete,

fuzzy and conflicting information (LIESIO, MILD et al., 2007).

In the process for formulating the problem, it is necessary to determine the dimensions

in which the projects will be evaluated based on the perception and values of the decision

makers. These performance criteria are scales in which the judgments and decisions will be

based to evaluate different projects (CHIEN, 2002).

Important to note that portfolio management is under political pressure, given the

personal interests of executives to use the power to impose their individual preferences

(ENGLUND & GRAHAM, 1999; CHIEN, 2002; ELONEN & ARTTO, 2003; ENGWALL &

JERBRANT, 2003). Thus, a decision-making process is structured and formalized through a

decision-maker in managing pressures of interest groups, justifying their decisions and

communicating with other organization elements (CHIEN, 2002).

The criteria must be sought in the strategic objectives of the people responsible for

portfolio management (KEENEY, 1992). The socio-political values of the decision maker and

the objective properties of the projects set the environment where the criteria should be

sought (KEENEY & RAIFFA, 1976; ROY, 1993; LANDRY, 1995; ROY, 1996; BANA E

COSTA, ENSSLIN et al., 1999). Care must be taken to ensure that the criteria are built from

the decision maker values and not from the differences between the alternatives and / or

sought from past similar situations even successful (KENNERLEY & NEELY, 2003;

ENSSLIN, GIFFHORN et al., 2010).

Although there are several generic proposals for structuring the criteria for selection,

the most important step is to identify the criteria that has greater significance for the

organization (ENGLUND & GRAHAM, 1999; SHENHAR, TISHLER et al., 2002; CHEN,

2008), taking into account the preferences of decision makers (ROY, 1993) and the adoption

of appropriate management techniques to each particular situation (SHENHAR, 2001;

LEWIS, WELSH et al., 2002).

Since the establishment of criteria and measurement takes into account the values and

preferences of the manager responsible for the area being evaluated, this step in performance

evaluation is personalized to the manager responsible for the decision. Roy (1996) and

Landry (1995) refer to this condition as the limit of objectivity.

After identifying what are the necessary and sufficient properties of the portfolio that

explain the decision-maker‟s values (LACERDA, 2009), it is necessary to translate them into

a measurable and unambiguous set of scales. At this stage, the decision-makers are prompted

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

5

by the facilitator to clarify the direction of preferences for each specific goal set in the

previous step.

Firstly, the ordinal scales built has a limited degree of knowledge since all de

information they provide is qualitative. If the decision-maker want to improve the quality of

the information he have to transform the ordinal scales into cardinal scales providing the

information of the difference of attractivity between the level of the scales.

2.2.2 Project information collection

Once clarified the criteria, the teams can "shape" the projects scope according to these

criteria, in order to be aligned to the strategy (SALOMO, WEISE et al., 2007).

Thus, the second step defined proposed by Englund and Graham (1999) for selection

of projects is the projects information collection. The main way out to this phase is a list of

projects with their charters and information necessary to confront the projects properties with

pre-established criteria for the organization.

2.2.3 Evaluation and Recommendations

This stage of evaluation is to determine the impact of projects on performance

indicators to understand its consequences (KEENEY, 1992). A recurring problem in portfolio

management is to give priority to short duration projects, low risk and consequently that with

little impact on improving competitiveness (FRICKE & SHENHAR, 2000).

Thus, measuring the impact of each element to the portfolio management, the third

stage ends with the establishment of prioritization in projects within the available investment

resources.

2.2.4 Monitoring the portfolio management

The fourth step is proposed to allocate the financial and human resources in prioritized

projects, communicate project teams and continue managing the initiatives. In this stage,

monitoring of projects should return information to executives in order to close the cycle of

four stages, as proposed by Englund and Graham (1999).

Noting that there is not a single tool for portfolio effective monitoring, the

methodology presented in this article may be used to build models in line with the two

approaches described by Cooper et al (2000) for monitoring the portfolio management.

2.3 PORTFOLIO MANAGEMENT AND PERFORMANCE EVALUATION

As shown, the contribution of a structured process on the constructivist performance

evaluation allows the identification of opportunities for improving portfolio management.

Thus, we draw the following parallel with the paradigms for decision aid (ENSSLIN, 2009) to

be recognized, as illustrated in Table 1. Decision Aiding

Paradigms Paradigm Description Context in Portfolio Management

Reference in

Portfolio Management

P1 = Uniqueness,

Identity

Decision maker values and

preferences

Criteria for evaluating projects

must be contextualized and

developed in each case the

projects performance evaluation.

(ENGLUND &

GRAHAM, 1999;

SHENHAR, 2001;

CHIEN, 2002;

ENGWALL, 2003)

P2 = Limited

Knowledge

Decision makers‟ need to

improve their understanding

of the decision

consequences.

Approaches that expand the

understanding of decision makers

on the context must be used

(WILLIAMS, 2005;

CÁÑEZ & GARFIAS,

2006; LIESIO, MILD et

al., 2007)

P3 = Social Entity

To favor the stakeholders

with interests in the

decision to submit their

Recognize that the criteria for

evaluation are influenced by social

participants in the portfolio

(ENGLUND &

GRAHAM, 1999;

CHIEN, 2002;

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

6

interests to the decision management decision process ENGWALL, 2003;

WILLIAMS, 2005)

P4 = Recursive

Participatory Learning

The dynamic recursive

process of participants

learning

Recognition that the learning

process is cyclical and that

organization needs a mechanism

to incorporate such knowledge in

the organizational culture.

(ENGLUND &

GRAHAM, 1999;

COOPER, EDGETT et

al., 2000; ENGWALL,

2003; WILLIAMS,

2005)

P5 = Principles of

Measurement

Properties of ordinal scales,

interval, and ratio

Among references of portfolio

management found, none

increases the understanding of the

process of measuring the level of

differences recognition between

ordinal and cardinal scales.

(ROBERTS, 1979;

BARZILAI, 2001)

P6 = Legitimacy and

Validation

Transparency of

participation, recognition of

usefulness of knowledge

generated and the scientific

status of the construction of

knowledge used

Recognition by the decision

maker, knowledge built by the

decision aid process was useful to

understand the consequences of

projects in strategic objectives as

well as having scientific support

for corporate use

(ENGLUND &

GRAHAM, 1999;

CHIEN, 2002; CÁÑEZ

& GARFIAS, 2006;

ENSSLIN, GIFFHORN

et al., 2010)

Table 1: Paradigms for Decision Aid (ENSSLIN, 2009) and Portfolio Management

3 RESEARCH METHODOLOGY

This item is divided into three sections, first section presents a methodological

framework, second section presents the procedures used to the literature review and the third

section discusses the adopted intervention instrument.

3.1 METHODOLOGICAL FRAMEWORK

This research is exploratory, applied and developed in a case study way because the

goal resides in deepening the knowledge related to innovation within a software development

company. Data used are primary, since they were obtained through unstructured interviews

with the technology director and manager of PMO (Project Management Office) of the

company in which the research was performed. The research method is qualitative and

quantitative. The intervention tool used was the Multicriteria Decision Aid Methodology -

Constructivist (MCDA-C) (ENSSLIN, GIFFHORN et al., 2010).

3.2 LITERATURE REVIEW

The theoretical framework of this article began searching for articles in each journal

cited by Shenhar and Dvir (2007), using the keyword "project management", which created a

database of 200 articles that, after systematic reading of their abstracts before the previously

defined criteria: (i) view of knowledge is not reduce to the positivist paradigm, (ii) recognize

that the dimensions of project success are socially constructed and (iii) create opportunities to

expand the understanding of what the degree of success in project and portfolio management.

Finally, after reading the full articles, eight articles were selected (COOMBS,

MCMEEKIN et al., 1998; FRICKE & SHENHAR, 2000; CHIEN, 2002; DVIR &

LECHLER, 2004; SHENHAR, 2004; WILLIAMS, 2005; TUKEL, ROM et al., 2006;

COOPER, 2007) to set the core of the bibliography.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

7

3.3 CONSTRUCTION OF THE MULTICRITERIA MODEL – PROCEDURES

The construction of the model of performance evaluation following the MCDA-C

methodology is divided into eight steps, described in this sub-section.

3.3.1 STEP 1: CONTEXTUALIZATION

The Structuring Phase aims to explain the context and achieve an understanding of the

problem to be discussed. To achieve such an aim, the players involved in the context are

identified and the problem statement is legitimated with them.

3.3.2 STEP 2: HIERARCHICAL STRUCTURE OF VALUE

The facilitator then encourages the decision maker to talk about the context, and by

interpreting the interviews the Primary Elements of Evaluation (PEE) are identified. These

elements are the essential factors in the decision maker‟s system of values and concerns. For

each PEE, a concept representing the decision maker‟s choice of preference direction is

constructed, as well as its psychological opposite pole.

Then, the decision maker is encouraged to group the concepts in areas of concern.

With the concepts of each area of concern, cognitive maps are constructed (BITITCI,

SUWIGNJO et al., 2001). In the means-ends relationship map, the clusters of concepts are

identified (EDEN, JONES et al., 1985) and they represent the map in an exhaustive way.

Each cluster in the cognitive map has an equivalent point of view in the hierarchical structure

of value. This association makes it possible to transfer knowledge from cognitive map to the

hierarchical structure of value (ENSSLIN, DUTRA et al., 2000).

3.3.3 STEP 3: CONSTRUCTION OF DESCRIPTORS

The hierarchical structure of value represents the strategic dimension called

Fundamental Points of View (FPsV) and their connections to the operational activities, called

Elemental Points of View (EPsV). Next, it is necessary to use the information in the cognitive

maps to build ordinal scales in the hierarchical structure of value, named descriptors, in order

to measure the range of what is measured (BANA E COSTA, ENSSLIN et al., 1999). In order

to establish the basis for comparing the performance between descriptors, the decision maker

must identify the reference levels „neutral‟ and „good‟ (ENSSLIN, DUTRA et al., 2000). The

process of qualitative knowledge generation is finished with the descriptors.

3.3.4 STEP 4: INDEPENDENCE ANALYSIS

To continue the process of building knowledge, the qualitative scales of descriptors

must be transformed onto cardinal scales and then integrated. The MCDA-C uses a

compensatory model to build the global evaluation model. This model assumes that the

conversion rates used in the integration are constant. To achieve this condition, the criteria

must be independent (ENSSLIN, DUTRA et al., 2000).

3.3.5 STEP 5: CONSTRUCTION OF VALUES FUNCTIONS AND IDENTIFICATION OF CONVERSION

RATES

The next step in the methodology is the transformation of the descriptors into cardinal

scales called value functions. This transformation requires the decision makers to describe the

different levels of attractiveness for all the levels of the descriptor. The integration is achieved

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

8

by associating the conversion rates with the increase in performance when improving from

the „neutral‟ reference level to the „good‟ reference level for each descriptor.

3.3.6 STEP 6: IDENTIFICATION OF IMPACT PROFILE OF ALTERNATIVES

Then it is possible to evaluate the performance of every alternative and, among them,

the status quo. The models constructed by the MCDA-C methodology make possible an

explicit evaluation in numerical and/or graphic form, facilitating the identification and

understanding of the intensity of strong as well as weak points of the alternatives under

evaluation.

3.3.7 STEP 7: SENSITIVITY ANALYSIS

In order to provide a broad view of the stability of alternative performances, the model

allows for the development of a sensitivity analysis of the impact of alternatives in the scales,

in the attractiveness difference in cardinal scales as well as in the conversion rates.

3.3.8 STEP 8: FORMULATION OF RECOMMENDATIONS

The knowledge generated allows the decision-makers to visualize for each criterion

where the performance of a given alternative is „good‟, „normal‟ or „weak‟. The scale of the

descriptors allows the identification of actions to improve performance. Combining this

knowledge with the global evaluation obtained in the previous step, it is possible to generate

alternatives and measure their impact in the context.

4 MULTICRITERIA MODEL OF PROJECT EVALUATION

STEP 1: CONTEXTUALIZATION

The research commenced with meetings with the decision makers of the company in

order to contextualize the problem and expose its function. The company wanted to have a

mechanism to identify and evaluate projects proposals of new software products.

The interview resulted in the establishment of a problem focus, with definition of: (i)

Label: To evaluate the competitive advantage of a new company product; (ii) Players: Core

Decision-makers: CTO (Chief Technology Officer) with the CPO (Chief Project Officer);

Relevant Stakeholders: Product Manager and Sales Executive; those directly affected by

decisions: users; those indirectly affected by decisions: customers; (iii) Problem statement:

Product managers demand new software, but they do not have a mechanism to evaluate the

competitiveness of a product at the beginning of a project.

STEP 2: HIERARCHICAL STRUCTURE OF VALUE

After legitimating the context of the problem with the decision maker, the PEEs were

mapped. 60 PEEs were identified, and classified in two areas of concern for the decision

maker: “compatible products with global players” and “life cycle cost”.

As the study continued, the decision makers were questioned in order to guide the

PEEs to action and to determine the concepts inherent in each PEE. When this activity was

concluded, the decision maker started to construct two means-ends relationship maps, one for

each area of concern. With the maps, it was possible to identify the sub-clusters. Next, the

clusters were associated with the concerns and transferred to the hierarchical structure of

value, forming the fundamental points of view and the elementary points of view.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

9

Figure 1 shows the means-ends relationship map constructed for the area of concern

“compatible products with international players”, as well as the identification of its clusters

that generate the FPsV. At this point, a hierarchical structure or a tree of value was proposed

for the decision context.

Figure 1: Means-ends relationship map of an area of concern highlighting the clusters “Strategic Alliances”,

“Pioneering” and “SOA”

STEP 3: CONSTRUCTION OF DESCRIPTORS

After the identification of the FPsV, the construction of scales for the multicriteria

model was started in order to measure the performance of each action on an ordinal scale,

using the decision maker‟s construction. In order to construct the scales, in some cases it was

necessary to decompose the FPsV in the elementary points of view (EPsV) in order to

operationalize the descriptors.

From the hierarchical value structure, the construction of descriptors was started in

order to express the decision maker‟s strategic objectives.

STEP 4: INDEPENDENCE ANALYSIS

All the criteria were analyzed to check the independence of preferences.

STEP 5: CONSTRUCTION OF VALUES FUNCTIONS AND IDENTIFICATION OF CONVERSION RATES

When the structuring of the decision context was concluded, the evaluation phase was

started with the transformation of the descriptors into cardinal scales, so as to express the

attractiveness of the impact levels of the descriptors. With the use of the MACBETH

software, Figure 2 shows the transformation phases from the ordinal scale to the cardinal, for

one of the EPV descriptor “sign services” from FPV-“SOA”.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

10

Figure 2: Transformation process of a descriptor in a specific criterion (Cardinal Scale), using the MACBETH software

Once having the value function, and defined the unit, of all the points of view, the

process to integrate them and form the representation of the model of global evaluation

according to the perception of the decision maker was started. Integration was achieved

applying the conversion rates to the points of view. At this point, the global model of the

multicriteria support was used to evaluate the success of a software project of the company

investigated.

STEP 6: IDENTIFICATION OF IMPACT PROFILE OF ALTERNATIVES

The project under consideration obtained 50 points, and hence was rated as

competitive according to the decision maker‟s perception. The evaluation in graphic form is

presented in Figure 3, where the arrows beside the points of view are the scale representation

of success in each dimension evaluated.

Figure 3: Hierarchical value structure and the graphic of consequences of the project

STEP 7: SENSITIVITY ANALYSIS

Finally, changes to conversion rates and the impact of each criterion were simulated,

as a sensitivity analysis of the model. This analysis resulted in an instrument to show to the

decision maker and project agents where the best opportunities for improvement in

performance are, thus improving the competitiveness of new products developed.

STEP 8: FORMULATION OF RECOMMENDATIONS

The generated knowledge has helped to measure, on a cardinal scale, the contribution

that improvement actions may make to the strategic objectives as can be observed in the

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

11

measurement of 50 points for the initial version of the project evaluated. With this knowledge,

the chief project officer (CPO) started a new activity to sort the improvement requests and

improve the project‟s performance, linking with the FPV “Speedy in developing new

features”, that would improve the global performance by 13 points (see Figure 4). This would

involve the average rate of errors dropping from 4 to 2 (EPV-“Project Management”), the

project deploying 10 features requested by stakeholders per months instead of 8 (EPV-

“Productivity”), and the project reusing 20% of the lines of code instead of the original 15%

(EPV-“Components Reuse”). This illustrate the Step 8 of the MCDA-C, recommendations.

Figure 4: The representation of the impact of improvement action a’ in the global performance against the current project a

5 CONCLUSIONS AND RECOMMENDATIONS FOR FURTHER RESEARCH

The present research used recent views of knowledge to propose solutions to some

problems observed in portfolio management, proposing a methodology that enables the

identification, organization, measurement and integration of criteria judged necessary and

sufficient by the decision maker, in order to understand the context, and thus, to understand

the consequences of different decisions.

With the points above in mind, the research question guiding this work is revisited:

How to measure a project charter in light of the strategic objectives of a company in order to

aid decisions on portfolio management?

To answer this question, the present work has presented a methodology that, by using

the instruments described here, helps the creation of possible improvement actions to increase

alignment with de strategic planning.

Regarding the achievement of the objective of this research, as set out in Section 3.3,

“Construction of the Multicriteria Model – procedures” – the MCDA-C methodology has

been presented and Section 4.

The decision maker‟s understanding of the contribution of the project was presented

graphically and numerically, as shown in Figure 3. This knowledge emerges from developing

the ability to foresee the consequences of the operational characteristics of the project in the

strategic objectives of the company. This contributes to foreseeing the opportunities for

project improvement, as well as generating, starting from this instrument, actions to improve

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

12

its chances of success. Moreover, this cardinal measurement allows the comparability and

project ordering the evaluated faced with other projects proposals.

The decision makers may apply the model to guide the search for actions in order to

select and prioritize projects put forward by managers. Before the model, the decisions were

ad-hoc, resulting in many negotiations in order to choose which project to begin.

A limitation of the present research is its tightly centered focus on the technical

concerns of product development, which is the decision maker‟s main concern. So, the model

could be expanded by the unpacking of the core problem in order to observe other company

areas, such as services, sales and marketing. The MCDA-C methodology has demonstrated its

usefulness in the process of supporting portfolio management, and it is recommended that it

be applied in other contexts, so as to test its generality in similar situations. However, as

models generated in each situation are specific to the context, the model will need to be

adapted, even in contexts that are very similar.

6 REFERENCES

ANDREWS, K. R. The Concept of Corporate Strategy, rev. ed. Homewood, IL: Irwin. 1980.

ATKINSON, R. Project management: cost, time and quality, two best guesses and a

phenomenon, its time to accept other success criteria. International Journal of Project

Management, v.17, n.6, p.337-342. 1999.

ATKINSON, R., CRAWFORD, L. & WARD, S. Fundamental uncertainties in projects

and the scope of project management. International Journal of Project Management, v.24, n.8,

p.687-698. 2006.

BACCARINI, D. The logical framework method for defining project success. Project

Management Journal, v.30, n.4, p.25. 1999.

BANA E COSTA, C. A., ENSSLIN, L., CORREA, E. C. & VANSNICK, J.-C. Decision

Support Systems in action: Integrated application in a multicriteria decision aid process.

European Journal of Operational Research, v.113, n.2, p.315-335. 1999.

BARZILAI, J. On the foundations of measurement. 2001 IEEE International Conference on

Systems, Man, and Cybernetics, v.1. 2001.

BITITCI, U. S., SUWIGNJO, P. & CARRIE, A. S. Strategy management through

quantitative modelling of performance measurement systems. International Journal of

Production Economics, v.69, n.1, Jan, p.15-22. 2001.

BOURNE, M., MILLS, J., WILCOX, M., NEELY, A. & PLATTS, K. Designing,

implementing and updating performance measurement systems. International Journal of

Operations & Production Management, v.20, n.7, p.754-771. 2000.

CÁÑEZ, L. & GARFIAS, M. PORTFOLIO MANAGEMENT AT THE MEXICAN

PETROLEUM INSTITUTE. Research Technology Management, v.49, n.4, p.46. 2006.

CHEN, C. C. An objective-oriented and product-line-based manufacturing performance

measurement. International Journal of Production Economics, v.112, n.1, Mar, p.380-390.

2008.

CHIEN, C.-F. A portfolio-evaluation framework for selecting R&D projects. R&D

Management, v.32, n.4, p.359-368. 2002.

COOMBS, R., MCMEEKIN, A. & PYBUS, R. Toward the development of benchmarking

tools for R&D project management. R&D Management, v.28, n.3, p.175-186. 1998.

COOPER, R. G. Managing technology development projects. Engineering Management

Review, IEEE, v.35, n.1, p.67-67. 2007.

COOPER, R. G., EDGETT, S. J. & KLEINSCHMIDT, E. J. New Problems, New

Solutions: Making Portfolio Management More Effective. Research Technology

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

13

Management, v.43, n.2, p.18-33. 2000.

DE MEYER, A., LOCH, C. H. & PICH, M. T. From Variation to Chaos. MIT Sloan

Management Review, winter. 2002.

DVIR, D. & LECHLER, T. Plans are nothing, changing plans is everything: the impact of

changes on project success. Research Policy, v.33, n.1, p.1-15. 2004.

DVIR, D., LIPOVETSKY, S., SHENHAR, A. & TISHLER, A. In search of project

classification: a non-universal approach to project success factors. Research Policy, v.27,

n.9, p.915-935. 1998.

EDEN, C., JONES, S. & SIMMS, D. Messing about in Problems. R&D Management, v.15,

n.3, p.255-255. 1985.

ELONEN, S. & ARTTO, K. A. Problems in managing internal development projects in

multi-project environments. International Journal of Project Management, v.21, n.6, p.395-

402. 2003.

ENGLUND, R. L. & GRAHAM, R. J. From Experience: Linking Projects to Strategy.

Journal of Product Innovation Management, v.16, n.1, p.52-64. 1999.

ENGWALL, M. No project is an island: linking projects to history and context. Research

Policy, v.32, n.5, p.789-808. 2003.

ENGWALL, M. & JERBRANT, A. The resource allocation syndrome: the prime challenge

of multi-project management? International Journal of Project Management, v.21, n.6, p.403-

409. 2003.

ENSSLIN, L. Processo Decisório e Aspectos Cognitivos. XII Simpósio de Pesquisa

Operacional e Logística da Marinha. 2009.

ENSSLIN, L., DUTRA, A. & ENSSLIN, S. R. MCDA: a constructivist approach to the

management of human resources at a governmental agency. International Transactions in

Operational Research, v.7, n.1, p.79-100. 2000.

ENSSLIN, L., GIFFHORN, E., ENSSLIN, S. R., PETRI, S. M. & VIANNA, W. B.

Avaliação do Desempenho de Empresas Terceirizadas com o Uso da Metodologia

Multicritério de Apoio à Decisão- Construtivista. Revista Pesquisa Operacional, v.v.30, n.1,

p.125-152. 2010.

FRICKE, S. E. & SHENHAR, A. J. Managing multiple engineering projects in a

manufacturing support environment. Engineering Management, IEEE Transactions on, v.47,

n.2, p.258-268. 2000.

GANN, D. M. & SALTER, A. J. Innovation in project-based, service-enhanced firms: the

construction of complex products and systems. Research Policy, v.29, n.7-8, p.955-972. 2000.

JUGDEV, K. & MÜLLER, R. A retrospective look at our evolving understanding of project

success. Project Management Journal, v.36, n.4, p.19-31. 2006.

KEENEY, R. L. Value-Focused Thinking: A Path to Creative Decisionmaking: Harvard

University Press. 1992

KEENEY, R. L. & RAIFFA, H. Decisions with multiple objectives: preferences and value

tradeoffs. New York: Wesley, p.224-241. 1976.

KENNERLEY, M. & NEELY, A. Measuring performance in a changing business

environment. International Journal of Operations & Production Management, v.23, n.2, p.213-

229. 2003.

LACERDA, R. T. O. O sucesso em gerenciamento de projetos: a estruturação de um modelo

de avaliação a partir de uma visão construtivista. Florianópolis. Dissertação de mestrado em

Engenharia de Produção da Universidade Federal de Santa Catarina. 2009.

LANDRY, M. A Note on the Concept of 'Problem'. Organization Studies, v.16, n.2, p.315.

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

14

1995.

LEWIS, M. W., WELSH, M. A., DEHLER, G. E. & GREEN, S. G. Product development

tensions: Exploring contrasting styles of project management. Academy of Management

Journal, v.45, n.3, Jun, p.546-564. 2002.

LIESIO, J., MILD, P. & SALO, A. Preference programming for robust portfolio modeling

and project selection. European Journal of Operational Research, v.181, n.3, p.1488-1505.

2007.

LIPOVETSKY, S., TISHLER, A., DVIR, D. & SHENHAR, A. The relative importance of

project success dimensions. R&D Management, v.27, n.2, p.97-106. 1997.

MINARRO-VISERAS, E., BAINES, T. & SWEENEY, M. Key success factors when

implementing strategic manufacturing initiatives. International Journal of Operations &

Production Management, v.25, n.2, p.151. 2005.

MINTZBERG, H. The Rise and Fall of Strategic Planning: Reconceiving Roles for Planning,

Plans, Planners: Free Press. 1994

PICH, M. T., LOCH, C. H. & MEYER, A. D. On uncertainty, ambiguity, and complexity in

project management. Management Science, v.48, n.8, p.1008. 2002.

POLLACK, J. The changing paradigms of project management. International Journal of

Project Management, v.25, n.3, p.266-274. 2007.

PWC. Boosting business performance through programme and project management.

Connected Thinking. PriceWaterhouseCoopers. 2004.

ROBERTS, F. S. Measurement theory: Addison-Wesley Reading, MA. 1979

ROY, B. Decision science or decision-aid science? European Journal of Operational

Research, v.66, n.2, p.184-203. 1993.

______. On operational research and decision aid. European Journal of Operational

Research, v.73, n.1, p.23-26. 1994.

______. Multicriteria Methodology for Decision Aiding: Kluwer Academic Pub. 1996

SALOMO, S., WEISE, J. & GEMUNDEN, H. G. NPD Planning Activities and Innovation

Performance: The Mediating Role of Process Management and the Moderating Effect of

Product Innovativeness. Journal of Product Innovation Management, v.24, n.4, p.285-302.

2007.

SHENHAR, A. J. One Size Does Not Fit All Projects: Exploring Classical Contingency

Domains. MANAGEMENT SCIENCE, v.47, n.3, March 1, 2001, p.394-414. 2001.

______. Strategic Project Leadership: Toward a strategic approach to project management.

R&D Management, v.34, n.5, p.569-578. 2004.

SHENHAR, A. J. & DVIR, D. PROJECT MANAGEMENT RESEARCH-THE CHALLENGE

AND OPPORTUNITY. Project Management Journal, v.38, n.2, p.93. 2007.

SHENHAR, A. J., TISHLER, A., DVIR, D., LIPOVETSKY, S. & LECHLER, T.

Refining the search for project success factors: a multivariate, typological approach. R&D

Management, v.32, n.2, p.111-126. 2002.

SOMMER, S. C. & LOCH, C. H. Selectionism and Learning in Projects with Complexity

and Unforeseeable Uncertainty. Management Science, v.50, n.10, p.1334. 2004.

TUKEL, O. I. & ROM, W. O. An empirical investigation of project evaluation criteria.

International Journal of Operations & Production Management, v.21, n.3, p.400. 2001.

TUKEL, O. I., ROM, W. O. & EKSIOGLU, S. D. An investigation of buffer sizing

techniques in critical chain scheduling. European Journal of Operational Research, v.172, n.2,

p.401-416. 2006.

WILLIAMS, T. Assessing and moving on from the dominant project management discourse

XVI INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND OPERATIONS MANAGEMENT

Challenges and Maturity of Production Engineering: competitiveness of enterprises, working conditions, environment. São Carlos, SP, Brazil, 12 to 15 October – 2010.

15

in the light of project overruns. Engineering Management, IEEE Transactions on, v.52, n.4,

p.497-508. 2005.

WINTER, M. & CHECKLAND, P. Soft systems- a fresh perspective for project

management. Proceedings of the Institution of Civil Engineers: Civil Engineering, v.156, n.4,

p.187-192. 2003.