978-1-4244-2173-2/08/$25.00 ©2008 IEEE

Implications of a Rationalist Inductive Approach in System of Systems Engineering Research

Andres Sousa-Poza Ph.D.

Engineering Management and Systems Engineering

Old Dominion University asousapo@odu.edu

Jose J. Padilla Engineering Management and

Systems Engineering Old Dominion University

jpadi002@odu.edu

Ipek Bozkurt Engineering Management and

Systems Engineering Old Dominion University

ibozkurt@odu.edu

Abstract. System of Systems Engineering (SoSE) is a rapidly growing discipline that has emerged in response to increasingly complex situations. Complexity arises from the use of reducible perspectives on irreducible problems, from emergence, and other sources that serve to create uncertainty. As these factors violate or challenge fundamental assumptions of systems engineering, new approaches to deal with new problems have been developed. Similarly, complex situations inherently provide challenges for research. Traditional management and engineering research approaches, which are based largely on empiricist paradigms, face serious limitations when addressing complex problems. It is argued in this paper that additional paradigms are needed supported by appropriate methodologies. The rationalist inductive approach is discussed in detail, guidelines for a modeling based rationalist inductive approach are presented, and the implications for SoSE are addressed. Keywords: System of Systems, rationalism, induction, complexity

1 Introduction

It is safe to say that today, more than ever, scholars and practitioners from all disciplines realize the complex nature of the problems that are being dealt with. On one hand, technology is developing at an increasing speed; on the other hand, human behavior seems to be the area that is under the microscope. The word “system” does not seem to be sufficient anymore; we are now dealing with “System of Systems” (SoS), enterprise systems or just complex systems. SoSE is an emerging discipline intended to help with the design and management of System of Systems domains, and in general, complex domains [1].

In order to better deal with these complex

problems, a crucial shift in the philosophical research paradigm is necessary. Just as traditional Systems Engineering tools are not sufficient

anymore to deal with today’s problems, the Hypothetico-Deductive method of theory testing common in the dominant discourse will no longer serve the needs of the contemporary research to build theory or address SoS problems. Based on this argument a Rationalist Inductive approach is proposed to deal with complex conditions that arise when researching problems under the system of system perspective. Given that SoS is a nascent discipline the work presented by many researchers in the field is theoretical by nature and in the best case exploratory. Some may claim empirical work but solely under narrow conditions in which the idea of SoS is lost or void of social context.

The purpose of this work is to propose a

methodology for conducting theoretical work in a more rigorous manner. This methodology uses the premise of the coherence theory of truth and coherence theory of knowledge as foundations to build knowledge with the intention of making the process repeatable knowing that the final results may not. This rationalist perspective is presented with an inductive method of inference which suggests a more exploratory framework. In the following sections a detailed argument on why a Rationalist Inductive approach is vital for research in terms of building theory, addressing old and new models, and formulating new methodological approaches is presented. 2 Concept of Research

The ultimate goal of doing research is to create knowledge. A well established definition of knowledge is provided by Plato establishing knowledge as a justified true belief (JTB). Although this definition has received criticism [2], it is the most enduring construct. Our interest in this paper relates mostly to justification.

Arguably, the most accepted form of

justification is the hypothetico-deductive method, in which a hypothesis is presented and deductively tested [3]. Using this type of method we can

establish correlations, and ultimately causality. There are, however, a vast number of justification approaches that can be taken. Presently, the strongest challenge is from neo-positivistic or constructivist perspectives, loosely encapsulated in what are generally referred to as the qualitative research paradigm. Ultimately, the empiricist base of these paradigms results in similar limitations to their effectiveness. Some of the fundamental limitations encountered include:

1. The loss of traceability between observations

and abstracted generalizations when emergent conditions are present,

2. The rapidly divergent contexts that are established when attempting to address complex problems holistically, and inter alia,

3. The teleological limitations (or inappropriateness) conveyed from the empiricist perspectives to non-ergodic, non-linear problems. When we are dealing with complex systems in

which establishing causality is pervasively evasive, to say the least, how can we cope with it? Is the hypothetico-deductive method or empiricist approaches the most appropriate methods to conduct research and create knowledge? These questions are the reason for this paper and for presenting a formal alternative. It must be noted that this alternative is and has been used extensively in different formats and in different fields. 3 Rationalist Inductive Approach

As a starting point, we need to separate the

term “rationalist inductive” into its two components: the one referring to induction and the one referring to rationalism.

According to [4], “induction is defined in

general ways as a process of inferring from the particular to the universal. That is, whenever we derive a general statement from a particular statement or facts, we have induction”. Thilly, in the same paper, elaborates further on the concept of induction; we present a few highlights:

• In order to infer, we must consciously relate

one judgment with another, otherwise it would be just association of ideas.

• Induction and deduction are both processes of inference, the former moving from particular to

general, the latter moving from general to particular.

• Induction seeks to discover not the casual, but the causal connections.

[5] presents that as humans we are moderately

good at deductive logic, while we are superb at recognizing or matching patterns and constructing temporary internal models/hypotheses/schemata to work with. “We carry out localized deductions based on our current hypotheses and act on them. And, as feedback from the environment comes in, we may strengthen or weaken our beliefs in our current hypotheses, discarding some when we cease to perform, and replacing them as needed with new ones. In other words, we cannot fully reason or lack full definition of the problem; we use simple models to fill the gaps in our understanding. Such behavior is inductive” [5].

Induction is important not only because it is the

main way we learn, we are familiar with, and we can extrapolate from, but also because induction provides elements to establish an overall framework of study by creating boundaries of a narrow field of study or an ill defined situation by establishing causal connections.

Rationalism is a way of thinking, which basically states that knowledge can be obtained deductively by appealing to mental constructs such as concepts, laws, or theories [6]. Rationalism justifies knowledge by the coherence of a system of premises [7], [8]. The Coherence Theory of truth is a doctrine which argues that all our concepts are related to one another in such a way that we cannot be said fully to have grasped any one of them unless we have grasped all the others: they form an organic conceptual scheme, it is said, a system of meanings which cohere in such a way that introducing a new concept at any one point in the system has repercussions which are felt through the system [9]. Coherence is a system of premises which should be sufficient to establish truth; the coherence theory of knowledge holds that knowledge claims require justification, but also that no belief can be justified except by reference to other beliefs [8].

Although in the previously quoted definition of rationalism it is stated that knowledge can be obtained deductively, it can also be used inductively to create -through coherence- justified true beliefs, or knowledge. Furthermore, it helps us build new models and theories using existing models and theories in an inductive manner. In this

sense, the feedback of our hypotheses comes in the coherence of the models used. Figure 1 is a simple representation of this approach.

Figure 1. Inductively building theory through

coherence In order to achieve truth through coherence,

two things need to take place: the premises must be true and they must hold together. For this, the axiological structure of assumptions and presuppositions by the inquirer must be clearly established. The axiological structure establishes the context of the study, and under this axiological structure the proposed theory must be true based on a coherent system of beliefs achieved through the inductive process. This in turn, placed under a methodological structure, can be used to formalize the theory or model.

4 Rationalist Inductive Approach and

SoSE Like in any emerging discipline, it is difficult

to agree on definitions of concepts that shape and bound SoSE. Unlike Systems Engineering, which is a well-established discipline built on solid foundations; SoSE is still being shaped by academics and practitioners. Keeping this perspective in mind, it can be said that most of the concepts that are related to SoSE do not have empirical foundations. After a review of literature on SoSE, [10] have stated that a rigorous development of SoSE concepts is notably absent. To be more specific, they state that the “empirical development of an associated body of knowledge for use as a guide, furthering knowledge development and practical application” is missing [10].

Nevertheless, the foundations of SoSE are still

being built; and as research builds upon the previous work, SoSE boundaries are established. Issues such as how a system of systems should behave; what purposeful actions it should follow; what the nature of interrelationships between entities should be like etc. are being addressed. The conclusions of these discussions make way for new premises and propositions that the SoSE discipline is being built upon.

As stated above, these premises, assumptions and propositions have no empirical basis; this is so for couple of reasons. Firstly, when dealing with SoS, most of the time it is not possible to have a complete understanding of what goes on within a system, due to the nature of the complexity of the system. Uncertainty and complexity tend to have a “black box” effect on the system. Emergence also presents an immediate challenge in that direct correspondence is not possible; also, computational or modeling challenges with complex situations may make direct correspondence intractable due to the nature of the problem. Another reason why establishing an empirical basis for some of the SoSE concepts is not possible is that a system of systems is a combination of both technical and social aspects. The technical issues in SoSE have higher reducibility than the more “soft” components of the systems, such as human factors, contextual issues, etc. As such, it becomes impossible to empirically establish a unified perspective that incorporates empirically derived social and technical components

Taking these into consideration, it is safe to say

that many concepts and premises of SoSE are derived in a rationalistic manner. This is where the Rationalist Inductive approach becomes crucial when dealing with system of systems research. The less-traditional ways of reaching knowledge, such as the rationalist inductive approach, should be acknowledged and accepted in the scholar community. The rationalist inductive approach should not be taken into consideration as a second-best option of doing research, but rather, an appropriate paradigm to be considered when dealing with complex, emerging and dynamic issues. 5 Proposed Methodology

Philosophical theories are neither right nor wrong, forming different perspectives to discuss the problem in question. In addition, epistemological and truth theories highlight the importance and dynamics of non-mathematical human beliefs that are essential for science in general and simulation in particular [11]. When basing truth on correspondence we focus on the association of action and observable results, while when using coherence we focus on the strength of the belief system. The question that is raised is how do we justify such coherent structures? For correspondence many approaches exist; with the most prevalent possibly being the hypothetic-deductive model. Few formal methods are however

available to support coherence. The following methodology (figure 2) presents the building blocks for it.

Figure 2. Proposed Methodology for Rationalist Inductive Approach

1. Exploration: This is the heart of the inductive

process in terms that it explores the possible related domains associated with the research question. The research problem can be identified via different means, such as literature reviews, case studies, etc. In this exploration step, the premises which form the foundation of the argument are defined. The context is set, which provides the axiomatic foundation of the research.

2. Structuration: This is the heart of the coherence theory of truth process, in which a coherent formulation of the New Theory is presented, which is then strengthened by bounding and fine tuning the system of beliefs and premises, to finally establish the model. The conditions under which the model will be developed and executed are also identified. We can represent the model itself as a specification, according to [12]. This specification can be a mathematical equation, a logical statement, or a computer simulation.

3. Conclusion: The last block of the Rationalist Inductive methodology is the conclusion, which is where the developed model and the output are presented. The conclusion can include an array of analysis, from very deterministic statistical analysis to the more interpretative analysis.

6 Validity

Validity is conditio sine qua non in the

execution of research. It appears in the literature as stamps of guarantee of the quality of theories and is

almost as synonymous with scientific rigor or hard science. Validity involves concepts such as well-grounded, justifiable, executed with legality and formality, logical correctness, appropriate, and conforming to accepted principles.

Internal validity in a quantitative empirical

context is defined as the extent to which the instrument measures what is supposed to measure ([13]; [14]. Schwandt in [15] provides a more qualitative empirical definition as validity being is defined as how accurately the account represents participants’ realities of the social phenomena and it is credible to them.

Internal validity from a rationalist perspective

is a function of the coherence with which the participants’ view has been successfully conveyed which is very similar to the premise of validity in qualitative research; different being that the latter is focused on soft or secondary data while the former uses premises from ideas or existing knowledge claims as a form of data.

Construct validity is applicable in the

structuration stage, where the premises are being made explicit, in order to be used within the specific tool or technique chosen, such as formal logic, mathematics or computer simulation. Just as in internal validity, it is based on the coherence structure of the premises and the way they are formulated and structured in order to explain knowledge claims.

External validation is focused on the extension

of the results beyond the settings in which they were obtained, for instance, from sample to population; in the rationalist perspective it is focused on the applicability to the context from which the premises were derived.

This approach presents a challenge in terms of

validation similar to the ones in qualitative research, where the final outcome is highly dependent on the inquirer’s perspective, biases and assumptions. Questions such as degree of coherence, completeness and parsimony come to mind when arguing the minimum conditions required for something to be considered knowledge. In literature there are no criteria suggested to address these questions. Some approaches explored by the researchers include:

• After exhaustive exploration and formulation

of an initial model/theory, the number of iterations (additions and subtractions) on the

model/theory during the structuration tends to become a matter of format more than substance. In this stage the theory can be said to be reaching a maturity point.

• When additions or subtractions of premises alter the final theory in a substantial manner in relation to the original research question and the domain in which it was formulated. For instance, going to a different level (deeper or higher) of inquiry that takes the theory to a different domain or context than where the question was initially formulated.

• When the theory compared to existing theories or situations explain, illustrate or even predict conditions and outputs that the initial theories did not address.

• Documenting the inductive process, biases and assumption of the inquirer, and the reason why some premises are more relevant than others to warrant their choosing by the inquirer.

7 Method Development

A methodology provides the foundation for the development of robust methods. It is the methods, however, that provide the scholar the necessary basis for a robust defense against criticism. It is imperative in a rationalist method, that approaches be identified to support coherence; and consequently providing the necessary supporting arguments for validity of the conclusions that are drawn.

As is the case in any research, when simple conditions are encountered, the approaches attain an extremely high level of formality. As the problems that we address increase in complexity it becomes increasingly difficult to maintain the formality. A simple illustration of this can be presented in the argument: If A = B and B = C, then A = C. By relaxing the conditionality of the relationships the argument rapidly becomes possibilistic in nature. For example; if A is not significantly different to B, and B is not significantly different from C, often leads to the incorrect conclusion that A is not significantly different to C, when in fact the only valid conclusion that can be drawn is that either A is not significantly different to C, or A is significantly different to C. This possibilistic condition is fundamentally acceptable in the rationalist paradigm. As problems become more intricate, methods are necessary to explore the diversity of possibilities, and to develop conclusions from the patterns that emerge in these conditions.

As conditions progress from deterministic, to probabilistic, possibilistic convergence, to possibilistic divergence, the manner in which conclusions are drawn changes as well. In a convergent condition, conclusions can be made directly from the possible states that emerge. In Divergent conditions conclusions are drawn from the patterns that can be discerned in the states. This requires a degree of interpretation where coherence is difficult to establish formally. The models used in such cases (e.g. Agent Based Modeling and System Dynamics) ensure that coherence is maintained and can be supported.

Figure 3. Approach Development for the Rational Inductive Methodology

Figure 3 presents the mapping between the

induction-based methodology and the research process that evolves from this using modeling and simulation techniques. The modeling and simulation techniques form the heart of the justification process by ensuring that the coherence of the belief system is maintained in a manner that all assumptions, premises, and presuppositions are explicit and hold together with the new theory.

Core to this process is the construction of the

model, the verification of the model, and exploration of the model via different simulation scenarios or analytical techniques. As Davis, Eisenhardt, and Bingham [16] mention: “While verification confirms existing theory and increase

internal validity, experimentation adds the creative and theoretical contributions that build new theory”. It allows for the exploration of the New Theory in the created context by varying parameters values, adding or subtracting parameters or creating new combination of the existing ones.

8 Conclusions

SoSE is an emerging discipline. Axiological and methodological structures must be addressed to delineate its growth and ultimately transfer into practice. Conditions that common to the System of Systems context must be incorporated in the axiological that are formed. System of Systems problems are inherently complex; exhibiting typical emergent, non-monotonic, non-ergodic behaviors that must be addressed in terms of possibilities rather than probabilities.

The main purpose of the paper is to define

specific methodological guidelines that may be used in research where complex problems need to be addressed. A Rationalist Inductive approach is proposed that utilizes modeling and simulation. The modeling and simulation does not rely on correspondence to empirically observed phenomena. Rather modeling and simulation are used to capture relationships based on a context that is theoretically constructed. In this manner the modeling and simulation ensure the coherence of the arguments presented.

This rationalist inductive approach

complements other research methodologies, such as the hypothetico-deductive methodology. It overcomes challenges confronted by other methodologies

Acknowledgements: The authors would like

to thank Van E. Brewer and Samuel F. Kovacic for their contributions to the ideas of this paper. References [1] Henrie, M.; Delaney, E.E. (2005). Towards a

common system of systems vocabulary. IEEE International Conference on Systems, Man and Cybernetics, 2005 3, pp.10-12

[2] Gettier, E. (1963). Is Justified True Belief Knowledge? Analysis (23), 121-123.

[3] Popper, K. (1968). The Logic of Scientific Discovery. London: Hutchinson.

[4] Thilly, F. (1903). The Theory of Induction. The Philosophical Review, 12(4), 401-411.

[5] Arthur, B. W. (1994). Inductive Reasoning and Bounded Rationality. The American Economic Review, 84(2), pp. 406-411.

[6] Nonaka, S. and Takeuchi, N. (1995). The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. New York, NY: Oxford University Press.

[7] Dauer, F. (1974). In Defense of the Coherence Theory of Truth. The Journal of Philosophy, 71(21), 791-811.

[8] Walker, R.C.S. (1985). Spinoza and the Coherence Theory of Truth. Mind, 94(373), 1-18.

[9] Firth, R. (1964). Coherence, Certainty, and Epistemic Priority. The Journal of Philosophy, 61(19), American Philosophical Association Eastern Division Sixty-First Annual Meeting., pp. 545-557.

[10] Keating, C.B., Unal, R., Sousa-Poza, A., Dryer, D. Rogers, R., Safford, R., Peterson, W., Rabadi, G., (2003). System of Systems Engineering. Engineering Management Journal, 15(3), 36-45.

[11] Schmid, A. (2005). What is the Truth of Simulation? Journal of Artificial Societies and Social Simulation, 8(4) Retrieved on January 25 of 2008 from <http://jasss.soc.surrey.ac.uk/8/4/5.html>

[12] Gilbert, N. (2000). Models, Processes and Algorithms: Towards a Simulation Toolkit. In Suleiman, R., Troitzsch, K., Gilbert, N. (Eds). Tools and Techniques for Social Science Simulation. pp. 3-16. Germany: Physica-Verlag.

[13] Leedy, P.D. and Ormrod, J.E. Practical Research, Planning and Design. 7th Ed. Merrill, Prentice Hall.

[14] Black, J. A. & Champion, D. J. (1976). Methods and Issues in Social Research. John Wiley & Sons, Inc. pp: 59-61

[15] Creswell, J.W. and Miller, D.L. (2000). Determining Validity in Qualitative InquiryTheory into Practice. 39(3), Getting Good Qualitative Data to Improve Educational Practice. pp. 124-130.

[16] Davis, J., Eisenhardt, K., and Bingham, C. (2007). Developing Theory Through Simulation Methods. Academy of Management Review, 32(2), 480-499.