The Simple Elegance of Causal Enquiry
Robert O. BriggsInstitute for Collaboration ScienceUniversity of Nebraska at Omaha
Today’s program
Epistemology The Philosophy of Science The Scientific Approach
Three Ways to Think About Academia
The Philosophical The Pragmatic The Publishable
Epistemology:The Philosophical View
A way of knowing A way of creating knowledge
Prevailing Epistomologies
Interpretivism Criticalism Causal enquiry
Interpretivism
Creating knowledge about… The inferences people draw
from and the meanings people ascribe to the words and actions of others.
Key Assumption: No objective reality
Key Discipline: Induction
Criticalism
Creating knowledge about… Social Justice Key Assumption:
Injustice is socially and historically constituted
Systemic injustice can be redressed through action
Causal (Science)
Creating knowledge about… Cause-and-effect Key assumption: Objective
reality Key Discipline: Deduction
Epistemology Myths
Causal and Interpretivist enquiry are mutually exclusive world views
Objective Reality vs. No Objective Reality?
What is Reality?
Epistemology Myths
Causal scholars are (or think they are) objective. Reality is objective Scientists are subjective Validation = Intersubjective concurrence
Interpretivists don’t believe in gravity Interpretivists seek to explain meaning,
not causation.
Epistemology Myths
Interpretivism is qualitative Causal enquiry is
quantitative
Epistemology Myths
An epistemology is something you “are” I’m an interpretivist I’m a positivist
Pragmatic Epistemology
A set of mental disciplines To keep us from drawing (and then
publishing) bone-headed conclusions
No Embarrassment
Epistemology: The Publishing Perspective
Bad PapersAre Forever!
Interpretivism, Criticalism, and Causal epistemology (science) are not mutually exclusive. They are, in fact, interdependent; all three are necessary to a complete understanding of Information Systems
Provocation
The Philosophy of Science
Assumptions of Causal Enquiry
Regular patterns of causation
Independent from human mind
“knowable”
The Boundaries of Science
If it’s not about cause-and-effect…
It’s not science Period.
Goals of Science
Discover and describe phenomena and their correlates
Create causal models for phenomena of interest
Test the usefulness of those models Use those models to increase the
likelihood people will survive and thrive.
Goals of Science Discover and describe phenomena of
interest (Exploratory Science) Create causal models for phenomena of
interest (Theoretical) Test the usefulness of the models
(Experimental Science) Use those models to increase the
likelihood people will survive and thrive. (Applied Science / Engineering)
XKED.com
In which aspect of scientific method is this scientist engaged?
A.ExploratoryB.TheoreticalC.ExperimentalD.Applied Science / Engineering
The three most exciting words in science are, “Gee, that’s funny…”
- Isaac Asimov
•Why did that happen?
•Does it always happen that way?
•Does it have to happen that way?
•Could I make it happen on purpose?
Why Should You care?
Good science will Make it more likely that people will survive and
thrive Make you work smart Get you published in good journals
Bad Science will Harm others Waste effort, time, and money Embarrass you for years…
The Causal Disciplines
Phenomenon-of-Interest Who Cares? Theory Hypotheses Research Methods Analysis
The First Discipline
Explicitly Define The Phenomenon of Interest
The Phenomenon of Interest
In the world of cause-and-effect… The phenomenon-of-interest is the
EFFECT The EFFECT is what you seek to explain The EFFECT is what you seek to
improve The EFFECT is the outcome you
measure
The First DisciplineDefine The Phenomenon of Interest
Explicitly In writing(!) Refine the definition as your
understanding deepens Challenge your definition
continuously
Explicitly Define thePhenomenon of Interest
Satisfaction First definition:
The degree to which needs are fulfilled Measures
I am satisfied My needs are fulfilled I feel satisfied
Better definition A valanced affective arousal with respect to goal
attainment Measures
I feel satisfied with… …gave me a feeling of satisfaction I feel good about…
Phenomenon vs. Domain:The Philosophical View
The phenomenon-of-interest is the OUTCOME you hope to improve measurably Productivity Creativity
The domain is the setting in which the outcome manifests Requirements Negotiation Data Mining
Phenomenon vs. DomainThe Pragmatic View
You study the phenomenon of interest …Don’t ever forget it
You sell the domain To funding agencies To reviewers To readers To yourself
The Second Discipline
Who Cares?!?
Who Cares?!?
Why is this phenomenon-of-interest is worthy of study?
Philosophical “Who cares?”
A goal of scientific enquiry is to increase the likelihood that people will survive and thrive
Society provides the scarce resources for scientific enquiry. You must be able to justify your use of them.
Pragmatic “Who Cares”
Reviewer Perspective: “What I do is important. What you do is trivial.”
Your reviewer just had a “much better” paper rejected by the same journal.
Publishable “Who Cares?”
1. The phenomenon of interest is worth studying1.1 People are more likely to survive
and thrive if we understand the cause of this phenomenon
1.2 The existing literature does not fully explain the causes of this phenomenon
Publishable “Who cares?”
You must define explicitly the phenomenon-of-interest in the who-cares argument
It’s your anchor for all that follows You are the most important target
for the argument
Good “Who Cares?” 1.1 Organizations exist to create value for
stakeholders Organizations operate under risk Mitigate risk, the organization may survive Internal risk assessments can mitigate risk Risk assessments must be run by groups If we can make risk assessment groups more
productive, we may increase that people will survive and thrive!
Productivity is…. This study examines the use of GSS to make
risk assessment groups more productive.
Bad “Who cares” 1.1 Organizations do risk assessments
frequently We studied collaborative risk
assessment workshops
Ugly “Who Cares” 1.1
We collected some data about risk assessment workshops
Good “Who Cares 1.2” Connolly et al (1992) showed that productivity
of brainstorming teams could be improved by making them anonymous.
However, Johnson and Stephens (2003) found higher productivity when brainstorming teams were identified
A causal theory of productivity might be useful for explaining these seemingly disparate results, and might allow the development of even better brainstorming techniques.
This paper offers and tests such a theory
Bad “Who Cares” 1.2 Jones (1983) said nothing has been
done about productivity Smith (1978) called for more
research on productivity Johnson (1981) studied productivity
among factory workers I studied productivity among
brainstorming groups
Ugly “Who Cares” 1.2
I searched 3 on-line databases and browsed 6 web search engines and only found 2 articles on this topic.
Little is known about this topic Nobody has studied this topic yet.
Publishable Causal enquiryThe Opening Argument
Section 1. Who Cares?!?Argument: This phenomenon is worth studying.
1.1 People will be better off if we understand this phenomenon
1.2 Current literature does not yet fully explain it
The Third Discipline
Deriving An Explanatory
Theory
‘theory’ With a Small ‘T’
Exploratory Taxonomies Frameworks Descriptive models Correlational models (grounded theory)
Applied Design Theory (design methodologies)
Explanatory Theory
A causal model to explain variations in the phenomenon-of-interest
Data have no scientific meaning
except with respect to
the Explanatory Theory
from which they spring
Today’s Message:Today’s Message:
Goals of Science Discover and describe phenomena of
interest (Exploratory) Create causal models for phenomena
of interest (Theory) Test the usefulness of the models
(Experiment) Use those models to increase the
likelihood people will survive and thrive. (Application)
Anything Missing?
TruthTruthTruthTruth
Anything Missing?
Positivist Perspective
Science = Useful Science = Useful
Science <> TrueScience <> True
A useful model is better than Truth
Useful Is Better Than True
Useful Is Better Than True
Name the PhenomenonName the Phenomenon
BobeziteBlock
Describe the PhenomenonDescribe the Phenomenon
BobeziteBobeziteBlockBlock
A
B
Explore the PhenomenonExplore the
Phenomenon
BobeziteBobeziteBlockBlock
A
B
BobeziteBobeziteBlockBlock
Explore the PhenomenonExplore the
Phenomenon
BobeziteBobeziteBlockBlockBobeziteBobezite
BlockBlockBobeziteBobeziteBlockBlockBobeziteBobezite
BlockBlock
A
B
BobeziteBobeziteBlockBlock
A
B
Describe the dynamics of the phenomenon
Describe the dynamics of the phenomenon
A Useful ModelA Useful Model
One Gear
TruthTruth
One Thousand Gears
When does the Model Become Useful?
When does the Model Become Useful?
When you want todo something newWhen you want todo something new
Therefore
For matters of cause-and-effectA useful model (Theory)
is better than Truth
An experiment, without a Theory is
meaningless
What is a Theory?
An excuse to not do anything meaningful?
Pie-in-the-sky disconnect from reality?
There is nothingmore useful
than a good theory
Theory All Drives:
Hypothesis Experimental design Measures Treatments Statistics
What is a theory?
Causal Model Internally Consistent Explains and/or predicts Proposes mechanisms of causation Testable
Structure of a Theory
Axioms Propositions
Axioms
Assumptions about the fundamental nature of the universe
The beginning of suppositional logic What if we were to assume the world
works like X…would that explain observed variations in Y?
Axioms Are “Received”
Source is irrelevant Axioms cannot be derived or defended Feynman’s Inspiration
Example Axioms
Axiom 1: Human attention is limited Axiom 2: A subconscious cognitive mechanism ascribes utility to salient goals
Propositions Functional Statements of cause-and-
effect that must be logically true if the axioms are true
Force is a function of Mass and Acceleration (F=MA)
Satisfaction is a function of shifts in yield for the set of salient goals.
n
jj
m
ii YYfS
11
Propositions are...
Causal Composed of constructs Without empirical content Logically derivable from
axioms
Propositions of Direct Causation
Proposition 1: Productivity is a function of effortProposition 2: Effort is a function of goal congruenceProposition 3: Effort is an inverse function of distraction
ProductivityEffort
Distraction
Goal Congruence
+
-
+12
3
Debugging A Theory
EffortDesire for outcome
+1
Proposition 1: Effort is a function of desire-for-outcome.
Propositions of Moderating Causation
EffortPerceived
Effort Required
Desire for Outcome
Mathematical Notation of Propositions
P = (E)Where
P = Productivity
E = Effort
E = -(D)Where
E = EffortD = Distraction
Problematic Propositions
WORK DESIGN & EXECUTION OUTCOMES
IMPLICIT INCENTIVES
EXPLICITINCENTIVES
SOCIAL ENVIRONMENT
TECHNICAL ENVIRONMENT
RESOURCE ENVIRONMENT
ORGANIZATIONAL STRUCTURE
ENVIRONMENT
DISTRIBUTED WORK
ARRANGEMENT
ORGANIZATIONAL LEVEL
GROUP LEVEL
INDIVIDUAL LEVEL
INCENTIVE STRATEGY
(e.g. Reward & Compensation)
WORK DESIGN & EXECUTION OUTCOMES
IMPLICIT INCENTIVES
EXPLICITINCENTIVES
SOCIAL ENVIRONMENT
TECHNICAL ENVIRONMENT
RESOURCE ENVIRONMENT
ORGANIZATIONAL STRUCTURE
ENVIRONMENT
DISTRIBUTED WORK
ARRANGEMENT
ORGANIZATIONAL LEVEL
GROUP LEVEL
INDIVIDUAL LEVEL
INCENTIVE STRATEGY
(e.g. Reward & Compensation)
Qualities of a Good Theory
Parsimony Explanation/Prediction Boundaries
Pragmatic Theory
You usually start with propositions and work backward to axioms
You usually start badly and get better
You use someone else’s theory whenever you can
Your technology is not in your theory
Pragmatic Theory
A good theory will get you to the moon and back safely on the first try
Good theory will do more to save you from drawing bone-headed conclusions than any other discipline of Causal enquiry
Good theory will make you look like a genius
Publishing PerspectiveAlternative Wordings for Propositions
Y is a function of Z Z causes Y Z determine Y The more Z you do, the more Y you
get Z has a positive influence on Y
Publishable Causal enquiry
Section 2. TheoryArgument: I understand what causes Y
If we assume that:Axiom 1: The world is like X
Then it must be that: Proposition 1: Y is a function of Z.
The Fourth Discipline
Deriving Hypotheses
The Fourth DisciplineHypotheses Comparative statements
Contrasts value of a dependent variable across at least two treatments that instantiate differing values of the independent variable
Dependent Variable Measures consequent construct of a proposition
Independent Variable Invokes differing levels of causal
construct
Hypotheses
MUST be logically derived from propositions
Test the proposition Should have empirical content
Example Hypothesis
H1: Brainstorming teams with access to an automated feedback graph will produce more unique ideas than teams with no automated graph
Example Hypothesis
H2: During brainstorming, the more we pound randomly on the walls, the fewer unique ideas a team will produce.
Problematic Hypotheses
H3: Groups using richer media will exhibit higher levels of cohesion initially
Problematic Hypotheses
H4: On negotiation tasks, face-to-face groups will outperform computer mediated groups, will experience less process difficulty, than computer-mediated groups, and will have more favorable reactions to their group task performance, interaction process, and communication medium
Publishable Causal enquiry
Section 4. HypothesesArgument: This theory is testable
If, as Proposition 1 posits, Y is a function of Z, and if I do W with Technology-1 to invoke higher values of Z then it must be that: H1. People doing W with Technology-1 will score higher on the Y-test than people doing NOT-W
The Fifth Discipline
Experimental Research Methods
Experimental Design
Construct Validity Statistical Validity Internal Validity External Validity
An experiment without a theory is meaningless
Today’s Message:Today’s Message:
Experiment
Compare outcomes Different treatments Control other possible causes
Experimental InquiryExperimental Inquiry
TreatmentTreatment11
TreatmentTreatment11
TreatmentTreatment22
TreatmentTreatment22
IdenticalSubjectPools
IdenticalSubjectPools
ResultsResultsResultsResults
ResultsResultsResultsResults
}} CompareCompare
Investigative InquiryInvestigative Inquiry
PopulationPopulation11
PopulationPopulation11
PopulationPopulation22
PopulationPopulation22
ResultsResultsResultsResults
ResultsResultsResultsResults
}} CompareCompareOneTreat-ment
OneTreat-ment
Positive Experimental Results may mean...
Manipulation caused difference Hypothesis has support Theory has support
Negative Results Mean
Experiment Flawed? Hypothesis Flawed? Propositions Flawed? Axioms Broken?
The Only Scientific Truth
The Model is No Good
Publishable Causal enquiry
Section 4. MethodsArgument: I found a reasonable way to test the hypotheses
4.1 My DV instantiates the phenomenon of interest4.2 My IV instantiates a causal construct4.3 My approach would reveal a difference if there were one4.4 There are few alternative explanations for any difference discovered
An Experiment without a theory is
meaningless
Phenomena: Phenomena: Large, Odd-Smelling BoxesLarge, Odd-Smelling Boxes
Scientific Instrument: Scientific Instrument: DrillDrill
Collecting Data without A TheoryCollecting Data without A Theory
Collecting Data Without A TheoryCollecting Data Without A Theory
Collecting Data Without A TheoryCollecting Data Without A Theory
Collecting Data With a TheoryCollecting Data With a Theory
Collecting Data With a TheoryCollecting Data With a Theory
Collecting Data With a TheoryCollecting Data With a Theory
Collecting Data With a TheoryCollecting Data With a Theory
Collecting Data With a TheoryCollecting Data With a Theory
A Physicist Uses the A Physicist Uses the Elephant TheoryElephant Theory
+
=Fission!
A Farmer Uses the Elephant A Farmer Uses the Elephant TheoryTheory
A Farmer Uses the TheoryA Farmer Uses the Theory
There is nothingmore useful
than A Good Theory
An Experiment without a theory is
meaningless
Data have no meaning except in reference to the
theory from which they spring.
Points to Ponder
You don’t have to measure a cause, you only have to manipulate it.
You have to measure effects For experimental research you must have a
theoretical explanation for every effect you measure
Not required for exploratory work
Scientific Method
Discover Phenomenon Theorize Hypothesize Fastest Falsifications Experiment Conclude Apply
Truth
Powerful theory will outperform powerful statistics every time!
Truth There is No Perfect Study You must pilot your experiments
Truth No Theory is made or broken by a single study
Remember
Experiments without theories are meaningless
Remember
Data Have No Meaning except in reference to the
theory from which they spring
Worth Reading Stebbins, Robert. Exploratory Research in the
Social Sciences. 2001. Popper, Karl -- The Logic of Scientific Discovery
(Any edition from 1934-1981) Shadish, W.R., Cook, T.D., & Campbell, D.T.
(2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton-Mifflin.
Hevner, A. and Chatterjee, S. (2010). Design Science in Information Systems: Theory and Practice.