bpm cluster meeting 2014

1/31www.janclaes.info

FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION

Jan ClaesTeaching assistant : PhD 2009 – 2015

Supervisors UGent : Geert Poels & Frederik GaillySupervisors TU/e : Paul Grefen & Irene Vanderfeesten

Why do people struggle with the complexity of constructing a process model?


Business Process Modeling (BPM)Assumptions

Constructing process models is useful Constructing the models is cognitively challenging

Business Process Modeling (BPM) Process model quality

“what is a good process model?” Process of process modeling (PPM)

“how to make a good process model (effectiveness) cccccccc using a good approach (efficiency)?”



How do people construct process models?


PPMChart visualization

CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITY DELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE

Start event Edge Activity

Gateway

Edge

Activity

Edge

Edge

Activity

Edge Gateway

Edge

123

5

7

9

4

6

8

1112

10

time

mod

el e

lem

ents


Flow-oriented modeling


Aspect-oriented modeling


Undirected modeling


Observations (objective)

Obs. 1. Majority used serialized approachObs. 2. Large group flow-oriented modelingObs. 3. Small group aspect-oriented

modelingObs. 4. Large group combine FO and AOObs. 5. Small group undirected modelingObs. 6. Remainder is uncategorizedObs. 7. Undirected sessions lasted longer



What about the quality of the corresponding models?


Impressions (subjective)

Impr. 1. Modelers need serializationImpr. 2. Structured serialization

helps avoiding ‘mistakes’Impr. 3. Structured serialization

does not support every modeler to the same extent



Is there a rational for these observations and impressions?


Kinds of human memory

input Sensory memory

Workingmemory

Long term memory

performanceOBSERVATION SELECTION

ORGANIZATION

RETRIEVALTRANSFER

Workingmemory


Types of cognitive load

Time

Inst

anta

neou

s lo

ad

overall load

intrinsicload

extraneousload

germane load

freecapacity

assumed capacity

instantaneous load

overload


Cognitive Load Theory (CLT)

case complexity prior knowledgeinput material

representation fitworking memory

capacity

extraneous cognitive load

germane cognitive load

schema construction

process model quality

A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term

overall construction effort

cognitive overload

intrinsic cognitive load

++

+

+

+++

+ +

–

– – –

–

–


Cognitive Fit Theory (CFT)

Need for structure

Learning styleField dependencySensory

Visual

Inductive

Active

Sequential

Intuitive

Auditory

Deductive

Reflective

Global

I ♥︎� chaos I #$@&! chaos

DON’T SEE THE FOREST THROUGH THE TREES



What can we do with this knowledge?


ImpressionsCognitive theoryObservations

Theory building

Structured Process Modeling Theory



Structured Process Modeling Theory (SPMT)

A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B

learning style

degree of serialization

adopted serialization style

need for structure

field-dependency

– +

intrinsic cognitive load for modeling phases

intrinsic cognitive load for aggregation phases

cognitive overload

intrinsic cognitive load for strategy building phases

+ + +

serialization style fitstructuredness of serialization

– – – –

1 2 3



How will the developed theory be tested?


Methodology1. Cognitive tests prior to the test session

Collect modeler data2. Observational modeling session

At TU/e in BPM lecture of September 26 Collect modeler, modeling and model data

3. Post-experiment interviews with selection of participants (e.g.,

outliers) Collect additional, qualitative data


Experiment set-upNo treatmentMeasure every involved construct of the theory

Modeler, modeling process, process modelCalculate (cor)relations of the theory


PRIOR TO THE EXPERIMENT SESSION

HOMEWORK


Experiment set-up1. Prior to the experiment session

Software to be installed on personal laptop ( = download from website and unzip)

5 cognitive tests (time range 5-30 min)• Reading span test (RSPAN)• Operation span test (OSPAN)• Counting span test (CSPAN)• Hidden figures test (HFT)• Need for structure (NFS)



To be completed at home Before deadline Software collects data in one-zip file per test

which participants have to submit



Students will receive their individual results as incentive to not cheat on these tests

Late responses or too much errors reveals cheating Student is not allowed to start actual experiment if

the test results are not submitted in time


AT THE EXPERIMENT SESSION

MODELING TASK


Experiment set-up2. At the experiment session

Modeling task and post-task survey In the same, already installed program

(using a different start-up code) Software automatically collects data

• Each operation in the tool ( = each activity in the modeling process)

• Constructed process model, number of operations, modeling time

in one-zip file which they have to submit


Experiment set-up2. At the experiment session

Student does not get credit if not all data (6 zip files) are submitted

Technical problems > replacement assignment(e.g., help me with coding the collected data)


AFTER THE EXPERIMENT SESSION

INTERVIEW


Experiment set-up3. After the experiment session

Data will be analyzed and outliners will be defined Selection of students will be invited for post-

experiment interview (voluntarily) Collect qualitative data

• Explain how they have modeled• Explain their cognitive profile• Explain which results we expected and why• Ask about their opinion: why did they (not) achieve the

predicted results?


Thanks for you attention!

> Do you have feedback on my research plans? <> set-up is useful for theory testing? <

> set-up is feasible (enough)? <

> other suggestions? <

Does someone has some time to test the software?

Jan [email protected]

http://www.janclaes.infoTwitter: @janclaesbelgium



How can the concepts of the theory be measured?





capacity



schema construction




cognitive overload


++

+

+

+++

+ +

–

– – –

–

–





representation fit



schema construction



cognitive overload


++

+

+

+++

+ +

–

– – –

–

–

working memory capacity

Reading span - Operation span - Counting span

Reading spanFor many years, my family and friends have been working on the farm. SPOT

Because the stuffy was room, Bob went air for some fresh outside. TRAILWe were fifty miles out at sea before we lost sight of the land. BAND

Counting span

Count the dark blue circles

Operation spanIS (8 / 4) – 1 = 1 ? BEAR

IS (6 X 2) – 2 = 10 ? BEANSIS (10 X 2) – 6 = 12 ? DAD




case complexity prior knowledgeworking memory

capacity



schema construction



cognitive overload


++

+

+

+++

+ +

–

– – –

–

–

input material representation fit

Fit with modeling task = constantFit with modeler through questions

Example questionsI did experience problems using the modeling tool (5-point Likert)

Using the modeling language was not difficult (5-point Likert)I found it easy to understand the case description (5-point Likert)



prior knowledgeinput material


capacity



schema construction




cognitive overload


++

+

+

+++

+ +

–

– – –

–

–

case complexity

Case complexity = constant(number of activities and arrows in solution model can be used as proxy for number of concepts and relations)




case complexityinput material representation fit

working memory capacity



schema construction



cognitive overload


++

+

+

+++

+ +

–

– – –

–

–

prior knowledge

Prior domain, modeling language and modeling method knowledgethrough questions

Example questionsI am familiar with the mortgage handling process (5 point-Likert)

How many process models did you ever create?I am experienced in process modeling (5-point-Likert)






capacity

schema construction



++

+

+

+++

+ +

–

– – –

–

–



cognitive overload


Through questions

Example questionsI found it hard to complete the task (5-point-Likert)





capacity






cognitive overload


++

+

+

+++

+ +

–

– – –

–

– schema construction





capacity



schema construction



cognitive overload


++

+

+

+++

+ +

–

– – –

–

–


Syntactical quality: number of cognitive mistakesSemantic quality: assessment by experts using checklist (penalty for over- and underfit)





capacity



schema construction



cognitive overload


++

+

+

+++

+ +

–

– – –

–

–


Time between start of reading case description and last operation in the toolNumber of operations




learning style



need for structure

field-dependency

– +



cognitive overload


+ + +


– – – –




learning style



need for structure

field-dependency

– +



cognitive overload


+ + +


– – – –

Through questions

Example questionsWhen I write a text:

• I usually work on content, structure, wording and format right from the start• I usually first decide on the content, then the structure, next I write sentences

and finally I work on the formatting




learning style



need for structure

field-dependency

– +



cognitive overload


+ + +


– – – –

Through questions

Need for Structure scale12 questions with 6-point Likert scale




learning style



need for structure

field-dependency

– +



cognitive overload


+ + +


– – – –

Hidden Figures Test




learning style


need for structure

field-dependency

– +



cognitive overload


+ + +

serialization style fit– – – –


structuredness of serialization

Assessment by experts

Flow-oriented – Aspect-oriented – CombinationHappy Path First – Undirected

Uncategorized




learning style



need for structure

field-dependency

– +



cognitive overload


+ + +

structuredness of serialization

– – – –

serialization style fit

Theoretical matchPost-experiment interview

Global learner = aspect-oriented, sequential = flow-orientedHigh need for structure = no undirected

Field-dependent = flow-oriented




learning style



need for structure

field-dependency

– +

+ + +


– – – –



cognitive overload


See earlier: through questions

bpm cluster meeting 2014

Business

modeling process

process models

faculty of economics

cognitive overloadwww

test session

observational modeling

business administration931what

business administration1131is