alignment-based precision checking a. adriansyah 1, j. munoz gamma 2, j. carmona 2, b.f. van dongen...
TRANSCRIPT
Alignment-based Precision CheckingA. Adriansyah1, J. Munoz Gamma2,
J. Carmona2, B.F. van Dongen1,
W.M.P. van der Aalst1
Tallinn, 3 September 2012
1) Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands
2) Software Department, Universitat Politècnica de Catalunya, Spain
2
Model needs to be precise
Event Log
Process Models
a da b
bc
end
c
a b
d
end
aabdabdacd
aabdc
ad
Unfitting
How to measure precision for unfitting event logs?
d
3
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting TracesAlignment
Precision Metrics
4
Optimal alignment: least #deviations
a da b
bc
end
Trace adab
aAlignment
d ba a b
ad
Process Model
Fitting trace, closest to the original
Deviation
Deviation
Log Moves
Model Moves
aabd Reference: W.M.P. van der Aalst, A. Adriansyah, B.F. van Dongen (2012). Replaying history on process models for conformance checking and performance analysis. WIREs Data Mining and Knowledge Discovery, 2(2), 182-192.
5
Optimal alignments can be many
a da b
bc
end
Trace ad
Alignment 1 Alignment 2
Process Model
baa d
d caa d
d
Both alignments are optimal
abd or acd
6
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments Precision Metrics
aabdaabdacd…
Fitting TracesAlignment
7
a
b
b
a
d
24 4
2 2 2
d 2
aabdabdabd or acdaabd
1-Alignment automaton (Prefix-based)
Fitting Traces
a da b
bc
end
Process Model
1-Align Automaton
aabdabdabdaabd
8
b d
b
a
ad
4 4
2 2 2
1-Alignment automaton (Prefix-based)
0c
Fitting Traces
a da b
bc
end
Process Model
1-Align Automaton
aabdabdabdaabd
Imprecision
2 2
…
…
9
a22 3
db1.51.5 11
b1a
d
3
All-Alignment Automaton (Prefix-based)
c
Fitting Traces
a da b
bc
end
Process Model
All-Alignments Automaton
aabdabdabd or acd aabd
1 1
0.5 d 0.5
4 4
2 2 2
10
Comparison of log automata
aabdabdadadab
Event Log
a da b
bc
end
Process Model
Without alignment
1-Align
All-AlignPre
fix-
bas
ed L
og
Au
tom
ata
aabdabdabd/acd aabd
Fitting Traces
11
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting TracesAlignment
Precision Metrics
Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in Process Conformance. 8th International Conference of Business Process Management (BPM): 211-226.
Use calculation
12
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting Traces
Precision Metrics
Alignment
13
Implementation in ProM 6
• Package: “ETConformance”
14
Experiment settings
Event Log
Flower (F)
Precise (P)
Flower’ (F’)
Precise’ (P’)
Duplicate Log Combined Log
Precise-Precise (PP’)
Precise-Flower (PF’)
Flower -Precise(FP’)
Flower-Flower(FF’)
15
Experiment 1: Measuring precision stability
Perfectly fitting logs and models Non-fitting logs and models
P FP’ PF’ PP’F P FF’ FP’ PF’ PP’
Values of change significantly, but not and
16
Experiment 2: Sensitivity to unfitting trace
• fluctuates when traces are unfitting
Model P
0.80
0.85
0.90
0.95
1.00
0 1 2 3 4
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Prec
isio
n
Number of Removed Events
Noise Sensitivity - P
ETC
1-Align
All-Align𝑎𝑝❑𝑎𝑝1
𝑒𝑡𝑐𝑝❑
Pre
cisi
on
#Removed events
0.000.200.400.600.801.00
0 1 2 3 4
Model PF’
Pre
cisi
on
#Removed events
0.00
0.10
0.20
0.30
0.40
0 1 2 3 4
Pre
cisi
on
#Removed events
Model FP’
0.90
0.93
0.95
0.98
1.00
0 1 2 3 4
Pre
cisi
on
#Removed events
Model PP’
17
Experiment 3: Real life cases
• Using 5 logs and models from municipalities in the Netherlands
• Computation time of is much shorter than , yet provide close estimation
18
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting Traces
Precision Metrics
Alignment
19
Conclusion and Future Work
• Pre-alignment provides more accurate precision measurements in cases where logs are non-fitting
• Applicable to models with duplicate tasks/unlogged activities
• gives close estimation to with lower computation time• Log-Model evaluation needs to consider multiple
dimension of conformance• Potential use of the technique: auditing, conformance
diagnosis, process repair
20
Questions
21
Thanks!