on realism of architectural procedural modelsjaroslav/papers/2017-realism/2017-benes... · –what...
TRANSCRIPT
-
On Realism of
Architectural Procedural ModelsJan Beneš, Tom Kelly, Filip Děchtěrenko, Jaroslav Křivánek, Pascal Müller
-
Quick Quiz
2Jan Beneš - On Realism of Architectural Procedural Models
-
Quick Quiz
3Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Photograph
-
Overview
4Jan Beneš - On Realism of Architectural Procedural Models
Procedural Modeling&
Realism
Questions Previous Work
Experiment &
Data Acquisition
Answers
-
Procedural Modeling
• Procedural modeling– Tool for artists
– Or fully automatic
– Wide range of outputs
• Wide use
5Jan Beneš - On Realism of Architectural Procedural Models
[Independence Day]
[Palubicki09]
-
Procedural Modeling
• Procedural modeling– Tool for artists
– Or fully automatic
– Wide range of outputs
• Wide use
6Jan Beneš - On Realism of Architectural Procedural Models
[Independence Day]
[Palubicki09]
-
Realism
Realism
Plausibility
Immersion
• Model & material & display
• Should be studied
• For procedural modelling
7Jan Beneš - On Realism of Architectural Procedural Models
[Velocipedia, Gianluca Gimini]
-
Realism
Realism
Plausibility
Immersion
• Model & material & display
• Should be studied
• For procedural modelling
8Jan Beneš - On Realism of Architectural Procedural Models
[Velocipedia, Gianluca Gimini]
-
What we did
9Jan Beneš - On Realism of Architectural Procedural Models
Procedural
architecture
-
What we did
10Jan Beneš - On Realism of Architectural Procedural Models
User Study
-
Question #1
(1) Can people tell procedurally generated
buildings from real ones?
11Jan Beneš - On Realism of Architectural Procedural Models
-
Question #2
(2) Is realism carried in the detail or in the
larger structure?
12Jan Beneš - On Realism of Architectural Procedural Models
-
Question #2
(2) Is realism carried in the detail or in the
larger structure?
13Jan Beneš - On Realism of Architectural Procedural Models
-
Question #2
(2) Is realism carried in the detail or in the
larger structure?
14Jan Beneš - On Realism of Architectural Procedural Models
-
Question #3
(3) What factors do the users think influence
the perception of realism?
15Jan Beneš - On Realism of Architectural Procedural Models
I think…
-
Questions
(1) Can people tell procedurally generated buildings from real ones?
(2) Is realism carried in the detail or in the larger structure?
(3) What factors do the users think influence the perception of realism?
16Jan Beneš - On Realism of Architectural Procedural Models
I think…
-
Previous Work
17
-
Previous Work - Realism
18Jan Beneš - On Realism of Architectural Procedural Models
“There is no dirt, no dust, no fingerprints on the glass table…simply too beautiful, too clean and polished…” [Reinhard13]
-
Previous Work - Realism
19Jan Beneš - On Realism of Architectural Procedural Models
CG2Real [Johnson11]
-
Previous Work - Realism
20Jan Beneš - On Realism of Architectural Procedural Models
[Rademacher01]
-
Previous Work - Buildings
21Jan Beneš - On Realism of Architectural Procedural Models
• Building generation
– Grammar [Stiny80, Wonka03, Muller06, Schwarz15]
– Data [Fan16]
– Sketch [Nishida16]
– Predefined parts [Kalogerakis12, Talton12]
-
Previous Work - Miscellaneous
22Jan Beneš - On Realism of Architectural Procedural Models
• Machine Learning– What makes Paris look like Paris [Doersch12]
– Architectural Style Recognition [Mathias11]
• Image Quality & Similarity– Visible Differences Predictor [Daly92]
– Visual Equivalence and Aggregates [Ramanarayanan07,08]
– Structural Similarity - SSIM [Wang04]
-
Experimental Setup
23
-
Datasets
24Jan Beneš - On Realism of Architectural Procedural Models
Favela Medieval Paris Venice
-
Datasets
25Jan Beneš - On Realism of Architectural Procedural Models
Favela Medieval Paris Venice
Photos
Computer
Generated
-
X
Classification Screen
26Jan Beneš - On Realism of Architectural Procedural Models
X
-
Blurs
27Jan Beneš - On Realism of Architectural Procedural Models
13px blur7px blur1px – no blur
55px blur32px blur25px blur
-
Summary
28Jan Beneš - On Realism of Architectural Procedural Models
TRAINING
-
Summary
29Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
TRAINING
-
Summary
30Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
TRAINING
-
Summary
31Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
Venice-1px
Medieval-1px
Paris-1px
Favela-1px
TRAINING
…
-
Summary
32Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
Venice-1px
Medieval-1px
Paris-1px
Favela-1px
DEBRIEFING
TRAINING
…
-
Data preparation
33
-
Photographs
34Jan Beneš - On Realism of Architectural Procedural Models
-
Renders
35Jan Beneš - On Realism of Architectural Procedural Models
-
Data Characteristics
36Jan Beneš - On Realism of Architectural Procedural Models
300px
300px
-
Data Characteristics
37Jan Beneš - On Realism of Architectural Procedural Models
300px
300px
Choose subset
-
Quick Peek at Results
38
-
Quick Results
39Jan Beneš - On Realism of Architectural Procedural Models
random choice = 0.5
-
Quick Results
40Jan Beneš - On Realism of Architectural Procedural Models
random choice = 0.5
-
Quick Results
41Jan Beneš - On Realism of Architectural Procedural Models
random choice
Favela
Best Dataset
Medieval
Worst Dataset
-
Q1: CG vs Real
42
-
Q1: CG vs Real
43Jan Beneš - On Realism of Architectural Procedural Models
• HDIFF: participants can tell photographs and generated buildings apart.
– Random choice = 0.5 accuracy
– No blur, overall – p
-
Q1: CG vs Real
44Jan Beneš - On Realism of Architectural Procedural Models
• HDIFF: participants can tell photographs and generated buildings apart.
– Random choice = 0.5 accuracy
– No blur, overall – p
-
Q2: Details vs Structure
45
-
Q2: Details vs Structure
46Jan Beneš - On Realism of Architectural Procedural Models
• HSCALE: the detail that allows participants
to tell photographs and generated
images apart is present at various scales.
• ANOVA
-
What’s ANOVA?
47Jan Beneš - On Realism of Architectural Procedural Models
Black box
Independent
variable
Independent
variable
Independent
variable
Dependent
variable
-
What’s ANOVA?
48Jan Beneš - On Realism of Architectural Procedural Models
Black box
Blur Size
Original
image class
Dataset
Accuracy
-
What’s ANOVA?
49Jan Beneš - On Realism of Architectural Procedural Models
• ANOVA
– Linear model
– Effects of independent
variables
-
Significant Effects
50Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
-
HSCALE
51Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
-
More on Blur
52Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
-
Post-Hoc - Blur
53Jan Beneš - On Realism of Architectural Procedural Models
Not significantSignificant
-
Blurs
54Jan Beneš - On Realism of Architectural Procedural Models
13px blur7px blur1px – no blur 55px blur32px blur25px blur
HSCALE: the detail that allows participants to tell photographs
and generated images apart is present at various scales.
-
Two- & Three-Way Interactions
55Jan Beneš - On Realism of Architectural Procedural Models
• Three factors
• Interactions
• More in paper
• Significant
• Not Significant
Dataset
Original Image
Class
Blur Size
-
Q2: Summary
56Jan Beneš - On Realism of Architectural Procedural Models
• Blur
– pHSCALE accepted
– Realism at different scales
• Exploratory Analysis
– Further insight
– Two- & Three-way interactions
– Mostly significant
-
Q3: What users thought
57
-
Participants
58Jan Beneš - On Realism of Architectural Procedural Models
• 52 total
– 11 female
– 24.3 years old
-
What users thought
59Jan Beneš - On Realism of Architectural Procedural Models
• Manually tallied debriefings
1. Imperfections & small detail (30/52 ~ 58%)
2. Texture (19/52 ~ 37%)
3. Reflections in windows (18/52 ~ 35%)
4. “Weird” or uniform color (17/52 ~ 33%)
5. Things in & around windows (16/52 ~ 31%)
6. Model Structure (14/52 ~ 27%)
7. Lighting (12/52 ~ 23%)
8. Shadow (12/52 ~ 23%)
9. Regularity (11/52 ~ 21%)
-
1. Imperfections & Small Detail
60Jan Beneš - On Realism of Architectural Procedural Models
• Imperfections & small detail (30/52 ~ 58%)
PhotographComputer GeneratedComputer Generated
-
2. Texture
61Jan Beneš - On Realism of Architectural Procedural Models
• Texture (19/52 ~ 37%)
Computer GeneratedComputer Generated
-
5. Windows
62Jan Beneš - On Realism of Architectural Procedural Models
• Things in & around windows (16/52 ~ 31%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated PhotographPhotograph
-
6. Model Structure
63Jan Beneš - On Realism of Architectural Procedural Models
• Model Structure (14/52 ~ 27%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Computer Generated
-
9. Regularity
64Jan Beneš - On Realism of Architectural Procedural Models
• Regularity (11/52 ~ 21%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Computer Generated
-
Biases
65Jan Beneš - On Realism of Architectural Procedural Models
• Cut-Out Edges– 21/52 ~ 40% – “played role”
• Camera Angles– 18/52 ~35% – Influenced for at
least one
• Background– 8/52 ~ 15% – Made me choose
computer generated more often
-
Opinions Summary
66Jan Beneš - On Realism of Architectural Procedural Models
• Multi-faceted
– Guidelines, guesses
– Not hard facts
• Biases
Textures
Color
-
More Exploration
67
-
Confounding Buildings @ 55px
68Jan Beneš - On Realism of Architectural Procedural Models
Computer
Generated
Photographs
-
Summary and Future Work
69
-
Summary
70Jan Beneš - On Realism of Architectural Procedural Models
• Realism in – Procedural Modelling
– Buildings
• Methodology for experiment– Procedural content
• Verified– People can tell CG and Photos apart
– Realism is carried at different scales
• “Soft results”– What people consider
– Exploration of results
-
Future Work & Limitations
71Jan Beneš - On Realism of Architectural Procedural Models
• Future Work– Understand asset reuse better
– Understand importance of structure better
– Study buildings in context
– Neural networks to automate testing & drive rule generation
• Limitations– Design limitations & Biases
– Generalization
-
Acknowledgements
72Jan Beneš - On Realism of Architectural Procedural Models
• ESRI
• Charles University GrantSVV-2016-260332
• Czech Science Foundation grant16-18964S.
• Filip Děchtěrenko was supported by Czech Academy of Sciences Strategy AV21 –RVO 68081740
• vrbn.io
• Corona Renderer
-
Thank You!
73
http://JanBenes.net
Supplemental Material
http://janbenes.net/
-
Biases
74Jan Beneš - On Realism of Architectural Procedural Models
• Cut-Out Edges– 21/52 ~ 40% – “played role”
• Camera Angles– 18/52 ~35% – Influenced for at
least one
• Background– 8/52 ~ 15% – Made me choose
computer generated more often
-
What users thought
75Jan Beneš - On Realism of Architectural Procedural Models
• Manually tallied debriefings
1. Imperfections & small detail (30/52 ~ 58%)
2. Texture (19/52 ~ 37%)
3. Reflections in windows (18/52 ~ 35%)
4. “Weird” or uniform color (17/52 ~ 33%)
5. Things in & around windows (16/52 ~ 31%)
6. Model Structure (14/52 ~ 27%)
7. Lighting (12/52 ~ 23%)
8. Shadow (12/52 ~ 23%)
9. Regularity (11/52 ~ 21%)