structure and aesthetics in non-photorealistic images
DESCRIPTION
Structure and Aesthetics in Non-Photorealistic Images. Hua Li, David Mould, and Jim Davies Carleton University. Artistic or Messed. 2 /35. Related Work on Evaluating Non-Photorealistic Algorithms. Performance based on processing speed ill-suited for stylization - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/1.jpg)
Structure and Aesthetics in Non-Photorealistic Images
Hua Li, David Mould, and Jim DaviesCarleton University
![Page 2: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/2.jpg)
Artistic or Messed
2/35
![Page 3: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/3.jpg)
Related Work on Evaluating Non-Photorealistic Algorithms
• Performance based on processing speed– ill-suited for stylization
• Side-by-side comparisons – not fully convinced by audience
3/35
![Page 4: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/4.jpg)
Perceptual Evaluation on Non-Photorealistic Algorithms
• Quantitative evaluation– rating scores [Schumann et al. 96, Gooch and
Willemsen 02, Mandryk et al. 2011, Mould et al. 2012]
– response time [Gooch et al. 04]– eye-tracking data [Mandryk et al. 2011, Mould et
al. 2012]• Qualitative evaluation– questionnaire-based
4/35
![Page 5: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/5.jpg)
Motivation of Our StudyTone Structure
Tone-based Structure-based
Halftoning
[Pang et al. 08] [Chang et al. 09] [Ours 10]
Stippling [Secord 02]
[Mould 07] [Martin et al. 11] [Ours 11]
Screening [Ulichney 98]
Abstraction [Kyprianidis 11] [Mould 12, 13]
[Qu et al. 08] [Ours 11]
[Floyd and Steinberg 76]
[Ostromoukhov 01]
5/35
![Page 6: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/6.jpg)
Questions to Answer
• Are structural and aesthetic quality related? • Do images matter for side-by-side
comparisons?
6/35
![Page 7: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/7.jpg)
Participants
• 30 participants– 15 female and 15 male– 11 artists– aged 18 to 33
7/35
![Page 8: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/8.jpg)
Study Overview
• 1 ~ 1.5 hours to complete the experiment• Using the keyboard or the mouse to enter
their responses• Tasks:– rating structural and aesthetic quality– collecting response times for rendered images
8/35
![Page 9: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/9.jpg)
Image Stimuli
• Seven categories – include cars, cats, persons, flowers, buildings, mugs,
and birds.• Each category contains 13 different images
including– one unprocessed image– and 12 rendered images using 12 algorithms.
• Images are black and white, or greyscale to remove the influence of color.
9/36
![Page 10: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/10.jpg)
Procedure
• Step 1: verbal introduction• Step 2: training• Step 3: formal study• Step 4: questionnaire• Step 5: ranking
10/35
![Page 11: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/11.jpg)
Interfaces Used
Interface for collecting the response time 11/35
![Page 12: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/12.jpg)
Interfaces Used
Aesthetic rating Structural rating12/35
![Page 13: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/13.jpg)
Experimental Images -Bird Category
Unprocessed13/35
![Page 14: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/14.jpg)
Experimental Images - Bird Category
Structure-Preserving Stippling (SPS)
Structure-Aware
14/35
![Page 15: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/15.jpg)
Experimental Images - Bird Category
Content-Sensitive Screening (CSS)
Structure-Aware
15/35
![Page 16: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/16.jpg)
Experimental Images - Bird Category
SPS with Exclusion Masks (SPH)
Structure-Aware
16/35
![Page 17: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/17.jpg)
Experimental Images - Bird Category
Line Art using edge tangent field (ETF)
Structure-Aware
17/35
![Page 18: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/18.jpg)
Experimental Images - Bird Category
Artistic Tessellation (AT)
Structure-Aware
18/35
![Page 19: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/19.jpg)
Experimental Images - Bird Category
Line Art from SPS (Drawing)
Structure-Aware
19/35
![Page 20: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/20.jpg)
Experimental Images - Bird Category
Secord’s Stippling Method (Secord)
Tone-based
20/35
![Page 21: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/21.jpg)
Experimental Images - Bird Category
Line Art using edge tangent field (Mmosaics)
Tone-based
21/35
![Page 22: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/22.jpg)
Experimental Images - Bird Category
Contrast-Aware Halftoning (CAH)22/35
![Page 23: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/23.jpg)
Experimental Images - Bird Category
Black and White (BW)23/35
![Page 24: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/24.jpg)
Experimental Images - Bird Category
Adding 50% salt and pepper noise (Noisy)
Reduced Information
24/35
![Page 25: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/25.jpg)
Experimental Images - Bird Category
Gaussian filter (Blurring)
Reduced Information
25/35
![Page 26: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/26.jpg)
Positive Correlation Between Structural and Aesthetic Ratings
26/35
![Page 27: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/27.jpg)
Dot-based Methods (Stippling)
Aesthetic Structure1
1.5
2
2.5
3
3.5
4
4.5
SecordSPS
Mea
n Ae
sthe
tic\S
truc
ture
Rati
ngs
Structure-Aware
Tone-basedStructure-
Aware
Tone-based
27/35
![Page 28: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/28.jpg)
Region-based Methods (Mosaics)
Aesthetic Structure1
1.5
2
2.5
3
3.5
MmosaicsAT
Mea
n Ae
sthe
tic\S
truc
ture
Rati
ngs
Structure-Aware
Tone-basedStructure-
Aware
Tone-based
28/35
![Page 29: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/29.jpg)
Effect of Category on Ratings
Aesthetic Structure 1
1.5
2
2.5
3
3.5
4
4.5CarCatBirdPersonMugFlowerBuilding
Mea
n Ae
sthe
tic\S
truc
ture
Rati
ngs
29/35
![Page 30: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/30.jpg)
Effect of Category on Response Time
Series1
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
CarCatBirdPersonMugFlowerBuilding
Z va
lues
of R
espo
nse
Tim
e
30/35Building < Flower < Bird < Cat < Person < Mug < Car
![Page 31: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/31.jpg)
Artists and Non-Artists
Aesthetic Structure1
1.5
2
2.5
3
3.5
4
4.5
5
ArtistNon-Artist
Mea
n Ae
sthe
tic\S
truc
ture
Rati
ngs
31/35
![Page 32: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/32.jpg)
Overall Ranks after Study
• Participants preferred the AT images (7/30 responses) the most, CAH second (6/30).
• Participants’ least favorite – blurred images most often (20/30 responses), and
with AT second (5/30). • Controversial ranking for stylized images
rendered by the AT method.
32/35
1st Ran
k
2nd Rank
3rd Rank
4th Rank
5th Rank
6th Ran
k
7th Ran
k
8th Ran
k
9th Ran
k
10th Ran
k
11th Ran
k
12th Ran
k0
5
10
15
20
25
CAHATNoisyBlurring
From Most Favourite to Least Favourite
Num
ber o
f Par
ticip
ants
![Page 33: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/33.jpg)
Conclusions
• Considering structure as a possible way to increase aesthetic appeal.
• Considering the choice of the images used.– Generally, bird images were the easiest images to
abstract, while Person images were the most difficult.
33/35
![Page 34: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/34.jpg)
Future Work
• More Participants• More Categories• More NPR Algorithms• Eye tracker
34/35
![Page 35: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/35.jpg)
Thanks for Your Attention.
35/35
Questions?
![Page 36: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/36.jpg)
Effect of Algorithms on Ratings (skip)
Aesthetic Structure1
1.5
2
2.5
3
3.5
4
4.5
5
CAHSPSCSSSPHETFATBWDrawingUnprocessedNoisyBlurring
Mea
n Ae
sthe
tic\S
truc
ture
Rati
ngs
36/41
![Page 37: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/37.jpg)
Effect of Algorithms on Response Time (skip)
Series1
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
CAHSPSCSSSPHETFATBWDrawingUnprocessedNoisyBlurringZ
valu
es o
f Rec
ogni
tion
Tim
e
37/41
![Page 38: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/38.jpg)
Interaction between Categories and Algorithms (skip)
Car Cat Bird Person Mug Flower Building1
1.5
2
2.5
3
3.5
4
4.5
5CAHSecordSPSCSSSPHETFMmosaicsATBWDrawingUnprocessedNoisyBlurring
Mea
n Ae
sthe
tic R
ating
s
Aesthetic scores38/41
![Page 39: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/39.jpg)
Interaction between Categories and Algorithms (skip)
CAHSe
cord SP
SCSS SP
H ETF
Mmosaics AT
BW
Drawing
Unprocesse
dNoisy
Blurring
1
1.5
2
2.5
3
3.5
4
4.5
5
CarCatBirdPersonMugFlowerBuilding
Mea
n Ae
sthe
tic R
ating
s
39/41
![Page 40: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/40.jpg)
Interaction between Categories and Algorithms (skip)
Car Cat Bird Person Mug Flower Building
-1
-0.5
0
0.5
1
1.5
CAHSPSCSSSPHETFATBWDrawingUnprocessedNoisyBlurringZ
valu
es o
f Rec
ogni
tion
Tim
e
40/41
![Page 41: Structure and Aesthetics in Non-Photorealistic Images](https://reader035.vdocuments.mx/reader035/viewer/2022070423/56816643550346895dd9b9de/html5/thumbnails/41.jpg)
1st Ran
k
2nd Rank
3rd Rank
4th Rank
5th Rank
6th Ran
k
7th Ran
k
8th Ran
k
9th Ran
k
10th Ran
k
11th Ran
k
12th Ran
k0
5
10
15
20
25
CAHATNoisyBlurring
From Most Favourite to Least Favourite
Num
ber o
f Par
ticip
ants