20111127 iccv祭り shirasy

1

CV勉強会＠関東（第18回）

shirasy

2011.11.27ICCV祭り

Content-Based Photo Quality Assessment

紹介する論文テーマ：

本資料は、以下の学会発表論文を引用しております。

Content-Based Photo Quality Assessment

W. Luo, X. Wang and X. Tang

in Proceedings of IEEE International Conference on Computer Vision (ICCV) 2011.

http://www.ee.cuhk.edu.hk/~xgwang/papers/luoWTiccv11.pdf

2

自己紹介

最近、参加している活動

CV関連

デザイン関連

写真関連

3

はじめに

視覚的な美的観点からの写真の自動品質評価の関心が高まる

4

関連研究

写真の自動品質評価の先行研究

○ブースティングTong et al. [18] used boosting to combine global low-level features for the classification of professional and amateurish photos

○写真撮影の経験則Ke et al. [11] designed a set of high-level semantic features based on rulesof thumb of photography. They measured the global distributions of edges, blurriness, hue.

These features were not specially designed for photo quality assessment.

5

○着目領域

Some approaches employed regional features by detecting subject areas,

since human beings percept subject areas differently from the background.

• Datta et al. [5] divided a photo into 3 × 3 blocks and assumed that the central block is the subject area.

• Luo et al. [12] assumed that in a high quality photo the subject area has a higher clarity than the background.

写真の自動品質評価の先行研究

関連研究

6

高品質の指針

様々なタイプの写真を撮る時、プロカメラマンは、異なる美的基準を持ち、異なる写真技術を用いることが知られている[2,19]

7

Photos are manually divided into seven categories based on photo content:

本論文のポイント

風景植物動物夜景人物静止物構造物

既存手法の大きな問題は、コンテンツの多様性を考慮せずに全ての写真を同じように扱うこと

8

本論文のポイント

ポイント

• Content based Content based Content based Content based photo qualityphoto qualityphoto qualityphoto quality assessmentassessmentassessmentassessment

• new subject area detection methods

• new global and regional features

• This work is construct This work is construct This work is construct This work is construct a large and diversified benchmark a large and diversified benchmark a large and diversified benchmark a large and diversified benchmark databasedatabasedatabasedatabase for the research of photo quality assessment for the research of photo quality assessment for the research of photo quality assessment for the research of photo quality assessment

プロやアマチュアカメラマンの写真：17,613枚

9

写真の品質評価方法

写真は評価者10名の内8名が、その評価に同意した場合にのみ、

高または低品質として分類される。 ※評価者の主観

A photo is classified as high or low quality only if eight out of the ten viewers agree on its assessment.

10

高品質写真/低品質写真例

高品質

低品質



所感：低品質の写真も、「白飛び」、「黒潰れ」、「ピントが甘い」と言うことはない様子。「プロならば、このように撮るだろうという推測」が（評価者の）判断基準になっていると思われる。

11

写真の品質

（補足）品質とは・・・

User Experience（UX）：品質は、実用的品質と快楽的品質に大別

○(本論文での観点と思われる品質）

快楽的品質

○実用的品質

累積的UXエピソード的UX

一時的UX予測的UX

本論文では、職業写真家以外の利用が対象と思われる。また、撮影物との関係性は無視。

実用的品質：使いやすさ、便利さ、機能性など

快楽的品質：魅力さ、満足感など

時間

12

提案手法


3. Global FeaturesProfessionals follow certain rules of color composition and scene composition to produce aesthetically pleasing photographs. ・3.1. Hue Composition Feature

4. Subject Area Extraction MethodsThe way to detect subject areas in photos depends on photo content. ・4.1. Clarity based region detection ・4.2. Layout based region detection ・4.3. Human based region detection

5. Regional FeaturesWe propose a new dark channel feature to measure both the clarity and the colorfulness of the subject areas. ・5.1. Dark Channel Feature ・5.2. Human based Feature

・5.3. Complexity Feature

13

提案手法


3. Global Features Professionals follow certain rules of color composition and scene composition to produce aesthetically pleasing photographs. ・3.1. Hue Composition Feature

写真を攻勢する色の組み合わせに着目⇒高品質のパターンと低品質のパターンを混合ガウス分布にて学習

14

提案手法


3. Global Features Professionals follow certain rules of color composition and scene composition to produce aesthetically pleasing photographs. ・3.1. Hue Composition Feature

15


4. Subject Area Extraction MethodsThe way to detect subject areas in photos depends on photo content. ・4.1. Clarity based region detection・4.2. Layout based region detection ・4.3. Human based region detection

提案手法

A clarity based subject area detection method was proposed in [12]. Since it used a rectangle to represent the subject area and fitted it to pixels with high clarity, the detection results were not accurate.

This paper improves the accuracy by oversegmentation.

16


4. Subject Area Extraction MethodsThe way to detect subject areas in photos depends on photo content. ・4.1. Clarity based region detection・4.2. Layout based region detection ・4.3. Human based region detection

提案手法

17


4. Subject Area Extraction MethodsThe way to detect subject areas in photos depends on photo content. ・4.1. Clarity based region detection ・4.2. Layout based region detection・4.3. Human based region detection

提案手法

Hoiem et al. [9] proposed a method to recover the surface layout from an outdoor image.

The scene is segmented into sky regions, ground regions, and vertical standing objects as shown in Figure 6.

This Paper take vertical standing objects as subject areas.

18


4. Subject Area Extraction MethodsThe way to detect subject areas in photos depends on photo content. ・4.1. Clarity based region detection ・4.2. Layout based region detection・4.3. Human based region detection

提案手法

Face detection [21] to extract faces from human photos. For images where face detection fails, this paper uses human detection [4] to roughly estimate the locations of faces.

19




提案手法

•特徴量Dark channelは、透明性、彩度、色相の成分が複合。• Dark channelの値は、画像のボケ度合いが高い程、Dark channelの値も高くなる。

20


5. Regional FeaturesWe propose a new dark channel feature to measure both the clarity and the colorfulness of the subject areas. ・5.1. Dark Channel Feature・5.2. Human based Feature


提案手法

写真撮影時、ライティングは品質に影響するため、ハイライト、陰影、明瞭度（周波数成分を分析）を特徴として着目

21




提案手法

空間的な複雑さを測定するためセグメンテーション結果を利用

Let Ns and Nb be the numbers of super-pixels in the subject area and the background, ||S|| and ||B|| be the areas of the subject area and the background.

22

実験結果

Regional featuresRegional featuresRegional featuresRegional features

23

実験結果

Global featuresGlobal featuresGlobal featuresGlobal features

24

実験結果

特徴量特徴量特徴量特徴量のののの組組組組みみみみ合合合合わせわせわせわせ

25

実験結果

特徴量の組み合わせ場合のROC曲線

26

まとめ

•新たな対象領域の検出法、全体および局所の特徴量に着目したコンテンツベースの写真の品質評価を提案

•ベンチマーク用データベースを用意

The integration of automatic photo categorization and quality assessment.

今後の課題

従来法と比較し、提案手法は品質の評価判断結果の一致度が高い

27

[1] S. Bhattacharya, R. Sukthankar, and M. Shah. A Framework for Photo-Quality Assessment and Enhancement based on Visual Aesthetics. In Proc. ACM MM, 2010. 4, 6, 7 http://www.cs.ucf.edu/~subh/docs/pubs/mmfat05123-bhattacharya.pdf

[2] J. Carucci. Capturing the Night with Your Camera: How to Take Great Photographs After Dark. Amphoto, 1995. 1 http://www.amazon.com/Capturing-Night-Your-Camera-Photographs/dp/0817436618

[3] D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y. Xu. Color harmonization. In Proc. ACM SIGGRAPH, 2006. 3, 4

[4] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Proc. CVPR, 2005. 5 http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf

[5] R. Datta, D. Joshi, J. Li, and J. Wang. Studying aesthetics in photographic images using a computational approach. In Proc. ECCV, 2006. 1, 3, 6, 7 http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ECCV06/datta.pdf

[6] C. Grey. Master Lighting Guide for Portrait Photographers. Amherst Media, Inc., 2004. 1 http://www.amazon.co.jp/Master-Lighting-Guide-Portrait-Photographers/dp/1584281251

[7] K. He, J. Sun, and X. Tang. Single image haze removal using dark channel prior. In Proc. CVPR, 2009. 5 http://research.microsoft.com/en-us/um/people/jiansun/papers/dehaze_cvpr2009.pdf

[8] K. He, J. Sun, and X. Tang. Single image haze removal using dark channel prior. IEEE Trans. on PAMI, 2010. 5 http://research.microsoft.com/en-us/um/people/jiansun/papers/dehaze_cvpr2009.pdf

[9] D. Hoiem, A. Efros, and M. Hebert. Recovering surface layout from an image. Int’l Journal of Computer

Vision, 2007. 4, 5 http://cvcl.mit.edu/SUNSeminar/hoiem_efros_scenelayout_IJCV06.pdf

参考文献（1/2）

28

[10] X. Jin, M. Zhao, X. Chen, Q. Zhao, and S. Zhu. Learning Artistic Lighting Template from Portrait Photographs. In Proc. ECCV, 2010. 1 http://blog.scse.buaa.edu.cn/%E9%87%91%E9%91%AB.pdf

[11] Y. Ke, X. Tang, and F. Jing. The design of high-level features for photo quality assessment. In Proc. CVPR, 2006. 1, 2, 6,7 http://www.yanke.org/papers/cvpr06photo.pdf

[12] Y. Luo and X. Tang. Photo and video quality evaluation: Focusing on the subject. In Proc. ECCV, 2008. 1, 3, 4, 5, 6,7 http://www.csd.uwo.ca/courses/CS9840a/StudentPapers/ECCV2008LuoTang.pdf

[13] H. Mante and E. Linssen. Color design in photography. Focal Press, 1972. 3 [14] M. Nishiyama, T. Okabe, Y. Sato, and I. Sato. Sensationbased photo cropping. In Proc. ACM MM, 2009. 3 [15] X. Ren and J. Malik. Learning a classfication model for segmentation. In Proc. ICCV, 2003. 5 [16] A. Savakis and S. Etz. Method for automatic assessment of emphasis and appeal in consumer images, Dec.

30 2003. US Patent 6,671,405. 1 [17] M. Tokumaru, N. Muranaka, and S. Imanishi. Color design support system considering color harmony. In

Proc.IEEE International Conference on Fuzzy Systems, 2002. 3 [18] H. Tong, M. Li, H. Zhang, J. He, and C. Zhang. Classfication of digital photos taken by photographers or

home users. In Proc. PCM, 2004. 1, 2 [19] L. White. Infrared Photography Handbook. Amherst Media, Inc., 1995. 1 [20] L. Wong and K. Low. Saliency-enhanced image aesthetics class prediction. In Proc. ICIP, 2009. 1, 3

http://www.comp.nus.edu.sg/~lowkl/publications/wonglk_icip2009.pdf [21] R. Xiao, H. Zhu, H. Sun, and X. Tang. Dynamic cascades for face detection. In Proc. ICCV, 2007.

参考文献（2/2）

20111127 iccv祭り shirasy

Engineering