digital video processing and analysis · 2019. 9. 20. · advanced digital media high dynamic range...
TRANSCRIPT
![Page 1: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/1.jpg)
Digital Video Processing and
Analysis : Introduction
Cheolkon Jung
School of Electronic Engineering
Xidian University, China
![Page 2: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/2.jpg)
8 PhD & 17 Master Students
(Alumni: 3 PhD, 44 Masters)
Research Lab (2010~Present) 2
![Page 3: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/3.jpg)
Advanced digital media
High dynamic range (HDR) video
Ultra high definition (UHD) video
Stereoscopic 3D (S3D) video
Multimedia big data
Surveillance video
Main research topics
Image restoration & Super-resolution
S3D video processing & communications
Computational photography & Display enhancement
Multimedia annotation, retrieval & security
H.265 and perceptual video coding
Deep learning for computer vision (2015~)
Research Interests 3
![Page 4: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/4.jpg)
Instructor Information 4
• Instructor: Cheolkon Jung
• Office: #203, 100 Building
• Email: [email protected]
• Webpage: http://web.xidian.edu.cn/zhengzk
http://see.xidian.edu.cn/media
• Time:
• Formal: Wednesday PM 4:30-6:00 (Every week);
• Informal: Appointed time;
• Place: #210, 100 Building
http://web.xidian.edu.cn/zhengzk
![Page 5: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/5.jpg)
Images 5
Pixel
(Picture Element)
Image: 2D vector
![Page 6: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/6.jpg)
Videos 6 Image sequences
![Page 7: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/7.jpg)
Difference Between Images and Videos 7
• Images • Shape
• Color
• Texture
• Videos • Shape
• Color
• Texture
• Motion
![Page 8: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/8.jpg)
Motion 8 Motion Estimation
Motion Compensation
![Page 9: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/9.jpg)
3D Videos 9
2009
3DTV
At Home
Theater
![Page 10: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/10.jpg)
3D Videos 10
3D Display With Glasses
![Page 11: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/11.jpg)
• Images • Shape
• Color
• Texture
• Videos • Shape
• Color
• Texture
• Motion
• Depth in 3D videos
Difference Between 2D and 3D Videos 11
![Page 12: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/12.jpg)
• Human 3D: perceive depth information from two slightly different views by two
eyes.
Depth Perception
Depth 12 Depth Estimation
3D Reconstruction
![Page 13: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/13.jpg)
• Exactness • Perfect reproduction without degradation
• Perfect duplication of processing result
• Convenient & powerful computer-aided processing • Can perform rather sophisticated processing through hardware or Software
• Even kids can do it!
• Easy storage and transmission • 1 CD can store 1 video or hundreds of family photos!
• Paperless transmission of high quality photos through network within seconds
Films Sensors
Digital Videos 13
![Page 14: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/14.jpg)
• Source of videos: • Captured with an analog or digital TV/movie camera
• Time-based sequences: radar, satellite, or drones
• Video units:
Digital Videos 14
![Page 15: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/15.jpg)
• Popular mobile imaging devices:
• Smart phone, video camera, personal media player (PMP), etc
• Storage size expansion:
• HDD, CD, DVD, flash memory, etc
• Digital videos everywhere:
• Social, commercial, industry and military applications
• Entertainment, education, medicine, databases, security, etc.
• High-speed computing environments in most digital devices
Digital Videos 15
![Page 16: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/16.jpg)
Digital Video Processing & Analysis
• Video compression
• Video enhancement
• Video surveillance
• Video content analysis
• Video indexing & retrieval
• Object detection & recognition
• Motion estimation
• Stereo video processing
16
![Page 18: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/18.jpg)
HDR Video Compression
IEEE VCIP 2015 & 2016
SIGNAL PROCESS: IMAGE (2018)
Free energy
principle
Perceptual Block Merging
Whole Framework of the Proposed Method
Perceptual Encoder
Calculate
FEJNDCU &
FEJNDframe
Adjust RD cost
minimization FEJND map Block merging
Perceptual
weight
18
![Page 19: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/19.jpg)
IEEE TMM (2014)
Example
Learning
2D to 3D Video Conversion 19
![Page 20: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/20.jpg)
IEEE ICASSP 2016
IEEE TCSVT (2018)
Ambient Light Adaptive Video Enhancement
![Page 21: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/21.jpg)
Edge Guided Depth Image Super-resolution (DISR) Using CNN
DISR by HR edge prediction, instead of HR texture prediction IEEE ICASSP 2018
Deep Learning for Machine Intelligence 21
![Page 22: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/22.jpg)
Main Contents 22
• Video Basics
• Video Modeling
• Deep Leaning for Visual Understanding
• Motion Estimation
• HEVC for Video Compression
• Stereo and 3D Videos (S3D)
• 360 Videos
• Text books: TPAMI/IJCV/TIP/TMM/TCSVT/CVPR/ICCV/ECCV
Video Processing and Communications (Wang et al.)
Student Projects:
- Paper selection
- One-page proposal
- Proposal presentation
- Reproduction of methods
- Final presentation
- Demonstration
- Technical report and manual
Source codes should not be open!
* Quiz: 5 times
![Page 23: Digital Video Processing and Analysis · 2019. 9. 20. · Advanced digital media High dynamic range (HDR) video Ultra high definition (UHD) video Stereoscopic 3D (S3D) video Multimedia](https://reader036.vdocuments.mx/reader036/viewer/2022070219/61283d3bf365492d6f053212/html5/thumbnails/23.jpg)