objective estimation of video and speech quality to ...objective estimation of video and speech...
TRANSCRIPT
![Page 1: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/1.jpg)
Objective Estimation of Video andSpeech Quality to SupportNetwork QoS Efforts
2nd Internet2/DoE Quality of Service WorkshopFebruary 10, 2000, Houston, Texas
Stephen Voran & Stephen WolfInstitute for Telecommunication Sciences
Boulder, Colorado
![Page 2: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/2.jpg)
Motivation! User-perceived QoS is driven by many
factors, including! Perceived quality of delivered video! Perceived quality of delivered speech
! From the user point of view, these aretwo important components of QoS
! From the network point of view, theseare demanding applications
![Page 3: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/3.jpg)
Example MultimediaCommunication System
SpeechCodec
VideoCodec
Other
Net
wor
k In
terf
aceSpeech
Codec
VideoCodec
Other
Net
wor
k In
terf
ace
Terminal Equipment
Network
Terminal Equipment
![Page 4: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/4.jpg)
Quality Estimation for MultimediaCommunication System (In Service)
FeatureExtraction
Estimates ofPerceived
VideoQuality
SpeechCodec
VideoCodec
Other
Net
wor
k In
terf
aceSpeech
Codec
VideoCodec
Other
Net
wor
k In
terf
ace
Terminal Equipment
Network
FeatureExtraction
FeatureExtraction
Terminal Equipment
DistanceMeasures
QualityMapping
DistanceMeasures
QualityMapping
Estimates ofPerceived
Speech Quality
FeatureExtraction
![Page 5: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/5.jpg)
Quality-Estimation Components! Feature Extraction:
! Extract perceptually-relevant, quality-relatedattributes of signals
! Also called perceptual transformations! .0025 < (feature bit-rate/signal bit-rate) < 1
! Distance Measures:! Calculate perceptually-relevant distance between
two streams of features! Quality Mappings:
! Relate perceptual distances to estimates ofperceived quality
![Page 6: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/6.jpg)
Quality-EstimationConfigurations (In service)
(Feature Extraction)
SpeechCodec
VideoCodec
Other
Net
wor
k In
terf
aceSpeech
Codec
VideoCodec
Other
Net
wor
k In
terf
ace
Terminal Equipment
Network
Terminal Equipment
(Feature Extraction)
Distance Measures
Quality Mapping
Estimates of PerceivedSpeech and Video
Quality
(Feature Extraction)
(Storage) (Storage)
(Parentheses denote configuration options.)
![Page 7: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/7.jpg)
Quality-EstimationConfigurations (Out of service)
(FeatureExtraction)
SpeechCodec
VideoCodec
Other
Net
wor
k In
terf
aceSpeech
Codec
VideoCodec
Other
Net
wor
k In
terf
ace
Terminal Equipment
Network
Terminal Equipment
(Feature Extraction)
Distance Measures
Quality Mapping
Estimates of PerceivedSpeech and Video
Quality
(Feature Extraction)
Storage
(Storage)Storage
(Parentheses denoteconfiguration options.)
![Page 8: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/8.jpg)
Speech Quality EstimationDetails! Feature extraction or perceptual
transformation:! Transformation from Hz scale to critical band or
Bark scale! Transformation from signal amplitude to an
approximation of loudness
! Distance measure:! Measures and normalizes out spectral errors at
successively smaller time and frequency scales
![Page 9: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/9.jpg)
Speech Quality EstimationStatus! Measuring Normalizing Block (MNB)
algorithms provide reliable quality estimatesfor 4-kHz bandwidth speech, throughwaveform and non-waveform codecs with bit-errors and frame erasures and constant delay
! Continuing challenges include:! Variable transmission delay! Wider bandwidth speech! More general audio signals
![Page 10: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/10.jpg)
Video Quality EstimationDetails
! Feature Extraction! Transform video using perceptual filters
! Spatial information (magnitude, direction)! Temporal information! Chroma information
! Extract summary statistics from S-T regions! Standard deviation, mean! Region size determines compression factor! 8 lines x 8 pixels x 6 frames
![Page 11: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/11.jpg)
Video Quality EstimationDetails! Distance Measure
! Apply perceptibility threshold! Apply visual masking function
! Loss: {out-in}/in! Gain: log {out/in}
! Pool error over space (e.g., worst 5%)! Pool error over time
![Page 12: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/12.jpg)
Spatial Information Loss(e.g., Blurring)
SpatialSpatial
FilterFilter
SpatialSpatial
FilterFilter
![Page 13: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/13.jpg)
Spatial Information Gain(e.g., Tiling)
SpatialSpatial
FilterFilter
SpatialSpatial
FilterFilter
![Page 14: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/14.jpg)
Temporal Information Loss(e.g., Dropped Frames)
Temporal FilterTemporal Filter
![Page 15: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/15.jpg)
Temporal Information Gain(e.g., Error Blocks)
Frame nFrame n Frame n-1Frame n-1
__ ==
![Page 16: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/16.jpg)
Video Quality EstimationStatus
! Wide range of scenes, systems, bit-rates! Spatial detail, motion, contrast, brightness,
noise, saturation, hue! VTC, MPEG-1/2/4, DS3, analog, digital
transmission errors, multi-generation! Continuing challenges include
! Variable transmission delay systems! Continuous quality evaluation! Impairment visibility for some scenes
![Page 17: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/17.jpg)
Typical Performance ofEstimators! Measured through correlation to results
of controlled, formal, subjectivelistening and viewing tests
! Per System Correlation Coefficient! .70 ≤ ρ2 ≤ .98 (speech)! .74 ≤ ρ2 ≤ .98 (video)
![Page 18: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/18.jpg)
Example Scatter Plot for VideoSubjective vs. Objective
! Eleven DataSets! 164 systems
! ρ2 = .95! Zero = no
impairment! One = max
impairment
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Objective Quality
Sub
ject
ive
Qua
lity
![Page 19: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/19.jpg)
Standardization! Published standards
! Video: ANSI T1.801.03-1996 and ITU-TP.910, Section 5.3 and Annex A, 1996
! Speech: ANSI T1.518-1998 and ITU-TP.861 Appendix II, 1998
! Extensions, refinements and additionalstandardization efforts continue
![Page 20: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/20.jpg)
Conclusions! Perception-based objective estimators
of video and speech quality have beenderived and verified
! Might they support network QoSefforts?
! We welcome opportunities forcollaborative experimentation
![Page 21: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/21.jpg)
Institute for Telecommunication Sciences
Etc.
NationalInstitute ofStandards &Technology
Institute forTelecommunication
Sciences(ITS)
Etc.
NationalTelecommunications &
Information Administration(NTIA)
Etc.
Etc.
U.S. Departmentof
Commerce
ITS is the chief research and engineering arm of NTIA. ITSmission includes:
" Promotion of advanced telecommunications and informationinfrastructure development in the US
" Facilitation of more efficient and effective use of radiospectrum
" Enhancement of domestic competitiveness and foreign tradeopportunities of US telecommunications firms
![Page 22: Objective Estimation of Video and Speech Quality to ...Objective Estimation of Video and Speech Quality to Support Network QoS Efforts 2nd Internet2/DoE Quality of Service Workshop](https://reader031.vdocuments.mx/reader031/viewer/2022041814/5e59b7ec2f7b50503c646bc6/html5/thumbnails/22.jpg)
For more details! www.its.bldrdoc.gov! S. Voran, "Objective Estimation of Perceived Speech Quality,
Part I: Development of the Measuring Normalizing BlockTechnique," IEEE Transactions on Speech and Audio Processing,July 1999
! S. Voran, "Objective Estimation of Perceived Speech Quality,Part II: Evaluation of the Measuring Normalizing BlockTechnique," IEEE Transactions on Speech and Audio Processing,July 1999
! G.W. Cermak, S. Wolf, E.P. Tweedy, M.H. Pinson, and A.A.Webster, “Validating Objective Measures of MPEG VideoQuality," SMPTE Journal, April 1998
! S. Wolf, "Measuring the End-to-End Performance of DigitalVideo Systems, "IEEE Transactions on Broadcasting, September1997