methods for low bitrate coding enhancement part i ... · audio decoder car head unit (degraded) pcm...
TRANSCRIPT
© Fraunhofer IIS 1 08.09.2017
Methods for Low Bitrate Coding Enhancement Part I: Spectral Restoration
Patrick Gampp1, Christian Uhle1,2,Oliver Hellmuth1, Peter Prokein1,
Jürgen Herre2,1, Sascha Disch1,2, Julia Havenstein1, Antonios Karampourniotis1
1 Fraunhofer IIS, Erlangen, Germany2 International Audio Laboratories Erlangen, Germany
© Fraunhofer IIS 2 08.09.2017
Methods for Low Bitrate Coding Enhancement Part I: Spectral Restoration
1. Introduction
2. System Overview
3. Bandwidth Extension
4. Birdie Suppression
5. Evaluation
6. Conclusion
© Fraunhofer IIS 3 08.09.2017
1. IntroductionMotivation
Perceptual audio coders such as MP3 and AAC are used for efficient Storage Transmission
With good encoder and sufficiently high bitrates Uncoded and coded signals are perceptually indistinguishable!
Perceptual transparency At very low bitrates, sound quality is degraded Today`s car audio systems: Lots of possible audio sources with varying audio quality, e.g. Streaming, satellite radio Personal audio collection, etc.
High quality car audio systems unsparingly reveal coding artifacts
© Fraunhofer IIS 4 08.09.2017
1. IntroductionAim
Post‐processing of decoded signal without using meta data Easy, versatile integration Can handle transcoding Repeated encoding of already encoded signals E.g. MP3 at 32 kbps is decoded and then encoded with
AAC at 256 kbps Only meta data of the last encoder will be shown!
Enhancement algorithms shall increase perceptual pleasantness In contrast to perceptual transparency
Remove of perceptual audio coding artifacts at low bitrates
© Fraunhofer IIS 5 08.09.2017
2. System Overview
Audio Decoder
Car Head Unit
(Degraded)PCM Audio
SignalAudio Source Manager
(Enhanced) PCM Audio
Signal
)))
)))
)))
))))))
(Compressed) Audio
Bitstream
PCM Loudspeaker
Signal
Car Amplifier
Car Sound Processing
Low Bitrate Coding Enhancement Suite
Spectral Restoration
Spatial Enhancement
...
Audio Decoder
Car Head Unit
(Degraded)PCM Audio
SignalX’Audio
Source Manager
Low Bitrate Coding Enhancement Suite
Spectral Restoration
Spatial Enhancement
(Enhanced) PCM Audio
Signal
)))
)))
)))
))))))
(Compressed) Audio
Bitstream
PCM Loudspeaker
Signal
...
Car Amplifier
Car Sound Processing
Operation in the Head Unit
Operation in the Amplifier
© Fraunhofer IIS 6 08.09.2017
3. Bandwidth ExtensionBandwidth Limitation
Occurs at low bitrates Static lowpass filtering: dull and muffled sound
© Fraunhofer IIS 7 08.09.2017
3. Bandwidth ExtensionOverview
Processing frequency domain (STFT) Transient‐sustained decomposition for separate processing of
Transient path Sustained path
STFT
Delay LPF
Transient Suppression
Copy Up
Envelope Shaping Whitening ISTFT Anti‐Roughness
Filter
Copy Up
Envelope Shaping Whitening ISTFT Anti‐Roughness
FilterHPF
x[n] y[n]gt
gs
+
+
+
© Fraunhofer IIS 8 08.09.2017
3. Bandwidth ExtensionWhy Transient‐Sustained Decomposition?
Transient signal portions in the uncoded signal usually have Higher bandwidth Higher correlation within one spectral frame
Advantage of different parameter tunings, e.g. Create higher bandwidth for transient path More anti‐roughness filtering on sustained path
© Fraunhofer IIS 9 08.09.2017
3. Bandwidth ExtensionCopy Up and Spectral Envelope Shaping
Copy up of complex spectral coefficients Source patch: [fc, fc‐w] Destination patches: [fc+(k‐1)w, fc+kw] with k = 1,2,3,…
In order to avoid discontinuities between patches Attenuate destination patches by factor T Factor T is based on spectral tilt computed on source patch
|X|
ffcw w w
TT
TT
© Fraunhofer IIS 10 08.09.2017
3. Bandwidth ExtensionSpectral Whitening
Reduce spectral variance Compute and apply spectral
weights Attenuate spectral peaks Raise spectral valleys
PG11
Slide 10
PG11 Andere Farbe für gainsPatrick Gampp, 9/7/2017
© Fraunhofer IIS 11 08.09.2017
3. Bandwidth ExtensionAnti‐Roughness Filter
Aim: suppress roughness which is caused by copy up This is achieved by filtering of destination patches Two nested allpass filters in parallel with short impulse responses Mainly beneficial for sustained signal part. Therefore the transient‐sustained
decomposition is required
PG3
Slide 11
PG3 Bild von Allpässen?Impulsantwort?Patrick Gampp, 9/5/2017
© Fraunhofer IIS 12 08.09.2017
4. Birdies SuppressionBirdies Artifact
Can occur at low bitrates Groups of spectral bins are very coarsely quantized to Very low spectral magnitudes (spectral gap) Very high spectral magnitudes (spectral island)
Over time spectral gaps and islands may toggle or can occur isolated Introduction of temporal modulation Artifact is reminiscent of a bird`s twitter
Separate processing of Spectral islands Spectral gaps
© Fraunhofer IIS 13 08.09.2017
4. Birdies SuppressionExample for Islands and Gaps
© Fraunhofer IIS 14 08.09.2017
4. Birdie SuppressionOverview
Processing in frequency domain (STFT) Signals are processed using spectral weighting Scaling of spectral coefficients with real‐valued weights
XSTFTx[n] ISTFT y[n]
Spectral Gap Weight Calculation
Fill Source Signal Generation
Spectral Island Weight Calculation
G WS
F
Z Y
© Fraunhofer IIS 15 08.09.2017
5. EvaluationListening Test
Performed in quiet listening room using closed studio headphones 11 listeners Listening test with multiple stimuli Seven‐grade comparison test scale (ITU‐R BS.1284)
Condition Description
Reference Coded signal
Hidden reference Coded signal
BS Birdie suppression
BWE Bandwidth extension
BWE + BS Combination of BWE and BS
NLD BWE algorithm (Larsen&Aarts, 2005)
© Fraunhofer IIS 16 08.09.2017
5. EvaluationListening Test (2)
10 test signals of length between 8 s and 20 s each Loudness normalization (ITU‐R BS.1770)
Three different codec implementations1. MP3 encoder, 64 kbps, bandwidth 10 kHz (less Birdies than codec 2,3)2. MP3 encoder, 96 kbps, bandwidth 12 kHz (more Birdies than codec 1)3. MP3 encoder, 64 kbps, bandwidth 12 kHz (more Birdies than codec 1) Listeners should rate according to1. Quality of bandwidth‐extension2. Quality of concealment of spectral islands and gaps3. Overall sound quality
© Fraunhofer IIS 17 08.09.2017
Results per item, for all items and groups of codecs Proposed algorithms improve sound quality over all items Speech item (Spe3) also benefits of enhancement
5. EvaluationListening Test (3)
All 1Lower BWLess Birdies
All 2,3Higher BWMore Birdies
© Fraunhofer IIS 18 08.09.2017
6. Conclusion
Methods for Restoration of spectral islands and gaps Bandwidth extension
Methods do not use meta data Methods work in real‐time Evaluation in a listening test yielded Proposed algorithms improve sound quality over all items Speech also benefits of proposed algorithms
© Fraunhofer IIS 19 08.09.2017
Listen to Sonamic Enhancement Right NowWe Invite You to Get an Impression
Regency Ballroom
Listen also to
Symphoria 3D
Sonamic Loudness
© Fraunhofer IIS 20 08.09.2017
Thank you for your attention!