d1 - 27/10/2015 the present document contains information that remains the property of france...
Post on 03-Jan-2016
216 Views
Preview:
TRANSCRIPT
D1 - 20/04/23
The present document contains information that remains the property of France Telecom. The recipient’s acceptance of this document implies his or her acknowledgement of the confidential nature of its contents and his or her obligation not to reproduce, transmit to a third party, disclose or use for commercial purposes any of its contents whatsoever without France Telecom’s prior written agreement.
confidential
Comparison of Narrowband and Wideband
Speech codecs in noisy environnement
Noel Chateau noel.chateau@francetelecom.com, Laetitia Gros laetitia.gros@francetelecom.com,
Catherine Quinquis catherine.quinquis@francetelecom.com, Jean Yves Monfort
jeanyves.monfort@francetelecom.com
Workshop on Wideband Speech Quality in Terminals and Networks: Assessment and
Prediction
8th and 9th June 2004 - Mainz, Germany
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D2 - 20/04/23
confidential
Content
1. 3Gpp/VoIP context
2. Methodology
3. Description of Experiment
4. Results
5. Conclusion
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D3 - 20/04/23
confidential
3GPP/VoIP Context
s Rel 99 : March 2000Q only narrow band speech and Circuit switched communications
s Rel4: March 2001QPacket switched for download QSelection of wide band speech codec
s Rel5: March 2002QPacket switched conversationnal communicationsQnarrow band and wide band codecs
s Rel6: September 2004QCharacterisation of default codecs for Packet switched conversationnal communications
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D4 - 20/04/23
confidential
3GPP/ Characterisation of default codecs for Packet switched conversationnal communications
s 1st phase QAMR codec in quiet and noisy environmentQAMRWB codec in quiet and noisy environment
s 2nd phaseQAMR and AMRWB versus legacy codecs in quiet environment
s Further experimentQAMR and AMRWB versus legacy codecs in noisy environment : car noise
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D5 - 20/04/23
confidential
Methodology
s VoIP in UMTSQ delay is one major issue
–Interleaving due to radio environment–Buffering in decoder side to cope with jitter–IP transport–Packetisation in IP as well as on radio leg
QNot only quality of codecs but acceptability of service
s Conversation test QMore accurate to assess acceptability of service
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D6 - 20/04/23
confidential
Experiment: Description
s Noisy environment : car noise @ 60dB PaQ Car noise is in one of the room, the other room is quiet
s Codecs under testQAMR (6.7 and 12.2 modes)QAMRWB (12.65 and 15.85 modes)QG.723.1@ 6.3 kb/sQG.729@ 8 kb/sQG.711@ 64 kb/sQG.722 @ 64 kb/s
s Environmental conditionsQIP packet loss : 0% or 3%
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D7 - 20/04/23
confidential
Experiment: Description
PC 1 : VOIP Terminal
Simulator
Network Board A
PC 2 : VOIP Terminal
Simulator
Network Board B
PC 3 : Network Simulator
Hub 1 Hub 2
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D8 - 20/04/23
confidential
Results on voice quality criterion s ANOVA
Qsignificant main effects: noise, PLP & codecQsignificant interactions: codec x PLP
s Planned comparisons QInteraction bandth x noise for PLP=0%: NSQInteraction Bandth x PLP for Noise: NS
s Tuckey tests show a superiority of some codecs on others for noisy and/or lossy conditions, but not systematic. Without noise and packet losses: no significant differences.
s The superiority of WB Codecs is higher for noisy and packet-loss conditions, but there is no systematic benefit of WB in noise.
Factor F
noise * F(1,30) = 20.23 - p<0.001
packet loss percentage (PLP) * F(1,30) = 13.61 - p<0.001
codec * F(7,210) = 14.26 - p<0.001
noise x PLP F(1,30) = 0.68 - p=0.42
noise x codec F(7,210) = 1.25 - p=0.28
codec x PLP* F(7,210) = 2.3 - p<0.05
noise x codec x PLP F(7,210) = 0.76 - p=0.62
Voice Quality
Ca r noise W ithout noise
Pa cke t loss = 0%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc2 ,5
3,0
3,5
4,0
4,5
5,0
MO
S
P a cke t loss = 3%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc
A N
O V
A
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D9 - 20/04/23
confidential
Results on understanding criterion
Factor F
noise * F(1,30) = 68.11 - p<0.001
packet loss percentage (PLP) * F(1,30) = 11.07 - p<0.01
codec * F(7,210) = 20.3 - p<0.001
noise x PLP F(1,30) = 0.53 - p=0.47
noise x codec F(7,210) = 1.86 - p=0.08
codec x PLP* F(7,210) = 2.09 - p<0.05
noise x codec x PLP F(7,210) = 1.88 - p=0.07
s ANOVAQsignificant main effects: noise, PLP & codecQsignificant interactions: codec x PLP
s Planned comparisons QInteraction bandth x noise for PLP = 0%: Signif.QInteraction Bandth x PLP for Noise: NS
s Tuckey tests show an almost systematic superiority of WB and G.711 codecs on NB codecs (except AMR NB 12.2 kbs) for noisy and/or lossy conditions. Without noise and packet losses: no significant differences.
s Considered globally, WB and G.711 codecs better resist to noise for PLP=0%.
U nderstanding of the partner
Ca r noise W ithout noise
Pa cke t loss = 0%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc2 ,5
3,0
3,5
4,0
4,5
5,0
MO
S
P a cke t loss = 3%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc
A N
O V
A
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D10 - 20/04/23
confidential
Results on interaction criterion
Factor F
noise * F(1,30) = 24.58 - p<0.001
packet loss percentage (PLP) * F(1,30) = 12.66 - p<0.001
codec * F(7,210) = 6.97 - p<0.001
noise x PLP F(1,30) = 0.3 - p=0.59
noise x codec F(7,210) = 2.67 - p<0.05
codec x PLP* F(7,210) = 0.63 - p=0.73
noise x codec x PLP F(7,210) = 0.96 - p=0.46
s ANOVAQsignificant main effects: noise, PLP & codec
Qsignificant interactions: codec x noise
s Planned comparisons QInteraction bandth x noise for PLP = 0%: Signif.
QInteraction Bandth x PLP for Noise: NS
s Tuckey tests show only a few significant differences for noisy conditions only.
s As for the understandy criterion, considered globally, WB codecs better
resist to noise for PLP=0%.
A N
O V
A
Interactivity
Ca r noise W ithout noise
Pa cke t loss = 0%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc2 ,5
3,0
3,5
4,0
4,5
5,0
MO
S
Pa cke t loss = 3%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D11 - 20/04/23
confidential
Results on default perception criterion
Factor F
noise * F(1,30) = 29.5 - p<0.001
packet loss percentage (PLP) * F(1,30) = 11.95 - p<0.01
codec * F(7,210) = 6.48 - p<0.001
noise x PLP F(1,30) = 0.21 - p=0.65
noise x codec F(7,210) = 1.67 - p=0.12
codec x PLP* F(7,210) = 2.47 - p<0.05
noise x codec x PLP F(7,210) = 0.82 - p=0.57
A N
O V
A
s ANOVAQsignificant main effects: noise, PLP & codec
Qsignificant interactions: codec x PLP
s Planned comparisons QInteraction bandth x noise for PLP = 0%: NS
QInteraction Bandth x PLP for Noise: NS
s Tuckey tests show no significant differences between codecs for any condition.
s The default perception criterion does not seem relevant to detect a possible advantage of WB codecs in noise.
D efault perception
Ca r noise W ithout noise
Pa cke t loss = 0%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc2 ,0
2,5
3,0
3,5
4,0
4,5
5,0
MO
S
Pa cke t loss = 3%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D12 - 20/04/23
confidential
Results on global quality criterion
Factor F
noise * F(1,30) = 57.37 - p<0.001
packet loss percentage (PLP) * F(1,30) = 14.51 - p<0.001
codec * F(7,210) = 13.54 - p<0.001
noise x PLP F(1,30) = 0.29 - p=0.59
noise x codec F(7,210) = 1.28 - p=0.26
codec x PLP* F(7,210) = 1.68 - p=0.11
noise x codec x PLP F(7,210) = 0.4 - p=0.9
A N
O V
A
s ANOVAQsignificant main effects: noise, PLP & codecQsignificant interactions: none
s Planned comparisons QInteraction bandth x noise for PLP = 0%: NSQInteraction Bandth x PLP for Noise: NS
s Tuckey tests show an almost systematic superiority of WB and G.711 codecs on NB codecs (except AMR NB 12.2 kbs) for noisy and/or lossy conditions. Without noise and packet losses: no significant differences.
s As for the voice-quality criterion, the superiority of WB Codecs is higher for noisy
and packet-loss conditions, but there is no systematic benefit of WB in noise.
Globa l qua lity of the com m unica tion
Ca r noise W ithout noise
Pa cke t loss = 0%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc2 ,0
2,5
3,0
3,5
4,0
4,5
5,0
MO
S
Pa cke t loss = 3%
AM
R N
B 6.7 kbs
AM
R N
B 12.2 kbs
AM
R W
B 12.65 kbs
AM
R W
B 15.85 kbs
G.723.1 6.4 kbs
G.729 8 kbs
G.722 64 kbs +
plc
G.711 +
plc
France Telecom R&D Distribution of this document is subject to France Telecom’s authorization D13 - 20/04/23
confidential
Summary of subjective tests resultss ANOVA is not useful for determining if WB codecs are significantly better than NB
codecs in noisy environment but show that noise was the strongest experimental factor that affected the subjective data.
s Planned comparisons and Tuckey tests show that:Qthe MOS differences between WB and NB are significantly larger in noisy conditions than in silent conditions for no packet losses, for "Understanding" and "Interactivity" criteria.
Q the MOS differences between WB and NB are often but not systematically larger in noisy and/or 3% packet-loss conditions than in the silent and 0 % packet-loss condition, for the "Voice quality" and "Global quality" criteria.
Qthe differences between WB and NB is independant on the presence of noise and packet loss for the "Default perception" criterion.
s It can be concluded that without packet loss, in noise, the advantage of WB compared to NB is that it enhances the comprehension and the interaction with the other partner, but not voice and global quality, since these criteria are highly affected by the presence of noise.
top related