ad pesq veroct2011

7/31/2019 Ad PESQ VerOct2011

1/19

Speech Quality Measurement

In TEMS Investigation


2/19

Speech quality measurement

AQM overview

How AQM works in TI Accuracy of AQM in TI

Typical AQM values in different environments

What if AQM worse than expected


3/19

AQM overview

Estimate the user experienced quality.

ITU recommendation P.862.1 Compares a distorted speech recording to the original.

Reference speech

(original)

Convert into

psychoacoustic domain

Transmitted speech

(distorted)

Convert into

psychoacoustic domain

PESQ valueCognitive

modelling


4/19

MOS Meaning

5 Excellent

4 Good

3 Fair

2 Poor

1 Bad

AQM results: MOS-LQO

MOS Scale results fromsubjective tests (listening tests).

MOS-LQO =Mean Opinion Score

Listening Quality Objective

using AQM 862.1

Scale is: 4.5 (best) to 1.0 (worst).

Best possible score in a live

network depends on the speechcoder used:

- Best EFR: 4.2

VoCoder Reference PESQ[MOS-LQO]

EFR 0 4.22

EFR 1 4.16

EFR 2 4.15

EFR 3 4.13

EVRC 0 3.96

EVRC 1 3.63

EVRC 2 3.82

EVRC 3 3.57


5/19

AQM in TEMS

AQM works by sending speech references and comparing the

recording to the same known reference.

TI uses four speech references ,two female and two males eakers. Each reference contains of two sentences and each

sentence is 5.5 sec long. The references are in accordance with theITU standard P. 830, which stipulates how subjective MOS testsshould be performed.. One AQM score is given for each reference.

Sentence 0 Sentence 1

Reference 0


6/19

Test units

The AQM algorithm isimplemented in the Call

Generator and in the AQMModule of the TEMSInvestigaton.

Call Generators can beconnected to analogue lines or

DL audio

measurement

to ISDN. Call Generators and AQM

module store the recorded audiofor later analysis.

Results from UL and DL are

joined into the database so youcan see them together in a routeor make statistics on both.

Digital CellularNetwork

Public Switched

Telephone Network(PSTN)

Call

Generators

UL audiomeasurement


7/19

Measurement procedure DTMF syncs the call

Half duplex

Start when last. . .

TIRecord 5.5 sec

and

synch to the CG

Play 5.5 sec speech Record 5.5 sec

and

synch to the CG

AQM DL AQM UL AQM DL

Send DTMFto order call.Start afterlast digit.

digit received

FreqAQM FreqAQM FreqAQMFreqAQM

FreqAQM FreqAQM


8/19

Accuracy of AQM in TI

Three steps of verification:

1. ITU standardizationChose the algorithm which compares best to listening

tests.

3) Live testsprove PESQ

in TA system

2) Conformance tests

prove the PESQ algorithms

2. Conformance testsWhen a measurment tool implements the

in TA

1) ITU has held a competition were PESQ was proven to bethe best speech quality algorithm

a gor m mus pass e con ormance

test. The test consists of 1800 speechsentences and only three scores may differ

more than 0.05 MOS from the ITU version.

3. Live testsBoth the audio collection parts of a

measurement tool and the handset candegrade the recorded audio. To verify that thedegradations are insignificant we compare live

tests to simulations and listening tests.


9/19

Live tests

TEMS has a lab network with base station and BSC,

which makes it possible to test with controlled radioenvironment. Results are then compared tosimulations.

AMR

Codec rate

PESQmeasured on

live network on

DL

PESQmeasured on

live network on

UL

PESQmeasured on

simulation

12.2 4.11 4.08 4.17

7.95 3.81 3.84 3.84

5.90 3.50 3.50 3.50

4.75 3.21 3.18 3.26


10/19

Typical AQM values

Environment AQM

Best EFR / AMR FR (12.2) 4.15Best AMR HR (7.95) 3.85

Best CDMA EVRC 3.8

es . = .

Best 3G (AMR 12.2) BLER = 1% UL 3.85

Typical 3G (AMR 12.2) BLER = 1% 3.8

Good 3G/2G >= 3.5

Acceptable >=3.0


11/19

AQM measurements

PESQ algorithm (ITU Recommendation P.862) Estimates the listening speech quality

Speech-quality score based on the comparison betweenthe original speech and the speech as degraded bythe network

AQM in TEMS Investigation p on n nves ga on .

MO PESQ voice calls (command sequence)

UL and DL MOS score

Frequent AQM

Presentation of AQM scores in TEMS Investigation/Data Collection

TEMS Investigation/TEMS Discovery

Supported terminals Sony Ericsson K790i, K790a

Sony Ericsson K800i

LG CU-500


12/19

TEMS Investigation Test TerminalAQM Module

AQM measurements

Transmitted audio

Received degraded audio(from Call Generator)

Reference audio

PESQ DL Score

Command sequence

MOBILE-TO-FIXED AQM/PESQ MEASUREMENTS

TEMS InvestigationRoute Analysis (Converter)

Call Generator(with PESQ module)

Merged log file containing bothPESQ DL and UL scores

Log file containingPESQ DL scores

PESQ UL scores in XML files(copy/download)

Voice call

PESQ UL Score

Reference audio


13/19

AQM/PESQ MEASUREMENTS TEMS Investigation supports AQM/PESQ measurements

- Mobile-to-mobile and mobile-to-fixed

AQM is more than PESQ

TEMS Investigations AQM/PESQ solution

- TEMS Investigation with AQM license option

[ SERVICE QUALITY TESTING ]

- - Sony Ericsson W995, W995a, and W995 EDGE

- Kyocera KX5

- Samsung SPH-M320 and M330

- AQM module including cable kit

- Call Generator and/or Mobile Receiving Unit(MRU)

- Stand-alone or mounted in backpack or

equipment case

Full reference speech quality measurements


14/19

TEMS Investigation Test terminalAQM Module

Command sequence

Initiate AQM call

Received degraded audio(from Mobile Receiver Unit)

PESQ DL Score

MOBILE-TO-MOBILE AQM/PESQ MEASUREMENTS

[ SERVICE QUALITY TESTING ]

Mobile Receiver Unit(MRU)

TEMS InvestigationExport functionality

Logfile containing PESQ DL andpositioned DL Frequent AQM Scores

Logfile containingPESQ DL scores Voice call

Answer callReference audio

TEMS Investigation


15/19

AUDIO QUALITYMEASUREMENTS

Use TEMS Investigation to monitor end-to-end speech quality

Perform AQM/PESQ measurements with a Call Generator, mobile receivingunits, or a mix of both

- Mobile-to-fixed and mobile-to-mobile

Note the difference between the two solutions, in areas like measurements,

, ,

- PESQ UL and DL vs. PESQ DL- One radio link affecting each PESQ score vs. two radio links

affecting each PESQ score

- Impact of PSTN vs. impact of MRU location

- Capacity of Call Generator vs. number of MRUs

- Workflow and control

Use SQI measurements as a complement, when troubleshooting

Create your own solution


16/19

AQM is worse is it bad radio on DL?

1. Check the radio for the DL: GSM: SQI low (< 30)? Analyze further:

Check no of HO;s. Each HO gives an avg drop of~0.8 MOS.

RxQual (5-7) and FER (>0),

3G: SQI low? Analyze further: Avg BLER < BLER target. Check BLER target. A

BLER target of 1% is default and gives AQM =~3.8MOS.

Is it AMR HR? Check no of Inter RAT HO.


17/19

AQM is worse is it bad radio on UL?

2. Check the radio for the UL:1. All: Use UE traffic recording.

2. All:Look at Frequent AQM does it ever reach maximum? Check codec used: Is it AMR HR, FR or HR?

3. GSM: Look at Frequent AQM does it vary a lot? Analyzeur er:

Check DL radio. Bad DL is clue to bad UL. HO directly affect UL.

4. 3G: Remember for 3G there is always BLER on the UL!Will give varying AQM and Frequent AQM scores.


18/19

AQM is worse not radio

No or too few AQM scores. Reason No synch.:Voice message. Analyze by setting CG to record no sync .

Round trip delay too long. CG trace parameter Start ofRecording > 2600 samples?

Bad AQM due to low or high volume: , - . -

6dB. Check UL volume, should be > -15dB and < 2dB. Typical -

2dB.

Bad AQM due to background noise:

Record DL or UL audio. Look and listen at audio in silentparts in the beginning and end.


19/19

Some questions from customers How do you know you show the correct AQM scores?

See Accuracy of AQM in TA

Agilent / QVoice shows lower AQM scores than TEMS.WHY?

The audio recordin e ui ment is not the same. Our

equipment has low background noise, very little filtering,correct volume settings from factory.

The mobiles are not the same.

Audio references are different.

Remember we have verified our scores to simulations andin a controlled test environment (Techlab).

ad pesq veroct2011

Documents