MOS-- Mean Opinion Score

KeywordsSpeech quality, MOS, mute, noise, garble noise

AbstractThis guide describes common speech quality problems and their general principles and summarizes the ideas and methods for testing, tracing, isolating, and locating the problems, and KPI improvement of speech quality.

Acronyms and AbbreviationsAcronyms and Abbreviations

Full Spelling

MOS mean opinion score

PESQ perceptual evaluation of speech quality

PESQ_LQ PESQ listening quality

SQI speech quality index

All network elements used by the two parties of a speech call may cause speech quality problems. In problem analysis, possible reasons need to be considered from the end-to-end angle.

. All operations must strictly follow operation processes. It also has very detailed and strict rules on experiment parameters, such as:

Recording system Speech samples Speech input levels

Listening levels Different talkers (eight male, eight female, and eight children) Multiple talkers (Several persons speak simultaneously.) Errors Interoperability with other speech coding standards Mismatch Environmental noise Music

P.862 PESQPESQ is developed by British Telecom and KPN and is adopted by ITU as the P.862 specification in 2001. PESQ compares the voice signals and deterioration signals and provides a MOS score, similar to manual hearing evaluation test. The functions of PESQ, however, are more powerful. PESQ can test both effects of network elements such as decoders and quality of end-to-end voices. It provides test results mainly based on different reasons of signal deterioration, such as encoding and decoding distortion, errors, packet loss, delays, jitter, and filtering.

PESQ scores range from -0.5 to 4.5. The larger the value is, the better the speech quality is. Generally, the scores range from 1 to 4.5.

PESQ-LQPESQ scores, however, have certain tendencies that are likely to bring optimistic evaluation for poor speech and pessimistic evaluation for good speech. Therefore, the standard is optimized. The function for converting PESQ scores to PESQ-LQ scores is provided. The score is similar to subjective MOS values. describes the relationship between PESQ-LQ scores and subjective experience.

Scores and subjective perceptions

Score Quality of the speech

5 Excellent

4 Good

3 Fair

2 Poor

1 Bad

The conversion formula is as follows:

P862.1ITU also proposes an optimization standard, namely P.862.1 and provides a way of mapping PESQ scores directly to MOS-LQO scores. The conversion formula is as follows:

Speech Quality Testers Third-party testers:

− DSLA

− Swissqual

− Comarco

− Rising

Vendors' testers:

− Ericsson: TEMS

− Huawei: Probe

− Nokia: Nemo

DSLADSLA is short for digital speech level analyser.

Manufacturer: Malden Standards supported: PESQ/PESQ-LQ/P862.1 Functions: speech quality test Availability: Huawei bought many and 15 are available in the asset base.

Advantages:

Not expensive, medium size, and easy to carry No requirements of terminal types Able to test delay of voice service on the user plane

Disadvantages:

Complicated debugging and unstable test results

Cable connection of terminals and testers, requiring close locations of the calling and called parties; not suitable for commercial test between UE and PSTN

Moderate recognition among customers

TEMSTEMS

Manufacturer: Ericsson Standards supported: SQI and PESQ (additional hardware support needed;

integrated in version 8.0, 2004) Functions: all-round KPI drive test tool, not limited to speech quality test Availability: one in Germany (network planning), supporting test of 3G

speech quality

Advantages:

SQI is a speech quality evaluation algorithm at the wireless side. It can avoid the strong correlation with speech quality samples.

Supporting PESQ

Disadvantages:

Expensive (not confirmed) Equipment providers' drive test tool, with average recognition among

customers

Recommended Tool: ProbeProbe is a drive test tool developed by Huawei. It provides the speech evaluation function since version 1.5 and supports automatic test and scoring.

Probe requires support by Huawei U88000. The most outstanding characteristic of Probe is that it does not need to be connected to any independent speech evaluation device. Probe is easy to carry. What is more important is that digital transmission of speech frames can avoid interference caused by traditional analog lines.

The connection diagram is as follows:

Manufacturer: Huawei Standards supported: PESQ and P862.1 Functions: all-round KPI drive test tool, not limited to speech quality test Availability: Huawei product, easy to obtain

Advantages:

Test speech samples are played and recorded on UE; therefore, external interference and cable loss are eliminated.

No special hardware requirement and easy to carry Huawei product without purchase

Disadvantages:

Average recognition among customers Supporting few Huawei terminals only

Voice Recording Collection by Probe

After tests, users need to collect the following data for analysis:

gen files recorded by Probe 2.2, for example, Probe_20090423112749.gen If users do not change the save path, .gen files are saved in the default path C:\GENEX Logfile.

System log files (optional) recorded by Probe 2.2 are .log files named by dates, for example, GENEX Probe2009-05-11.log. The default path for saving system log files is the software installation directory\log.

Voice recording files with low scores (degraded voice files) are .wav file named by dates. If users do not change the path for saving voice recording files when they configure test plans, the voice recording files will be saved in the path shown in .

Path for saving voice recording files

Wideband AMRVoice services on UMTS networks use the AMR standard for speech coding. The speech coding technology with a maximum rate of 12.2 Kbps is called narrowband AMR. The speech coding technology with a maximum rate of 23.8 Kbps is called wideband AMR (released in 1999). Wideband AMR uses large bandwidths and is able to keep high-frequency information in voices.

Speech Quality Improvement Features

Wideband AMR (WAMR)

Recommendation LevelMedium

Few commercial terminals support WAMR.

Gains

Different speech coding technologies are compared in WCDMA for UMTS - Radio Access for Third Generation Mobile Communications, 3rd Ed. The book reveals that WAMR can provide better subjective experience given similar bandwidths.

DeploymentThis feature needs to be supported by terminals, RNS, and CN.This feature is optional and needs license support.

Serial number of the feature: WRFD-010613

Availability: This feature is introduced since version RAN6.0 and is a 3GPP R5 feature.

Risks and CautionsCurrently, no mobile phones support the WAMR feature. Certain Nokia terminals, however, can support WAMR after Nokia helps to update the software.

TFO/TrFO

Recommendation LevelHigh

PrincipleTrFO is short for Transcoder Free Operation, that is, calls from mobile terminal UE to UE do not go through TC coding and decoding processes.

Coding and decoding units TC on WCDMA networks belong to MGW. When two mobile terminals on the R4 network communicate, AMR speech used on the WCDMA terminal is transmitted on the entire path and does not need to go through TC coding and decoding conversion, that is conversion from AMR speech to G.711PCM speech and then to AMR speech.

TrFO is implemented through outband signaling. It can also be implemented through in-band signaling, namely TFO. During decoding, the TFO module inserts indicator bits according to certain rules into the frame data sent by a mobile phone. During coding, either TFO or TC is used. If TFO can extract the indicator bit from the code stream, TC coding is not used. Otherwise, TC coding is used.

That means compared with TrFO, TFO occupies certain TC resources for extracting TFO indicator bits. TFO uses two lower bits of the PCM frame for transmission. That is, the medium and the low bits in the TDM line transmit AMR/EFR speech data. The six bits above are wasteful, and therefore do not save bandwidths. Improvement of speech quality, however, is the same.

GainsImprovement of speech quality is ensured only for mobile phone to mobile phone calls, at about 0.2.

DeploymentBoth the CN and RAN need to support this function.

The TrFO function is activated if the RNC supports IUUP Version2. The TrFO function is enabled if both the CN and RNC support it.

RAN sideVersion requirement: This feature is introduced since version RAN3.0 and is a 3GPP R4 feature.Serial number of the feature: WRFD-011600. This feature is optional and customers need to buy licenses.Switch: In addition to licenses, the IUUP Version 2 must also be enabled.

RNC LMTCommand: SET UCORRMALGOSWITCHParameter: IUUP_V2_SPT_SWITCH. Value 1 indicates that the switch is on. Value 0 indicates that the switch is off.Description of the command:IUUP_V2_SPT_SWITCH (switch of supporting IUUP version 2): When the switch is on and "supporting IUUP version 2" is activated, the RNC supports the FTO/TrFO function.

CN sideFrom the access signaling of voice services, we can check whether the CN supports the function.Signaling is RAB ASS REQ of the Iu interface. The source is uP-ModeVersions:0000000000000011 in the figure below.

The IUUP version that the CN supports is indicated with 16 bits, indicating whether V16 to V1 are supported separately. If a version is supported, the value is 1.In up-ModeVersions:00000000000011 in the above figure, the last two bits are 1 and all the other bits are 0. Therefore, V2 and V1 are supported and other versions are not supported.Huawei products generally support V1 and V2.The CN TrFO function is enabled on the MSC. Enable the TrFO function for specific peer end devices. For example, enable the TrFO function for the RNC210 by running the following command:MOD OFC: ON="RNC210", SUPTTRFO=YES;

Risks and CautionsUE compatibility issues may cause call drops.

In RAB assignment, the CN sends the coding formats allocated when a user opens an account to the RNC. AMR has up to nine possible rates.

In IUUP version 1, Huawei RNC does not support all transmission formats, reserving only rates GBR and MBR, which are generally 12.2 Kbps and 4.75 Kbps. In this case, AMRC cannot be implemented.

Huawei IUUP version 2 supports all transmission formats.

After IUUP version 2 is opened, up to nine RFCIs can be sent by the RNC. Certain Samsung mobile phones that use earlier versions of Qualcomm chips do not support blind detection of transmission formats when more than three AMR rates are sent, and therefore cause call drops.

Workaround: Disable the algorithm for blind detection of transmission formats.

SET UCORRMALGOSWITCH: ChSwitch= CFG_DL_BLIND_DETECTION_SWITCH-0;

According to theoretical estimate, the gain of blind detection is 0.3 dB, corresponding to a 7% increase of capacity. Disabling blind detection reduces the capacity of air interfaces.

CN compatibility issues may cause call drops.

When the TrFO function is enabled, throughout the speech call, speech data coded through compression is not encoded or decoded, but transmitted in a transparent mode from the calling party to the access network, to the core network, then to the peer core network, and back to the access network.

In this case, in the process of setting up IUUP for the first time on the core network, the set of rates supported by the remote core network is not known. After negotiation, the remote core network does not support the rate set and requires the local core network to re-negotiate.

Therefore, the core network initiates re-negotiation of IUUP by another RAB assignment. The RNC side finds that the first IUUP negotiation is not completed (Processing of the RAB assignment process is not reasonable at the RNC side to a certain extent. In the process of setting up the RAB signaling, RNC does not wait for initialization of IUUP users. That is, IUUP initialization may take a long time and the signaling of RAB is already completed.), and initiates IUUP reconfig. In the process of reconfig, however, the status of IUUP needs to be checked. The IUUP status is found incorrect and a failure is returned to L3, causing call drop.

AMRC

Recommendation LevelLow

This function brings small gains and big compatibility risks.

PrincipleAdaptive multi rate control (AMRC) can control eight types of voice services through RNC, namely 12.2 kbit/s, 10.2 kbit/s, 7.95 kbit/s, 7.4 kbit/s, 6.7 kbit/s, 5.9 kbit/s, 5.15 kbit/s, and 4.75 kbit/s as well as WAMR 6.60 kbit/s, 8.85 kbit/s, 12.65 kbit/s, 15.85 kbit/s, and 23.85 kbit/s. This feature can improve speech quality, expand uplink coverage, and reduce system load level.

Before RAN5.0, the downlink transmit power and uplink UE transmit power are considered in AMR adjustment. If the transmit power exceeds the threshold, the link quality is poor.

In RAN5.1, cell load is used for triggering AMRC. RNC continuously monitors cell load in dynamic mode and adjusts user speech code rate according to load changes. When the load is heavy, the low rate of AMR speech CODEC is used for reducing cell load. When the load is light, the high rate of AMR speech CODEC is used for providing higher quality speech.

AMRC-is an operation item in the process of load reshuffling. When the downlink common measurement (TCP) and the uplink common measurement (RTWP) of NodeB exceed the threshold of LDR, LDR is triggered. LDR is a way of controlling congestion. Then, the system enters the basic congestion status. After LDR is triggered, AMRC can be used for reducing system load. The RNC selects candidate AMR users on the basis of ARP and current user rate. Users with low ARP are first selected for rate adjustment. If ARPs are the same, users with high speech rate are selected first.

After the user speech rate is degraded, the rate is improved by using the downlink transmit power and UE uplink transmit power. The mechanism is the same as that used in RAN5.0.

GainsExperiment 1a - Test Results

1.0

2.0

3.0

4.0

5.0

Conditions

M OS

EFR12.210.27.957.46.75.95.154.75

EFR 4.01 4.01 3.65 3.05 1.53

12.2 4.01 4.06 4.13 3.93 3.44 1.46

10.2 4.06 3.96 4.05 3.80 2.04

7.95 3.91 4.01 4.08 3.96 3.26 1.43

7.4 3.83 3.94 3.98 3.84 3.11 1.39

6.7 3.77 3.80 3.86 3.29 1.87

5.9 3.72 3.69 3.59 2.20

5.15 3.50 3.58 3.44 2.43

4.75 3.50 3.52 3.43 2.66

No Errors C/I=16 dB C/I=13 dB C/I=10 dB C/I= 7 dB C/I= 4 dB C/I= 1 dB

Low rate can ensure better speech quality in areas with poor coverage. On most sites, however, only a small proportion of areas with poor coverage can make gains.

DeploymentThe switch needs to be turned on and the license needs to be obtained.

Switch: SET UCORRMALGOSWITCH: CsSwitch=CS_AMRC_SWITCH-1;

License: serial number of the feature: WRFD-020701.

Availability: This feature is first introduced since version RAN2.0 and is a 3GPP R99 feature.

Risks and CautionsCompatibility problems may occur when this function is integrated with the core network or RAN of other companies. Service may not be set up, calls may be dropped, and one way audio may occur.

Currently, users are not recommended to turn on this switch. According to information obtained, other vendors do not turn on this switch either.

Formulating Optimization SolutionGenerally, Huawei lacks experience in this aspect and not many features are available in the current version.

Feature optimizationIn UE2UE scenarios, speech quality can be improved through the TFO/TrFO feature. It is not effective for UE2PSTN.

For areas with poor coverage, the AMRC feature can be used for improving the speech quality. However, not much commercial use experience is accumulated about this feature.

Parameter OptimizationFor networks with a small number of users, power control parameters of the outer ringlet of voice services can be modified. That is, sacrifice the capacity for quality improvement.

Discuss the specific modified values of parameters with R&D and make comprehensive evaluation.

The following four methods are common methods for optimizing MOS:

Increase the maximum downlink transmit power by 3 dB for 12.2 K CS services:MOD UCELLRLPWR: CNDomainId=CS_DOMAIN, MaxBitRate=12200, RlMaxDlPwr=30The benefit of the method is that it increase MOS scores by decreasing downlink BLERs in poor coverage areas.The disadvantage is that it increases the downlink transmit power, which worsens congestion in some congested cells and increases Ec/No.

Increase the minimum downlink transmit power by 3 dB for 12.2 K CS services:MOD UCELLRLPWR: CNDomainId=CS_DOMAIN, MaxBitRate=12200, RlMinDlPwr=-120The benefit of this method is that it increases average MOS scores by decreasing the downlink BLERs in areas with good coverage.The disadvantage is that it increases the downlink transmit power, which worsens congestion in some congested cells and increases Ec/No.

Increase the minimum value of SIR target:MOD UTYPRABOLPC: RABINDEX=0, SUBFLOWINDEX=0, TRCHTYPE=TRCH_DCH, DELAYCLASS=1, MINSIRTARGET=82, (MINSIRTARGET is an internal parameter. Enter it in the command box and run it.)The benefit of this method is that it restrains BLERs when signals are strong and it brings low risk. However, it only applies to uplink outer loop power controls and effects are insignificant.

Increase BLER values based on objectives:

MOD UTYPRABOLPC: RABINDEX=0, SUBFLOWINDEX=0, TRCHTYPE=TRCH_DCH, DELAYCLASS=1, BLERQUALITY=-30, SIRADJUSTSTEP=1Adjusting BLERQUALITY can greatly restrain BLERs to increase MOS scores, but you need to adjust SUR adjustment steps synchronously. This method significantly increases uplink and downlink transmit power and causes congestion. Exercise caution when adjusting this parameter.

CASE:

Due to contractual cluster signed off between huawei and XXX operator of MOS=3.7, but drive test result not more than 3.5 found so could not submit to report customer for acceptance.

Summary of issue:

• Contractual signed off between huawei and XXX operator for MOS=3.7 but after cluster drive test result of MOS found=3.45

Drive test KPI of cluster:

MOS PLOT:

To get target value of MOS ie 3.7, we have done:

Checked cluster audit properly, no more obstruction found Checked NBR of whole cluster No major alram found in site No issue of RTWP No issue of BLER All KPI found normal.

Parameter OptimizationRNC LMTCommand: SET UCORRMALGOSWITCHParameter: IUUP_V2_SPT_SWITCH. Value 1 indicates that the switch is on. Value 0 indicates that the switch is off.

SETUCORRMALGOSWITCH:CsSwitch=CS_IUUP_V2_SUPPORT_SWITCH-1;

After implementaion of TRFO, drive test of cluster has done:

Reference Information

Tool Guide

Probe User Guidehttp://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=searchManualTableOfContents&node=000001104256&colID=ROOTWEB|CO0000000064&web_doc_id=SC0000575221&doc_type=null&p_line=29-7

RNC LMT User GuideBSC 6810:

http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=searchManualTableOfContents&node=000001416232&colID=ROOTWEB|CO0000000064&web_doc_id=SC0000486471&doc_type=null&p_line=29-7

BSC 6900:

http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=searchManualTableOfContents&node=000001056886&colID=ROOTWEB|CO0000000064&web_doc_id=SC0000578329&doc_type=null&p_line=29-7