open accessible coded test for 3gpp mobile conformance

1

Open Accessible Coded TestOpen Accessible Coded Testfor 3GPP Mobile Conformancefor 3GPP Mobile Conformance

Shicheng HuShicheng Hu

Leader of MCC task 160 / 272Leader of MCC task 160 / 272

16th Dec. 200416th Dec. 2004

Open Source Application & Standards Open Source Application & Standards WorkshopWorkshop

2

About myselfAbout myself

• Born in Shanghai, ChinaBorn in Shanghai, China• Home city, Nuremberg, GermanyHome city, Nuremberg, Germany• Nationality: FrenchNationality: French• Having worked for ETSI > 11 years Having worked for ETSI > 11 years • Protocol Testing Competence Centre Protocol Testing Competence Centre • Leading mobile testing for 3GPPLeading mobile testing for 3GPP• [email protected]@etsi.org

3

ETSI Open Test StandardsETSI Open Test Standards

• ETSI has a large pool of open test standardsETSI has a large pool of open test standards• Coded abstract test suites in TTCN languageCoded abstract test suites in TTCN language• IPR belongs to ETSIIPR belongs to ETSI• Open access, free of chargeOpen access, free of charge• ETSI has led the workETSI has led the work• Internet testInternet test

– VoIP – H.225, SIP testVoIP – H.225, SIP test• Fix network testFix network test

– ISDN / BISDN, Supplement Service test, IN testISDN / BISDN, Supplement Service test, IN test• Wireless testWireless test

– DECT, TETRA, Hipper-LAN/MAN/Access testDECT, TETRA, Hipper-LAN/MAN/Access test• Mobile testMobile test

– GSMGSM– 3GPP mobile TTCN-coded test3GPP mobile TTCN-coded test is the latest open test standard is the latest open test standard

4

ContentsContents

• Why 3GPP testWhy 3GPP test• What to testWhat to test• How to reachHow to reach• Who worksWho works• ResultResult• ConclusionsConclusions

5

ContentsContents


6

3GPP complex & New Hope3GPP complex & New Hope• Huge investment for 3GPP deploymentHuge investment for 3GPP deployment• 3GPP system provides high bit rate access, 0.38, 2, 10, … 3GPP system provides high bit rate access, 0.38, 2, 10, …

Mbits/sMbits/s• Capable of providing wide range of servicesCapable of providing wide range of services• 3GPP is a complex3GPP is a complex

– Three complicated radio technologies, FDD, TDD, GERANThree complicated radio technologies, FDD, TDD, GERAN– Enhanced (U)SIM applicationsEnhanced (U)SIM applications

• User equipment (UE) integrates service access, the User equipment (UE) integrates service access, the technologies and USIM in an intelligent, sexy, secure, technologies and USIM in an intelligent, sexy, secure, easy-use … easy-use … mobile toymobile toy

– Let the Let the toys toys in the hand of everybody for playing/use in the hand of everybody for playing/use everywhere at anytimeeverywhere at anytime

– TheyThey should help to make money in a smart way for all 3G should help to make money in a smart way for all 3G Operators, manufacturers, service / contents providers …Operators, manufacturers, service / contents providers …

7

3GPP System Bottleneck3GPP System Bottleneck• 3GPP provides a system solution for a generation3GPP provides a system solution for a generation• 3GPP system components:3GPP system components:

– Core networksCore networks– InfrastructuresInfrastructures– MobilesMobiles– USIM cardsUSIM cards

• Bottleneck of the component chain is mobiles (UEs)Bottleneck of the component chain is mobiles (UEs)• Mobiles are the Hi-Tech products ofMobiles are the Hi-Tech products of

– Radio technologyRadio technology– MicroelectronicsMicroelectronics– Protocol stacksProtocol stacks– Software platformSoftware platform– Entertainment electronicsEntertainment electronics

• Mobile test is always the last step before system Mobile test is always the last step before system deploymentdeployment

8

Conformant UE = Interoperability Conformant UE = Interoperability

• UE InteroperabilityUE Interoperability towards access technologies is the towards access technologies is the key issuekey issue for mobility for mobility

• Conformant (U)SIM against different UEsConformant (U)SIM against different UEs• Nobody can make money without an interoperable UENobody can make money without an interoperable UE

– This truth is well understood by the 3GPP industryThis truth is well understood by the 3GPP industry

• Conformant UE = Interoperability (of UE in 3G networks) Conformant UE = Interoperability (of UE in 3G networks) • Solution: conformance testSolution: conformance test

– Very detailed test specificationsVery detailed test specifications

– Based on the 3GPP core specsBased on the 3GPP core specs

– Low level testLow level test

– Before commercial products entering marketBefore commercial products entering market

– Voluntary declaration, but certifiedVoluntary declaration, but certified

– One of the most important UE qualityOne of the most important UE quality benchmarks benchmarks

9

ContentsContents


10

Scope of Conformance TestScope of Conformance Test

• USIM functional test (TS 31.121)USIM functional test (TS 31.121)– Identification, PIN, account charging, network priority, phonebookIdentification, PIN, account charging, network priority, phonebook

• UICC test (SIM electrical test) (TS 102.230)UICC test (SIM electrical test) (TS 102.230)– power on/off, 1.8v/3v detection, presence detectionpower on/off, 1.8v/3v detection, presence detection– bit/character transmitting duration, command processing and error bit/character transmitting duration, command processing and error

handling, … handling, … • Acoustic test (TS 26.132)Acoustic test (TS 26.132)

– Loudness rating, idle noise, echo …Loudness rating, idle noise, echo …• Radio Frequency & Radio Resource Management test (TS Radio Frequency & Radio Resource Management test (TS

34.121)34.121)– Many characteristics tests of transmitter & receiverMany characteristics tests of transmitter & receiver– Many difficult performance testsMany difficult performance tests

• Protocol and signalling test via radio interface (TS 34.123)Protocol and signalling test via radio interface (TS 34.123)– Heart of conformanceHeart of conformance– Most difficult part Most difficult part – Unambiguous & less tolerant (leading to verdict Pass or Fail)Unambiguous & less tolerant (leading to verdict Pass or Fail)– Time consumedTime consumed at specificationat specification

11

ContentsContents


12

How to Reach 3GPP mobileHow to Reach 3GPP mobileProtocol /signalling ConformanceProtocol /signalling Conformance

• Need rigorous test specs in TTCN (Need rigorous test specs in TTCN (coded testscoded tests))– Decided by 3GPP in 2000 Decided by 3GPP in 2000 – Black box test methodBlack box test method– A commercially available UE (UMTS mobile) under testA commercially available UE (UMTS mobile) under test

• Use ETSI “Specialist Task Force” (STF) tool and facilities to Use ETSI “Specialist Task Force” (STF) tool and facilities to accelerateaccelerate

• Request serviceRequest service• Long term commitments from 3GPP and the test industry Long term commitments from 3GPP and the test industry

– The project takes several yearsThe project takes several years– Costs millions €Costs millions €

• Share the costs and ensure the fundingShare the costs and ensure the funding– 3GPP funding from Organisation Partners 3GPP funding from Organisation Partners

• Main source, nearly 800k€ (61 mm) /year) Main source, nearly 800k€ (61 mm) /year) – Voluntary contributions as cash from the marketing partnersVoluntary contributions as cash from the marketing partners

• Range from 65k€ - 150k€ (5 – 12 mm) /yearRange from 65k€ - 150k€ (5 – 12 mm) /year– Man-power as free-of-chargeMan-power as free-of-charge

• Range from 15 - 18 mm /yearRange from 15 - 18 mm /year

13

3G Test industry commitments3G Test industry commitments

• The TTCN test cases are implemented in the The TTCN test cases are implemented in the System Simulators in different platformsSystem Simulators in different platforms

– Secure the implementation commitments before Secure the implementation commitments before starting the projectstarting the project

• SS vendors organise TC verifications and SS vendors organise TC verifications and regression testsregression tests

– To make implemented TC runningTo make implemented TC running– A running TC has a high real valueA running TC has a high real value– A run TC will continue running after changesA run TC will continue running after changes

• Early identify the clients who deploy the TTCN Early identify the clients who deploy the TTCN test casestest cases

– Prioritising TCsPrioritising TCs– Scheduling the deploymentScheduling the deployment– CertificationsCertifications

14

Clients of 3GPP testClients of 3GPP test• Global Certification Forum (GCF)Global Certification Forum (GCF)• PCS Type Certification Review Board (PTCRB )PCS Type Certification Review Board (PTCRB )

– GCF + PTCRB are the major clients for W-CDMA (FDD) GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs& GSM/GPRS test specs

• TD-SCDMA Industry Alliance (TDIA)TD-SCDMA Industry Alliance (TDIA)– TDIA is the major clients for TD-SCDMA (LCR TDD) TDIA is the major clients for TD-SCDMA (LCR TDD)

test specstest specs

• GCF WI-10 selected 400 TCs as high priorityGCF WI-10 selected 400 TCs as high priority– Divided in 4 packagesDivided in 4 packages– Progressing sequentially one by one (milestone 80%)Progressing sequentially one by one (milestone 80%)

• UMTS mobile certification will be started if 80% UMTS mobile certification will be started if 80% of each package are validatedof each package are validated

15

ContentsContents


16

Who writes TTCN TCsWho writes TTCN TCs• MCC task 160 created 2000MCC task 160 created 2000• 13 experts with excellent 3G + TTCN knowledge13 experts with excellent 3G + TTCN knowledge

– from 4 organisation partnersfrom 4 organisation partners• Currently from 3 partnersCurrently from 3 partners

• MCC task 272 split out from MCC task 160 for 2G/2.5G -> MCC task 272 split out from MCC task 160 for 2G/2.5G -> 3G TCs3G TCs

• 8 ATSs have been generated8 ATSs have been generated– NAS CS and PS signalling testNAS CS and PS signalling test– RRC testRRC test– 3G <-> 2G/2.5G interRAT tests3G <-> 2G/2.5G interRAT tests– L2 tests (RLC, MAC …)L2 tests (RLC, MAC …)– Radio access bearer testRadio access bearer test– SMS testSMS test

• 4 formal + 17 interim working deliveries / year4 formal + 17 interim working deliveries / year

17

Methodology for 3G testMethodology for 3G test

Test Cases

(TTCN)

34.123-3, 51.010-5

PICS (capability

applicability)

34.123-2, 51.010-2

Test Purposes,34.123-1 Descript (prose), 34.108,

51.010-1

Test Report

logging and

analysis

Core specs

25.331, 25.321, 2 24.008, 44.018, 44.060

implementation

testing

SSMobile Under Test

compilation

Executable Test Suite

(e.g., C++)

18

How to approve TCHow to approve TC• Test cases are approved by e-mailTest cases are approved by e-mail

– No quality compromiseNo quality compromise– Two weeks consultation periodTwo weeks consultation period– TC approval one by oneTC approval one by one

• A golden ruleA golden rule: TC must run at least on one SS platform against : TC must run at least on one SS platform against two independent commercial mobilestwo independent commercial mobiles

– SS vendors raise CR for a new TC approvalSS vendors raise CR for a new TC approval– Log files and radio conditions as support information for approvalLog files and radio conditions as support information for approval

• Reason behindReason behind– TTCN TC, SS and UE are developed at the same timeTTCN TC, SS and UE are developed at the same time– None of them is perfectNone of them is perfect

• Taken two years to approve the GCF prioritised TCs for 3G Taken two years to approve the GCF prioritised TCs for 3G interoperabilityinteroperability

– 100 TC approved in 2003, 100 TC approved in 2003, – 250 TC approved in 2004250 TC approved in 2004– The development of these TCs took also two years beforehandThe development of these TCs took also two years beforehand

• All SDOs of 3GPP OP own IPR of 3GPP ATSsAll SDOs of 3GPP OP own IPR of 3GPP ATSs– Open access to TTCN codes Open access to TTCN codes

19

ContentsContents


20

ResultResult• 400 TCs in 8 ATSs, 25 MB codes400 TCs in 8 ATSs, 25 MB codes

– Platform independentPlatform independent– Running on > 4 platformsRunning on > 4 platforms

• We have reached the GCF We have reached the GCF 80%80% milestone milestone– 3G mobile certification starting at beginning of 20053G mobile certification starting at beginning of 2005– The light is green for mass 3G mobiles entering the marketsThe light is green for mass 3G mobiles entering the markets– 4 types of 3G mobiles are ready to go4 types of 3G mobiles are ready to go

• Open the real 3G test businessOpen the real 3G test business– 4 validated 3G SS platforms available in the markets4 validated 3G SS platforms available in the markets– 4 test houses in Europe opening 3G test services4 test houses in Europe opening 3G test services– 3G TTCN experts becoming hot in the labour market3G TTCN experts becoming hot in the labour market

• We know the price of each 3G coded TCWe know the price of each 3G coded TC– 10 k€ /TC10 k€ /TC– 3G investment for 4 years: 4 mil.€ for 400 TCs3G investment for 4 years: 4 mil.€ for 400 TCs

• All TTCN codes can be downloaded free of chargeAll TTCN codes can be downloaded free of charge• These codes are continuously maintained and deliveredThese codes are continuously maintained and delivered

– Every three weeks for 3G membersEvery three weeks for 3G members– Every three months for everybodyEvery three months for everybody

21

ContentsContents


22

ConclusionsConclusions

• It is a successful software projectIt is a successful software project

• 3GPP investment starts the value return3GPP investment starts the value return

• The 3G mobile bottleneck has goneThe 3G mobile bottleneck has gone

• key for successkey for success– Use ETSI tool having created STFs Use ETSI tool having created STFs

– TTCN teams work closely with test industryTTCN teams work closely with test industry

open accessible coded test for 3gpp mobile conformance

Documents