gruppo telecom italia etsi ttcn-3 user conference and model based testing workshop 11-14 june, 2012...
Post on 30-Dec-2015
228 Views
Preview:
TRANSCRIPT
GRUPPO TELECOM ITALIA
ETSI TTCN-3 User Conference and Model Based Testing Workshop11-14 June, 2012 – Bangalore, INDIA
Telecom Italia LAB / T.TG.TL.APSERAZIO Luca
MONET (MObile NEtwork Tester)Telecom Italia experience
Telecom Italia LAB / T.TG.TL.APSERAZIO Luca
Index
► Testing Lab and Telecom Italia’s Test Plant► Testing areas and objectives► Testing automation► Testing Methodology ► The MoNeT solution► Test Scenarios► A test scenario► Monet history ► System framework► Economics► Next steps
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
3
Telecom Italia offers technological infrastructures and platforms in which voice and data are converted into advanced telecommunications services - as well as the latest ICT and Media solutions. These tools cater for the Group's as well as the country's growth.
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
Telecom Italia Group
Our presence and role in the Italian market
Key numbers - First Half 2011
» 31.3 million mobile lines
» 9.1 million broadband accesses, of which 1.9 million wholesale customers
» 15 million retail network connections to the fixed network in Italy, of which 7.2 million retail broadband accesses
» 3.65% daily average audience share by La7
» 49.3 page views and 4.1 million single visitors to Virgilio (at March 31, 2011)
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 4
Telecom Italia in the world
Italy, Brazil and Argentina are the strategic markets where the Telecom Italia Group operates.
The leadership of our Group at national level originates from an exhaustive offer aimed at transforming traditional communication into digital communication: activities carried out through companies operating on the domestic market range from land to mobile telephony, internet, research and development, office products and services, television and entertainment, media and ICT.
The Group is present in Brazil with TIM Brazil, one of the most important players because of the quantity of mobile lines. In Argentina the Group operates through Telecom Argentina in the sectors of land and mobile lines, internet and data transmission. Telecom Argentina also offers mobile telephony services in Paraguay.
The Telecom Italia Group is also active in other parts of Europe, North America, Africa and Asia through Sparkle, Lan Nautilus and Med Nautilus, Telecom Italia’s local operational companies.
The Group can be accessed throughout the world by contacting Telecom Italia and Telecom Italia Sparkle’s representative offices.
Telecom Italia in the world
Our presence and role in Latin America
Abroad Group's focus is on Latin America.It operates in Argentina and Paraguay, providing fixed and mobile telephony services and internet through Telecom Argentina group.In Brazil, TIM Brasil remains one of the major players.
Key numbers - First Half 2011
Brazil» 55.5 million mobile lines» 25.5% of market share
Argentina
» 4.1 million fixed lines» 1.5 million broaband accesses» 17.4 million mobile customers
Paraguay
» 2 million mobile lines
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
5SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
Testing LabTesting Lab is responsible of the maintenance and the management of test laboratories such as the Mobile Network Lab used for the validation of mobile network platforms and architectures, terminals and services.
Services development phases are paired with testing in order to reduce the impact and the probability of faults in older phases of the process.
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 6
Telecom Italia’s test plant: an overview
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 7
► 2 HLRs
► 5 MSCs
► 5 MGWs
► 3 SGSNs
► 3 GGSNs
► Several service nodes in continuous configuration and adaptaion to services (SMSC, TGDS, INS, IVR, …)
► Interconnecting IP backbone
Telecom Italia’s test plant: the numbers of the core network
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 8
► Verification and validation of:
► New releases, patches and data
► New network elements and new hardware
► New network architectures
► Regression testing
► Network analysis under traffic conditions
► Toll Ticketing validation
► Test plant chains control using an end-to-end approach oriented to the final user
► Testing remotization
Testing areas and objectives
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 9
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.
Automation benefits
► Reduction of time and costs for test execution and verdict evaluation
► Avoiding repetitive manual activity which is prone to error
► Saving costs related to human experts and test bed occupation
► Reduction of time to market for services, products and network architectures
Testing automation
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
10
Testing Methodology The testing methodology applied is End-to-End testing with protocol conformance checks at the main SUT interfaces which is aimed to ensure the delivery of service to the final user with the correct communication between network elements.
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
SUT
EUTA
EUTB
USERA
USERB
Interface conformance checks
E2E functionality
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 11
► MoNeT (MObile NEtwork Tester) is an automatic tool for the testing of the Mobile CS Core Network
► The tool is connected directly to the mobile core Network by network interfaces: A interface for GSM and Iu-CS interface for UMTS and acts over these interface with the same signalling generated by an end user over the radio interface
► The platform offers, from a web based remote GUI:
► A tool for the automatic execution of test lists and test scenarios
► A virtual mobile terminal with all its functionalities
The MoNeT solution
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 12
SUT .
The MoNeT solution: test plant deployment
MONET
MONET 3GMONET 2G
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
13
► Basic call► Supplementary Services
► CFB Call Forwarding on Busy► CFU Call Forwarding Unconditional► CFNRy, Call Forwarding no Reply► CFNRc, Call Forwarding no Reachable► CW Call Waiting► CH Call Holding► MPTY Multi Party Service► CLIP Calling Line Identification Presentation► CLIR Calling Line Identification Restriction► COLP Connected Line Identification Presentation ► COLR Connected Line Identification Restriction► Call Barring
► Emergency call► Narrow Band and Wide Band call► SMS► Unsuccesful scenarios► Multi toll ticketing► Intelligent Network Services (TIM custom VAS services: TCA, TIT, RIF, I&B, 2in1, PN,
Premium rate, PagoIO, SOS PayForMe), ► Hand over (3G-2G, 3G-3G, 2G-3G, 2G-2G)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP
Test Scenarios
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 14
A test scenario: Call Forwarding on Busy - qualitative
User A User B User C User DMobile
Core Network
CFB activation, destination USER D
Auxiliary call from B to A to establish busy state
Active call
Call from C to A forwarded to D
Active call
End calls
CFB deactivation
Preamble
Postamble
Test
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 15
A test scenario: Basic call - signalling
User A User B Mobile Call Network
REGISTER (Deactivate SS - CF and CW)
Preâmbulo
Test
RELEASE COMPLETE
REGISTER (Deactivate SS - CF and CW)
RELEASE COMPLETE
SETUP (Speech, CdPN = B)
CALL PROCEEDING
SETUP
CALL CONFIRM
ALERTING
ALERTING
CONNECT
CONNECT ACKNOWLEDGE
CONNECT
CONNECT ACKNOWLEDGE
Active call
DISCONNECT
RELEASE
REL COMP
DISCONNECT
RELEASE
REL COMP
MONET emulates the behaviour of the users at the mobile core network interfaces (A or Iu-CS) enabling for each user to:► Perform protocol procedures
(including settimg of test pre-conditions)
► set the value of each field of sent signalling messages at Call Control level (e.g. Bearer Capability, CdPN, …)
► check (or ignore) each field of received signalling messages at call control level (e.g. Release cause,received Bearer Capability, CgPN, …)
These features enable MoNeT to:► Verify the correctness of the
service and of the signalling procedures
► Produce the logs, the traces and the verdict of each test
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 16
► 2001: feasibility study
► 2002: platform development and integration with scenarios from Eurescom P613 project (TTCN-2)
► 2003: debugging of call scenarios
► 2004-5: continuous extension of call scenarios with TIM custom serveces
► 2006-2008: project frozen for the realization of Mobile Network Labs in Turin
► 2009: extension of call scenarios
► 2010: extension of call scenarios with Hand-over procedures
► 2011: feasibility study for migration to TTCN-3
► 2012: migration to TTCN-3
Monet history
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 17
The MoNeT is composed of:
► Linux server with:
► web server for remote control
► signalling board for interconnection with 2G network (A interface)
► PSTN modem for calls involving PSTN lines
► Ixia Catapult for interconnection with 3G network (Iu-CS interface)
► A generic web client
System hardware framework
Control Unit
Sch
eda S
TM
1
SS
7 In
terf
WEB Client
MONET
Ixia DCT2000
IPIP A Interfaces
CS 2G/3G Core Network
CS 2G/3G Core Network
Iu-CS Interfaces
IPIPW
EB
Serv
er
PSTN
PSTN
PS
TN
Inte
f.
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 18
System software framework
STM-1 Board
Linux Based PC
Test Script Language
User Plane Control Plane
Physical Layer
Emulated Protocol layers (MTP3b, SSCF, SSCOP, AAL5, ATM)
Simulated Protocol layers
(CC/SS/SMS, MM)
Adaptation and custom functions
TTCN tool
TTCNTest Suites
WEB Server
GUI
Iu-CS Interface
Emulated Protocol layers (RANAP, SCCP)
Emulated Protocol layers (ALCAP, STC)
Emulated Protocol layers
(Iu-UP, AAL2, ATM)
Test Manager
Interfaccia utente
GSM-A Interface Manager
Report
PSTN Interface Manager
Database
TestList
GSM-A Interface Driver PSTN Interface Driver
GSM-A Interfaces PSTN Interface
Terminale Virtuale
IU Interface Manager
IU Interface Driver (ddriver)
IU Interfaces
Terminale Virtuale
Test Manager
User Interface
Tests
GSM-A Interface Manager
Report
PSTN Interface Manager
Database
TestList
GSM-A Interface Driver PSTN Interface Driver
GSM-A Interface PSTN Interface
IU Interface Manager
IU Interface Driver (ddriver)
IU Interface
VirtualTerminal
(ETS)VirtualTerminal
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 19
Migration from TTCN-2 to TTCN-3
TTCN-2 Source Code Translation• 30 kLoC• more than 60 test cases• more than 130 test functions
Migration of External Functionality• Test Suite Operations• Test Suite Operations Procedures
Test Adapter Implementation• CP Implementation• PCO Implementation• Codec Implementation
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 20
► Automated translation of
► test types
► test data
► test behavior
► Because of a good quality (TTCN-2 standard compliance) of the TTCN-2
► nearly no manual efforts
TTCN-2 Source Code Translation and Functionality Migration
► Extern Functionality► TSOs
► TSOPs
► Translated to► TTCN-3 Predefined functions
► TTCN-3 Functions
► External functions that have to implemented in the platform adapter
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 21
Coordination Point / CP► No need for implementation.
Provided by TTCN-3 tools
► Cleary separated at runtime
Test adapter implementation (I)Communication Implementation
Point of Control and Observation / PCO
► Repackaging
RepackagingFunctionality
OS spec. handling
TTCN-2 specific integration
Functionality
OS spec. handling
TTCN-3 Port Impl. / SA
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 22
► ASN.1 Based Codecs
► No effort as TTCN-3 tool support all of the ASN.1 requirements and encoding rules, built-in
► Non-ASN.1 Based Codecs
► Decision is to reimplement the non-ASN.1 codecs
► Codec generation approach suitable
► Low-efforts, high-performance
► Low-risk
► Only component that really has to be fully touched!
Test Adapter Implementation (II) Codec Implementation
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 23
TTCN-3 based MoNet Architecture
TM / TL
TECH
Test System User
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
Test System User
WEB Server
GUI
CMD Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Line Interface
TECH
CD
SA
Plu
gin
s
SA
Plu
gin
s
SA
Plu
gin
s
PAPlu
gin
s
CMD Line
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 24
► Context:
► Regression test for FOA CNCS8.0
► Activity duration: 4 months
► Test cases for each session: 470
► Rate of usage: several daily execution on each testing chain (6 testing chains combining 3 MSCs and 2 HLRs) after every new software installation or correction
► Overall usage: over 200 session which means 200x6x470>550K tests (about 5000 tests/day)
Economics: an application revenue
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 25
Economics: the BEP between automated and manual testing
The BEP (Break Even Point) is reached with a limitated number of executions of the test list.
Ipothesys Manaul Testing Automated Testing
Hardware (terminals, PC, Interfaces, ...) 10 [KEuro] 230 [KEuro]
Automatic testing platform development - 3 Person Years
Test Execution Time (user selection, execution, analysis) ~30 [min] ~5 [sec]
Testing environment setting ~0,2 Person YearsWorking days/year 214Number of tests 500Expert people cost/year ~60 [KEuro]
1
11
21
31
41
51
0
100
200
300
400
500
600
700
800
GSM Network Regression Testing - Costs
Total Cost (manual)
Total Cost (automated)
Test list executions
KE
uro
BEP
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 26
Economics:the effort required for automated and manual testing
During the FOA CNCS8.0 activity, the numer of test executed to validate software and corrections is so high that it shouldn’t be possible to apply the methodology with a manual approach.
1 201 401 601 801 1001 12010
50
100
150
200
250
300
GSM Network Regression Testing - Total Time
Test execution Time (manual)
Test execution Time (automated)
Test List Executions
Pe
rso
n Y
ea
rs
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 27
► CS scenarios extension
► PS scenarios
► User plane handling and check
► Interfaces migration over IP
Next steps
ETSI TTCN-3 User Conference and Model Based Testing WorkshopMONET (MObile NEtwork Tester)
SERAZIO Luca, Telecom Italia LAB / T.TG.TL.AP 28
THANK YOU FOR YOUR ATTENTION!
QUESTIONS?
top related