![Page 1: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/1.jpg)
dfgfdhsjfgdghjgh'(gk(jsrt
Dedicatedtestenvironments–Moneypitorqualitystamp
John-HelgeHofstadEVRY
![Page 2: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/2.jpg)
>10,000
SWE
FIN NOR
UK
LAT
UKR
IN
#1Norway#4Sweden#1FinancialServicesintheNordics
8,600employees
RevenueofEUR*
1.37bn.CommiWedorderbacklogEUR*
2.48bn.
GlobaldeliveryUkraine–Latvia–India
EVRY–TheleadingNordicITServices&So^warevendor
Servingcustomersinpublicandprivatesector
Officesin 8 countries
MarketshareITServicesinNorway
25% LeadingverccalexpercseFinancialservices,localgovernment,publicsector,healthcare,insurance
*Operacngrevenueandbacklog2017
customers
BUL
![Page 3: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/3.jpg)
SWE
FIN
NOR
UK
LAT
UKR
CommiWedorderbacklogMNOK
9 194 *Operacngrevenue2015
SignificantfocusonFinancialServices
RevenueMNOK*
3 576
EVRYFinancialServicesoffices/employees
**OrderbacklogperJan2017
**
IN
IN
Chandigarh
Bangalore
150 customers
12 countries
Servingmorethan GlobaldeliverycapabilitydedicatedworkingexclusivelyforFinancialServicesclients
#1 intheNordics
LargestcommercialITworkforceservingfinancialservicesintheNordics,with
1 299
![Page 4: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/4.jpg)
Customerbase
![Page 5: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/5.jpg)
SOURCE:GARTNER(NOVEMBER2014BY2013REVENUETIER
Amongthetop25BusinessserviceprovidersinbankingacrossWesternEurope
Pan-Europeancustomerbase
![Page 6: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/6.jpg)
CorporateStrategy
FOCUSONVERTICALS
Drivebusinesstowardsvalueaddingindustry-specificsolucons
CreacngDigitalAdvantagetoourcustomers
LEVERAGINGSTRATEGICPARTNERS
SERVICEEXCELLENCE
CloudServices
PROXIMITYTOCUSTOMERS
Investinclosepartnerships
OperaconsDigitalServices GlobalDeliveryCloudServices
![Page 7: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/7.jpg)
Moneypitorqualitystamp?
• TescnginLIVEProduccon • TescngindedicatedtestEnvironments
![Page 8: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/8.jpg)
• ORG• Admin• Environments• Problemscllinger• Hvaløservi• Hvormangeparalelleprosjekterogoppdrag
![Page 9: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/9.jpg)
OrganisaconFinancialServices
WiljarNesse
OperaBonsJonBerntsen
SourcingSisselS.Warhuus
Architecture,Security&SoluBons
RunarRobertsen
SalesPerBjarneValstad
FSFinancialSuiteOperaBons
TrygveLauritzen
DNBOperaBonsPer-OlavNæss
Quality&ChangeManagement
CamillaB.BjerckeOperaBonsCentre
KalleKellgren
DatabaseKjellRuneJørgensen
BOServerTomThorstensen
AOCoreOleChriscanSjøberg
Dev.mentServicesPeWerStørseth
PreProducconoperaconArveLoraas
NonFunc.TescngJohnHelgeHofstad
AOPaymentsTrondGodø
AOChannelsJohnHaraldBrobak
AOCardsOleJacobSelberg
![Page 10: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/10.jpg)
Testorg
![Page 11: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/11.jpg)
TestCentreOfExcellence
TCOE
TOOLS
NFT
FTEnvironments
Automacon
![Page 12: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/12.jpg)
Testmanagement
![Page 13: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/13.jpg)
CCM
![Page 14: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/14.jpg)
CCM–Valuechain
![Page 15: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/15.jpg)
DevOps
![Page 16: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/16.jpg)
SecureDevelopment
• IntegratedDevelopmentEnvironment
• SourceCodeControl
CodingComplete
• CodeQualityCheck• FlawDeteccon• AlignmentwithStandards
CodeReviewComplete • CommitandMerge
SourceCode• AutomaccBuilds• AutomatedTescng
BuildFinished
• StaccCodeAnalysis• AuditTrailControl
SecurityTescngSourcecodeComplete • DeploymentbyTest
EnvironmentAdministrator
• Globalvs.Restricted
ReadyforSystemTest
• Securitytescng(OWASP)
• Performancetescng• FuncconalTescng
ReadyforProduccon• DeploymentbyProducconEnvironmentAdministrator
Finished
![Page 17: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/17.jpg)
Configuraconmanagement
![Page 18: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/18.jpg)
DeploymentwithSalt
![Page 19: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/19.jpg)
ADCM
Newinfrastructuretohandlefiles
Uniformmethodsofdeployment
![Page 20: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/20.jpg)
Admin–Configuraconmanagement
Sharedinstallaconscriptsforallenvironments
Managedconfiguraconfor
easycomparisonofenvironments
Configuraconandinstallaconscriptsin
versioncontrol
![Page 21: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/21.jpg)
Environmentadmin-ChallengesScripBng• RunCommands
OrchestraBon• Setupserver• Createenvironment
![Page 22: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/22.jpg)
DataMasking
![Page 23: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/23.jpg)
Environments
Production environment
Staging area
Testdata masking factory
FS GlobalFS Restricted
GLOBALMasked environment
Only masked or fictive testdata
RESTRICTEDUnmasked environment
Production like or semi-masked testdata
FS CTS
Customer test network
FS Global
FS Restricted
Global source resources Onsite resources Customers
FS CTS
USER
S
![Page 24: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/24.jpg)
Developmentview
Chan
ge fr
eque
ncy
Developer
Developer
Source code management
Automated building
Code Releases
Release Quality
Automateddeployment
Masked
Production ready
Offshore and onshore• Development• System testing
Onshore• Acceptance testing• Reference
Customer• Acceptance testing• Customer testing
Production
Application environmentsDevelopment services
Masked and Unmasked
Unmasked
Releasedepot
Releases
Systemadministrator
![Page 25: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/25.jpg)
FINODS
Card Services
Issuing, Acquiring and Security
Internetbank and “non card clients”
WebServices–load-balancers/MQ
EDB ESB WS_PROXY Card Portal / Clients
Bank Services (non-Card)
Batch, Analysis, Security, Online
Disk SAN – dedicated SAN's to critical systems
Database servers – serving area C and E
http-servers, MQ, filetransfer, SQLproxy, Internet Proxy
Loadbalancers
= Security areas
Area A
Area B
Area C
Area D
Area E
Area F
Area G
Database servers – Cards
![Page 26: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/26.jpg)
Environments-Architecture
![Page 27: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/27.jpg)
BF_PreProd
Internet VPN
IP_Transport
Customer Customer Desktop cli
App
BigIP
bankportal.preprod-restricted.evry.com esb-office.preprod-restricted.evry.com customerportal.preprod-restricted.evry.com (esb-internet.preprodrestricted.evry.com)
Zone: DMZ_Bank Zone: DMZ_Internet bankfilex bankmq banksql
Zone: Bankkontor
Zone: Kjerne Zone: Kanal Zone: MSCloud
Zone: Mainframe
Zone: Infrastruktur
Zone: Tools
Desktop cli
WEB WEB PLP EMQ OCM
CPU3/ZMQ
fs.preprod.no
preprod.local
preprod.local
BigIP
APP APP
IMQ NMQ
ESB
AD
AD
AD
SP CRM
LDAP
Salt
Cli Srv/ app
=> FinODS
DB DB
APP APP
SAN – storage area network Area A
Area B
Area C
Area D
Area E
Area F
Area G Load Balancing Load Balancing
=> Web applications
=> Web services
(other)
App
=> MQ services
Load Balancing
HAProxy HAProxy Load Balancing
=> File extentions
=> DB services
![Page 28: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/28.jpg)
Environmentorders
• Orders• Defects• KPI’s
![Page 29: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/29.jpg)
EnvironmentKPIMetric Explanation Value
Audit Deviation External
ISAE 3402 audits on Test Management Process. Measure in number of deviations last 12 months.
0 >0
Error distribution A-Test vs S-Test
Error distribution A-Test/Pilot vs System Test. Mean value from last 12 months. Large variance from one quarter to another might be experienced.
<10% 10-20% >20%
Delivery precision Test Environments
Percentage of orders registered in the project Test environment in ALM set to “Retest” before the delivery date. Mean value for last quarter.
>90% 80-90% <80%
Test Environment Quality Indicator
Ratio of proven Test environment Defects to the total number of proven defects from System test last quarter.
<5% 5-15% >15%
![Page 30: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/30.jpg)
Nocficacons
![Page 31: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/31.jpg)
ChallengeProducBontesBng
• Priorices• Servicewindows• Consumeddata• Stressandscalability
Dedicatedtestenvironments
• Limitedresourcepool• Systemresources• Cost• Versioncontrol• Plaworm• Mulccustomercontracts• Scaling
![Page 32: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/32.jpg)
BenefitsProducBontesBng
• Environment• Configuracon• Livescenarios• Resultdataaccuracy
Dedicatedtestenvironments
• Systemdowncme• Dataconsumpcon• Business&customerimpact• Stressandspiketescng• Scalability
![Page 33: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/33.jpg)
AnnualProjects• Startedin2016
• Completedin2016
RowLabels CountofProjectCodeNN 8Delivery 1Development 1Infrastructure 1MigraBon 5Process 2SystemDevelopment 17SystemImplementaBon 4TransiBonandTransformaBon 1GrandTotal 40
RowLabels CountofProjectCodeNN 6Delivery 1Infrastructure 1MigraBon 6Process 2SystemDevelopment 28SystemImplementaBon 3GrandTotal 47
![Page 34: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/34.jpg)
Summary
• Unaffectedcustomers• Lessparallelchallenges&rollbacks• Buildsandsoluconpre-approved• Producconaccviceslimitedtovalidacon• LowernumberofProducconevents
![Page 35: dfgfdhsjfgdghjgh’(gk(jsrt Dedicated test environments – Money … · 2017-05-07 · Cloud Services Digital Services Operaons Global Delivery ... VPN Internet IP_Transport Customer](https://reader035.vdocuments.mx/reader035/viewer/2022070917/5fb7d007ae5b6466a4736ec3/html5/thumbnails/35.jpg)
dfgfdhsjfgdghjgh'(gk(jsrtDedicatedtestenvironments–Moneypitorqualitystamp
Thankyouforlistening
[email protected]+4795154023hWps://www.linkedin.com/in/johnhofstad/