dfgfdhsjfgdghjgh'(gk(jsrt

Dedicatedtestenvironments–Moneypitorqualitystamp

John-HelgeHofstadEVRY

>10,000

SWE

FIN NOR

UK

LAT

UKR

IN

#1Norway#4Sweden#1FinancialServicesintheNordics

8,600employees

RevenueofEUR*

1.37bn.CommiWedorderbacklogEUR*

2.48bn.

GlobaldeliveryUkraine–Latvia–India

EVRY–TheleadingNordicITServices&So^warevendor

Servingcustomersinpublicandprivatesector

Officesin 8 countries

MarketshareITServicesinNorway

25% LeadingverccalexpercseFinancialservices,localgovernment,publicsector,healthcare,insurance

*Operacngrevenueandbacklog2017

customers

BUL

SWE

FIN

NOR

UK

LAT

UKR

CommiWedorderbacklogMNOK

9 194 *Operacngrevenue2015

SignificantfocusonFinancialServices

RevenueMNOK*

3 576

EVRYFinancialServicesoffices/employees

**OrderbacklogperJan2017

**

IN

IN

Chandigarh

Bangalore

150 customers

12 countries

Servingmorethan GlobaldeliverycapabilitydedicatedworkingexclusivelyforFinancialServicesclients

#1 intheNordics

LargestcommercialITworkforceservingfinancialservicesintheNordics,with

1 299

Customerbase

SOURCE:GARTNER(NOVEMBER2014BY2013REVENUETIER

Amongthetop25BusinessserviceprovidersinbankingacrossWesternEurope

Pan-Europeancustomerbase

CorporateStrategy

FOCUSONVERTICALS

Drivebusinesstowardsvalueaddingindustry-specificsolucons

CreacngDigitalAdvantagetoourcustomers

LEVERAGINGSTRATEGICPARTNERS

SERVICEEXCELLENCE

CloudServices

PROXIMITYTOCUSTOMERS

Investinclosepartnerships

OperaconsDigitalServices GlobalDeliveryCloudServices

Moneypitorqualitystamp?

•  TescnginLIVEProduccon •  TescngindedicatedtestEnvironments

•  ORG•  Admin•  Environments•  Problemscllinger•  Hvaløservi•  Hvormangeparalelleprosjekterogoppdrag

OrganisaconFinancialServices

WiljarNesse

OperaBonsJonBerntsen

SourcingSisselS.Warhuus

Architecture,Security&SoluBons

RunarRobertsen

SalesPerBjarneValstad

FSFinancialSuiteOperaBons

TrygveLauritzen

DNBOperaBonsPer-OlavNæss

Quality&ChangeManagement

CamillaB.BjerckeOperaBonsCentre

KalleKellgren

DatabaseKjellRuneJørgensen

BOServerTomThorstensen

AOCoreOleChriscanSjøberg

Dev.mentServicesPeWerStørseth

PreProducconoperaconArveLoraas

NonFunc.TescngJohnHelgeHofstad

AOPaymentsTrondGodø

AOChannelsJohnHaraldBrobak

AOCardsOleJacobSelberg

Testorg

TestCentreOfExcellence

TCOE

TOOLS

NFT

FTEnvironments

Automacon

Testmanagement

CCM

CCM–Valuechain

DevOps

SecureDevelopment

• IntegratedDevelopmentEnvironment

• SourceCodeControl

CodingComplete

• CodeQualityCheck• FlawDeteccon• AlignmentwithStandards

CodeReviewComplete • CommitandMerge

SourceCode• AutomaccBuilds• AutomatedTescng

BuildFinished

• StaccCodeAnalysis• AuditTrailControl

SecurityTescngSourcecodeComplete • DeploymentbyTest

EnvironmentAdministrator

• Globalvs.Restricted

ReadyforSystemTest

• Securitytescng(OWASP)

• Performancetescng• FuncconalTescng

ReadyforProduccon• DeploymentbyProducconEnvironmentAdministrator

Finished

Configuraconmanagement

DeploymentwithSalt

ADCM

Newinfrastructuretohandlefiles

Uniformmethodsofdeployment

Admin–Configuraconmanagement

Sharedinstallaconscriptsforallenvironments

Managedconfiguraconfor

easycomparisonofenvironments

Configuraconandinstallaconscriptsin

versioncontrol

Environmentadmin-ChallengesScripBng•  RunCommands

OrchestraBon•  Setupserver•  Createenvironment

DataMasking

Environments

Production environment

Staging area

Testdata masking factory

FS GlobalFS Restricted

GLOBALMasked environment

Only masked or fictive testdata

RESTRICTEDUnmasked environment

Production like or semi-masked testdata

FS CTS

Customer test network

FS Global

FS Restricted

Global source resources Onsite resources Customers

FS CTS

USER

S

Developmentview

Chan

ge fr

eque

ncy

Developer

Developer

Source code management

Automated building

Code Releases

Release Quality

Automateddeployment

Masked

Production ready

Offshore and onshore• Development• System testing

Onshore• Acceptance testing• Reference

Customer• Acceptance testing• Customer testing

Production

Application environmentsDevelopment services

Masked and Unmasked

Unmasked

Releasedepot

Releases

Systemadministrator

FINODS

Card Services

Issuing, Acquiring and Security

Internetbank and “non card clients”

WebServices–load-balancers/MQ

EDB ESB WS_PROXY Card Portal / Clients

Bank Services (non-Card)

Batch, Analysis, Security, Online

Disk SAN – dedicated SAN's to critical systems

Database servers – serving area C and E

http-servers, MQ, filetransfer, SQLproxy, Internet Proxy

Loadbalancers

= Security areas

Area A

Area B

Area C

Area D

Area E

Area F

Area G

Database servers – Cards

Environments-Architecture

BF_PreProd

Internet VPN

IP_Transport

Customer Customer Desktop cli

App

BigIP

bankportal.preprod-restricted.evry.com esb-office.preprod-restricted.evry.com customerportal.preprod-restricted.evry.com (esb-internet.preprodrestricted.evry.com)

Zone: DMZ_Bank Zone: DMZ_Internet bankfilex bankmq banksql

Zone: Bankkontor

Zone: Kjerne Zone: Kanal Zone: MSCloud

Zone: Mainframe

Zone: Infrastruktur

Zone: Tools

Desktop cli

WEB WEB PLP EMQ OCM

CPU3/ZMQ

fs.preprod.no

preprod.local

preprod.local

BigIP

APP APP

IMQ NMQ

ESB

AD

AD

AD

SP CRM

LDAP

Salt

Cli Srv/ app

=> FinODS

DB DB

APP APP

SAN – storage area network Area A

Area B

Area C

Area D

Area E

Area F

Area G Load Balancing Load Balancing

=> Web applications

=> Web services

(other)

App

=> MQ services

Load Balancing

HAProxy HAProxy Load Balancing

=> File extentions

=> DB services

Environmentorders

•  Orders•  Defects•  KPI’s

EnvironmentKPIMetric Explanation Value

Audit Deviation External

ISAE 3402 audits on Test Management Process. Measure in number of deviations last 12 months.

0 >0

Error distribution A-Test vs S-Test

Error distribution A-Test/Pilot vs System Test. Mean value from last 12 months. Large variance from one quarter to another might be experienced.

<10% 10-20% >20%

Delivery precision Test Environments

Percentage of orders registered in the project Test environment in ALM set to “Retest” before the delivery date. Mean value for last quarter.

>90% 80-90% <80%

Test Environment Quality Indicator

Ratio of proven Test environment Defects to the total number of proven defects from System test last quarter.

<5% 5-15% >15%

Nocficacons

ChallengeProducBontesBng

•  Priorices•  Servicewindows•  Consumeddata•  Stressandscalability

Dedicatedtestenvironments

•  Limitedresourcepool•  Systemresources•  Cost•  Versioncontrol•  Plaworm•  Mulccustomercontracts•  Scaling

BenefitsProducBontesBng

•  Environment•  Configuracon•  Livescenarios•  Resultdataaccuracy

Dedicatedtestenvironments

•  Systemdowncme•  Dataconsumpcon•  Business&customerimpact•  Stressandspiketescng•  Scalability

AnnualProjects•  Startedin2016

•  Completedin2016

RowLabels CountofProjectCodeNN 8Delivery 1Development 1Infrastructure 1MigraBon 5Process 2SystemDevelopment 17SystemImplementaBon 4TransiBonandTransformaBon 1GrandTotal 40

RowLabels CountofProjectCodeNN 6Delivery 1Infrastructure 1MigraBon 6Process 2SystemDevelopment 28SystemImplementaBon 3GrandTotal 47

Summary

•  Unaffectedcustomers•  Lessparallelchallenges&rollbacks•  Buildsandsoluconpre-approved•  Producconaccviceslimitedtovalidacon•  LowernumberofProducconevents

dfgfdhsjfgdghjgh'(gk(jsrtDedicatedtestenvironments–Moneypitorqualitystamp

Thankyouforlistening

John-HelgeHofstadJohn.helge.hofstad@evry.com+4795154023hWps://www.linkedin.com/in/johnhofstad/