six sigma (black belt) project (hakeem-ur-rehman)

61
SIX SIGMA (BLACK BELT)PROJECT Page 1 SIX SIGMA BLACK BELT PROJECT REPORT MANAGING & ENHANCING QUALITY OF IT OPERATIONS BY REDUCING THE UNPLANNED DOWNTIME HAKEEMURREHMAN 27022012 Lahore, Pakistan SUPERVISOR: DR. NADEEM SHAFIQ BUTT THIS PROJECT REPORT IS PART OF THE REQURMENT FOR THE SIX SIGMA BLACK BELT COURSE ORGANISED BY PIQC INSTITUTE OF UALITY AND SINGAPORE QUALITY INSTITUTE

Upload: khurram-shahzad

Post on 27-Oct-2014

294 views

Category:

Documents


29 download

TRANSCRIPT

Page 1: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 1

SIX SIGMA BLACK BELT

PROJECT REPORT

MANAGING & ENHANCING QUALITY OF IT OPERATIONS BY REDUCING THE UNPLANNED DOWNTIME

HAKEEM–UR–REHMAN 27–02–2012

Lahore, Pakistan

SUPERVISOR: DR. NADEEM SHAFIQ BUTT

THIS PROJECT REPORT IS PART OF THE REQURMENT FOR THE

SIX SIGMA BLACK BELT COURSE

ORGANISED BY

PIQC INSTITUTE OF UALITY

AND

SINGAPORE QUALITY INSTITUTE

Page 2: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 2

TABLE OF CONTENTS

Sr. No. Topic Page No.

1 DEFINE PHASE 3 Project Charter 4

Deployment Process Flow Diagram 5 SIPOC Diagram 5 KPI’s and KPO’s 8

2 MEASURE PHASE 9

Measurement System Analysis (MSA) 10 Line Chart (Month Wise Downtime) 12 Pie Chart (Category Wise Downtime) 13 Pareto Analysis (Category Wise Downtime) 13 Cause & Effect Diagram (Downtime) 14 Pie Chart Country Wise Downtime 14 City Wise Downtime 14 Bar Chart Shift Wise Average Downtime 15 Gender Wise Average Downtime 16 Qualification Wise Downtime 16 Experience Wise Downtime 17 Age Wise Downtime 17 Multiple Bar Chart Country & City Wise Downtime 17 Qualification & Experience Wise Downtime 18 Age & Experience Wise Downtime 18 Probability Plot 18 Current Sigma Level Calculation 19

3 ANALYSIS PHASE 20

Testing of Hypothesis 21

4 IMPROVE PHASE 33 Design of Experiment 34

5 CONTROL PHASE 42 Process Failure Mode & Effect Analysis 43

Control Charts 59

Page 3: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 3

DEFINE PHASE:

Project Charter Process Flow Diagram SIPOC Diagram

KPI’s & KPO’s

DEFINE

PHASE

Page 4: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 4

PROJECT CHARTER: PROJECT TITLE:

Managing & Enhancing Quality of It Operations By Reducing The Unplanned Downtime

BUSINESS CASE:

Last year (from February to August, 2011) due to the different issues in the IT operations and

development company faces unplanned downtime which causes the loss of revenue,

reputation and customers. By considering these issues the overall losses are approximately

Average Rs. 23981100 per month (Average Downtime per month X Average Cost (Human

Resource) per Minute X 7 Months).

By reducing above said losses by 30%, we will be able to save approximately average Rs.7194330 per month.

PROBLEM STATEMENT:

Managing quality by controlling and improving processes of identified major downtime categories which are causing 45% of total down time.

OBJECTIVE: Reduction in %age downtime up to 30% of the total downtime.

METRICS: Primary Metric = Downtime (%) = (Total Downtime / Total Production Time) * 100

Secondary Metric = % Yield

PROJECT SCOPE:

IT Operations & Development

BEFORE IMPROVEMENT MONTHLY DOWNTIME:

AFTER IMPROVEMENT MONTHLY DOWNTIME:

Schedule:

Feb Mar Apr May Jun Jul Aug

Total 8079 6961 7047 6263 6559 5405 6645

0

2000

4000

6000

8000

10000

To

tal D

ow

nti

me

Month wise Total Downtime

Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Total Downtime 8079 6961 7047 6263 6559 5405 6645 5430 4355 2122 2276

0

2000

4000

6000

8000

10000

To

tal D

ow

nti

me

Month Wise Total Downtime

ID Task Name Start Finish Duration

Oct 2011 Nov 2011 Dec 2011 Jan 2012 Feb 2012

23/10 30/10 6/11 13/11 20/11 27/11 4/12 11/12 18/12 25/12 1/1 8/1 15/1 22/1 29/1 5/2 12/2

1 10d10/28/201110/17/2011Define

2 15d11/18/201110/31/2011Measure

3 15d12/9/201111/21/2011Analyze

4 35d1/27/201212/12/2011Improve

5 15d2/17/20121/30/2012Control

Page 5: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 5

DEPLOYMENT PROCESS FLOW DIAGRAM:

Process Flow Diagram; IT Operations and Development

GN

OC

NT

Ad

min

Te

am

He

lp D

esk

De

ve

lop

me

nt

Te

am

DB

A T

ea

mD

iale

r T

ea

mO

pe

ratio

ns

Te

am

Clie

nt

Se

rvic

es

Develop Query

to Load the

Leads

Correct

formatLoad the Leads

Update Filter

Getting

LeadsStart DialingYes

Yes

Open the Bridge

No

Request to

update the filter

No

Get Required

Teams on the

Bridge

Login Agents

Give Rights to the

Agents

Request to Load

the Leads

Develop

Campaign Script

Document that contains

requirements for new

campaignRequest to Create

New Campaign

Data which

Need to be

Loaded

Deploy Script

Approval for Script

Deployment

Yes

Update ScriptNo

Record

Downtime

Analyze Current

System and

Request

Maintainance

Perform System

Maintainance

Have Login

Rights

Report any

Production

Impacting Issue

Provide Downtime

Details to

Concerned Person

Perform Database

Maintanance

Apply Filter Reboot Dialers

Analyze

Performance

Satisfied with

Performance

Inform

Client

Change Dialing

Parameters

Yes

No

Assign/Unassign

Machine

Develop Filter for

Leads Selection

Perform Leads

Scrubbing

Set Rules to

Access the

Network

Monitor Network

Traffic

Perform

Maintenance

Activity of Telco

Lines with Career

Analyse Network

Health in Case of

Issues

SIPOC DIAGRAM:

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Development Team

Operations Team

Strategy Analyst

Dialing

Filter Dialing

Criteria Email

APPLY FILTER

Operations team

requests filter update

Development team

develops the filter

Dialer team implements

the filter and checks the

leads count

Dialer team provide the

updated leads count and

filter to development team

Correct Leads Enough Leads

Available

Operations

Team Agents

Page 6: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 6

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

NT Admin

Credentials with the rights to reboot

Reboot Dialer

Dialer team analyst

stops the controller

Analyst stops the

CTIEngine and Dialogic

Restart the machine

Once machine is backup

restart the dialogic

Restart the CTIEngine

and Controller

applications

Dialer is ready for

production

Controller up

and running CTI Engine up

and running Dialogic up and

running

Dialer Team Agents

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Client Client

Services Team

Client

Requirements Client's

Approval Client

Services Approval

UPDATE SCRIPT

Client services requests a

change in script

Developer makes changes

in the script

Script is tested on test

server

Developer sends the script

to Dialer Team for

deployment in production

Dialer Team deploys the

script after getting

appropriate approvals

Correct Script

With Updated Information

Agents Client

Services Operations

Manager Developer

Page 7: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 7

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Client Services

Strategy Analyst

Previous

Dialing Data New Strategy

CHANGE DAILING

PARAMETERS

Client Services/Strategy Analyst

request change in parameters

Dialer Team analyst implements

the strategy

Dialer Team supervisor verifies

that the change has been

implemented correctly

Dialer Team inform the requestor

that change has been implemented

Target

Strategy Met Dialing with

New Parameters

Strategy

Analyst Client

Services Team

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Client Services

Client

Lead files

LEADS LOADING

Operations team

requests filter update

Development team

develops the filter

Dialer team implements

the filter and checks the

leads count

Dialer team provide the

updated leads count and

filter to development team

New Leads

Operations

Team Client

Services Dialer Team

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

NT Admins Dialer

Team

X-ten NT login

credentials Agent

application login credentials

LOGIN AGENTS

Agent starts soft phone

Agent logs into terminal

server using his

credentials

Agents starts agent

application and login

Once agent is logged

into agent application

he logs into the

campaign

Required campaign is

loaded and agent starts

dialing

Agent

Application Campaign

Loaded

Agents

Page 8: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 8

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Machine Owner

Required

approvals Required

machine information

MAINTENANCE

Dialer Team request

field support to

perform maintenance

Field support team

performs the

maintenance and

restarts the machine

Dialer Team restarts

the appropriate

applications

Machine up and

running

Dialer Team NT Admin

KPI’s and KPO’s:

KPI’s X KPO’s Y

Month X1 Downtime Y1

Gender X2

Qualification X3

Experience X4

Country X5

City X6

Shift X7

Page 9: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 9

MEASURE PHASE:

Measurement System Analysis Line Chart Pie Chart Pareto Chart Cause & Effect Diagram Bar Chart Multiple Bar Chart Probability Plot Current Sigma Level

MEASURE

PHASE

Page 10: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 10

Process performance is measured in terms of downtime per month. This time is measured from the clock and noted on an Excel sheet. Different cases of intervals were provided to operators to evaluate those intervals as “Good” or “Bad”. While conducting study, Downtime 60 minutes as good and others as bad. After that operators’ performance was put in Minitab-16 and Measurement System Analysis for Attribute data was analyzed. Results as shown below:

MEASUREMENT SYSTEM ANALYSIS: According to AIAG 10% overall error rate is acceptable in measurement system. Below report indicating that our Measurement system has acceptance up to 93.3%. Hence our Measurement system is acceptable.

standard 93.3% of the time.

The appraisals of the test items correctly matched the

100%< 50%

YesNo

93.3%

Najeeb RehmaFahad JavaidHassan Rauf

120

100

80

60

40

20

0

93.3%

thus very difficult to assess.

the study were borderline cases between Good and Bad,

-- High percentage of mixed ratings: May indicate items in

items are being passed on to the consumer (or both).

many Good items are being rejected, or too many Bad

-- High misclassification rates: May indicate that either too

or incorrect standards.

problems, such as poor operating definitions, poor training,

Low rates for all appraisers may indicate more systematic

indicate a need for additional training for those appraisers.

-- Low accuracy rates: Low rates for some appraisers may

measurement system can be improved:

Consider the following when assessing how the

Overall error rate 6.7%

Good rated Bad 10.0%

Bad rated Good 3.3%

ways)

Mixed ratings (same item rated both 13.3%

Misclassification Rates

85.0

100.095.0

Attribute Agreement Analysis for Results

Summary Report

% Accuracy by Appraiser Comments

Is the overall % accuracy acceptable?

Page 11: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 11

Item 9

Item 7

Item 2

Item 5

Item 1

3020100

Item 10

Item 8

Item 6

Item 4

Item 3

3020100

Najeeb Rehma

Fahad Javaid

Hassan Rauf

30150

Najeeb Rehma

Fahad Javaid

Hassan Rauf

30150

Najeeb Rehma

Fahad Javaid

Hassan Rauf

30150

Most Frequently Misclassified Items

Appraiser Misclassification Rates

Overall Error Rate = 6.7%

Attribute Agreement Analysis for Results

Misclassification Report

% Good rated Bad % Bad rated Good

% Good rated Bad % Bad rated Good % Rated both ways

Najeeb Rehma

Fahad Javaid

Hassan Rauf

100806040

Bad

Good

100806040

2

1

100806040

Najeeb Rehma

Fahad Javaid

Hassan Rauf

Najeeb Rehma

Fahad Javaid

Hassan Rauf

100806040

Good

Bad

% by Appraiser

% by Standard

% by Trial

% by Appraiser and Standard

Attribute Agreement Analysis for Results

Accuracy ReportAll graphs show 95% confidence intervals for accuracy rates.

Intervals that do not overlap are likely to be different.

Page 12: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 12

i

iItems

Mix of

reduce your ability to assess how well the appraisers rate that type of item.

have 50% Good items and 50% Bad items. If you have a small percentage of items of one type, you

It is good practice to have a fairly balanced mix of Good and Bad items. Your data shows that you

Rates

and Error

Accuracy

Rated both ways= 13.3%

Bad rated Good= 3.3%

Good rated Bad= 10.0%

Error= 6.7%

Accuracy= 93.3%

(10 items x 3 appraisers = 30). The accuracy and error rates are:

would have 30 (5 x 3 x 2). To calculate the error for items rated both ways, you would have 30 values

calculate the error rates for Good items, you would have 30 appraisals (5 x 3 x 2); for Bad items you

and 5 Bad, and three appraisers test each item twice for a total of 60 appraisals (10 x 3 x 2 = 60). To

The accuracy and error rates are calculated across all appraisals. Suppose you test 10 items, 5 Good

Check Status Description

Attribute Agreement Analysis for Results

Report Card

LINE CHART:

Above line chart and descriptive statistics shows last seven months trend of the total production downtime on monthly basis.

Feb Mar Apr May Jun Jul Aug

Total 8079 6961 7047 6263 6559 5405 6645

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

To

tal D

ow

nti

me

Month wise Total Downtime

Page 13: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 13

PIE CHART:

From pie chart of category wise production downtime we can clearly observe that “unknown” was the largest downtime category with the share of 25% of total downtime. “Dialer Software” and “Procedure Error” were the second largest downtime category with 12% share of each catagory. Overall, we have a clear picture of percentage of downtime associated with each root cause category through pie chart.

PARETO ANALYSIS:

Downtime 1933 568111661 5465 5379 5237 4652 2842 2066 2043

Percent 4.1 12.124.8 11.6 11.5 11.2 9.9 6.1 4.4 4.4

Cum % 87.9 100.024.8 36.5 47.9 59.1 69.0 75.0 79.4 83.8

Root Cause Category

Other

Networ

k Har

dwar

e

Networ

k

Client

Related

Telco

Facilities

Chan

ge E

rror

Dialer S

oftw

are

Proc

edur

e Er

ror

Unkn

own

50000

40000

30000

20000

10000

0

100

80

60

40

20

0

Do

wn

tim

e

Perc

en

t

Category Vs Downtime

Change Error 11%

Client Related 4%

Database 1%

Dialer Hardware 3%

Dialer Software 12%

Facilities 10%

Maintenance 1%

Network 4% Network Hardware

4% Procedure Error

12%

Service Provider 0%

Software 3%

Telco 6%

Unknown 25%

Windows server 4%

Category Wise Downtime

Page 14: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 14

After constructing the Pareto chart of downtime of causes it was observed that out of remaining 14 categories (except “Unknown”) only 4 were responsible for a downtime of 44.2%. Procedure error and Dialer software proved to be the first and second largest category of downtime which was responsible for an 11.6% and 11.5% of the total downtime respectively. Change error caused 11.2% while facilities caused around 9.9% of total downtime. From above information, main four major categories identified which are Change Error, Procedure Error, Facilities Failure and Dialer Software. These four categories alone were responsible for 44.2% of total downtime. By managing these four categories we can considerably increase the quality by reducing unplanned downtime.

CAUSE & EFFECT DIAGRAM: Below cause & effect diagram indicating that there are number of root causes which are responsible for downtime in the system. Four major causes (i.e.: Procedure Error, Facilities Error, Change Error and Dialer Software Failure) are considered to identify the root causes.

Downtime

Procedure Error

Dialer Software Failure

Facilities Failure

Change Error

Natural disaster

Procedure misunderstanding

Lack of Education

Not clearly defined

Procedure wrongly followed

Inexperienced technician

Documentation not available

Not considered important

Not upgraded

Lack of job knowledge

Not enough trained resource

Trainings not arranged

Lack of automation

Resistance to change

Resources not available to

develop customized tools

Resources are busy in other tasks

Tools are considered

unimportant

Lack of ownership

Change manager/implementer

not properly defined

Change was not properly tested

Lack of time

Campaign needs to go live

on short noticeNot good enough

testing environment

Change implemented wrongly

Change was misunderstood

Lack of communication

between stakeholders

Earthquake

Virus attack

Virus database

was not updated

Some malicious

software installed

Poor Maintenance

System upgrade is costly

Maintenance performed

on wrong time

Misunderstanding of timings

Software crash

Incompatibility with

other components

Not thoroughly tested

Unhandled exception

Too much load

Load unbalancing

QA not performed before release

New version needs to be

released immediately

Wrong parameters applied

Lack of

experience/knowledge

Lack of education/training

Parameters were not

double checked

Agents logging into

wrong machine

Proper login information

was not provided

System crash

Too much load

Incompatibility after

system upgrade

Storm

FireFlood

Hardware failure

Antivirus was

not installed

Technology failed during change

Current system does not

support upgrade

PIE CHARTS:

95%

0% 1%

4% Country Wise Downtime

Pakistan

Philippines

Senegal

United States of America

Page 15: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 15

From pie chart of Country wise downtime we can clearly observe that “Pakistan” was the largest downtime country with the share of 95% of total downtime.

From pie chart of City wise downtime we can clearly observe that “Lahore” was the largest downtime city with the share of 85% of the total downtime. BAR CHARTS:

Above bar chart indicates that average downtime occurs in the evening and morning shifts are much higher than the night shift.

Beckley 0%

Bend 1%

Charlston 0%

Dakar 1%

Karachi 10%

King of Prussia 1%

Lahore 85%

Manila 0%

Pittsburg 2%

City Wise Downtime

Evening Morning Night

Total 58.94 56.94 39.95

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

Av

era

ge D

ow

nti

me

Shift wise Average Downtime

Page 16: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 16

Above bar chart indicates that average downtime occurs due to male is more than female.

Legend:

A Levels A-Levels

BBIT Bachelors in Business and Information Technology

BCE Bachelors in Computer Engineering

BCS Bachelors in Computer Science

BEE Bachelors in Electrical Engineering

BIS Bachelors in Information Systems

BIT Bachelors in Information Technology

BT Bachelors in Telecommunication

MCS Masters in Computer Science

MIS Masters in Information Systems

MISS Masters in Information Systems Security

MT Masters in Telecommunication

Due to lesser counts in few categories’ regarding the downtime bar chart used to high light major responsible qualification levels. So, most of the downtime are due to Graduate (i.e.: BIT, BT and BCS).

Female Male

Total 39.79 50.28

0.00

10.00

20.00

30.00

40.00

50.00

60.00

Av

era

ge

Do

wn

tim

e

Gender Wise Average Downtime

ALevels

BBIT BCE BCS BEE BIS BIT BT MCS MIS MT

Sum of Downtime 4985 5145 156 6827 1704 3289 11470 7488 86 420 5389

Count of Downtime 142 132 2 144 59 52 190 117 2 1 104

0

2000

4000

6000

8000

10000

12000

14000

To

tal D

ow

nti

me

Qualification Wise Total Downtime

Page 17: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 17

Above bar chart indicates that the lesser experience persons are more responsible for downtime.

Above bar chart indicates that the people having less than 34 years age are more responsible for downtime. MULTIPLE BAR CHARTS:

2-3 4-5 6-7 8-9 10-11 12-14

Sum of Downtime 15236 16884 10967 3033 809 30

Count of Downtime 334 291 230 74 15 1

02000400060008000

1000012000140001600018000

Tota

l Do

wn

tim

e

Experience Wise Total Downtime

28-29 30-31 32-33 34-35 36-37 38-39 40-41 50-51 54-55

Sum of Downtime 8664 20699 7833 2647 3954 1702 651 457 352

Count of Downtime 198 459 120 50 50 43 10 8 7

0

5000

10000

15000

20000

25000

Tota

l Do

wn

tim

e

Age Wise Total Downtime

Karachi Lahore Manila Dakar Beckley BendCharlst

onKing ofPrussia

Pittsburg

Pakistan PhilippinesSenegal United States of America

Sum of Downtime 4668 40016 195 451 92 250 58 352 877

Count of Downtime 93 808 2 16 5 4 1 7 9

05000

1000015000200002500030000350004000045000

To

tal D

ow

nti

me

Country & City Wise Total Downtime

Page 18: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 18

Above multiple–bar chart indicates that the most of downtime occurs in Lahore, Pakistan.

Above multiple–bar chart indicates that the most of downtime occurs due to lesser experience and having Graduate (i.e.: BIT, BT, BCS) qualification.

7006005004003002001000-100-200

99.99

99

95

80

50

20

5

1

0.01

Downtime

Pe

rce

nt

Mean 49.69

StDev 62.08

N 945

AD 82.236

P-Value <0.005

Probability Plot of DowntimeNormal

Above probability plot of the overall downtime for all root cause categories indicating a non–

normal distribution because the p-values for the Anderson-Darling test are less than 0.05

0100020003000400050006000

6-7

2-3

4-5

6-7

8-9

4-5

2-3

4-5

6-7

8-9

2-3

2-3

4-5

8-9

2-3

4-5

8-9

2-3

4-5

6-7

10

-11

4-5

12

-14

4-5

4-5

A Levels BBIT BCE BCS BEE BIS BIT BT MCS MIS MT

Tota

l Do

wn

tim

e

Qualification & Experience

Qualification & Expereince Wise Total Downtime

02000400060008000

100001200014000

2-3

4-5

2-3

4-5

6-7

2-3

4-5

6-7

8-9

2-3

4-5

8-9

6-7

2-3

4-5

8-9

2-3

4-5

8-9

12

-14

10

-11

10

-11

28-29 30-31 32-33 34-35 36-37

38-39 40-41 50-51

54-55

Tota

l Do

wn

tim

e

Age & Experience

Age & Expereince Wise Total Downtime

Page 19: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 19

MEASURING THE CURRENT PROCESS SIGMA LEVEL:

Root Cause Categories

Downtime (Minutes per Year)

Root Cause Categories Downtime

(Minutes per Year)

Dialer Software 5379 Procedure Error 5465

Dialer Hardware 1431 Network Hardware 1933

Network 2043 Database 677

Windows Server 1685 Telecommunication (Telco) 2842

Facilities 4652 Software 1257

Client Related 2066 Service Provider 140

Maintenance 491 Unknown/Other 11661

Change Error 5237 TOTAL 46959

COST OF DOWNTIME:

Total Production time (Minutes) per Month = 15 hours X 60 Minutes X 26 Days = 23400

Average Down Time (Minutes) per Month = 6708

Average Cost per minute (Human Resources) = Rs. 3575

Average down time Cost / month = Rs. 23981100

SIGMA LEVEL CALCULATIONS:

LAST SEVEN MONTH STUDY:

Total Downtime (Minutes) = 46959

Total Production Time (Minutes) = 163800

Opportunity per Unit = 1

DPU (Defects / Unit) = 46959 / 163800 = 0.2867

DPO (Defects/opportunity) = 46959 / (163800 X 1) =0.2867

Yield = 1 – DPO = 1 – 0.2867 = 0.7133

DPMO = 286685 (defects/million opportunity)

Sigma Level = 2.06 (Using M.S. Excel)

CONCLUSION: From above said investigations it is concluded that most of the downtime occurs in Lahore, Pakistan. On average in evening and morning shift downtime occurs more than night shift and male are more responsible than female for this downtime. Persons’ having lesser experience and having graduate level qualification (i.e: BIT, BT, BCS) are more responsible than others.

Page 20: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 20

ANALYSIS PHASE:

Testing of Hypothesis

ANALYZE

PHASE

Page 21: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 21

500

250

0

500

250

0

500

250

0

500

250

0

Pakistan

Philippines

Senegal

United State

United State

Senegal

Philippines

Pakistan

5002500

Data in Worksheet Order

Investigate outliers (marked in red).

Distribution of Data

Compare the location and spread.

One-Way ANOVA for Downtime by Country

Diagnostic Report

Page 22: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 22

Differences among the means are not significant (p > 0.05).

> 0.50.10.050

NoYes

P = 0.076

Philippines

United State

Pakistan

Senegal

4002000-200-400

means at the 0.05 level of significance.

You cannot conclude that there are differences among the

1 Senegal

2 Pakistan

3 United State None Identified

4 Philippines

# Sample Differs from

Which means differ?

One-Way ANOVA for Downtime by Country

Summary Report

Do the means differ?

Means Comparison Chart

Comments

The result of the above test shows that County is an insignificant factor as the P–value is more than 0.05.

Page 23: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 23

The result of the above test shows that City is an insignificant factor as the P–value is more than 0.05.

Page 24: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 24

500

250

0

500

250

0

500

250

0

Evening

Morning

Night

6005004003002001000

Evening

Morning

Night

Data in Worksheet Order

Investigate outliers (marked in red).

Distribution of Data

Compare the location and spread.

One-Way ANOVA for Downtime by Shift

Diagnostic Report

Differences among the means are significant (p < 0.05).

> 0.50.10.050

NoYes

P = 0.000

Morning

Evening

Night

806040

implications.

the size of the differences to determine if they have practical

overlap indicate means that differ from each other. Consider

Chart to identify means that differ. Red intervals that do not

means at the 0.05 level of significance. Use the Comparison

You can conclude that there are differences among the

1 Night 2

2 Evening 1

3 Morning

# Sample Differs from

Which means differ?

One-Way ANOVA for Downtime by Shift

Summary Report

Do the means differ?

Means Comparison Chart

Red intervals that do not overlap differ.

Comments

Page 25: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 25

The result of the above test shows that shift is a significant factor as the P–value is less than 0.05.

891802713624535446357268179901

500

250

0

51464136312621161161

Downtime (Fe, you would have a 90% chance.

the difference. If Downtime (Ma was 22.253 greater than

Downtime (Fe, you would have a 60% chance of detecting

If the true mean of Downtime (Ma was 14.433 greater than

For alpha = 0.05 and sample sizes = 892, 53:

100%

22.253

90%

14.433

60%< 40%

Data in Worksheet OrderInvestigate outliers (marked in red).

14.433 60.0

16.495 70.0

18.907 80.0

22.253 90.0

Difference Power

sample sizes of 892 and 53?

What difference can you detect with your

sample sizes.

Power is a function of the sample sizes and the standard deviations. To detect a difference smaller than 18.907, consider increasing the

2-Sample t Test for the Mean of Downtime (Male and Female)

Diagnostic Report

Downtime (Ma Downtime (Fe

Power

What is the chance of detecting a difference?

Page 26: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 26

6005004003002001000-100

Downtime (Ma

Downtime (Fe

the mean of Downtime (Fe (p > 0.05).

The mean of Downtime (Ma is not significantly greater than

> 0.50.10.050

NoYes

P = 0.084

20151050

results of the test.

samples. Look for unusual data before interpreting the

-- Distribution of Data: Compare the location and means of

that the true difference is between -2.0710 and 23.047.

the difference from sample data. You can be 90% confident

-- CI: Quantifies the uncertainty associated with estimating

0.05 level of significance.

mean of Downtime (Ma is greater than Downtime (Fe at the

-- Test: There is not enough evidence to conclude that the

Sample size 892 53

Mean 50.280 39.792

90% CI (46.83, 53.73) (27.699, 51.886)

Standard deviation 62.571 52.573

Statistics Downtime (Ma Downtime (Fe

10.488

(-2.0710, 23.047)

Difference between means*

90% CI

* The difference is defined as Downtime (Ma - Downtime (Fe.

2-Sample t Test for the Mean of Downtime (Male and Female)

Summary Report

Distribution of Data

Compare the data and means of the samples.

Mean Test

Is Downtime (Ma greater than Downtime (Fe?

90% CI for the Difference

Does the interval include zero?

Comments

Page 27: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 27

at most a 60% chance of detecting a difference of 23.539.

least a 90% chance of detecting a difference of 159.82, and

Based on your samples and alpha level (0.05), you have at

100%

159.82

90%

23.539

60%< 40%

23.539 3.5 - 60.0

126.15 60.0 - 100.0

139.35 70.0 - 100.0

150.83 80.0 - 100.0

159.82 90.0 - 100.0

Difference Power

with your sample sizes?

What difference can you detect

Masters in_1 104 51.817 57.211 (40.691, 62.943)

A Levels 142 35.106 43.816 (27.836, 42.375)

Bachelors _2 132 38.977 41.490 (31.833, 46.121)

Bachelors _3 2 78 56.569 (-430.25, 586.25)

Bachelors _4 144 47.410 51.121 (38.989, 55.831)

Bachelors _5 58 29.034 38.518 (18.907, 39.162)

Bachelors _6 52 63.25 66.703 (44.680, 81.820)

Bachelors _7 190 60.368 75.072 (49.625, 71.112)

Bachelors _8 117 64 82.220 (48.945, 79.055)

Masters in_9 2 43 18.385 (-122.18, 208.18)

Qualification Size

Sample

Mean Deviation

Standard

95% CI for Mean

Individual

Statistics

150.83, consider increasing the sample sizes.

Power is a function of the sample sizes and the standard deviations. To detect differences smaller than

One-Way ANOVA for Downtime by Qualification

Power Report

Power

What is the chance of detecting a difference?

Differences among the means are significant (p < 0.05).

> 0.50.10.050

NoYes

P = 0.016

Bachelors _3

Masters in_9

Bachelors _6

Bachelors _8

Bachelors _7

Masters in_1

Bachelors _4

Bachelors _2

A Levels

Bachelors _5

3002001000-100

implications.

the size of the differences to determine if they have practical

overlap indicate means that differ from each other. Consider

Chart to identify means that differ. Red intervals that do not

means at the 0.05 level of significance. Use the Comparison

You can conclude that there are differences among the

10 Bachelors _3

1 Bachelors _5 6 7

2 A Levels 6 7

3 Bachelors _2

4 Bachelors _4

5 Masters in_1

6 Bachelors _7 1 2

7 Bachelors _8 1 2

8 Bachelors _6

9 Masters in_9

# Sample Differs from

Which means differ?

One-Way ANOVA for Downtime by Qualification

Summary Report

Do the means differ?

Means Comparison Chart

Red intervals that do not overlap differ.

Comments

Page 28: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 28

The result of the above test shows that qualification is a significant factor as the P–value is less than 0.05.

Page 29: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 29

500

250

0

500

250

0

500

250

0

500

250

0

28 29 30

31 32 33

35 36 38

40 50 55

55

50

40

38

36

35

33

32

31

30

29

28

5002500

Data in Worksheet Order

Investigate outliers (marked in red).

Distribution of Data

Compare the location and spread.

One-Way ANOVA for Downtime by Age

Diagnostic Report

Differences among the means are significant (p < 0.05).

> 0.50.10.050

NoYes

P = 0.000

33

50

36

55

31

32

29

35

38

28

30

40

3002001000-100

implications.

the size of the differences to determine if they have practical

overlap indicate means that differ from each other. Consider

Chart to identify means that differ. Red intervals that do not

means at the 0.05 level of significance. Use the Comparison

You can conclude that there are differences among the

10 36 1 2 3 4

11 50

12 33

1 40 7 8 10

2 30 10

3 28 10

4 38 10

5 35

6 29

7 32 1

8 31 1

9 55

# Sample Differs from

Which means differ?

One-Way ANOVA for Downtime by Age

Summary Report

Do the means differ?

Means Comparison Chart

Red intervals that do not overlap differ.

Comments

Page 30: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 30

The result of the above test shows that age is a significant factor as the P–value is less than 0.05.

Page 31: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 31

500

250

0

500

250

0

500

250

0

2 3 4

5 6 7

8 9 10

10

9

8

7

6

5

4

3

2

5002500

Data in Worksheet Order

Investigate outliers (marked in red).

Distribution of Data

Compare the location and spread.

One-Way ANOVA for Downtime by Experience

Diagnostic Report

Differences among the means are significant (p < 0.05).

> 0.50.10.050

NoYes

P = 0.000

9

5

10

4

6

3

2

7

8

16012080400

implications.

the size of the differences to determine if they have practical

overlap indicate means that differ from each other. Consider

Chart to identify means that differ. Red intervals that do not

means at the 0.05 level of significance. Use the Comparison

You can conclude that there are differences among the

1 8 9

2 7 9

3 2 9

4 3 9

5 6 9

6 4 9

7 10 9

8 5

9 9 1 2 3 4 5 6 7

# Sample Differs from

Which means differ?

One-Way ANOVA for Downtime by Experience

Summary Report

Do the means differ?

Means Comparison Chart

Red intervals that do not overlap differ.

Comments

Page 32: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 32

The result of the above test shows that experience is a significant factor as the P–value is less than 0.05.

CONCLUSIONS:

Testing of Hypothesis indicating that the few Key Input Variables (KPI’s) are became significant

like Shift, Qualification, Experience and Age. In Improve phase, further experimentation will be

done by considering following significant factors that is Shift, Qualification, and Experience.

While age is a significant factor but it’s a noise factor that’s why not considered it further.

Page 33: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 33

IMPROVE PHASE:

Design of Experiment

IMPROVE

PHASE

Page 34: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 34

300250200150100500

Median

Mean

18016014012010080

1st Q uartile 55.00

Median 135.00

3rd Q uartile 180.00

Maximum 280.00

90.34 175.29

85.76 180.00

58.88 123.36

A -Squared 0.33

P-V alue 0.483

Mean 132.81

StDev 79.71

V ariance 6353.23

Skewness 0.040952

Kurtosis -0.641979

N 16

Minimum 10.00

A nderson-Darling Normality Test

95% C onfidence Interv al for Mean

95% C onfidence Interv al for Median

95% C onfidence Interv al for StDev

95% Confidence Intervals

Summary for Downtime

Graphical summary of the response clears that the data generated (i.e.: response) against the Design of Experiments is normally distributed and hence the assumption of the normality is fulfilled.

Page 35: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 35

20-2-4-6-8

99

95

90

80

70

60

50

40

30

20

10

5

1

Standardized Effect

Pe

rce

nt

A Shift

B Q ualification

C Experience

Factor Name

Not Significant

Significant

Effect Type

BC

B

Normal Plot of the Standardized Effects(response is Downtime, Alpha = 0.05)

Page 36: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 36

A

AB

AC

ABC

C

BC

B

876543210

Te

rm

Standardized Effect

2.306

A Shift

B Q ualification

C Experience

Factor Name

Pareto Chart of the Standardized Effects(response is Downtime, Alpha = 0.05)

50250-25-50

99

90

50

10

1

Residual

Pe

rce

nt

200150100500

50

25

0

-25

-50

Fitted Value

Re

sid

ua

l

40200-20-40

6.0

4.5

3.0

1.5

0.0

Residual

Fre

qu

en

cy

Mean -4.44089E-15

StDev 24.31

N 16

16151413121110987654321

50

25

0

-25

-50

Observation Order

Re

sid

ua

l

Normal Probability Plot Versus Fits

Histogram Versus Order

Residual Plots for Downtime

Page 37: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 37

EveningNight

200

150

100

50

MatersA-Levels

142

200

150

100

50

Shift

Me

an

Qualification

Experience

Main Effects Plot for DowntimeData Means

Main effect plots of shift, qualification & experience shows that the most dominating factor is the

qualification. As it gives visible variation in the downtime required for the reducing the

unplanned downtime.

14

2

Maters

A-Levels

EveningNight

Experience

Qualification

Shift

37.5

185.0230.0

15.0

115.0

210.0165.0

105.0

Cube Plot (data means) for Downtime

Cube plot for downtime shows the optimum conditions for the desired results. If minimum

unplanned downtime is required than organization must work with the combination of Night shift,

Master Qualification, and 14 year of experience (mean’s maximum experience).

Page 38: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 38

RESPONSE OPTIMIZATION:

When optimizer is run for the goal of minimum downtime evaluation, it gives minimum downtime

which is 15 minutes.

CurHigh

Low1.0000D

Optimal

d = 1.0000

Minimum

Downtime

y = 15.0

1.0000

Desirability

Composite

2.0

14.0

A-Levels

Maters

Night

EveningQualific ExperienShift

Night Maters [14.0]

Page 39: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 39

3002001000-100

99

95

90

80

70

60

50

40

30

20

10

5

1

Downtime

Pe

rce

nt

Mean 132.8

StDev 79.71

N 16

AD 0.328

P-Value 0.483

Probability Plot of DowntimeNormal

Shift Qualification Experience

Evening

Night

Maters

A-Levels

Maters

A-Levels

14

2

14

2

14

2

14

2

20000150001000050000

95% Bonferroni Confidence Intervals for StDevs

Test Statistic 11.06

P-Value 0.136

Bartlett's Test

Test for Equal Variances for Downtime

Page 40: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 40

Downtime 1933 568111661 5465 5379 5237 4652 2842 2066 2043

Percent 4.1 12.124.8 11.6 11.5 11.2 9.9 6.1 4.4 4.4

Cum % 87.9 100.024.8 36.5 47.9 59.1 69.0 75.0 79.4 83.8

Root Cause Category

Other

Networ

k Har

dwar

e

Networ

k

Clien

t Rela

ted

Telco

Facil

ities

Chan

ge E

rror

Dialer

Softw

are

Proc

edur

e Er

ror

Unkn

own

50000

40000

30000

20000

10000

0

100

80

60

40

20

0

Do

wn

tim

e

Perc

en

t

Pareto Chart of Root Cause Catagories Before Improvement

Total Downtime 1025 1005 967 965 960 17033258 1759 1503 1198 1180 1175 1106 1089

Percent 5 5 5 5 5 917 9 8 6 6 6 6 6

Cum % 70 76 81 86 91 10017 27 35 41 47 53 59 65

Root Cause Catagories

Other

Dialer S

oftw

are

Service

Pro

vider

Proc

edur

e Er

ror

Facil

ities

Networ

k Har

dwar

e

Telco

Chan

ge E

rror

Softw

are

Windo

ws se

rver

Dialer H

ardw

are

Clien

t Related

Networ

k

Unkn

own

20000

15000

10000

5000

0

100

80

60

40

20

0

To

tal

Do

wn

tim

e

Perc

en

t

Pareto Chart of Root Cause Catagories After Improvement

Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Total Downtime 8079 6961 7047 6263 6559 5405 6645 5430 4355 2122 2276

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

To

tal D

ow

nti

me

Month Wise Total Downtime (After Improvement "Sep. to Dec.")

Page 41: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 41

SIGMA LEVEL CALCULATIONS (AFTER IMPROVEMENT):

FOUR MONTH STUDY:

Total Downtime (Minutes) = 14183

Total Production Time (Minutes) = 93600

Opportunity per Unit = 1

DPU (Defects / Unit) = 14183 / 93600 = 0.1515

DPO (Defects/opportunity) = 14183 / (93600 X 1) = 0.1515

Yield = 1 – DPO = 1 – 0.1515 = 0.8485

DPMO = 151528 (defects/million opportunity)

Sigma Level = 2.53 (Using M.S. Excel)

SIGMA LEVEL

BEFORE THE IMPROVEMENT AFTER THE IMPROVEMENT

2.06 2.53

CONCLUSIONS: With new suggested combination provided by the Design of Experiment; the overall system

performance is improved as shown in the above sigma level calculation.

Page 42: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 42

CONTROL PHASE:

Process Failure Mode & Effect Analysis Control Charts

CONTROL

PHASE

Page 43: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 43

Process Failure Mode and Effect Analysis: CHANGE ERROR

Process Function /

Requirement

Potential Failure Mode

Potential Effects of

Failure

S

Potential Causes / Mechanisms of

Failure

O

Current Process Controls

D

RP

N

Recommended Actions

Responsibility

Apply Filter Leads never get buffered

High wait time

8

Applied wrong filter

3 Filter is just overviewed to find out any kind of error

5 120 Check the filter in database and get the leads count before actually applying it

Jawad Hassan (Analyst Systems and Production) Wrong filter was

provided 3 6 144

Wrong leads gets buffered

Agents talking to wrong customers

7

Import date of leads not updated 4

No process controls available

7 196

Double check the filter with developer and with Ops manager to discuss the result of implementation and avoid requesting filter change himself

Zeeshan Jamil (Supervisor Systems and Production)

Part of filter missing while

applying

4 6 168

Filter reverted to old one

Agents getting wrong leads

6

Applied for only one day instead of until further notice

4

Usually asked from OPS team if change is only for one day or for until further notice

6 144

Always ask the requester if change is only for one day or its for until further notice

Ali Gohar (Analyst Systems and Production)

Current tasks finished

3

No process controls available

3 54

Tasks audit should be performed every month

Afriaz Cheema (Manager Systems and Production)

Leads Loading Loaded to wrong database

Agent in downtime waiting for leads

8

Wrong information provided by developer 2

Normally developers mentions in which database to

2 32

Always get a confirmation from dialer team and developer

Hammad Haasan (Database Administrator)

Page 44: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 44

load the leads

about the location of the database

Misunderstanding by the DBA 2

No process controls available

2 32

Loaded with wrong scrubbing information

Customers complaining about DNC

6

Wrong scrubbing information

provided by client services/developer 2

Client services normally mentions the scrubbing criteria

6 72

Double check with the client services before actually scrubbing the leads

Arsalan Ahmed (Database Administrator)

Update Script Incorrect script deployed

Missing some information from script

5

Miscommunication between

developer and client services

5

Developer just relies on the information provided

4 100

Communication regarding script must be documented and understanding must be developed by asking questions

Bilal Sheikh (Software Engineer)

Developer missed the script

6

No process controls available

4 120

Script must be verified after development and before UAT

Bilal Sheikh (Software Engineer)

Missing some call disposition term codes

6

Functionality not working in production

environment

4

Basic testing is performed before going live

7 168

There should be more focus to improve testing environment and employees should be encouraged to do more and more testing in test

Afaq Ali (Deputy Head of Production)

Term codes were not added

6 6 216

Page 45: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 45

environment before going live

Dialer Application Update

Applications crashing

Agents get kicked out of system

8

Unhandled Exception

4

Some level of exception handling is done by the developer

4 128

More importance needs to be given to the exception handling in programming

Abul Asim (Senior Software Engineer)

Applications not starting

Production affected

7

Parameters not adjusted properly

3

Once issues are reported the parameters are double checked to find out if anything was missed

3 63

Updated parameters should be verified by the supervisor or checked by other team member in order make sure changes are correct

Ahmed Usman (Supervisor Systems and Production)

Application not working properly

Some functionalities stop working like TPV

6

Controller and CTI Engine are not synchronized

4

Information only flows through formal emails

5 120

Both dialer development and CTI Engine development team should have a closer interaction and communication before making any changes in any component. There should be more informal communication

Abul Asim (Senior Software Engineer) Ahmed Raza (Software Engineer)

Page 46: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 46

Windows Server Update

Dialer Applications crashing

Dialer applications not starting and agents can't login

8

Wrong version upgrade

2

Normally version upgrade is done over the phone as some times clients need not to be aware of the change

6 96

Everything regarding new and old version must be documented including what is current version, what is new version and are the back out steps in case a change is failed

Afaq Ali (Deputy Head of Production)

Unnecessary upgrade to some application and

frameworks 2

Normally upgrades are not allowed

6 96

Automatic upgrade must be turned off as it affect custom build application

Rizwan Ali (System Administrator)

Change dialing parameters

Dialing stopped

agents not getting calls

8

Applied wrong parameter

2

No process controls available

6

Parameters should be double checked

Zeeshan Jamil (Supervisor Systems and Production)

Dialing with wrong parameters

Too fast dialing

3

TAR updated with high value

4 Normally when the issue arises, parameters are checked to make sure if these were updated correctly or not

6 72 Change management system must be properly implemented and every change must be properly documented and approved before actual change is implemented. Change must be verified by supervisor or another person

Asim Zafar (Director IT Operations) Shoib Sakoor (Head of Production)

TAR updated with low value

4 6 72

Dialing in wrong time zones

6 Wrong time zones

opened 5 5 150

Dialing in wrong mode

3

Incorrect dialing mode applied

2 5 30

Page 47: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 47

Process Failure Mode and Effect Analysis: DIALER SOFTWARE FAILURE

Process Function /

Requirements

Potential Failure Mode

Potential Effect of Failure

S

Potential Cause /

Mechanism of Failure

O

Current Process Controls

D

RP

N

Recommended Action(s)

Responsibility

Software Crash

Application closed itself

Agents logged out and are in downtime

8

Unhandled exception

2

Some known exceptions are handled through programming

6 96

There should be a more focus on exception handling and development team should be more concerned about this

Abul Asim (Senior Software Engineer/Program Architect)

Too many processes

running 3

No process control available

4 96

Application threads should be properly controlled and memory should be managed in a better way

Abul Asim (Senior Software Engineer/Program Architect)

Capacity issues

4

No process control available

2 64

Capacity should be continuously monitored and whenever there are new agents added to the system, the capacity must be rechecked to analyze if current resources are enough to handle this or not

Afriaz Cheema (Manager Systems and Production)

QA/Testing not performed

extensively

4

No process control available

6 192

Before deployment of application, an extensive QA/testing should be performed and specially stress testing should also be performed

Abdullah (Head of QA)

Page 48: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 48

Compatibility issues with

other components

3

No process control available 4 96

All relevant stakeholders should be involved during development process

Afriaz Cheema (Manager Systems and Production)

Parameter Configuration

Application not starting or

crashing again and

again

Agents logged out and are in downtime

8

Wrong parameters

applied

3

Implemented himself checks the parameters

6 144

Parameters should be double checked and if possible there should be tools available instead of updating anything from backend

Ahmed Usman (Supervisor Systems and Production)

Agents are not able to

perform their jobs properly

5

Syntax mistake

2

Implemented himself checks the parameters

3 30

Training should be given to the technical teams on the current tools being used

Rabia (Manager Training and Development)

Some values missed

3

Implemented himself checks the parameters

6 90

Parameters should be double check with another team member immediately after any updating

Ali Gohar (Analyst Systems and Production)

Too much load Software crashes or starts malfunctioning

Leads are unnecessarily burned

5

Stuck channels are not released so rest of the

leads are marked as

ND

5

No process control available

5 125

This needs to be taken care at programming level

Abul Asim (Senior Software Engineer/Program Architect)

No further agents are able to login

Resources utilization is too high and no further resources are available to perform further actins

3

No process control available

4 60

OPS team should always inform technical teams if there is any change in current resources allocation

Matthew Debelak (Operations Manager)

Page 49: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 49

Lead disposition is not properly updated

6

Application is responding slow to database

4

No process control available

5 120

There should a more focus performance optimization

Abul Asim (Senior Software Engineer/Program Architect)

Page 50: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 50

Process Failure Mode and Effect Analysis: FACILITIES FAILURE

Process Function /

Requirements

Potential Failure Mode

Potential Effect of Failure

S

Potential Causes / Mechanisms of

Failure

O

Current Process Controls

D

RP

N

Recommended Actions

Responsibility

System Crash System shutdown

All relevant resources are in downtime

8

Too much workload

2 Network utilization is normally monitored by GNOC

4 64 Systems should be monitored continuously to check their capacity. Stress testing should also be performed after every couple of months

Afriaz Cheema (Manager Systems and Production)

Incompatibility while updating

with new software/hardware

3 Head of production normally checks with the development team about incompatibility issues

5 120 Compatibility issues must be taken into consideration before updating any software and all stake holders should be informed about the update

Afriaz Cheema (Manager Systems and Production) Afaq Ali (Deputy Head of Production)

Unexpected error 2 No process control available

6 96 Nothing can be done

System never comes backup

All resources are in downtime

8

Hardware failure 3 Nothing can be done

6 144 Hardware’s must be continuously updated

Ehtesham Opel (Manager System Administrator)

Facility Evacuation

All systems are shutdown

All relevant resources are logged out

8

Storm 2 UPS are available in case of electricity failure

3 48 Generators should be ready with extra fuel available

John Skubis (Field Support Officer) Heavy Rains 2 3 48

Page 51: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 51

Earthquake 2 No process control available

6 96 Nothing can be done

Maintenance System not available for production

Production cannot be started

8

Maintenance started on wrong

time due to misunderstanding

3 Normally emails mention the time

5 120 Time should be properly mentioned either its Pakistan time, US time or UK time

Munam Khalid (Helpdesk Analyst)

Hardware failure during

maintenance

2 No process control available

6 96 Hardware’s must be continuously updated

Afriaz Cheema (Manager Systems and Production)

No one available to restart system

after maintenance as the person is

moved to another site for some

other maintenance

3 On contact person is normally available

5 120 An extra resource should be hired who could be contacted during off contact hours of primary person

Shoib Sakoor (Head of Production)

Virus Attack System shutdown

All user disconnected

8

Antivirus was not updated

2 Antivirus is manually updated

3 48 Antivirus must be regularly updated and security policies must be tightened without exception to anyone. WSUS server must be installed so that all users are automatically updated on regular basis

Ehtesham Opel (Manager System Administrator) Shakeel Husain (Manager Networks)

Some malicious software was

installed

2 All users are not allowed to install the software

4 64

System not recognizing the users

Some users cannot login

6

Windows domain server was

attacked due to lack of security

4 Antivirus and security policies are in place

4 96

Some applications not functioning properly

Users are not able to perform their jobs properly

5

Some malicious software/patch

was installed that affected the

software

3 4 60

Page 52: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 52

Process Failure Mode and Effect Analysis: PROCEDURE ERROR

Process Function /

Requirements

Potential Failure Mode

Potential Effect of Failure

S

Potential Causes /

Mechanisms of Failure

O

Current Process Controls

D

RP

N

Recommended Actions

Responsibility

Login Agents Agents Stuck on agent application

Can't login

8

Dialer is down and was not restarted

2

No process control available

4 64

Before production starts check all the dialers that these are restarted

Jawad Hassan (Analyst Systems and Production)

Controller is down and was not restarted

4

No process control available

9 288

Before production login a test agent to make sure that dialer applications are up and running

Jawad Hassan (Analyst Systems and Production)

CTI Engine is not properly

working 3 9 216

Dialogic is not working

properly 3 9 216

Agents kicked out of agent application immediately after they login

Can't start production

8

X-ten /dial pad is not configured properly 4

No process control available

6 192

Agents should be properly given training regarding how to configure the phone before they login

Jason Oliver(Operations Manager)

Audio codec is not

configured 3 7 168

Agents should always sit on their designated stations

Joane (Operations Supervisor)

No campaigns loaded

Can't start production

8

Not assigned to the

specific campaign

3

Specific agents are assigned to the campaign when it created

7 168

New agents must always be assigned to the campaign by requesting the required

Joane (Operations Supervisor)

Page 53: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 53

technical teams before they try to login

Logging in to wrong dialer

3

Supervisors inform the agents about their current dialer

7 168

OPS team should be informed if dialer is changed and it should be documented and OPS should acknowledge the receipt of email as well

Afriaz Cheema (Manager Systems and Production)

No dialing Long wait time

7

Campaign not setup to dial during the specific time agents try to login

4

Production time is normally communicated to Technical teams before production starts

6 168

OPS team must ensure that they always inform technical teams if there is any change in production time

Jason Oliver(Operations Manager)

Reboot Dialer Dialer never comes backup

Loss of production

8

Was shutdown instead of

reboot

3

No process control available

7 168

Shutdown option should be disabled for the user

Nadeem (System Administrator)

Hardware failure 2

No process control available

7 112 Nothing can be done

Applications Failing to start

Dialer is not ready for production

8

Dialogic was not started

with optimum settings

2

No process control available

4 64

Technical support teams should be given proper training and their knowledge should be continuously updated regarding the

Afriaz Cheema (Manager Systems and Production) Shoib Sakoor (Head of Production)

CTI Engine threads were

not stable when next application was started

4 8 256

Page 54: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 54

Applications were started

in wrong sequence

2 7 112

current processes. Checklists should be maintained and followed if a process always follows the same steps. A test agent should also be logged in before production to ensure applications are properly working

One of the applications

were not started

3 7 168

Script Deployment

Script Missing

Agents can't see the script and can't properly communicate with customers

5

Script was not deployed on one of the

terminal servers

3

Script and files are manually checked on all terminal servers

4 60

An automated application should be developed that deploys the script itself

Afaq Ali (Deputy Head of Production)

Some files were not copied

4 6 120

Wrong Script shown

Performance impacting as agents can't properly communicate with customers

4

Version number was not updated

4

Script version is checked by the person who updates the script

6 96

An automated application should be developed that updates the script version automatically

Afaq Ali (Deputy Head of Production)

Page 55: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 55

Critical identified processes which should be monitored on regular basis according to corresponding recommended actions.

Process Function /

Requirements

Potential Failure Mode

Potential Effects of

Failure

S

Potential Causes / Mechanisms of

Failure

O

Current Process Controls

D

RP

N

Recommended Action(s)

Responsibility Root

Cause Category

Login agents Agents Stuck on agent application

Can't login 8 Controller is down and was not restarted

4 No process control available

9 288

Before production login a test agent to make sure that dialer applications are up and running

Jawad Hassan (Analyst Systems and Production)

Procedure Error

Reboot dialer Applications Failing to start

Dialer is not ready for production

8

CTI Engine threads were not stable when next application was started

4 No process control available

8 256

Technical support teams should be given proper training and their knowledge should be continuously updated regarding the current processes. Checklists should be maintained and followed if a process always follows the same steps. A test agent should also be logged in before production to ensure applications are properly

Afriaz Cheema (Manager Systems and Production) Shoib Sakoor (Head of Production)

Procedure Error

Page 56: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 56

working

Update Script Incorrect script deployed

Missing some call disposition term codes

6 Term codes were not added

6

Basic testing is performed before going live

6 216

There should be more focus to improve testing environment and employees should be encouraged to do more and more testing in test environment before going live

Afaq Ali (Deputy Head of Production)

Change Error

Login agents Agents Stuck on agent application

Can't login 8 CTI Engine is not properly working

3 No process control available

9 216

Before production login a test agent to make sure that dialer applications are up and running

Jawad Hassan (Analyst Systems and Production)

Procedure Error

Apply filter wrong leads get buffered

Agents talking to wrong customers

7 Import date of leads not updated

4 no process controls available

7 196

Double check the filter with developer and with Ops manager to discuss the result of implementation and avoid requesting filter change himself

Zeeshan Jamil (Supervisor Systems and Production)

Change Error

Software Crash

Application closed itself

Agents logged out and are in downtime

8 QA/Testing not performed extensively

4 No process control available

6 192

Before deployment of application, an extensive QA/testing should be performed and

Abdullah (Head of QA)

Dialer Software

Page 57: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 57

specially stress testing should also be performed

Login agents

Agents kicked out of agent application immediately after they login

Can't start production

8 x-ten / dial pad is not configured properly

4 No process control available

6 192

Agents should be properly given training regarding how to configure the phone before they login

Jason Oliver(Operations Manager)

Procedure Error

Change dialing parameters

Dialing with wrong parameters

Dialing in wrong time zones

6 Wrong time zones opened

5

Normally when the issue arises, parameters are checked to make sure if these were updated correctly or not

5 150

Change management system must be properly implemented and every change must be properly documented and approved before actual change is implemented. Change must be verified by supervisor or another person

Asim Zafar (Director IT Operations) Shoib Sakoor (Head of Production)

Change Error

Parameter configuration

Application not starting or crashing again and again

Agents logged out and are in downtime

8 Wrong parameters applied

3

Implemented himself checks the parameters

6 144

Parameters should be double checked and if possible there should be tools available instead of updating anything from backend

Ahmed Usman (Supervisor Systems and Production)

Dialer Software

Page 58: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 58

System Crash System never comes backup

All resources are in downtime

8 Hardware failure 3 Nothing can be done

6 144

Hardware’s must be continuously updated

Ehtesham Opel (Manager System Administrator)

Facilities Failure

Too much load

Software crashes or starts malfunctioning

Leads are unnecessarily burned

5

Stuck channels are not released so rest of the leads are marked as ND

5 No process control available

5 125

This needs to be taken care at programming level

Abul Asim (Senior Software Engineer/Program Architect)

Dialer Software

System Crash System shutdown

All relevant resources are in downtime

8

Incompatibility while updating with new software/hardware

3

Head of production normally checks with the development team about incompatibility issues

5 120

Compatibility issues must be taken into consideration before updating any software and all stake holders should be informed about the update

Afriaz Cheema (Manager Systems and Production) Afaq Ali (Deputy Head of Production)

Facilities Failure

Maintenance System not available for production

Production cannot be started

8

Maintenance started on wrong time due to misunderstanding

3

Normally emails mention the time

5 120

Time should be properly mentioned either its Pakistan time, US time or UK time

Munam Khalid (Helpdesk Analyst)

Facilities Failure

Page 59: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 59

191715131197531

60

40

20

0

Subgroup

Mean __

X=30.16

UCL=57.42

LCL=2.89

191715131197531

150

100

50

0

Subgroup

Ran

ge _

R=88.5

UCL=157.2

LCL=19.7

Xbar-R Chart of Downtime

Stability Report

Is the process mean stable?

Investigate out-of-control subgroups. Look for patterns and trends.

Is the process variation stable?

Investigate out-of-control subgroups. Look for patterns and trends.

Page 60: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 60

> 5% 0%

NoYes

0.0%

on the Xbar chart.

The process mean is stable. No subgroups are out of control191715131197531

60

40

20

0

Subgroup

Mean __

X=30.16

UCL=57.42

LCL=2.89

191715131197531

150

100

50

0

Subgroup

Ran

ge _

R=88.5

UCL=157.2

LCL=19.7

Comments

Xbar-R Chart of Downtime

Summary Report

Is the process mean stable?

Evaluate the % of out-of-control subgroups.Xbar Chart

Investigate out-of-control subgroups.

R Chart

Investigate out-of-control subgroups.

CONCLUSIONS: Process Failure Mode & Effect Analysis (PFMEA), X-bar & R Charts are suitable for monitoring and controlling the system.

Page 61: Six Sigma (Black Belt) Project (Hakeem-Ur-Rehman)

SIX SIGMA (BLACK BELT)–PROJECT Page 61