callq scope and user specification summary

Author and Developer: Chris Morton

1

Scope

And

User Specification

Summary

Overview..........................................................................................................................................................................................................5

User Roles.........................................................................................................................................................................................................5

Workflow..........................................................................................................................................................................................................6

Quality Assessment..........................................................................................................................................................................................7

Quality Assessment Scorecard Selection Form...........................................................................................................................................7

Call Details Capture Form............................................................................................................................................................................7

Scorecard Details Capture Form.................................................................................................................................................................8

Compliance Questions............................................................................................................................................................................8

Weighted Questions...............................................................................................................................................................................9

Quality Assessment Population..............................................................................................................................................................9

Notepad, Paraphrases and Listwords....................................................................................................................................................9

Notepad.............................................................................................................................................................................................9

Paraphrases.....................................................................................................................................................................................10

Listwords.........................................................................................................................................................................................10

Other Quality Assessment Capture Forms................................................................................................................................................10

Client Notebook...................................................................................................................................................................................10

Flag Call.................................................................................................................................................................................................11

Training And Development..................................................................................................................................................................11

Calculators And Tools...........................................................................................................................................................................11

Ratings..................................................................................................................................................................................................12

Quality Assessment Summary Form.........................................................................................................................................................12

Scorecard Builder...........................................................................................................................................................................................13

Select an Existing Scorecard or Create a New Scorecard..........................................................................................................................13

Scorecard Questions.................................................................................................................................................................................13

Scorecard Words.......................................................................................................................................................................................15

Scorecard Settings.....................................................................................................................................................................................15

Scorecard Reports.....................................................................................................................................................................................16

Scorecard Category Question Order.........................................................................................................................................................16

Attributes and Wordcloud Reports................................................................................................................................................................17

Dashboard......................................................................................................................................................................................................20

Bar Graphs/Linear Graphs.........................................................................................................................................................................21

Total Compliant Percent/Average Quality Score.................................................................................................................................21

2

Total Non-Compliant Calls/Total Compliant (%) Calls..........................................................................................................................22

Total Trifactor Distribution Percentage of Quality Scores...................................................................................................................22

Cumulative Evaluated Calls with Quality Scores <= 85% and Compliance Fails..................................................................................22

Cumulative Total Evaluated Calls with Quality Scores Below 85%......................................................................................................23

Total Failed Evaluated Calls with Quality Scores above 95%...............................................................................................................23

Total Evaluated Calls per day...............................................................................................................................................................24

Cumulative Total Evaluated Calls.........................................................................................................................................................24

Average Handling Time (Seconds)........................................................................................................................................................24

Average № Calls Per QA Agent Per Day...............................................................................................................................................25

Overall Total Cumulative № Evaluated Calls Per Dialler Agent...........................................................................................................25

Overall Total Cumulative № Evaluated Calls Compliance/Non Compliance ratio Per Dialler Agent..................................................26

Overall Total Trifactor Distribution Percentage per Dialler Agent.......................................................................................................26

Overall Total Cumulative № Evaluated Calls Per Dialler Team............................................................................................................26

Overall Total Cumulative № Evaluated Calls Per QA Agent.................................................................................................................27

Pie Graphs and Area Graphs.....................................................................................................................................................................27

Percent Certainty.................................................................................................................................................................................27

Formula Reference (Sample Certainty)...........................................................................................................................................27

Compliant/Ave Quality Scores.............................................................................................................................................................28

Quality Markdowns..............................................................................................................................................................................28

Compliance Markdowns.......................................................................................................................................................................29

Agent Ranking Data...................................................................................................................................................................................29

Top 5 Agents and Bottom 5 Agents Formula Reference.................................................................................................................29

Date Range................................................................................................................................................................................................30

Wordclouds...............................................................................................................................................................................................30

Feedback...................................................................................................................................................................................................31

Configuration Manager..................................................................................................................................................................................32

Team/Agent/Scorecard Configuration......................................................................................................................................................32

Client/Dashboard Configuration...............................................................................................................................................................33

Client User Configuration..........................................................................................................................................................................34

Manager User Configuration.....................................................................................................................................................................35

QA User Configuration..............................................................................................................................................................................36

Scorecard Configuration............................................................................................................................................................................36

Filename Decoder..........................................................................................................................................................................................37

3

Calculators......................................................................................................................................................................................................38

Stats Model Calculator – Flash..................................................................................................................................................................38

Percent to be Audited..........................................................................................................................................................................38

Sample Certainty..................................................................................................................................................................................38

Sample Certainty Comparison..............................................................................................................................................................39

Stat Model Assessment Calculator............................................................................................................................................................39

Trend Analysis Tool........................................................................................................................................................................................40

Combinator – Select Ordinal Column........................................................................................................................................................40

Combinator – Select Comparison Columns...............................................................................................................................................41

Combinator – Triplets...........................................................................................................................................................................41

Combinator – Quads............................................................................................................................................................................42

Trend Findings...........................................................................................................................................................................................42

Dynamic Import..............................................................................................................................................................................................43

Import Pre-Analysis...................................................................................................................................................................................43

4

OVERVIEW

This document serves to summarise all intended functionality related to the CallQ Infinity Quality Assurance System as commissioned by Call Quality to meet the business needs described to the developer.

The CallQ Infinity system is designed to meet the following core functionality requirements:

1. Enable the user to define and publish customised scorecards to be processed by agents in order to evaluate customer interactions

2. Collate and report on data collected in customised scorecards for the purposes of qualitative and quantitative analysis of Compliance Pass/Fail Data and numerically weighted Scorecard Questionnaire Data to aid the management to make correct and timely decisions pertaining to a specific campaign, as would be deduced from the cited reports

3. Real time and chronologic graphical/numerical dashboard display of collated data4. Facilitate a user friendly and feature rich graphical user interface to replace existing quality assurance procedures specific to

Call Qualitys’ internal business models and processes 5. Hierarchical organisation of human resources related to internal operation of Call Quality in order to create work flow

efficiencies and correct dissemination of collected data to relevant people of both client and internal data consumers

CallQ Infinity a generic and dynamic software system that is developed to replace existing quality assurance and customer interaction evaluation procedures with a standardized, accurate and consistent methodology. The purpose of CallQ is to provide high quality and reliable data that clients and executives can refer to, to support decision making of senior executives both internally within CCL and externally by our clients, relating to interactions of call centre agents and customers of the subscribed client or campaign.

USER ROLES

In order to achieve the required functionality a number of user roles have been defined on the system. User roles serve to allow correct access permissions to certain functionality within the system and to reflect the human resources hierarchy within Call Quality to achieve a structured quality assessment environment; this ensures that quality assessment procedures are adhered to thereby ensuring that data published by Call Quality is reliable and consistent. Examine the table below for descriptions of the roles defined in CallQ.

Role Description Access NotesAdministrator Native WordPress role allows

for full access to the entire frontend system, including the WordPress Dashboard

Full access, cannot capture quality assurance data that is reflected in reporting functionality

Recommended that this role is limited to only senior technical staff. Administrators must have a configured login and can only be added by the WordPress native user ‘admin’.

Client Client subscribers to the CallQ system can view data as represented on the client dashboard. Navigation is limited to the Client Dashboard only and is exclusively a data consumer role. A Client will be able to login via a VPN connection from a remote location

Assigned Client Dashboards only. Cannot generate documents using the document export functionality. No navigational options.

Clients must have a configured login.

Campaign Manager Campaign Manager Users can be configured for senior staff members that are responsible for the daily operations on the dialling floor.

Access is restricted to Dashboard Access and cannot generate documents from the CallQ document export functionality. Dashboard functionality includes access to all scorecard data that are assigned to the campaign manager.

Campaign Managers are assigned an automatically generated CallQ login.

Configuration Manager Configuration Manager Users are of key importance concerning the addition of new users, teams, definition of and visibility of new/existing scorecards, assignment of

Full Access to the Configuration Manager interface, Scorecard Builder and List Manager

Configuration Manager users should be assigned to only senior staff members who are responsible at a high level for all human resources (besides Administrators) and the

5

scorecards to teams and to specific QA Agent Users. Additionally Configuration Manager users can define which Dashboards a Client User has access to.

definition and publication of scorecards. Configuration Managers are assigned an automatically generated CallQ login.

Data Manager Data Manager Users have access to the reporting functionality and the scorecard building functionality of CallQ.

Full Access to Attribute and Wordcloud Reporting, All Dashboards (all campaigns including document exports), Trend Analysis Tool, Dynamic Import

Data Managers can access all campaign data within the system, therefore must be in a trusted position in order to ensure that data is distributed only to relevant data consumers. Data Managers can use CallQ to provide data to be further analysed by MI Analysts. Data Managers are assigned an automatically generated CallQ login.

Director Director Users are similar to Campaign Manager Users, however can use the document export features to export data from the dashboards they have access to.

Dashboard functionality includes access to all scorecard data that are assigned to the Director. Full Access to Trend Analysis Tool and Dynamic Import Access

Director Users are assigned an automatically generated CallQ login.

QA Agent The QA Agent users on the system make up the majority of users of the CallQ users. QA Agent users are assigned to a Team as defined in CallQ, and can be assigned scorecards either individually or inherit the scorecards already configured for all users in a specified team.

QA Agents have access restricted to the Quality Assessment Functionality only and are the only users who can capture data relating to the quality assessments according to the scorecards assigned to the user.

QA Agents are assigned an automatically generated CallQ login.

WORKFLOW

The workflow captured in CallQ includes the population of user defined scorecards specific to any campaign by QA Agents. This data is then collated and reported on using two main reporting tools, namely the Attributes and Wordcloud Reports and the Dashboard.

The business model of CallQ corresponds to the Call Quality quality assessment procedure, whereby QA Agents listen to calls and evaluate customer/agent interactions against a set of questions formulated to identify what interactions are most relevant when determining the interaction outcome, be it favourable or unfavourable. After a number of assessments have been conducted against a set of questions specific to the campaign, a data set is analysed to reveal key performance indicators that are most relevant to the interaction outcome. The data is represented graphically and in the form of downloadable documents that can be compiled ad hoc from one of the following data reporting features:

Dashboard Attribute and Wordcloud Reporting Trend Analysis Tool

From the data reporting, one can derive which factors, according to the set of questions specific to the campaign, which agent and customer behaviours are most influential when determining a particular outcome. Once conclusions are drawn from analysis, senior company executives can use the information to support or refute certain decisions regarding campaign performance. Such information that can support executive decisions from the conclusions drawn from CallQ include and are not limited to:

Identification of Dialler Agents who are not performing well enough Identification of Dialler Agents who are exceptionally good at their jobs Identification of Dialler Agent behaviours areas that need improvement Identification of Customer behaviours that determine a particular call outcome Identification of relationships of scorecard questions with regard to a certain interaction outcome Quality and Compliance metrics concerning the efficacy of internal training Data that can support Speculative Analysis of marketing activity efficacy Data that can support Speculative Analysis concerning consumer behaviours

6

Data that can support Speculative Analysis of future trending and current market behaviours

CallQ is programmed to allow the objective and automated analysis of data captured by the QA Agents; the scientific approach allows a report data consumer to be confident in the report findings, in order to give the management decision makers the necessary information to make informed judgements concerning aspects of day to day business operations.

QUALITY ASSESSMENT

The Quality Assessment Capture Form facilitates the process whereby QA Agents can evaluate a customer interaction against a Scorecard defined for a particular campaign. Scorecards are assigned to a particular user or team, from which the QA Agent can select and proceed to populate and submit to the database. From all the user roles in CallQ, only QA Agents can capture data that can be used to report on. Please note that these screenshots show that an administrator user is logged in, and therefore the data that is being captured is not added to the databases, since the populated form is not submitted by a QA user.

QUALITY ASSESSMENT SCORECARD SELECTION FORM

The screenshot on the right is the first step in the Quality Assessment Process. This form allows the QA Agent to select the scorecard relevant to the customer interaction that they are about to evaluate.

CALL DETAILS CAPTURE FORM

The Call Details Capture form includes fields that are used to enter the most basic data related to the customer interaction, including details such as Dialler Agent Name, Team Name and CLI. This form is a common form for all Quality Assessments, collecting information to uniquely identify the customer interaction on the database.

To ensure the highest possible quality of data capture, including cleanliness and consistency of the captured data the form includes validated fields that check the type of data that that is being captured against what is expected to be captured. For example the CLI can only accept a correctly entered phone number, which does not contain letters; similarly where name fields are entered, numbers are not allowed. Additionally CallQ ensures the correct casing of person names

is entered by automatically correcting syntax errors upon entry of erroneous data. To ensure a higher accuracy and consistency of captured data, special dropdown lists are included to allow the user to first select the correct value, and if it does not exist in the list, then add it manually.

To optionally match the unique quality assessment to a specific customer interaction, the Call Details Capture form allow the user to upload a recordings of the interaction, so that the customer interaction can be matched against the quality assessment, for the purposes of ‘dip-checking’ and evaluated call; this ensures that the internal quality procedures of Call Quality are being adhered to.

7

SCORECARD DETAILS CAPTURE FORM

The Scorecard Details Capture Form displays the specific questionnaire details according to the Scorecard created on the Scorecard Builder. This form includes a navigational ‘tree’ of all questions on the scorecard. In most cases the questions are distinctly of two kinds: Compliance Questions and Weighted Questions.

COMPLIANCE QUESTIONS

Compliance Questions represent a series of questions that must all be affirmative in order for a compliance pass to be evaluated. If a single Compliance question is not answered correctly, then the customer interaction will fail the Compliance Evaluation section of the quality assessment.

8

WEIGHTED QUESTIONS

Weighted Questions are associated with a numerical ‘points’ value. Upon selecting a correct answer for a weighted question, the weighted points are accrued in the total Quality Score of the particular assessment, which upon submitting the questionnaire to the database is saved as a total Quality Score as well as a percentage value of the total maximum Quality Score for a particular scorecard.

Weighted Questions are grouped into categories as specified in the Scorecard Builder, creating an organised and topically related question tree for orderly and scientific navigation of the scorecard question set.

QUALITY ASSESSMENT POPULATION

To populate the form the QA Agent is required to navigate the question tree and select a question, where upon the question is displayed and the agent input can be entered into the inputs displayed, as defined for that particular question. The inputs that can be displayed for each question can differ, and the following inputs can be entered:

Radio Buttons Dropdown Lists Multi-select Lists Check Boxes Single Line Text Inputs Text Area Fields Calendar Date Selection

Upon capturing information and clicking save, the answers for the question are showed in a quality assessment history window, to help the QA Agent view a summary of the quality assessment as they proceed to populate the form. After all appropriate questions have been answered by the QA Agent, the agent proceeds to submit the form to the database, and the QA Agent can continue to move on to the next quality assessment.

NOTEPAD, PARAPHRASES AND LISTWORDS

NOTEPAD

The functionality enabled by Notepad allows a QA Agent to edit free text in text area sections that will not be captured to the database, to give the QA Agent an area to edit their commentary to a suitable standard which can be accepted into the database. Such editing may include the correction of spelling and grammar or the truncation of text to make it as concise as possible.

9

PARAPHRASES

Paraphrases represent the essence of a particular phrase that could be said in a number of ways, include the verbatim expression as shown in the left most listbox of the paraphrase. For example, referring to the screenshots, the phrase ‘I enjoy playing Eurochance’ shows (2) in the right box. In this example this may mean that the QA Agent has gleaned from the customer interaction that the customer has said a paraphrase that has equivalent meaning twice during the call. This may mean the customer said ‘I love playing Eurochance’ and ‘Eurochance is fun to play’ which in this case are both paraphrases of ‘I enjoy playing Eurochance’.

The value that this adds to the quality assessment procedure is that calls can be grouped by the essence of the communication, adding an additional layer to the customer interaction reporting to be able to better identify what kind of things customers might say that relate to a particular interaction outcome. The definition of paraphrases may also include phrases said by the Dialler Agent, which may also influence the outcome.

Paraphrases are defined during the definition of the scorecard on the Scorecard Builder, as a targeted phrase collection, intentionally to pre-emptively distinguish what customers might be saying. The quantitative result can be used to analyse the frequency of certain ‘communication essence’ as related to how this affects a certain interaction outcome.

LISTWORDS

The functionality of Listwords enables the QA Agent to capture verbatim words mentioned either by the Dialler Agent or by a customer. This serves the purpose of identifying certain words specific to the clients’ product or service, or the customer interactions that are being assessed. For example by using Listwords it is possible to quantify how often the clients competitor product, brand or service is being mentioned. Specifically for example, if a client campaign is specifically trying to promote ‘Product X’ (from the client ‘Vendor A’), yet the direct competitor to this product is ‘Product Y’ (from ‘Vendor B’), one can ascertain that customers are mentioning Product Y on average of 3 time per customer interaction when the outcome of the customer interaction promoting Product X leads to a particular outcome (e.g. a non-sale). These data, once reported on, may help determine certain customer behaviours in relation to Vendor A’s marketing activities.

OTHER QUALITY ASSESSMENT CAPTURE FORMS

To ensure that the correct volume of information is captured, the Quality Assessment Form includes the following additional forms for capturing data that adds value to a particular quality assessment:

CLIENT NOTEBOOK

The Client Notebook is a free text field to provide the client with important notes that the client should be aware of as captured by the QA Agent.

10

FLAG CALL

The Flag Call Form enables a QA Agent to flag an exceptionally good or bad call, and to bring this to the attention of persons whom are configured to receive an email of this alert. This form allows for emergency interventions to be taken when appropriate, or alternatively allows a collection of useful training resources to be collected.

TRAINING AND DEVELOPMENT

To capture freetext comments particularly to do with improving Dialler Agent performance, for the purposes assisting with training and development of Dialler Agents this form enables the capture of three specific types of feedback that can be used by Dialler Agent Coaches.

CALCULATORS AND TOOLS

In certain instances it is necessary for QA Agents to calculate and convert numbers. To allow this in a single interface, without having to depend on external software the Calculators And Tools form assists QA Agents to do their job. In the current CallQ system a simple calculator, a minutes to seconds conversion tool and a seconds to minutes conversion tool has been included in this form.

RATINGS

11

The capture of certain ratings is important in terms of base metrics to be analysed by reporting. The ratings form enables a QA Agent to rate the Dialler Agent by the following metrics:

Overall Rating Speech Clarity Rating Product Knowledge Rating Customer Rapport Rating Total Talk Time Total Hold Time Number of Hold Intervals

The collection of such metrics may help identify which agents are not adequately performing according to the rating system, and may additionally reveal how certain Dialler Agent behaviours in regards to hold intervals and total hold times may or may not influence a certain campaign.

QUALITY ASSESSMENT SUMMARY FORM

Once a quality assessment is submitted, the summary of the quality assessment capturing process is displayed before re-selecting the next Scorecard Quality assessment. The Quality Assessment Summary Form gives the QA Agent some feedback, enhancing the user experience of the QA Agent, by allowing the QA Agent a chance to review and amend their answers before moving on to the next quality assessment.

If necessary to amend the capture of the previous quality assessment, the QA Agent can use the browser back button to load the previous form and recapture as required.

12

SCORECARD BUILDER

The Scorecard Builder Form is the starting point from which a quality assessment can be initiated. The Scorecard Builder includes an intuitive interface whereby a scorecard can be authored by a Configuration Manager or an Administrator.

SELECT AN EXISTING SCORECARD OR CREATE A NEW SCORECARD

The Scorecard Builder facilitates the creation of a new scorecard or the modification of an existing one. The Configuration Manager can enter a new scorecard name or select an existing scorecard from the dropdown list. The Scorecard Builder loads the Scorecard Questions Form and the Configuration Manager can proceed to define a scorecard by adding a

new question using the green plus icon, or remove a question by clicking the red cross icon.

SCORECARD QUESTIONS

The Scorecard Questions Form includes a template question construction interface. The Configuration Manager must first populate the following fields:

The Question Text field is limited to 150 characters of alphanumeric (including punctuation) text.

From Category Field the Configuration Manager can select from the list of Categories or add a new category (50 characters of alphanumeric text). To create a Compliance Question the user must either ‘Compliance’ or ‘Critical Fail’ from the dropdown. To create a Weighted Question any other category can be selected or entered.

The Question Weighting represents the points value of a Weighted Question. The user may select a value from 0-20, the sum of all the weighting from Weighted Questions is the

total points value of the Scorecard. When defining a question, the Configuration Manager must assign appropriate question weightings according to the relevance of the question in the context of the quality assessment for a particular campaign.

All questions are indexed by default, and should the Client requirements include a specific tracking of a particular KPI, the Index for Analysis checkbox should be selected.

In order to temporarily hide a from the Scorecard in the Quality Assessment Form the show question checkbox can be unchecked.

To create the answer template for the scorecard question, icons representing the various answer input types are single clicked and display in the question preview.

To accommodate a dynamic interface, the Scorecard Question Form facilitates the construction of an answer template in a WYSIWIG style way, so that the scorecard author can view and test the questions as the scorecard is built, with the Question Preview. The Question Preview is an approximation of what the QA Agent can see on the published scorecard.

13

Certain pre-defined dropdown lists exist, including a year, month, day and time selection. All other dropdown selections can be defined using the popup that displays when the list icon is clicked. From the list popup the user can select how to represent the list in the scorecard, which can be one of the following:

Radio Buttons Dropdown Lists Multi-select Lists Check Boxes

To define the values for a list a comma separated list of values can be entered. The comma separate values will be displayed as options according to the type of list selected.

When the single line text icon is selected a dialogue displays that allows the user to select a validated text type according to what is required by the question. The different options create a corresponding single line text field in the Question Preview. The validation on the text boxes ensures that only data conforming to the type of text box can be captured, helping achieve data cleanliness and correct data capture as required.

14

SCORECARD WORDS

To populate the Listwords and Paraphrases for a specific scorecard this can be done in the Scorecard Words form. This involves compiling a comma separated list of words or phrases and entering the data into the appropriate sections. Note that phrases cannot contain commas, since this would result in the phrase being spilt where the comma exists.

Listwords should represent a list of ‘targeted words’ that the QA Agent is tasked to listen for and count. The Quality Assessment Listword Counter allows the QA Agent to easily identify and count the frequency of specific words by clicking on the corresponding word in the Listword Counter.

Similarly the paraphrases should consist of a list of ‘targeted phrases’ that are synonymous to an expression that either the Dialler Agent or customer may say. The QA Agent can use the Paraphrase counter in the Quality Assessment page to count the frequency of synonymous paraphrases as the customer interaction is assessed.

SCORECARD SETTINGS

The Scorecard Settings allow a Configuration user to upload a Scorecard Logo that will be displayed on the Quality Assessment Form in order to create a graphic identity for the scorecard. Additionally a scorecard description can be edit that serves as a guide for the QA Agent. The Scorecard Description should contain instructions and information concerning the particular quality assessment related to the scorecard.

Call flag email addresses are also configured here, as a comma separated list. People who are likely to appreciate real-time email feedback of customer interactions should be listed in here.

15

SCORECARD REPORTS

From the Scorecard Reports Form the Configuration Manager can set the schedule of automatic reporting. The Scheduled Automatic Reporting works in the background to generate reports as scheduled to increase the speed of other reporting functionality.

By including a comma separated list of Report Subscriber email addresses, the subscribers are automatically sent an alert email when a scheduled report completes successfully, prompting the user to login to a specified url to view the compiled report.

SCORECARD CATEGORY QUESTION ORDER

To accommodate the required functionality for displaying the scorecard question on the Quality Assessment in the correct order, the Scorecard Category Order Form enables a Configuration Manager to order questions and categories into the order required by the QA Agents.

Questions are grouped together by categories, the categories can be ordered into place by dragging and dropping the Category Texts accordingly.

Similarly the questions belonging to each category can be dragged and dropped into the correct order. The order in which the questions

are displayed in the Question Tree is set here, and is important in terms of ensuring a logical order for the QA Agents to capture the data for each quality assessment.

16

ATTRIBUTES AND WORDCLOUD REPORTS

The Attributes and Wordcloud Reports generate a collated report of all question answer pairs for a particular scorecard. The first page of the Attributes and Wordcloud Reports includes a number of selection criteria to generate a collation according to the filter criteria selected.

The filter fields which can be selected include the following:

Scorecard – the scorecard for the collated report to compile data from Dialler Campaign – the Dialler Campaign which the QA Agents have captured on a specified Scorecard Dialler Team – the Dialler Team as captured by the QA Agents on a specified Scorecard Dialler Agent – the Dialler Agent as captured by the QA Agents on a specified Scorecard QA Team – the QA Team of QA Agents who captured against a specified Scorecard QA Agent – the QA Agent of who captured against a particular CF Outcome – a flag to filter either Compliance Fails or Compliance passes for a specified Scorecard Min Percent – a filter to collate records above a particular Quality Score Percent for a specified Scorecard Max Percent – a filter to collate records below a particular Quality Score Percent for a specified Scorecard Assessment Start Date – the earliest date from which to compile data for a specified Scorecard Assessment End Date – the latest date from which to compile data for a specified Scorecard

Should non-matching filter criteria be selected (for example if a certain QA Team does not capture against a particular Scorecard) the report will prompt the user that no data exists for the selected filter criteria.

17

The Attributes and Wordcloud reports can be accessed by Data Managers and Administrators.

By selecting the ‘Do Word Cloud’ check box, Wordclouds for questions that have free text fields will be analysed and Wordclouds generated for each qualifying question.

The Attributes and Wordcloud Report also consists of a dialogue that can be used to define a custom colours to use when colour coding the Wordcloud Report. Each word is coloured according to its frequency and the corresponding colour selected.

The next form in the Attributes and Wordcloud Reports allows a user to customize the fields to be displayed in the collated report. By default all questions are displayed and checked for a particular scorecard. Unchecking a question will not show that field in the scorecard. The user may order the column order of the collated report by drag and dropping the questions into the order required by the user.

The result of the Attributes and Wordclouds Report is a downloadable .csv file that can be opened in a spread sheet program such as Microsoft Excel.

The downloadable file is automatically named with a unique filename which can be entered into the Filename decoder to obtain information pertaining to the files creation.

The .csv file includes all the data that has been captured for a particular scorecard which meets the filter criteria as selected in the first and second pages of the Attributes and Wordcloud reports. The file can be further analysed by data analyst staff to reveal the information that is being sought.

The Wordcloud report reflects the frequency of unique words that occur in freetext fields as captured by QA Agents for questions that have freetext fields in the answer template. Depending on the number of occurrences of a

particular word, more frequent words show in the report are larger in size and are coloured in darker shades and the inverse applies for words that occur less frequently. The Wordcloud Report form includes the Wordcloud Bar that allows the user to edit the colour theme of the reported words and order them according to the number of occurrences and alphabetically. The Wordcloud bar also allows a user to select a particular wordcloud for export as PowerPoint, Word or Excel files which become available for download.

18

DASHBOARD

The Dashboard is an important feature of CallQ and is the primary graphical interface for data visualisation of the quality assessment process as captured for a specific scorecard.

The Dashboard includes several features that can be used to accurately ascertain the current performance of a quality assessment process, including certain KPI’s which are useful for determining the efficacy of certain executive level decisions.

The Dashboard is accessible to the following roles in CallQ:

Administrator – Full access to export functionality (All Scorecards) Client – No Access to export functionality (Assigned Scorecards) Campaign Manager – No access to export functionality (Assigned Scorecards) Data Manager – Full access to export functionality (All Scorecards) Director – No access to export functionality (Assigned Scorecards)

The first stage of loading the Dashboard includes selecting a scorecard from the specified list, as determined by the assignment of applicable scorecards as configured by a Configuration Manager.

The Dashboard does not conform to the rest of the standard CallQ navigation or layout by design. There is no navigation menu to guide the user to any of the other pages, and specifically for Client user, no navigation is possible beyond the Dashboard Page.

20

The Dashboard shows the synopsis of data related to a selected Scorecard quality assessment over a period of time. The initial load page displays data representing up to 50 days of collated data. The data visualization attempts to show graphically performance metrics of the selected Scorecard.

The Navigation options for the Dashboard page include the Campaign Overview (Up to 50 days of Scorecard Data), Date Range (Configurable Date Range), Wordclouds and Feedback.

BAR GRAPHS/LINEAR GRAPHS

The Bar Graphs and Linear Data represented on the dashboard are displayed in an interactive interface that must be navigated to show all data visualisations. The graphics are generated as the user navigates via the arrows or

alternatively via the menu for the section of the page. Each graph represents data that is extracted from the Collated Scorecard using a particular query to calculate the metrics.

Upon clicking the graphic in a certain region the graphic is enlarged and the corresponding tabular data is shown, as represented by the graph.

The enlarged display graph can be selected for export by clicking the office export item, adding the graphic and the data to a compilation of data from the dashboard, made available for export into PowerPoint, Word or Excel.

TOTAL COMPLIANT PERCENT/AVERAGE QUALITY SCORE

The Total Compliant Percent/Average Quality Score Linear Graph displays the calculated totals of customer interactions that are compliant and the average quality scores on a daily basis. This particular graph includes functionality that allows the user to zoom into a particular date range to see the data in a detailed display. The graph also includes the ability to scroll the date range of the data displayed, as shown below.

21

TOTAL NON-COMPLIANT CALLS/TOTAL COMPLIANT (%) CALLS

Total Non-Compliant/Total Compliant Calls displays a stacked bar graph of the percent ratio of calls that are compliant against the calls that are non-compliant on a day by day basis. Compliant calls represent all calls (customer interactions) where the QA Agents have passed all compliance questions according to the Scorecard for the particular quality assessment. Non-Compliant calls represent customer interactions in which

the QA agents have failed one or more compliance questions for the Scorecard for the quality assessment.

The value of this graph helps identify ‘bad days’ and ‘good days’ for a particular quality assessment allocation of calls. The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.

TOTAL TRIFACTOR DISTRIBUTION PERCENTAGE OF QUALITY SCORES

The Total Trifactor Distribution Percentage of Quality Scores shows a stacked bar chart representing how the distribution of quality scores varies over a period of time on a day to day basis. The Trifactors are represented in three colours:

Green represents the percentage proportion of customer interactions that have been evaluated with Quality Scores between 95 and 100% (including 95%)

Yellow represents the percentage proportion of customer interactions that have been evaluated with Quality Scores of 85 up to 95% (not including calls with 85% or 95% Quality Scores)

Red represents the percentage proportion of customer interactions that have been evaluated with Quality Scores less than 85% (including 85%)

The value of this graph is that it allows the user to see how well the customer interactions are performing in terms of overall quality over time. The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.

CUMULATIVE EVALUATED CALLS WITH QUALITY SCORES <= 85% AND COMPLIANCE FAILS

The Cumulative Evaluate Calls with Quality Scores <= 85% and Compliance Fails represent the total percentage of calls with scores below 85% and calls with Compliance fails in relation to the total number of evaluated calls over a period of time on a day to day basis, as a compound line graph. This is an important graph since spikes or dips in this graph will indicate a significant change in quality of the

customer interactions being assessed.

22

One can see that in this particular graph that the percent values plateau, which is indicating that the quality is stabilising. From this plateau the data analysts may determine a baseline proportion of what is acceptable in terms of Quality Scores and Compliance failures. The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.

CUMULATIVE TOTAL EVALUATED CALLS WITH QUALITY SCORES BELOW 85%

The Cumulative Total Evaluated Calls with Quality Scores below 85% shows a bar graph of the total number of calls that have been evaluated with Quality Scores equal to or below 85% over a period of time on a day to day basis. Sharp rises in this graph would indicate a sudden drop in quality. The number displayed in the bars of the graph is the total sum of calls that have been evaluated with a Quality Score of

below 85%. As time progresses this number will increase as more customer interactions are processed, the last number (bar) in the graph always represents the overall total number of calls with low quality scores.

The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.

TOTAL FAILED EVALUATED CALLS WITH QUALITY SCORES ABOVE 95%

The Total Failed Evaluated Calls with Quality Scores above 95% graph represents a cumulative total of calls with high quality, above or equal to 95% that were also compliance fails. These customer interactions would have failed on a Compliance markdown yet have achieved the required quality, therefore the calls that belong to this data set are of particular interest in terms of the quality assessment procedure.

The quantification and the identification of this data is important since these calls may have been erroneously failed or may indicate that the Dialler Agents that are involved in these customer interactions are not trained properly, or may not be conforming with instructions of the Client brief. Also the customer interactions of this data set may represent potential follow ups for the Dialler agents to revisit. The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.

23

TOTAL EVALUATED CALLS PER DAY

The Total Evaluated Calls Per Day Graph shows how many calls where evaluated on a given day, on a day by day basis. The value of this KPI will help identify how well the QA Team are working in relation to the prescribed target for the day. As more QA Agents are assigned to the quality assessment for a specified Scorecard the total evaluated number of customer interactions should increase, similarly the inverse

applies when QA Agents are removed from the specified Scorecard. A decrease or increase in total number of calls evaluated per day could also offer insight into how motivated a particular staff contingent assigned to a specific Scorecard are.


CUMULATIVE TOTAL EVALUATED CALLS

The Cumulative Total Evaluated Calls shows a bar graph indicating the total number of calls evaluated over a period of time, on a day by day basis. This KPI indicates the gross productivity of a quality assessment and is important in terms of determining the certainty of the results of the other KPI’s displayed. In relation to overall quality assessment targets it can help provide information about how

close the QA Agents are to achieving the number of customer interactions required by the client in order to achieve an estimate on when a certain target allocation will be reached.


AVERAGE HANDLING TIME (SECONDS)

The Average Handling Time graph shows the average time in seconds used to evaluate a call. Since customer interaction vary greatly in time it is important to display the average time spent on quality assessments. By establishing and monitoring average handling times the user can easily see on the data visualisation where insufficient or excessive handling times are

evident; this may indicate that the quality assessments are not being carried out properly or that the customer interactions are either increasing or decreasing in duration as the campaign ages. By establishing a average handling time baseline, management can allocate sufficient staff resources for a particular campaign, determine the estimated cost of a quality assessment activity and ensure that a consistent quality standard is adhered to in terms of how much time a staff member should be spending on a quality assessment of a customer interaction.

24


AVERAGE № CALLS PER QA AGENT PER DAY

The Average Number of Calls Per QA Agent Per Day graph displays a stacked bar graph of the average number of quality assessments that are being carried out for the specified scorecard, compared with the number of QA Agents that have been capturing for a particular day.

Ideally the average number of calls a day should be constant, since each individual QA Agent should be processing the same number of calls per day. In this stacked bar chart, the red indicates the total number of QA Agents that have submitted data, and the green indicates the average number of assessments per agent.


OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER DIALLER AGENT

The Overall Cumulative Number of Evaluated Calls Per Agent bar graph shows the number of customer interactions evaluated for each Dialler Agent Recorded on the system. Ideally all Dialler Agents should be equally evaluated in the beginning of a quality assurance against a particular Scorecard. As Dialler Agents according to the trifactors calculated for each agent, the quality assessment can target certain

customer interactions belonging to agents to who are underperforming or are performing exceptionally well, and over time the distribution of overall total for evaluated customer interactions will be intentionally skewed according to internal business protocol.

The labels of each Dialler Agent displayed correspond to the number of quality assessments evaluated for the agent.

OVERALL TOTAL CUMULATIVE № EVALUATED CALLS COMPLIANCE/NON COMPLIANCE RATIO PER DIALLER AGENT

25

The Overall Total Cumulative Number of Evaluated Calls Compliance/Non-Compliance Ratio Per Dialler Agent stacked bar graphs show the percentage proportion of customer interactions evaluations that passed and failed the Compliance Questions for the labelled Dialler Agent. The proportion of Failed/Passed customer interactions can be used to identify agents whom are not sufficiently trained or are not adhering to compliance dictated by the Campaign Manager or Client. Additionally it allows interventions to be taken should

a Dialler Agent consistently fail on the compliance criteria of the Scorecard, helping deliver a higher quality outcome for our clients. The red proportion represents failed evaluations; the green portion represents passed evaluations.

The labels of each Dialler Agent displayed correspond to the proportion of Failed/Passed call interactions evaluated for the agent.

OVERALL TOTAL TRIFACTOR DISTRIBUTION PERCENTAGE PER DIALLER AGENT

The Overall Total Trifactor Distribution Percentage Per Dialler Agent displays a stacked bar graph that show how each individual Dialler Agent quality scores are distributed in terms of quality score as determined by the Trifactor Criteria:

Green represents the percentage proportion of customer interactions that have been evaluated with Quality Scores between 95 and 100% (including 95%)

Yellow represents the percentage proportion of customer interactions that have been evaluated with Quality Scores of 85 up to 95% (not including calls with 85% or 95% Quality Scores)

Red represents the percentage proportion of customer interactions that have been evaluated with Quality Scores less than 85% (including 85%)

Once can deduce that the agents with higher proportions of green are producing higher quality customer interactions, and similarly Dialler Agents with higher proportions of red are not producing customer interactions that meet the minimum standards expected.

The labels of each Dialler Agent displayed correspond to the proportion of Failed/Passed call interactions evaluated for the agent.

OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER DIALLER TEAM

The Overall Total Number Evaluated Calls per Dialler Team bar graph shows the numerical distribution of calls evaluated for each Dialler Team as recorded by the QA Agents. Ideally upon beginning a quality assurance campaign, all teams should receive the same number of evaluations, and as time goes by, and customer interactions from the Dialler Teams can be identified against Quality and Compliance standards, by using the other KPI’s displayed on the Dashboard (In particular Compliant/Ave Quality Scores (Area Graph)). The internal quality assessment protocol can be

enforced and specific customer interactions from particular teams can then be targeted and the number of each Dialler Team’s evaluated customer interaction can be adjusted accordingly. The image here represents that two teams have received more attention by QA Agents than the other 4 Dialler Teams.

26

OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER QA AGENT

The Overall Cumulative Number of Evaluated Calls Per QA Agent displays a bar graph of the number of quality assessments under taken by each individual QA Agent who has been allocated onto the Scorecard. This graph clearly shows that most of the calls assessed in this scorecard have been performed by a particular QA Agent. Ideally the graph should indicate that all QA Agents are processing the same number of customer interactions.

PIE GRAPHS AND AREA GRAPHS

PERCENT CERTAINTY

Since most quality assessment campaigns can consist of several hundred thousand (or million) customer interactions it is not possible to evaluate each individual customer interaction, therefore a way of sampling the total customer interaction pool is necessary to evaluate calls. A statistically accepted model has been included in the CallQ code to calculate exactly the number of customer interactions required to be evaluated in order to achieve a prescribed percentage of certainty. Call Quality prescribed that a minimum of 95% certainty needs to be delivered in order to give our clients information that can be used accurately to support executive decisions.

The image on the left shows the current number of evaluations (sample size) that have been assessed and the percent certainty (confidence level) of the results of the rest of the dashboard. One can see that from 10000 allocated customer interactions, 571 quality assessments have been done, which is calculated as 100% confidence of the results on the dashboard. The minimum number of assessments that need to be done to achieve a 95% confidence result for 10000 interactions is 370. For more information about these calculations please refer to the Calculators section in this document.

The formulas necessary to calculate the required sample sizes to achieve the minimum 95% percent certainty are listed below:

FORMULA REFERENCE (SAMPLE CERTAINTY)

(confidence) x = Z(c/100)2r(100-r)

(sample size) n = N x/((N-1)E2 + x)

(margin of error) E = Sqrt[(N - n)x/n(N-1)]

where:

N is the population size

r is the fraction of responses

Z(c/100) is the critical value for the confidence level c

COMPLIANT/AVE QUALITY SCORES

27

The Compliant/Ave Quality Scores area graph shows the distribution of Compliance Passes (percent of total evaluations) and Average Quality Scores as distinguished by each team. This graph easily helps with the identification as whole of how teams are performing in regards to overall performance. To view the numerical values more clearly the user can click on the image, resulting in a zoomed popup window, displaying the values in human readable format, as illustrated below.

QUALITY MARKDOWNS

The Quality Markdowns pie graph shows the top 10 Quality Questions that have been marked by the QA Agent as fails for the particular question. For example if the Scorecard has 40 questions that can be marked as ‘Yes’ or ‘No’ , the top most questions marked as ‘No’ will be shown on this graph as a percentage total of the top 10 ten marked down Quality Questions. The user can click to image to show a zoom window, clearly showing the associated values represented the total ‘No’

answers for each question. The user can also use the navigation menu or arrows to scroll into each question in detail to see the break down of each questions answers as captured by the QA Agents.

From the data represented under Quality Markdowns, one can deduce which quality aspects of the Dialler Agent customer interactions need most attention, helping direct training resources to the area’s most needing improvement for the purposes of improving overall dialler campaign customer interactions.

COMPLIANCE MARKDOWNS

28

The Compliance Markdowns pie graph shows the top 10 Compliance Markdowns for the selected Scorecard. In this example one can see 4 questions displayed, and this can be attributed to the fact that only 4 Compliance Questions exist or that from the total pool of compliance questions, only 4 have been marked with ‘No’ answers, so in this case only the top 4 Compliance Questions are represented. Navigation of the specific question breakdowns can be done using the arrows or menu. Additionally the user may click on the image to show a zoomed in view of the data with human readable values.

From the data represented under Compliance Markdowns, one can deduce which compliance aspects of the Dialler Agent customer interactions need most attention, helping direct training resources to the area’s most needing improvement for the purposes of maintaining an acceptable compliance as prescribed by the Client.

AGENT RANKING DATA

The Agent Ranking Data rates agents according to the top 5 Dialler Agents and the bottom 5 Dialler Agents as captured by the QA Agents. The importance of this statistic must be considered in regards to the Overall Cumulative Number of Evaluated Calls Per Agent bar graph, since the results will alter significantly if as Dialler Agents are targeted more specifically.

This statistic ranks Dialler Agents who have at least 4 quality evaluations in the CallQ system, specific to the Scorecard. At least 10 agents who have 4 evaluations each need to exist in the system for the ranking system to display any data (minimum of 40 evaluated customer interactions). The data represented is entered into a formula to generate a ‘Quality Factor’ score, then ranked accordingly, considering tie positions if applicable. The general formula used to calculate the top 5 and bottom 5 is explained below.

TOP 5 AGENTS AND BOTTOM 5 AGENTS FORMULA REFERENCE

Dialler Agents are ranked according to their ‘Quality Factor’ as can be derived from the following formulas:

passrate = total number of compliance passes/total number evaluated customer interactions

Quality Factor = (passrate*average quality score)/(total number evaluated customer interactions-total number of compliance passes) +0.95

DATE RANGE

29

Upon loading the Dashboard shows by default the last 50 days of activity for the selected scorecard, or approximately the last two months of working days. In order to allow a further drill down of data according

the date of customer interactions evaluated, the Dashboard interface includes a date range dialog.

The dialog includes 7 buttons to easily drill down into the periods represented by those dates, however the Assessment Start Date and Assessment End Date can be selected by using the calendar that displays when the user clicks the input field, as in the illustration.

Should no data exist for a particular date range query then the user is advised to change the date range. When a certain date range is applicable the dashboard is generated with all the data represented in the default 50 day report albeit as a chronologically applicable dataset (Bar Graphs, Pie and Area Graphs, Agent Ranking Data, Wordclouds).

WORDCLOUDS

The Dashboard includes a navigation option that displays automatically generated Wordclouds that are derived from question answer templates that contain inputs for freetext capture by the QA Agents. The Wordclouds represent the most common words captured in the free text of the QA Agents assessments. (See section on Attributes and Wordcloud Reports)

30

FEEDBACK

The Dashboard includes a section for a user to communicate internally with the Call Quality staffs that are nominated to receive communication from the Dashboard. The client cannot see the name of the nominated recipient of communication, however can view the seniority of the nominated recipient (reflected as the role they perform on CallQ).

The Dashboard Feedback interface allows a data consumer to type a message, upload attachments and request a new question to be submitted for the selected Scorecard.

File attachments can include any of the following extensions, limited by a total of 5 attachments: doc,docx,xls,xlsx,ppt,pptx,jpg,

bmp,gif,swf,png,zip,rar,csv,txt,rtf,pdf,tif,tiff,mp3,wav,ogg,wma,mpeg,avi,wmv,flv,mp4,mov

The value of feedback is apparent since it allows a clear line of communication to nominated recipients of the communication, enabling an anonymous and streamlined communication route to most effectively deal with day to day running of the customer interaction campaign as required by the Client.

The Dashboard enables a user to edit and submit a new question for evaluation, specific to the selected Scorecard for which the data is shown for. The interface is exactly the same in regards to the Scorecard Builder – Scorecard Questions sections. Upon submitting the proposed new question, the data is emailed to a nominated Data Manager, to be captured on the Scorecard Builder. It is important to note that new questions are not automatically added to the scorecard, since QA Agent training might be applicable, and the question may need to be edited for spelling, correct answer template publication etc. If a new scorecard is selected, it is attached as a file to the email that is sent to the QA Agent, with a filename that resembles ‘5177A575C7E.callq’. The file is a proprietary format that can only be opened in CallQ.

31

CONFIGURATION MANAGER

The Configuration Manager is an important feature of CallQ since it allows for the management of the Users, Teams and Scorecards. The business model captured by CallQ involves a hierarchical role (user) structure, each role having access to different parts of the application thereby ensuring that the only the correctly assigned users have access to only the part of the application that they need to do their job.

The Configuration Manager consists of seven different sections as illustrated below:

TEAM/AGENT/SCORECARD CONFIGURATION

The Team/Agent/Scorecard Configuration serves the purpose of mapping applicable Scorecards to their denoted teams and users. The form allows any of the available scorecards to be assigned to any QA Team that has been configured in the system.

The form consists of a list of configured teams, as a menu button; upon clicking the Team Name the Available Scorecards and Assigned Scorecards can be seen. The Available Scorecards are listed according to the Scorecards that have been configured as visible on the Scorecard Configuration Form (also part of the Configuration Manager).

If Scorecards have already been assigned to the QA Team, the Assigned Scorecards are visible in this list. Assigned Scorecards are available to the team members of the applicable team, either to be assigned individually or to the whole QA team, by checking the radio button for each user as listed in the Team Members section of the form, specific to the team.

In order for the Configuration Manager to assign/remove Scorecards, either from the Available Scorecards list to/from the Assigned Scorecards list, or from the Assigned Scorecards list to/from and individual user, the Configuration Manager is required to double click the text in the applicable box. If the ‘All Team Members’ radio button is selected, then the Assigned Scorecard will be assigned to all team members in that team. If a team member is transferred to another team, all scorecards need to be re-assigned accordingly.

The tool bar for each Team includes an icon that allow the Configuration Manager to clear all individual and team scorecard assignments, and an icon that allows the configuration manager to deselect the currently selected team member. The team members displayed on the Team Member List will always be QA Agent users who are currently employed.

32

CLIENT/DASHBOARD CONFIGURATION

The Client/Dashboard Configuration form facilitates the creation of a new Client Entity on CallQ. The Client represents an important data consumer of the Dashboard; Client Users configured for the Client inherit the access privileges given to the Client, in regards to which dashboards are assigned to each Client. CallQ caters for a Client to have several Dashboards at their disposal, fulfilling the business requirement stipulated that a Clients customer interactions may need to be assessed using a number of different scorecards since each Client may have at least one product or customer service campaigns running concurrently.

In order to assign a dashboard to a specific Client (access is cascaded to Client Users configured for the Client), the Configuration Manager must double click the Dashboard title in the list of Available Dashboards to add it to the Assigned Dashboards list. Similarly to remove access to a Dashboard, the Configuration Manager must double click the Dashboard title in the Assigned Dashboards.

The Client/Dashboard Configuration includes the following fields to be populated:

Client Name - the name of the company Contact Name – the name of the person who is most relevant for communication from Call Quality (this is not a user on the

system necessarily) Contact Email – the email of the primary contact person Contact Phone – the primary phone number for the contact person Client Country – for internal statistical purposes Website – for internal information purposes Industry – for internal statistical purposes Color Theme – the colour theme that the clients Dashboard is themed by, usually representing company colours of the client

33

CLIENT USER CONFIGURATION

The Client User Configuration Form facilitates the creation of new Client Users for a particular Client. Client Users inherit the permissions of the Client they are configured for; additionally Client users have no access to any of the other pages in CallQ. Should a Client user try to navigate off the page by manually altering the URL, they are automatically redirected to the ‘Select Scorecard’ dialog of the Dashboard.

Once a Client User is added on to the system, the Client User can be seen on the front end, with the associated data for that user.

CallQ automatically generates a login name for client users, with the general formula ‘firstname_lastname[_number]’. A number is appended to the automatically generated username should the username already exist in the WordPress users.

The Client User Configuration includes the following fields to be populated:

Password – password the user will login with, the user can change their own password on the system. The password is securely stored in the database using native MD5 encryption.

Organisational Level – useful to determine what level of the Client Company hierarchy a specific user belongs to. Title – indicates gender, marital status or education Firstname Lastname Photo – a file of the user image that can be used to identify clients graphically Client – the Client to which a specific user belongs, and from which the user inherits access permissions to the Assigned

Dashboards for the client. Contact Number Email Address – must be a unique email address in the WordPress installation, an email is sent to this address when the user is

configured on the system, for the Client User records (login details). Notes – any notes about a particular client that may be useful when corresponding with the client or communicating via phone.

34

MANAGER USER CONFIGURATION

The Manager User Configuration Form allows the Configuration Manager to create users that perform management roles within CallQ. Each role has specific access to the application as documented in the User Roles section of this document. Management Roles that can be inherited when creating a new user include:

Campaign Manager Configuration Manager Data Manager Director

The CallQ login for Manager Users is generated automatically. For Directors the username conforms to the general format ‘firstname_lastname[_number]’, for Campaign Managers, Configuration Managers and Data Managers the username conforms to the general format ‘CALLQ####’ where # represents a 4 digit number, if less than 1000, with zero placed holders for the three left most numbers.

The following fields must be populated when a new Manager User is created:

Password - password the user will login with, the user can change their own password on the system. The password is securely stored in the database using native MD5 encryption.

Organisational Level – The level of seniority of users within the CCL/Call Quality staff compliment First Name Last Name CCL Login – the login of the user on the CCL LAN Domain (or employee code) Photo – a file of the user image that can be used to identify users graphically User Type – what type of management user is the new user Team Name - the team that the Manager User belongs to, if the Team Name is not already in dropdown list, the Team can be

added as the user is created Contact Number – the phone number that the staff member can be contacted by Email Address – must be a unique email address in the WordPress installation, an email is sent to this address when the user is

configured on the system, for the Manager User records (login details) Notes – any notes about a particular user that may be useful when corresponding with the user or communicating via phone

QA USER CONFIGURATION

35

The QA User (Agent) Configuration form allows a Configuration Manager to add a QA Agent to the WordPress installation. QA Agents are unique on the system since QA Agents are the only users that can capture valid Scorecard Data that can be reported on.

When a new QA Agent is created on the System the QA Agent is assigned to a QA Team, and inherits scorecards that have been assigned to all team members in that team if applicable. For QA Agents an automatically generated username is created for login, conforming to the format ‘CALLQ####’ where # represents a 4 digit number, if less than 1000, with zero placed holders for the three left most numbers.

The following fields must be populated when a new QA Agent user is created:

Password - password the user will login with, the user can change their own password on the system. The password is securely stored in the database using native MD5 encryption.

Organisational Level – The level of seniority of users within the CCL/Call Quality staff compliment First Name Last Name CCL Login – the login of the user on the CCL LAN Domain (or employee code) Photo – a file of the user image that can be used to identify users graphically Team Name - the team that the QA Agent user belongs to, if the Team Name is not already in dropdown list, the Team can be

added as the user is created Contact Number – the phone number that the staff member can be contacted by Email Address – must be a unique email address in the WordPress installation, an email is sent to this address when the user is

configured on the system, for the QA Agent records (login details) Notes – any notes about a particular user that may be useful when corresponding with the user or communicating via phone

SCORECARD CONFIGURATION

The Scorecard Configuration form allows the Configuration Manager to hide and show Scorecards/Dashboards configured on the system. If the Scorecards box is unchecked this indicates that the specific Scorecard is not available to any user on the CallQ installation, accept for Configuration Managers and Administrators. The feature ensures that only the most up to date and relevant scorecards can be made available for QA Agents and other users (including users whom are authorised on the Dashboard and Attribute and Wordcloud Report), verifying that the activities undertaken in CallQ are currently applicable to the Call Quality business objectives at the time.

FILENAME DECODER

36

CallQ produces a number of different types of exports for various different User Roles in the system. In order to guarantee that all files are unique in system, each file named by default with an encoded file name, uniquely generated for the file.

CallQ also recommends a human readable filename, these file names are longer and provide a description of the contents of the file, however may be impractical for many users. In order to allow a user to recover useful information from an encoded filename, the filename decoder shows information including the following:

System Recommended File Name – a longer descriptive version of the contents of the file Page – the page of CallQ on which the file was generated Link – a link to the file as stored on the server (for data redundancy) Created Time – the time the file was created User – the user who created the file Filter – if applicable, the dynamically generated filter that was applied when the file was created (applies to Attributes and

Wordcloud Reports) Number of Records – if applicable, the number of records in the file (applies to .csv and .xlsx files only)

37

CALCULATORS

STATS MODEL CALCULATOR – FLASH

The Stats Model Calculator – Flash page in CallQ displays a number of flash calculators for calculating various critical numbers relating to sample certainty, volumes and staff compliment. The calculators on this page are compiled as Flash Applets (.swf) and the browser must be updated with the Adobe Flash Plugin in order to be viewed.

PERCENT TO BE AUDITED

The Percent to be Audited Calculator calculates the percent of the total sample required to achieve the entered certainty, which is 95% by default.

The calculation allows for two inputs for total allocation size, namely sales calls and non-sales calls and calculates the actual number of required sample size of each stream of input and the percent representation of those numbers from the total allocation size in the analogue gauges on the right.

Additionally the calculator calculates the number of required QA Agents in order to process the given sample size according to the call length and the Average handling time. The number represented here is the sum of the QA Agent required for Sales calls and Non-Sales calls.

SAMPLE CERTAINTY

The Sample Certainty Calculator calculates what certainty can be achieved from a specified sample size from a total allocation size.

For more information about the mathematics of the calculations, please refer to the Formula Reference (Sample Certainty) section of this document.

Additionally the calculator calculates the number of required QA Agents in order to process the given sample size according to the call length and the Average handling time. The number represented here is the sum of the QA Agent required for Sales calls and Non-Sales calls.

38

SAMPLE CERTAINTY COMPARISON

The Sample Certainty Comparison Calculator calculates the current sample certainty of a specified number of samples, compared with required number of samples in order to achieve a 95% certainty.

The calculator also calculates the number of QA Agents required to perform assessments in relation to the campaign complexity and interaction length as Low, Medium and High buttons.

The animated dial on the right of the calculator shows the Sample Certainty as entered compared to the prescribed 95% certainty for Call Quality assessments.

STAT MODEL ASSESSMENT CALCULATOR

The Stat Model Assessment Calculator is a JavaScript calculator that does all volume and certainty calculations needed to ensure that quality standards are adhered to in terms of correct numbers of assessments being used. This calculator is the most comprehensive calculator necessary for the internal business processes at Call Quality.

In addition to the basic functionality need to calculate certainties and volumes, the revealed ‘Recommended Values’ section allows the user fine tune the calculated sample size considering margin of error, population size, confidence percent and response percent.

The calculator also allows for entry of detailed parameters related to the calculation of staff allocation required to perform assessments. The calculation includes fields for entry of the following:

Agent Performance Split QA Daily Productive Hours Average Sales Call Length Average Non-Sales Call Length QA AHT (Average Handling Time) QA Working Days (Week)

The estimate of the number of QA Agents required is also broken down into a high, medium and low performers (referring to QA Agents productivity) and how many calls should be allocated to each class of QA Agent.

For more information about the mathematics of the calculations, please refer to the Formula Reference (Sample Certainty) section of this document.

39

TREND ANALYSIS TOOL

The Trend Analysis Tool is a tool for extracting the meaning of data and representing it as human readable text that can be quoted in other written text. The Trend Analysis Tool is a powerful means of analysing data for trends. The Trend Analysis tool can be used either for ad hoc analysis of data from a Data Catalog (refer to Dynamic Import section of this document) or data collated from a specific Scorecard. The tool involves a brute force analysis algorithm that calculates all possible permutations of a specific data set (up to 5 columns at a time), and numerically evaluates the frequency of each permutation in a given data set, allowing for the extraction of the most significant permutations to be identified as influencing the result of a selected ‘ordinal column’.

The brute force calculation currently uses large amounts of memory and increases disk activity and is restricted for use to the Administrator, Director and Data Manager roles in CallQ.

COMBINATOR – SELECT ORDINAL COLUMN

The Combinator requires the user to select the ‘Ordinal Column’ that a trend needs to be calculated for. Typically the Ordinal Column should be the information the user is seeking to answer a question about. For example, if one wants to find the most significant influencing factors that relate to ‘Call Outcome’, one would select that column.

The columns displayed for selection of an ordinal column are defined by their distinctness of the data in those columns. The columns that can be selected for an ordinal column contain up to ten different values, and columns with free text are ignored.

For example, using the ‘Call Outcome’ column in this data set, this column has only 6 possible values, as can be seen in the image below.

The data can be analysed for specific outcomes by checking/unchecking the values represented

for the column. A significant performance advantage can be achieved by analysing only the necessary data.

40

COMBINATOR – SELECT COMPARISON COLUMNS

Once an ordinal column is selected the user is required to select columns to compare the ordinal values against, in order to calculate all possible permutations for the selected columns. Although all columns can be selected for analysis it is highly recommended that the fewest possible columns are selected for analysis. For the user to select the correct columns that may relate most significantly to the ordinal column selected, the user should have an in depth knowledge of the data being analysed.

The Comparison columns incudes a section for selecting ‘triplets’ and ‘quads’ by checking the appropriate checkbox, to refine the trend extracting methodology to examine 4 or 5 columns respectively.

If required the user may examine the results in detail by unchecking the ‘Do not show workings’ check box, which will result in a tabular graphical display of all permutations identified and the frequency of those permutations in the data set examined.

Upon selecting the Comparison Columns the user can use a dialog such as the one below to specify which values to analyse for trending. By default all values in each column are selected, except for blank or null values.

Significant performance gains can be made depending on how the value options are configured and the fewer values that are selected the better the performance will be.

One must keep in mind when comparing the comparison values that significant trending for values that are not checked will not show in any of the analytic results, therefore the results must

be appreciated against the original dataset mindful that other trends may exist that are not displayed.

By default, if triplets or quads are not selected, the trend analysis algorithm will only compare two columns against each value in the selected ordinal column. By calculating all permutations of all paired columns selected on the Comparison Columns form and comparing them against all selected values of the ordinal column, the frequency of each permutation is counted in the original data source and tabulated into a trend extraction.

COMBINATOR – TRIPLETS

Similar to the default pair comparison, the triplet comparison evaluates trends by calculating all permutations of the comparison columns selected (and the values selected for those columns), in sets of three, and comparing against the ordinal column. The frequency of each permutation is counted and tabulated for trend extraction. In order for triplet trending data to be displayed at least 3 columns must be selected from the Comparison Columns.

41

One must note the processor and memory overhead incurred when processing triplet trending is significantly greater than it would be for pair trending in a specific data set.

COMBINATOR – QUADS

Using the same methodology as pair and triplet trending, quad trending reveals trends involving permutations of 4 columns against the selected values of the ordinal column. Similarly it must be noted that the processor and memory overhead is significantly greater than that of the pair and triplet trend extractions. The user should be aware that analysing trending can take a long time. In order for quad trending data to be displayed at least 4 columns must be selected from the Comparison Columns.

TREND FINDINGS

The results from the Combinator show the trends identified as human readable text in plain English, an optionally as tabular data. If either or both of the ‘Do Triplets’ or ‘Do Quads’ checkboxes have been checked and the minimum respective number of columns have been selected the trending is displayed respectively, according to algorithmic extraction.

The trends are shown in the Pair, Triplet and Quad findings sections respectively. The output displays all trending information that has a minimum frequency of 2.5% of the total data set, displaying the top three most significant results in respect to the ordinal column values selected.

If no trending information is compiled by the algorithm, then where applicable the text ‘No Trends Found’ will be displayed.

The image displayed here shows the fully expanded version of the Trend Analysis tool results; the interface conveniently includes collapsible sections to facilitate a more user friendly interaction by allowing the user to only view the results they are interested in.

42

DYNAMIC IMPORT

The Dynamic Import feature of CallQ allows a Data Manager or Administrator user to upload a .csv file for ad hoc analysis and data cleansing operations.

The Dynamic Import feature works by using a special algorithm to recognise what type of data is in each column of the uploaded .csv file, assigning a ‘Data Expression’ to the column, and applying data cleansing rules specific for the type of data in each column.

Upon import a table is created in the database, with column data types corresponding to native MySQL data types. The Data Expression algorithm evaluates the data using regular expressions, differentiating between fields such as person names, dates, long text and short text. In order to standardize in disparate data (for example data that contains dates in various formats), the cleansing algorithm includes features that standardize date/times, remove trailing/leading/extra spaces, apply correct casing to person names, replace empty text with null values and correct decimal numbers that have commas to have points instead. The underlying database supports Unicode encoding for text fields (UTF-8 collations) where applicable, supporting characters that are not found in the English alphabet.

IMPORT PRE-ANALYSIS

The import pre-analysis applies a ‘top-values’ analysis to show the top 10 most frequent values in qualifying columns of the imported CSV file.

The top 10 most frequent values are shown in order of the frequency, displaying the actual count of the particular value, and the percent of the total those values represent of the total row count in the .csv file. The graphic representation is shown as a pie graph.

The result of the pre-analysis also includes summary of the import details and the corresponding table in the database. The Table Data Summary displays information as listed below:

Column Name (the original name of the column in the csv file)

Serial Name (the internal system name of the column)

System Data Type (MySQL data type) Data Expression (name, email, integer,

decimal, phone number, website, text select, short text, long text)

Unique Value Count Non-blank Value Count Blank Count Index Type (none, reference (clustered

index), text search (full text index))

43

The Data Catalog produced by the Dynamic Import can be appended to by importing another .csv file that exactly matches the format (column names, number of columns and data expression) original Data Catalog produced in the initial data import.

44

callq scope and user specification summary

Technology