Quality Measurement for Cloud Based E-Commerce Applications

Jay Kiruthika Dr.Gerard Horgan Dr.Souheil Khaddaj Kingston University I. T. Consultant, Kingston University

jay.kiruthika@gmail.com gmhorgan@hotmail.com s.khaddaj@kingston.ac.uk

Abstract— Cloud based e-Commerce applications are favored over traditional systems due to their capacity to reduce costs in various aspects like use of resources, low operating costs, eliminate capital costs, low maintenance and service costs. Its core functionality of optimizing performance and its automatic system recovery is crucial in web applications. Using a cloud platform for web applications increases productivity and decreases the replication of business documents saving businesses money in the current economic climate. A stable system is needed to achieve this and quality measurement is crucial to establish baselines to help predict resources for the future of the business. The proposed quality measurement model is one such designed for Cloud based e-Commerce applications. It aims to create a repository or an error-Knowledge Management System(e-KMS) for known online defects with capacity to add in future defects as they occur when using the applications. By mapping these defects directly to quality factors affected, accurate quality measurement can be achieved.

Keywords-Software Quality; Quality measurement; Software Quality Management; Software Defects; Cloud Computing.

I. INTRODUCTION

Cloud based web applications differ from business to business depending upon their quality criteria. Current businesses use a business specific model to measure quality. In this paper a generic model is proposed which can be used for all cloud based web applications. As all web applications including the cloud based ones log in user requests as it executes in web logs, it can be used to calculate the frequency of a particular error occurrence. The web logs store defects from user perspective as well as system perspective. The System Event logs, Application logs, Security logs log the defects mostly from system perspective. As the cloud based web applications are geographically distributed the weblogs can be used as fingerprint to map quality. Cloud service providers store individual weblogs for separate applications. Using these web logs an e-KMS can be developed using the known defects as a common repository and the failed requests or defects can be classified according to its criticality of impact on the business. Gerald Weinberg defines quality in Quality Software Management: Systems Thinking, as "Quality is value to some person" [1]. This definition stresses that quality is inherently subjective – industries view of software quality varies depending on their goals and management policies. A good software quality management model is driven by essential views of stake holders, project managers, business

managers, financial managers, operational managers, applications managers and the end users. Businesses tend to measure quality and align its business goals by using Key performance indicators for measuring and monitoring revenues, gross margins, return on investment, productivity, customer satisfaction etc. [4].

II. KEY QUALITY FACTORS The cloud based web applications normally rely on key quality factors like performance, availability for quality measurement. They are given priority from a cloud service provider’s point of view to user’s requirement. For example in air traffic control management system maintainability, correctness and reliability are given priority over performance as they need to be stable. The businesses tend to focus on the quality criteria and factors needed for applications which are referred to as Key Quality Factors (KQF) [6] [7] and these need not be the same for all the software it develops. This proposed generic model takes in the most of the quality factors and based on the criteria, essential views and tradeoffs can be done depending on Service Level Agreements (SLAs) [8] [9] to measure quality. The quality factors in this proposed model are defined using ISO 9126 [5] as listed below: Performance 5 – Performed well and used required resources 4 - Performed satisfactory and used required resources 3 - Performed satisfactory and used resources slightly more than acceptable 2 - Performed poorly and used vast resources 1 - Did not perform and used all resources

Maintainability 5 - Easy to maintain 4 - Mostly easy to maintain 3 - Maintainable on few certain conditions 2 - Difficult to maintain 1 - Cannot be maintained

Security 5 - Not accessible to unauthorized users 4- Accessed by unauthorized users in certain areas but no damage to system 3- Accessed by unauthorized users in certain areas and damages are short term 2 – Accessed by unauthorized users damage is long term 1 – Accessed by anyone, no restrictions

Cost/Benefit 5 - Benefits far exceed Cost 4 - Benefits exceed Cost comfortably 3 - Benefits match Cost 2 - Low Benefits, Cost Exceed 1 - Higher Cost than Benefits

2012 11th International Symposium on Distributed Computing and Applications to Business, Engineering & Science

978-0-7695-4818-0/12 $26.00 © 2012 IEEE

DOI 10.1109/DCABES.2012.62

209

Timeliness 5 - System is delivered way ahead of delivery date 4 - System is delivered ahead the delivery date 3 - System is delivered on the delivery date 2 - System is delivered late after the delivery date 1 - System is undelivered

Correctness 5 – System meets and supports all its functional objectives 4 - System meets and supports most of its functional objectives 3 - System meets and supports some and missed few 2 - System missed 90% functional objectives and only few are achieved 1 - System did not meet any functional objectives

Reliability 5 - System executes and produces same results whenever executed 4 – System executes and produces mostly the same results whenever and wherever executed 3 – System executes and produces often the same results whenever and wherever executed 2 - System executes and produces rarely the same results whenever and wherever executed 1 - System executes and produces nothing when executed

Usability 5 - Easy to use by experienced and inexperienced users 4 - Easy to use by experienced but inexperienced users need training 3 - Often easy to use by experienced with some training and inexperienced users need extensive training 2 - used by experienced and inexperienced users with extensive training 1 - Very difficult/Cannot be used by experienced and inexperienced users and can be used only by a select staff

Availability/Consistency 5 - System required/expected to function in a period of time – Very satisfactory 4 - System required/expected to function in a period of time – Expected level 3 - System required/expected to function in a period of time – Below expected level but not to a very damaging level, down time manageable 2 - System required/expected to function in a period of time – Below expected level damaging level, more downtime 1 - System required/expected to function in a period of time – Not functioning at all

Scalability 5 - Ability/Capacity of a system to perform a given task under increased load -- Very Satisfactory 4 - Ability/Capacity of a system to perform a given task under increased load – Expected level 3 - Ability/Capacity of a system to perform a given task under increased load – Below expected level, slightly damaging 2 - Ability/Capacity of a system to perform a given task under increased load – Below expected level, hugely damaging 1 - Ability/Capacity of a system to perform a given task under increased load – Ceased functioning

III. CONSTRUCTION OF E-KMS FOR HTML AND USER ERRORS

Usually when a web page responds to a request, the underlying html responds with a three digit integer result which may be informational, successful responses, re-directional responses or error page [2] depending upon the situation. The following Table 1 lists few of the web responses and possible reasons with a detailed description. The table also classifies the web responses connected to Key

Quality Factors (KQFs) and its impact which are given a value subjective 1-postive, 2-Neutral and 3-Negative depending upon the response.

Normally HTML numbered response can be classed as:

• 1xx - Informational responses - request received and continuing process

• 2xx - Successful responses - Requests successfully received, understood and accepted.

• 3xx – Re-directional responses - Further action must be taken in order to complete the request

• 4xx - Client Error responses - The request contains bad syntax or cannot be fulfilled

• 5xx – Server Error responses - The server failed to fulfill an apparently valid request

Table 1. Html Repository

Web Response/ Error Number

Reason KQFs affected directly/indirectly affected

Impact Negative –3 Positive – 1 Neutral - 2

Description of web responses/ messages

100

Continue Performance Usability Maintainability

1 The client may continue with its request. This interim response is used to inform the client that the initial part of the request has been received and has not yet been rejected by the server.

400

Bad Request

Performance Usability Maintainability

3 Usually means the syntax used in the URL is incorrect (e.g., uppercase letter should be lowercase letter; wrong punctuation marks).

401 Etc..

Unauthorized

Security Usability Interoperability Maintainability Correctness

3 Server is looking for some encryption key from the client and is not getting it. Also, wrong password may have been entered.

The defects start from HTML code 400. The various key quality factors affected by few of the error codes are also listed in the table and the reasons of the occurrence of the

210

error are also listed. This repository is used as a foundation to develop the quality model to measure the quality of the software. This repository can be tuned to specific business by including user defined errors or specific application error which might show up in the weblogs. For example database connection error can come under HTML code 500. By sifting through the logs and isolating the reasons which might be logged as part of the application a separate repository can be created to include application errors which can then be included in the quality measurement for an accurate value. Figure 1 below shows the illustration of the processes involved in creating the e-KMS. The impact/severity scale based on the business is given a subjective value according to Likert [3] scale of 1 to 5 and listed in table 2 below. To summarize the proposed e-KMS system has • Application defects with KQF scoring • User defects with KQF scoring • HTML defects with KQF scoring

Figure 1. Proposed Repository to measure software quality for cloud

based e-Commerce applications using defects Table 2. Critical Level and their values based on Likert scaling system

Impact Level/Severity on business Value Very High/Critical or Red Alert 1

High 2 Medium 3

Low 4 Very Low 5

Most of the common defects can be mapped to solutions and form a repository with added information of what can be done to deal with them which can then be handed over to trained professionals, thereby reducing the burden on the resources, man power and IT development. In an online cloud based e-business, most of the defects occurring in the software can be logged. This serves as a treasure trove for predicting uptime, downtime, meantime between failure for the business. This information is very crucial for reliability engineering. If key quality factor reliability can be measured using the logs, the rest of the key quality factors can be measured to a certain degree from the same logs provided a subjective value is given to the KQFs based on the essential view. A generic model can then be created using the logs as the repository.

IV. CALCULATING THRESHOLD USING DEFECT FREQUENCY

Defects and predicting them has always been a complex process. Linear equations [10], fuzzy logic, binomial distribution are some of the techniques used in quality models for predicting them. The "defect potentials" are the total numbers of defects found in five sources in a software lifecycle and can be found in: • Requirements • Design • Source code • User documents • Bad fixes or bugs in defect repairs

Frequency of a defect plays an important role in recalculating the critical level of a defect’s impact has on quality. It can be defined as the number of times it occurred in a specified interval of time. There is a need to normalize the frequency of defects to calculate the quality as they are needed to recalculate the KQF scoring from the original e-KMS used as a repository. For example a harmless “file not found” 400 error might be a crucial ebook file which the client has paid for it. If the frequency of the error goes beyond a satisfactory or a baselined value its priority has to be moved up as its criticality has gone up a notch. On the other hand the file not found on the client’s perspective might be an order file download which has all the details of the clients ordered including credit card and vital crucial private encrypted data. Even if the frequency of this error occurence is below the threshold its criticality is high because of the nature of the defect, it not only affects the clients business but the credibility of the software itself [3] will be questioned. These issues need to be addressed while designing the e-KMS and while prioritising the defects to be handled in the queue. As part of case study, weblogs from a cloud based e-Commerce online business whose products include ebooks, music, films, stationary etc., were examined and the frequency of html defects are extracted and stored. The weblogs which had logged few of the user defects occurring in a 12 week period were also extracted and stored in the Table 3 below.

Table 3. Frequency of HTML defects from Logs

Html error

W1 W2 W3 W4 W5… W12

400 6305 4493 3663 3124 3586 4436

401 4675 3497 3242 2672 3246 3562

402 4946 3675 3172 2634 2936 3475

403 15667 5079 6208 7445 6647 5141

404 53359 37688 43801 70576 39792 62715

405 4273 2890 2876 2587 3050 2747

N.w 8 7 32 137 200 46

St td 9 3 58 758 3 16

211

The frequency of HTML defects from weblogs are extracted for the period of 12 weeks lists error codes from 400 to 500 and includes few application error specific to the software which are the Network error, ASP files missing, HTML files missing, PDF files missing, syntax error or bad search terms which comes under user errors, syntax errors. Their defect threshold/midpoint is calculated for each of the codes using the compounded averages which are then mapped to specific quality criteria. Table 4. Re-evaluated KQF values using the defects frequency

Html error

W1 W2 W3 W4 W5… W12

400 5 4 3 2 3 4 401 5 3 3 2 3 3 402 5 4 3 2 3 3 403 RedA 2 3 4 3 2 404 4 3 3 4 3 5 405 5 3 3 3 4 3 N.w 1 1 1 3 4 1 St. td 1 1 2 RedA 1 1

The re-evaluated KQF values listed in Table 4 lists the red-alerts where the frequency has exceeded the baselines or the threshold limits set by the businesses. This threshold limit can be reduced or increased depending on the sensitivity and nature of the software. Using these values graphs can be plotted to see the quality measurement for individual KQFs. Figure 2 is one such example for KQF performance and availability over 12 weeks.

Figure 2. KQF values for Performance and Availability for 12 weeks

In the graph around week6 a new change in the existing software was introduced resulting in teething problems and few red alerts. This was resolved quickly in the week and as shown in week 7 it went down. The KQF value calculated changed due to • Changes in the software • Changes in the hardware • Part of expansion to the future technologies • Fixing Defects

New set of baselines will emerge resulting in the new values for the quality. Care should be taken during these stages to re-evaluate the essential views as users’ views or the operation manager’s views always change depending upon the platform and usability of the software. Change management is pivotal in businesses’ quality management and defects management. Since cost/benefit is involved in every step of the way, crucial care is taken when implementing them as one misstep will cost the business.

V. CONCLUSIONS AND FUTURE WORK

Software metrics for quality is never a straightforward or a done deal. As new technology emerges the metrics to measure it will also differ due to its complexity. Particularly in a cloud based e-Commerce business there are issues such as the physical location, time difference, software platform, hardware platform to be taken into account when developing such models. Due to the diversity of hardware platforms like virtualization, the system defects tend to become hard to measure unlike the standardised individualised platforms. The proposed model generalizes and automates the measurement of quality for online cloud based businesses and can be adopted easily. For example, businesses like cloud based e-travel agent system, on-line product purchase stores or an online cloud-based music store can easily incorporate this model into the existing ones to measure quality. This model can be enhanced and tuned to measure key quality factors for individual business and can be deployed for large scale systems. There is future work to be done by calculating the percentage weight of more than one key quality factors affecting a defect. By reading the pattern of defects and scoring through the system logs, quality measurement from system perspective can be calculated and added into the overall quality measurement which can also be automated.

REFERENCES [1] Gerald Weinberg, Quality Software Management: Systems Thinking,

- Dorset House, 1991 - 978-0932633224 [2] Berkun, Scott, "Issue #10: The Web Shouldn't Be a Comedy of

Errors," UIWeb.com, July 2000, www.uiweb.com/issues/issue10.htm [3] Trochim, William M. "Likert Scaling". Research Methods

Knowledge Base, 2nd Edition. October 20, 2006. Retrieved April 30, 2009.

[4] David Parmenter, Key Performance Indicators: Developing, Implementing, and Using Winning KPIs - Wiley; 2 edition, 2010 978-0470545157.

[5] ISO/IEC 9126-1:2001 Software Engineering — Product quality — Part 1: Quality model.

[6] Kiruthika J., Khaddaj S. and Horgan G, “Software Quality Factors and Defects Issues”, Proc. International Conference on Semantic e-Business and Enterprise Computing, London., 2011.

[7] Khaddaj S. and Horgan G., A proposed adaptable quality model for software quality assurance. Journal of computer sciences, 1(4):82-487-2005.

[8] Khaddaj S. and Horgan, G. The evaluation of software quality factors in very large information systems. Electronic Journal of Information Systems Evaluation, 7(1), pp. 43-48, 2004.

[9] Horgan G. and Khaddaj S., Use of an adaptable quality model approach in a production support environment. Journal of Systems and Software, 82(4), pp. 730-738, 2009.

212

[10] Xiuxiang Zhao, Ying Liu, and Shi Yong, “Predicting Software Defects using Multiple Criteria Linear Programming”, Proceedings of

the International Symposium on Intelligent Information Systems and Applications (IISA’09), pp. 583-585, 2009.

213