a embedded software component quality framework fernando ferreira de carvalho advisor: silvio romero...

A Embedded software component quality

framework

Fernando Ferreira de CarvalhoAdvisor: Silvio Romero de Lemos Meira

Informatics Center - Federal University of PernambucoC.E.S.A.R. - Recife Center for Advanced Studies and Systems

[email protected]

12-Junho-2008

http://www.rise.com.brFernando Carvalho, PhD Candidate

A Embedded software component quality framework

Introduction / Motivation

Embedded system is at the heart of many systemsSo, embedded system industry needs, • Low production cost• Short time to market• High Quality

to be more efficient and competitive (Brown, 2000)

The CBD with reuse technique had been a nice direction to reach this objectives…

But, Component reuse without quality assurance give catastrophic results [ariane].



SAAP 2008.1


The CBD technologies do not take into account the specifics needs of embedded-systems development: Timing, Memory, power, hardware constraints and others.



SAAP 2008.1

Introduction /

• Solutiono Certification is the future of software

components [Wallnau, Heineman, Councill, Shaw]

o According to Weber et al. (Weber et al., 2002), the need for quality assurance in software development has exponentially increased in the past few years



SAAP 2008.1


• Benefitso More reliabilityo Short time-to-market;o More product quality;o higher quality levels, o reduced maintenance time, o investment return, o reduced time-to-market, among otherso Wide used, where certification is mandatory (aircraft)



SAAP 2008.1

Rise Framework



SAAP 2008.1

Component Certification Process



SAAP 2008.1

Component Certification Processfocused in embedded systems

An Embedded Computer System: A computer system that is part of a larger system and performs some of the requirements of that system; for example, a computer system used in an aircraft or rapid transit system. (IEEE,1992).



SAAP 2008.1


Embedded systems :• Used for specifics propose• Used to control systems (ex: mechanical machines)• Ultra small devices with simple specific functionality• Small systems with sophisticated functions • Produced in large scale



SAAP 2008.1


Embedded systems has a specific requirements:- real-time- hi reliability - low power consumption- low data and code memory- low resources - low CPU capabilities- others



SAAP 2008.1

Problem Formulation

The CBD is an efficient and effective way for design of simple and complex embedded systems.

However, quality assurance of components is must be done to take advantage of CBD.

The Software industry still far to reach the maturity level the hardware industry which it have catalogues and datasheets available for its components.

For this reason, a well-defined and consistent embedded software component quality assurance is essential for CBD and reuse adoption.



SAAP 2008.1

Proposed solution

An Embedded Software Component Quality Verification Framework

It is composed of four inter-relation module:, based on a set of activities, metrics and guidelines.

• Embedded software component Quality Model (EQM)

• Maturity Level evaluation Techniques

• Metrics Approach

• Component Certification

based on a set of activities, metrics and guidelines.



SAAP 2008.1

Proposed solution

This Framework is based in the standards

• ISO/IEC 9126, 2001 - Quality Model for Software Product

• ISO/IEC 14598, 1998 - Software Product Evaluation Process

This two standards converged to:

• ISO/IEC 25010, 2005 - Software product quality - requirements and evaluation

the Framework adapted the quality model and evaluation to component context and embedded domain.



SAAP 2008.1

Out of scope

This Framework is part of broad context, some aspects were expected since initial definition. Nevertheless, other process can be added in the future.

•Cost Model

•Formal Proof

•Prediction of the component assembly



SAAP 2008.1

2 – Embedded System DesignEmbedded system design comprise:

• Ultra-small device x simple functionality

• Small system x sophisticated functions

• Large systems and distributed systems

• Systems produced in large quantities x low production cost

• Systems produced in low volume x important features



SAAP 2008.1

But the specifics requirements depends on the application (domain)- Industrial Automation - Automotive- Medical - Consumer electronics- Aircrafts- others …




SAAP 2008.1


So, embedded component certification process must be different that general propose component, because the component evaluation is realized focused in other requirementsWe divided in two groups:• General propose sw component certification process

o desktops, servers, x86 architecture

• Specific propose sw component certification process (embedded systems, others architectures)



SAAP 2008.1

Component Certification Processfocused in embedded systems (requiriments)

• Industrial Automation (Crnkovic et al., 2005) In the lowest levels: availability, timeliness, and reliability At higher levels: performance, usability, and integrability



SAAP 2008.1

Component Certification Processfocused in embedded systems (requirements)

• Automotive (Åkerholm e.t. al, 2004) Extendibility Maintainability Usability Predictability Security Safety Reliability Testability Flexibility Efficiency



SAAP 2008.1

Component Certification Processfocused in embedded systems (requirements)

• Medical (Wijnstra e.t. al,2001) Reliability Safety Functionality Portability Modifiability

Configurability Extensibility and Evolvability

Testability Serviceability



SAAP 2008.1

• Certification is the future of software components [Wallnau, Heineman, Councill, Shaw]

• RiSE Approach1.Embedded Component Quality Model (ECQM)

Embedded Software Component Maturity Model (ESCMM)2.Certification Techniques Framework

Defining techniques for evaluate quality attributes of ECQM3.Metrics Framework

Track the properties of the ECQM, certification techniques and process.

4.Embedded Software Component Certification Process Defining the steps for certify a component

Embedded Software Component Quality Process



SAAP 2008.1

1. Embedded Component Quality Model (ECQM)o Embedded Software Component

Maturity Model (ESCMM)o Based ISO/IEC 25010 (*)

standard, with some news and extended characteristics.

Embedded Component Certification Process

*ISO/IEC 9126 - Software Products Quality ISO/IEC 14598 - Evaluation software product

Development timeCostTime to marketTargeted marketAffordabilityLicensing

Marketability

DeployabilityReplaceabilityAdaptabilityReusability

Portability

AnalyzabilityStabilityChangeabilityTestability

Maintainability

Time BehaviorResource behaviorScalability

Efficiency

UnderstandabilityConfigurabilityLearnabilityOperability

Usability

MaturityRecoverabilityFault Tolerance

Reliability

SuitabilityAccuracyInteroperabilitySecurityComplianceSelf-contained

Functionality

Sub-Characteristics

Characteristics

Changes in the Proposed Component QualityModel, in relation to ISO/IEC 25010



SAAP 2008.1

1. Embedded Component Quality Model (ECQM)

Embedded Component Certification Process

Changes in the Proposed Component QualitModel, in relation to ISO/IEC 25010

Development timeCostTime to marketTargeted marketAffordabilityLicensing

Marketability

DeployabilityReplaceabilityAdaptabilityReusability

Portability

AnalyzabilityStabilityChangeabilityTestability

Maintainability

Time BehaviorResource behaviorScalability

Efficiency

UnderstandabilityConfigurabilityLearnabilityOperability

Usability

MaturityRecoverabilityFault Tolerance

Reliability

SuitabilityAccuracyInteroperabilitySecurityComplianceSelf-contained

Functionality

Sub-Characteristics

Characteristics



SAAP 2008.1

2. Certification Techniques Frameworko Defining techniques for evaluate quality attributes of ECQM

Guidelines for selecting evaluation level.

Financial disasterMany people killedUnrecoverable environmental damage

ESCMM V

Large economic gross

Threat to human livesRecoverable environment damage

ESCMM IV

Significant economic loss

Large number of people disabled

Damage properlyESCMM III

Few economic loss

Few people disabledSmall/Medium damage properly

ESCMM II

Negligible economic loss

Few material damage; No specific risk

No damageESCMM I

EconomicSafety/SecurityEnvironmentLevel



SAAP 2008.1

2. Certification Techniques Frameworko Defining techniques for evaluate quality attributes of ECQM

Guidelines for selecting evaluation level.

• Programming LanguageFacilities (BestPractices)• Maturity analysis

ESCMM II

• Fault toleranceanalysis• Error Manipulationanalysis

ESCMM III

Formal Proof

Reliability growthmodel

Suitability analysis

Reliability

ESCMM VESCMM IVESCMM ILevel



SAAP 2008.1

3. Metrics Frameworko Track the properties of the ECQM, certification techniques and

process.

0 <= x <= 1; which closer to 1 is betterInterpretation

Precision on results / Amount of testsMetric

Based on the amount of tests executed, how much test results return with precision?

Question

Evaluates the percentage of the results that were obtained with precision

Goal

CorrectnessQuality Attribute

AccuracySub-Characteristic

Functionality



SAAP 2008.1

4. Embedded Software Component Certification Processo Defining the steps for certify a component

4.1 Establish Evaluation Requirements activity 4.2 Specify the Evaluation activity 4.3 Design the Evaluation activity 4.4 Execute the Evaluation activity



SAAP 2008.1

4. Embedded Software Component Certification Process



SAAP 2008.1

4.1 Establish Evaluation Requirements activity



SAAP 2008.1

4.2 Specify the Evaluation activity



SAAP 2008.1

4.3 Design the Evaluation activity



SAAP 2008.1

4.4 Execute the Evaluation activity



SAAP 2008.1

Submissão de artigo dia 05/06/2008 - SBCARS



SAAP 2008.1



SAAP 2008.1

Thank you !

Questions ?

• Fernando F. de Carvalhoo [email protected]



SAAP 2008.1

Embedded Component Certification Timeline

a embedded software component quality framework fernando ferreira de carvalho advisor: silvio romero...

Documents