software prediction models forecasting the costs of software development

Software Prediction Models Forecasting the costs of software development

Upload: alaina-knight

Post on 04-Jan-2016

217 views

Category:

Documents

1 download

Report

Download

Embed Size (px):

TRANSCRIPT

Software Prediction Models

Forecasting the costs of software development

Prediction Study Outcomes Vary

Estimation-by-analogy beats regression Or not Classification and regression trees (CART)

beats regression Or not Artificial neural networks beat regression Or not

Page 3: Software Prediction Models Forecasting the costs of software development

Why Are The Results Conflicting?

Poor data or research procedure Complex techniques may require expert

users; hence applications may vary Small sample size Measurement process that is flawed Selective use of differing parameters may

result in different rankings

Page 4: Software Prediction Models Forecasting the costs of software development

Key Terms

Accuracy indicator– Some measure of a process– A summary statistic based on that measure

Leave-one-out cross-validation Arbitrary function approximator taxonomy

– Many-data versus sparse-data– Linear versus nonlinear– Supervised versus unsupervised

Reliability versus validity

Page 5: Software Prediction Models Forecasting the costs of software development

Indicator 1: MMRE

Mean magnitude of relative error (MMRE) is an average where the MRE=|actual-prediction|/actual

Claimed advantages of MMRE– Compare across data sets*– Independent of units– Compare across differing prediction models*– Scale independence

*An hypothesis challenged by this paper

Page 6: Software Prediction Models Forecasting the costs of software development

Indicator 2: MER

Magnitude of the error relative to the estimate (MER) is defined asMER = |actual-prediction|/prediction

Page 7: Software Prediction Models Forecasting the costs of software development

Indicator 3: AR

The absolute residual (AR) is defined asAR = |actual-prediction|

Page 8: Software Prediction Models Forecasting the costs of software development

Other Measures

Standard deviation (SD) Relative standard deviation (RSD) Log standard deviation (LSD) Balanced relative error (BRE) Inverted balanced relative error (IBRE)

Page 9: Software Prediction Models Forecasting the costs of software development

Standard Deviation of Residuals, Denoted SD

zero is ˆ residuals

theofmean thebecause occurstion simplifica The 2.

StDevfor formula general theNOT is formula This 1.

:Notes

)1(

SD 1

iii

yyr

Page 10: Software Prediction Models Forecasting the costs of software development

Algebraic Simplification

1)(

residuals. theofmean the and

residual therepresent ˆLet

rrresidualsStDev

iyyr

iii

thiii

Page 11: Software Prediction Models Forecasting the costs of software development

Relative Standard Deviation (RSD)

)1(

RSD 1

xyyn

i i

Page 12: Software Prediction Models Forecasting the costs of software development

Log Standard Deviation (LSD)

?ˆlnln

and residual theof variance the

where

)1(

2 LSD

iii

yye

Page 13: Software Prediction Models Forecasting the costs of software development

Balanced Relative Error (BRE)

0ˆ,

ˆ)ˆ(

0ˆ,)ˆ(

BREyy

yyy

Page 14: Software Prediction Models Forecasting the costs of software development

Inverted Balanced Relative Error (IBRE)

0ˆ,

ˆ)ˆ(

0ˆ,)ˆ(

IBREyy

yyy

Crime Prediction and Forecasting: Feature Selection and

Earthquake Prediction and Forecasting

Prediction Markets for Economic Forecasting · for many prediction market researchers, he will be sorely missed. ... Prediction Markets for Economic Forecasting Erik Snowberg, Justin

TOWARDS EPIDEMIC PREDICTION FEDERAL EFFORTS … · TOWARDS EPIDEMIC PREDICTION: FEDERAL EFFORTS AND OPPORTUNITIES IN OUTBREAK MODELING PRODUCT OF THE Pandemic Prediction and Forecasting

1 Survey on Software Defect Prediction - GitHub Pageslifove.github.io/files/PQE_Survey_JC.pdf · 1 Survey on Software Defect Prediction Jaechang Nam Abstract Software defect prediction

Software Reliability Prediction - SoftRel, LLC · Survey Based Defect Density Models ... Software prediction confidence bounds are a function of 0 ... Software Reliability Prediction

A General Approach to Prediction and Forecasting Crime Rates with Gaussian … · 2020-03-06 · A General Approach to Prediction and Forecasting Crime Rates with Gaussian Processes

ATM 401/501 Status of Forecasting: Spring 2013 1. Forecasting at NCEP Environmental Modeling Center Ocean Prediction Center

Operational Hydrologic Ensemble Forecasting - CNRFC · Operational Hydrologic Ensemble Forecasting ... NWSRFS configuration. probablistic ... Ensemble Streamflow Prediction. Climate

Flood prediction forecasting using machine Learning Algorithms

FORECASTING ELECTIONS COMPARING PREDICTION MARKETS, POLLS, AND

Neural networks for the prediction and forecasting of water resources variables

Weather Research and Forecasting (WRF) Performance ...Weather Research and Forecasting (WRF) • The Weather Research and Forecasting (WRF) Model – Numerical weather prediction system

Software Consulting Forecasting

FORECASTING BANKRUPTCY PREDICTION USING CONDITIONAL …

Solar Energy Forecasting Using Numerical Weather Prediction (NWP) Models

National Air Quality Forecasting Capability: Nationwide Prediction and Future Enhancements

Survey on Software Defect Prediction Using Machine ... · Survey on Software Defect Prediction Using ... Software defect prediction is seen as a highly important ability when planning

NUMERICAL WEATHER PREDICTION IN VERY SHORT TERM WIND FORECASTING

Neural networks for the prediction and forecasting of water.pdf

Reliability Prediction for Fault-Tolerant Software ...This section outlines our reliability prediction approach forfault-tolerant software architectures, software families, and software

Baseline Forecasting for Population Ageing and · PDF fileBaseline Forecasting for Population Ageing and Greenhouse Gas ... forecasting with exogenous population prediction input as

Quantitative Precipitation Forecasting at the Hydrometeorological Prediction Center (HPC)

Software Reliability Prediction using Fuzzy Inference ...€¦ · Software Reliability Prediction using Fuzzy Inference ... Software Metrics, Software Reliability ... because with

Operational Hydrologic Ensemble Forecasting - ECMWF · Operational Hydrologic Ensemble Forecasting. ... Prediction Center ... NWSRFS Hydrologic Models Time Streamflow April-July

Pelatihan software eviews 6 forecasting

Simulation Software for Real Time Forecasting as an ... · PDF fileSimulation Software for Real Time Forecasting as an Operational Support ... Simulation Software for Real Time Forecasting

An Empirical Study on Software Defect Prediction with a ... · An Empirical Study on Software Defect Prediction with a Simpliﬁed Metric Set ... defect prediction, software metrics,

Software Defect Prediction Techniques in Automotive Domain ... · of machine learning based software defect prediction techniques. The effective use of software defect prediction

ENSEMBLE FORECASTING – A NEW PARADIGM IN NUMERICAL WEATHER PREDICTION

Ensemble Prediction Systems and Probabilistic Forecasting Richard (Rick) Jones SWFDP Training Workshop on Severe Weather Forecasting Bujumbura, Burundi,

Big Data and Machine Learning Using MATLAB · Software Servers and Databases Repositories –SQL, NoSQL, etc. ... Weather Forecasting Energy Load Forecasting Stock Market Prediction

ProfitSage Software Forecasting & Budgeting Manual

Lightning Forecasting at the Storm Prediction Center

Time-series Forecasting - Parametric prediction models, ARIMA › lehre › ba-18w › script › arima.pdf · Time-series Forecasting - Parametric prediction models, ARIMA Non-Seasonal