Research

ROBUST is the product of Colorado State University's ongoing research into software reliability. A great deal of this research has been published, and more is forth coming. The key papers, describing the theory, equations, and algorithms used by ROBUST, are available here in both postscript and Adobe Acrobat format. For you convince, a BibTex file for these references is available.

Estimating Defect Density Using Test Coverage
By Y. K. Malaiya and J. A. Denton
Submitted to the 9th International Symposium On Software Reliability Engineering
Defect density is one of the most important factors that allow one to decide if a piece of software is ready to be released. In theory, one can find all the defects and count them, however it is impossible to find all the defects within any reasonable amount of time. Estimating defect density can become difficult for high reliability software, since the remaining defects can be extremely hard to test. Defect seeding will work only if the distribution of seeded defects is similar to the existing defects. One possible way is to apply the exponential SRGM and thus estimate the total number of defects present at the beginning of testing. Here we show the problems with this approach and present a new approach based on software test coverage. Software test coverage directly measures the thoroughness of testing avoiding the problem of variations of test effectiveness. Here we present interpretations of the parameters of the coverage- defect-density model presented by Malaiya et al. We apply this model to actual test data to project the residual defect density. The results show that this method results in estimates that are more stable than the existing methods. This method is easier to understand and the convergence to the estimate can be visually observed.

What do Software Reliability Parameters Means?
By Y. K. Malaiya and J. A. Denton
Appeared in Proc. 8th International Symposium On Software Reliability Engineering, 1997, Albuquerque, NM. Pages 124-135
Here we investigate the underlying basis connecting the software reliability growth models to the software testing and debugging process. This is important for several reasons. First, if the parameters have an interpretation, then they constitute a metric for the software test process and the software under test. Secondly, it may be possible to estimate the parameters even before testing begins. These {\em a priori} values can serve as a check for the values computed at the beginning of testing, when the test-data is dominated by short term noise. They can also serve as initial estimates when iterative computations are used.

Among the two-parameter models, the exponential model is characterized by its simplicity. Both its parameters have a simple interpretation. However, in some studies it has been found that the logarithmic Poisson model has superior predictive capability. Here we present a new interpretation for the logarithmic model parameters. The problem of a priori parameter estimation is considered using actual data available. Use of the results obtained is illustrated using examples. Variability of the parameters with the testing process is examined.

The Relationship Between Test Coverage and Reliability
By Y. K. Malaiya, J. Bieman, R. Karcich, and B. Skibbe
Appeared in 5th International Symposium On Software Reliability Engineering, 1994
We model the relation among testing effort, coverage and reliability, and present a logarithmic model that relates testing effort to test coverage (block, branch, c-use or p-use). The model is based on the hypothesis that the enumerables (like branches or blocks) for any coverage measure have different detectability, just like the individual defects. This model allows us to relate a test coverage measure directly with defect coverage. Data sets for programs with real defects are used to validate the model. The results are consistent with the known inclusion relationships among block, branch and p-use coverage measures. We show how defect density controls time to next failure.

The model can eliminate the variables like test application strategy from consideration. It is suitable for high reliability applications where automatic (or manual) test generation is used to cover enumerables which have not yet been tested.

Enhancing Accuracy of Software Reliability Prediction
By N. Li and Y. K Malaiya
Appeared in 4th International Symposium On Software Reliability Engineering, 1993, Pages 71-79
The measurement and prediction of software reliability require the use of the Software Reliability Growth Models (SRGMs). The predictive quality can be measured by the average end-point projection error. In this paper, the effects of two orthogonal classes of approaches to improve prediction capability of a SRGM have been examined using a large number of data sets. the first approach is preprocessing of data to filter out short term noise. The second is to overcome the bias inherent in the model. The results show that proper application of these two approaches can be more important the the selection of the model.

Accurate Software Reliability Estimation
Masters Thesis for Jason Denton, Colorado State University 1999
This masters thesis provides an overview of the exponential and logarithmic software relability models, presents the techniques used by ROBUST to estimate their parameters, and presents the initial work on stabilization.