INTRODUCTION
As one of the most important aspects of software quality, software reliability
is often defined as the probability of failurefree software operation for a
specified period of time in a specified environment (Musa
et al., 1987; Lyu, 1996; Gokhale
et al., 2004; Williams, 2006). Many Software
Reliability Growth Models (SRGMs) have been developed to describe the errordetection
process and estimate the growth reliability of products during software development
process (Musa et al., 1987; Lyu,
1996; Pham and Zhang, 1999; Zhao
et al., 2006). Most existing SRGMs consider only the software fault
detection process in the testing stage, assuming immediate fault correction
activity (Schneidewind, 2001; Xie
et al., 2007; Huang and Huang, 2008). However
in practice, each detected fault is reported, located and then corrected. The
software fault correction process plays a significant role in improving software
reliability and ensures software quality (Schneidewind, 2003;
Huang and Lin, 2006). Thus, the fault correction process
cannot be simply treated as an immediately achieved process. In addition to
predicting and assessing software failure occurrence, it is also m eaningful
for analyzing and modeling the software fault correction.
Recently, some researchers have emphasized the great importance of fault correction
process modeling. Since, the fault correction process is usually lagging behind
the fault detection process, the correction lag is a common phenomenon in the
software testing. Schneidewind (2001,
2003) first modeled the fault correction process as a constant delay fault
detection process. Within this framework, some extensions have been proposed
by substituting various timedependent delay functions for the constant delay
(Huang and Lin, 2006; Lo, 2007;
Xie et al., 2007). The introduction of timedelay functions is very useful
to separate the fault correction from the fault detection. It can help people
find more useful models. Wu et al. (2007) summarized
several kinds of timedelay distribute functions in constructing the fault correction
process models. Following the similar idea of time delay, some other attempts
have also been made to model the fault detection and correction process. For
instance, Huang and Huang (2008) incorporated server
queuing models into the fault correction modeling by considering imperfect debugging;
Gokhale et al. (2004, 2006)
modeled the fault detection and correction processes with a nonhomogeneous
Markov chain. Lo and Huang (2006) proposed a framework
to integrate two processes by the simultaneous differential equations. Kupar
et al. (2008) indicated that there are two types of detected faults
in the software which need different testing effort for correction. They used
the simultaneous differential equations to integrate the two processes.
In the work mentioned earlier, the correction lag has been treated as a time
delay between the fault detection and correction processes. It is represented
by many kinds of timedelay functions. However, these timedelay functions have
different forms because they are established under different assumptions. In
reality, the time to remove a fault depends on many factors such as the number
of the detected faults, the complexity of the software structure, the skill
of the correction personnel, etc. It seems difficult to choose an accurate timedelay
function to describe the correction lag in various testing circumstances. In
reality, as a fault must be corrected and removed after its detection, the number
of corrected faults is naturally depended on the number of detected faults.
Since the correction lag cannot be ignored, some detected but not corrected
faults exist in the software. These latent faults are caused by the correction
lag and reflect the relationship between the fault detection and correction
processes. We can use them to integrate the two processes together.
In this study, the correction lag is defined as the difference between the number of detected faults and the number of corrected faults. We are going to model both fault detection and correction processes based on this lag.
THE CORRECTION LAG
The purpose of software testing is finding and correcting faults before the software product delivery to the customs. When a failure is detected, the fault correction personnel need a period of time to locate the fault and modify some codes to remove it. The correction lag is a common phenomenon in software testing. Some detected but not corrected faults exist in the software. We call them the remaining uncorrected faults in this study. These faults directly represent the correction lag between the fault detection and correction processes. The number of remaining uncorrected faults is the difference between the number of detected faults and the number of corrected faults, which is φ(t) = m_{d}(t) = m_{c}(t).
Xie et al., 2007 used an actual set of data from the testing on a middle
size software project to study the fault detection and correction processes
with several kinds of timedelay distribute functions. Figure
1 plots the number of detected, corrected, remaining uncorrected faults
of this data set. It is analyzed that the number of detected faults varies like
the number of corrected faults means they are dependent on each other. Moreover,
the number of remaining uncorrected faults is not constant, increasing from
the beginning of software testing and decreasing to minimum value at the later
stage. Musa et al. (1987) reported a data set of
System T1 which includes the number of detected and corrected faults. System
T1 was used for a realtime command and control application.

Fig. 1: 
The number of detected and corrected faults of a middle size
software project 

Fig. 2: 
The number of detected and corrected faults of system T1 
Figure 2 depicts the number of detected and corrected faults
of System T1. Figure 2 shows that the number of remaining
uncorrected faults has an increasing and then decreasing scenario. Next, we
will discuss about why the correction lag has such a varying trend.
At the beginning of software testing, there are many faults in the software
and they are easy to be detected. While in this time, the effective debugging
is not easy, the testing personnel need time to read and analyze the failure
report to find out the root cause of failure (Huang and
Lin, 2006). Thus, the correction process gradually falls far behind the
fault detection process and the correction lag is increasing. As testing proceeds,
Fig. 1 and 2 show that the correction lag
is decreasing. Although, the number of remaining uncorrected faults may not
be monotonously decreasing versus time, it is approximately reduced at the latter
stage of testing. During this time, faults in the software are almost completely
detected and the number of detected faults increases very slowly. Simultaneity,
the fault correction personnel are engaging to correct each detected fault.
Thus, the difference between the number of detected faults and the number of
corrected faults is decreasing. Generally speaking, although, some of noncritical
detected faults might be postponed to be corrected until next release, most
of detected faults should be corrected before software release. A large number
of remaining uncorrected faults cannot be acceptable for both software developers
and users. It is natural to see that the number of remaining uncorrected faults
is decreasing to the minimum value at the end of testing process. From the analysis
above, it can conclude that the number of remaining uncorrected faults has an
increasing and then decreasing scenario. This feature of correction lag is also
testified by experimental results from Huang’s server queuing models (Huang
and Huang, 2008).
To characterize the varying trend of the remaining uncorrected faults, we use
the Gamma curve and other curves (such as Weibull, Raleigh, Lognormal curves)
are used as the correction lag function to fit it on these two data sets. Through
simulation, we find out that the Sum of Squares Errors (SSE) value of the Gamma
curve on these data sets are 93.89 and 169.48, which are the lowest SSE value
compared with other curves. As the Gamma curve has the best goodness of fit,
we select the Gamma curve as the correction lag function:
SOFTWARE RELIABILITY GROWTH MODELING
Here, the fault detection and correction processes are modeled based on NonHomogeneous
Poison Process (NHPP). The correction lag between fault detection and correction
processes is characterized by the remaining uncorrected faults. As studied in
the former section, the number of remaining uncorrected faults has an increasing
and then decreasing feature and the Gamma curve is selected to represent it.
The followings are the explicit assumptions for both fault detection and correction
processes models:
• 
The failure observation/fault correction phenomenon is modeled
by NHPP 
• 
The software system is subject to failures during execution caused by
faults remaining in the software 
• 
All faults are independent and equally detectable 
• 
The software failure rate at any time is affected by the number of faults
remaining in the software at that time 
• 
Each time a failure occurs, the fault that caused it is not
immediately removed because the correction lag φ exists in the debugging
process 
According to these assumptions, we have:
and
To solve the simultaneous differential equations simply and easily, we assume
a(t), b(t) are constant, which means no fault is introduced and the failure
rate is not changing during the software testing. So, Eq. 2
is changed into:
Equation 3 is directly deduced from assumption 5. It can
be transformed as:
Substituting Eq. 5 into 4, we obtain a
new differential equation as:
When c = 0, then
It means the correction lag does not exist in the software testing and the
fault correction model is just equal to the fault detection model, m_{d}(t)
= m_{c}(t). In this case, by solving the Eq. 6 with
marginal conditions m_{d}(0) = 0 and m_{d}(+∞) = a, we
can get m_{d}(t) = m_{c}(t) = m_{c}(t) = m_{c}(t)
= a(le^{bt}), the model becomes the GoelOkumoto (GO) model, which
is a fundamental SRGM. Otherwise c ≠ 0, by solving Eq. 6,
we get a new SRGM for the fault detection process:
From Eq. 5 and 7, we can get the fault
correction model which paired with the fault detection model as:
MODEL EVALUATION
Here, we will evaluate the proposed model for both fault detection and correction
processes by using two real data sets. The first data set (DS1) is from a middle
size software project reported by Xie et al., 2007 and the second data
set (DS2) is from System T1 of the Roma Air Development Center project (Musa
et al., 1987). Table 1 summarizes some existing
SRGMs which modeled both fault detection and correction processes. These models
are from Schneidewind et al. (2001, 2003),
Xie et al., 2007, Lo, 2007and Huang
and Huang (2008).
Parameter estimation: With the model which combined the fault detection
process with the fault correction process, a software testing can be described
more practically than conventional SRGMs. There are unknown parameters in the
model and they can be estimated by using the method of least squares. Against
observations of fault detection and correction processes, the parameters of
the new model are estimated by minimizing the Sum of Square Residual (SSR) (Xie et al., 2007), which is defined as follows:
where, yd_{k} and yc_{k} denote the cumulative number of detected
and corrected faults collected until t_{k} respectively, t_{k},
k = 1,2,…, are the running times from the begging of testing.
Substituting m_{d}(t) and m_{c}(t) into Eq. 9
with a data set which includes the fault detection and correction processes,
then the equation of SSR is obtained. If we want to estimate the parameters
of the proposed model on the data set, the proposed model is substituted into
Eq. 9 and the minimum value of SSR can be deduced by solving
the differential equations:
The parameters of the proposed model can be estimated as the results of the
differential equations. Commonly, numerical procedures have to be developed
in order to obtain the Least Squares Estimates (LSEs).
Criteria for model comparison: In the following comparison criteria
expressions, every variable or function is used for both fault detection and
correction processes:
Table 1: 
Summarization of models 

• 
The goodness of fit of the curve is measured by the sum of
squares errors, SSE. Sum of Square residual SSE sums up the squares of the
residuals between the actual data and the mean value function m(t) of each
model in terms of the number of actual faults at any time. The SSE function
can be expresses as follows: 

where, y_{k} is the total number of faults observed
at time t_{k} according to the testing time and
is the estimated cumulative number of faults at time t_{k} obtained
from the fitted value function, k = 1,2,..n. Therefore, the lower the SSE
value, the better the model performs 
• 
After the proposed model is fitted to the actual observed
data, the deviation between observed and fitted values can also be evaluated
by a using the KolmogorovSmirov (KS) test. The KolmogorovDistance (KD)
is defined as: 

Where, k is the sample size, F*(x) is the normalized observed
cumulative distribution at xpoint and F(x) is the expected cumulative distribution
at xpoint, based on the model 
• 
The capability of the model to predict failure behavior from
present and past failure behavior is called predictive validity, which can
be represented by computing the Mean value of Relative Errors (MRE) for
a data set: 

Suppose Z failures have been observed by the end of testing
time t_{z}, we use the failure data up to time t_{i}(t_{i}≤t_{z})
to estimate the parameters of m(t). Moreover, a lower value of MRE indicates
a better performance of fitting the real software data 
Experiments on DS1: We estimate the parameters of the selected models
and the proposed models by using the LSEs method on DS1. The estimated parameters
of all models are listed in Table 2. Table 3
gives the performance comparisons in terms of SSE, KD and MRE. Table
3, show the proposed model provides the lowest value of all criteria only
except the KD value on the detection process. On the average, it can figure
out that the proposed model gives a better fit and prediction in both detection
and correction processes than other models. As can be seen from Table
3, the proposed model, Schneidewind model and Lo model do well in fitting
this data set on both fault detection and correction processes. Figure
3 and 4 depict the estimated data by these three models.
We can see that all of these three models have a good performance on DS1.
It is also calculate the estimated number of remaining uncorrected faults by
using the proposed model, Schneidewind model and Lo model. Figure
5 graphically shows the actual and estimated number of remaining uncorrected
faults versus time. The difference between detected and corrected faults is
initially zero at the beginning of software testing and it is increasing rapidly
as test proceeds. This phenomena means effective debugging is difficult. The
testing personnel need time to analyze the failure report and find the root
of failure. The correction process falls far behind the fault detection process
and the remaining uncorrected faults increase. After reaching the maximum value,
the number of remaining uncorrected faults is decreasing. At this time, faults
in the software are almost completely detected, the detected fault number increases
very slowly. On the other hand, the testing personnel master the whole software
system better than before the detected faults can be corrected in time. So,
the corrected fault number gets gradually closer to the detected fault number.
The remaining uncorrected faults decrease quickly to the minimum value. All
three models can characterize the increasing and then decreasing feature of
remaining uncorrected faults. Furthermore, the proposed model is the best one
for representing the correction lag and it also has the best goodness of fit
on DS1. Therefore, it conclude that using the remaining uncorrected faults is
a better way to integrate the fault detection and correction processes than
using the time delay functions.
Table 2: 
Parameter estimation of different SRGMs on DS1 

Table 3: 
Comparison results on DS1 


Fig. 3: 
Detected fault data on DS1: fitted versus actual 

Fig. 4: 
Corrected fault data on DS1: fitted versus actual 
Experiments on DS2: Similarly, we also estimated the parameters of the
proposed model and other selected SRGMs by using the methods of LSEs on DS2.
The estimated parameters are listed in Table 4. Table
5 lists the comparisons results in terms of SSE, KD and MRE. The proposed
model has the lowest value of SSE, MRE on both fault detection and correction
processes.

Fig. 5: 
Actual/estimated remaining uncorrected fault for DS1 

Fig. 6: 
Detected and corrected fault data on DS2: fitted versus actual 

Fig. 7: 
Actual/estimated remaining uncorrected fault on DS 
Although, the proposed model does not provide the lowest value of KD on the
two processes, it also gives very close value to them. Figure
6 shows the estimated data by the proposed model versus the actual data
set. It can see that the proposed model fit the data set very well in both detection
and correction processes.
Table 4: 
Parameter estimation of different SRGMs on DS2 

Table 5: 
Comparison results on DS2 

Overall, the proposed model gives a better fit and prediction on DS2.
As can be seen from Table 4, the proposed model, Xie model and Lo model do well in fitting DS2. Figure 7 graphically shows the actual and estimated number of remaining uncorrected faults versus time. We can see that the proposed model can characterize the actual remaining uncorrected faults better than other two models. As can be seen from Table 4, the Xie model also fits DS2 very well and it provides a very close SSE value to the proposed model. But Fig. 7 shows that the Xie model cannot characterize the correction lag as well as the proposed model. This fact means the Xie model cannot fit the DS2 as precisely as the proposed model does. Hence, the relationship between the fault detection and correction processes can be more accurately described by the proposed model from the viewpoint of fault amount.
SOFTWARE RELEASE PROBLEM
Although, testing is an efficient way to find and eliminate the faults in software
systems, the exhaustive testing is impractical in fact. The software developers
need to decide when to stop the testing and release the software to customers.
As software reliability growth models can capture the quantitative aspects of
the testing, they are used to provide a reasonable software release time. Some
researchers have studied the release time problem from the costbenefit viewpoint
and they suggested that the optimal release time T* can be obtained by minimizing
the expected total software system cost (Lyu, 1996;
Pham and Zhang, 1999, 2003; Huang
and Lyu, 2005). For instance, Pham and Zhang (1999)
proposed a generalized software cost model including removal cost, warranty
cost and risk cost, which is:
where, c_{1} is the software test cost per unit time, c_{2}
is the cost of correcting a fault per unit time during testing period, c_{3}
is the cost of correcting a fault per unit time during the warranty period,
c_{4} is the loss due to software failure, C(T) is the expect total
cost of software systems at time T, α is the discount rate of the testing
cost, u_{y} is the expected time to correct a fault during the testing
period, u_{w} is the expected time to correct a fault during the warranty
period; T_{w} is the period of warranty time and R(x/T) is the reliability
function of software by time T for a mission time x which can be calculate by
the SRGMs and it is expressed as:
R(x/T) = e^{[m(T+x)m(T)]} 
The conventional SRGMs, m(T), only describe the fault detection process, which
is represented by m_{d}(T) in this study. As the proposed SRGM models
both fault detection and correction processes, the traditional cost function
C(T), can be revised to be more practical by incorporating the fault correction
process model m_{c}(T):
where, m_{c}(T) is the expected number of corrected faults at the time
of release T, m_{d}(T+T_{w})m_{c}(T) is the expected
number of corrected faults during the warranty period, which includes two components:
undetected faults m_{d}(T+T_{w})m_{d}(T) and remaining
uncorrected faults m_{d}(T+T_{w})m_{c}(T).
The latter component represents the correction lag between the fault detection and correction processes which has been specifically discussed in our study and it exists in most software testing. These detected but uncorrected faults remain in the software even after the software release for some potential reasons, such as the limited testing time or the testing resource, the detected fault is uncritical, etc. The correction of these detected faults will cost more efforts in the operation period. We can conclude that the revised software cost model C’(T), incorporates the fault detection and correction processes and reflect the correction lag phenomena, so it is very practical.
Pham and Zhang (1999) used DS2 to show how to determine
the optimal release time by their generalized software cost model C(T) with
the conventional SRGMs. They considered the coefficients as c_{0} =
50, c_{1} = 700, c_{2} = 60, c_{3} = 3,600, c_{4}
= 50,000, T_{w} = 20, u_{y} = 0.1, u_{w} = 0.5, x =
0.05, α = 0.95. Substituting the proposed fault detection process model m_{d}(t)
to Eq. 13 with the estimated parameters on DS2,
= 138.65,
=0.1351 c = 5.44, =8.65,
we get that T* is 30.1 h and the lowest cost is $22,760 by minimizing C(T).

Fig. 8: 
Plot of the total cost 
The corresponding cost function is plotted as a dash line in Fig.
8.
To show the difference in using the new software cost model, we also substitute the proposed model for both fault detection process and fault correction process, m_{d}(t) and m_{c}(t), into the revised cost function C’(T). With the same coefficients and estimated parameters value, the optimal release time can be obtained as T’* = 33.5 h with the lowest cost of $24,538. The revised cost function is also plotted as a solid line in Fig. 8. From the numbers and the figure, we can find out that a later optimal release time and a larger total cost are provided by the revised cost model. This result shows the effects of the correction lag. In reality, people need more time and recourse to perform the fault correction activity. As the traditional cost model does not consider the correction lag, it is optimistic in determining the software release time.
CONCLUSIONS
The fault correction process plays a significant role in ensuring the software quality and it always lags behind the fault detection process. In this study, the correction lag between the fault detection and correction processes was directly studied from a novel viewpoint of fault number. We investigated the number of remaining uncorrected faults on real data sets and found that it had an increasing and then decreasing feature. The Gamma curve was appropriate in characterizing the remaining uncorrected faults. Then we modeled the fault detection and correction processes based on the correction lag. Numerical examples showed that the proposed model successfully described the two processes and the comparison results also indicated that using the remaining uncorrected faults to integrate the two processes was more effective than using the time delay functions. Furthermore, we studied the software release problem. The proposed SRGM was applied to extend the software cost model and a more reasonable software release time was achieved by considering the effect of the correction lag.
ACKNOWLEDGMENTS
The work described in this study is jointly supported by HiTech Research and Development Program of China (2008AA01A201) and National Nature Science Foundation of China (60503015).
NOTATIONS
MVF 
: 
Mean value function 
SSE 
: 
Sum of squared errors 
MRE 
: 
Means of relative errors 
NHPP 
: 
Nonhomogeneous Poison process 
SRGMs 
: 
Software reliability growth models 
m_{d}(t) 
: 
The expected number of faults detected in time (0, t) 
m_{c}(t) 
: 
The expected number of faults corrected in time (0, t) 
φ(t) 
: 
The number of remaining uncorrected faults at time t 
a 
: 
Expected number of initial faults 
b 
: 
Fault detection rate 
y_{k} 
: 
No. of actual faults observed at time t_{k} 