Entropy based Software Reliability Growth Modelling for Open Source Software Evolution

: During Open Source Software (OSS) development, users submit "new features (NFs)", "feature improvements (IMPs)" and bugs to fix. A proportion of these issues get fixed before the next software release. During the introduction of NFs and IMPs, the source code files change. A proportion of these source code changes may result in generation of bugs. We have developed calendar time and entropy-dependent mathematical models to represent the growth of OSS based on the rate at which NFs are added, IMPs are added, and bugs introduction rate.The empirical validation has been conducted on five products, namely "Avro, Pig, Hive, jUDDI and Whirr" of the Apache open source project. We compared the proposed models with eminent reliability growth models, Goel and Okumoto (1979) and Yamada et al. (1983) and found that the proposed models exhibit better goodness of fit.


INTRODUCTION
OSS evolution is based on the bug triaging process where different reports about the issues are filed. The various attributes related to the issues are also filed during the reporting. The reported issues are assigned to different developers for fixing. The issues which are reported by the users are mainly NFs, IMPs and bugs [28].
In line with the OSS architecture model proposed in [1], we expanded it by incorporating requests, namely feature improvements shown as green colour boxes in Fig.  1. Fig. 1 shows that to fix different issues, developers modify the source code which may result in bugs. In order to fix the issues a lot of changes must be made in the source code. These modifications to the source code were quantified using measurement dependent upon entropy called the "complexity of code changes" [2,3]. Code changes are quantified based on Shannon entropy [6]. Cobb-Douglas based two dimensional and three dimensional models have been proposed to predict the entropy of software systems by considering bugs, NFs and IMPs [13].

Figure 1
A classification of open source users and developers [1] In this paper, we proposed calendar time and entropy based models for a software product to estimate the number of issues fixed and to predict the leftover issues which need to be fixed over a long run. The models consider different rates at which different issues are fixed and the rate at which bugs are generated during fixing of these issues.
We have taken into consideration two existing SRGMs [11,12] to compare with our proposed models. The existing and proposed models have been validated using data collected from the various products of Apache open source project. Results show that the proposed models exhibit better goodness of fit.
In this paper, we have extended the work proposed in [28]. The authors presented a model to represent the OSS growth using the IMPs rate due to fixing of NFs and IMPs. The proposed models have been validated on five products, namely "Avro, Pig, Hive, jUDDI and Whirr" of Apache project.
The remaining part of the paper has been divided into 5 sections. A review of the available literature related to the proposed work in the paper has been presented in section 2. In section 3, the data collection and the mathematical formulation to embody the OSS development have been proposed. Experimental setup has been presented in section 4. The numerical illustrations to validate the proposed models presented in section 5. The paper has been concluded in sections 6.

RELATED WORK
Many current OSS quality models come from the standard scheme ISO 9126 [29,30]. Many different models of open source efficiency and maturity, and their comparative analysis is available in [31].
In a study [32] it was reported that a majority of OSS quality assurance models primarily concentrate on datadominant software evaluations. A study proposed a framework of process to address the challenges for the evaluation and selection of OSS [33]. The evolution of OSS is characterized by the number of issues that are reported or requests made for their enhancements. The major requests made by users are for the NFs and IMPs in addition to the bugs reports. Therefore, when to release open source software depends not only on the bugs fixed, but also on NFs and IMPs implementation. In OSS development paradigm software are released frequently without waiting to fix all the requested features [10]. A study has been proposed by incorporating uncertainties in the SRGM [14]. A model based on Dempster-Shafter theory and improved differential evolution has been proposed for allocation of reliability growth and predefined budget in multimedia systems [15]. An optimal release time problem by applying risk-reduction approach with the delay incurred cost has been proposed in [16]. Multicriteria based dynamic selection model has been used to improve a software reliability prediction model [17]. The modelling of the software correction process consists of bug detection and correction process. After a fault is found the bug correction process will be delayed for some time. By considering these factors, a model has been proposed and validated in [18].
A testing effort and multivariate function based fault prediction rate has been proposed for SRGM [19]. Recently, entropy based SRGMs were proposed to measure the software reliability growth and these models were further used to predict software release time [23,34]. In another studies [4,25] remaining software faults from previous release and current faults based multirelease SRGM has been proposed.
Here, a quantified approach for multi version software system has been proposed by considering the NFs and IMPs implementations and bugs generated from these implementations. The proposed model based on Non Homogenous Poisson Process(NHPP), which has been widely used in SRGM [7,8,9,20,27]. Our model is unique and has the novelty in the sense that it incorporates the rates at which NFs and IMPs are introduced and the bugs generated from the introduction of NFs and IMPs. Our proposed model is an improved model and quite different from the models proposed in [7,8,9,20,21,22,27].

DESCRIPTION OF DATASETS AND MODEL CONSTRUCTION
In this section, different datasets used to validate the proposed models followed by the proposed models have been described.

Description of Datasets
Apache open source products data has been used to validate the proposed models [5].
Tab. 1 shows the data collection time period of different Apache products considered for study. Tab. 2 shows the versions for which we have taken the cumulative data and renamed as different release numbers [23].
Once the different issues are reported by the users, these issues are fixed by modifying the code in different files of the software products. Due to these modifications in the code , the complexity of code changes increases and hence an uncertainty arises. This measure of uncertainty is called entropy. Based on information theory [6], the "complexity of code changes/Entropy" was firstly proposed by Hassan [2]. Shannon entropy [6] is defined as:

Developing Models For Multi Release Software Product
In OSS, once the issues are reported, the triaging takes place and different issues are assigned to developers. Once the issues are fixed, the source code modifies and in result of this new releases take place. But, there are some issues which are still left in the current release, which get fixed in the next release. A mathematical model is necessary here, which will predict the issues that can be fixed over a long period of time. It is important to note that all issues of the current release could not be fixed due to time constraint and can be remained unresolved and added to the next release.
Y(t) -cumulative number of fixed errors at any time t; ipotential errors; b -rate of error fixing per remaining error.
NHPP based time-dependent SRGMs have been proven to be successful models for quantitative quality evaluation of the software products [7,26].
Let N(t) denote the total number of issues detected at time t, and Y(t) denote its expectation. Then Y(t) = E[N(t)], and the failure rate λ(t) are related as follows: Here, i denotes the different issues, i.e. the sum of bugs, NFs and IMPs reported in the issue tracking system. These issues are fixed by active users. Let the rate at which new features are introduced be p and the rate at which feature improvements are incorporated for these new features introductions, be q. It has been assumed that fixing of these issues may result in bugs. Let r be the rate of bugs generated from the introduction of new features fixed out of total issues and s be the rate of bugs generated from feature improvements. This results in the following equation: The number of issues fixed at any time t is denoted by Y(t). In the above Eq. (5), the number i − Y(t) represents the left over issues at time t. p(i − Y(t)) is the number of new features introduced with rate p in the software.
is the number of feature improvements incorporated with rate q for the new features added with rate p.
is the number of bugs generated with rate r from the addition of new features added with rate p.
is the number of bugs generated with rate s from the feature improvements incorporated with rate q. Solving (5) at t = 0, Y(t) = 0 results in the following equation: For n th release, Eq. (6) can be written as n n n n n n n n n n n n n n n n n n n where Y n (t n ) is the issues fixed at any time t n in n th release. i n is the potential issue to be fixed over a long run in the n th release. p n and q n are the rates at which new features and improvements in features are introduced in the n th release of the software. r n and s n are the rates of bugs generations from the incorporation of these new features and improvements in features. The model given in (6) can also be written in the following form: where, G(t) follows a distribution function, at t = 0, G(0) = = 0 and at t = ∞, G(∞) = 1.
For n th release, Eq. (8) can be written as: n n n n n n n n n n n n n n n n n n The issues that can be fixed over the long run can be predicted by using the model given in Eq. (6).
In the following section, we have assimilated the code changes in order to fix different issues. Eentropy based model for issues prediction has been proposed by considering the code changes in the software.
In the line of proposed model given in (6), we have developed the following entropy i.e. H(t), based model for issues prediction. Y(H(t)) is the mean value function. This results in (10): A software is evolved with its multi-versions. For each version the issues are reported by open source community to meet their own requirements. It is not possible to fix all the reported issues in the current version of the software, therefore the remaining issues are carried over for the next release. The equation given in (6), has been used to evaluate the mean value of the issues.
In order to express mathematically the different releases of the software where a proportion of issues of a long run fixed in the current release and remaining issues passed on to the next release, we formulated the Eq. (11), Eq. (12), Eq. (13), and Eq. (14). The following models presented in Eq. (11), Eq. (12), and Eq. (13) are for Release 1, Release 2, Release 3 and Eq. (14) represents for the n th release of the software product [4,23,24]. 1 1 The number of issues which will be addressed over the future for the first release is i 1 at time t 1 . The quantity i 1 (1 -G 1 (t 1 )) is the leftover issues of Release 1 and it will be added to the Release 2 to be fixed at the rate G 2 (t -t 1 ). The following expression presents the issues for the Release 2.
( ) ( ) Here, the number of issues over long run to be fixed in Release 2 is i 2 .
The cumulative number of issues fixed for Release 3 by considering the leftover issues of just previous release is given by: Here, the number of issues over long run to be fixed in Release 3 is i 3 . Based on the approaches applied for the issues fixing modelling for Release 1, Release 2 and Release 3, the following mathematical equation for n th release can be written.
( ) ( ) Similarly, entropy based mathematical expressions for different releases can be derived. The following equation shows the entropy based cumulative modelling for n th release.

EXPERIMENTAL DESIGN
We have investigated the leftover issues of previous releases that will be remained unresolved and added to the issue content of the next releases. We formulated the following cases in our study (Tab. 3). The proposed models discussed above have validated for issues datasets of five Apache products [5].
The parameters i, b, p, q, r and s of the models have been estimated in Statistical Package for Social Sciences (SPSS) software using Nonlinear regression (NLR).

NUMERICAL ILLUSTRATION
For different cases given in Tab. 3, different issues estimation results have been observed across all the datasets. We have taken 5 products of Apache project for model validation and discuss the findings.
The results of only 3 products have been presented due to space limitation. We have predicted the number of issues contributing on the content of subsequent release for each case. Tab. 4 to Tab. 6 show parameter estimates and performance measures for issues of different products for different cases given in Tab. 3. In shows the real number of issues fixed in n th release of the software product. i n is the potential number of issues that need to be fixed in n th release and (i n − I n ) is the left over issues of that release.  From Tab. 4, the estimated issues (potential issues) is 364 for case 3 given in Tab. 3 for Release 1. The real number of issues that have been fixed for Release 1 is 332. This implies that 364 -332 = 32 issues are unresolved and added to Release 2 issue content. The estimated number of issues in Release 2 is (i 2 + i 1 (1 − G 1 (t 1 ))) = 191. The real number of issues that have been fixed for Release 2 is 183. This implies that 191 -183 = 8 issues are unresolved and added to Release 3 issue content. Similarly, 24 issues added to Release 4 from Release 3. In Release 4, 68 issues remained unresolved, and added to the intial issue content of Release 5.
We observed from the results of the proposed entropy based model (case 4 of Tab. 3) that 332 issues have been fixed in Release 1 and the estimated value of the issues is 363. This shows that 363 -332 = 31 issues are still to be fixed in Release 1 but, they are now added to Release 2 issue content. In Release 2, 183 issues have been fixed and the estimated value for issues is 194 which shows that 11 issues of Release 2 will be now fixed in Release 3. 142 issues have been fixed in Release 4 and the predicted issue is 212, it means 70 issues need to be fixed in Release 4 but they are now added to Release 5. Similar inference can be drawn for GO and S-shaped models.
In the proposed models (case 3 and case 4 of Tab. 3), to analyze the rate at which bugs are generated due to NFs and IMPs incorporated, we have taken the average values of p, q, r and s across different releases of a product.
In Avro product, the rates at which new features and feature improvements are incorporated are 0.03 and 0.26. Bugs are generated at the rate of 0.32 and 0.39 due to the addition of NFs and incorporation of IMPs. This shows that the rate of bugs generated due to feature improvements is greater than the rate of bugs generated due to new features incorporation. We get maximum R 2 value 0.995 (in case 3 and case 4); 0.99 (in case 1, case 3 and case 4); 0.98 (in case 1, case 3 and case 4); 0.99 (in case 4) and 0.99 (in case 1, case 3 and case 4) for different releases respectively. In Pig product, the rates at which new features and feature improvements incorporated are 0.06 and 0.11. Bugs are generated at the rate of 0.32 and 0.51 due to the addition of NFs and incorporation of IMPs. This shows that the rate of bugs generated due to feature improvements is greater than the rate of bugs generated due to new features incorporation. In Hive product, the rates at which NFs and IMPs are incorporated in the software are 0.04 and 0.16. Bugs are generated at the rate of 0.76 and 0.04 due to addition of NFs and incorporation of IMPs. This shows that the rate of bugs generated due to new features addition is greater than the rate of bugs generated due to feature improvements.
In case of jUDDI, we get maximum R 2 value, i.e. 0.97 (in case 3 and case 4), 0.95 (in case 2), 0.87 (in case 4) and 0.96 (in case 3 and case 4) for different releases respectively.
In jUDDI product, the rates at which NFs and IMPs are incorporated are 0.11 and 0.12. Bugs are generated at the rate of 0.33 and 0.24 due to the addition of NFs and incorporation of IMPs. This shows that the rate of bugs generated due to new features addition is greater than the rate of bugs generated due to feature improvements.
In case of Whirr product, we get maximum R 2 value, i.e. 0.99 (in case 3 and case 4), 0.96 (in case 3 and case 4), 0.81(in case 4) and 0.76 (in case 3 and case 4) for different releases respectively.
In Whirr product, the rates at which new features and feature improvements are incorporated are 0.08 and 0.11. Bugs are generated at the rate of 0.29 and 0.32 due to the addition of NFs and incorporation of IMPs. This shows that the rate of bugs generated due to feature improvements is greater than the rate of bugs generated due to new features incorporation.
The proposed models give better performance in terms of R 2 for all the cases described in Tab. 3. For all the releases of every product we have taken maximum R 2 across all the four cases defined in Tab. 3. Results show that we have 21 cases of maximum R 2 for case 4 (proposed model), 18 cases of maximum R 2 for case 3 (proposed model), 5 cases of maximum R 2 for case 2 and 4 cases of maximum R 2 for case 1 out of total 48 cases of maximum R 2 across all the products and releases. During our experiment, it has been observed that proposed models based on total issues fixed mentioned in case 4 defined in Tab. 3, give the highest cases of maximum goodness of fit.

Applications of the Proposed Models
The reliability of a software product is an important performance quality attribute. To meet the enormous and frequent requirements, a software is released frequently. The frequent releases must also meet a predefined reliability level. The model predicts the rate at which NFs and IMPs are introduced in the software and also the bugs generated from these introductions. This information will also help in understanding the growth pattern of the software. Based on the proposed models optimal release time can be determined which will assist the release managers in taking release related decisions based on the number of issues fixed.

CONCLUSION
We have developed the first mathematical model for issues prediction based on OSS development paradigms: NFs introduction, IMPs and bugs generated from these additions. We have also extended the proposed model by considering the entropy using information theory based measures. We estimated the potential (over a long run) value of different issues in multiple releases of Avro, Pig, Hive, jUDDI and Whirr products of the Apache project. During experimentation, it has been observed that a proportion of issues which is unresolved in the current release are included to the content of future release. Results show that we have 21 cases of maximum R 2 for proposed entropy based model, 18 cases of maximum R 2 for the proposed time based model, 5 cases of maximum R 2 for Sshaped model and 4 cases of maximum R 2 for GO model out of total 48 cases of maximum R 2 across all the products and releases. Entropy based model proposed in this paper gives maximum cases of maximum R 2 .
The proposed research work will help managers to evaluate the progress in the software products' reliability. In the future, the proposed work will be extended by using Bio inspired algorithms.