Matthieu Foucault, Marc Palyart, Jean-Rémy Falleri and Xavier Blanc
Software metrics have been developed to measure the quality of software systems. A proper use of metrics requires thresholds to determine whether the value of a metric is acceptable or not. Many approaches propose to define thresholds based on large analyses of software systems. However it has been shown that thresholds depend greatly on the context of the project.
Thus there is a need for an approach that computes thresholds by taking into account this context. In this paper we propose such approach with the objective to reach a trade-off between representativeness of the threshold and computation cost. Our approach is based on an unbiased selection of software entities and makes no assumptions on the statistical properties of the software metrics values. It can therefore be used by anyone, ranging from developer to manager, for computing a representative metric threshold tailored to their context.
Due to space limitation, some figures and tables are not included in the version of the paper available in the SAC proceedings (they are available in the preprint version though) :
Root Mean Square Errors for estimated 90% thresholds of the NOA+NOM metric| Number of Classes | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Number of Repositories | 200 | 400 | 600 | 800 | 1000 | 1200 | 1400 | 1600 | 1800 | 2000 | |
| 3 | N/A | N/A | N/A | N/A | N/A | 2.09 | N/A | N/A | N/A | N/A | |
| 5 | 3.57 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 8 | N/A | 3.57 | N/A | N/A | 2.09 | N/A | N/A | N/A | N/A | N/A | |
| 13 | 3.57 | 5.47 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 21 | 7,63 | 5.33 | 6.40 | N/A | N/A | N/A | 12.24 | 2.08 | N/A | N/A | |
| 34 | 2.08 | 3.12 | 3.89 | 2.83 | 6.40 | N/A | 6.40 | 3.62 | N/A | N/A | |
| 55 | 4.96 | 3.57 | 2.32 | 5.33 | 2.05 | N/A | 2.32 | 4.96 | N/A | 3.57 | |
| 89 | 3.89 | 7.63 | 3.57 | 6.40 | 2.83 | 5.33 | 2.83 | 4.33 | 2.45 | 2.09 | |
| 144 | 2.08 | 3.38 | 2.84 | 6.40 | 2.84 | 2.08 | 3.12 | 2.09 | 2.32 | 2.45 | |
| 233 | 2.08 | 2.08 | 2.05 | 2.08 | 2.45 | 2.08 | 2.04 | 2.45 | 2.08 | 2.45 | |
| 377 | 3.38 | 3.38 | 6.40 | 2.45 | 4.12 | 2.56 | 2.56 | 2.08 | 2.45 | 2.83 | |
| Number of Classes | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Number of Repositories | 200 | 400 | 600 | 800 | 1000 | 1200 | 1400 | 1600 | 1800 | 2000 |
| 3 | N/A | N/A | N/A | N/A | N/A | 3.23 | N/A | N/A | N/A | N/A |
| 5 | 3.23 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 8 | N/A | 4.79 | N/A | N/A | 3.30 | N/A | N/A | N/A | N/A | N/A |
| 13 | 7.26 | 5.74 | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 21 | 10.55 | 4.42 | 10.55 | N/A | N/A | N/A | 8.85 | 3.92 | N/A | N/A |
| 34 | 2.75 | 2.94 | 2.92 | 5.92 | 5.92 | N/A | 4.79 | 4.71 | N/A | N/A |
| 55 | 5.73 | 2.72 | 4.79 | 3.92 | 7.26 | N/A | 8.85 | 4.71 | N/A | 5.91 |
| 89 | 5.37 | 3.31 | 3.92 | 3.31 | 7.26 | 3.92 | 2.75 | 4.79 | 3.87 | 2.94 |
| 144 | 3.51 | 3.51 | 2.92 | 8.85 | 3.23 | 3.31 | 4.17 | 3.23 | 3.87 | 2.91 |
| 233 | 3.23 | 2.72 | 3.87 | 2.72 | 2.94 | 2.72 | 2.92 | 4.79 | 3.92 | 3.31 |
| 377 | 3.92 | 3.23 | 2.92 | 2.75 | 4.71 | 3.51 | 3.87 | 2.72 | 2.94 | 2.72 |
The sampling and threshold computation process described in this paper has been implemented using the harmony framework. Harmony is a framework which assists researchers when mining software repositories. Harmony can be installed as a set of Java plugins. Its source code, and information about how to install harmony (such as screencasts) are available here. To be sure to install a version of harmony which is compatible with the implementation of the process, we recommand that you use this url for the update site when installing the eclipse plugins : http://se.labri.fr/data/harmony/update-site/archive/012013
When harmony is installed, the next step is to download the source code of the process : download here. Then follow these steps: