Additional Data

Due to space limitation, some figures and tables are not included in the version of the paper available in the SAC proceedings (they are available in the preprint version though) :

Root Mean Square Errors for estimated 90% thresholds of the NOA+NOM metric
Number of Classes
Number of Repositories 200 400 600 800 100012001400160018002000
3 N/A N/A N/A N/A N/A 2.09 N/A N/A N/A N/A
5 3.57 N/A N/A N/A N/A N/A N/A N/A N/A N/A
8 N/A 3.57 N/A N/A 2.09 N/A N/A N/A N/A N/A
13 3.57 5.47 N/A N/A N/A N/A N/A N/A N/A N/A
21 7,63 5.33 6.40 N/A N/A N/A 12.24 2.08 N/A N/A
34 2.08 3.12 3.89 2.83 6.40 N/A 6.40 3.62 N/A N/A
55 4.96 3.57 2.32 5.33 2.05 N/A 2.32 4.96 N/A 3.57
89 3.89 7.63 3.57 6.40 2.83 5.33 2.83 4.33 2.45 2.09
144 2.08 3.38 2.84 6.40 2.84 2.08 3.12 2.09 2.32 2.45
233 2.08 2.08 2.05 2.08 2.45 2.08 2.04 2.45 2.08 2.45
377 3.38 3.38 6.40 2.45 4.12 2.56 2.56 2.08 2.45 2.83
Root Mean Square Errors for estimated 90% thresholds of the CBO metric
Number of Classes
Number of Repositories 200 400 600 800 100012001400160018002000
3 N/A N/A N/A N/A N/A 3.23 N/A N/A N/A N/A
5 3.23 N/A N/A N/A N/A N/A N/A N/A N/A N/A
8 N/A 4.79 N/A N/A 3.30 N/A N/A N/A N/A N/A
13 7.26 5.74 N/A N/A N/A N/A N/A N/A N/A N/A
21 10.55 4.42 10.55 N/A N/A N/A 8.85 3.92 N/A N/A
34 2.75 2.94 2.92 5.92 5.92 N/A 4.79 4.71 N/A N/A
55 5.73 2.72 4.79 3.92 7.26 N/A 8.85 4.71 N/A 5.91
89 5.37 3.31 3.92 3.31 7.26 3.92 2.75 4.79 3.87 2.94
144 3.51 3.51 2.92 8.85 3.23 3.31 4.17 3.23 3.87 2.91
233 3.23 2.72 3.87 2.72 2.94 2.72 2.92 4.79 3.92 3.31
377 3.92 3.23 2.92 2.75 4.71 3.51 3.87 2.72 2.94 2.72

Process Implementation

Prerequisites

Installation process

The sampling and threshold computation process described in this paper has been implemented using the harmony framework. Harmony is a framework which assists researchers when mining software repositories. Harmony can be installed as a set of Java plugins. Its source code, and information about how to install harmony (such as screencasts) are available here. To be sure to install a version of harmony which is compatible with the implementation of the process, we recommand that you use this url for the update site when installing the eclipse plugins : http://se.labri.fr/data/harmony/update-site/archive/012013

When harmony is installed, the next step is to download the source code of the process : download here. Then follow these steps:

  1. The sampler chooses random projects on github. To do so, we use a list of more than two million github projects, stored in a MariaDB/MySQL database. You need to import this list of projects in your local server, in a database named "github". The MySQL dump is available here
  2. Open Eclipse
  3. Import the projects in your workspace.
  4. Look at the configuration files
  5. Open the "run configurations" dialog in Eclipse, and click on the "add required bundles" button
  6. Run the configuration
  7. To run only the sampling part, type sampler in the eclipse console, to run the sampling part plus the harmony analysis, type "samplerHarmony"