Replication Package for: "Analyzing APIs Documentation and Code to Detect Directive Defects"


Application Programming Interface (API) documents represent one of the most important references for API users. However, it is frequently reported that the documentation is inconsistent with the source code and deviates from the API itself. Such inconsistencies in the  documents inevitably confuse the API users hampering considerably their API comprehension and the quality of software built from such APIs. In this paper, we propose an automated approach to detect defects of API documents by leveraging techniques from program comprehension and natural language processing. Particularly, we focus on the directives of the API documents which are related to parameter constraints and exception throwing declarations. A first-order logic  based constraint solver is employed to detect such defects based on the obtained analysis results. We evaluate our approach on parts of well documented JDK 1.8 APIs. Experiment results show that, out of around 2000 API usage constraints, our approach can detect 1158 defective document directives, with a precision rate of 81.6%, and a recall rate of 82.0%,  which demonstrates its practical feasibility.



This page provides the replication package with (i) the material and working data sets of our study (e.g., including input data, and output defective reports); (ii) NLP patterns and the defined NLP heuristics. 

Download Replication Package


Pre-processing Rules


Yu Zhou

 College of Computer Science, NUAA, 

Nanjing, China 

Ruihang Gu

 College of Computer Science, NUAA, 

Nanjing, China 

Taolue Chen

 Dept. of computer science, Middlesex University, London 

Nanjing, China 

Zhiqiu Huang

 College of Computer Science, NUAA, 

Nanjing, China 

Sebastiano Panichella

 University of Zurich, Switzerland

Harald Gall University of Zurich, Switzerland