An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites - Fundamentals of Software Engineering Access content directly
Conference Papers Year : 2019

An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites

Samad Paydar
  • Function : Author
  • PersonId : 1030382
Aidin Azamnouri
  • Function : Author
  • PersonId : 1161215

Abstract

Randoop is a well-known tool that proposes a feedback-directed algorithm for automatic and random generation of unit tests for a given Java class. It automatically generates two test suites for the class under test: (1) an error-revealing test suite, and (2) a regression test suite. Despite successful experiences with applying Randoop on real world projects like Java Development Kit (JDK) which have led to creation of error-revealing tests and identification of real bugs, it has not been investigated in the literature how useful are the regression test suites generated by Randoop. In this paper, we have investigated flakiness and fragility of Randoop’s regression tests during evolution of 5 open source Java projects with a total of 78 versions. The results demonstrate that the flakiness of the regression tests is not generally noticeable, since in our dataset, only 5% of the classes have at least one flaky regression tests. In addition, test fragility analysis reveals that in most versions of the projects under study, the regression tests generated by Randoop could be successfully executed on many of later versions. Actually, for 2 out of 5 projects in the experiments, the regression tests generated for each version could be successfully executed on all the later versions of the project.
Fichier principal
Vignette du fichier
490001_1_En_8_Chapter.pdf (676.47 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03769117 , version 1 (05-09-2022)

Licence

Attribution

Identifiers

Cite

Samad Paydar, Aidin Azamnouri. An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites. 8th International Conference on Fundamentals of Software Engineering (FSEN), May 2019, Tehran, Iran. pp.111-126, ⟨10.1007/978-3-030-31517-7_8⟩. ⟨hal-03769117⟩
15 View
86 Download

Altmetric

Share

Gmail Facebook X LinkedIn More