Extending Coverage Criteria by Evaluating Their Robustness to Code Structure Changes - Testing Software and Systems Access content directly
Conference Papers Year : 2012

Extending Coverage Criteria by Evaluating Their Robustness to Code Structure Changes

Abstract

Code coverage is usually used as a measurement of testing quality and as adequacy criterion. Unfortunately, code coverage is very sensitive to modifications of the code structure, and, therefore, the same test suite can achieve different degrees of coverage on the same program written in two syntactically different ways. For this reason, code coverage can provide the tester with misleading information.In order to understand how a testing criterion is affected by code structure modifications, we introduce a way to measure the sensitivity of coverage to code changes. We formalize the modifications of the code structure using semantic preserving code-to-code transformations and we propose a framework to evaluate coverage robustness to these transformations, extending actual existing coverage criteria.This allows us to define which programs and which test suites can be considered robust with respect to a certain set of transformations. We can identify when the obtained coverage is fragile and we extend the concept of coverage criterion by introducing an index that measures the fragility of the coverage of a given test suite. We show how to compute the fragility index and we evidence that also well-written industrial code and realistic test suites can be fragile. Moreover, we suggest how to deal with this kind of testing fragility.
Fichier principal
Vignette du fichier
978-3-642-34691-0_13_Chapter.pdf (383.24 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01482400 , version 1 (03-03-2017)

Licence

Attribution

Identifiers

Cite

Angelo Gargantini, Marco Guarnieri, Eros Magri. Extending Coverage Criteria by Evaluating Their Robustness to Code Structure Changes. 24th International Conference on Testing Software and Systems (ICTSS), Nov 2012, Aalborg, Denmark. pp.168-183, ⟨10.1007/978-3-642-34691-0_13⟩. ⟨hal-01482400⟩
58 View
66 Download

Altmetric

Share

Gmail Facebook X LinkedIn More