Variable Strength T-Way Test Suite Generator with Constraints Support

Abstract

T-way testing (or interaction testing) is a common test planning method used to sample a complete or exhaustive test suite systematically.In t-way testing, it is assumed that interaction only occurs between t numbers of parameters (where t is the interaction strength). Therefore, all t-way strategies generate the t-way test suite with the intention to cover every possible combination produces by the interacting parameters (or also known as tuples). However, for some systems under test(SUT), there are some combinations that are known to produce invalid outputs or eventrigger unwanted errors. Additionally, there are also some known combinations that are impossible to occur due to requirements set to the system. As such, these combinations (termed constraints) have to be excluded from the final test suite. Whilemany t-way strategies have been proposed in literature for the past 20 years (e.g. GTWay, MIPOG, TConfig and TCG), only IPOG and PICT strategies have been known to support constraints in variable strength test suite generation. However, as t-way test suite generation process is an NP-hard problem, no single strategy can claim dominance over the others. Motivated by the challenges, this paper proposes a new strategy named General Variable Strength with Constraints (GVS_CONST) that support variable strength interaction with constraints consideration. Empirical evidence demonstrates that in most cases GVS_CONST outperforms other competing strategies in term of test suite size.