Test oracles based on metamorphic relations for image processing applications

Jameel, Tahir; Lin, Min; Liu, Chao

doi:10.1109/snpd.2015.7176238

Cited by 16 publications

(21 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Mayer and Guderlei [19] NA + NA + − − − Just and Schweiggert [21] NA + NA + − − − Kuo et al [22] NA + NA + − − − Jameel et al [149] NA + NA + − − − Chan et al [150], [151] NA + NA + − − − Chen et al [152] NA + NA + − − − Ding et al [153] NA + NA + − − − Murphy et al […”

Section: Metamorphicunclassified

Metamorphic Testing for Web System Security

Chaleshtari¹,

Pastore

Göknil

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Security testing aims at verifying that the software meets its security properties. In modern Web systems, however, this often entails the verification of the outputs generated when exercising the system with a very large set of inputs. Full automation is thus required to lower costs and increase the effectiveness of security testing. Unfortunately, to achieve such automation, in addition to strategies for automatically deriving test inputs, we need to address the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior (e.g., the response to be received after a specific HTTP GET request). In this paper, we propose Metamorphic Security Testing for Web-interactions (MST-wi), a metamorphic testing approach that integrates test input generation strategies inspired by mutational fuzzing and alleviates the oracle problem in security testing. It enables engineers to specify metamorphic relations (MRs) that capture many security properties of Web systems. To facilitate the specification of such MRs, we provide a domain-specific language accompanied by an Eclipse editor. MST-wi automatically collects the input data and transforms the MRs into executable Java code to automatically perform security testing. It automatically tests Web systems to detect vulnerabilities based on the relations and collected data. We provide a catalog of 76 system-agnostic MRs to automate security testing in Web systems. It covers 39% of the OWASP security testing activities not automated by state-of-the-art techniques; further, our MRs can automatically discover 102 different types of vulnerabilities, which correspond to 45% of the vulnerabilities due to violations of security design principles according to the MITRE CWE database. We also define guidelines that enable test engineers to improve the testability of the system under test with respect to our approach. We evaluated MST-wi effectiveness and scalability with two well-known Web systems (i.e., Jenkins and Joomla). It automatically detected 85% of their vulnerabilities and showed a high specificity (99.81% of the generated inputs do not lead to a false positive); our findings include a new security vulnerability detected in Jenkins. Finally, our results demonstrate that the approach scale, thus enabling automated security testing overnight.

show abstract

Section: Metamorphicunclassified

Metamorphic Testing for Web System Security

Chaleshtari¹,

Pastore

Göknil

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…The idea of metamorphic testing was proposed in [3] for the first time, and since that time used successfully in several various sectors, for instance, in the development of web applications [5,6,7], compilers [8,9], computer graphics [10,11] and bioinformatics [12,13] applications, so on.…”

Section: Related Workmentioning

confidence: 99%

Metamorphic Testing and Serverless Computing: A Basic Architecture

Yusyn

Заболотня

2022

IJCAI

View full text Add to dashboard Cite

Automated testing of complex software systems can be a challenging task, and today there are a large number of methods for its implementation. One such method is metamorphic testing, which effectively solves the problems of usual methods and is gaining popularity. However, performing metamorphic tests can take a long time, so the question arises of their distributed running, including in the cloud. Thus, the authors of this study considered the designing of a cloud serverless architecture of software for metamorphic testing. The serverless architecture for metamorphic testing is proposed, which is based on the composition of the entire system from 5 individual components: models, data generator, software artifact under test, metamorphic relations, and serverless functions. For each of the main possible types of software artifacts, the possibility of using the serverless architecture for metamorphic testing is considered. The developed architecture is presented in the form of component, deployment, and sequence diagrams. The use of the proposed architecture in practice is shown by the example of testing two software artifactsa class library and a web application. Performance measurements have shown that despite the additional network delay when running one test, the performance of all tests in general in the case of the serverless architecture is closer to local startup and will be faster with increasing complexity and number of tests. Povzetek: Predstavljena je arhitektura brez strežnika za metamorfno testiranje zapletene programske opreme.

show abstract

“…These four MRs (rotation at 90 degrees, transposition, reflection at ordinate, and reflection at abscissa) are generally applicable to all image processing operations. Furthermore, Jameel et al [14] furthered the research by using two of these four to ascertain the fault detection rate of dilation and erosion MRs. In total, these authors have presented eight MRs (two general and six specific) for dilation and erosion operation.…”

Section: Introductionmentioning

confidence: 99%

“…In total, these authors have presented eight MRs (two general and six specific) for dilation and erosion operation. Jameel et al [14] used only two general MRs, i.e., reflection at ordinate and reflection at abscissa. However, the fault detection rate of the remaining two MRs (rotation and transposition) is not determined.…”

Section: Introductionmentioning

confidence: 99%

“…Afterward, mutation testing is performed to evaluate MRs. Mutation operators always play an important role in generating the mutants. In existing literature [12,14,16,17], only a few mutation operators are used that have generated a very small number of mutants. The authors did not discuss the effectiveness of mutation operators or which operator is effective enough to generate and kill a maximum number of mutants.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing

Jafari,

Nadeem

2024

J. Imaging

View full text Add to dashboard Cite

Testing an intricate plexus of advanced software system architecture is quite challenging due to the absence of test oracle. Metamorphic testing is a popular technique to alleviate the test oracle problem. The effectiveness of metamorphic testing is dependent on metamorphic relations (MRs). MRs represent the essential properties of the system under test and are evaluated by their fault detection rates. The existing techniques for the evaluation of MRs are not comprehensive, as very few mutation operators are used to generate very few mutants. In this research, we have proposed six new MRs for dilation and erosion operations. The fault detection rate of six newly proposed MRs is determined using mutation testing. We have used eight applicable mutation operators and determined their effectiveness. By using these applicable operators, we have ensured that all the possible numbers of mutants are generated, which shows that all the faults in the system under test are fully identified. Results of the evaluation of four MRs for edge detection show an improvement in all the respective MRs, especially in MR1 and MR4, with a fault detection rate of 76.54% and 69.13%, respectively, which is 32% and 24% higher than the existing technique. The fault detection rate of MR2 and MR3 is also improved by 1%. Similarly, results of dilation and erosion show that out of 8 MRs, the fault detection rates of four MRs are higher than the existing technique. In the proposed technique, MR1 is improved by 39%, MR4 is improved by 0.5%, MR6 is improved by 17%, and MR8 is improved by 29%. We have also compared the results of our proposed MRs with the existing MRs of dilation and erosion operations. Results show that the proposed MRs complement the existing MRs effectively as the new MRs can find those faults that are not identified by the existing MRs.

show abstract

Test oracles based on metamorphic relations for image processing applications

Cited by 16 publications

References 11 publications

Metamorphic Testing for Web System Security

Metamorphic Testing for Web System Security

Metamorphic Testing and Serverless Computing: A Basic Architecture

Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing

Contact Info

Product

Resources

About