Testing Smart Contracts

Abstract

Background: Executing, verifying and enforcing credible transactions on permissionless blockchains is done using smart contracts.A key challenge with smart contracts is ensuring their correctness and security. Several test input generation techniques for detecting vulnerabilities in smart contracts have been proposed in the last few years. However, a comparison of proposed techniques to gauge their effectiveness is missing. Aim: This paper conducts an empirical evaluation of testing techniques for smart contracts. The testing techniques we evaluated are: (1) Blackbox fuzzing, (2) Adaptive fuzzing, (3) Coverage-guided fuzzing with an SMT solver and (4) Genetic algorithm. We do not consider static analysis tools, as several recent studies have assessed and compared effectiveness of these tools. Method: We evaluate effectiveness of the test generation techniques using (1) Coverage achieved -we use four code coverage metrics targeting smart contracts, (2) Fault finding ability -using artificially seeded and real security vulnerabilities of different types. We used two datasets in our evaluation -one with 1665 real smart contracts from Etherscan, and another with 90 real contracts with known vulnerabilities to assess fault finding ability. Result: We find Adaptive fuzzing performs best in terms of coverage and fault finding over contracts in both datasets. Conclusion: However, we believe considering dependencies between functions and handling Solidity specific features will help improve the performance of all techniques considerably. CCS CONCEPTS• Security and privacy → Software security engineering.

Blockchain-based key-value store to support dynamic smart contract interaction in the agricultural sector

“…Mocha is renowned for its flexibility and comprehensive reporting, which are crucial for debugging and validating complex smart contract interactions [53]. Aligning with Ethereum.org's standards, the study ensured that the testing procedures adhered to industry best practices, thereby enhancing the credibility and security of the smart contract [54]. This alignment was particularly important for validating the security aspects of the contract, as Ethereum.org's standards are specifically tailored to address the unique challenges and threats in the blockchain environment.…”

Functional suitability assessment of smart contracts: A survey and first proposal

“…Akca et al 75 Coverage tests help create a complete set of tests and consequently produce a measure of completeness. The relationship between coverage and completeness is very important because it follows a very high correlation between them.…”

“…Akca et al 75 empirically investigated the different techniques available for smart contract testing, including black‐box fuzzing, adaptive fuzzing, coverage‐guided fuzzing, and genetic algorithms. By leveraging two datasets of 1665 and 90 real smart contracts extracted from Etherscan, they compared the different techniques in terms of reached coverage and ability to find faults.…”

“…In the literature, four types have been defined for coverage: (i) branch coverage, (ii) opcode coverage, (iii) call coverage, and (iv) event coverage 75 . These four types of coverage can be computed through the CovCal framework, 75 which operates as an extension of the EVM. An alternative tool to measure the coverage of the solidity source code is solidity‐coverage §§ .…”

Systematic Mapping of Testing Smart Contracts for Blockchain Applications

“…In the papers [17], [19], [29], [30], [31], [34], [45], and [61], a newer method called mutation testing was introduced for the specific purpose of testing smart contracts. In addition, in the papers [20], [21], [23], [24], [26], [41], [42], [48], [53], [54], [55], and [56], another method called fuzz testing was introduced as well.…”