ChainSecurity is happy to release PolPatrol, an automated validator for testing the stability and security of Polkadot runtimes with respect to generic security and performance properties. Since Polkadot’s relay chain runtime lies at the core of the Polkadot network, the current version of PolPatrol focuses on ensuring that relay chain runtimes are secure and functionally correct.
Peter Czaban, Executive Director of Web3 Foundation, said:
“Polkadot Runtimes make it easier than ever before to flexibly describe and deploy blockchain logic. To ensure that this richness of capabilities leads to secure developments, we are committed to funding tools which can support developers during their creation. We are excited to work with a leading blockchain security firm ChainSecurity to research and develop new approaches to automated blockchain logic analysis”.
Petar Tsankov, Chief Scientist of ChainSecurity, said:
“We are dedicated to bringing to Polkadot’s developers and users practical and useful security tools that enable them to easily assess the safety and security of Polkadot. Starting with manual testing and random fuzzing of Polkadot Runtimes, we are looking forward to building further tools capable of deep security fuzzing and verification. We are grateful to Web3 Foundation for their support in this project”.
Who is PolPatrol for?
Two user groups will primarily use PolPatrol:
- Developers of new relay chain runtimes who would like to test the correctness of their code.
- DOT token holders who can vote on newly proposed relay chain runtimes and would like to check whether a proposed runtime is safe and secure.
For both user groups, PolPatrol automatically analyzes any given relay chain Web assembly binary (or, a Wasm blob) and warns users upon violation of important security and performance properties.
How can I use PolPatrol?
The source code of PolPatrol is available at https://github.com/chainsecurity/polpatrol.
To learn how to install PolPatrol and how to run it on a custom relay chain runtime, please refer to the official documentation at https://polpatrol.ch.
How does it work?
PolPatrol uses an instrumented Polkadot runtime environment to run the provided relay chain runtime and log all calls that it makes to the environment. In addition to the calls, it also monitors essential performance metrics, such as execution time and memory usage. This enables PolPatrol to check relevant trace and performance properties. PolPatrol aggregates the collected runtime information to let developers draw conclusions on the overall behavior of the runtime and compare different implementations.
What properties does PolPatrol check?
The focus of PolPatrol is on performance and safety properties.
Performance of runtime entries
PolPatrol collects statistics about all calls to runtime entries and reports the total number of calls made while testing the run-time, the maximum execution time (in seconds), the maximum memory usage (in MB), and the maximum number of storage-related calls to the environment:
==================================================================== Entry Name | # Calls | Max Time | Max Mem | Max #Storage -------------------------------------------------------------------- initialize_block | 31 | 0.0324 | 68.294 | 59 execute_block | 31 | 0.9438 | 72.484 | 341 inherent_extrinsics | 31 | 0.0404 | 68.290 | 1 apply_extrinsic | 109 | 0.6399 | 71.959 | 79 finalize_block | 31 | 0.3210 | 69.862 | 110 ==================================================================== |
For example, the above sample output identifies that the initialize_block entry was called 31 times, each call took at most 0.0324 seconds and consumed at most 68.294 MB of memory, making at most 59 storage-related calls. We note that PolPatrol outputs additional metrics (not shown above), such as the mean execution time and mean memory usage.
Performance of extrinsics
PolPatrol also reports statistics about applied extrinsics, reporting the same metrics as the ones reported for environment entries:
==================================================================== Entry Name Max Time | Max Mem | Max Storage| Weight -------------------------------------------------------------------- Balances::force_transf 0.031489 | 68.296 | 57 | 1000000 Balances::transfer 0.031372 | 68.296 | 66 | 1000000 Democracy::propose 0.028740 | 68.290 | 39 | 5000000 Council::execute 0.029009 | 68.290 | 47 | 100000 Slots::fix_deploy_data 0.028606 | 68.290 | 38 | 500000 Democracy::fast_track 0.028877 | 68.290 | 39 | 200000 ImOnline::heartbeat 0.026807 | 68.289 | 13 | 10000 Democracy::undelegate 0.029554 | 68.289 | 54 | 500000 Balances::set_balance 0.029181 | 68.290 | 102 | 0 Democracy::resign_proxy 0.028444 | 68.289 | 40 | 100000 Council::vote 0.030299 | 68.290 | 107 | 200000 Parachains::set_heads 0.028418 | 68.289 | 13 | 1000000 Democracy::set_proxy 0.028941 | 68.290 | 41 | 100000 Democracy::cancel_queu 0.028394 | 68.289 | 39 | 10000 Council::propose 0.029840 | 68.290 | 95 | 5000000 Democracy::external_pro 0.028710 | 68.290 | 39 | 5000000 Claims::claim 0.027391 | 68.289 | 11 | 1000000 Slots::new_auction 0.028643 | 68.290 | 56 | 0 Democracy::second 0.028793 | 68.289 | 40 | 5000000 Democracy::delegate 0.029207 | 68.290 | 52 | 500000 Democracy::vote 0.028790 | 68.289 | 40 | 200000 FinalityTracker::final 0.026426 | 68.289 | 13 | 10000 Democracy::cancel_refe 0.028563 | 68.289 | 39 | 10000 Democracy::remove_proxy 0.028664 | 68.290 | 41 | 100000 Council::set_members 0.029432 | 68.290 | 51 | 0 Democracy::emergency 0.028735 | 68.289 | 39 | 500000 Slots::elaborate_deploy 0.659817 | 71.959 | 51 | 5000000 Timestamp::set 0.028862 | 68.289 | 29 | 10000 Democracy::proxy_vote 0.028574 | 68.289 | 41 | 200000 Democracy::veto_external 0.028529 | 68.290 | 39 | 200000 Slots::bid 0.029268 | 68.289 | 57 | 500000 Democracy::external_prop 0.028528 | 68.290 | 39 | 5000000 ==================================================================== |
In addition to reporting the execution time, memory usage, and storage calls, PolPatrol also reports the assigned weight of each extrinsic. This allows developers and users to judge whether the weights are adequately defined based on the actual performance of the extrinsic.
Safety properties
Based on the performance information, PolPatrol checks several safety properties, such as:
==================================================================== Maximum block execution time: 0.944 s - OK Maximum block memory: 72.484 MB - OK Maximum block length: 0.375 MB - OK ==================================================================== |
In particular, PolPatrol ensures that the maximum block execution time is below 2 seconds, the maximum block memory is below 10,000 MB and the block length does not exceed 10,000 MB (measured as bytes of a block, scale-encoded). Further safety properties can be easily added.
Acknowledgments
ChainSecurity would like to thank the Web3 Foundation for supporting the research and development of PolPatrol with a research grant.
* * *
About Polkadot
Polkadot is the first interoperability protocol that enables blockchain networks to work together under the protection of shared security.
For more information on Polkadot, visit polkadot.network.
About Web3 Foundation
Established in Zug, Switzerland by Ethereum co-founder and Chief Technology Officer Dr. Gavin Wood, Web3 Foundation funds research and development teams building the stack of technologies behind the decentralized web.
For more information on Web3 Foundation, visit web3.foundation.
About ChainSecurity
ChainSecurity provides security services and develops custom security solutions for blockchain projects.
Learn more about ChainSecurity at chainsecurity.com.