PolPatrol – Validator for Polkadot Runtimes

Peter Czaban, Executive Director of Web3 Foundation, said:

“Polkadot Runtimes make it easier than ever before to flexibly describe and deploy blockchain logic. To ensure that this richness of capabilities leads to secure developments, we are committed to funding tools which can support developers during their creation. We are excited to work with a leading blockchain security firm ChainSecurity to research and develop new approaches to automated blockchain logic analysis”.

Petar Tsankov, Chief Scientist of ChainSecurity, said:

“We are dedicated to bringing to Polkadot’s developers and users practical and useful security tools that enable them to easily assess the safety and security of Polkadot. Starting with manual testing and random fuzzing of Polkadot Runtimes, we are looking forward to building further tools capable of deep security fuzzing and verification. We are grateful to Web3 Foundation for their support in this project”.

Who is PolPatrol for?

Two user groups will primarily use PolPatrol:

• Developers of new relay chain runtimes who would like to test the correctness of their code.
• DOT token holders who can vote on newly proposed relay chain runtimes and would like to check whether a proposed runtime is safe and secure.

For both user groups, PolPatrol automatically analyzes any given relay chain Web assembly binary (or, a Wasm blob) and warns users upon violation of important security and performance properties.

How can I use PolPatrol?

The source code of PolPatrol is available #here

To learn how to install PolPatrol and how to run it on a custom relay chain runtime, please refer to the official documentation #here

How does it work?

PolPatrol uses an instrumented Polkadot runtime environment to run the provided relay chain runtime and log all calls that it makes to the environment. In addition to the calls, it also monitors essential performance metrics, such as execution time and memory usage. This enables PolPatrol to check relevant trace and performance properties. PolPatrol aggregates the collected runtime information to let developers draw conclusions on the overall behavior of the runtime and compare different implementations.

What properties does PolPatrol check?

The focus of PolPatrol is on performance and safety properties.

Performance of runtime entries

PolPatrol collects statistics about all calls to runtime entries and reports the total number of calls made while testing the run-time, the maximum execution time (in seconds), the maximum memory usage (in MB), and the maximum number of storage-related calls to the environment:

====================================================================
Entry Name            | # Calls | Max Time | Max Mem | Max #Storage
--------------------------------------------------------------------
initialize_block      |      31 |   0.0324 |  68.294 |          59
execute_block         |      31 |   0.9438 |  72.484 |         341
inherent_extrinsics   |      31 |   0.0404 |  68.290 |           1
apply_extrinsic       |     109 |   0.6399 |  71.959 |          79
finalize_block        |      31 |   0.3210 |  69.862 |         110
====================================================================

For example, the above sample output identifies that the initialize_block entry was called 31 times, each call took at most 0.0324 seconds and consumed at most 68.294 MB of memory, making at most 59 storage-related calls. We note that PolPatrol outputs additional metrics (not shown above), such as the mean execution time and mean memory usage.

Performance of extrinsics

PolPatrol also reports statistics about applied extrinsics, reporting the same metrics as the ones reported for environment entries:

====================================================================
Entry Name               Max Time | Max Mem | Max Storage| Weight
--------------------------------------------------------------------
Balances::force_transf   0.031489 |  68.296 |         57 | 1000000
Balances::transfer       0.031372 |  68.296 |         66 | 1000000
Democracy::propose       0.028740 |  68.290 |         39 | 5000000
Council::execute         0.029009 |  68.290 |         47 |  100000
Slots::fix_deploy_data   0.028606 |  68.290 |         38 |  500000
Democracy::fast_track    0.028877 |  68.290 |         39 |  200000
ImOnline::heartbeat      0.026807 |  68.289 |         13 |   10000
Democracy::undelegate    0.029554 |  68.289 |         54 |  500000
Balances::set_balance    0.029181 |  68.290 |        102 |       0
Democracy::resign_proxy  0.028444 |  68.289 |         40 |  100000
Council::vote            0.030299 |  68.290 |        107 |  200000
Parachains::set_heads    0.028418 |  68.289 |         13 | 1000000
Democracy::set_proxy     0.028941 |  68.290 |         41 |  100000
Democracy::cancel_queu   0.028394 |  68.289 |         39 |   10000
Council::propose         0.029840 |  68.290 |         95 | 5000000
Democracy::external_pro  0.028710 |  68.290 |         39 | 5000000
Claims::claim            0.027391 |  68.289 |         11 | 1000000
Slots::new_auction       0.028643 |  68.290 |         56 |       0
Democracy::second        0.028793 |  68.289 |         40 | 5000000
Democracy::delegate      0.029207 |  68.290 |         52 |  500000
Democracy::vote          0.028790 |  68.289 |         40 |  200000
FinalityTracker::final   0.026426 |  68.289 |         13 |   10000
Democracy::cancel_refe   0.028563 |  68.289 |         39 |   10000
Democracy::remove_proxy  0.028664 |  68.290 |         41 |  100000
Council::set_members     0.029432 |  68.290 |         51 |       0
Democracy::emergency     0.028735 |  68.289 |         39 |  500000
Slots::elaborate_deploy  0.659817 |  71.959 |         51 | 5000000
Timestamp::set           0.028862 |  68.289 |         29 |   10000
Democracy::proxy_vote    0.028574 |  68.289 |         41 |  200000
Democracy::veto_external 0.028529 |  68.290 |         39 |  200000
Slots::bid               0.029268 |  68.289 |         57 |  500000
Democracy::external_prop 0.028528 |  68.290 |         39 | 5000000
====================================================================

In addition to reporting the execution time, memory usage, and storage calls, PolPatrol also reports the assigned weight of each extrinsic. This allows developers and users to judge whether the weights are adequately defined based on the actual performance of the extrinsic.

Safety properties

Based on the performance information, PolPatrol checks several safety properties, such as:

====================================================================
Maximum block execution time:     0.944 s  - OK
Maximum block memory:            72.484 MB - OK
Maximum block length:             0.375 MB - OK
====================================================================

In particular, PolPatrol ensures that the maximum block execution time is below 2 seconds, the maximum block memory is below 10,000 MB and the block length does not exceed 10,000 MB (measured as bytes of a block, scale-encoded). Further safety properties can be easily added.

Acknowledgments

ChainSecurity would like to thank the Web3 Foundation for supporting the research and development of PolPatrol with a research grant.

* * *

About Polkadot

Polkadot is the first interoperability protocol that enables blockchain networks to work together under the protection of shared security.

For more information on Polkadot, visit #polkadot.network.

About Web3 Foundation

Established in Zug, Switzerland by Ethereum co-founder and Chief Technology Officer Dr. Gavin Wood, Web3 Foundation funds research and development teams building the stack of technologies behind the decentralized web.

For more information on Web3 Foundation, visit #web3.foundation.