1

I have problems with trying to benchmark my custom pallet. I tried different scenarios and received different errors:

Using the same binary:

  1. I built my node with cargo build --release --features runtime-benchmarks
  2. I generated a chainspec with the binary (/tmp/chainspec.json)
  3. When running ./target/release/aleph_node --execution wasm benchmark pallet --chain /tmp/chainspec.json --list (or the actual benchmarking command) I get:
Error: Input("Did not find the benchmarking metadata. This could mean that you either did not build
the node correctly with the `--features runtime-benchmarks` flag, or the chain spec that you are using was
not created by a node that was compiled with the flag: runtime requires function imports which are
not present on the host:
'env:ext_benchmarking_proof_size_version_1', 'env:ext_benchmarking_wipe_db_version_1', 
'env:ext_benchmarking_current_time_version_1', 
'env:ext_benchmarking_get_read_and_written_keys_version_1', 
'env:ext_benchmarking_reset_read_write_count_version_1', 
'env:ext_benchmarking_read_write_count_version_1', 
'env:ext_benchmarking_set_whitelist_version_1', 
'env:ext_benchmarking_commit_db_version_1'")

Using different binaries:

  1. I built my node with cargo build --release (no --features runtime-benchmarks)
  2. I generated a chainspec with the binary (/tmp/chainspec.json)
  3. I built my node (overwriting the old binary) with cargo build --release --features runtime-benchmarks
  4. I run ./target/release/aleph_node --execution wasm benchmark pallet --chain /tmp/chainspec.json --list and got expected result:
pallet, benchmark
pallet_custom, custom_method
  1. I ran ./target/release/aleph_node benchmark pallet --chain /tmp/chainspec.json --execution=wasm --wasm-execution=compiled --pallet pallet_custom --extrinsic "*" --steps 50 --repeat 20 and got error:
Error: Input("Did not find the benchmarking metadata. This could mean that you either did not build the 
node correctly with the `--features runtime-benchmarks` flag, or the chain spec that you are using was not
created by a node that was compiled with the flag: Other: Exported method Benchmark_benchmark_metadata is 
not found")

which makes perfect sense, since the wasm blob in the chainspec has been built without the feature. 6. I ran ./target/release/aleph_node benchmark pallet --chain /tmp/chainspec.json --pallet pallet_custom --extrinsic "*" --steps 50 --repeat 20 and this succeeds, however it's the wasm execution that should be used for benchmarking.

I suspect that this may be connected with misimplementing frame_benchmarking::Benchmark:

#[cfg(feature = "runtime-benchmarks")]
#[macro_use]
extern crate frame_benchmarking;

#[cfg(feature = "runtime-benchmarks")]
mod benches {
    define_benchmarks!([pallet_custom, Custom]);
}

impl_runtime_apis! {

    #[cfg(feature = "runtime-benchmarks")]
    impl frame_benchmarking::Benchmark<Block> for Runtime {
        fn benchmark_metadata(extra: bool) -> (
            Vec<frame_benchmarking::BenchmarkList>,
            Vec<frame_support::traits::StorageInfo>,
        ) {
            use frame_benchmarking::{Benchmarking, BenchmarkList};
            use frame_support::traits::StorageInfoTrait;

            let mut list = Vec::<BenchmarkList>::new();
            list_benchmarks!(list, extra);

            let storage_info = AllPalletsWithSystem::storage_info();

            (list, storage_info)
        }

        fn dispatch_benchmark(
            config: frame_benchmarking::BenchmarkConfig
        ) -> Result<Vec<frame_benchmarking::BenchmarkBatch>, sp_runtime::RuntimeString> {
            use frame_benchmarking::{Benchmarking, BenchmarkBatch, TrackedStorageKey};

            // I'm using Substrate in version polkadot-v0.9.26 so the new API is not available yet:
            // let whitelist: Vec<TrackedStorageKey> = AllPalletsWithSystem::whitelisted_storage_keys();
            let whitelist: Vec<TrackedStorageKey> = vec![
                // Block Number
                // Total Issuance
                // Execution Phase
                // Event Count
                // System Events
            ];

            let params = (&config, &whitelist);
            let mut batches = Vec::<BenchmarkBatch>::new();
            add_benchmarks!(params, batches);

            Ok(batches)
        }
    }

}
6
  • The fact that it works for wasm could mean that you are not passing the build flags somewhere. Did this happen after a Substrate update? Where can I find the code? Oct 11, 2022 at 10:56
  • @OliverTale-Yazdi I'm based on the old Substrate version (~polkadot-v0.9.26); the code is here: github.com/Cardinal-Cryptography/aleph-node/pull/655 Oct 11, 2022 at 11:45
  • I dont quite understand the code. You dont have a --dev flag for testing? Or --chain my-runtime-dev? Oct 11, 2022 at 12:37
  • nah, we don't have... we have a command bootstrap-chain for generating chainspec in a dev manner Oct 11, 2022 at 13:04
  • In Polkadot we only run the benchmarks with a dev runtime. I dont quite remember why, but there was a good reason for it… Oct 14, 2022 at 12:27

1 Answer 1

5

It turned out that I was missing one crucial thing:

impl sc_executor::NativeExecutionDispatch for ExecutorDispatch {
    #[cfg(feature = "runtime-benchmarks")]
    type ExtendHostFunctions = frame_benchmarking::benchmarking::HostFunctions;
    #[cfg(not(feature = "runtime-benchmarks"))]
    type ExtendHostFunctions = ();
...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.