Cannot run benchmarking: "Did not find the benchmarking metadata."

Question

I have problems with trying to benchmark my custom pallet. I tried different scenarios and received different errors:

Using the same binary:

1. I built my node with cargo build --release --features runtime-benchmarks
2. I generated a chainspec with the binary (/tmp/chainspec.json)
3. When running ./target/release/aleph_node --execution wasm benchmark pallet --chain /tmp/chainspec.json --list (or the actual benchmarking command) I get:

Error: Input("Did not find the benchmarking metadata. This could mean that you either did not build
the node correctly with the `--features runtime-benchmarks` flag, or the chain spec that you are using was
not created by a node that was compiled with the flag: runtime requires function imports which are
not present on the host:
'env:ext_benchmarking_proof_size_version_1', 'env:ext_benchmarking_wipe_db_version_1', 
'env:ext_benchmarking_current_time_version_1', 
'env:ext_benchmarking_get_read_and_written_keys_version_1', 
'env:ext_benchmarking_reset_read_write_count_version_1', 
'env:ext_benchmarking_read_write_count_version_1', 
'env:ext_benchmarking_set_whitelist_version_1', 
'env:ext_benchmarking_commit_db_version_1'")

Using different binaries:

1. I built my node with cargo build --release (no --features runtime-benchmarks)
2. I generated a chainspec with the binary (/tmp/chainspec.json)
3. I built my node (overwriting the old binary) with cargo build --release --features runtime-benchmarks
4. I run ./target/release/aleph_node --execution wasm benchmark pallet --chain /tmp/chainspec.json --list and got expected result:

pallet, benchmark
pallet_custom, custom_method

5. I ran ./target/release/aleph_node benchmark pallet --chain /tmp/chainspec.json --execution=wasm --wasm-execution=compiled --pallet pallet_custom --extrinsic "*" --steps 50 --repeat 20 and got error:

Error: Input("Did not find the benchmarking metadata. This could mean that you either did not build the 
node correctly with the `--features runtime-benchmarks` flag, or the chain spec that you are using was not
created by a node that was compiled with the flag: Other: Exported method Benchmark_benchmark_metadata is 
not found")

which makes perfect sense, since the wasm blob in the chainspec has been built without the feature. 6. I ran ./target/release/aleph_node benchmark pallet --chain /tmp/chainspec.json --pallet pallet_custom --extrinsic "*" --steps 50 --repeat 20 and this succeeds, however it's the wasm execution that should be used for benchmarking.

I suspect that this may be connected with misimplementing frame_benchmarking::Benchmark:

#[cfg(feature = "runtime-benchmarks")]
#[macro_use]
extern crate frame_benchmarking;

#[cfg(feature = "runtime-benchmarks")]
mod benches {
    define_benchmarks!([pallet_custom, Custom]);
}

impl_runtime_apis! {

    #[cfg(feature = "runtime-benchmarks")]
    impl frame_benchmarking::Benchmark<Block> for Runtime {
        fn benchmark_metadata(extra: bool) -> (
            Vec<frame_benchmarking::BenchmarkList>,
            Vec<frame_support::traits::StorageInfo>,
        ) {
            use frame_benchmarking::{Benchmarking, BenchmarkList};
            use frame_support::traits::StorageInfoTrait;

            let mut list = Vec::<BenchmarkList>::new();
            list_benchmarks!(list, extra);

            let storage_info = AllPalletsWithSystem::storage_info();

            (list, storage_info)
        }

        fn dispatch_benchmark(
            config: frame_benchmarking::BenchmarkConfig
        ) -> Result<Vec<frame_benchmarking::BenchmarkBatch>, sp_runtime::RuntimeString> {
            use frame_benchmarking::{Benchmarking, BenchmarkBatch, TrackedStorageKey};

            // I'm using Substrate in version polkadot-v0.9.26 so the new API is not available yet:
            // let whitelist: Vec<TrackedStorageKey> = AllPalletsWithSystem::whitelisted_storage_keys();
            let whitelist: Vec<TrackedStorageKey> = vec![
                // Block Number
                // Total Issuance
                // Execution Phase
                // Event Count
                // System Events
            ];

            let params = (&config, &whitelist);
            let mut batches = Vec::<BenchmarkBatch>::new();
            add_benchmarks!(params, batches);

            Ok(batches)
        }
    }

}

Answer 1

It turned out that I was missing one crucial thing:

impl sc_executor::NativeExecutionDispatch for ExecutorDispatch {
    #[cfg(feature = "runtime-benchmarks")]
    type ExtendHostFunctions = frame_benchmarking::benchmarking::HostFunctions;
    #[cfg(not(feature = "runtime-benchmarks"))]
    type ExtendHostFunctions = ();
...