1

Hey valuable Substrate pros,

do you know a way to use impl_benchmark_test_suite inside another crate like the runtime? I have multiple pallets. There are no mocks inside these pallets. There is only one runtime mock in my project. Since impl_benchmark_test_suite needs the mock, I would like to find a way to call the benchmark tests in an outer crate like the runtime.

#[cfg(test)]
#[cfg(feature = "runtime-benchmarks")]
mod benchmark_tests {
    use super::*;
    use frame_benchmarking::impl_benchmark_test_suite;

    impl_benchmark_test_suite!(
        my_pallet_1::Pallet,
        crate::mock::ExtBuilder::default().build(),
        crate::mock::Test,
        benchmarks_path = my_pallet_1::benchmarking,
    );

    impl_benchmark_test_suite!(
        my_pallet_2::Pallet,
        crate::mock::ExtBuilder::default().build(),
        crate::mock::Test,
        benchmarks_path = my_pallet_2::benchmarking,
    );

    impl_benchmark_test_suite!(
        my_pallet_3::Pallet,
        crate::mock::ExtBuilder::default().build(),
        crate::mock::Test,
        benchmarks_path = my_pallet_3::benchmarking,
    );
}

This produces that error:

error: no rules expected the token `::`

my_pallet_1::Pallet

no rules expected this token in macro call

EDIT:

Trying to use the benchmark test module inside the mock of the runtime:

#[cfg(feature = "runtime-benchmarks")]
mod benchmark_tests {
    use frame_benchmarking::impl_benchmark_test_suite;

    use my_pallet::Pallet;

    impl_benchmark_test_suite!(
        Pallet,
        ExtBuilder::default().build(),
        Test,
        benchmarks_path = my_pallet::benchmarking,
    );
}

This produces this error:

error: no rules expected the token `)`
375 | /     impl_benchmark_test_suite!(
376 | |         Pallet,
377 | |         ExtBuilder::default().build(),
378 | |         Test,
379 | |         benchmarks_path = my_pallet::benchmarking,
380 | |     );
    | |_____^ no rules expected this token in macro call

EDIT 2:

I tried another approach: Just using the mock runtime and ExtBuilder from another pallet.

Inside my_pallet/benchmarking.rs:

impl_benchmark_test_suite!(Pallet, pallet_tests::ExtBuilder::default().build(), pallet_tests::Test);

Although ExtBuilder and Test is public in pallet_tests it produces the following error:

failed to resolve: could not find `ExtBuilder` in `pallet_tests`

could not find `ExtBuilder` in `pallet_tests`

cannot find type `Test` in crate `pallet_tests`

1 Answer 1

1

I suspect the problem in your approach is using fully qualified paths for the first and fourth argument. Checking in the docs I found the following information:

because of macro syntax limitations, neither Pallet nor benches can be paths, but both have to be idents in the scope of impl_benchmark_test_suite.

So I guess you might try doing something like this:

use my_pallet_1::{Pallet as Pallet1, benchmarking as bench1};

impl_benchmark_test_suite!(
    Pallet1,
    crate::mock::ExtBuilder::default().build(),
    crate::mock::Test,
    benchmarks_path = bench1
);

I haven't tested it though, it's just my suggestion based on the docs.


EDIT: This doesn't work, unfortunately. See comments.

6
  • Unfortunately this doesn't work. This error: error: no rules expected the token ')' and in addition this warning: unused imports: Pallet as Pallet1, 'benchmarking as bench1'.
    – Chralt
    Oct 5, 2022 at 11:57
  • Another error, which comes after doing this: error[E0599]: no function or associated item named 'test_bench_by_name' found for struct 'my_pallet::Pallet' in the current scope
    – Chralt
    Oct 5, 2022 at 13:23
  • I've tried to reproduce what you attempt to do, but with my crate, and got the same error. So after some digging in Substrate I learned that test_bench_by_name is indeed implemented by the benchmarks! macro, but it has a #[cfg(test)] annotation, which AFAIR makes it invisible to other crates. :( You might take a look at the source and try to implement it manually, but I'm not sure if it's worth. Oct 5, 2022 at 15:08
  • 1
    test doesn't work like other features in Rust. You can't use code from other crates annotated with #[cfg(test)], even when the code using it would have #[cfg(test)] too. Oct 6, 2022 at 9:42
  • 1
    I just found a way to get a better result with just ignoring impl_benchmark_test_suite and use the actual runtime config parameters with this script: ./target/release/my-node benchmark pallet --chain dev --execution=native --wasm-execution=compiled --heap-pages=4096 --pallet "*" --extrinsic "*" --steps 1 --repeat 0 --detailed-log-output
    – Chralt
    Oct 6, 2022 at 9:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.