Questions tagged [benchmarking]
The benchmarking tag has no usage guidance.
118
questions
3
votes
2
answers
68
views
Get results of some std operations in benchmarks
We faced a situation the other day where we needed some result of std operations inside of our benchmarks. Specifically, we were looking to generate key pairs and signatures from types given by ...
- 1,080
0
votes
2
answers
98
views
How to set timestamp in benchmarks?
Is it possible to use std for benchmarks? If not, how can I get timestamps in a benchmark?
I tried using this but it didn't compile:
#[cfg(feature = "runtime-benchmarks")] extern crate std;...
- 1,080
0
votes
1
answer
32
views
Use criterion for statistic driven benchmarking of Substrate pallets
Criterion version 0.4.0 seems to support WASM builds: https://www.tweag.io/blog/2022-03-03-criterion-rs/#first-class-wasm-support
Can we us it to benchmark extrinsics of Substrate pallets?
- 1,080
3
votes
0
answers
101
views
Cannot create a runtime on benchmark run
Using polkadot-v0.9.19 branch I've created benchmarks for a pallet, added it into runtime, compiled.
On run of any of created benchmarks getting this error:
Error:
Service(Client(RuntimeApiError(...
CommunityBot
- 1
0
votes
1
answer
25
views
Weight consumed inside the on_idle hook?
The documentation of the on_idle hook states that "Any implementation should always respect _remaining_weight and never consume (and therefore return) more than this amount."
In the ...
3
votes
2
answers
52
views
How to use bounded_vec macro in benchmarking.rs
I was using substrate-node-template, and created some benchmarking functions.
I was able to use bounded_vec macro in tests.rs but, can't able to use it in the benchmarking.rs file
my code
#
- 616
1
vote
1
answer
33
views
Weight for an extrinsic that schedule another call in the future?
I have an extrinsic for a poll that will end in the future. I used scheduller pallet to make the end of the poll execute. The end of the poll is then made by a root call.
Should I add the weight of ...
CommunityBot
- 1
5
votes
2
answers
108
views
Runtime api not available just after client creation? Exported method is not found
I am trying to incorporate benchmark overhead command to our parachain. The issue i have is that runtime api methods are not available right away after client is created.
https://github.com/mangata-...
CommunityBot
- 1
2
votes
1
answer
33
views
Witness data for Benchmarking
I have been looking into the hrmp pallets where some of the extrinsics needs to have a witness data variable provided as a parameter.
Example:
/// Total number of opening channels must be provided as ...
- 8,463
1
vote
0
answers
39
views
Error when trying to run benchmarks on runtime but no in test benchmarks
i´m writing benchmarks for a pallet that manages sending tokens from assets pallet from one account to other.
I have written the benchmarks and run it with no problem using the command:
cargo test -p ...
- 3,759
1
vote
1
answer
53
views
Testing benchmarks with V2
Im writing benchmarks for a pallet using benchmarking version 2. And it seems I cant call Pallet::<Test>::test_benchmarks_"extrinsic_name" inside of a test anymore.
Is it no longer ...
- 2,598
0
votes
1
answer
21
views
Do struct allocations' weight consumption depend linearly on input data length?
I am writing some benchmarking code and I came across a situation where I had to benchmark a few let bindings.
let sig = ...;
let data : Vec<u8> = vec![...];
// data is actually variable. Ranges ...
- 3,702
2
votes
1
answer
66
views
Benchmark weights output
Can someone please explain the following output:
Why did the weights generate a zero input?
Weight::from_ref_time(0 as u64)
What is the fastest way to eyeball this and conclude whether these weights ...
CommunityBot
- 1
1
vote
1
answer
22
views
How to account for ensure_signed calls?
Let's say our pallet dispatchable returns DispatchResultWithPostInfo, where we are trying to manually calculate the weight, albeit with helpers obtained from benchmarking, but that's not relevant to ...
- 8,463
0
votes
0
answers
19
views
How does the benchmarking code add_benchmarks! work on the runtime?
I am trying to understand the runtime-benchmarks gated code on the runtime/src/lib.rs and I realize that we need to "list" and "add" benchmarks that we want to perform : https://...
- 501
2
votes
1
answer
78
views
Is there any benchmarking tool for parachain collator node performance?
Like there is e.g. polkadot benchmark machine --chain kusama --verify-duration 60 for the relaychain, is there something similar for the parachains? If not maybe some custom tool for measuring ...
CommunityBot
- 1
1
vote
1
answer
21
views
How to generate database weights?
As a chain grows the DB operations should get costlier. Is there a standard procedure that outlines how to generate these weights? Or should we continue using the default substrate weights for this ...
- 8,463
2
votes
1
answer
36
views
Benchmarking an extrinsic that validates a signature
I have an extrinsic in one of my pallets that looks validates a signature over a message, which is separate from the actual extrinsic being submitted.
The code can be simplified to look like the below
...
- 530
2
votes
1
answer
34
views
Benchmarking in a cusom pallet requires a storage put operation in Staking pallet
Let's say I have a pallet_x. In this pallet i have an extrinsic where I get the active_era using the below code :
Staking::<T>::active_era()
.map(|e| e....
- 8,463
3
votes
1
answer
74
views
Quadratic complexity in Benchmarking
I am trying to understand if quadratic complexity is bad (gives inaccurate results) when used for benchmarking:
I have a pallet whose weight would ideally depend on x * y where x and y are lengths of ...
- 8,463
4
votes
1
answer
29
views
Can we use std crates in benchmarking?
For benchmarking I can see a need to use random number generators for maybe verification of a random storage or such. Do we have utilities provided by the frame_benchmarking framework itself or is it ...
- 8,463
0
votes
1
answer
40
views
Who are whitelisted_callers?
On benchmarks one can often see whitelisted_caller() being used to test extrinsics and I wonder, how do they bypass the need for fee, and even pay for extrinsics like transfer_keep_alive for ...
- 759
3
votes
1
answer
48
views
How to benchmark frame-system on parachain?
After upgrading the Substrate to polkadot-v0.9.43.
I got the following error while running the runtime benchmarks.
2023-07-21 19:55:56 Starting benchmark: frame_system::set_code
Error: Input("...
- 9,499
1
vote
3
answers
108
views
Setting static parameter types in benchmarks
There is very little information to be found on static parameter types. All I could find is a docstring that says: In testing, StaticArgument can be altered later: StaticArgument::set(8).
I need to ...
CommunityBot
- 1
3
votes
1
answer
166
views
How to mock trait in benchmarking
Can you tell me how to mock a trait in benchmarking?
We have a pallet which takes NFTTrait as one of the configs. See below.
#[pallet::config]
pub trait Config: frame_system::Config {
...
- 254
7
votes
1
answer
66
views
Proof Skipped in benchmark
We're updating our weights to weights 2.0 but we're getting a lot of Proof Skipped lines in the output. What is the impact of this, and how do we fix it?
Example output:
/// Proof Skipped: Escrow ...
- 664
6
votes
1
answer
63
views
Benchmarking multiple instances
We have two instances of a pallet with different runtime configurations. How can we run the benchmarks separately to generate two distinct sets of weights?
As far as I can tell this is not possible ...
- 664
4
votes
1
answer
76
views
Missing XCM benchmark `reserve_asset_deposited`
I'm upgrading our parachain based on Polkadot 0.9.40 here. Now in the process of benchmarking xcm operations, I notice there is no benchmark for reserve_asset_deposited in XCM fungible benchmarks! I ...
- 664
5
votes
0
answers
33
views
Benchmarks not getting skipped with Err(BenchmarkError::Skip)
I am using the pallet_xcm_benchmarks, in the config for pallet_xcm_benchmarks::generic,
I have used
impl pallet_xcm_benchmarks::generic::Config for Runtime {
fn worst_case_asset_exchange() -> ...
- 664
3
votes
1
answer
41
views
Error: "Did not find the benchmarking metadata" after building runtime for benchmarking
After upgrading our chain to polkadot 0.9.38 I managed to build the benchmarking using cargo build --release --features runtime-benchmarks but when I try to benchmark any pallet after the build I ...
- 325
7
votes
1
answer
151
views
How does the benchmark calculate the proof size?
I have no idea why this call uses such a lot proof size (around 4MB).
It looks like it doesn't change too many storage items.
https://github.com/darwinia-network/darwinia-messages-substrate/blob/...
- 8,463
5
votes
1
answer
48
views
frame-benchmarking::v2 doesn't work for force_origin
In Config of my pallet I have:
type FilteredOrigin: EnsureOriginWithArg<
Self::RuntimeOrigin,
SomeEnum,
Success = Self::AccountId
>;
And there is an ...
- 8,463
4
votes
1
answer
34
views
Associated types in benchmark `where` bound
I'm trying to constrain the value of an associated type of a trait inside a benchmark's where bound, like this:
#[benchmarks(where T: currency::Config<Balance = u128>)]
pub mod benchmarks {
...
CommunityBot
- 1
3
votes
1
answer
55
views
`Currency::create_deposit` of `pallet_balance` fail on actual runtime but passes on mock rutime
The benchmark is written as:
one_extrinsic {
let caller: <T as frame_system::Config>::AccountId = make_account::<T>(1);
assert_eq!(
<T as crate::Config&...
CommunityBot
- 1
4
votes
1
answer
65
views
How to create a bunch of key pairs when benchmarking?
I want to benchmark verifying sr25519 signatures and for that I need to create a bunch of key pairs, sign some data, then benchmark the verification process, however using sp_core::sr25519::Pair is ...
- 1,073
3
votes
1
answer
56
views
Can't write to StorageMap when benchmarking, EncodeLike is not implemented for any type even if it's the same in pallet
I have the following StorageMap defined in my pallet:
#[pallet::storage]
pub(super) type Signatures<T: Config> = StorageMap<
_,
Blake2_128Concat,
T::Hash,
BoundedBTreeMap<...
- 159
3
votes
1
answer
52
views
Benchmark failing extrinsics
Is there a way to benchmark an extrinsic whose expected behavior is to trigger an error?
I stumbled on a project where we have two instances of the same pallet. Instance1 will proceed normally for a ...
- 8,463
3
votes
2
answers
27
views
Does substrate offer benchmarking utilities in outer node?
I have some heavy computation done off-chain (in outer node) and i want to benchmark those functions, does substrate offer tools for doing it or should i rely on a third party library to do that?
3
votes
3
answers
80
views
Custom benchmarks for external pallets
We use some external pallets that are highly configurable. The weight of the extrinsics in these pallets is highly dependent on the callback supplied in the config, and the existing benchmarks don't ...
- 372
6
votes
1
answer
107
views
cannot create module: instance allocation for this module requires 74376 bytes which exceeds the configured maximum of 65536 bytes
Try in to benchmark node pallets after Polkadot upgrade from 0.9.36 to 0.9.38 and get cannot create module: instance allocation for this module requires 74376 bytes which exceeds the configured ...
- 10.5k
2
votes
1
answer
55
views
Estimated proof size less than measured
I was running some benchmark, and noticed that the estimated proof size was less than the measured size, despite MaxEncodedLen being used in this crate. The estimation of 0 is clearly wrong, but this ...
- 8,463
5
votes
2
answers
152
views
Hardware: Benchmark Standard vs Ecosystem Nodes
During the last Substrate Builders Program Office Hours, the Mangata team raised awareness around some potential confusion around the Substrate Benchmarking System and how it is being applied in the ...
- 1,707
2
votes
1
answer
76
views
What is Dev Mode? And how can I use it?
I heard there is something called Dev Mode in Substrate for pallet development.
What is Dev Mode? And how can I use it?
- 8,130
0
votes
1
answer
198
views
1010: Invalid Transaction: Transaction would exhaust the block limits
How to specify limit weight for extrinsic?
- 1,280
0
votes
0
answers
44
views
Error: Input Runtime benchmarking wasn't enabled when building the node
I'm following along this benchmark docs to run benchmarking on my Substrate node (dependencies on branch = "polkadot-v0.9.25") with the command cargo build --release --features runtime-...
- 71
3
votes
2
answers
73
views
Using Autogenerated Weights
I'm wondering about weights. The docs are quite outdated here and I do not understand what is going on. I've managed to create benchmarks and generate a weight.rs.
https://github.com/paritytech/...
CommunityBot
- 1
0
votes
0
answers
23
views
Use sightglass for wasmtime benchmarking
Did anyone ever successfully use sightglass for benchmarking of Substrate in wasmtime?
I'm trying to build the following sightglass benchmarking suite / Dockerfile:
RUN echo 'APT::Install-Suggests &...
- 575
0
votes
1
answer
41
views
How to benchmark huge bounded vectors?
I want to have a queue system in a pallet using a storage value and a bounded vector.
/// Data related to tranmission queues.
#[pallet::storage]
#[pallet::getter(fn at_block_queue)]
pub type ...
- 8,463
0
votes
1
answer
29
views
How many (order of magnitude) Parachains can run on a Relaychain?
Does anyone have data/experience that indicates how many Substrate Parachains can run under one Substrate Relaychain?
I understand this will be hardware specific, so it will help if you can give some ...
- 2,126