Questions tagged [benchmarking]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
19 views

Do struct allocations' weight consumption depend linearly on input data length?

I am writing some benchmarking code and I came across a situation where I had to benchmark a few let bindings. let sig = ...; let data : Vec<u8> = vec![...]; // data is actually variable. Ranges ...
Purple_Turtle's user avatar
0 votes
0 answers
17 views

How does the benchmarking code add_benchmarks! work on the runtime?

I am trying to understand the runtime-benchmarks gated code on the runtime/src/lib.rs and I realize that we need to "list" and "add" benchmarks that we want to perform : https://...
b0zero's user avatar
  • 502
1 vote
1 answer
18 views

How to generate database weights?

As a chain grows the DB operations should get costlier. Is there a standard procedure that outlines how to generate these weights? Or should we continue using the default substrate weights for this ...
Purple_Turtle's user avatar
2 votes
1 answer
24 views

Benchmarking an extrinsic that validates a signature

I have an extrinsic in one of my pallets that looks validates a signature over a message, which is separate from the actual extrinsic being submitted. The code can be simplified to look like the below ...
Nate Armstrong's user avatar
2 votes
1 answer
30 views

Benchmarking in a cusom pallet requires a storage put operation in Staking pallet

Let's say I have a pallet_x. In this pallet i have an extrinsic where I get the active_era using the below code : Staking::<T>::active_era() .map(|e| e....
Immanuel John's user avatar
3 votes
1 answer
68 views

Quadratic complexity in Benchmarking

I am trying to understand if quadratic complexity is bad (gives inaccurate results) when used for benchmarking: I have a pallet whose weight would ideally depend on x * y where x and y are lengths of ...
Purple_Turtle's user avatar
1 vote
1 answer
20 views

How to account for ensure_signed calls?

Let's say our pallet dispatchable returns DispatchResultWithPostInfo, where we are trying to manually calculate the weight, albeit with helpers obtained from benchmarking, but that's not relevant to ...
Purple_Turtle's user avatar
4 votes
1 answer
29 views

Can we use std crates in benchmarking?

For benchmarking I can see a need to use random number generators for maybe verification of a random storage or such. Do we have utilities provided by the frame_benchmarking framework itself or is it ...
Purple_Turtle's user avatar
0 votes
1 answer
25 views

Who are whitelisted_callers?

On benchmarks one can often see whitelisted_caller() being used to test extrinsics and I wonder, how do they bypass the need for fee, and even pay for extrinsics like transfer_keep_alive for ...
Purple_Turtle's user avatar
2 votes
1 answer
40 views

How to benchmark frame-system on parachain?

After upgrading the Substrate to polkadot-v0.9.43. I got the following error while running the runtime benchmarks. 2023-07-21 19:55:56 Starting benchmark: frame_system::set_code Error: Input("...
AurevoirXavier's user avatar
3 votes
1 answer
30 views

Error: "Did not find the benchmarking metadata" after building runtime for benchmarking

After upgrading our chain to polkadot 0.9.38 I managed to build the benchmarking using cargo build --release --features runtime-benchmarks but when I try to benchmark any pallet after the build I ...
chexware's user avatar
  • 273
7 votes
1 answer
140 views

How does the benchmark calculate the proof size?

I have no idea why this call uses such a lot proof size (around 4MB). It looks like it doesn't change too many storage items. https://github.com/darwinia-network/darwinia-messages-substrate/blob/...
AurevoirXavier's user avatar
5 votes
1 answer
39 views

frame-benchmarking::v2 doesn't work for force_origin

In Config of my pallet I have: type FilteredOrigin: EnsureOriginWithArg< Self::RuntimeOrigin, SomeEnum, Success = Self::AccountId >; And there is an ...
Данила Алексеев's user avatar
5 votes
0 answers
32 views

Benchmarks not getting skipped with Err(BenchmarkError::Skip)

I am using the pallet_xcm_benchmarks, in the config for pallet_xcm_benchmarks::generic, I have used impl pallet_xcm_benchmarks::generic::Config for Runtime { fn worst_case_asset_exchange() -> ...
hooda's user avatar
  • 51
4 votes
1 answer
74 views

Missing XCM benchmark `reserve_asset_deposited`

I'm upgrading our parachain based on Polkadot 0.9.40 here. Now in the process of benchmarking xcm operations, I notice there is no benchmark for reserve_asset_deposited in XCM fungible benchmarks! I ...
Alex Sed's user avatar
  • 493
4 votes
1 answer
32 views

Associated types in benchmark `where` bound

I'm trying to constrain the value of an associated type of a trait inside a benchmark's where bound, like this: #[benchmarks(where T: currency::Config<Balance = u128>)] pub mod benchmarks { ...
Sander's user avatar
  • 362
3 votes
1 answer
47 views

Can't write to StorageMap when benchmarking, EncodeLike is not implemented for any type even if it's the same in pallet

I have the following StorageMap defined in my pallet: #[pallet::storage] pub(super) type Signatures<T: Config> = StorageMap< _, Blake2_128Concat, T::Hash, BoundedBTreeMap<...
dadzerlaze's user avatar
  • 1,053
3 votes
2 answers
26 views

Does substrate offer benchmarking utilities in outer node?

I have some heavy computation done off-chain (in outer node) and i want to benchmark those functions, does substrate offer tools for doing it or should i rely on a third party library to do that?
dadzerlaze's user avatar
  • 1,053
3 votes
1 answer
47 views

How to create a bunch of key pairs when benchmarking?

I want to benchmark verifying sr25519 signatures and for that I need to create a bunch of key pairs, sign some data, then benchmark the verification process, however using sp_core::sr25519::Pair is ...
dadzerlaze's user avatar
  • 1,053
3 votes
1 answer
47 views

Benchmark failing extrinsics

Is there a way to benchmark an extrinsic whose expected behavior is to trigger an error? I stumbled on a project where we have two instances of the same pallet. Instance1 will proceed normally for a ...
Benjamin Gallois's user avatar
3 votes
3 answers
64 views

Custom benchmarks for external pallets

We use some external pallets that are highly configurable. The weight of the extrinsics in these pallets is highly dependent on the callback supplied in the config, and the existing benchmarks don't ...
Sander's user avatar
  • 362
6 votes
1 answer
105 views

cannot create module: instance allocation for this module requires 74376 bytes which exceeds the configured maximum of 65536 bytes

Try in to benchmark node pallets after Polkadot upgrade from 0.9.36 to 0.9.38 and get cannot create module: instance allocation for this module requires 74376 bytes which exceeds the configured ...
Dzmitry Lahoda's user avatar
2 votes
1 answer
54 views

Estimated proof size less than measured

I was running some benchmark, and noticed that the estimated proof size was less than the measured size, despite MaxEncodedLen being used in this crate. The estimation of 0 is clearly wrong, but this ...
Sander's user avatar
  • 362
2 votes
1 answer
58 views

What is Dev Mode? And how can I use it?

I heard there is something called Dev Mode in Substrate for pallet development. What is Dev Mode? And how can I use it?
Bruno's user avatar
  • 7,705
6 votes
1 answer
63 views

Benchmarking multiple instances

We have two instances of a pallet with different runtime configurations. How can we run the benchmarks separately to generate two distinct sets of weights? As far as I can tell this is not possible ...
Gregory Hill's user avatar
6 votes
1 answer
62 views

Proof Skipped in benchmark

We're updating our weights to weights 2.0 but we're getting a lot of Proof Skipped lines in the output. What is the impact of this, and how do we fix it? Example output: /// Proof Skipped: Escrow ...
Sander's user avatar
  • 362
3 votes
1 answer
52 views

`Currency::create_deposit` of `pallet_balance` fail on actual runtime but passes on mock rutime

The benchmark is written as: one_extrinsic { let caller: <T as frame_system::Config>::AccountId = make_account::<T>(1); assert_eq!( <T as crate::Config&...
Sudip Ghimire's user avatar
0 votes
1 answer
145 views

1010: Invalid Transaction: Transaction would exhaust the block limits

How to specify limit weight for extrinsic?
rust.dev's user avatar
  • 511
0 votes
0 answers
41 views

Error: Input Runtime benchmarking wasn't enabled when building the node

I'm following along this benchmark docs to run benchmarking on my Substrate node (dependencies on branch = "polkadot-v0.9.25") with the command cargo build --release --features runtime-...
Kresna's user avatar
  • 71
0 votes
0 answers
21 views

Use sightglass for wasmtime benchmarking

Did anyone ever successfully use sightglass for benchmarking of Substrate in wasmtime? I'm trying to build the following sightglass benchmarking suite / Dockerfile: RUN echo 'APT::Install-Suggests &...
Achim's user avatar
  • 565
0 votes
1 answer
37 views

How to benchmark huge bounded vectors?

I want to have a queue system in a pallet using a storage value and a bounded vector. /// Data related to tranmission queues. #[pallet::storage] #[pallet::getter(fn at_block_queue)] pub type ...
Leouarz's user avatar
  • 177
0 votes
1 answer
25 views

How many (order of magnitude) Parachains can run on a Relaychain?

Does anyone have data/experience that indicates how many Substrate Parachains can run under one Substrate Relaychain? I understand this will be hardware specific, so it will help if you can give some ...
Begley Brothers Inc's user avatar
0 votes
1 answer
29 views

Use criterion for statistic driven benchmarking of Substrate pallets

Criterion version 0.4.0 seems to support WASM builds: https://www.tweag.io/blog/2022-03-03-criterion-rs/#first-class-wasm-support Can we us it to benchmark extrinsics of Substrate pallets?
Achim's user avatar
  • 565
0 votes
0 answers
20 views

How do i generate signatures in benchmarking.rs? [duplicate]

I am trying to generate an sr25519 signature within benchmarks. But i am facing an error that doesn't make sense to me. let sig: T::Signature = acc_pair.sign(&schema.encode()); | -...
master chief's user avatar
3 votes
2 answers
59 views

Using Autogenerated Weights

I'm wondering about weights. The docs are quite outdated here and I do not understand what is going on. I've managed to create benchmarks and generate a weight.rs. https://github.com/paritytech/...
shredding's user avatar
  • 284
1 vote
1 answer
72 views

What reference hardware can a parachain use to benchmark & generate weights?

The 2022 tracking issue for Substrate benchmarks mentions that Polkadot uses this reference hardware of this specification to generate weights: Reference Hardware The transaction weights in Polkadot ...
Nuke's user avatar
  • 1,802
0 votes
0 answers
19 views

How can I get time in ms for executing pallet calls?

I'm trying to measure the time a pallet call takes, in milliseconds. So I can compare for different setups the call 'impact', for example use different numbers of collators, and gather useful data to ...
Alexandre Ribeiro's user avatar
0 votes
1 answer
60 views

How to decrease time it takes to benchmark?

I have the following benchmark that takes a very long time to complete. What measures can I take to improve the time it takes to run the benchmark without losing its effectiveness? on_initialize { ...
Yatusabes's user avatar
  • 736
2 votes
1 answer
61 views

Benchmark weights output

Can someone please explain the following output: Why did the weights generate a zero input? Weight::from_ref_time(0 as u64) What is the fastest way to eyeball this and conclude whether these weights ...
Yatusabes's user avatar
  • 736
0 votes
1 answer
19 views

What does #[skip_meta] do?

I notice some benchmarks in Frame have #[skip_meta]. What does this macro do? Please explain when one should use this macro when benchmarking.
Yatusabes's user avatar
  • 736
0 votes
0 answers
22 views

How to check weather calculated weight are applied to the relevant function? [duplicate]

Normally, a developer write pallet public functions and assigning some hard coded weight to the function. Later on when everything is locked and development of pallet is complete. After that, the ...
Syed Kamran's user avatar
2 votes
1 answer
65 views

How to benchmark functions with multiple logical paths? [duplicate]

The purpose of a benchmarking functions is to find the resources consumed by a functions in the worst case scenario. Is there any tools/methodlogy available so that , we can find the most resource ...
Syed Kamran's user avatar
1 vote
1 answer
64 views

How to test my bench-marking functions?

I have written benchmarking functions from my pallet. How I can test these benchmarking functions please?
Syed Kamran's user avatar
2 votes
1 answer
55 views

why are complexity parameter or accounts seeding unnecessary for benchmarking transfers?

I am trying to understand why there is no complexity parameter on transfer benchmarks. The substrate benchmarks documentation states: For benchmarks, you want to execute the most computationally ...
Yatusabes's user avatar
  • 736
2 votes
1 answer
64 views

Benchmark extra macro

What does the #[extra] macro do in benchmarks? I was looking at how benchmarks are being implemented on the Balances pallet. I notice the following benchmarks: benchmarks_instance_pallet { transfer {...
Yatusabes's user avatar
  • 736
2 votes
1 answer
38 views

Benchmarking pallet code which is not an extrinsic

I am currently writing code for the hooks on_initialize and on_idle. They share functionality in the form of 2 methods, each using these methods multiple times. I would like to accurately benchmark ...
gatsey's user avatar
  • 183
1 vote
2 answers
87 views

How to get the current weights of a chain for all extrinsics to verify all benchmarks has been integrated?

I have just ran benchmarks and updated all the weights of the pallets in my parachain... or have I? How can I verify that all my changes "bite"? Is there a way to check the weights of a ...
Simson's user avatar
  • 929
0 votes
0 answers
36 views

Howto customize Babe and Grandpa default weights

It seems polkadot and kusama runtimes both which use babe and grandpa use the default weights for those pallets. The weights are handcrafted ref In default_weights.rs and default_weights.rs The ...
Mokhtar Naamani's user avatar
0 votes
2 answers
87 views

How to set timestamp in benchmarks?

Is it possible to use std for benchmarks? If not, how can I get timestamps in a benchmark? I tried using this but it didn't compile: #[cfg(feature = "runtime-benchmarks")] extern crate std;...
Robert La Ferla's user avatar
3 votes
1 answer
43 views

Block number in the benchmark code is 0, but is 1 in the running code

If I'm running a benchmark and I get the current block number from inside the benchmark code (i.e. in the setup before I actually call the call I'm trying to benchmark) using frame_system::Pallet::<...
Kyle's user avatar
  • 268