Questions tagged [benchmarking]
The benchmarking tag has no usage guidance.
119
questions
2
votes
1
answer
60
views
why are complexity parameter or accounts seeding unnecessary for benchmarking transfers?
I am trying to understand why there is no complexity parameter on transfer benchmarks.
The substrate benchmarks documentation states:
For benchmarks, you want to execute the most computationally ...
- 766
2
votes
1
answer
68
views
Benchmark extra macro
What does the #[extra] macro do in benchmarks?
I was looking at how benchmarks are being implemented on the Balances pallet. I notice the following benchmarks:
benchmarks_instance_pallet {
transfer {...
- 766
2
votes
1
answer
41
views
Benchmarking pallet code which is not an extrinsic
I am currently writing code for the hooks on_initialize and on_idle. They share functionality in the form of 2 methods, each using these methods multiple times. I would like to accurately benchmark ...
- 183
1
vote
2
answers
90
views
How to get the current weights of a chain for all extrinsics to verify all benchmarks has been integrated?
I have just ran benchmarks and updated all the weights of the pallets in my parachain... or have I?
How can I verify that all my changes "bite"?
Is there a way to check the weights of a ...
- 1,110
0
votes
2
answers
98
views
How to set timestamp in benchmarks?
Is it possible to use std for benchmarks? If not, how can I get timestamps in a benchmark?
I tried using this but it didn't compile:
#[cfg(feature = "runtime-benchmarks")] extern crate std;...
- 530
3
votes
1
answer
46
views
Block number in the benchmark code is 0, but is 1 in the running code
If I'm running a benchmark and I get the current block number from inside the benchmark code (i.e. in the setup before I actually call the call I'm trying to benchmark) using frame_system::Pallet::<...
- 299
0
votes
1
answer
44
views
How setup Storage for a proper benchmarking?
Right now by benchmarks look like the following
some_pallet_func_using_storage{
..filling storage with garbage data to create the worst scenario..
}:_(RawOrigin::Signed(caller), bla,bla,bla)
verify{
....
- 1
1
vote
1
answer
221
views
Cannot run benchmarking: "Did not find the benchmarking metadata."
I have problems with trying to benchmark my custom pallet. I tried different scenarios and received different errors:
Using the same binary:
I built my node with cargo build --release --features ...
- 234
3
votes
1
answer
67
views
Why would benchmarking a simple setter has a linear behaviour
I came across this basic example in the substrate repo, where a range 1..1000 is used when benchmarking it. The generated weights.rs shows
fn set_dummy_benchmark(b: u32, ) -> Weight {
Weight::...
- 363
1
vote
1
answer
83
views
How to use an outer mock inside `impl_benchmark_test_suite`?
Hey valuable Substrate pros,
do you know a way to use impl_benchmark_test_suite inside another crate like the runtime? I have multiple pallets. There are no mocks inside these pallets. There is only ...
- 503
1
vote
0
answers
63
views
Another Benchmarking Wasm error [closed]
I asked a similar question a month ago here, and the answer/explanation was found in a failing cargo t -p pallet-nft --features runtime-benchmarks command.
This time, the building part works, the ...
- 385
5
votes
2
answers
109
views
Runtime api not available just after client creation? Exported method is not found
I am trying to incorporate benchmark overhead command to our parachain. The issue i have is that runtime api methods are not available right away after client is created.
https://github.com/mangata-...
2
votes
1
answer
121
views
How can I access the emitted runtime events inside the benchmarking.rs file?
I read in this answer that you can access everything from mock runtime in the benchmarking file by just adding #[cfg(test)] above the import use crate::mock::*; and that it is not a good idea since it ...
- 85
2
votes
1
answer
210
views
Problem with running Benchmarks on substrate pallet: recursion limit reached while expanding `stringify!`
When I add and run a new benchmark via
cargo test --package pallet-<name> --features runtime-benchmarks
I get an error which is unclear and unrelated to the actual benchmark i added:
error: ...
0
votes
2
answers
85
views
Error: Expected associated type but found enum. I'm getting this error even after defining the associated type as a custom enum in the mock runtime
I'm trying to run the following benchmark test:
use primitives::{CurrencyId, TokenId};
benchmarks! {
onboard_asset {
let caller: T::AccountId = whitelisted_caller();
let ...
- 85
2
votes
1
answer
164
views
Error when trying to benchmark the substrate-node-template -- "Found argument 'Cargo.toml' which wasn't expected, or isn't valid in this context"
I'm trying to benchmark the pallet_template from the substrate-node-template example but am getting the error: Found argument 'Cargo.toml' which wasn't expected, or isn't valid in this context.
I have ...
- 85
2
votes
1
answer
79
views
Use BoundedVec in benchmark
I have some code that benchmarks one of my extrinsic.
benchmarks! {
create_supersig {
let z in 0 .. 10;
let alice: T::AccountId = get_account::<T>("ALICE");
...
1
vote
1
answer
28
views
Unknown storage interactions during benchmarking custom pallet
I was running benchmarks for my pallet and i had a note about some unknown db interactions besides the ones i expect.
// Storage: unknown [0x3a7472616e73616374696f6e5f6c6576656c3a] (r:1 w:1)
I tried ...
- 227
1
vote
1
answer
75
views
How to persist TestExternalities via impl_benchmark_test_suite testing?
I set up a state via
impl ExtBuilder {
pub fn build(self) -> sp_io::TestExternalities {
let mut t = system::GenesisConfig::default().build_storage::<TestRuntime>().unwrap();
let mut ext =...
- 125
1
vote
1
answer
80
views
Benchmarking Wasm error
I am trying to do benchmarking for my pallet,but I get the following error:
Error executing and verifying runtime benchmark: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction ...
- 385
5
votes
1
answer
215
views
How to weight `on_initialize`?
Is there any best practice how to count the weights in on_initialize correctly?
I read through the Polkadot and Substrate on_initialize weight calculations and couldn't always figure out: When to ...
- 503
3
votes
2
answers
57
views
benchmarks: Significant different between 99th and worst case
I'm running benchmark-cli storage tool to calculate average weights for DB read and writes and one thing that I've noticed is that there are significant difference between even 99th percentile and ...
- 97
2
votes
1
answer
34
views
Do I need to re-run frame-support weight constants?
I notice that frame-support has some constant weights:
ExtrinsicBaseWeight
RocksDbWeight
BlockExecutionWeight
ParityDbWeight (not using)
Sean recommends that we run weights on all pallets as ...
- 766
5
votes
2
answers
120
views
Large event payloads unsafe?
Premise
Substrate will, by default, store every event in a block in a storage value Event of type Vec<EventRecord> in the system module, which involves reading & writing this full vector ...
- 451
1
vote
1
answer
145
views
Do `on_finalized` and `on_initialize` impact block weight budget?
on_initialize returns a weight, but what is the effect of this weight exceeding block weight limit? Is the block valid? can extrinsic still be included?
on_finalized does not return a weight, does ...
- 451
3
votes
2
answers
68
views
Get results of some std operations in benchmarks
We faced a situation the other day where we needed some result of std operations inside of our benchmarks. Specifically, we were looking to generate key pairs and signatures from types given by ...
- 530
1
vote
1
answer
130
views
Benchmarking whitelist
"Whitelist account from further DB operations."
What does it actually means, and when should it be used? Looking into the examples, I see that basically every caller of the extrinsic is ...
- 148
2
votes
2
answers
206
views
Why are events stored?
The documentation around emitting events states that events are written to storage, and then cleared next block.
This has considerable costs
You add extra db write/read overhead to practically every ...
- 451
3
votes
0
answers
101
views
Cannot create a runtime on benchmark run
Using polkadot-v0.9.19 branch I've created benchmarks for a pallet, added it into runtime, compiled.
On run of any of created benchmarks getting this error:
Error:
Service(Client(RuntimeApiError(...
- 133
3
votes
1
answer
111
views
How to see the hardware benchmark results?
Substrate has introduced a flag called --no-hardware-benchmarks to disable automatic hardware benchmarks. This implies if the flag is not set, the benchmark should run automatically at startup and ...
- 545
1
vote
0
answers
71
views
Error when trying to use an account in Benchmarks expected associated type, found enum `RawOrigin`
I have the following account initialisation on benchmarking.rs file. The following configuration works on mock.rs and test.rs files but getting errors in benchmarking
pub(crate) fn test_pub(n: u8) -&...
- 343
2
votes
1
answer
82
views
How to benchmark pallet Grandpa?
at the moment I struggle a bit with the question of "How to benchmark the Grandpa pallet". Like it seems it's not possible to auto-generate weights at the moment (see here: https://github....
- 319
3
votes
3
answers
128
views
How to create and mint assets in benchmarks
I am building a pallet which uses assets in a loosely-coupled manner. In the pallet's Config, there's an associated type Assets with Inspect and Transfer bounds.
Now, when writing benchmarks, I need ...
- 1,053
0
votes
1
answer
85
views
cannot find trait WeightInfo in this scope
I've got the error:
cannot find trait WeightInfo in this scope
I've added this in runtime :
impl my_pallet::Config for Runtime {
type Event = Event;
type WeightInfo = my_pallet::weights::...
- 418
0
votes
1
answer
76
views
Benchmarks command error
need help with benchmarking: the command cargo build --release --features runtime-benchmarks completes without any problem, but then I try ./target/release/fs-node benchmark pallet --chain dev --...
- 385
1
vote
1
answer
77
views
How to calculate weight on cumulus
I am a bit confused about this. If I have a pr on Substrate that requires a companion on Cumulus that adds to the weight file for a given pallet, how do I calculate the weight? I can run the benchmark....
- 623
2
votes
1
answer
80
views
Is there any benchmarking tool for parachain collator node performance?
Like there is e.g. polkadot benchmark machine --chain kusama --verify-duration 60 for the relaychain, is there something similar for the parachains? If not maybe some custom tool for measuring ...
- 93
1
vote
1
answer
42
views
Cumulus extrinsic weight
Are the benchmarks run on the same machine in the cumulus repository as the ones in the substrate? That is, is the weight in weights.rs files the same in these two repositories?
Edit: The weights in ...
- 623
2
votes
1
answer
48
views
What's the difference between add_benchmarks and add_benchmark?
I am trying to migrate my pallet over to the parachain template, and while adding it to the runtime I noticed a slight difference between how benchmarks are defined.
On the old runtime, all (or most) ...
- 561
2
votes
1
answer
40
views
Weight selection for extrinsic in case of multiple bechmarks
In my benchmarks, I've written multiple benchmarks for an extrinsic to cover different scenarios, which gave me WeightIinfo for each of my benchmarks.
Now the question here is how I can pick the worst ...
- 493
8
votes
1
answer
184
views
Performance tuning of Substrate node
I was recently wondering what kind of performance tuning options can be specified while building a Substrate node. Motivation for doing this is based on the benchmark I ran on machines that utilize ...
- 161
2
votes
1
answer
79
views
ink! contract extrinsics only take 10-11% block weights when stress test
I have an ink! contract project and I'm doing stress test to benchmark the performance of this ink! contract project in a production chain.
My stress test includes these steps:
Deploy ink! projects
...
- 291
5
votes
1
answer
133
views
Benchmark session pallet fails without pallet_staking
my lovely Substrate-Heros,
at the moment I try to benchmark the FRAME pallets of our custom runtime. Unfortunately, I ran into an error while benchmarking the session pallet. I copied the ...
- 319
5
votes
1
answer
160
views
Benchmark FRAME pallets
Hello Substrate Experts,
at the moment I try to figure out how to benchmark FRAME pallets for our runtime and I have basically three points that are not 100% clear to me.
Based on my understanding it'...
- 319
2
votes
2
answers
219
views
Benchmarking Mock
We’re trying to do the exact thing as referenced in How to mock trait in benchmarking, and we have implemented the struct + trait we need in our mocks, however, the benchmarks are calling the runtime ...
- 766
6
votes
1
answer
374
views
What is the possible minimum block time?
Is there a way to determine/calculate what the minimum block time would be? If not, what could we consider before lowering the block time, and how to monitor the side effects of small block time?
I ...
- 893
9
votes
1
answer
212
views
Single node performance
I've a couple of questions:
I'm running a the node-template on a machine that has 8 cores and 64GB of RAM and its using an SSD. I have some pallets working there and I'm using the python client to ...
- 1,104
1
vote
0
answers
73
views
Function or associated item cannot be called upon benchmarking
I’m trying to run benchmarks on the latest commit in this PR https://github.com/InvArch/InvArch-Node/pull/87 The pallet library pallet library is separate so the runtime depends on this latest commit ...
- 71
2
votes
1
answer
48
views
Do benchmarks consider cached reads?
When running benchmarks do the result consider that multiple reads to the same value per block are cached as mentioned in the documentation referenced below?
If caching is not considered in the ...
- 766
2
votes
1
answer
112
views
Error: Input("InsufficientBalance") when running benchmark
Trying to benchmark an extrinsic, but I keep getting error:
./target/release/rmrk-substrate benchmark --pallet pallet-rmrk-core --extrinsic 'create_collection'
Error: Input("InsufficientBalance&...
- 227