We use some external pallets that are highly configurable. The weight of the extrinsics in these pallets is highly dependent on the callback supplied in the config, and the existing benchmarks don't properly account for that. Is there any way to write a custom benchmark for an externally defined pallet?
3 Answers
In general it should not be necessary to implement custom benchmarks for external pallets.
You just have to configure the concrete callbacks in your runtime and re-run the benchmark. The weight of the callbacks will then be included in the output.
This is easier that to annotate weights in interface. Otherwise all interfaces that can be used as callback would need to expose their weight info as well.
In your example i dont see a difference. Since every runtime config needs to re-run benches, the effort for the CurrencyHooks will be included.
-
But the specific
CurrencyHooksthat we use only have a high cost for some of the currencies, and only when the chain is a particular state. So we need the benchmark to setup that state and use that particular currency, otherwise it won't hit the worst-case– SanderApr 20 at 10:45
Yes, you always should generate the weights based on your runtime configuration and your hardware requirements.
See and example from one of the FRAME pallets: the pallet_balances:
In Polkadot has its own Weights file and is different than the Wights file in another chain, Kusama for example.
Then in your Config have to specify your Weights. See the runtime file in Kusama.
Take a look into this previous answer on StackExchange: Benchmark FRAME pallets, for a more complete explanation about it.
-
I am aware of how to run benchmarks for external pallets in general, using the benchmarks supplied by the pallets themselves. The question is about supplying a custom benchmark for an external pallet, not about running existing benchmarks– SanderApr 14 at 13:47
-
Can you be more specific of the pallet and the configuration you are using? to understand better why the benchmarks are not accurate for your case Apr 14 at 14:09
-
To give an example, orml pallets come without benchmarks: github.com/open-web3-stack/open-runtime-module-library/tree/… . You could argue that the project should just add them, but some of these pallets are highly customizable by supplying custom hooks, making generic benchmarks difficult– SanderApr 14 at 14:51
For anyone finding this post in the future.. We got a working solution based on the frame_system benchmarks. See our custom benchmark here. The idea is to define a new dummy Pallet and Config item, write benchmarks for them, and apply the generated weights to the original pallet.
CurrencyHooksin the runtime configuration oforml-tokensbut obviously the upstream benchmarks can't account for those. What is the recommended approach here?