Calculating actual weights for extrinsics

Question

I have a substrate pallet with a number of extrinsics, some simple, some more complex. Depending on the chain state and the input variables, the amount of computation and the number of DB reads for the same extrinsic call can be quite different.

I've done my weight annotations based on a worst-case scenario. I'm happy with those for estimating the transaction fees in advance.

Now I'm trying to return the actual weight via DispatchResultWithPostInfo so that the transaction-payment pallet can calculate the real fee and return the difference to the user. I, perhaps naively, thought that the actual weight of an extrinsic call would be calculated for me somewhere and I could just pass this back in DispatchResultWithPostInfo, but this doesn't seem to be the case.

So I've created a weight variable in each extrinsic which I update each time there is a DB read/write operation so that I have something to create my PostInfo that reflects what actually happened. This raises a few questions for me:

1/ When do DB reads get cached, and should therefore not add extra weight?

2/ How can I get the actual computation time? Or can I not and have to use estimates from benchmarking?

3/ Is this really the best way to calculate the actual weight? I feel I have to be missing something.

Thanks in advance for any help.

Answer 1

First, the transaction_payment pallet does this for you with compute_actual_fee from the post_dispatch and configuring it in your runtime with the ChargeTransactionPayment signed extension via SignedExtra, as done in Kusama. Some additional info.

Going to your questions.

1. The first thing that needs to be understood is the existence of a Storage Overlay, which caches values which have been read or written to inside of a block. This means, if you read a storage item, the first time, it will need to navigate through the whole merkle trie, and fetch that storage item. But the second time, it just reads from the storage overlay, since that value has been cached by the client (ref).

2. Correct, you should benchmark. Regarding your cache concern, the benchmarking tool/framework is smart enough to know that the reads that are cached don't add up the same cost as a read from db (ref).

Just in case, for benchmarking in general, check out what whitelisting is.

3. Correct again, based on your benchmark results you should do weight annotations based on a worst-case scenario. The post_dispatch will handle the difference between the worst-case and actual weight. In the official FRAME docs the weights are generated dynamically with a weight function in the weight.rs file. You could consider this "the best way".

Example for answer three:

• extrinsic: frame/identity/src/lib.rs
• dynamic weight function: frame/identity/src/weight.rs

For people who like to watch a in-depth video about benchmarking, watch the substrate seminar about it!

For a good explanation about the importance of weights.