0

I am writing some benchmarking code and I came across a situation where I had to benchmark a few let bindings.

let sig = ...;
let data : Vec<u8> = vec![...];
// data is actually variable. Ranges from 0 bytes to 200k bytes
let tx = Transaction { ..., data };

If I put this code in a #[block] and benchmark it I get a constant number: 222_000. However. If I benchmark it using Linear<1, 200_000> I will get a linear dependence on length 400_000 + 10.saturating_mul(x).

I took care to not put the vec![] allocation inside the benchmark #[block]. Basically what you see in the code-snip above is also inside #[block].

What surprises me now is that if the vec allocation is happening outside, why should the vector length matter when creating a let binding?

Doing the math on both, including the linear component gives me a 3-5x increase in estimated weight which I consider significant enough to warrant a question. Would love if someone can explain me the apparent discrepancy that I see here.

3
  • For these kinds of questions around the output of benchmarks, it is best to start looking at the raw data, not just the linear forumula that is outputted. Configure your benchmarks with the json_file flag, and graph the results. Usually these questions can answer themselves.
    – Shawn Tabrizi
    Sep 17, 2023 at 21:19
  • @ShawnTabrizi thanks I didn't know about json_file. It helped me answer the question. Sep 18, 2023 at 7:11
  • If you have an answer to your own question, please write down the steps you took to solve it and what the solution was, that way others with a similar problem can get help.
    – Shawn Tabrizi
    Sep 19, 2023 at 20:15

1 Answer 1

0

So as I was expecting, let bindings shouldn't have a significant effect on benchmarking depending on the the vector size as in this particular case, the vector allocation was being done elsewhere (not being benchmarked). Just to be sure that this was the case and to find out the cause of getting a small linear correlation on data length, I ran the benchmarks with json_file, and noted that there was no dependence, but only minor outliers that were causing the benchmarking framework to falsely think that there was a linear upwards correlation. The extrinsic time and storage_root_time remained unchanged with the datum size.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.