Why would benchmarking a simple setter has a linear behaviour

Question

I came across this basic example in the substrate repo, where a range 1..1000 is used when benchmarking it. The generated weights.rs shows

fn set_dummy_benchmark(b: u32, ) -> Weight {
    Weight::from_ref_time(5_834_000 as u64)
        .saturating_add(Weight::from_ref_time(24_000 as u64).saturating_mul(b as u64))
        .saturating_add(T::DbWeight::get().writes(1 as u64))
}

I've got two questions:

1. Isn't that a bit counterintuitive? I understand the whole benchmarking will finally build a linear model, but why would set_dummy(1) and set_dummy(1000) make such a big difference? I mean the saturating_mul(b as u64) part: if b is big enough it can consume a significant amount of weights? OK b is set to be 1000 max, but I thought it was as simple as updating storage with an integer(or alike) so it should be constant?

2. If there's no such range variance, I could imagine the benchmarking outcome would be constant. So which one is correct? And which one should we follow in real practice? I'm just concerned that defining the upper bound for every value would be infeasible..

Answer 1

The benchmarking process is not an exact science. Rather it is simply measuring the real time execution of a process.

In this case, the benchmark creator has implied there should be a linear weight function by providing a range of 0 - 1000. The function, you are right, is constant time, but small variance in executing the code my have created enough to make the linear fitting function think there is some amount of linear fit.

The solution here is to update the benchmark to remove the linear range as I have done here: https://github.com/paritytech/substrate/pull/12456

So thanks, this is basically a "bug" report.