11
  1. How expensive are storage reads and writes?
  2. Does the cost depend on the size of the item?

1 Answer 1

18

Substrate itself is not concerned with Cost but with Weight.
How Weight is translated to Cost depends on the specific chain.
Weight is the time to complete a specific task. This will be updated to a multivariate struct here, for now its just time.

How expensive are storage reads and writes?

The Weight of one storage read/write is a constant in Substrate.
Currently the values are set to Read: 25 µs and Write: 100 µs, here and here.
A Substrate chain (like Polkadot) can use these or custom values instead.
We want to give Substrate developers the tools to calculate their own values in the future.
This way they are not reliant on hard-coded values and get more precise results.

Does the cost depend on the size of the item?

No. These constants are not multiplied with the size of the accessed items.
This will soon change but for now these constants are used as approximation.
Now you ask how small values can be as fast as large values, let's analyze Polkadot as example.

Storage Weights in Polkadot

Here is a graphic how the storage read/write times look on Polkadot at Block 9091874.
The top part shows how many values per size exist; most values are 80 Byte.
In the bottom graphic you see the time it took for each value size to be read and written.
The black lines are the average +25%, which produces nearly the constants that are currently used.

RocksDB DOT#9091874      Same but with lower Alpha
RocksDB DOT#9091874 RocksDB DOT#9091874

The outliers to the top right (very large and slow) are currently ignored, since they are only accessed very rarely.
In the future we want to make the Weight dependent on the value size to have more customizable results.

Conclusion

For Polkadot these constants are fine since most values are 80 Byte and small values indeed have a constant access time.
We are currently introducing tooling to calculate Weights for any Substrate chain to remove such assumptions.
This way it will be possible to have correct and generally applicable Weights.

The underlying Database

An interesting observation in the graphics above is that small values seem to have constant access time.
Let's look at how the underlying DB performs on its own:

RocksDB Read 8-32 KiB RocksDB Read 0-64 KiB
RocksDB Read jump @14600 RocksDB Read jump @14600

It seems tightly clustered for small values and makes a jump at ~14,600 Byte.
Up or down, depending maybe on total DB size or value size distribution, I don't know.
RocksDB is difficult to analyze like this, since it does not seem to behave very deterministic in my lab settings.

I assume they chunk the data and small pieces which fit in one chunk all have the same performance.
ParityDB for example does this in a more predictable way.
It puts data up to 32KiB in its own chunk while larger values are split up into 4KiB chunks. enter image description here

This shows a 100% jump and a clear 4KiB "staircase" at 32KiB for ParityDB.
RocksDB is linear in this range.

That the underlying Database influences the timing so much is another reason to make the Weights more customizable.
The tracking issue for this is Substrate/10921.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.