What are the benefits of rocksdb over leveldb? Can Substrate use leveldb?
I understand that rocksdb was the database of choice for Parity Ethereum client, and paritydb aims to fix a lot of the overheads and issues that rocksdb has for Substrate archive nodes. But considering that most Ethereum client uses leveldb, I was wondering if there are any technical reasons for this outside of convenience on Parity’s side. Also, is it possible to use leveldb for Substrate?
Leveldb and rocksdb are pretty similar on the architecture level (I believe rocksdb actually shared some code with leveldb in the early days, but don't quote me on that) and picking one over the other probably doesn't matter that much, or at least that was the case back then.
I have not followed leveldb development much in the last few years and it could be that it has improved a lot in the meantime; back when the choice was made, rocksdb had better performance, a better feature set and a very active development team.
The readme on the repo gives the reasons: https://github.com/paritytech/parity-db
Essentially as we have a very constrained use case, we can make optimisations that more generic kv data stores like rocksdb and leveldb can not make. In particular we can make use of the following:
Which means most of the keys are fixed size and uniformly distributed. Most values are small. Values over 16 kbytes are rare.
As we're coding in rust there's every reason to assume that we can improve on the generic databases' performance (which will be written in C/C++). As a rule of thumb not writing your own DB is generally a good idea, but in this case I believe it could well be justified.
