Scalability beyond Asynchronous Backing

Question

With the release of Asynchronous Backing, a good performance increase is expected (lower block time, increased block weight).

Which further efforts are being undertaken with regards to Polkadot scalability?

From @alexander-theißen, I learned that:

• At the pallet-contracts layer, there's work done on: code merkelization, user space gas metering, using register based byte code for wasmi, maybe a switch to a jit compiler, speeding up validation and instrumentation.
• At the FRAME-level, there's work to reduce the weights of all kind of extrinsics
• On the protocol level there seems to be something like "nested parachains" being developed.

I'd love to learn more about the scalability efforts. Further links and resources would be much welcomed.

Answer 1

We have a number of protocol updates that will help improve scalability at the protocol level.

Torrent-style fetching of PoVs: https://github.com/paritytech/polkadot/issues/3307 . This includes a "PreVF" mechanism that can be used to prove light client validity of a parachain header to a validator, which it can then use to participate in torrent-style networking to download the PoV (the Proof-of-Validity). Faster PoV downloads/uploads means more data and more time spent executing - higher throughput.

"Fat availability cores": https://github.com/paritytech/polkadot/issues/4951

This is an extension to asynchronous backing that allows parachains to progress faster than the relay-chain by having more than one block pending availability at a time.

"Methods to avoid ever including parachain code in critical-path data": https://github.com/paritytech/polkadot/issues/3310 . This makes code upgrades for parachains cheaper and less likely to clog up the gossip network.

There are also a lot of general engineering improvements (Wasm execution speed, database accesses, etc.) which will help with scalability.

There are rough plans for Polkadot 2.0, which will involve nested relay chains or at least multiple relay chains sharing a validator-set. But nothing written or published yet.

Answer 2

We expect careful benchmarking work to expose many further optimizations, which falls under rob's "general engineering improvements".

We've experimental low level ideas, like I suspect QUIC would improve networking performance somewhat, which requires either developing nQUIC or else using TLS 1.3.

XCMP shall permit far more messages than XCM, because transport happens off-chain.

We've "whole block optimizations" which typically also help resist MEV. These come in cryptographic flavors like Schnorr half-aggregation or SnarkPack but also non-cryptographic ones, like doing uniswaps with all buys & sell in a block receiving the same common price, thus making the tx commutative & thread friendly.

We'll move heavy relay chain functionality onto parachains, what we call system parahreads.

Sassafras permits removing memepools, which saves busy parachains considerable CPU time.

We'll do multiple relay chains eventually. It'll resemble OmniLedger, meaning it's an algorithm for shuffling validators between relay chains, for which the security analysis uses a stronger honest assumption and concentration bounds. We think the OmniLedger analysis came out somewhat flawed however, so redoing it.. very slowly.