How to start block on_finalize before we reach block time?

Question

I am currently trying to understand how we could improve our block production performance at Avail.

By looking at this answer on a previous question, I get that for a 6 second block time, we allocate 2sec for block production, 2sec for syncing and 2sec for block propagation.

From what I can witness by adding some metrics to a node is that block propagation and import happen no matter what after the on_finalize hook is called, which is logic, but it's always called at the end of a block.

Also by checking this answer, and more especially this bit :

Let's assume we give the block production 500 ms. Of this 500ms 67% are allocated for the actual block initialization plus transaction execution. If we are now above these 67% of execution time we will stop the block production and finalize the block. Then you send out the block to the network.

I wanted to know how can this be actually triggered or if this even mean that block production can be stopped mid blocktime and start the on_finalize processes. For example, I filled a block with the maximum normal ratio (85%) but it still waited to call the on_finalize.

To give some context, in our chain, after all the transaction in a block, we want to do some additional process which is done right now in the on_finalize hook, the goal would be to do it just before or ideally trigger the on_finalize sooner.

My question being : Is there a way to adjust the timing of on_finalize to start it before reaching the end of the block time, particularly after reaching a certain block fill ratio ?

Don't hesitate in case more details are needed !

Answer 1

I can add a new perspective to this by illuminating how block production works in Substrate. Hopefully it would also answer your question:

First, make sure to read up on the division of Runtime and node here, and how they communicate together.

The block authoring is done in a client component called sc_basic_authorship and sc_block_builder. It follows the following broad rules:

ruintime_apis::initialize_block()
for tx in txs: 
   runtime_apis::apply_extrinsic(tx)
end for
runtime_apis::finalize_block()

(ref)

The inner loop of this pseudocode can either because of two reasons:

1. This component is configured to only run up until a fixed slot_duration of time for applying transactions. If this deadline is reached, the block production stops. The 2/3 ratio is applied here. This is the number that you typically see set to 6000ms and is returned here.
2. More commonly, at any given time, the runtime might signal to the block proposer that it is full. This error is returned when the weight limti is reached in the runtime, aka. the 2s block weight limit. See here. This translates to ExhaustResources errors, which is intercepted by the block building process and will also cause the block production to stop. This means the inner loop of the above code stops. Afterwards, the block finalization will runn as planned. Note that block finalization (FRAME's Hooks::on_finalize) is a mandatory operation and must happen.

This is crucially why the weight of on_finalize must already be accounted for as a part of on_initialize. Because it is executed after the runtime has already decided that it is out of weight.