Transaction pool management

Question

I have to send thousands of transactions to a custom substrate chain, it will probably take a few days to process all of them. Of course I can't just send all the transactions at once because they probably won't fit in the transaction pool and also if I am overwhelming the tx-pool there's a chance that some of the transactions are dropped from the pool which (considering the high volume of transactions to process) would not be easy to handle.

The extrinsic that I will be executing this many times is pretty simple, in only stores data and emits an event

What would be a good approach to managing the amount of transactions to send to the transaction pool at a certain point and what considerations should I take into account?

Answer 1

You can set --pool-kbytes and --pool-limit in your node, to allow it to not drop some extrinsics.

But better than that I'd recommend you to keep track of the txpool size by calling the RPC method author_pendingExtrinsics.

That way you will be able to stop sending more extinsics when the txpool is full (or near to that), waiting for some time and then start sending them again.

PS: evm-tps has a yarn substrate part where we use that approach.