I know that the best practices suggest designing transactions (pallet calls) in a way that executes the operation atomically and doesn't keep any intermediate states. This guarantees, that in case a block where the atomic operation has been applied, is reverted by the Block Finalization Consensus mechanism (i.e. GRANDPA or any other), the operation will not hang in an intermediate state. It will be fully rolled back due to its atomicity, and the transaction sender can retry it later.
However, in a system that I'm currently working on, there is a requirement to process a single operation in several transactions/blocks. That means I need to keep some intermediate state (in the pallet's storage) between several transactions/blocks before stating the operation as completed.
My concern here is that because I keep an intermediate state of a single operation during several transactions/blocks, some of those transactions/blocks may be reverted due to the Block Finalization Consensus mechanism. In this case, I won't be able to state the operation as completed in the last transaction/call for that operation, as I'm missing some important changes from all intermediate calls/transactions for that operation that were reverted.
It seems to me, that the most obvious way to address this issue is to send every new transaction/call for the operation only after the block where the previous transaction/call for that operation occurred is finalized by the Finalization Consensus (GRANDPA). And if there is some block that is reverted during the whole operation, implement some kind of retry strategy.
To implement that, I plan to rely on the Substrate subscription API described in the Transaction inclusion section in polkadot.js docs.
Can someone confirm my thoughts regarding the above and suggest any better approach if there are any?
Is there something I need to be aware of, in case I rely on the result.status.isFinalized field from the polkadot.js docs when I send several subsequent transactions/calls (that are parts of the same domain operation), and wait for every transaction/call to be finalized (i.e. updates the intermediate state of domain operation), before sending the next transaction/call?
TL;DR: Is there any good way to implement sending a stream of data via an extrinsic stream_data, that should be called several times while transferring the stream (every call transfers a part of the stream), and each time it updates some intermediate state of buffer in the storage?
To complete the data streaming and do something with the accumulated buffer, there will be another extrinsic end_stream.
How to implement this properly, considering the fact that the intermediate state of the buffer can be inconsistent in case the GRANPDA reverts some blocks with previously executed stream_data extrinsics?
