2

We observe a strange phenomenon: We send many lightweight extrinsics (~2 per block) signed by the same account to one full node (node1) which is peered up with two authority collators (auth1 and auth2) producing blocks regularly.

At some point, the extrinsics stop being included.

The last extrinsic nonce included is 3836

then I ask all nodes what the next nonce should be.

curl http://localhost:9933 -X POST -H "Content-Type: application/json" --data '{"method":"system_accountNextIndex","params":["5Gtt7oCnD3aZPUMx5B2QrXAt5nfoi4LfBSG8xeTMLmDq9qJN"],"id":1,"jsonrpc":"2.0"}'

auth1 and node1 both return 3837

but when I query node1 for pending extrinsics using author_pendingExtrinsics

I get a list of 8044 extrinsics and the first one in the list indeed has nonce 3837

querying auth1 and auth2 for pending xt returns none. Verifying again: they are peered up mutually querying system_peers

I can now pick the first pending extrinsic from node1's list and manually send it to auth1 using author_submitExtrinsic, it gets included. This tells me there is no issue with that extrinsic. But this doesn't lead to an avalanche of the remaining extrinsics, I can only pick one by one manually and they will be included (so they are not banned).

another strange behavior: If we send a different extrinsic with the expected next nonce, we see the avalanche of pending extrinsics included immediately (except that one extrinsic with the same nocne, obviously)

environment: Paseo relay chain (polkadot-sdk-1.9.0), integritee parachain (polkadot-sdk-1.6.0, same with polkadot-sdk-1.9.0)

Is there a way to force a node to re-broadcast their pending extrinsics?

Is there a hidden rate limiting implemented that postpones or bans extrinsics from the same sender?

Can this have anything to do with "retracted" status of extrinsics?

Observation: If I restart only auth1 the pending extrinsics get included in the next block(s)

1 Answer 1

1

Basically your issue should be this. So, both authority nodes are seeing the transactions at some point, but then removing all of them as invalid (which is a bug).

The reason why node-1 is not sending them again is that it already send the transactions to both collators and thus, it will not send them again (it keeps internally some list of send transactions per node). By sending them manually, you are basically skipping this list.

Is there a way to force a node to re-broadcast their pending extrinsics?

No

Observation: If I restart only auth1 the pending extrinsics get included in the next block(s)

Because node-1 clears the internal list of already send transactions to the collator and then, sends them again.

1
  • thank you. our current workaround is a cronjob that restarts one of the collators every 5 min. Waiting for a proper fix
    – brenzi
    Apr 3 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.