1

I'm running collators for both kusama and polkadot network, and both started failing to import blocks after running for a while, these are the logs that I'm getting:

2022-12-13 22:49:39 [Relaychain] 💔 Error importing block 0x6843471352677f914d51277815a5ebc17ba6af7564b6fea1bfdf2b544bd9477f: consensus error: Import failed: Import failed: Database    
2022-12-13 22:49:39 [Relaychain] 💔 Error importing block 0x53c82a1ca03c9ddd3c3cfb6c3715a1a1c1ce2320c5cf8da118f95d5b14e47ea0: block has an unknown parent    
2022-12-13 22:49:39 [Relaychain] Block import error: Database    
2022-12-13 22:49:39 [Relaychain] 💔 Error importing block 0x6843471352677f914d51277815a5ebc17ba6af7564b6fea1bfdf2b544bd9477f: consensus error: Import failed: Import failed: Database    
2022-12-13 22:49:39 [Relaychain] 💔 Error importing block 0x53c82a1ca03c9ddd3c3cfb6c3715a1a1c1ce2320c5cf8da118f95d5b14e47ea0: block has an unknown parent    
2022-12-13 22:49:39 [Relaychain] ⚙️  Syncing  0.0 bps, target=#13344031 (24 peers), best: #9352018 (0x973d…5cf4), finalized #9351680 (0x50f4…b6c1), ⬇ 55.2MiB/s ⬆ 49.2kiB/s    
2022-12-13 22:49:40 [Relaychain] Block import error: Database    
2022-12-13 22:49:40 [Relaychain] 💔 Error importing block 0x6843471352677f914d51277815a5ebc17ba6af7564b6fea1bfdf2b544bd9477f: consensus error: Import failed: Import failed: Database    
^[[A2022-12-13 22:49:40 [Relaychain] Block import error: Database    
2022-12-13 22:49:40 [Relaychain] 💔 Error importing block 0x6843471352677f914d51277815a5ebc17ba6af7564b6fea1bfdf2b544bd9477f: consensus error: Import failed: Import failed: Database    
2022-12-13 22:49:41 failed to associate send_message response to the sender    
2022-12-13 22:49:41 [Parachain] 💤 Idle (0 peers), best: #0 (0xba71…7873), finalized #0 (0xba71…7873), ⬇ 0 ⬆ 0    
2022-12-13 22:49:41 failed to associate send_message response to the sender    
2022-12-13 22:49:41 failed to associate send_message response to the sender    
2022-12-13 22:49:41 failed to associate send_message response to the sender    
2022-12-13 22:49:42 [Relaychain] Block import error: Database    
2022-12-13 22:49:42 [Relaychain] 💔 Error importing block 0x6843471352677f914d51277815a5ebc17ba6af7564b6fea1bfdf2b544bd9477f: consensus error: Import failed: Import failed: Database    
2022-12-13 22:49:42 [Relaychain] 💔 Error importing block 0xbe59ff69c5698389d999ddadf8aa2624994a28656198d8c8e4ccd53619dba4ce: block has an unknown parent    

Does this mean that the database got corrupted? Is there a way to fix or prevent this?

Edited:

The polkadot version being used is: polkadot-v0.9.27. I didn't make any changes to the collators, before the issue started. The parachain the collator is for does not have a slot yet.

It seems that the issue was that the node ran out of disk, how much storage does a collator need, it filled a 50GB disk in a couple of days, is this usual?

Thanks.

4
  • Did you get this issue after making some kind of a change? What version of Polkadot are you using i.e. polkadot-v0.9.34. Please include as much information as you can to be able to reproduce your issue.
    – Bruno
    Dec 13, 2022 at 23:23
  • Are you able to reproduce this issue ? If in addition to what @Bruno has requested, you could provide logs with the following targetsdb=trace,sync=trace would add more information. On top of that, I am curious to know if forcing a fresh sync changes this behavior, understand getting rid of your actual db. Dec 14, 2022 at 9:44
  • I've updated the post, it seems that the issue was that the node ran out of disk
    – Sebastian
    Dec 14, 2022 at 20:20
  • That is great that you figured out the issue. Can you describe how you were able to figure out what the issue was? This can be helpful for others.
    – Bruno
    Dec 14, 2022 at 20:23

1 Answer 1

1

It seems that the issue was that the node ran out of disk, how much storage does a collator need, it filled a 50GB disk in a couple of days, is this usual?

It should fill that in a day at most. Even a pruned relay node for Kusama or Polkadot requires more than that.
You should plan to have AT LEAST 250 GB, but 500 would be more sane IMO. The reference hardware recommends a terrabyte to be future proof.
In newer Substrate versions you can opt for a remote relay node, if you have access to a trusted one. This would only require local space for the parachain data, which will minimize the required space.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.