How to unbrick a Substrate Chain? --revert?

Question

I've been running a local relay chain with two validators, and tried doing a runtime upgrade that failed showing the following errors on the validators:

2022-02-11 18:11:48 Running migration to v1 for session_historical with storage version StorageVersion(1)
2022-02-11 18:11:48 Attempted to apply migration to v1 but failed because storage version is StorageVersion(1)

and

panicked at 'Timestamp slot must match `CurrentSlot`', /Users/santibalaguer/.cargo/git/checkouts/substrate-7e08433d4c370a21/4aeb95f/frame/babe/src/lib.rs:845:9
2022-02-11 18:11:54 ❗️ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Other("Wasm execution trapped: wasm trap: wasm `unreachable` instruction executed\nwasm backtrace:\n    0: 0x693c - <unknown>!rust_begin_unwind\n"). Dropping.
2022-02-11 18:11:54 panicked at 'Timestamp must be updated once in the block', /Users/santibalaguer/.cargo/git/checkouts/substrate-7e08433d4c370a21/4aeb95f/frame/timestamp/src/lib.rs:175:13
2022-02-11 18:11:54 Proposing failed: ClientImport("RuntimeApiError(Application(Execution(Other(\"Wasm execution trapped: wasm trap: wasm `unreachable` instruction executed\\nwasm backtrace:\\n    0: 0x693c - <unknown>!rust_begin_unwind\\n\"))))")

Given that this is a local setup and that I was just testing things out, I'd be curious of what's the best way to recover this chain. I tried looking for how to use the --revert flag but could not find much (don't know either if it's the right way).

Answer 1

Recovering from bricked blockchain:

• Time warp - try reversing the system clocks on all the validators to when it was previously able to produce and finalize blocks. See Kusama's First Adventure
• Hard spoon - to avoid purging any state you could try using the export-state command to extract raw storage state from your old chain for use in a new blockchain, or use tools like fork-off-substrate or chainSpecUtils.js
• Purge chain - try purging the blockchain database using the purge-chain command or removing it manually if there isn't any important information you want to try to backup first

Prevent bricking your blockchain:

• Swap from Babe to Aura consensus - try using Aura consensus instead, since if you are using Babe you might brick your blockchain if you stop your validators for more than one epoch and you may also experience an intermittent block delay issue when you're using Babe and aren't running enough validators

Are you using polkadot-launch to run your local relay chain with two validators?