Is it possible to use the chain specification file or some other mechanism to configure a Substrate network with injected block history? In this case, the motivation is CI/CD/testing, but I can see how this could also be useful for network migrations.
You should be able to do this either by persisting the whole database and always using that as a starting point, or using the export / import functionality supported by Substrate. In order to export the historical block data you could do:
substrate export-blocks --chain my-chain -d my-chain-db blocks.bin
You could then persist this blob somewhere and share it with your test environment. You can then re-import the block data (into a blank database) as the first step before starting your tests:
substrate import-blocks --chain my-chain -d my-chain-db blocks.bin
When using BABE this isn't possible / pratical due to the liveness requirements of the protocol (i.e. if the chain is down for more than one epoch then it gets bricked and you get the dreaded "unexpected epoch change" error).
You can build up a local chain with the
--dev flag and then import it in your CI.
There are several ways to restore snapshots, depending on your needs.
- For Polkadot/Kusama you can download a chain snapshot from Polkachu. This would be a real chain snapshot, so probably to heavy for testing.
- Import some blocks with the
import-blockscommand that were exported with the
export-blockscommand before. This works fine for real chain specs, but
--devcreate a new genesis and therfore the import fails.
export-stateand pass that the resulting file to
--chain. This will put all the chain state into the genesis block. The new chain will have the same state as the old one, but start at block zero.
- Just copy paste the
~/.local/share/polkadot/chains/directory from some backup. Works for real specs. For
--devspecs you need to copy the directory you specify with
You can also use fork-off-substrate script to generate chainspec containing storage of any running chain - even your live network.
Once great resource and reference on generation of snapshots is the polkadot-snapshot-generator tool, that is hosted on https://polkashots.io/ for Polkadot and Kusama (and some other networks/parachains may as well.
When running a node,
--base-path <my/base/path> will generate the following structure with the
db directory containing the
<my/base/path>/chains ==> <the chains set from chainspec> ==> db | ==> keystore | ==> network
You want to copy only the
db section here, assuming that you do not want to share the
keystore with your priv keys (potentially sensitive!) and
network with your network libp2p info and peering priv keys.