Testing a migration with `try-runtime`

Question

I have written a migration for the phragmen pallet in Substrate and now I want to test it. I would want to use the current state of Kusama/Polkadot to make sure the migration works correctly. To achieve this AFAIU I need to use the try-runtime feature.

I have a problem and a question regarding the try-runtime feature.

1. When I specify the --uri argument to be equal to wss://rpc.polkadot.io(as provided in the documentation) I get the following error:

2022-12-03 10:28:32 error: "`WsClientBuilder` failed to build: Transport(Invalid URL: No port number in URL (default port is not supported))"    
Error: Input("failed to build ws client")

2. When running the on-runtime-upgrade subcommand which runtime upgrade is being executed and how can I specify it so that it executes the on_runtime_upgrade function that I have wrote for my migration?

Answer 1

Good question! try-runtime is the right idea indeed. (Note for Parity devs at the end)

Currently the try-runtime CLI scraps the chain state from the an RPC node. Using the public node is probably really slow.
This is the procedure I use with success:

0. Write a new migration in Substrate, eg MyMigration.
1. Integrate the migration into the runtime by adding it to the Executive type. This needs to be done for all runtimes where you want the migration to execute. Example from Kusama:

pub type Executive = frame_executive::Executive<
    Runtime,
    Block,
    frame_system::ChainContext<Runtime>,
    Runtime,
    AllPalletsWithSystem,
    (
        MyMigration, // Added here
        SomeOtherMigration
    ),
>;

2. Run diener in the Polkadot repo to patch your Substrate version:

# If you have it on a branch:
diener update --substrate --branch tmp-oty-debug-bench-bot-revert
# ... or in a local folder:
diener patch --crates-to-patch ../substrate --substrate

3. Build a release binary with the try-runtime feature. So for example: cargo b -r --bin polkadot --features try-runtime.
4. Start a local Polkadot node by using the current release binary. This node will be used instead of a public endpoint to speed things up tremendously. The node requires some extra arguments. This works well for me (could probably be optimized):

./polkadot --chain polkadot --db paritydb -d empty-dir/ --sync warp --rpc-max-request-size 100000 --rpc-max-response-size 100000 --rpc-external --rpc-cors=all --unsafe-ws-external

Wait until you see Warp sync is complete. Which looks like this:

⏩ Warping, Importing state, 1044.21 Mib (32 peers), best: #15600673 (0xef7d…adca), finalized #15600673 (0xef7d…adca), ⬇ 13.8kiB/s ⬆ 0.7kiB/s    
Warp sync is complete (1044 MiB), restarting block sync.    
⏩ Block history, #2688 (32 peers), best: #15600705 (0x53c2…55b1), finalized #15600701 (0x55fc…001a), ⬇ 958.7kiB/s ⬆ 18.3kiB/s

5. Start the try-runtime CLI from your locally build binary in a second terminal with the same chain-spec:

RUST_LOG=remote-ext=debug,runtime=trace ./target/release/polkadot try-runtime --chain=polkadot-dev --execution=Wasm --no-spec-check-panic on-runtime-upgrade live --uri ws://localhost:9944

This will first download the whole state from your local node and then run all migrations. Should look like this:

# Lots of these:
DEBUG main remote-ext: new total = 241000, full page received: …

# Lots of these:
DEBUG main remote-ext: progress = 0.05 [40000 / 806314] …

# Then the interesting part:
main polkadot_runtime: try-runtime::on_runtime_upgrade polkadot

# Then you will see the log output of your migrations
# Finally if it was successful: 
main try-runtime::cli: TryRuntime_on_runtime_upgrade executed without errors. 
Consumed weight = (83471000 ps, 0 byte), total weight = (2000000000000 ps, 5242880 byte) (0.00 %, 0.00 %).

6. Repeat steps 4 and 5 for all runtimes by changing the --chain arguments.

Note: If you are a Parity dev you can use the try-runtime bot. Usage looks like this.
In the future the try-runtime checks will run automatically on all Polkadot MRs in the CI, which should make manual testing superfluous.