try-runtime fails with error RestartNeeded - Invalid request ID

Question

I am trying to run try-runtime locally to see the execution results of a particular storage migration, using live subcommand to upload a production state from a remote node.

I am getting an error: RestartNeeded("Invalid request ID")

What could be wrong?

╰─➤  RUST_LOG=runtime=trace,try-runtime::cli=trace,executor=trace \                                                            1 ↵
./target/release/polkadot \
try-runtime \
--execution Native \
--chain kusama-dev \
on-runtime-upgrade \
live \
--uri wss://kusama-rpc.polkadot.io:443
2022-12-06 15:47:45.003 TRACE main runtime::staking: [0] 💸 inserting genesis staker: be5ddb1579b72e84524fc29e78609e3caf42e85aa118ebfe0b0ad404b5bdd25f (GsvVmjr1...) => 100000000000000 => StakerStatus::Validator    
2022-12-06 15:47:45.003 DEBUG main runtime::bags_list: [0] 👜 [VoterList]inserted be5ddb1579b72e84524fc29e78609e3caf42e85aa118ebfe0b0ad404b5bdd25f (GsvVmjr1...) with score 100000000000000 into bag 103277785738759, new count is 1    
2022-12-06 15:47:45.003 TRACE main runtime::staking: [0] 💸 planning new session 0 at genesis    
2022-12-06 15:47:45.003 DEBUG main runtime::staking: [0] 💸 Starting the first era.    
2022-12-06 15:47:45.004  INFO main runtime::staking: [0] 💸 generated 1 npos voters, 1 from validators and 0 nominators    
2022-12-06 15:47:45.004  INFO main runtime::staking: [0] 💸 generated 1 npos targets    
2022-12-06 15:47:45.004 TRACE main runtime::staking: [0] 💸 planning new session 1 at genesis    
2022-12-06 15:47:45.004  WARN main runtime::parachains: Took active validators from set with wrong size    
2022-12-06 15:47:45.004  WARN main runtime::parachains: Took active validators from set with wrong size    
2022-12-06 15:47:45.004  WARN main runtime::parachains: Took active validators from set with wrong size.    
2022-12-06 15:47:45.004  WARN main runtime::parachains: Took active validators from set with wrong size    
2022-12-06 15:47:45.004 TRACE main runtime::staking: [0] 💸 starting session 0    
2022-12-06 15:47:45.004 DEBUG main runtime::staking: [0] 💸 found 0 slashes scheduled to be executed in era 0    
2022-12-06 15:50:45.349  INFO main jsonrpsee_client_transport::ws: Connection established to target: Target { sockaddrs: [], host: "kusama-rpc.polkadot.io", host_header: "kusama-rpc.polkadot.io:443", _mode: Tls, path_and_query: "/" }
2022-12-06 15:50:45.383  INFO main remote-ext: scraping key-pairs from remote @ 0x4a65c043ab64f28f988e5ae54958f8d23e11cfbdba41fe711c06ddaabbedf009    
2022-12-06 15:50:45.383  INFO main remote-ext: downloading data for all pallets.    
2022-12-06 15:56:45.393  WARN tokio-runtime-worker jsonrpsee_core::client::async_client: InvalidRequestId
2022-12-06 15:56:45.393 ERROR                 main remote-ext: failed to execute batch: [c2261276cc9d1f8598ea4b6a74b15c2f218f26c73add634897550b4003b26bc6fb00ce09311f4fbcb468430ee0ac7190f42b7fead1bb8eec5d895fb3d3bd145e85f0b6d5c8bb3d7edb4a6fa9dda2c021,
....
.... lots of data
....
296a81d6ead301607072d2bbf9dd308c3606eb1b795a62bcf2f]. Error: RestartNeeded("Invalid request ID")    
Error: 
   0: Invalid input: batch failed.

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Answer 1

You are using the public Kusama endpoint; this will not work. The node times out before the batch which tries to download all the state can complete.
Please follow this description about how to use it: Testing a migration with try-runtime.

Answer 2

The underlying reason for this error is the target not not being able to send back large payloads. You need to run against nodes that execute with:

--rpc-max-request-size 100000 
--rpc-max-response-size 100000

Answer 3

What also has helped me, is pallet option of a live subcommand (uploads only a state of a pallet). Since my test was requiring only a state of a particular pallet, with the option I could do that.