5

I've been working on getting a test relay up and running in the cloud (Docker containers running on Debian virtual machines) for remote testing, and we've got everything running. The relay chain is able to sync, and sees the parachain. The collators see each other. We then noticed the following:

  1. The parachain is trying to collate, but the blocks are never picked up by the relay.
  2. We're getting a log that collation was not advertised to any validators.
  3. That seems to be because finality is dropping behind on the relay chain, to the point where the relay's finalized block was 200 behind the current best.
  4. 2 of our 3 validators are always offline, and it cycles almost perfectly. So the online validator will be A, then next block B, then next block C. That would cause the slow and stalled finality, and probably the lack of collation being advertised.

For reference, we also just registered a parathread on Rococo and put a request through for a temporary parachain slot, currently waiting for onboarding. My hope was to check whether that parachain worked appropriately.

I also tested this exact relay/para chain spec pair locally and it worked perfectly.

Has anyone seen something like this before? Thanks.

EDIT: Both the relay and the parachain are on polkadot-v0.9.37. We're running the parachain with

parachain-node --collator --force-authoring --chain raw-parachain-chainspec.json --base-path /tmp/parachain/alice --port 40333 --ws-port 9944 --rpc-port 9934 --telemetry-url "wss://telemetry.polkadot.io/submit/ 0" --rpc-methods Unsafe --name Fennel0 --node-key $(cat para-key.sh) --unsafe-ws-external --rpc-external --rpc-cors=all -- --execution wasm --chain /app/polkadotspec.json --port 30343 --ws-port 9977 --bootnodes /ip4/[relay_ip]/tcp/30333/p2p/12D3KooWLj8FgbwDtLQbA5ozWqqx6vM1n2rmjwQMczSiDveyWa5G --rpc-external --ws-external --prometheus-external --rpc-cors all

and the relay bootnode with

polkadot --validator --base-path /tmp/relay/alice --chain /app/chainspec.json --port 30333 --ws-port 9944 --rpc-port 9934 --telemetry-url "wss://telemetry.polkadot.io/submit/ 0" --name FnlDot0 --node-key $(cat node_key) --rpc-cors all --prometheus-external --rpc-methods Unsafe --rpc-external --ws-external

Update: After re-running, our Relay network is working again. All three nodes are in and finality has been running fine. Our parachain is, however, still not collating.

Here's a pastebin for the boot node's log: https://pastebin.com/raw/VAC1a50W

And one of the collator logs: https://pastebin.com/raw/EyCUCRKD

The Dockerfile for our relay nodes is https://github.com/fennelLabs/polkadot/blob/release-v0.9.37/Dockerfile

And the Dockerfile for our parachain nodes is https://github.com/fennelLabs/Fennel-Protocol/blob/terraform/Dockerfile

We are exposing more ports at runtime through docker run.

UPDATED LOGS WITH -lparachain::collator-protocol=trace

Bootnode: https://pastebin.com/raw/CnYe4M3r

Collator: https://pastebin.com/raw/EG2C7uY7

And just for more information and context, here's some sample scripts we're using to provision our VMs and run the nodes:

Bootnode: https://github.com/fennelLabs/Fennel-Protocol/blob/terraform/fennel-protocol-boot-terraform-start.sh

Collator: https://github.com/fennelLabs/Fennel-Protocol/blob/terraform/fennel-protocol-collator-1-terraform-start.sh

8
  • 2
    What version of Polkadot is the Relay chain running? What version of Polkadot is your parachain running? Please include logs of both the Relay chain and the parachain from the beginning. Also include the commands you use to start up the validators and collators with the flags.
    – Bruno
    Jun 15, 2023 at 17:02
  • 1
    Edited with version and commands, logs to come.
    – Romulus10
    Jun 15, 2023 at 17:16
  • Added a pastebin showing the log for the relay bootnode and one collator.
    – Romulus10
    Jun 15, 2023 at 22:51
  • 1
    There was a similar issue here. However, in your case, you're running polkadot-v0.9.37. Please create a new issue here with all the details that you have in this post.
    – Bruno
    Jun 16, 2023 at 1:35
  • Done. Issue is github.com/paritytech/polkadot/issues/7384. Thanks @Bruno!
    – Romulus10
    Jun 16, 2023 at 1:56

1 Answer 1

2

What eventually fixed this issue was an upgrade to the latest version of Polkadot. After the upgrade, and after the answer found in https://substrate.stackexchange.com/a/9036/4330, we're now collating properly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.