Zombienet: cannot exec into a container in a completed pod; current phase is Succeeded

Question

My network definition file:

[relaychain]
default_image = "docker.io/parity/polkadot:v0.9.18"
default_command = "polkadot"
default_args = [ "-lparachain=debug" ]

chain = "rococo-local"

  [[relaychain.nodes]]
  name = "alice"
  validator = true

  [[relaychain.nodes]]
  name = "bob"
  validator = true

[[parachains]]
id = 2000

  [parachains.collator]
  name = "collator01"
  image = "gcr.io/p-network-202104/collator:latest"
  command = "adder-collator"
  
[types.Header]
number = "u64"
parent_hash = "Hash"
post_state = "Hash"

Then I tried to run that network:

$ ./zombienet-linux1232 -p kubernetes spawn zombienet/0001-small-network.toml
{
  name: 'alice',
  key: '2bd806c97f0e00af1a1fc3328fa763a9269723c8db8fac4f93af71db186d6e90',
  accounts: {
    seed: '0xa9e19ee39ecb24b7be164e6d0fef2e90de607f380c4e7591feff539bbadf5cb9',
    sr_account: {
      address: '5HnGdsS8zYozLApMaHxDn2eeKujz38mWzqQnspstFtKyBxLi',
      publicKey: '0xfce092ed3296c4285c9bf6eb365e098231878a67337633381c517722a83c1508'
    },
    sr_stash: {
      address: '5GKMjxQCKiJFgvu6UJdjdAjkVtRw8EiffdA7K8i17Fjfq1j9',
      publicKey: '0xbc1de63683a13c534954aa4481f2618d20550a64b9f890797a849274b17f5014'
    },
    ed_account: {
      address: '5FBjn9VWAeTP8fVUGa9JJznRYVWAppkFUGSVvZ7CeUgPFqK1',
      publicKey: '0x8a11e36ba8eda17f9a5c62fce3098aaaebd1080d0d89b15d3c8f0fb432a1deda'
    },
    ec_account: { publicKey: [Uint8Array] }
  },
  command: 'polkadot',
  commandWithArgs: undefined,
  image: 'docker.io/parity/polkadot:v0.9.18',
  chain: 'rococo-local',
  validator: true,
  args: [ '-lparachain=debug' ],
  env: [
    { name: 'COLORBT_SHOW_HIDDEN', value: '1' },
    { name: 'RUST_BACKTRACE', value: 'FULL' }
  ],
  bootnodes: [],
  telemetryUrl: '',
  telemetry: false,
  prometheus: true,
  overrides: [],
  addToBootnodes: false,
  resources: undefined,
  zombieRole: 'node',
  group: 'alice'
}
{
  name: 'bob',
  key: '81b637d8fcd2c6da6359e6963113a1170de795e4b725b84d1e0b4cfd9ec58ce9',
  accounts: {
    seed: '0x37b832313162309329437ce00e18f93c952fb27ec123da882091515d78b67965',
    sr_account: {
      address: '5FhfT9HUqdBS2tSqNugHJEKmyrUYuJzTrQvhiakLbb8FHYHm',
      publicKey: '0xa0e4aead4127aa67e8612d840a27e1b78f8fe42855f0999a65a82db2761bb90b'
    },
    sr_stash: {
      address: '5ECnDLoU7e6RP4NNKonA27i7sGXSj43J5dcAvyr86MuTgziP',
      publicKey: '0x5ea1186076be44587524d22c84c39aaf4ae1c4c47369b1e04eabb6658e07a50f'
    },
    ed_account: {
      address: '5DoTVfs8SqxQmDDSYZjWFPyp7CNY6edcXmaBVcqfDwzPaC77',
      publicKey: '0x4cd7705dc5551ec3f5e1f5b4d08d7a4adb97a021cf8348f17b91d1d03152bef4'
    },
    ec_account: { publicKey: [Uint8Array] }
  },
  command: 'polkadot',
  commandWithArgs: undefined,
  image: 'docker.io/parity/polkadot:v0.9.18',
  chain: 'rococo-local',
  validator: true,
  args: [ '-lparachain=debug' ],
  env: [
    { name: 'COLORBT_SHOW_HIDDEN', value: '1' },
    { name: 'RUST_BACKTRACE', value: 'FULL' }
  ],
  bootnodes: [],
  telemetryUrl: '',
  telemetry: false,
  prometheus: true,
  overrides: [],
  addToBootnodes: false,
  resources: undefined,
  zombieRole: 'node',
  group: 'bob'
}

     Using provider: kubernetes

     Launching network under namespace: zombie-f081cefea6b6ef81e4c104376994654b
         Using temporary directory: /tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a

    launching temp pod with image docker.io/parity/polkadot:v0.9.18
         with command: bash -c polkadot build-spec --chain rococo-local --disable-default-bootnode > /cfg/rococo-local-plain.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
        temp pod is ready!

        🧹 Starting with a fresh authority set...
              👤 Added Genesis Authority alice - 5GKMjxQCKiJFgvu6UJdjdAjkVtRw8EiffdA7K8i17Fjfq1j9
              👤 Added Genesis Authority bob - 5ECnDLoU7e6RP4NNKonA27i7sGXSj43J5dcAvyr86MuTgziP

    launching temp-collator pod with image gcr.io/p-network-202104/collator:latest
         with command: bash -c adder-collator export-genesis-state > /cfg/genesis-state && adder-collator export-genesis-wasm > /cfg/genesis-wasm && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
        temp-collator pod is ready!
Error: Command failed with exit code 1: kubectl --kubeconfig /home/user2038/.kube/config --namespace zombie-f081cefea6b6ef81e4c104376994654b cp temp-collator:/cfg/genesis-state /tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a/2000/genesis-state
error: cannot exec into a container in a completed pod; current phase is Succeeded
    at makeError (/snapshot/zombienet/node_modules/execa/lib/error.js:60:11)
    at handlePromise (/snapshot/zombienet/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  shortMessage: 'Command failed with exit code 1: kubectl --kubeconfig /home/user2038/.kube/config --namespace zombie-f081cefea6b6ef81e4c104376994654b cp temp-collator:/cfg/genesis-state /tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a/2000/genesis-state',
  command: 'kubectl --kubeconfig /home/user2038/.kube/config --namespace zombie-f081cefea6b6ef81e4c104376994654b cp temp-collator:/cfg/genesis-state /tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a/2000/genesis-state',
  escapedCommand: 'kubectl --kubeconfig "/home/user2038/.kube/config" --namespace zombie-f081cefea6b6ef81e4c104376994654b cp "temp-collator:/cfg/genesis-state" "/tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a/2000/genesis-state"',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: 'error: cannot exec into a container in a completed pod; current phase is Succeeded',
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}

     Node's logs are available in /tmp/zombie-f081cefea6b6ef81e4c104376994654b_-32386-La9yjKBxWV2a/logs

It seems there is no process running and that causes the "cannot exec" error... but why???
What is wrong in my network definition?

I suspect the relaychain.nodes command should be:

commandWithArgs = './target/release/polkadot --alice --validator --base-path /tmp/relay/alice --chain rococo-custom-2-raw.json --port 30333 --ws-port 9944 --unsafe-ws-external --rpc-cors all --rpc-external --allow-private-ipv4 --discover-local --rpc-methods Unsafe > alice.out 2>&1; tail -f /dev/null'

Or my parachains.collator command should be:

command = './target/release/parachain-collator --alice --collator --force-authoring --chain rococo-local --unsafe-ws-external --parachain-id 2000 --rpc-external --allow-private-ipv4 --discover-local --rpc-methods Unsafe --base-path /tmp/parachain/alice --port 40333 --ws-port 8844 --rpc-port 6969 --rpc-cors all -- --execution wasm --chain res/dev/rococo-custom-2-raw.json --port 30343 --ws-port 9977 > collator.out 2>&1; tail -f /dev/null'

any idea? Thank you

Answer 1

the issue here is the command for exporting the genesis-state / genesis-wasm.

launching temp-collator pod with image gcr.io/p-network-202104/collator:latest
         with command: bash -c adder-collator export-genesis-state > /cfg/genesis-state && adder-collator export-genesis-wasm > /cfg/genesis-wasm && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished

Does this image gcr.io/p-network-202104/collator:latest have the adder-collator binary?

Answer 2

You need to add your collator command, which is target/release/parachain-collator if your parachain is based on substrate-parachain-template. Notice there is no DOT "." before the command!

  [parachains.collator]
  name = "collator01"
  image = "gcr.io/x-network/collator:latest"
  command = "target/release/parachain-collator"