1

it seems to be a Github environment setup problem for running Zombienet test with my custom parachain.

I have made a template repo trying to reproduce this problem.

When I used my template repo to run Zombienet tests with its own docker image, it passed. My settings in .github/workflow/test-code.yml

  POLKADOT_VERSION: "v0.9.22"
  COLLATOR_VERSION: "z1template"

when I changed the images to my custom image with corresponding Polkadot image, it failed...

  POLKADOT_VERSION: "v0.9.21"
  COLLATOR_VERSION: "z21"

But that z21 collator image can pass all Zombienet tests in my local machine, but the same image cannot run properly in Github's kubectl & minikube environment for unknown reason...

Here are the error message... I use this repo to run the same Zombienet tests with different collator and polkadot images: https://github.com/AuroraLantean/substrate-parachain-template-zombienet2/runs/6772959979?check_suite_focus=true

Regarding the kubectl and minikube setup in the workflow .github/workflow/test-code.yml

      - name: Start minikube
        uses: medyagh/setup-minikube@master

This was working until changing "parachain-collator" and "parachain" to our custom name.

my error message:


✓ Added Genesis Parachain 2000
    launching temp-3 pod with image docker.io/parity/polkadot:v0.9.21
         with command: bash -c polkadot build-spec --chain /cfg/rococo-local-plain.json --disable-default-bootnode  --raw > /cfg/rococo-local-raw.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
uploading to fileserver: /tmp/zombie-1e58b1b5b91023abb10deeeb5f6a16ec_-8674-VFF15057SbAm/rococo-local-plain.json as:4a888a6781a58f346fa733098c0ae468338b8690d4b74f97473f6b191579d3d4
temp-3 pod is ready!
Error: Command failed with exit code 1: kubectl --kubeconfig /home/runner/.kube/config --namespace zombie-1e58b1b5b91023abb10deeeb5f6a16ec cp temp-3:/cfg/rococo-local-raw.json /tmp/zombie-1e58b1b5b91023abb10deeeb5f6a16ec_-8674-VFF15057SbAm/rococo-local.json -c temp-3
error: unable to upgrade connection: container not found ("temp-3")
    at makeError (/snapshot/zombienet/node_modules/execa/lib/error.js:60:11)
    at handlePromise (/snapshot/zombienet/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  shortMessage: 'Command failed with exit code 1: kubectl --kubeconfig /home/runner/.kube/config --namespace zombie-1e58b1b5b91023abb10deeeb5f6a16ec cp temp-3:/cfg/rococo-local-raw.json /tmp/zombie-1e58b1b5b91023abb10deeeb5f6a16ec_-8674-VFF15057SbAm/rococo-local.json -c temp-3',
  command: 'kubectl --kubeconfig /home/runner/.kube/config --namespace zombie-1e58b1b5b91023abb10deeeb5f6a16ec cp temp-3:/cfg/rococo-local-raw.json /tmp/zombie-1e58b1b5b91023abb10deeeb5f6a16ec_-8674-VFF15057SbAm/rococo-local.json -c temp-3',
  escapedCommand: 'kubectl --kubeconfig "/home/runner/.kube/config" --namespace zombie-1e58b1b5b91023abb10deeeb5f6a16ec cp "temp-3:/cfg/rococo-local-raw.json" "/tmp/zombie-1e58b1b5b91023abb10deeeb5f6a16ec_-8674-VFF15057SbAm/rococo-local.json" -c temp-3',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: 'error: unable to upgrade connection: container not found ("temp-3")',
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}

Then I reverted my repo to the commit before using custom parachain name... error message:

✓ Added Genesis Parachain 2000
    launching temp-3 pod with image docker.io/parity/polkadot:v0.9.20
         with command: bash -c polkadot build-spec --chain /cfg/rococo-local-plain.json --disable-default-bootnode  --raw > /cfg/rococo-local-raw.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
uploading to fileserver: /tmp/zombie-a05726da326756e3d08fc7b074628325_-5931-GI5IM7Cfo2Qi/rococo-local-plain.json as:9a5d73ac877a90feef59aa30ada40121958c41c0feb8180cb548fd69aca0cb92
temp-3 pod is ready!
Error: Invalid chain spec raw file generated.
    at /snapshot/zombienet/dist/providers/k8s/chain-spec.js:102:19
    at Generator.throw (<anonymous>)
    at rejected (/snapshot/zombienet/dist/providers/k8s/chain-spec.js:6:65)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Not sure if this is causing issues, but noticed this in Zombienet code https://github.com/paritytech/zombienet/blob/main/src/providers/k8s/chain-spec.ts#L20 chain-spec.ts

//We have two options to get the chain-spec file, neither should use the `raw` file/argument

I see "--raw" in the log:

with command: bash -c polkadot build-spec --chain /cfg/rococo-local-plain.json --disable-default-bootnode  --raw > /cfg/rococo-local-raw.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished

This function was triggered by something else, not by me directly. So there must be a setting somewhere. Do I need to change the chainName argument in chain-spec.ts#L17 ?

export async function setupChainSpec(    namespace: string,    networkSpec: ComputedNetwork,    chainName: string,     chainFullPath: string){...}

I found out that one commit from last week can run Zombienet tests in cicd... I just used the commit to run the tests and it passed! but after that commit, the new codes all failed the tests(although they can pass tests in my local machine)

So what part of the code could affect Zombienet tests in Github's CICD?

please advise.

1 Answer 1

1

The cause 1: git rebase did not work properly!

Solution: overwrite the code with your latest main branch code, then add your Zombienet & docker files:

add new files: .dockerignore, docker/, zombienet/

modify files: .github/workflows/ci.yml, .gitignore

For custom parachain name, just copy the compiled binary into the regular parachain-collator in your new images:

COPY --from=builder /builds/collator/target/release/customName /usr/local/bin/parachain-collator

This cicd code works in Github action:

      - name: Start minikube
        uses: medyagh/setup-minikube@master

The cause 2: git cicd environment is not good for such high disk, high memory, and high bandwidth process!

Solution: use Google cloud or other cloud virtual machines to run your Github's cicd with Zombienet

See here about hosting your own (CICD) runners and customize the environment used to run jobs in your GitHub Actions workflows. https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.