Previously, I have already made a Rust docker image:
FROM debian:latest
RUN apt update && apt install -y git clang curl libssl-dev llvm libudev-dev procps vim make wget build-essential
RUN curl https://sh.rustup.rs -sSf >> install-rust.sh
RUN chmod u+x install-rust.sh
RUN ./install-rust.sh -y
RUN PATH=$PATH:$HOME/.cargo/env
SHELL ["bash", "-lc"]
RUN rustup update
RUN rustup update nightly
RUN rustup target add wasm32-unknown-unknown --toolchain nightly
CMD [ "tail", "-f", "/dev/null" ]
Then I cloned Substrate-Parachain-Template repo: https://github.com/substrate-developer-hub/substrate-parachain-template and checked out its polkadot-v0.9.18 branch,
Then after adding my own Dockerfile.template in the cloned repo above:
FROM gcr.io/p-network-202104/rust:1.60.0 as builder
ARG PROFILE=release
#older rust made collator:v4
WORKDIR /builds/collator
COPY . .
RUN . $HOME/.cargo/env && cargo build --${PROFILE}
# here I made a mistake copying files from an incorrect path so the docker build process was stopped here
# the resulted local docker image was built up to here...
# That is okay because I tagged it to be used in other docker files below...
# but I should add the EXPOSE and CMD commands below for a normal scenario
EXPOSE 40333-40335 8844-8848 6965-6969 30343-30345 9977-9979 30333 9933 9944
# TODO: Make this run the CMD used in the docker-compose-collator.yaml
CMD tail -f /dev/null
Also with a .dockerignore file below...
[Notice] I added target folder below into the original .dockerignore file from the repo because this can avoid Sending build context to Docker daemon sending huge size context data!
.git
target
**/target/
**/*.txt
**/*.md
/docker/
!/target/release/polkadot-collator
# dotfiles in the repo root
/.*
Then I built template:v1 docker image:
docker build -f Dockerfile.template -t gcr.io/p-network-202104/template:v1 .
because this template:v1 image is huge(11.3GB), it is okay to use it locally without pushing it to the cloud.
My third and final docker image should be slim:
FROM gcr.io/p-network-202104/template:v1 as builder
FROM debian:latest
#FROM debian:stretch-slim
#buster-slimHi @pepoviola, I logged into that image then ran
root@23006a9e7a5d:/# /target/release/parachain-collator
Error: Input("Relay chain argument error: Invalid input: `rococo-local` only supported with `rococo-native` feature enabled.")
2022-05-19 11:31:21 Parachain Collator Template
2022-05-19 11:31:21 ✌️ version 0.1.0-unknown-x86_64-linux-gnu
2022-05-19 11:31:21 ❤️ by Anonymous, 2020-2022
2022-05-19 11:31:21 📋 Chain specification: Local Testnet
2022-05-19 11:31:21 🏷 Node name: hypnotic-glue-7138
2022-05-19 11:31:21 👤 Role: FULL
2022-05-19 11:31:21 💾 Database: RocksDb at /root/.local/share/parachain-collator/chains/local_testnet/db/full
2022-05-19 11:31:21 ⛓ Native runtime: template-parachain-1 (template-parachain-0.tx1.au1)
2022-05-19 11:31:21 assembling new collators for new session 0 at #0
2022-05-19 11:31:21 assembling new collators for new session 1 at #0
root@23006a9e7a5d:/# ls
# v1
# compiled binaries are in ./builds/collator
COPY --from=builder ./builds/collator/target/release/parachain-collator ./target/release/
COPY ./polkadot-launch ./polkadot-launch
RUN ls
EXPOSE 40333-40335 8844-8848 6965-6969 30343-30345 9977-9979 30333 9933 9944
# TODO: Make this run the CMD used in the docker-compose-collator.yaml
CMD tail -f /dev/null
then I built it: docker build -f Dockerfile.slim -t gcr.io/p-network-202104/template:slim1 .
and pushed the new template:slim1 image to the cloud.
Now I am ready for Zombienet...
This Substrate-Parachain-Template uses Cumulus, so I set:
cumulus_based = true in my network definition file:
[relaychain]
default_image = "docker.io/parity/polkadot:v0.9.18"
default_command = "polkadot"
default_args = [ "-lparachain=debug" ]
chain = "rococo-local"
[[relaychain.nodes]]
name = "alice"
validator = true
[[relaychain.nodes]]
name = "bob"
validator = true
[[parachains]]
id = 2000
cumulus_based = true
[parachains.collator]
name = "collator01"
image = "gcr.io/p-network-202104/template:slim1"
command = "target/release/parachain-collator"
[types.Header]
number = "u64"
parent_hash = "Hash"
post_state = "Hash"
With the test file:
Description: Small Network test
Network: ./z04-small-network.toml
Creds: config
# to run this file: ./zombienet-linux -p kubernetes test zombienet/z04-small-network.feature
# well know functions
alice: is up
bob: is up
alice: parachain 2000 is registered within 225 seconds
# logs
bob: log line matches glob "*rted #1*" within 10 seconds
bob: log line matches "Imported #[0-9]+" within 10 seconds
bob: log line matches "Imported new block." within 10 seconds
alice: parachain 2000 block height is at least 6 within 150 seconds
#alice: parachain 2000 block height is at least 10 within 250 seconds
# Tracing
# alice: trace with traceID 94c1501a78a0d83c498cc92deec264d9 contains ["answer-chunk-request", "answer-chunk-request"]
# metrics
alice: reports node_roles is 4
alice: reports sub_libp2p_is_major_syncing is 0
# histogram
#alice: reports histogram polkadot_pvf_execution_time has at least 2 samples in buckets ["0.1", "0.25", "0.5", "+Inf"] within 100 seconds
# system events
bob: system event contains "A candidate was included" within 20 seconds
alice: system event matches glob "*was backed*" within 10 seconds
With the latest zombienet-linux v1.2.34, I ran it: ./zombienet-linux -p kubernetes test zombienet/z04-small-network.feature
The result:
small network( Small Network test )
runningNetworkSpecPath undefined
Launching network... this can take a while.
Using provider: kubernetes
Launching network under namespace: zombie-c1812f6eb712223f7cdb3e97384732a7
Using temporary directory: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt
launching temp pod with image docker.io/parity/polkadot:v0.9.18
with command: bash -c polkadot build-spec --chain rococo-local --disable-default-bootnode > /cfg/rococo-local-plain.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
temp pod is ready!
🧹 Starting with a fresh authority set...
👤 Added Genesis Authority alice - 5GNJqTPyNqANBkUVMN1LPPrxXnFouWXoe2wNSmmEoLctxiZY
👤 Added Genesis Authority bob - 5HpG9w8EBLe5XCrbczpwq5TSXvedjrBGCwqxK1iQ7qUsSWFc
launching temp-1 pod with image gcr.io/p-network-202104/template:slim1
with command: bash -c target/release/parachain-collator build-spec --disable-default-bootnode > /cfg/rococo-local-plain.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
temp-1 pod is ready!
launching temp-2 pod with image gcr.io/p-network-202104/template:slim1
with command: bash -c target/release/parachain-collator build-spec --chain /cfg/rococo-local-2000-plain.json --disable-default-bootnode --raw > /cfg/rococo-local-2000-raw.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
uploading to fileserver: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/rococo-local-2000-plain.json as:cdf720a7997f3f77d6e2509edff0820ba3af8eafaf59de281eabe9aa79e5dfac
temp-2 pod is ready!
launching temp-collator pod with image gcr.io/p-network-202104/template:slim1
with command: bash -c target/release/parachain-collator export-genesis-state --chain /cfg/rococo-local-2000.json > /cfg/genesis-state && target/release/parachain-collator export-genesis-wasm --chain /cfg/rococo-local-2000.json > /cfg/genesis-wasm && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
uploading to fileserver: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/rococo-local-2000.json as:60698b4418761572cdb18c86cbd4ea87fe643ba8e77ffc38965b822f5c50d952
temp-collator pod is ready!
✓ Added Genesis Parachain 2000
launching temp-3 pod with image docker.io/parity/polkadot:v0.9.18
with command: bash -c polkadot build-spec --chain /cfg/rococo-local-plain.json --disable-default-bootnode --raw > /cfg/rococo-local-raw.json && until [ -f /tmp/finished.txt ]; do echo waiting for copy files to finish; sleep 1; done; echo copy files has finished
uploading to fileserver: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/rococo-local-plain.json as:00de39b17f790e9209b5ca034e123711974a556cad8b2519269e00965b5ceb8c
temp-3 pod is ready!
Chain name: Rococo Local Testnet
⚙ Clear Boot Nodes
launching alice pod with image docker.io/parity/polkadot:v0.9.18
with command: /cfg/zombie-wrapper.sh polkadot --chain /cfg/rococo-local.json --name alice --rpc-cors all --unsafe-rpc-external --rpc-methods unsafe --unsafe-ws-external -lparachain=debug --no-mdns --node-key 2bd806c97f0e00af1a1fc3328fa763a9269723c8db8fac4f93af71db186d6e90 --no-telemetry --prometheus-external --jaeger-agent localhost:6831 --validator --listen-addr /ip4/0.0.0.0/tcp/30333/ws --base-path /data
uploading to fileserver: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/rococo-local.json as:74e1ac30e59861832272c2925e9d22c7bb63f47ae10e8010d8bd3b59e24996ce
uploading to fileserver: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/zombie-wrapper.sh as:80d9cdcda7802f9a0fbab8739bf5ab65556f4d6606dd81606529f8b3dc141768
alice pod is ready!
alice running
You can follow the logs of the node by running this command:
kubectl logs -f alice -c alice -n zombie-c1812f6eb712223f7cdb3e97384732a7
⚙ Added Boot Nodes: /ip4/172.17.0.6/tcp/30333/ws/p2p/12D3KooWQCkBm1BYtkHpocxCwMgR8yjitEeHGx8spzcDLGt2gkBm
launching bob pod with image docker.io/parity/polkadot:v0.9.18
with command: /cfg/zombie-wrapper.sh polkadot --chain /cfg/rococo-local.json --name bob --rpc-cors all --unsafe-rpc-external --rpc-methods unsafe --unsafe-ws-external -lparachain=debug --no-mdns --node-key 81b637d8fcd2c6da6359e6963113a1170de795e4b725b84d1e0b4cfd9ec58ce9 --no-telemetry --prometheus-external --jaeger-agent localhost:6831 --validator --bootnodes /ip4/172.17.0.6/tcp/30333/ws/p2p/12D3KooWQCkBm1BYtkHpocxCwMgR8yjitEeHGx8spzcDLGt2gkBm --listen-addr /ip4/0.0.0.0/tcp/30333/ws --base-path /data
bob pod is ready!
bob running
You can follow the logs of the node by running this command:
kubectl logs -f bob -c bob -n zombie-c1812f6eb712223f7cdb3e97384732a7
All relay chain nodes spawned...
launching collator01 pod with image gcr.io/p-network-202104/template:slim1
with command: /cfg/zombie-wrapper.sh target/release/parachain-collator --name collator01 --node-key 5e123980fad4770d9ab5b7df7bbee8499943822a8555d11995e8203210065342 --chain /cfg/rococo-local-2000.json --base-path /data --listen-addr /ip4/0.0.0.0/tcp/37663/ws --ws-port 39025 --prometheus-external --prometheus-port 44123 --alice --collator --force-authoring --jaeger-agent localhost:6831 -- --chain /cfg/rococo-local.json --execution wasm --port 45187 --ws-port 44467 --rpc-port 40445
collator01 pod is ready!
collator01 running
You can follow the logs of the node by running this command:
kubectl logs -f collator01 -c collator01 -n zombie-c1812f6eb712223f7cdb3e97384732a7
⚙ Added Boot Nodes: /ip4/172.17.0.8/tcp/30333/ws/p2p/12D3KooWAieokPB4zt3dJJokhVZw8KoQuTgXEEyempJtr6pGkhrd
Warn: Tracing collator service doesn't exist
---------------------------------------https://docs.docker.com/engine/reference/builder/#expose--
Network launched 🚀🚀
In namespace zombie-c1812f6eb712223f7cdb3e97384732a7 with kubernetes provider
Node name: alice
Node direct link: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A38615#/explorer
Node prometheus link: http://127.0.0.1:46855/metrics
---
Node name: bob
Node direct link: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A36869#/explorer
Node prometheus link: http://127.0.0.1:34721/metrics
---
Parachain ID: 2000
Parachain chainSpecPath path: /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/rococo-local-2000.json
Node name: collator01
Node direct link: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A43009#/explorer
Node prometheus link: http://127.0.0.1:41975/metrics
---
✔ alice: is up (66ms)
✔ bob: is up
2022-05-19 15:02:07 API/INIT: RPC methods not decorated: beefy_getFinalizedHead
✔ alice: parachain 2000 is registered within 225 seconds (2702ms)
✔ bob: log line matches glob "*rted #1*" within 10 seconds (69ms)
✔ bob: log line matches "Imported #[0-9]+" within 10 seconds (64ms)
✔ bob: log line matches "Imported new block." within 10 seconds (60ms)
Error:
Timeout(150), "getting desired parachain block height 6 within 150 secs".
1) alice: parachain 2000 block height is at least 6 within 150 seconds
✔ alice: reports node_roles is 4
✔ alice: reports sub_libp2p_is_major_syncing is 0
2022-05-19 15:04:40 API/INIT: RPC methods not decorated: beefy_getFinalizedHead
2) bob: system event contains "A candidate was included" within 20 seconds
2022-05-19 15:05:01 API/INIT: RPC methods not decorated: beefy_getFinalizedHead
3) alice: system event matches glob "*was backed*" within 10 seconds
Node's logs are available in /tmp/zombie-c1812f6eb712223f7cdb3e97384732a7_-188488-K4McZ572Plmt/logs
Deleting network
2022-05-19 15:05:11 RPC-CORE: health(): Health:: disconnected from ws://127.0.0.1:38615: 1000:: Normal connection closure
2022-05-19 15:05:19 RPC-CORE: getStorage(key: StorageKey, at?: BlockHash): StorageData:: WebSocket is not connected
child process exited
8 passing (6m)
3 failing
1) small network( Small Network test )
alice: parachain 2000 block height is at least 6 within 150 seconds:
AssertionError: expected 0 to be at least 6
+ expected - actual
-0
+6
at /snapshot/zombienet/dist/test-runner/index.js:265:37
at Generator.next (<anonymous>)
at fulfilled (/snapshot/zombienet/dist/test-runner/index.js:24:58)
2) small network( Small Network test )
bob: system event contains "A candidate was included" within 20 seconds:
AssertionError: expected false to be truthy
at /snapshot/zombienet/dist/test-runner/index.js:367:32
at Generator.next (<anonymous>)
at fulfilled (/snapshot/zombienet/dist/test-runner/index.js:24:58)
3) small network( Small Network test )
alice: system event matches glob "*was backed*" within 10 seconds:
AssertionError: expected false to be truthy
at /snapshot/zombienet/dist/test-runner/index.js:367:32
at Generator.next (<anonymous>)
at fulfilled (/snapshot/zombienet/dist/test-runner/index.js:24:58)
From above you can see the
- relaychain alice node test: ok
- relaychain bob node test: ok
- parachain 2000 is registered: ok
- 3 log tests: ok
- parachain 2000 block height test: failed!!?? Why did that fail?
I know the error 2) and 3) are caused by error 1), the block height test failure.
Why was the parachain collator not producing blocks?
When I checked the collator logs via kubectl logs -f collator01 -c collator01 -n zombie-c1812f6eb712223f7cdb3e97384732a7, the terminal returned nothing just like nothing happened!!??
When clicking on the Collator Node direct link: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A43009#/explorer , my browser could not connect to that collator node at all...
The docker "EXPOSE" does not actually publish the port according to https://docs.docker.com/engine/reference/builder/#expose And adding this to the network definition file didn't work:
args = ["-p 30333:30333/tcp", "-p 9933:9933/tcp", "-p 9944:9944/tcp"]
When I logged into that image via docker run -it 2647e63e75ee /bin/bash, then ran /target/release/parachain-collator, I got this error:
Error: Input("Relay chain argument error: Invalid input: `rococo-local` only supported with `rococo-native` feature enabled.")
See the whole terminal:
root@23006a9e7a5d:/# /target/release/parachain-collator
Error: Input('Relay chain argument error: Invalid input: "rococo-local" only supported with "rococo-native" feature enabled.')
2022-05-19 11:31:21 Parachain Collator Template
2022-05-19 11:31:21 ✌️ version 0.1.0-unknown-x86_64-linux-gnu
2022-05-19 11:31:21 ❤️ by Anonymous, 2020-2022
2022-05-19 11:31:21 📋 Chain specification: Local Testnet
2022-05-19 11:31:21 🏷 Node name: hypnotic-glue-7138
2022-05-19 11:31:21 👤 Role: FULL
2022-05-19 11:31:21 💾 Database: RocksDb at /root/.local/share/parachain-collator/chains/local_testnet/db/full
2022-05-19 11:31:21 ⛓ Native runtime: template-parachain-1 (template-parachain-0.tx1.au1)
2022-05-19 11:31:21 assembling new collators for new session 0 at #0
2022-05-19 11:31:21 assembling new collators for new session 1 at #0
root@23006a9e7a5d:/#
Please advise
target/release/parachain-collatorto check that the process starts as expected. Thanks!rococo-nativefeature isn't set. Is the project in a public repo?buildphase to inject then the binary.