How to connect two nodes running on different computers?

Question

I'm currently trying to deploy a private blockchain and I'm having problems trying to have nodes running on different machines but still communicating with each other. I've created a docker image which can be easily reused, it is simply the substrate contracts node template. Using this image I have set up a docker compose file that looks as follows:

version: '3'

services:
  bootnode:
    image: {my_image}
    ports:
      - "30333:30333"
      - "9933:9933"
      - "9944:9944"
      - "9615:9615"
    volumes:
      - bootnode:/tmp/node
    command: --ws-external --rpc-external --rpc-cors all --name "Bootnode 1" --pruning archive --allow-private-ipv4
    restart: always

  validator-1:
    image: {my_image}
    ports:
      - "30334:30333"
      - "9934:9933"
      - "9945:9944"
      - "9616:9615"
    volumes:
      - validator-1:/tmp/node
    command: --validator --name "Validator Node 1" --allow-private-ipv4
    restart: always

  validator-2:
    image: {my_image}
    ports:
      - "30335:30333"
      - "9935:9933"
      - "9946:9944"
      - "9617:9615"
    volumes:
      - validator-2:/tmp/node
    command: --validator --name "Validator Node 2" --allow-private-ipv4
    restart: always

volumes:
  bootnode:
  validator-1:
  validator-2:

All the containers refer to the same raw chain specification file, and that looks like:

{
  "name": "My Chain",
  "id": "my_chain",
  "chainType": "Live",
  "bootNodes": [
    "/ip4/{cloud_machine_ip}/tcp/30333/p2p/{boot_node_peer_id}"
  ],
  "telemetryEndpoints": [
    [
      "/dns/telemetry.polkadot.io/tcp/443/x-parity-wss/%2Fsubmit%2F",
      0
    ]
  ],
  "protocolId": "my_chain",
  "properties": null,
  "codeSubstitutes": {},
  "genesis": {
  ...

I'm running all the containers on a cloud hosted machine but when I try to run another node locally, or on another machine, using the same chainSpec file, it is unable to find any peers.

Does my setup make sense? How can I make it so nodes running on different machines can communicate with each other?

Side question

I also would like to not need to use the allow-private-ipv4 option, but if I don't use that, and the chain is set to 'Live', I cannot figure out a way to make the nodes communicate with each other locally either. Is that maybe related to my main issue? Is there something explanation that would help me understand why this happens?

Answer 1

You first will have to sort out the networking with your cloud provider. I would suggest doing your test locally first to remain away from networking issues between your machines and once that works, you can add your cloud networking to the picture.

The following blog post is now old but you still may find relevant information: https://medium.com/@wilfried.kopp/your-very-own-local-polkadot-substrate-network-in-less-than-30s-300ed7913895

Regarding the networking between your machine and some cloud machine, you will probably need to open some ports and configure your firewall and/or setup a VPN such as Wireguard and turn it on.

Before testing further, try pinging your various machines to ensure that your networking looks decent. Once you confirm the pings are working, if you need to troubleshoot further, you may wan to use nmap.