0

I am using a new Apple M2 laptop for Substrate development but I cannot install Substrate on it. If I follow the instructions at https://docs.substrate.io/install/macos/ it says I can install Rust and set up a Substrate development environment on Apple macOS computers with either Intel or an Apple M1 processor.

I have otherwise verified that the laptop meets the basic requirements. It is running:

  • sysctl -n machdep.cpu.brand_string outputs Apple M2 Max
  • /usr/bin/arch outputs arm64
  • macOS 13.4
  • 12 CPU cores, 30 GPU cores (CPU usage <10%)
  • 64 GB RAM (at least 20 GB RAM is free)
  • 1.8 TB available space

I run the following:

  • brew update && brew --version it outputs Homebrew 4.1.5
  • brew install protobuf openssl make

Then I checked if Rosetta 2 was running by checking if running the command pgrep oahd output a process id, which it did.

But I ran the command softwareupdate --install-rosetta --agree-to-license to install Rosetta 2 again anyway, and it output:

023-08-21 00:21:21.246 softwareupdate[52504:6762410] Package Authoring Error: 032-84877: Package reference com.apple.pkg.RosettaUpdateAuto is missing installKBytes attribute
Install of Rosetta 2 finished successfully

Then I followed the instructions here to install Substrate using Docker with the latest Docker Desktop version 4.22.0 (117440) installed and running. I clone the Substrate repository as follows, and ran the script to install Substrate within a Docker container, where the latest Substrate commit was 94be94be6d26becd2395b58ae09ca31f596afe7d

git clone https://github.com/paritytech/substrate
cd substrate
./docker/build.sh

But it outputs the following error:

Building parity/substrate:latest docker image, hang on!
[+] Building 2602.6s (14/15)                                                                                                              docker:desktop-linux
 => [internal] load build definition from substrate_builder.Dockerfile                                                                                    0.0s
 => => transferring dockerfile: 1.88kB                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                         0.0s
 => => transferring context: 136B                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                           5.0s
 => [internal] load metadata for docker.io/paritytech/ci-linux:production                                                                                 3.1s
 => [builder 1/4] FROM docker.io/paritytech/ci-linux:production@sha256:dc9abf9e877c5bad94828245406dac8a186530e1ad6a1b5f2072e5e19e1f64b4                   0.0s
 => CACHED [stage-1 1/6] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba                      0.0s
 => [internal] load build context                                                                                                                         0.6s
 => => transferring context: 46.95MB                                                                                                                      0.6s
 => CACHED [builder 2/4] WORKDIR /substrate                                                                                                               0.0s
 => CACHED [builder 3/4] COPY . /substrate                                                                                                                0.0s
 => CACHED [builder 4/4] RUN cargo build --locked --release                                                                                               0.0s
 => CACHED [stage-1 2/6] COPY --from=builder /substrate/target/release/substrate /usr/local/bin                                                           0.0s
 => ERROR [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin                                                               0.0s
 => ERROR [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin                                                        0.0s
 => ERROR [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin                                                   0.0s
------
 > [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin:
------
------
 > [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin:
------
------
 > [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin:
------
substrate_builder.Dockerfile:21
--------------------
  19 |     COPY --from=builder /substrate/target/release/subkey /usr/local/bin
  20 |     COPY --from=builder /substrate/target/release/node-template /usr/local/bin
  21 | >>> COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin
  22 |
  23 |     RUN useradd -m -u 1000 -U -s /bin/sh -d /substrate substrate && \
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 4b02f120-ba24-4d54-b2be-a38dc6dce883::kor4q7o2o99zc2mscfob9bp7v: "/substrate/target/release/chain-spec-builder": not found

real    43m22.870s
user    0m4.775s
sys 0m4.250s

So I then decided to run iTerm2 using Rosetta 2, by following the steps suggested here, as listed below:

  • Finder > Applications and found iTerm.app (alternative to Terminal app)
  • Right-clicked iTerm.app and selected "Duplicate"
  • Rename the duplicate to "iTerm x86_64"
  • Right-clicked "iTerm x86_64.app" > Get Info > Enabled checkbox called "Open using Rosetta".
  • Double-clicked "iTerm x86_64.app" to open the Rosetta terminal instead of the native arm64 terminal, typed arch and it output i386 (i was expecting it to output x86_64 instead)
  • Right-clicked "iTerm x86_64.app" in the Dock and clicked "Keep in Dock"
  • Changed to the same directory where Substrate had been cloned and ran:
./docker/build.sh

It was outputting:

...
=> => # <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
=> => # <jemalloc>: (This is the expected behaviour if you are running under QEMU)
...

And it output the following error:

Building parity/substrate:latest docker image, hang on!
[+] Building 4493.6s (14/15)                                                                                        docker:desktop-linux
 => [internal] load .dockerignore                                                                                                   0.0s
 => => transferring context: 136B                                                                                                   0.0s
 => [internal] load build definition from substrate_builder.Dockerfile                                                              0.0s
 => => transferring dockerfile: 1.88kB                                                                                              0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                     4.5s
 => [internal] load metadata for docker.io/paritytech/ci-linux:production                                                           2.5s
 => [builder 1/4] FROM docker.io/paritytech/ci-linux:production@sha256:dc9abf9e877c5bad94828245406dac8a186530e1ad6a1b5f2072e5e19e1  0.0s
 => [internal] load build context                                                                                                   0.1s
 => => transferring context: 322.12kB                                                                                               0.1s
 => CACHED [stage-1 1/6] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2fa  0.0s
 => CACHED [builder 2/4] WORKDIR /substrate                                                                                         0.0s
 => CACHED [builder 3/4] COPY . /substrate                                                                                          0.0s
 => [builder 4/4] RUN cargo build --locked --release                                                                             4488.6s
 => CACHED [stage-1 2/6] COPY --from=builder /substrate/target/release/substrate /usr/local/bin                                     0.0s
 => ERROR [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin                                         0.0s
 => ERROR [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin                                  0.0s
 => ERROR [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin                             0.0s
------
 > [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin:
------
------
 > [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin:
------
------
 > [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin:
------
substrate_builder.Dockerfile:21
--------------------
  19 |     COPY --from=builder /substrate/target/release/subkey /usr/local/bin
  20 |     COPY --from=builder /substrate/target/release/node-template /usr/local/bin
  21 | >>> COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin
  22 |
  23 |     RUN useradd -m -u 1000 -U -s /bin/sh -d /substrate substrate && \
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 4b02f120-ba24-4d54-b2be-a38dc6dce883::kor4q7o2o99zc2mscfob9bp7v: "/substrate/target/release/chain-spec-builder": not found

real    74m53.824s
user    0m10.714s
sys 0m9.167s

Update:

  • I tried following the suggestion mentioned here of adding the option --platform linux/arm64 here when running docker build and docker run but i got the same error.
  • I also tried a custom Docker build here where I pull from arm64v8/ubuntu instead of ubuntu:23.04, but I got the same error
1
  • so i removed lines 8-33 in the substrate_builder.Dockerfile github.com/paritytech/substrate/blob/master/docker/… and i was able to enter the Docker container with docker run -it parity/substrate but the /substrate/target/release directory only contained the substrate binary, but not the subkey, node-template, or chain-spec-builder binaries too like it used to, hence the error. so it does build substrate on an Apple M2, but the question now is why doesn't it generate the subkey, node-template, or chain-spec-builder binaries too anymore Aug 21, 2023 at 8:57

1 Answer 1

1

The reason for the error wasn't because I was using an Apple M2 chip, it was simply because for some reason if you build Substrate with Docker using substrate_builder.Dockerfile by following the instructions it now only generates the substrate binary in /substrate/target/release/substrate, but it no longer also generates the subkey, node-template, or chain-spec-builder binaries too like it used to.

So I think a pull request to the Substrate repository may be required to either change line 6 of substrate_builder.Dockerfile to instead run the following to generate all the binaries in the /substrate/target/release directory, so it doesn't generate an error when they are used later on lines 19-21:

RUN cargo build --locked --release && \
    cargo build -p subkey --release --locked && \
    cargo build -p node-template --release --locked && \
    cargo build -p chain-spec-builder --release --locked

So to overcome the error I simply removed lines of code 8-33 in the file substrate_builder.Dockerfile, and then entered the Docker container with the command docker run -it parity/substrate where I could then run the substrate binary with ./target/release/substrate --help and manually generate the other binaries by running:

cargo build -p subkey --release --locked && \
cargo build -p node-template --release --locked && \
cargo build -p chain-spec-builder --release --locked

Then I was able to use those binaries too with:

./target/release/subkey --help
./target/release/node-template --help
./target/release/chain-spec-builder --help

Note that another approach to obtaining the latest subkey binary is by running:

curl https://getsubstrate.io -sSf | bash -s -- --fast
cargo install --force subkey --git https://github.com/paritytech/substrate
subkey --help

Note that the chain-spec-builder binary is a "utility to easily create a testnet chain spec definition with a given set of authorities and endowed accounts and/or generate random accounts", and differs from the Substrate binary command build-spec that you may run with ./target/release/substrate build-spec --help is used to "serialize the chain specification into a JSON file".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.