12

Having a Wasm runtime is a key design decision in Substrate's architecture. It's clear that it's a key part of Substrate's forkless upgradability feature, but I'm still curious to understand what other motivations exist behind this decision.

Why does Substrate have a Wasm runtime? What advantages does using a Wasm execution environment provide and with what tradeoffs?

2 Answers 2

18

I'd divide the question further into two parts.

Does Substrate Really Need a Virtual Machine?

Historically, Substrate was split from Polkadot early on. So we will have to bring Polkadot into the context. Polkadot had two problems to solve.

  1. Polkadot is meant to be upgradable and governed on-chain.
  2. Polkadot's relay chain needs a way of verifying the correctness of parachain state transitions.

It is possible to solve those two without involving a virtual machine.

With regard to upgrading, for example, Ethereum is still leaning on hard-forks as their upgrade mechanism. This stuff works but is not perfect. One reason is that it conflates technical upgrading and governance. For a successful upgrade, a significant portion of users must upgrade their nodes in time. Not upgraded users will stay on the existing fork. Therefore the voting process is biased towards not upgrading.

When the state-transition logic is bytecode stored on the chain itself, voting is separate. In many cases, operators don't have to do anything for an upgrade.

Now, sometimes forkless code upgrades require intervention by node operators, such as when the code starts using functionality introduced by newer versions of the node software. In this case, though, the correct state transition is always on the chain. It's just that the older nodes cannot follow the canonical chain because they cannot execute its state transition function.

Hard forking is reserved only for more complicated situations, like social recovery. Upgrade becomes a routine.

You might say, "wait a minute, that does not require a VM", and you will be correct. E.g., Tendermint employs a rigid state machine by default. A tendermint-based chain can store a version number in the on-chain state. If a node does not have the logic for handling state transitions of that version, it can stop.

A VM allows abstracting from the specifics of the platform. In your language of choice, there are typically things that may produce slightly different results on different platforms. Those things are usually not significant, and regular programmers don't care about them.

However, in the case of blockchains, they do matter. Each observed bit of non-determinism leads to consensus divergence.

Take even undefined behavior. You don't want to invoke it in your code. If you did, anything could happen, including rm -rf /. That anything can be different depending on the platform and the compiler. On the other hand, if that happened inside a VM, the machine the node was running would certainly be fine. The behavior will be the same on each machine.

One of the costs is performance. Using a virtual machine most likely will lead to a performance loss compared to the native code.

Another cost is complexity. Using a VM requires another big component in your node software. A VM is often a quite complicated piece of machinery. However, this complexity is not systemic and is isolated in the execution engine.

Why was Wasm picked as the virtual machine?

WebAssembly has unique properties that make it a perfect fit for this use case.

  1. Performance. Wasm operations could be mapped to machine code quite directly. The performance is consistent without cliffs.
  2. Safe. Wasm programs do not have access to anything by default. The capabilities have to be exposed explicitly to the wasm environment. Wasm also has relatively simple semantics, which makes holes less probable.
  3. Determinstic-ish. Wasm instruction set has a small and well-defined number of non-deterministic operations. A not deterministic in this context means an operation can have different outputs given the same inputs. Making it deterministic is a matter of defining the actual output for those operations.
  4. Open standard. There is no single company that owns Wasm.
  5. Wide support. WASM has a lot of industry buy-in and wide support, so we can take advantage of all of the work that's being done in the ecosystem. Hitting a codegen bug in the compiler is a lot less likely when there are thousands of other people besides you compiling their code to WASM. The ecosystem also has a big incentive to make WASM faster, since it's a part of the web platform. The reason why JavaScript is essentially the fastest "scripting" language out there is not because it's particularly easy to optimize, but because it's a core component of a critical piece of software which every Internet-connected human being on the planet uses - the web browser - so an enormous amount of resources is poured into making JavaScript fast. WASM is on its way to similarly become a core component of the web, just like JavaScript is, and we want to take advantage of that.

It would take a lot of effort to make a platform like .NET or JVM work for our use-cases. They have too much surface. Making them safe and deterministic would be a problem.

I don't even know what the tradeoffs are here. Not much comes to mind. It generally feels like a perfect fit for this particular use case. Here I will try to leave my thoughts on this and might update if I remember something else.

  • Early in Polkadot's lifecycle Wasm and tooling around it were under heavy development. It wasn't good. But now it is almost perfect.
3
  • Thank you for the clear breakdown, this was exactly what I was looking for. For trade offs, isn’t there any overhead for interacting with a Wasm runtime in terms of speed ? Also harder to debug ? Or are these things just getting better too ? May 18, 2022 at 13:14
  • There is some overhead, I believe I mentioned that the cost is performance for using a VM relative to using no VM. In case of Wasm the inefficiencies are coming from different places. A notable one is copying data into the sandbox requirement.
    – pepyakin
    May 23, 2022 at 16:09
  • I am not entirely sure that there are problems with debugging. As I mentioned the tooling improves, wasmtime supports connecting lldb nowadays. That is not terribly useful though since debugging a runtime is not really convenient to do with a debugger anyway.
    – pepyakin
    May 23, 2022 at 16:14
3

WASM runtime provides more architecture flexibility, for eg. OCW's implementation is more neat on WASM runtime pattern.

Also, forkless upgrade itself is so important that illustrate WASM's worth. When go through BTC/BTH, ETC/ETH, and eth2.0, will find that upgrading itself needs miners consensus, and blockchain itself is very familiar with this problem, so why not just take it into WASM runtime.

3
  • What do you mean it's more "neat on WASM runtime pattern" ? Also, forkless upgradability could be achieved (with more difficulty, sure) with native runtimes so it doesn't really answer the question. May 16, 2022 at 15:59
  • @SachaLansky I think the second paragraph answer you question well, solving the miners/validators' consensus with on-chain-upgrade as a transaction is the true value of WASM which bring more autonomous and shorten the whole upgrade process. May 17, 2022 at 9:28
  • @SachaLansky By "neat on WASM runtime pattern", I mean one solution(WASM runtime) solves two problem (one is runtime's forkless upgradability, the other is ocw's easy upgradablity) May 18, 2022 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.