Why is the Runtime memory management designed such that the Host manages the memory shared with the Wasm Runtime (imports memory + allocation functions)? Since the Host might need to store things onto the memory, does it heap allocate things specific to itself on the linear memory shared with the Wasm runtime or it is in separate memory space?
Add a comment
|
1 Answer
Why is the Runtime memory management designed such that the Host manages the memory shared with the Wasm Runtime (imports memory + allocation functions)?
The design in which allocation functions are on the host side is dictated by the fact that some of the host functions might return buffers of data of unknown size. That means that the wasm code cannot efficiently provide buffers upfront. For example, let's examine the host function that returns a given storage value. The storage value's size is not known upfront in the general case, so the wasm caller cannot pre-allocate the buffer upfront. A potential solution is to first call the host function without a buffer, which will return the value's size, and then do the second call passing a buffer of the required size. For some host functions, caches could be put in place for mitigation, some other functions cannot be implemented in the such model at all. To solve this problem, it was chosen to place the allocator on the host side.
Note, however, that this is not the only possible solution. For instance, there is an ongoing discussion about moving the allocator into the wasm: substrate#11883.
The fact that the memory is imported does not matter too much in our setting. It could work with a declared and exported memory. In fact, this is how it worked in the beginning and importing memory. We almost certainly still support this to this day. Imported memory works a little bit better than exported memory since it avoids some edge cases, although it also has some downsides.
Since the Host might need to store things onto the memory, does it heap allocate things specific to itself on the linear memory shared with the Wasm runtime or it is in separate memory space?
Notably, the allocator maintains some of its data structures inside the linear memory and some other structures outside.