Resolving Sandbox Host Function Access After Deprecation in Polkadot v0.9.35

Question

I am currently working on a Substrate-based solo chain that has migrated from polkadot-v0.9.34 to polkadot-v0.9.35 and is now on the stable branch 2409. In v0.9.35, the Sandbox host functions were removed as part of this PR, which has impacted our node synchronization in full mode.

During the initial attempt to synchronize nodes from scratch in full mode, we encountered an error indicating that the Sandbox host functions were not found. As an initial approach, we tried a stub implementation by which this allowed the nodes to begin synchronization, the process ultimately failed when the nodes encountered a block that utilized these missing functions. Consequently, we decided to proceed with implementing the Sandbox trait ourselves to address this issue comprehensively.

Error: Service(Client(VersionInvalid("Other error happened while constructing the runtime:
runtime requires function imports which are not present on the host:
'env:ext_sandbox_memory_set_version_1',  'env:ext_sandbox_invoke_version_1', 
'env:ext_sandbox_instantiate_version_1', 'env:ext_sandbox_memory_teardown_version_1',
'env:ext_sandbox_memory_new_version_1', 'env:ext_sandbox_memory_get_version_1',
'env:ext_sandbox_instance_teardown_version_1'")))

What We Have Done So Far

To address this issue, I have started re-implementing the removed Sandbox host functions based on the deprecations mentioned in the PR. During this implementation, I first define all the host functions in a file called runtime_interface. After this I call these host functions in our node/client/lib.rs like this:

pub type HostFunctions =
    (sp_io::SubstrateHostFunctions, frame_benchmarking::benchmarking::HostFunctions,node_runtime_interfaces::sandbox::HostFunctions);

This ensured that the node recognized the availability of the Sandbox host functions.

For the implementation of these functions, I pulled the necessary code from the referenced PR. Additionally, I chose wasmi to execute WebAssembly, as the Sandbox functionality originally worked with wasmi before its removal. This decision was made to expedite the synchronization process for the node.

Below are some of the implemented files:

Sandbox Trait Definition

runtime_interface.rs :

#[runtime_interface(wasm_only)]
pub trait Sandbox {  
    fn memory_get(  
        &mut self,  
        memory_idx: u32,  
        offset: u32,  
        buf_ptr: Pointer<u8>,  
        buf_len: u32,  
    ) -> u32 {
        // Implementation  
        return 0;
    }  

    fn memory_set(  
        &mut self,  
        memory_idx: u32,  
        offset: u32,  
        val_ptr: Pointer<u8>,  
        val_len: u32,  
    ) -> u32 {
        // Implementation  
        return 0;
    }  

    fn instantiate(  
        &mut self,  
        dispatch_thunk: u32,  
        wasm_code: &[u8],  
        env_def: &[u8],  
        state_ptr: Pointer<u8>,  
    ) -> u32 {  
        return 0;  
    }  
    ...
}

Function Implementation

The Sandbox host functions are implemented in another file (sandbox_interface: It implements the several traits for FunctionExecutor), and an example of the memory_get function is as follows:

sandbox_interface.rs

impl Sandbox for FunctionExecutor {  
    fn memory_get(  
        &mut self,  
        memory_id: MemoryId,  
        offset: WordSize,  
        buf_ptr: Pointer<u8>,  
        buf_len: WordSize,  
    ) -> WResult<u32> {  
        let sandboxed_memory =  
            self.sandbox_store.borrow().memory(memory_id).map_err(|e| e.to_string())?;  
  
        let len = buf_len as usize;  
  
        let buffer = match sandboxed_memory.read(Pointer::new(offset as u32), len) {  
            Err(_) => return Ok(sandbox_env::ERR_OUT_OF_BOUNDS),  
            Ok(buffer) => buffer,  
        };  
  
        if self.memory.set(buf_ptr.into(), &buffer).is_err() {  
            return Ok(sandbox_env::ERR_OUT_OF_BOUNDS);  
        }  
  
        Ok(sandbox_env::ERR_OK)  
    }  
    ...
}

The Problem

The #[runtime_interface(wasm_only)] macro is critical to our implementation, as it provides functionalities such as:

• Trait Implementation: Implements the trait defined for the host functions.
• Host Function Registration: The macro registers the trait methods as host functions that can be called from the outside world.
• FFI (Foreign Function Interface) Wrappers: It generates FFI wrappers for the trait methods, allowing them to be called from the host environment.

We use the macro as follows:

#[runtime_interface(wasm_only)]
pub trait Sandbox {
    fn memory_get(
        &mut self,
        memory_idx: u32,
        offset: u32,
        buf_ptr: Pointer<u8>,
        buf_len: u32,
    ) -> u32 {
        self.sandbox()  // Implementation
            .memory_get(memory_idx, offset, buf_ptr, buf_len)
            .expect("Failed to get memory with sandbox");
    }
}

However, when the self.sandbox() method is called, it attempts to locate the sandbox functionality in the wasm-interface module via the sp-runtime-interface module. Since the sandbox functionality was removed in this PR, the implementation fails.

Attempted Solutions

We put some efforts to extract the logic of #[runtime_interface(wasm_only)] that is relevant to our use case to bypass this issue but the macro is essential for host function integration and cannot be omitted without sacrificing critical functionality like Trait Implementation, Host Function Registration, FFI Wrappers.

Question

Is it possible to access the Sandbox host function implementations within the current Substrate architecture by modifying the #[runtime_interface(wasm_only)] macro? Alternatively, is there another approach to resolve the issue of accessing the host function implementations effectively?

Thanks

Answer 1

You can create an Extension.

sp_externalities::decl_extension! {
    struct SandboxExt(TheRealExtension);
}

TheRealExtension being some struct that contains your actual sandbox logic.

Then you can implement your runtime interface roughly in the following way:

#[runtime_interface(wasm_only)]
pub trait Sandbox {  
    fn memory_get(  
        &mut self,  
        memory_idx: u32,  
        offset: u32,  
        buf_ptr: Pointer<u8>,  
        buf_len: u32,  
    ) -> u32 {
        let sandbox = self.extension::<SandboxExt>().expect("Sandbox should have been initialized before; qed");

        // Implementation  
        return 0;
    }  

    fn memory_set(  
        &mut self,  
        memory_idx: u32,  
        offset: u32,  
        val_ptr: Pointer<u8>,  
        val_len: u32,  
    ) -> u32 {
        let sandbox = self.extension::<SandboxExt>().expect("Sandbox should have been initialized before; qed");

        // Implementation  
        return 0;
    }  

    fn instantiate(  
        &mut self,  
        dispatch_thunk: u32,  
        wasm_code: &[u8],  
        env_def: &[u8],  
        state_ptr: Pointer<u8>,  
    ) -> u32 {   
        let sandbox = initialize(dispatch_thunk, wasm_code, env_def, state_ptr);
  
        self.register_extension(sandbox).expect("Sandbox was already initiated before?");

        return 0;  
    }  
}

The implementation above is just to demonstrate how it should work, very roughly. For example it just panics if instantiate is called twice, which is probably bad :P You could also first check if the extension exists and otherwise register it etc.

Answer 2

Thank you @bkchr for the response.

In our case, we are implementing the Sandbox trait by using the FunctionExecutor struct like this:

struct FunctionExecutor {  
    sandbox_store: Rc<RefCell<Store<wasmi::FuncRef>>>,  
    heap: RefCell<FreeingBumpHeapAllocator>,  
    memory: MemoryRef,  
    table: Option<TableRef>,  
    host_functions: Arc<Vec<&'static dyn Function>>,  
    allow_missing_func_imports: bool,  
    missing_functions: Arc<Vec<String>>,  
    panic_message: Option<String>,  
}

impl Sandbox for FunctionExecutor {  
    fn memory_get(  
        &mut self,  
        memory_id: MemoryId,  
        offset: WordSize,  
        buf_ptr: Pointer<u8>,  
        buf_len: WordSize,  
    ) -> WResult<u32> {  
        let sandboxed_memory =  
            self.sandbox_store.borrow().memory(memory_id).map_err(|e| e.to_string())?;  
  
        let len = buf_len as usize;  
  
        let buffer = match sandboxed_memory.read(Pointer::new(offset as u32), len) {  
            Err(_) => return Ok(sandbox_env::ERR_OUT_OF_BOUNDS),  
            Ok(buffer) => buffer,  
        };  
  
        if self.memory.set(buf_ptr.into(), &buffer).is_err() {  
            return Ok(sandbox_env::ERR_OUT_OF_BOUNDS);  
        }  
  
        Ok(sandbox_env::ERR_OK)  
    }  
    ...
}

The entry point of execution is memory_new, So currently we are registering the extension with FunctionExecutor (as the TheRealExtension) inside the memory_new function itself, as follows:

fn memory_new(&mut self, initial: u32, maximum: u32) -> u32 {
    log::info!("Going through the memory_new function");

    let function_executor: FunctionExecutor = FunctionExecutor::new(initial, maximum);
    let _ = self.register_extension(SandboxExt::from(function_executor));

    log::info!("Extension should be registered");

    let sandbox = self
        .extension::<SandboxExt>()
        .expect("memory_new: Sandbox should have been initialized before; qed");

    sandbox
        .memory_new(initial, maximum)
        .expect("Failed to create new memory with sandbox")
}

This requires an instance of FunctionExecutor (which is our TheRealExtension struct with the Sandbox trait) to be initialized, but it leads to the block execution failing during the sync process at instantiate with the following error:

Thread 'tokio-runtime-worker' panicked at 'Failed to instantiate a new sandbox: "dispatch_thunk_idx points on an empty table entry"', node/runtime-interfaces/src/lib.rs:166

The issue arises because the table provided in the constructor of the FunctionExecutor inside memory_new is currently using a mock version, as shown below:

impl FunctionExecutor {
    pub fn new(initial: u32, maximum: u32) -> Self {
        let sandbox_backend = SandboxBackend::Wasmi;
        FunctionExecutor {
            sandbox_store: Rc::new(RefCell::new(Store::new(sandbox_backend))),
            heap: RefCell::new(FreeingBumpHeapAllocator::new(0)),
            memory: wasmi::MemoryInstance::alloc(
                Pages(initial.try_into().unwrap()),
                Some(Pages(maximum.try_into().unwrap())),
            )
            .expect("Failed to allocate memory instance"),
            table: Some(TableInstance::alloc(100, Some(1000)).unwrap()), // Empty table
            host_functions: Arc::new(Vec::new()),
            allow_missing_func_imports: false,
            missing_functions: Arc::new(Vec::new()),
            panic_message: None,
        }
    }
}

From the last Substrate implementation where Sandbox was present, v0.9.34, we see that the table for FunctionExecutor is obtained from module_instance, as follows:

let table: Option<TableRef> = module_instance
    .export_by_name("__indirect_function_table")
    .and_then(|e| e.as_table().cloned());
let heap_base = get_heap_base(module_instance)?;

To fully understand the execution cycle of FunctionExecutor initialization and properly initialize it, we began syncing the nodes on an older version of the code (adding logs to trace the flow) where nodes were syncing successfully with sandbox functionality. During this process, we observed that logs were successfully generated from sandbox functions like memory_new(). However, we did not observe any logs related to the initialization of FunctionExecutor, leaving us uncertain whether this struct is indeed used to implement the Sandbox trait.

Currently, the Sandbox functions in Substate are being called like this:

#[runtime_interface(wasm_only)]  
pub trait Sandbox {
    fn memory_new(&mut self, initial: u32, maximum: u32) -> u32 {  
       self.sandbox()  
          .memory_new(initial, maximum)  
          .expect("Failed to create new memory with sandbox")  
    }
}

where self is a struct introduced by the runtime_interface(wasm_only)] macro.

To summarize, what we need is a struct that implements the trait and provides all the necessary arguments that are compatible with those executed in the problematic block during syncing. Therefore, we would like to ask the following:

• Is FunctionExecutor the correct way to implement the Sandbox trait?
• If so, how can we obtain the proper initialization arguments for it, or for another potential struct? In other words, if FunctionExecutor is not the appropriate choice and we need to create another custom struct, we would still require the correct initialization arguments to ensure that the syncing process executes correctly.