8

I'm trying to make an RPC call to a Substrate node, but I can't generate a proper Storage Key. The Module and Method parts are fine, the problem is the Twox64Concat part. The sp_core::twox_64 of value 33955c59b47da83d1cade50f724a6993 should return 61cb07b62bdbaa9980 but returns ef859d4c4bea1c66 instead.

enter image description here

Data storage structure:

#pallet::storage]
#[pallet::getter(fn addresses_owned)]
/// Keeps track of address owners.
pub(super) type AddressOwners<T: Config> =
   StorageMap<_, Twox64Concat, Address, BoundedVec<T::AccountId, T::MaxAddressOwners>, ValueQuery>;

The rest part of the pallet can be found here.

My code:

    let module_name = "Nolik";
    let module_name_twox = sp_core::twox_128(module_name.as_bytes());
    let module_name_hex = hex::encode(module_name_twox.clone());
    println!("MODULE {:?}", module_name_hex);

    let method_name = "AddressOwners";
    let method_name_twox = sp_core::twox_128(method_name.as_bytes());
    let method_name_hex = hex::encode(method_name_twox.clone());
    println!("METHOD {:?}", method_name_hex);

    let storage_key = "33955c59b47da83d1cade50f724a6993";
    let storage_key_twox = sp_core::twox_64(storage_key.as_bytes());
    let storage_key_hex = hex::encode(storage_key_twox.clone());
    println!("TWOX64 {:?}", storage_key_hex);

    let twox_64_concat: Vec<u8> = storage_key_twox
        .iter()
        .chain(storage_key.as_bytes())
        .cloned()
        .collect();

    let twox_64_concat_hex = hex::encode(twox_64_concat.clone());
    println!("TWOX64CONCAT {:?}", twox_64_concat_hex);

    let key = format!("0x{}{}{}", module_name_hex, method_name_hex, twox_64_concat_hex);
    println!("KEY {:?}", key);

The output

MODULE "f90f720da6ab97a2d8404b27e705dda8"
METHOD "82074de3756f76dc0e8434b877c54d17"
TWOX64 "ef859d4c4bea1c66"
TWOX64CONCAT "ef859d4c4bea1c663333393535633539623437646138336431636164653530663732346136393933"
KEY "0xf90f720da6ab97a2d8404b27e705dda882074de3756f76dc0e8434b877c54d17ef859d4c4bea1c663333393535633539623437646138336431636164653530663732346136393933"

Would appreciate any advice. Thanks!

Update

let storage_key_twox = sp_core::twox_64(storage_key.as_bytes().encode().as_ref());

solves the issue and I get the 61cb07b62bdbaa99 hash. But, there is extra 80 after it. It is possible to extend the original set of bytes and manually add 128 to byte array.

let mut with_80: Vec<u8> = vec![128];
with_80.extend(storage_key.as_bytes());

But I'm sure it's the wrong way of doing it.

enter image description here

4
  • Can you try using the hashed_key_for(k1, k2) function from here docs.rs/frame-support/3.0.0/frame_support/storage/…? Commented Mar 19, 2022 at 15:33
  • I could not use it directly because of conflicts in Cargo.toml, but I looked through the source code and found out that I need to encode the source bytes. I've changed let storage_key_twox = sp_core::twox_64(storage_key.as_bytes()); to let storage_key_twox = sp_core::twox_64(storage_key.as_bytes().encode().as_ref()); and I got the output 61cb07b62bdbaa99. That's a breakthrough ) The only difference is that the Storage Key contains 80 right after 61cb07b62bdbaa99. Still trying to understand the nature of it. Commented Mar 19, 2022 at 18:42
  • You should really avoid re-writing these functions on your own if you can. There should really be no issues to import the actual functions provided by frame_support, and if there is, you should ask that as a question we can answer fro you in another post.
    – Shawn Tabrizi
    Commented Mar 20, 2022 at 1:32
  • I've tried every library out there I could find. Unfortunately, each time I faced some issues and wasted so much time to understand how those libraries work. That's why I decided to make this move and understand how to generate these requests on my own. Commented Mar 20, 2022 at 3:09

1 Answer 1

7

I believe all the confusion here is related to the encoding of byte vectors.

Vectors in SCALE are encoded with a length prefix represented with a compact number.

A collection of same-typed values is encoded, prefixed with a compact encoding of the number of items, followed by each item's encoding concatenated in turn.

This is required because all of of a scale encoded blob is just bytes, so if at some point, some arbitrary number of bytes needs to be decoded, the decoder needs to know how many of those bytes should be used for the byte blob.

As you solved yourself, you need to encode() the bytes before you hash them, so that the length of the bytes is included in the hashing algorithm.

Lets look at your storage_key more closely then, before and after the encode:

let storage_key = "33955c59b47da83d1cade50f724a6993";
let storage_key_bytes = storage_key.as_bytes();
let storage_key_bytes_encoded = storage_key_bytes.encode();

println!("storage_key_bytes: {:?}", storage_key_bytes);
println!("storage_key_bytes_encoded: {:?}", storage_key_bytes_encoded);

storage_key_bytes: [51, 51, 57, 53, 53, 99, 53, 57, 98, 52, 55, 100, 97, 56, 51, 100, 49, 99, 97, 100, 101, 53, 48, 102, 55, 50, 52, 97, 54, 57, 57, 51]

storage_key_bytes_encoded: [128, 51, 51, 57, 53, 53, 99, 53, 57, 98, 52, 55, 100, 97, 56, 51, 100, 49, 99, 97, 100, 101, 53, 48, 102, 55, 50, 52, 97, 54, 57, 57, 51]

As you can see, there is an extra 128 at the front of storage_key_bytes_encoded.

So you need to also include the encode() to your twox_64_concat calculation:

let twox_64_concat: Vec<u8> = storage_key_twox
    .iter()
    .chain::<&[u8]>(storage_key.as_bytes().encode().as_ref())
    .cloned()
    .collect();

This results in the correct storage key with the extra 80, which is just hex representation of 128:

0xf90f720da6ab97a2d8404b27e705dda882074de3756f76dc0e8434b877c54d1761cb07b62bdbaa99803333393535633539623437646138336431636164653530663732346136393933

1
  • That's amazing! Thank you so much for your help Commented Mar 20, 2022 at 2:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.