Generating a Storage Key when making an RPC request

Question

I'm trying to make an RPC call to a Substrate node, but I can't generate a proper Storage Key. The Module and Method parts are fine, the problem is the Twox64Concat part. The sp_core::twox_64 of value 33955c59b47da83d1cade50f724a6993 should return 61cb07b62bdbaa9980 but returns ef859d4c4bea1c66 instead.

Data storage structure:

#pallet::storage]
#[pallet::getter(fn addresses_owned)]
/// Keeps track of address owners.
pub(super) type AddressOwners<T: Config> =
   StorageMap<_, Twox64Concat, Address, BoundedVec<T::AccountId, T::MaxAddressOwners>, ValueQuery>;

The rest part of the pallet can be found here.

My code:

    let module_name = "Nolik";
    let module_name_twox = sp_core::twox_128(module_name.as_bytes());
    let module_name_hex = hex::encode(module_name_twox.clone());
    println!("MODULE {:?}", module_name_hex);

    let method_name = "AddressOwners";
    let method_name_twox = sp_core::twox_128(method_name.as_bytes());
    let method_name_hex = hex::encode(method_name_twox.clone());
    println!("METHOD {:?}", method_name_hex);

    let storage_key = "33955c59b47da83d1cade50f724a6993";
    let storage_key_twox = sp_core::twox_64(storage_key.as_bytes());
    let storage_key_hex = hex::encode(storage_key_twox.clone());
    println!("TWOX64 {:?}", storage_key_hex);

    let twox_64_concat: Vec<u8> = storage_key_twox
        .iter()
        .chain(storage_key.as_bytes())
        .cloned()
        .collect();

    let twox_64_concat_hex = hex::encode(twox_64_concat.clone());
    println!("TWOX64CONCAT {:?}", twox_64_concat_hex);

    let key = format!("0x{}{}{}", module_name_hex, method_name_hex, twox_64_concat_hex);
    println!("KEY {:?}", key);

The output

MODULE "f90f720da6ab97a2d8404b27e705dda8"
METHOD "82074de3756f76dc0e8434b877c54d17"
TWOX64 "ef859d4c4bea1c66"
TWOX64CONCAT "ef859d4c4bea1c663333393535633539623437646138336431636164653530663732346136393933"
KEY "0xf90f720da6ab97a2d8404b27e705dda882074de3756f76dc0e8434b877c54d17ef859d4c4bea1c663333393535633539623437646138336431636164653530663732346136393933"

Would appreciate any advice. Thanks!

Update

let storage_key_twox = sp_core::twox_64(storage_key.as_bytes().encode().as_ref());

solves the issue and I get the 61cb07b62bdbaa99 hash. But, there is extra 80 after it. It is possible to extend the original set of bytes and manually add 128 to byte array.

let mut with_80: Vec<u8> = vec![128];
with_80.extend(storage_key.as_bytes());

But I'm sure it's the wrong way of doing it.

Answer 1

I believe all the confusion here is related to the encoding of byte vectors.

Vectors in SCALE are encoded with a length prefix represented with a compact number.

A collection of same-typed values is encoded, prefixed with a compact encoding of the number of items, followed by each item's encoding concatenated in turn.

This is required because all of of a scale encoded blob is just bytes, so if at some point, some arbitrary number of bytes needs to be decoded, the decoder needs to know how many of those bytes should be used for the byte blob.

As you solved yourself, you need to encode() the bytes before you hash them, so that the length of the bytes is included in the hashing algorithm.

Lets look at your storage_key more closely then, before and after the encode:

let storage_key = "33955c59b47da83d1cade50f724a6993";
let storage_key_bytes = storage_key.as_bytes();
let storage_key_bytes_encoded = storage_key_bytes.encode();

println!("storage_key_bytes: {:?}", storage_key_bytes);
println!("storage_key_bytes_encoded: {:?}", storage_key_bytes_encoded);

storage_key_bytes: [51, 51, 57, 53, 53, 99, 53, 57, 98, 52, 55, 100, 97, 56, 51, 100, 49, 99, 97, 100, 101, 53, 48, 102, 55, 50, 52, 97, 54, 57, 57, 51]

storage_key_bytes_encoded: [128, 51, 51, 57, 53, 53, 99, 53, 57, 98, 52, 55, 100, 97, 56, 51, 100, 49, 99, 97, 100, 101, 53, 48, 102, 55, 50, 52, 97, 54, 57, 57, 51]

As you can see, there is an extra 128 at the front of storage_key_bytes_encoded.

So you need to also include the encode() to your twox_64_concat calculation:

let twox_64_concat: Vec<u8> = storage_key_twox
    .iter()
    .chain::<&[u8]>(storage_key.as_bytes().encode().as_ref())
    .cloned()
    .collect();

This results in the correct storage key with the extra 80, which is just hex representation of 128:

0xf90f720da6ab97a2d8404b27e705dda882074de3756f76dc0e8434b877c54d1761cb07b62bdbaa99803333393535633539623437646138336431636164653530663732346136393933