0

I want to remove keys from the storage which cannot be decoded by using unhashed. I know there are undecodable keys because when counting the number of storage keys via MyStorage::<T>::iter_keys().count(), there are error logs.

After digging through the Polkadot SDK, I came up with the following script which is called as remove_undecodable_storage_keys::<TypeOfSecondKey>(MyStorage::<T>::final_prefix()). Unfortunately, this script only cleans up keys on my dev chain but not my production one as everything there can be decoded despite error logs from the MyStorage::<T>::iter_keys().count() check.

  1. Can someone help me spot the issue?
  2. Moreover, do you see any issues using this function for a StorageDoubleMap? The storage (orml_tokens::Accounts) I am trying to clean has two key types: AccountId and MyEnum. For some reason remove_undecodable_storage_keys only iterates over keys of the first enum variant of MyEnum.
fn remove_undecodable_storage_keys<T: parity_scale_codec::Decode + Sized>(
    prefix: [u8; 32],
) -> (u64, u64) {
    let mut previous_key = prefix.clone().to_vec();
    let mut reads: u64 = 1;
    let mut writes: u64 = 0;
    while let Some(next) =
        sp_io::storage::next_key(&previous_key).filter(|n| n.starts_with(&prefix))
    {
        reads.saturating_accrue(1);
        previous_key = next;
        let maybe_value = unhashed::get::<T>(&previous_key);
        match maybe_value {
            Some(_) => continue,
            None => {
                log::debug!(
                    "Removing key which could not be decoded at {:?}",
                    previous_key
                );
                unhashed::kill(&previous_key);
                writes.saturating_accrue(1);
                continue;
            }
        }
    }

    (reads, writes)
}

1 Answer 1

1

if the original issue is coming from decoding keys in the iter_keys of StorageDoubleMap, I would manually iterate all the keys and try to decode them. Also, in your current code, in unhashed::get::<T>(&key) T is the type of value not the key. So, afaiu, that line could be misleading when there is an invalid storage key that has some decodable value to T.

I would do something like this (this is basically modified implementation of IterableStorageDoubleMap::iter_keys()):

fn remove_undecodable_storage_keys<
   K1: Decode + Sized, 
   K2: Decode + Sized, 
   Hasher1: ReversibleStorageHasher, 
   Hasher2: ReversibleStorageHasher
>
(
    prefix: [u8; 32],
){
        // ... inside `while` loop
        reads.saturating_accrue(1);
        previous_key = next;

        if let Some(raw_key_without_prefix) = previous_key.get(32..) { 
                let mut k1_k2_material = Hasher1::reverse(raw_key_without_prefix);
                let k1 = match K1::decode(&mut k1_k2_material) {
                    Some(_) => // do nothing,
                    None => // kill the previous key and continue with iteration
                }
                let mut k2_material = Hasher2::reverse(k1_k2_material);
                let k2 = match K2::decode(&mut k2_material) {
                    // same thing as `k1`
                }
        }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.