12

I stumbled upon an unexpected behavior when doing storage migration.

Problem Description

I want to clean-up a large DoubleStorageMap in multiple steps. And that works just fine, however I also want to be able to test this using try-runtime. So I modified the code to recursively call the clean-up when try-runtime feature is enabled.

The code looks something like this:

        pub fn storage_cleanup<T: Config>(weight_limit: Weight) -> Weight {
            ...
            let limit = ...;
            match SomeDoubleStorageMap::<T>::remove_all(Some(limit)) {
                KillStorageResult::AllRemoved(removed_entries_num) => {
                    consumed_weight += deletion_weight * removed_entries_num as u64;
                }
                KillStorageResult::SomeRemaining(removed_entries_num) => {
                    consumed_weight += deletion_weight * removed_entries_num as u64;
                    if cfg!(feature = "try-runtime") {
                        return consumed_weight + storage_cleanup::<T>(weight_limit);
                    } else {
                        return consumed_weight;
                    }
                }
            }
            ...
        }

Scenario

  • Let's say that initially, SomeDoubleStorageMap has 3000 entries.
  • Limit is set to 100 meaning that at most, we'll delete 100 entries in one call.
  • I want to emphasize again that it works just fine when tested on-chain where each deletion is part of another block.
  • When this is tested with try-runtime, it will enter an infinite loop.
  • I've tried printing removed_entries_num and size of the storage map and it goes like this:
    • First Iteration
      • Initial map size is 3000
      • 100 entries are removed, 2900 entries remain
    • Second Iteration
      • Initial map size is 2900
      • entries are removed, 2900 entries remain
    • Previous step is performed until stack is exhausted

remove_all provides info as if something was removed, but the size doesn't change. I assume it has to be related to the fact I'm doing it all as part of a single block.

Question

Is this a bug or expected behavior? I've checked the docs and the code but couldn't find why this happens.

1 Answer 1

4

This is expected behavior. At least for now. I opened the following pr to improve the docs around this: https://github.com/paritytech/substrate/pull/11182

1
  • Thank you for the answer! I've went over the PR you linked and watches Shawn's video about storage (youtube.com/watch?v=9S8rmW8LD5o). It's more clear to me know what happens.
    – Dinonard
    Apr 8, 2022 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.