Factors affecting storage field key generation in ink!

Question

I've got a couple of questions regarding the storage fields' key generation using AutoKey in the context of upgradable contracts and partial ManualKey usage.

1. Does the use of ManualKey in some storage fields impact the keys generated by AutoKey for the rest of the fields?

2. Can the change in the ordering of the fields which uses ManualKey impact the fields (keys) generated by AutoKey?

3. Should we use ManualKey for every storage field in the case of upgradable contracts?

   3.a. If so does that imply that each field will be loaded lazily? and does this result in higher read/write overhead?

4. Can the use of sequentially numbered ManualKeys cause collisions?

5. What other things to keep in mind while using ManualKey for upgradable contracts?

A sample storage struct for reference:

#[ink(storage)]
pub struct SampleStruct {
    A: AccountId,
    B: Mapping<AccountId, Balance, ManualKey<100>>,
    C: String, 
    D: Lazy<Vec<AccountId>, ManualKey<101>>,
}

Answer 1

1. In your example - not. Each field has a unique auto-storage key based on the field's name.

   In more complex cases like the one below, the ManualKey of the Parent may affect the storage keys of the Child type:

   #[ink::storage_item]
   pub Parent {
       child_1: Child,
       child_2: Child<ManualKey<123>>,
   }
   
   #[ink::storage_item]
   pub struct Child<ParentKey: StorageKey = AutoKey> {
       A: AccountId,
       B: Mapping<AccountId, Balance, ManualKey<100, ParentKey>>,
       C: String, 
       D: Lazy<Vec<AccountId>, ManualKey<101, ParentKey>>,
   }
   

2. No. The field ordering works regardless of AutoKey or ManualKey. Because the auto storage key is generated based on the field's name, not on the position like in Solidity, so you can treat ManualKey in the same way as AutoKey, but instead of the name, you use a pre-defined number.

   The ordering matters for the layout of your structure. You can change the order of layoutless fields without any problem. You can put them in any order or place, shuffle as you wish. You can't change the order of layoutful fields or remove them.

   The field is layoutless if its type is Lazy, Mapping, or layoutless type. The type is layoutless if all its fields are layoutless. The field is layoutful if it is not layoutless=)

   Consequently, all packed are layoutful by default, but not each non-packed type is layoutless.

3. You can use ManualKey if you plan to change the field's name in the future, or if you are afraid that ink! will change the storage key calculation strategy; otherwise, AutoKey should be enough for you.

   Regarding upgradability: It depends on your use case. Making each field layoutless allows you to add/remove/shuffle them as you wish. It decreases the chance of corrupting the storage layout. However, doing that for every storage field may not always be necessary or efficient.

   a. Yes, it adds more overhead, but also may be useful in case of heavy fields that you don't use too much(so it's okay to load them lazily not too often). More about that you can read here.

4. In your example - no, if each key is unique within one struct. In the complex example from my answer to question 1: the usage of the ManualKey when you want to use the same type several times causes problems if you don't specify <ParentKey: StorageKey = AutoKey> during the declaration of the Child type and don't pass the ParentKey into ManualKey<101, ParentKey>.

5. I will put notes regarding upgradable contracts overall:

   1. The Usage of the ManualKey or AutoKey for packed types is useless.
   2. Use ManualKey if you plan to change the name of the field or are afraid that ink! can change the algorithm of the storage key calculation. Otherwise, AutoKey should be enough.
   3. Each non-packed type should specify <ParentKey: StorageKey> generic during declaration to avoid a potential collision with other types. It guarantees that if types have the same field's names the auto storage key will be unique.
   4. If you want to use ManualKey for your fields, then you need to propagate the parent storage key to each of your no-packed fields with ManualKey like ManualKey<123, ParentKey>.
   5. The fields that can be updated in the future should be layoutless. The fields that will remain during the full lifecycle of the contract or can be dropped/ignored later can be layoutful. 5a. To be able to drop the field, you can put it into Option. And if you don't need the field anymore, you can always set it to None.

   You can read the suggestions from the OpenBrush on how to write upgradable contracts.

Answers on the follow-up question:

If the ManualKey assigned to each field is unique, Can they still overlap with each other?

Storage keys can overlap in the complex struct case if you didn't propagate the parent storage key.

Is it possible that a given field (maybe some complex struct?) occupies more than one storage key?

Technically the field occupies either zero or one storage slot. But the type itself may have many fields that occupy many storage slots. For example, the Mapping itself doesn't occupy any storage slot, but each element occupies its own single storage.

Answer 2

answer for your questions about storage field key generation using AutoKey and ManualKey in the context of upgradable contracts.

1 - Does the use of ManualKey in some storage fields impact the keys generated by AutoKey for the rest of the fields?

No, using ManualKey in some storage fields does not impact the keys generated by AutoKey for the rest of the fields. AutoKey generates sequential integer keys starting from 0, regardless of whether some fields use ManualKey or not. So, you can use ManualKey for some fields and AutoKey for others without any issues.

2 - Can the change in the ordering of the fields which uses ManualKey impact the fields (keys) generated by AutoKey?

No, the ordering of the fields that use ManualKey does not impact the keys generated by AutoKey. AutoKey generates keys sequentially based on the number of storage fields, regardless of their order or the use of ManualKey.

3 - Should we use ManualKey for every storage field in the case of upgradable contracts?

It depends on your use case. ManualKey can be useful for controlling the storage key of a specific field, which can be important for upgradable contracts. However, using ManualKey for every storage field may not always be necessary or efficient.

3a - If so does that imply that each field will be loaded lazily? and does this result in higher read/write overhead?

Yes, if you use Lazy storage for fields that use ManualKey, it implies that those fields will be loaded lazily, which can result in higher read/write overhead. However, if you don't use Lazy storage, the field will be eagerly loaded.

4 - Can the use of sequentially numbered ManualKeys cause collisions?

Yes, if you use sequentially numbered ManualKeys for multiple fields, it can cause collisions. It's important to ensure that each ManualKey value is unique within the storage struct.

Here are some things to keep in mind while using ManualKey for upgradable contracts:

Make sure to choose unique values for each ManualKey to avoid collisions. Using Lazy storage for fields that use ManualKey can result in higher read/write overhead. Be mindful of the storage size and cost when choosing ManualKey values. If you plan on upgrading the contract, be sure to consider the impact on storage layout and key generation when making changes to the storage struct.