4

I noticed that my migration is too big and will take up more than one block.

How do I do a multi-block migration?

2 Answers 2

3

Here is a real-world example.

https://github.com/darwinia-network/darwinia-common/pull/673/files#diff-02470251f30c921643bfe724902ce5cc7a76ebac129128e1328c061295e66034R123-R148

In the past, our MMR pallet records all the nodes.

And the chain has run for over a year.

We need to prune around 10,000,000 nodes to leaves only.

Also, the log is useful during the migration.

https://github.com/darwinia-network/darwinia-common/pull/673#issuecomment-866571487

And a migration configuration might be helpful.

That you could configure how many items are to be migrated in a block.

If the number is too large it will affect the block production speed.

https://github.com/darwinia-network/darwinia-common/pull/673/files#diff-02470251f30c921643bfe724902ce5cc7a76ebac129128e1328c061295e66034R113-R116

2

There are some resources (and ongoing effort) here:

Depending on the migration, you may want to have logic to put the chain in "maintenance-mode", disable the pallet, or disable the extrinsics or code that can modify that storage or depend on that storage until the migration is fully complete. If the chain is in "maintenance-mode", it is possible to do the migration outside the on_initialize and take up more block weight.

Below is a walkthrough of how pallet-xcm does a multi-block migration.

pallet-xcm uses the fn on_runtime_upgrade() hook in combination with the fn on_initialize() hook.

When the runtime is upgraded the on_runtime_upgrade hook kicks off the migration:

fn on_runtime_upgrade() -> Weight {
    // Start a migration (this happens before on_initialize so it'll happen later in this
    // block, which should be good enough)...
    CurrentMigration::<T>::put(VersionMigrationStage::default());
    T::DbWeight::get().writes(1)
}

Then at the beginning of each block the on_initialize sets a 10% block limit and kicks off the migrations in respect to that limit:

fn on_initialize(_n: BlockNumberFor<T>) -> Weight {
    let mut weight_used = Weight::zero();
    if let Some(migration) = CurrentMigration::<T>::get() {
        // Consume 10% of block at most
        let max_weight = T::BlockWeights::get().max_block / 10;
        let (w, maybe_migration) = Self::check_xcm_version_change(migration, max_weight);
        CurrentMigration::<T>::set(maybe_migration);
        weight_used.saturating_accrue(w);
    }
    ....
}

The migrations starts (being careful not to exceed max weight -> 10%):

/// Will always make progress, and will do its best not to use much more than `weight_cutoff`
/// in doing so.
pub(crate) fn check_xcm_version_change(
    mut stage: VersionMigrationStage,
    weight_cutoff: Weight,
) -> (Weight, Option<VersionMigrationStage>) {
    let mut weight_used = Weight::zero();

    // TODO: Correct weights for the components of this:
    let todo_sv_migrate_weight: Weight = T::DbWeight::get().reads_writes(1, 1);
    let todo_vn_migrate_weight: Weight = T::DbWeight::get().reads_writes(1, 1);
    let todo_vnt_already_notified_weight: Weight = T::DbWeight::get().reads(1);
    let todo_vnt_notify_weight: Weight = T::DbWeight::get().reads_writes(1, 3);
    let todo_vnt_migrate_weight: Weight = T::DbWeight::get().reads_writes(1, 1);
    let todo_vnt_migrate_fail_weight: Weight = T::DbWeight::get().reads_writes(1, 1);
    let todo_vnt_notify_migrate_weight: Weight = T::DbWeight::get().reads_writes(1, 3);

    use VersionMigrationStage::*;

    if stage == MigrateSupportedVersion {
        // We assume that supported XCM version only ever increases, so just cycle through lower
        // XCM versioned from the current.
        for v in 0..XCM_VERSION {
            for (old_key, value) in SupportedVersion::<T>::drain_prefix(v) {
                if let Ok(new_key) = old_key.into_latest() {
                    SupportedVersion::<T>::insert(XCM_VERSION, new_key, value);
                }
                weight_used.saturating_accrue(todo_sv_migrate_weight);
                if weight_used.any_gte(weight_cutoff) {
                    return (weight_used, Some(stage))
                }
            }
        }
        stage = MigrateVersionNotifiers;
    }
    if stage == MigrateVersionNotifiers {
        for v in 0..XCM_VERSION {
            for (old_key, value) in VersionNotifiers::<T>::drain_prefix(v) {
                if let Ok(new_key) = old_key.into_latest() {
                    VersionNotifiers::<T>::insert(XCM_VERSION, new_key, value);
                }
                weight_used.saturating_accrue(todo_vn_migrate_weight);
                if weight_used.any_gte(weight_cutoff) {
                    return (weight_used, Some(stage))
                }
            }
        }
        stage = NotifyCurrentTargets(None);
    }

    let xcm_version = T::AdvertisedXcmVersion::get();

    if let NotifyCurrentTargets(maybe_last_raw_key) = stage {
        let mut iter = match maybe_last_raw_key {
            Some(k) => VersionNotifyTargets::<T>::iter_prefix_from(XCM_VERSION, k),
            None => VersionNotifyTargets::<T>::iter_prefix(XCM_VERSION),
        };
        while let Some((key, value)) = iter.next() {
            let (query_id, max_weight, target_xcm_version) = value;
            let new_key: MultiLocation = match key.clone().try_into() {
                Ok(k) if target_xcm_version != xcm_version => k,
                _ => {
                    // We don't early return here since we need to be certain that we
                    // make some progress.
                    weight_used.saturating_accrue(todo_vnt_already_notified_weight);
                    continue
                },
            };
            let response = Response::Version(xcm_version);
            let message = Xcm(vec![QueryResponse { query_id, response, max_weight }]);
            let event = match T::XcmRouter::send_xcm(new_key.clone(), message) {
                Ok(()) => {
                    let value = (query_id, max_weight, xcm_version);
                    VersionNotifyTargets::<T>::insert(XCM_VERSION, key, value);
                    Event::VersionChangeNotified(new_key, xcm_version)
                },
                Err(e) => {
                    VersionNotifyTargets::<T>::remove(XCM_VERSION, key);
                    Event::NotifyTargetSendFail(new_key, query_id, e.into())
                },
            };
            Self::deposit_event(event);
            weight_used.saturating_accrue(todo_vnt_notify_weight);
            if weight_used.any_gte(weight_cutoff) {
                let last = Some(iter.last_raw_key().into());
                return (weight_used, Some(NotifyCurrentTargets(last)))
            }
        }
        stage = MigrateAndNotifyOldTargets;
    }
    if stage == MigrateAndNotifyOldTargets {
        for v in 0..XCM_VERSION {
            for (old_key, value) in VersionNotifyTargets::<T>::drain_prefix(v) {
                let (query_id, max_weight, target_xcm_version) = value;
                let new_key = match MultiLocation::try_from(old_key.clone()) {
                    Ok(k) => k,
                    Err(()) => {
                        Self::deposit_event(Event::NotifyTargetMigrationFail(
                            old_key, value.0,
                        ));
                        weight_used.saturating_accrue(todo_vnt_migrate_fail_weight);
                        if weight_used.any_gte(weight_cutoff) {
                            return (weight_used, Some(stage))
                        }
                        continue
                    },
                };

                let versioned_key = LatestVersionedMultiLocation(&new_key);
                if target_xcm_version == xcm_version {
                    VersionNotifyTargets::<T>::insert(XCM_VERSION, versioned_key, value);
                    weight_used.saturating_accrue(todo_vnt_migrate_weight);
                } else {
                    // Need to notify target.
                    let response = Response::Version(xcm_version);
                    let message =
                        Xcm(vec![QueryResponse { query_id, response, max_weight }]);
                    let event = match T::XcmRouter::send_xcm(new_key.clone(), message) {
                        Ok(()) => {
                            VersionNotifyTargets::<T>::insert(
                                XCM_VERSION,
                                versioned_key,
                                (query_id, max_weight, xcm_version),
                            );
                            Event::VersionChangeNotified(new_key, xcm_version)
                        },
                        Err(e) => Event::NotifyTargetSendFail(new_key, query_id, e.into()),
                    };
                    Self::deposit_event(event);
                    weight_used.saturating_accrue(todo_vnt_notify_migrate_weight);
                }
                if weight_used.any_gte(weight_cutoff) {
                    return (weight_used, Some(stage))
                }
            }
        }
    }
    (weight_used, None)
}

The stage of the migration is stored in storage, so that the migration can continue in the next block if needed.

/// The current migration's stage, if any.
#[pallet::storage]
pub(super) type CurrentMigration<T: Config> =
    StorageValue<_, VersionMigrationStage, OptionQuery>;

Another example here:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.