Insert is just for inserting or overwriting any existing value. If you just want to put a specific value into storage, then use that.
Mutate is meant for cases where you want to modify the existing value or want to do some other case-decision depending on the current value.
For example when you want to increment a value, it can be written shorter with mutate
:
let v = match MyInt::<T>::get() {
// Calculate new value
};
MyInt::<T>::set(&v);
// But it is shorter with mutate
MyInt::<T>::mutate(|v| match v {
// Calculate and set new value
}