Mongoose append element to array if doesn’t exist remove if it does

I have a user model that can have an array of images;

const UserSchema = new mongoose.Schema({
  email: {
    type: String
  },
  password: {
    type: String
  },
    favorites:[imageschema]
});

I have a button on my frontend that I need to toggle the “favorite” status of the image; i.e. have it present in the “favorites” array of my user.

So essentially I want to get the user and then I want it to remove the image from the favorites array if its present in it, add it to the array if its not.

Assuming I use my user’s email address to find his/her document like so;

User.Find({email:email}).exec.then(...)

What do I need to add to accomplish this?