Resizing image without keeping the aspect ratio – three.js

So, I am trying to implement a feature of resizing an image like in PowerPoint: resizing the image without keeping the aspect ratio when the image is resized by dragging the edges.

For instance, if the user drags the top arrow upwards, the image height is increased, but the image is not moved at all and the bottom edge is not moved either. This logic also works when you rotate the image in powerpoint.

I am having some issues with my code in which when the top edge is dragged, the image is also moved horizontally. So, the horizontal vector of the top edge is NOT BLOCKED.

The important thing is that the solution should also work when the image is rotated.

I have no clue how to fix this.

This is the code for the resizing of the top edge:

            if (object === arrows[0]) {
                // Top edge
                // object.position.x = mesh.position.x;
                // object.position.z = mesh.position.z;
                const height = Math.abs((object.position.z - arrows[2].position.z) * cos + (object.position.x - arrows[2].position.x) * sin);
                mesh.scale.y = height/ (image.height * scale );
                // Get the direction from the current object to arrows[2]
                const direction = arrows[2].position.clone().sub(object.position).normalize();
                // Get the new position of the object
                const newPosition = object.position.clone().add(direction.multiplyScalar(height / 2));
                // Set the position of the object
                mesh.position.set(newPosition.x, mesh.position.y, newPosition.z); 
            }

I noticed that if I enable the object.position.x = mesh.position.x in the beginning of the if statement, the image is blocked from moving horizontally when the image IS NOT ROTATED. But when the image is rotated, the image moved very fast horizontally when dragging the top arrow.

If I change the code mesh.position.set(newPosition.x, mesh.position.y, newPosition.z); to mesh.position.set(mesh.position.x, mesh.position.y, newPosition.z); the image is blocked from moving horizontally when dragging the top edge, but then the bottom edge is moved.

So, my conclusions so far is that I cannot modify the code of the height, mesh.scale.y, direction, newPosition and the mesh.position.set. There is something else that I should add to block the horizontal vector of the top edge.