Disable scrolling on visual viewport size changes (e.g. when zooming)

When zooming in, the visual viewport size changes, and the scrollbars appear. Is there a way to disable scrolling in that situation?

Note that I mean zooming by touchpad (pinch zooming), not by ctrl + mouse wheel.

gif demo

I just want to disable scrolling for the visual viewport, not other possible scrollable elements.
(I’ve already found a way to disable scrolling for all scrollable elements here: How to disable scrolling temporarily?)

And note that overflow:hidden obviously doesn’t relate to this problem