OpenLayers – remove vertex from geometry bud disable editing (adding or dragging vertex)

I need in OpenLayers to make the Modify tool work only for deleting geometry vertices. I only managed to get the vertex to delete after clicking on the snapped vertex, but unfortunately it is still possible to grab the vertex and change its coordinates by dragging.

My code:

this._removeVertex = new Modify({ source: source, insertVertexCondition: never, deleteCondition: singleClick });

The problem is that when I add a condition for the Modify tool (for example condition: click) deleteCondtion doesn’t work for some reason.

So I tried to work around it this way:

let allowDragging = true;

this._removeVertex.on('modifystart', () => {
    allowDragging = false;
});

this._removeVertex.on('modifyend', () => {
    allowDragging = true;
});

this._map.on('pointermove', (event) => {
    if (!allowDragging) {
        event.stopPropagation();
    }
});

however, this disables dragging vertex after some time, so it’s not a good solution.

Question is:

Is there a solution in OpenLayers where one can only delete vertices from the drawn geometry using singleClick but disable any other editing at the same time?

Thank you