Is there a way I can prevent accidental selection on mobile for woocommerce website – User is scrolling through swatches

I have a website where users can choose swatches but the problem is when they are scrolling they end up selecting the colours which is a major issue. Is there a way we can limit it in CSS or js so it only selects an option when they are not scrolling. Please see the screenshot

The website is running on woodmart theme

enter image description here