Safari performance of custom mouse with javascript very low with Google Maps on page

I have this issue with smoothness on my webpage on safari, in particular a custom cursor which is set as following:

Moving the cursor (get called every mousemove)

const moveCursor = (e)=> {
  const mouseY = e.clientY;
  const mouseX = e.clientX;
  $cursor.style.transform = `translate(calc(${mouseX}px - ${$cursor.offsetWidth / 2}px), calc(${mouseY}px - ${$cursor.offsetWidth / 2}px))`; 
} 

When loading a google maps via Javascript even without adding any styling or markers, you can see a huge decrease in performance on the page.

Loading the map once

function initMap() {
    map = new google.maps.Map($map, {
             center: location,
             zoom: zoom,
             disableDefaultUI: true
          }
}

initMap();

Any idea how to prevent this from happening and have smooth cursor? when using a normal cursor you see the performance increases.

NOTE: This is particular in Safari, Chrome and others are totally not an issue