react useEffect handleKeyDown dubbleing output with every entry

every input results in doubble the amount of outputs each time resulting in getting stuck in a long loop after just a few inputs.
this is the code

  const [direction,setDirection] = useState(Array(1).fill(0));
  const directions = ["botNorth","botEast","botSouth","botWest"];
  const [squares, setSquares] = useState(Array(5).fill().map(()=> Array(5).fill(null)));

  useEffect(()=> {
    const handleKeyDown = (e) =>{
      e = e || window.event;
      const nextSquares = squares.slice();
      var nextDirection = direction.slice();
      console.log(e.which);
      if (e.keyCode == '38') {
          // up arrow
      }
      else if (e.keyCode == '37') {
         // left arrow
         nextDirection[0]--;
         if (nextDirection[0] < 0) {
          nextDirection[0] = 3;
         }

      }
      else if (e.keyCode == '39') {
         // right arrow
      }
      setDirection(nextDirection);
      nextSquares[location[0]][location[1]] = <img className={directions[nextDirection[0]]} src={bot}/>;
      setSquares(nextSquares);
    return () => document.removeEventListener('keydown', handleKeyDown);
    };
    document.addEventListener('keydown', handleKeyDown, true);
});

this is the resulting error each key was only pressed once
screenshot of error

I’ve tryed moving the the add and remvove listeners around both inside and outside the function but it doesnt seem to make a difference
when i comment out both setSquare() and setDirection() it outputs twice per input instead of doubbleing each time