Blob data conversion to image URLs taking 200x longer the second time

Using React, I am loading blob data from an API and saving it in cache using React Query. I am then converting those blobs into image URLs using URL.createObjectURL(blob). The first time the page loads, all of this goes just fine. If I navigate away from, and return to the page (using React Router), the same exact code loads the same exact images, yet it takes 200x longer. I am having a really hard time imagining why this is happening.

Network tab showing first, then second conversion:

On the side, I am caching the converted image URLs in memory, but even if I strip out all of this caching code, the same long conversion times occur. I will paste the blob conversion and caching code here however.

import { useState, useEffect, useRef } from "react";
import { useQuery } from "@tanstack/react-query";

import { downloadProfileImage } from "../services/api.js";

const imageUrlCache = {};

export function useProfileImage(userId, profileAvatarUpdateTime) {
  const [imageUrl, setImageUrl] = useState(null);
  const urlRef = useRef(null);
  const cacheKey = `${userId}@${profileAvatarUpdateTime}`;

  const { data: blob } = useQuery(
    downloadProfileImage(userId, profileAvatarUpdateTime)
  );

  useEffect(() => {
    if (blob) {
      if (cacheKey in imageUrlCache) {
        urlRef.current = imageUrlCache[cacheKey].imageUrl;
        imageUrlCache[cacheKey].subscribed++;
      } else {
        urlRef.current = URL.createObjectURL(blob);
        imageUrlCache[cacheKey] = {};
        imageUrlCache[cacheKey].imageUrl = urlRef.current;
        imageUrlCache[cacheKey].subscribed = 1;
      }
      setImageUrl(urlRef.current);
      // cleanup function to revoke object URL when the component unmounts
      return () => {
        if (urlRef.current) {
          imageUrlCache[cacheKey].subscribed--;
          if (imageUrlCache[cacheKey].subscribed < 1) {
            delete imageUrlCache[cacheKey];
            URL.revokeObjectURL(urlRef.current);
          }
          setImageUrl(null);
        }
      };
    }
  }, [blob, cacheKey]);

  return imageUrl;
}

Kind of struggling with this one. Anybody have any ideas what might be happening here?