Error related to pdf.worker.min says ‘import’, and ‘export’ cannot be used outside of module code

I am getting this error when pushing a build to Vercel:

Failed to compile.
static/media/pdf.worker.min.50acc843.mjs from Terser
  x 'import', and 'export' cannot be used outside of module code
    ,-[18:1]
 18 |  *
 19 |  * @licend The above is the entire license notice for the
 20 |  * JavaScript code in this page
    :                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
    `----
Caused by:
    0: failed to parse input file
    1: Syntax Error
Error: 
  x 'import', and 'export' cannot be used outside of module code
    ,-[18:1]

I tried reading here, here, here and here, but couldn’t find a solution that worked for me. So basically it is somehow related to pdfjs or pdf-lib, because after installing them, it has occur.

I am using pdf-lib version 1.17.1 and pdfjs-dist version 4.5.136 (according to package.json)

This is how I use them in my code:

import { PDFDocument } from 'pdf-lib';
import * as pdfjsLib from 'pdfjs-dist';

// Polyfill for Promise.withResolvers
if (typeof Promise.withResolvers === 'undefined') {
  (Promise as any).withResolvers = function () {
    let resolve: (value: unknown) => void;
    let reject: (reason?: any) => void;
    const promise = new Promise((res, rej) => {
      resolve = res;
      reject = rej;
    });
    return { promise, resolve: resolve!, reject: reject! };
  };
}

pdfjsLib.GlobalWorkerOptions.workerSrc = new URL(
  'pdfjs-dist/legacy/build/pdf.worker.min.mjs',
  import.meta.url
).toString();