JavaScript: How to grab thumbnail (first frame) of video with only raw byte data (mp4 and possibly other types)

I want to grab a thumbnail of a video (let’s say the thumbnail is at 0:00 of the video, so the first frame), however, I am streaming the video and want to display the thumbnail before the video fully loads (so fully loading it, saving it as a blob url and using the HTML5Video element isn’t an option).

Also, the URL that I am originally fetching the video from will not accept any more requests after I make the initial request to download the video, so I also can’t use the HTML5Video element to load that URL.

All I have access to are the raw bytes that are being streamed in from the response. So my question is how can I grab the first frame of the video when just enough bytes have loaded for it. Are there any tools for this, or will I have to interpret the file contents myself?

If I were to build my own solution, I assume the steps would be something like:

  • Wait for file metadata/headers to load first.
  • get the dimensions (x.y) of the video from the metadata.
  • the next x.y bytes will be the first frame of the video. Store this as an ArrayBuffer and create a data URL to use with an HTML img tag.

Let’s start with an mp4 solution but a solution to other formats would be appreciated as well, thanks in advance :).