I have this gnarly FFmpeg command:
ffmpeg -i music.mp3 -i video.mp4 -i speech.mp3 -filter_complex "[0:a]atrim=0:$(ffprobe -i speech.mp3 -show_entries format=duration -v quiet -of csv=p=0),volume=0.08[trimmed_music]; [2:a]volume=2[speech]; [1:v]loop=-1,trim=duration=$(ffprobe -i speech.mp3 -show_entries format=duration -v quiet -of csv=p=0),setdar=9/16[vout]; [trimmed_music][speech]amix=inputs=2:duration=first[aout]" -map "[vout]" -map "[aout]" -c:v libx264 -crf 23 -preset medium -c:a aac -b:a 192k final_output.mp4
What it does is:
- Get the duration of a speech.mp3 file
- Crop a video.mp4 to portrait dimensions
- Trim a longer music.mp3 file to the duration of the speech
- Loop a the video and trim the final loop so that the whole thing matches the duration of the speech
- Adjust the volume of the music and speech
- Combine them all into a single video with talking (speech) and music
I can’t figure out how to run it using the FFmpeg wasm. I realise ffprobe isn’t a thing with the wasm so we’ll have to find a different way to get the duration of the speech.mp3 by probably breaking it up into 2 or more exec functions, but I have no idea how to do that, which is why I’m here asking for help.
For reference, here’s the function into which I want to insert this exec function, but feel free to change it however needed. And let me know if I need to provide more information.
const processVideo = async (speech, video, music) => {
const ffmpeg = new FFmpeg();
// ffmpeg loading code goes here, assume that part works without issue
await ffmpeg.writeFile("video.mp4", new Uint8Array(video));
await ffmpeg.writeFile("speech.mp3", new Uint8Array(speech));
await ffmpeg.writeFile("music.mp3", new Uint8Array(music));
await ffmpeg.exec([
// command(s) should go here
]);
const fileData = await ffmpeg.readFile("final_output.mp4");
const blob = new Blob([fileData.buffer], { type: "video/mp4" });
const blobUrl = URL.createObjectURL(blob);
return blobUrl;
};