I’m working on a screen recording and streaming setup where the user records their entire screen and streams it to Twitch. The setup works fine initially, but when I switch tabs during recording, the stream breaks on the backend, and I get the following FFmpeg errors:
FFmpeg STDERR: [matroska,webm @ 0x7f9dcb904580] EBML header parsing failed
[in#0 @ 0x7f9dcb904380] Error opening input: Invalid data found when processing input
Error opening input file -.
Error opening input files: Invalid data found when processing input
My frontend code captures the screen and microphone and streams it via a WebSocket to the backend, where FFmpeg processes the stream. Below is my relevant frontend code:
const startRecording = async () => {
try {
const screenStream = await navigator.mediaDevices.getDisplayMedia({
preferCurrentTab: true,
systemAudio: 'include',
surfaceSwitching: 'include',
monitorTypeSurfaces: 'include',
video: {
displaySurface: 'browser',
height: 720,
width: 1280,
frameRate: { ideal: 24, max: 30 },
},
});
screenStream.getVideoTracks()[0].onended = () => {
console.log('Screen sharing ended. Stopping the recorder.');
stopRecording();
};
const micStream = await navigator.mediaDevices.getUserMedia({
audio: true,
});
const combinedStream = new MediaStream([
...screenStream.getVideoTracks(),
...micStream.getAudioTracks(),
]);
const recorder = new MediaRecorder(combinedStream, {
mimeType: 'video/webm; codecs=vp8,opus',
videoBitsPerSecond: 3 * 1024 * 1024,
});
const timeslice = 1000;
recorder.ondataavailable = async (event) => {
if (socket?.current?.connected && event.data.size > 0) {
console.log('Sending chunk data:', socket.current.id);
socket?.current.send(event.data);
recordedChunks.current.push(event.data);
} else if (!socket?.current?.connected) {
handleSocketDisconnection();
}
};
mediaRecorder.current = recorder;
recorder.start(timeslice);
setIsRecording(true);
} catch (error) {
console.log('Error starting screen recording:', error);
toast.error('Failed to start screen recording: ' + error);
}
};
const stopRecording = () => {
if (socket?.current && mediaRecorder) {
mediaRecorder?.current?.stop();
socket.current.close();
setIsRecording(false);
downloadRecordedVideo();
}
};
And here’s my backend code with FFmpeg settings for Twitch streaming:
const inputSettings = [
'-f', 'webm', '-i', '-', '-v', 'error', '-analyzeduration', '1000000', '-probesize', '5000000',
];
const twitchSettings = (twitch) => {
return [
'-c:v', 'libx264', '-preset', 'veryfast', '-tune', 'zerolatency',
'-g', '60', '-b:v', '2500k', '-maxrate', '3000k', '-bufsize', '8000k',
'-r', '30', '-vf', 'tpad=stop_mode=clone:stop_duration=2',
'-c:a', 'aac', '-ar', '44100', '-b:a', '96k',
'-use_wallclock_as_timestamps', '1', '-async', '1',
'-err_detect', 'ignore_err', '-reconnect', '1',
'-reconnect_streamed', '1', '-reconnect_delay_max', '5',
'-y', '-f', 'flv', twitch,
];
};
Problem: When switching tabs during screen sharing, it seems like the frame rate drops or the stream gets interrupted, leading to FFmpeg errors like EBML header parsing failed and Invalid data found when processing input. I suspect this happens because the browser deprioritizes resources when the tab is not active, which might lead to corrupt chunks being sent to FFmpeg.
Questions:
- Could switching tabs during screen capture be causing the issue by disrupting the frame rate or dropping frames?
- Is there a way to ensure FFmpeg doesn’t break due to these interruptions?
- Any suggestions on handling the stream more reliably when switching tabs or optimizing the FFmpeg setup for this scenario?
I tried adjusting the bitrate, frame rate, and buffer size but still experienced the same issue. I’m trying to figure out if the issue is related to how browsers handle screen capture when tab switching or something specific with FFmpeg handling the video stream.
Any insights would be greatly appreciated.
Thanks in advance!