How can I properly encode a GLTF while also preserving the encoding of other meshes and textures?

I’m building a web ar app that allows users to wear GLTF head models. (https://www.head5.camera/) In order to get the correct lighting on the GLTF, I have set renderer.outputEncoding = THREE.sRGBEncoding and this works great BUT it also adjusts the lighting of my THREE.VideoTexture from my video stream. Is there a way to encode just the GLTF but preserve the LinearEncoding? of the rest of the scene. Any help is much appreciated.

Here’s an example of what I mean:

The background is lighter than it is supposed to be because it is also being encoded.