I am trying to create an app that takes a good picture of you, worth for analyzing your skin.
I have 3 main conditions:
- The face should be in specific coordinates (I have them hardcoded)
- I wanna detect if the person is looking straight, not up, down, etc.
- Have good lighting.
They are not important for this question, just wanted to give you the big picture.
The feature is gonna work only on mobile devices via the browser (not mobile application).
I am getting this weird error, when I am running my detectFace function:
Box.constructor - expected box to be IBoundingBox | IRect, instead have {"x": null, "y": null, "width": null, "height": null}
The weird this is, this thing only happens whenever I am using TinyFaceDetector, this is how I am using it:
useEffect(() => {
const loadModel = async () => {
try {
await Promise.all([
faceapi.nets.tinyFaceDetector.loadFromUri("/models"),
faceapi.nets.faceLandmark68TinyNet.loadFromUri("/models"),
]);
initializeCamera();
setIsModelLoaded(true);
} catch (error) {
console.error("Error loading models:", error);
}
};
const detectFace = async () => {
if (isModelLoaded && webcamRef.current) {
const video = webcamRef.current.video as HTMLVideoElement;
const detections = await faceapi
.detectAllFaces(
video,
new faceapi.TinyFaceDetectorOptions({
scoreThreshold: 0.1,
inputSize: 128,
})
)
.withFaceLandmarks(true);
if (detections.length > 0) {
const minX = 0;
const maxX = 75;
const minY = 165;
const maxY = 230;
if (
detections[0].detection.box.x !== null &&
detections[0].detection.box.x >= minX &&
detections[0].detection.box.x <= maxX &&
detections[0].detection.box.y >= minY &&
detections[0].detection.box.y <= maxY
) {
setIsFaceInShape(true);
} else {
setIsFaceInShape(false);
}
const brightness = await calculateAverageBrightness(video);
if (brightness < 150) {
setIsLightGood(true);
} else {
setIsLightGood(false);
}
}
}
};
loadModel();
const intervalId = setInterval(detectFace, 200);
return () => {
clearInterval(intervalId);
};
}, [isModelLoaded]);
useEffect(() => {
if (isFaceInShape && isLightGood) {
capture();
}
}, [isFaceInShape, isLightGood]);
The weird thing is that when the function capture goes through, I see the error on my screen, but in the background I can see the image perfectly taken and the rest of the logic in capture is working great, which means that error doesn’t do anything, it doesn’t stop the app from working, where is it coming from ?
What I tried:
I noticed that when I lower the timeout interval, the errors happen less often, but I need it to scan fast so the app works as intended.
What worked:
When I am using ssdMobilenetv1 it works without any errors, but its a very heavy model especially for mobile and the camera starts lagging so much, because it makes those large scans every 0.2 seconds. The error happens mainly with TinyFaceDetector, which I am trying to use, because for mobile, it works perfectly, but I need the withFaceLandmarks prop.
Let me know if you need more code from my app.