I have this basic example of the problem:
//main.js
const { Worker } = require('worker_threads');
const worker = new Worker('./worker.js');
worker.on('message', function (message) {
//no-op
});
And:
//worker.js
const { parentPort } = require('worker_threads');
while (1) {
const message = {foo: true};
parentPort.postMessage(message);
}
This causes a rapid growth in memory usage. Within seconds it’s gigabytes. Within a couple of minutes it’s all the system memory, and the process is killed. I’ve tried adding delete message;
after the postMessage() call, and in the message handler of main.js. Neither of these seem to have any effect. I’ve tried running it natively, and in various node based docker containers; it’s always the same.
My questions are:
-
Why does it work this way? Why isn’t the object being garbage collected when main.js’s message handler is returning? And if it’s a case of postMessage() carelessly growing some message queue in the parent thread with freshly created objects, why does it do that, rather than waiting?
-
How do I get around this?
I imagine someone is thinking “Why would you send so many messages?”. The answer is to demonstrate the problem. In the real scenario it was larger but much more infrequent message sends, and that caused a much slower but still completely unacceptable memory leak.
What’s the deal here?