I’m encountering a RangeError: Array buffer allocation failed error in my JavaScript application after refreshing the page multiple times. The error occurs even though the memory usage appears to be well within limits. Here are the details from the last time I hit this error:
JavaScript Heap Size Limit: 4,294,705,152 bytes (approximately 4.29 GB)
Total JavaScript Heap Size: 300,268,829 bytes (approximately 300 MB)
Used JavaScript Heap Size: 292,110,613 bytes (approximately 292 MB)
Despite using less than 10% of the available memory, the error occurs after about 5 page refreshes. I have to clear the browser data and refresh the page to temporarily resolve the issue (ctrl-shift-r in chrome for hard refresh).
From my research into the issue, I’m guessing that it has to do with memory fragmentation, but that still seems pretty absurd to me that the memory is so fragmented that it can’t find a contiguous block of around 100mb when there are still gigabytes available. Unfortunately everything I found online never had a real resolution beyond “just use smaller array buffers but there’s no way to know exactly how much.
Some context on what I’m working on:
I’m playing around with generating and rendering simplex noise with web workers. I use shared array buffers to get that data between the workers and the main thread without incurring the cost of serializing that data.
If I’m rendering a 5000×5000 image of noise, I need a shared array buffer large enough to hold 25,000,000 f32s, which is about 100mb. Seeing as I have “4gb available”, I figured this is no big deal to hold a few of these at a time. Unfortunately if I do a normal reload too many times I hit the error above.
If I’m right about the fragmentation, I could break this up into a bunch of smaller buffers. Unfortunately with that I think i would lose some performance because I can no longer just loop over the entire map in one contiguous block (new to this so not positive on that). The other problem with this is I don’t know what that magic number is where I’ll stop hitting this error. I could put a try catch around all buffer allocations and if one fails, split that buffer into 4 or something, but that would add a lot of complexity to my project.
What else can I do about this?