I am currently working on a project that contains 160 million rows in a table. Directly fetching the data and converting it to CSV using sequilize Table.findAll() results in heap out of memory errors.
How should I efficiently provide CSV export to the client.
I am currently trying to implement stream to get data in chunks and create a csv in it, but not able to do it.
Note:
The CSV generated through dbeaver is of 50MB size.
Frontend: React
Backend: ExpressJs with Sequelize
Database: Postgresql