I need to insert records in a PG database from a json
file with 500k records.
Since the file is huge I’m creating a Read Stream and using JSONStream.parse
to send json
objects over to pipe
.
So far so good. This is not the problem, I’m just providing context.
After parsing the object, I need to insert the information using prisma
but I cannot insert records if certain field is already in the table. So first thing I think is that I should use an upsert
.
The problem is that this field is not a unique key of that table, therefore I cannot use it in the where
clause of the prisma upsert
.
Then, I did the following:
await prisma.pets
.findFirst({
where: { name: nameFromJson },
})
.then(async (existing_pet) => {
if (!existing_pet) {
await prisma.pets.create({
data: {
name: nameFromJson,
legs: numberOfLegs,
isAlive: isAlive,
},
})
}
})
.catch((error) => {
throw new Error(error)
})
My idea is to find the record with the same field first and when that promise is resolved, send the result to another promise where I can check the record and if it does not exist, then just shoot the create
.
But I keep getting duplicates in the table.
I’d like to understand what I’m doing wrong.