Using latest Papaparse to parse large CSV files.
It handles double quotes in the value but not when value starts with double quotes.
Using this code:
const parsePromise = new Promise<void>((resolve, reject) => {
Papa.parse<Equipment>(fileStream, {
header: true,
delimiter: "t",
dynamicTyping: true,
skipEmptyLines: true,
step: (result) => {
const rowData = {
vehicle_id: result.data.vehicle_id,
schema_id: result.data.schema_id,
option_id: result.data.option_id,
record_id: result.data.record_id,
location: result.data.location,
data_value: result.data.data_value,
condition: result.data.condition,
};
entities.push(rowData);
console.log(rowData)
},
complete: () => resolve(),
error: (error) => reject(error),
});
});
If I have the following csv data:
vehicle_id schema_id option_id record_id location data_value condition
425972620240523 15102 1266 7700 W "Första hjälpen"- förbandslåda med varningstriangel, 2 varselvästar
425972620240523 15104 1266 7700 W W
425972620240523 15101 1266 7800 INT S
425972620240523 15102 1266 7800 INT medical kit, warning triangle, 2 safety vests
425972620240523 15104 1266 7800 INT INT
425972620240523 15101 1267 7900 W S
425972620240523 15102 1267 7900 W Papperskorg (borttagbar)
It outputs
{
vehicle_id: 425972620240523,
schema_id: 15102,
option_id: 1266,
record_id: 7700,
location: 'W',
data_value: 'Första hjälpen"- förbandslåda med varningstriangel, 2 varselvästartrn' +
'425972620240523t15104t1266t7700tWtWtrn' +
'425972620240523t15101t1266t7800tINTtStrn' +
'425972620240523t15102t1266t7800tINTtmedical kit, warning triangle, 2 safety veststrn' +
'425972620240523t15104t1266t7800tINTtINTtrn' +
'425972620240523t15101t1267t7900tWtStrn' +
'425972620240523t15102t1267t7900tWtPapperskorg (borttagbar)trn',
condition: undefined
}
If I move the first double quote as in:
vehicle_id schema_id option_id record_id location data_value condition
425972620240523 15102 1266 7700 W Första "hjälpen"- förbandslåda med varningstriangel, 2 varselvästar
425972620240523 15104 1266 7700 W W
425972620240523 15101 1266 7800 INT S
425972620240523 15102 1266 7800 INT medical kit, warning triangle, 2 safety vests
425972620240523 15104 1266 7800 INT INT
425972620240523 15101 1267 7900 W S
425972620240523 15102 1267 7900 W Papperskorg (borttagbar)
The result is correct:
{
vehicle_id: 425972620240523,
schema_id: 15102,
option_id: 1266,
record_id: 7700,
location: 'W',
data_value: 'Första "hjälpen"- förbandslåda med varningstriangel, 2 varselvästar',
condition: null
}
{
vehicle_id: 425972620240523,
schema_id: 15104,
option_id: 1266,
record_id: 7700,
location: 'W',
data_value: 'W',
condition: null
}
....
How can Papaparse handle values starting with a double quote?