I am learning JavaScript (Node.js – using the Pipedream platform). I have been writing scripts to help automate some little tasks in my day to day work.
I am creating one that generates a report on recent interactions with clients.
As part of this I am using axios to get “engagements” from the Hubspot API (basically a list of identifiers I will use in later requests).
The API returns paginated responses. I have encountered pagination previously and understand the principle behind it, but have never written a script to handle it. This is my first.
It works. But I feel it could be improved. Below I’ve commented how I’ve approached it.
The endpoint returns up to 100 values ‘per page’ along with a "hasMore":true
flag and an "offset":987654321
value which can be passed as a query parameter in subsequent requests (if hasMore === true).
Example API response:
{"results":[1234,1235,1236],"hasMore":true,"offset":987654321}
My code:
import axios from 'axios';
//function to get each page of data
async function getAssoc(req){
const options = {
method: 'GET',
url: `https://api.hubapi.com/${req}`,
headers: {
Authorization: `Bearer ${auths}`,
},
};
return await axios(options);
}
//declare array in which to store all 'associations'
const assocs = [];
//store the ID that I get in an earlier step
const id = vid;
//declare variable in which to temporarily store each request response data
var resp;
//declare query parameter value, initially blank, but will be assigned a value upon subsequent iterations of do while
var offset = '';
do {
//make request and store response in resp variable
resp = await getAssoc(`crm-associations/v1/associations/${id}/HUBSPOT_DEFINED/9?offset=${offset}`);
//push the results into my 'assocs' (associations) array
resp.data.results.forEach(element => assocs.push(element));
//store offset value for use in next iteration's request
offset = resp.data.offset;
} while (resp.data.hasMore); //hasMore will be false when there's no more records to request
return assocs;
I feel it could be improved because:
- The DO WHILE loop, I believe, is making sequential requests. Is parallel a better/faster/more efficient option?
- I’m re-assigning new values to vars instead of using consts which seems simple and intuitive in my beginner’s mind, but I don’t understand a better way in this instance.
- I would welcome any feedback or suggestions on how I can improve this for my own learning.
Thank you in advance for your time and any assistance you can offer.