I am working on my university project. I want to extract data from my university LMS; I cracked the login authentication, and now I want to extract HTML of my subjects attendance and sessional marks data through Puppeeteer.
Here are my LMS credentials so you can understand the HTML code:
Link: https://my.kfueit.edu.pk/
username: COSC241103046
password: Sarmad.09
After entering the ‘home page’, go to ‘View my current enrollment.’
On this page, you can see all course details and 2 buttons, “Attendance” and “Sessional Marks.”
I want to get the course details like the course code, course name, credit hours, and section, and after all of that, click on the attendance button; a popup will appear; get the html code inside that; close the popup; click on the sessional marks button; a popup will show; get all the html code inside that; close the popup. This process repeats for all courses.
I tried to map through all courses, but it gives the wrong response.
if (targetFrame) {
await targetFrame.click('.fa.fa-book');
await targetFrame.waitForSelector('tbody');
const data = await targetFrame.evaluate(async () => {
const rows = Array.from(document.querySelectorAll('tbody tr')).slice(1);
console.log('33333333',rows)
return Promise.all(rows.map(async row => {
const cells = row.querySelectorAll('td');
const rowsData = {
courseCode: cells[0]?.innerText.trim(),
courseName: cells[1]?.innerText.trim(),
creditHours: cells[2]?.innerText.trim(),
sections: cells[3]?.innerText.trim(),
};
const buttons = cells[4]?.querySelectorAll('button');
if (buttons) {
buttons[0].click();
await new Promise(resolve => setTimeout(resolve, 500));
const attendancePopup = document.querySelector('#courseAttendaceModel');
const secondTable = attendancePopup?.querySelectorAll('table')[1];
if (secondTable) {
const attendanceRows = Array.from(secondTable.querySelectorAll('tr')).slice(1);
rowsData.attendance = attendanceRows.map(attendanceRow => {
const attendanceTds = attendanceRow.querySelectorAll('td');
return {
td2: attendanceTds[2]?.innerText.trim() || 'No data',
td3: attendanceTds[3]?.innerText.trim() || 'No data',
td5: attendanceTds[5]?.innerText.trim() || 'No data',
};
});
} else {
rowsData.attendance = 'No attendance data';
}
const closeButton = attendancePopup?.querySelector('.btn.btn-default');
if (closeButton) {
closeButton.click();
await new Promise(resolve => setTimeout(resolve, 500)); // Adjust delay to ensure popup closes
}
}
return rowsData;
}
));
});
console.log('enrollmentData', JSON.stringify(data, null, 2));
}
After that, I tried to get only one course detail first, but it gave empty string.
const data = await targetFrame.evaluate(() => {
const rows = document.querySelectorAll('tbody tr:nth-child(2) td');
return {
courseCode: rows[0]?.innerHTML.trim() || '',
courseName: rows[1]?.innerHTML.trim() || '',
creditHours: rows[2]?.innerHTML.trim() || '',
section: rows[3]?.innerHTML.trim() || '',
};
});
console.log('data: ', data)
I don’t know where I am wrong, but my end goal is to get attendance and sessional marks data for all courses.
Kindly help me to solve this and feel free to ask any question for clarification.
