Category: javascript

Category Added in a WPeMatico Campaign

InertiaJS router.get request with params not preserving state: always full refreshing page

In the docs, we can find:

State preservation

…
You can instruct Inertia to preserve the component’s state when using the get method by setting the preserveState option to true.

router.get('/users', { search: 'John' }, { preserveState: true })

…

I use this in my React application precisely for searching & filtering rows in, for example, a datatable.

For that, I have binded that search string to a text input content, so changing it launches a GET request that, in turn, changes the URL, e.g. http://localhost/example?search=Johnny. And because of that URL change, Inertia then always triggers a full page reload, making preserveState: true option totally useless in my case…

How is the “search functionality” supposed to be implemented then? What am I failing at?

Thanks in advance.

How to prevent slide change on “Tab” click in Swiper js

The title basically says it all. Using Tab on my swiper breaks it for some reason so I need to prevent it from switching slides after it went through all the clickable on tab elements in each of them. I’d prefer just using the navigation and manually clicking on each of them to go through the swiper. I don’t think there’s a need to provide any code, but here’s the options I used at least:

const swiper = new Swiper('.element', {
      loop: false,
      autoplay: false,
      autoHeight: true,
      keyboardControl: false,
      slidesPerView: 1,
      slidesPerGroup: 1,
      spaceBetween: 32,
      allowTouchMove: false,
      navigation: {
        nextEl: '.btn_right',
        prevEl: '.btn_left',
      },
    });

I tried using keyboardControl: false and also keyboard: {enabled: false, onlyInViewport: false} in the options, but that didn’t work

Cannot orchestrate typescript + node + ESNext modules

So I want to publish a typescript module.

I depend on some node resources: node-fetch & file-type.

I have played with a variety of settings to get imports to work across the board, but I haven’t figured it out.

Current state:

package.json:

  "type": "module"

tsconfig.json:

  "module":"ESNext", 
  "esModuleInterop": false,

In this mode, my attempt to import my dependencies simply fails.
Turning the interop to true doesn’t make a difference.

If I set "module":"commonjs"

Then I can import my dependencies, but nobody can see my exports:

import { caption } from "@m87/mdc-ts";
         ^^^^^^^
SyntaxError: The requested module '@m87/mdc-ts' does not provide an export named 'caption'

And yet, when I look at the output of the typescript build, I see:

const caption = async (file) => {
  // Code would be here
};
exports.caption = caption;

My test file is also set with "type":"module" in it’s own package.json.

Now, if I wasn’t importing those resources, I could publish my module successfully, by removing the "type":"module" from package.json in my module package — everything is successfully exported!

However, in that case, the resources I need are imported via require — which those packages do not support.

I feel like I have tried every mix-n-match set of options, including targeting other versions of javascript, switching the interop for all variations… and in the end I’m stumped.

Yarn resolution for loader-utils not processing

I have a react project with a dependency on [email protected]. Snyk flags a transitive vulnerability issue with [email protected] so I’ve added a yarn resolution to get the correct version. However, it seems to be getting ignored. How can I force yarn to load the 2.0.4 version of loader-utils?

I’ve tried this resolution format:

"resolutions": {
        "**/browserify-sign": "4.2.3",
        "**/@babel/traverse":"7.25.3",
        "**/decode-uri-component":"0.2.2",
        "**/nth-check":"2.0.1",
        "**/semver":"7.5.2",
        "**/ws":"7.5.10",
        "**/follow-redirects":"1.15.6",
        "**/elliptic": "6.5.7",
        "**/loader-utils":"2.0.4",
        "**/minimist":"1.2.6"
      }

And this resolution format:

"resolutions": {
        "**/browserify-sign": "4.2.3",
        "**/@babel/traverse":"7.25.3",
        "**/decode-uri-component":"0.2.2",
        "**/nth-check":"2.0.1",
        "**/semver":"7.5.2",
        "**/ws":"7.5.10",
        "**/follow-redirects":"1.15.6",
        "**/elliptic": "6.5.7",
        "**/[email protected]":"2.0.4",
        "**/minimist":"1.2.6",
        "**/[email protected]": "1.4.1"
      }

npm ls loader-utils provides this:


└─┬ [email protected] -> .applicationsmy-app
  ├─┬ [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ └── [email protected] deduped
  ├─┬ [email protected]
  │ └── [email protected] deduped
  ├─┬ [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ ├─┬ @pmmmwh/[email protected]
  │ │ └── [email protected]
  │ ├─┬ @svgr/[email protected]
  │ │ └── [email protected] deduped
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ └─┬ [email protected]
  │   ├─┬ [email protected]
  │   │ └── [email protected] deduped
  │   └── [email protected] deduped
  └─┬ [email protected]
    └── [email protected] deduped

The resolutions are all in a parent package.json file.

How do different browsers calculate a CSS translate animation containing images?

I am building a small app with a horizontal infinite scroll animation containing a set of images. In essence, it looks like this:

index.html

<!doctype html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <link rel="stylesheet" href="main.css">
  <title>Example</title>
</head>

<body>
  <div id="container">
    <div id="photos_container">
      <div id="photos_scroller">
        <img src="https://picsum.photos/1000" alt="ID 111111">
        <img src="https://picsum.photos/1000" alt="ID 111112">
        <img src="https://picsum.photos/1000" alt="ID 111113">
        <img src="https://picsum.photos/1000" alt="ID 111114">

        <img src="https://picsum.photos/1000" alt="ID 111111">
        <img src="https://picsum.photos/1000" alt="ID 111112">
        <img src="https://picsum.photos/1000" alt="ID 111113">
        <img src="https://picsum.photos/1000" alt="ID 111114">
      </div>
    </div>
  </div>
</body>

</html>

main.css

*,
*::before,
*::after {
  box-sizing: border-box;
}

* {
  margin: 0;
  padding: 0;
  font: inherit;
}

html,
body {
  height: 100%;
}

body {
  display: grid;
  place-items: center;
  background-color: #1f1f1f;
}

#container {
  width: 95vw;
  padding-inline: 1rem;
}

#photos_container {
  overflow: hidden;
}

#photos_scroller {
  display: flex;
  gap: 1rem;
  width: max-content;
  animation: slide var(--move_duration, 10s) linear infinite;
}

#photos_scroller > img {
  max-height: 50vh;
  transform: translateZ(0); /* to prevent flickering images */
}

@keyframes slide {
  to {
    transform: translate(calc(-50% - 0.5rem));
  }
}

In Chrome on a MacBook, this works fine. However, on Safari, as well as in Safari and Chrome (?!?) on an iPad the images are hardly moving.

Adding a delay to the animation solves the issue, so my hypothesis is that the animation and the corresponding distance are calculated differently in different browsers. Specifically, in the mentioned cases, I suppose the animation is based on the layout at a point in time when the photos have not yet been downloaded.

Also, after tilting the iPad from landscape to portrait and back to landscape orientation the animation runs correctly, so I guess, after the repaint the animation is correctly calculated based on the then available images.

Can someone please explain to me the mechanics that are at play here? What is the recommended way to handle this issue, in particular when images are added or removed dynamically with JavaScript?

How to send emails from Microsoft Workspace Email without disabling security defaults?

I have a simple sendMail function that is supposed to send emails from my Microsoft workspace email through a Node.js backend.

async function sendMail() {
    const transporter = nodemailer.createTransport({
        host: "smtp-mail.outlook.com",
        auth: {
            user: '[email protected]',
            pass: '****',
        },
        tls: {
            ciphers: "TLSv1.2",
        },
        port: 587,
    });
    return await transporter.sendMail({
        from: '[email protected]',
        to: ['[email protected]'],
        subject: 'subject',
        text: 'Welcome aboard...!',
    });
}

However, it is erroring out with the following error: 535 5.7.139 Authentication unsuccessful, user is locked by your organization's security defaults policy. Contact your administrator.

Now, I do not want to disable security defaults. What is the best and safest way to implement this?, and is the Microsoft Graph API a viable option here?, or anything else?

JS Programatically set the value of a file input field

I’m trying to post a large binary object via a <form> element. There is an old version that posts it using a b64 encoded string in a <input type="hidden"> field, but the payload data gets much larger than the web server allows.

As a solution, I’m trying to put the raw data in binary via a <input type="file"> field instead, so I can avoid the b64 inflation. Is this even possible?

The current code looks like this:

function submitToServer(evt) {
  evt?.preventDefault?.()

  const myForm = document.querySelector('form#RequestModal'),
    data = new FormData(myForm)

  let fetchData = [
    fetch('...', {method, body, headers...}),
    ... Other fetches...
  ]

  Promise.all(fetchData)
    .then([clientRes, tokenRes, otherData] => {
      ...
    })
    .catch(E => {
      ...
    })
}

As you might have guessed, the <form> element has a number of hidden input fields that are filled programmatically. I’m trying to replace these fields with file input fields (hidden by css).

Capture HTML element and it’s styles for re-rendering

How would you capture a div element and its computed styles so the individual element can be re-rendered in its own html file, so that it displays the same way it would in the original source?

I have written some code that I can use from within the developer console, but the rendered html file looks nothing like the original source.

function captureDivWithStyles(div) {
    div.id = 'captured-div'

    const styles = window.getComputedStyle(div);
    const cssText = Array.from(styles).reduce((str, property) => {
        return `${str}${property}:${styles.getPropertyValue(property)};`;
    }, '');

    return {
        html: div.outerHTML,
        css: cssText
    };
}

function createNewHtml(capturedDiv) {
    return `
<html>
<head>
    <title>Captured Div</title>
    <style>
        #captured-div {
            ${capturedDiv.css}
        }
    </style>
</head>
<body>
    ${capturedDiv.html}
</body>
</html>
    `;
}

function saveToFile(content, filename) {
    const blob = new Blob([content], { type: 'text/html' });
    const link = document.createElement('a');
    link.href = URL.createObjectURL(blob);
    link.download = filename;
    link.click();
    URL.revokeObjectURL(link.href);
}

function saveDivAsHtml(div) {
    const capturedDiv = captureDivWithStyles(div);
    const newHtml = createNewHtml(capturedDiv);
    saveToFile(newHtml, 'captured-div.html');

}

Usage via: (Where I am using my own query selector, which varies depeneding on what element I want to capture)

myDiv = document.querySelector()
saveDivAsHtml(myDiv);

Error in a script for Adobe Illustrator in charge of numbering

I’m working on a JavaScript script for Adobe Illustrator that will number multiple text fields in an.ai. The script should increment the numbering in multiple text fields at the same time and generate multiple copies of the document.

The script works well for the first text field, increasing the numbering from 0001 to 0005 as expected. However, for the second field, which must be numbered 0006 to 0010, the script stops at 0008 and continues printing the same number until the prints in the first field are completed. Then, move to the next number for the second field, but still with the same problem.

"use strict";

// Request the number of fields to number
var fieldCount = parseInt(prompt("Enter the number of fields to number:", "4"));
var fields = [];

// Request the number of copies to print
var numberOfCopies = parseInt(prompt("How many copies would you like to print? (e.g., for Original and Copy enter 2):", "2"));

// Configure fields with their numbering range
for (var i = 0; i < fieldCount; i++) {
    var fieldName = prompt("Enter the name of field " + (i + 1) + ":", "field_" + (i + 1)); // Field name
    var initialNumber = parseInt(prompt("Enter the starting number for field " + (i + 1) + ":", "0001"));
    var finalNumber = parseInt(prompt("Enter the final number for field " + (i + 1) + ":", "0005"));

    if (isNaN(initialNumber) || isNaN(finalNumber) || finalNumber < initialNumber) {
        alert("Please enter valid numbers.");
        exit();
    }

    fields.push({
        name: fieldName,
        current: initialNumber,
        last: finalNumber
    });
}

// Select Adobe Illustrator file
var baseFile = File.openDialog("Select the base file (.ai)");
if (baseFile === null) {
    alert("No file was selected.");
    exit();
}
var document = app.open(baseFile);

// Check if there are enough textFrames in the document
if (document.textFrames.length < fieldCount) {
    alert("The document does not have enough text fields. Please check the file.");
    exit();
}

// Loop to number and generate the copies
var continuePrinting = true;
while (continuePrinting) {
    continuePrinting = false;
    for (var i = 0; i < fields.length; i++) {
        // Find the corresponding textFrame by name
        var foundTextFrame = null;
        for (var j = 0; j < document.textFrames.length; j++) {
            if (document.textFrames[j].name === fields[i].name) {
                foundTextFrame = document.textFrames[j];
                break;
            }
        }
        if (foundTextFrame === null) {
            alert("No text field found with the name: " + fields[i].name);
            exit(); 
        }
        if (fields[i].current <= fields[i].last) {
            continuePrinting = true; // There are still fields to number
            var currentNumber = ("0000" + fields[i].current).slice(-4);
            foundTextFrame.contents = "No. " + currentNumber;
        }
    }

    // Print all updated fields before continuing numbering
    if (continuePrinting) {
        for (var copy = 0; copy < numberOfCopies; copy++) {
            document.print();
        }

        // Increment the numbers of all fields after printing
        for (var i = 0; i < fields.length; i++) {
            if (fields[i].current <= fields[i].last) {
                fields[i].current++; // Increment the field at the same time
            }
        }
    }
}
alert("Process completed. The receipts were printed in the specified fields.");

The code’s goal is for text fields in Adobe Illustrator to be numbered synchronously and in parallel. For example, if you have two fields with numbering ranges of 0001-0005 and 0006-0010, both should advance at the same time and in a coordinated way, such as 0001 and 0006, 0002 and 0007, etc. After updating the numbering in all the fields, the document should be printed according to the specified number of copies, so that each copy reflects the current numbering of all fields. Once the copies have been generated, the numbers in all fields must be incremented simultaneously to continue the sequence on the next print cycle.

I need to add a modal on one classic ASP page, but inly if a cookie has not been set. i’d like to avoid javascript

I have a classic ASP page that needs a modal to pop up asking people to sign up to our mailing list. If they sign up, a cookie is set. This page checks for that cookie and doesn’t harass the user if they’ve already signed up (a cookie is set).

I saw a similar routine using a lot of JavaScript, but I couldn’t get it to work. (I’m very weak in JavaScript.)

So I tried the following using classic ASP and the bare minimum of Javascript, which doesn’t display the modal at all.

Here’s the basic code I’m working with.

<code>
<body>
<%
' Get the cookie
KeepInTouch = Request.Cookies("KeepInTouch")

if KeepInTouch <> "Y" then
%>
<script>
    document.getElementById('popup').style.display='block';
</script>
<div id="popup" class="w3-modal">
    <div class="w3-modal-content">
        <div class="w3-container">
            <!-- ... rest of popup ... -->
        </div>
    </div>
</div>
</body>`
</code>
    

I can't get the modal to pop up, even without checking for the cookie.

Paysafe Checkout updates for 3DS compliance

I need to update a Paysafe Checkout integration due to a change in requirements from Visa[1]. From the suggested changes I can see that I have to add email/phone/cellphone values to a profile object on the setup options, but the API documentation does not detail any profile properties, so I’m at a loss as to what to do about this. Has anyone else had to deal with this that could offer some insights?

FWIW here’s a code snippet of the JS code from the place I think I need to add the profile data. Note: the code currently works without the profile data, so I can’t tell if just adding the profile data to paymentData has any effect at all :-/

const paymentData = {
    amount: amount,
    currency: 'GBP',
    accounts:{
        CC: accountNumber
    },
    billingAddress:{
        country: "GB",
        zip: postcode,
        state: state,
        city: city,
        street: address,
    },
    environment: 'TEST',
    companyName: 'myco',
    buttonColor: '#121212',
    imageUrl: 'https://myco.com/img/logo.jpg', 
    holderName: '',
    displayPaymentMethods: ['Cards'],
    threeDS: {
        useThreeDSecureVersion2: true
    }
};
const payeeData={
    amount,
    address,
    city,
    state,
    postcode,
    reference
};
const token = GetToken();
paysafe.checkout.setup(
    token,
    paymentData,
    function (instance, error, result) {
        postPaymentProcessing(payeeData, instance, error, result);
    },
    function(stage){
        console.log('closed at payment stage: ' + stage);
    }
);

[1] https://go.pardot.com/webmail/365931/1644206923/c2913a61471857eeea22a44effc737a3815c5cce267486bd0a60c8173b57b599
[2] https://developer.paysafe.com/en/api-docs/client-side-sdks/paysafe-checkout-powered-by-cards-api/setup-function/

Trying to get total change in value and average change in value for data set with errors, clean up meter raw data

The problem with the code currently, it doesnt properly handle all the errors it seems

what i need as output is

  • Total Change in value
  • Average change in value (based on number of vaild changes)
  • Starting value
  • Ending value (start value + total change in value)

for the most part it works except having a huge problem with one set of raw data, it appears the filtering to ignore bad changes in value is messing up with this data, not sure why.

Script – excert

  // Check if cellValue is a valid number
      if (isNaN(numericValue) || numericValue < 0) {
        numericValue = null; // Invalid data, treat as reset
      }

      if (numericValue !== null) {
        if (startValue === null) {
          startValue = numericValue; // This is the first valid reading
          firstValue = numericValue;
          lastValidValue = numericValue;
        } else {
          if (numericValue >= lastValidValue) {
            // Calculate the change (difference between current and last valid value)
            var change = numericValue - lastValidValue;
            totalUsage += change; // Increment total usage based on valid change
            lastValidValue = numericValue;
            if (change > 0) {
              countValidChanges++; // Increment the count only for valid changes > 0
            }
          } else {
            // If the value resets (lower than previous), ignore it and continue
            lastValidValue = numericValue; // Treat this as a reset point, but continue counting up from    here
          }
        }
      }
    }

    // === NEW LOGIC STARTS HERE ===

    // Calculate the initial average usage (without threshold) only if there are valid changes
    var initialAverageUsage = (countValidChanges > 0) ? (totalUsage / countValidChanges) : 0;

    // Reset totalUsage and countValidChanges to recalculate with threshold
    var threshold = initialAverageUsage + 2000;
    totalUsage = 0;
    countValidChanges = 0;
    lastValidValue = firstValue; // Reset to the first valid reading

    // Re-run the loop to apply the threshold
    for (var row = dataStartRow; row < data.length; row++) {
      var rowData = data[row];
      var cellValue = rowData[colIndex];
      var numericValue = parseFloat(cellValue);

      // Check if cellValue is a valid number
      if (isNaN(numericValue) || numericValue < 0) {
        numericValue = null; // Invalid data, treat as reset
      }

      if (numericValue !== null && numericValue >= lastValidValue) {
        // Calculate the change only if it doesn't exceed the threshold
        var change = numericValue - lastValidValue;
        if (change <= threshold) {
          totalUsage += change; // Increment total usage based on valid change
          lastValidValue = numericValue;
          if (change > 0) {
            countValidChanges++; // Increment the count only for valid changes > 0
          }
        }
      }
    }

    // Recalculate average usage with the threshold applied
    var finalAverageUsage = (countValidChanges > 0) ? (totalUsage / countValidChanges) : 0;

    // Ensure the ending value is never lower than the start value
    var endValue = (firstValue !== null) ? firstValue + totalUsage : 0; // If no valid first value, set to 0
    
    // Ensure that startValue and endValue default to 0 if no valid data was found
    reportData.push([point, totalUsage || 0, finalAverageUsage || 0, startValue || 0, endValue || 0]);
 }

here is the raw data that is causing the problem

66,360
66,360
66360
66360
66360
66360
66360
66360
66360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
66,360
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
66360
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
Data Loss
Data Loss
Data Loss
Data Loss
Data Loss
1.41E+06
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
Data Loss
Data Loss
Data Loss
1.41E+06
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
Data Loss
Data Loss
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
1.41E+06
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
Data Loss
Data Loss
Data Loss
Data Loss
Data Loss
Data Loss
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
Data Loss
Data Loss
1.41E+06
1.41E+06
1.41E+06
1.41E+06
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
6.64E+04
Data Loss
Data Loss
1.41E+06
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
Data Loss
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0.00E+00
0
0
0
0`

Ive tried updating it and adding threshold to the data, also it should be ignoring same values and improper values, it works great on most of the data, but this one data set is causing it to give absurdly high numbers.

Using Page Function in Apify Puppeteer for scraping articles (using Pagination)

I am trying to scrape news articles from this website using Apify Puppeteer crawler. I am trying to write a page-function in javascript to achieve the same but unable get it to work.
The webpage incorporates pagination so I am trying to navigate through all the relevant pages to extract all articles with publication date since ‘2024-08-15’.

I am new to javascript, so any help is highly appreciated. The page-function code I was trying to write is given below.

Can someone please help me to fix the page-function in javascript to use it in Apify Puppeteer scraper.

async function pageFunction(context) {
    const { request, $, log, jQuery } = context;
    const results = [];
    const url = context.request['url'];
    const pageopt = 'home';
    const pageoption = context.customData.pageopt;
    
    let i = 1;
    
    //Define startDate
    const startDate = new Date('2024-08-15');
    
    //Get the page object from the context
    const { page } = context;
    
    await page.goto(url, { waitUntil: "domcontentloaded" });
    const litags = null;
    
    if ( pageoption === 'True') {
        try {
            await page.waitForSelector('ul.bwNewsList li');
            liTags = await page.$$('ul.bwNewsList li');
        } catch (error) {
            console.error('An error occurred: ', error);
        }
    }
    else {
        try {
            await page.waitForSelector('ul.bwNewsList li');
            liTags = await page.$$('ul.bwNewsList li');
        } catch(error) {
            console.error('An error occurred: ', error);
        }
    }
    
    const urls = [];
    let date = null;
    
    for (let li of liTags) {
        date = new Date(await li.$eval('time', el => el.getAttribute('datetime')));
        
        const aTag = await li.$('a');
        const href = aTag ? await (await aTag.getPropert('href')).jsonValue() : null;
        
        if (date >= startDate) {
            urls.push(href);
        }
    }
    
    console.log(date)
    console.log(startDate)
    
    if (date >= startDate) {
        const nxthref = await page.$eval('div.pagingNext a', element => element.href);
        console.log(nxthref);
        context.customData.pageopt = 'False';
        await context.enqueRequest({ 'url' : nxthref});
    }
    
    for (let url of urls) {
        await page.goto(url, { waitUntil: " domcontentloaded" });
        
        let title = await page.$eval('title', el => el.textContent.replace(/[rns]+/g, " ") );
        const unique_article = url.concat(title);
        
        // Generate the hash of the title
        const titleHash = await page.evaluate(async (unique_article) => {
            const encoder = new TextEncoder();
            const data = encoder.encode(unique_article);
            const hash = await window.crypto.subtle.digest('SHA-256', data);
            const hashArray = Array.from(new Uint8Array(hash) );
            return hashArray.map(b => b.toString(16).padStart(2, '0')).join('');
        }, unique_article);
        
        const pgdate = new Date(await page.$eval('div.bw-release-timestamp time', el => el.getAttribute('datetime')));
        const divs = await page.$$('p');
        let text = '';
        for (let div of divs) {
            text += await page.evaluate(el => el.textContent, div);
        }
        text = text.replace(/[rns]+/g, " ");
        
        results.push({
            'article_id': titleHash,
            'url': url,
            'title': title,
            'content': text,
            'published_date': pgdate,
            'publisher': 'BW'
        });
    }
    
    return results;
}

Thanks in advance.

Browser’s tab freezes if i make too many request per second

I have a website that needs the user to make many requests in a very short time on the click of a button.

The problem I’m facing is that my tab freezes or become really slow when I send too many requests in a short time (like 300 requests in 5 seconds).

In each request I just use like void request() (don’t wait for the promise).

What do you think? Any advice?

Thank you in advance

P.S. I can’t send them in batches, each must be handled separately for legal reasons.