The Dialog is not showing

I’m on Library Project. The new book button is for adding books, and when i try adding another book, it duplicates. So I added container.textContent = '';
on my display function to remove duplicates. I successfully added one book, but when i tried adding another book, the modal doesn’t show up anymore and I inspected in on edge. This is what it says: Uncaught InvalidStateError: Failed to execute ‘showModal’ on ‘HTMLDialogElement’: The element is not in a Document. at HTMLButtonElement.

(even tho i have dialog.showModal(); and my dialog is present on html)

Here is my html:

`<dialog id = 'dialog'>
                <form>
                    <fieldset>
                        <label for="title">Title</label>
                        <input type="text" id="title" name="title">
                        <label for="author">Author</label>
                        <input type="text" id="author" name="author">
                        <label for="pages">Pages</label>
                        <input type="number" id="pages" name="pages">
                    </fieldset>
                   
        
                    <fieldset>
                        <legend>Have you read it?</legend>
                        <div class = 'yes'>
                            <input type="radio" id="read" name="read" value="read">
                            <label for="read">Yes</label>
                        </div>
                        <div class = 'no'>
                            <input type="radio" id="not_read" name="read" value="not_read">
                            <label for="not_read">No</label>
                        </div>
                       
                    </fieldset>
                    <div class = 'btn_submit'>
                        <button id = 'submit' type="submit">Submit</button>
                    </div>
                   
        
                </form>
            </dialog>`

My Javascript:

`document.addEventListener(‘DOMContentLoaded’, () => {

const container = document.querySelector('.container')
const Btn = document.querySelector('#mybtn');
const dialog = document.querySelector('#dialog');
const submitBtn = document.querySelector('#submit');

const myLibrary = [];

function Book (title,author,pages,read){
    this.title = title;
    this.author = author;
    this.pages = pages;
    this.read = read;

    this.info = function(){
      return  `Title: ${this.title}
               Author: ${this.author}
               Pages: ${this.pages}  
               Read: ${this.read}`;
    }
}

function addBookToLibrary() {
    const title = document.querySelector('#title').value;
    const author = document.querySelector('#author').value;
    const pages = document.querySelector('#pages').value;
    const read = document.querySelector('input[name="read"]:checked').value;
    const newBook = new Book(title,author,pages,read);

    myLibrary.push(newBook)
  }

  /*
let addAnother = true;

while(addAnother){
    addBookToLibrary()
    addAnother = prompt("Would you like to add another book? ") == 'yes';
}*/

function display(){
    container.textContent = ''; 

    myLibrary.forEach(lib => {
        const book = document.createElement('div');
        book.classList.add('book');
        const newP = document.createElement('p');  

        book.appendChild(newP);
        container.appendChild(book);

        newP.textContent = lib.info();
    })

}



Btn.addEventListener('click', () =>{
dialog.showModal();
});

submitBtn.addEventListener('click', (event) =>{
    addBookToLibrary();
    display();
    event.preventDefault();
    dialog.close();

});











});

`

I’d appreciate answers, thank you.

Converting from manifest V2 to V3 for chrome extension error

First off I am not a programmer. I am self taught and don’t do it for a living. So I don’t know a lot about chrome extensions but I still managed to create one that we use at my workplace. This is a chrome extension and the site that it is used for is only validated to run on chrome so using a different browser in not an option.
I converted my extension to manifest V3 about a year ago but there were a couple things that I couldn’t figure out how to make work. The extension only runs on a few work computers so the parts I couldn’t get to work in V3 were kept in a small extension in V2 that runs on the computer in developer mode. The problem is that I believe that soon V2 will stop working so I am trying to find a solution to get it to work in V3.

if (document.querySelector("[id^=GridViewTissueTeamSignatures_][id$=_TextBoxESigPassword]")) {
    // Select all elements with IDs starting with 'GridViewTissueTeamSignatures_' and ending with '_TextBoxESigPassword'
    document.querySelectorAll("[id^=GridViewTissueTeamSignatures_][id$=_TextBoxESigPassword]").forEach(function(element) {
        var num = element.id.split("_");
        var span = document.createElement('span');
        span.innerHTML = ' <button id="simbutton_' + num[1] + '" class="header-button blue" style="display:none;">Save</button>';
        element.insertAdjacentElement('afterend', span);
    });

    // Add click event listeners to checkboxes
    document.querySelectorAll("[id^=GridViewTissueTeamSignatures_][id$=_CheckBoxESigSign]").forEach(function(checkbox) {
        checkbox.addEventListener('click', function(e) {
            var num = this.id.split("_");
            var simbutton = document.getElementById("simbutton_" + num[1]);
            if (this.checked) {
                simbutton.style.display = 'inline';
            } else {
                simbutton.style.display = 'none';
            }
        });
    });

    // Add click event listener for simbuttons
    document.addEventListener('click', function(e) {
        if (e.target.id && e.target.id.startsWith('simbutton')) {
            e.preventDefault();
            //document.querySelector("#GridViewTissueTeamSignatures_ctl02_SignatureBlockTissueTeamMember_ctl00_CheckBoxESigInvalidate").click();
            document.querySelector("#FormLinkPanel_ButtonSave").click();
            return false;
        }
    });
}

when it gets to the line
document.querySelector(“#FormLinkPanel_ButtonSave”).click();
I get the following in my console:
Refused to run the JavaScript URL because it violates the following Content Security Policy directive: “script-src ‘self’ ‘wasm-unsafe-eval’ ‘inline-speculation-rules’ http://localhost:* http://127.0.0.1:*”. Either the ‘unsafe-inline’ keyword, a hash (‘sha256-…’), or a nonce (‘nonce-…’) is required to enable inline execution. Note that hashes do not apply to event handlers, style attributes and javascript: navigations unless the ‘unsafe-hashes’ keyword is present

Here is the manifest. I wasn’t 100% sure of what I was doing so I’m not sure if every thing in it is necessary. I am changing the site name in the upload because it is a private site and not allowed to give out the address.

{ 
   "name":"example.com Extension",
   "description":"Helps with the **** part of example.com.",
   "version":"3.0.8.6",
   "manifest_version":3,
   "icons":{ 
      "48":"icon48.png",
      "128":"icon128.png"
   },
   "permissions":[ 
      "https://example.com/*"
   ],
   "content_scripts":[ 
      { 
         "run_at":"document_end",
         "matches":[ 
            "https://example.com/*"
         ],
         "css":[ 
            "css/style.css",
        "css/jquery.multiselect.css"
         ],
         "js":[ 
        "js/dependencies/download.js",  
            "js/dependencies/pdf-lib.min.js",
            "js/dependencies/jquery.min.js",
            "js/dependencies/jquery-ui.min.js",
            "js/dependencies/simulate.js",
            "js/dependencies/JsBarcode.code128.min.js",
            "js/dependencies/DYMO.Label.Framework.2.0.2.js",
            "js/functions.js",
            "js/content.js",
            "js/main.js",
            "js/QAfunction.js",
        "js/addons/savebutton.js",
        "js/addons/hospitalNames.js",
            "js/addons/blood2.js",
        "js/addons/pdf-lib/trif.js",
            "js/addons/pdf-lib/versiti.js",
            "js/addons/pdf-lib/lnh.js",
        "js/addons/pdf-lib/VRL.js",
        "js/addons/pdf-lib/BBAcult.js",
            "js/addons/pdf-lib/Solvita.js",
            "js/addons/pdf-lib/BodyBag.js",
            "js/addons/pdf-lib/coroner.js",
        "js/addons/pdf-lib/chain.js",
            "js/addons/pdf-lib/property.js",
            "js/addons/pdf-lib/shipping.js",
        "js/addons/pdf-lib/VF.js",
        "js/addons/pdf-lib/Ossium.js",
        "js/addons/pdf-lib/OssiumCult.js"
         ]
      }
   ],
   "web_accessible_resources": [{
 "resources": ["images/*.gif","images/*.png","labels/*.label","pdf/*.pdf","html/*.html"],
 "matches": [ 
      "https://example.com/*"
      ],
 "extension_ids": []
}]
}

The line above it is a checkbox on the page and if I run that line instead it will click the checkbox. The code executes perfectly in V2.

How can I pass a variable to Handlebars helpers without exposing it in the template?

I am working on a chatbot project using Handlebars in NodeJS with TypeScript. I need some guidance on how to pass an object (e.g. TransactionManager) to Handlebars helper functions without exposing it to the template. I don’t want to expose the functions nor the variables in the template.

Requirements

  • For each session, I have a TransactionManager object that is unique
    to that session.
  • This TransactionManager object needs to be accessible inside
    Handlebars helpers when rendering the template, as it contains
    session specific data and methods (e.g., runSomeFunction() or m_DatabaseConnection).
  • I don’t want the TransactionManager object to be accessible directly
    in the template itself. So using
    {{TransactionManager.runSomeFunction}} or {{TransactionManager.m_DatabaseConnection}} or {{@TransactionManager}} in
    the template should not work.
  • The TransactionManager object is created dynamically at runtime when
    rendering a template, so it isn’t available when I define the
    Handlebars helper functions.

What I’ve tried:

I’ve considered adding the TransactionManager to the context or options.data, but this makes it accessible via {{@transactionManager}} in the template, which I want to avoid.
Using closures in helper functions doesn’t seem feasible because I need to create the TransactionManager on a per session basis, meaning it must be added dynamically when rendering the template and I don’t want to create a Handlebars context for each session.

Ideal behavior:

I want to pass the TransactionManager object to the helper functions when rendering the template without exposing it to the template itself.
Helper functions should be able to access this object (e.g., this.TransactionManager.runSomeFunction()), but the object with functions and variables should remain invisible to the template context.

How can I extract text with coordinates from a PDF using JavaScript?

I’m working on a project where I need to extract text along with its coordinates from a PDF document using JavaScript. I’m familiar with PDF.js, but I’m unsure how to use it to get the coordinates of the text. I’m also open to using other JavaScript libraries if they provide a more straightforward way to achieve this.

Here are the specifics of what I’m trying to do:

Extract text from a PDF page.
Obtain the coordinates (x, y) for each piece of text.
On click of paragraph, show action menu for that para and add a background color to the paragraph.

It would be great if i can get an examples of how to extract text with coordinates using PDF.js or any other JavaScript library.

ASP.Net MVC – TypeError $(…)camera is not a function

I am trying to get an image slider/carousel to work in an ASP.Net MVC view. I have successfully got it working in a normal HTML environment. It is called ‘camera wrap’.

I am using version 5.2.9.0 of MVC in Visual Studio 2022. I’ve been working with VB.Net for a long time but I am relatively new to the use of HTML, CSS and Javascript. I understand their interaction and what they do. I particularly want to use MVC (VB.Net).

I literally copied a working HTML doc into my MVC view. I have added the relevant css, javascript and images to the solution. The HTML includes the relative references to the scripts and css file. I have also tried loading them all using the ‘bundles’ option. I have also changed the $ sign for ‘jQuery’ in the function call. The script for this is…

    <script>
        jQuery(function () {

            jQuery('#camera_wrap_1').camera({
                thumbnails: true
            });

            jQuery('#camera_wrap_2').camera({
                height: '400px',
                loader: 'bar',
                pagination: false,
                thumbnails: true
            });
        });
    </script>

I get the following error…

TyepError

with this stack trace error.

stacktrace

I’m hoping this is something that is due to my lack of MVC knowledge, I have tried unsuccessfully to search for this issue in various locations.

This is the HTML I copied…

<!DOCTYPE html>
<!-- Camera is a Pixedelic free jQuery slideshow | Manuel Masia (designer and developer) --> 
<html> 
<head> 
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" > 
    <title>Camera | a free jQuery slideshow by Pixedelic</title> 
    <meta name="description" content="Camera a free jQuery slideshow with many effects, transitions, adaptive layout, easy to customize, using canvas and mobile ready"> 
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <!--///////////////////////////////////////////////////////////////////////////////////////////////////
    //
    //      Styles
    //
    ///////////////////////////////////////////////////////////////////////////////////////////////////--> 
    <link rel='stylesheet' id='camera-css'  href='../css/camera.css' type='text/css' media='all'> 
    <style>
        body {
            margin: 0;
            padding: 0;
        }
        a {
            color: #09f;
        }
        a:hover {
            text-decoration: none;
        }
        #back_to_camera {
            clear: both;
            display: block;
            height: 80px;
            line-height: 40px;
            padding: 20px;
        }
        .fluid_container {
            margin: 0 auto;
            max-width: 1000px;
            width: 90%;
        }
    </style>

    <!--///////////////////////////////////////////////////////////////////////////////////////////////////
    //
    //      Scripts
    //
    ///////////////////////////////////////////////////////////////////////////////////////////////////--> 
    
    <script type='text/javascript' src='../scripts/jquery.min.js'></script>
    <script type='text/javascript' src='../scripts/jquery.mobile.customized.min.js'></script>
    <script type='text/javascript' src='../scripts/jquery.easing.1.3.js'></script> 
    <script type='text/javascript' src='../scripts/camera.min.js'></script> 
    
    <script>
        $(function(){
            
            $('#camera_wrap_1').camera({
                thumbnails: true
            });

            $('#camera_wrap_2').camera({
                height: '400px',
                loader: 'bar',
                pagination: false,
                thumbnails: true
            });
        });
    </script>
 
</head>
<body>
    <div id="back_to_camera">
        <a href="http://www.pixedelic.com/plugins/camera/">&larr; Back to the Camera project</a>
    </div><!-- #back_to_camera -->
    
    <div class="fluid_container">
        <p>Pagination circles with the height relative to the width</p>
        <div class="camera_wrap camera_azure_skin" id="camera_wrap_1">
            <div data-thumb="../images/slides/thumbs/bridge.jpg" data-src="../images/slides/bridge.jpg">
                <div class="camera_caption fadeFromBottom">
                    Camera is a responsive/adaptive slideshow. <em>Try to resize the browser window</em>
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/leaf.jpg" data-src="../images/slides/leaf.jpg">
                <div class="camera_caption fadeFromBottom">
                    It uses a light version of jQuery mobile, <em>navigate the slides by swiping with your fingers</em>
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/road.jpg" data-src="../images/slides/road.jpg">
                <div class="camera_caption fadeFromBottom">
                    <em>It's completely free</em> (even if a donation is appreciated)
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/sea.jpg" data-src="../images/slides/sea.jpg">
                <div class="camera_caption fadeFromBottom">
                    Camera slideshow provides many options <em>to customize your project</em> as more as possible
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/shelter.jpg" data-src="../images/slides/shelter.jpg">
                <div class="camera_caption fadeFromBottom">
                    It supports captions, HTML elements and videos and <em>it's validated in HTML5</em> (<a href="http://validator.w3.org/check?uri=http%3A%2F%2Fwww.pixedelic.com%2Fplugins%2Fcamera%2F&amp;charset=%28detect+automatically%29&amp;doctype=Inline&amp;group=0&amp;user-agent=W3C_Validator%2F1.2" target="_blank">have a look</a>)
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/tree.jpg" data-src="../images/slides/tree.jpg">
                <div class="camera_caption fadeFromBottom">
                    Different color skins and layouts available, <em>fullscreen ready too</em>
                </div>
            </div>
        </div><!-- #camera_wrap_1 -->

        <p>Thumbnails with fixed height</p>
        <div class="camera_wrap camera_magenta_skin" id="camera_wrap_2">
            <div data-thumb="../images/slides/thumbs/bridge.jpg" data-src="../images/slides/bridge.jpg">
                <div class="camera_caption fadeFromBottom">
                    Camera is a responsive/adaptive slideshow. <em>Try to resize the browser window</em>
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/leaf.jpg" data-src="../images/slides/leaf.jpg">
                <div class="camera_caption fadeFromBottom">
                    It uses a light version of jQuery mobile, <em>navigate the slides by swiping with your fingers</em>
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/road.jpg" data-src="../images/slides/road.jpg">
                <div class="camera_caption fadeFromBottom">
                    <em>It's completely free</em> (even if a donation is appreciated)
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/sea.jpg" data-src="../images/slides/sea.jpg">
                <div class="camera_caption fadeFromBottom">
                    Camera slideshow provides many options <em>to customize your project</em> as more as possible
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/shelter.jpg" data-src="../images/slides/shelter.jpg">
                <div class="camera_caption fadeFromBottom">
                    It supports captions, HTML elements and videos and <em>it's validated in HTML5</em> (<a href="http://validator.w3.org/check?uri=http%3A%2F%2Fwww.pixedelic.com%2Fplugins%2Fcamera%2F&amp;charset=%28detect+automatically%29&amp;doctype=Inline&amp;group=0&amp;user-agent=W3C_Validator%2F1.2" target="_blank">have a look</a>)
                </div>
            </div>
            <div data-thumb="../images/slides/thumbs/tree.jpg" data-src="../images/slides/tree.jpg">
                <div class="camera_caption fadeFromBottom">
                    Different color skins and layouts available, <em>fullscreen ready too</em>
                </div>
            </div>
        </div><!-- #camera_wrap_2 -->
    </div><!-- .fluid_container -->
    
    <div style="clear:both; display:block; height:100px"></div>
</body> 
</html>

One point, the ‘camera’ function is in another script – camera.min.js not jquery.min.js as noted in the stack trace???

I have tried loading the sripts and css file using the ‘bundles’ option. I have ‘dragged’ the scripts + css references into the view from the solution explorer.

I have changed the $ sign for ‘jQuery’. One point, the ‘camera’ function is in another script – camera.min.js not jquery.min.js as noted in the stack trace???

Operationerror when decrypting key using subtle crypto api

I am encoding and decoding passwords using a master password, salt and iv. The weird thing is whenever the error happens (during the decryption process, I never get an error during encryption) and I restart the vite server, the error is temporarily fixed. I have checked the iv and salt and they stay the same during the encryption and decryption process. The master password is also the same since the user has to enter it every single time.

This is my code:

const isBrowser =
  typeof window !== "undefined" && typeof window.localStorage !== "undefined";

const getNewKey = async (password: string, salt: string) => {
  if (isBrowser) {
    const enc = new TextEncoder();
    const baseKey = await crypto.subtle.importKey(
      "raw",
      enc.encode(password),
      "PBKDF2",
      false,
      ["deriveBits", "deriveKey"]
    );
    const params = {
      name: "PBKDF2",
      salt: base64ToArrayBuffer(salt),
      iterations: 100000,
      hash: "SHA-256",
    };
    return await crypto.subtle.deriveKey(
      params,
      baseKey,
      { name: "AES-GCM", length: 256 },
      false,
      ["encrypt", "decrypt"]
    );
  }
};

export const encryptData = async (
  password: string,
  salt: string,
  data: string
) => {
  const key = await getNewKey(password, salt);
  if (!key) return "ERROR";
  console.log("Generated key: ", key)
  const iv = crypto.getRandomValues(new Uint8Array(12));
  const encryptedData = arrayBufferToBase64(
    await crypto.subtle.encrypt(
      {
        name: "AES-GCM",
        iv,
      },
      key,
      new TextEncoder().encode(data)
    )
  );

  return {
    iv: uint8ArrayToBase64(iv),
    encryptedData,
  };
};

export const decryptData = async (
  password: string,
  salt: string,
  data: string,
  iv: string
) => {
  console.log("IV: ", iv, " Salt: ", salt);
  const key = await getNewKey(password, salt);
  if (!key) return "ERROR";
  console.log("Generated key: ", key)
  const dataBuffer = base64ToArrayBuffer(data);
  const ivArray = base64ToUint8Array(iv);
  try {
    const decryptedData = await crypto.subtle.decrypt(
      {
        name: "AES-GCM",
        iv: ivArray,
      },
      key,
      dataBuffer
    );
    return new TextDecoder().decode(decryptedData);
  } catch (error) {
    console.error(error);
  }
};

const arrayBufferToBase64 = (buffer: ArrayBuffer) => {
  let binary = "";
  const bytes = new Uint8Array(buffer);
  const length = bytes.byteLength;
  for (let i = 0; i < length; i++) {
    binary += String.fromCharCode(bytes[i]);
  }
  return btoa(binary);
};

const base64ToArrayBuffer = (base64: string) => {
  const binaryString = atob(base64);
  const len = binaryString.length;
  const bytes = new Uint8Array(len);
  for (let i = 0; i < len; i++) {
    bytes[i] = binaryString.charCodeAt(i);
  }
  return bytes.buffer;
};

function uint8ArrayToBase64(uint8Array: Uint8Array) {
  let binaryString = "";
  const len = uint8Array.length;
  for (let i = 0; i < len; i++) {
    binaryString += String.fromCharCode(uint8Array[i]);
  }
  return btoa(binaryString);
}

function base64ToUint8Array(base64: string) {
  const binaryString = atob(base64);
  const len = binaryString.length;
  const uint8Array = new Uint8Array(len);
  for (let i = 0; i < len; i++) {
    uint8Array[i] = binaryString.charCodeAt(i);
  }
  return uint8Array;
}

Is it possible to find and call a function by its decorator?

What I want to achieve:

I want to develop a core browser library for a given product. There are some parts where function implementations will differ based on a customization ( let’s call them plugin functions ). I want to avoid having to manage these plugin functions inside the library code, I want to manage them in different code bases and only install them during CI/CD so I can include them in the bundle.

The app configuration tells the system which plugin function to choose, let’s assume customer 1 is going for foo and customer 2 for bar. It would be nice if the core library gets a parameter to identify the correct function to be called.


What I’ve done so far:

During runtime I had no idea how to call an implementation without an “explicit” if-statement. I think the only way that might work is to use decorators. The configuration will contain a unique name so I might search for a given function.

I started with this decorator every plugin should use

function pluginFunc(name: string) {
  console.log(`plugin func '${name}'`);

  return function (
    target: any,
    propertyKey: string,
    descriptor: PropertyDescriptor
  ) {
    console.log({ name, target, propertyKey, descriptor });
  };
}

export { pluginFunc };

And a custom plugin

import { pluginFunc } from "./pluginFunc";

@pluginFunc("foo")
function pluginOne() {
  console.log("hello from plugin 1");
}

export { pluginOne }

Now I want to test my approach in a simple index.ts file

/* async */ function callPlugin(name: string) {
  // find function by decorator and name and call it
}

callPlugin('foo');

Is it possible to find a function ( without importing it during build time! ) by a decorator pluginFunc and a given name?

If that’s not possible what about the simpler approach to simply call a function by its name ( without importing it ) ?

Simplest Chai Spies syntax

I’m having problems with Chai Spies. I have a hard time catching how the syntax works.

Here, I am trying the simple example. The myMap function which mimics the array.map method:

function myMap(inputArray, callback) {
  let newArr = [];

  for (i = 0; i < inputArray.length; i++) {
    let nEl = callback(inputArray[i]);
    newArr.push(nEl);
  };
  
  return newArr;
}

module.exports = myMap;

And the test:

const chai = require('chai');
const expect = chai.expect;
const spies = require('chai-spies');
chai.use(spies);

const myMap = require('../problems/my-map');

describe('function myMap', function () {
    const arr = [1, 2, 3];
    const cb1 = el => el*2;
    it('should return new array passed through callback', function () {
        
        expect(myMap(arr, cb1)).to.deep.equal([2, 4, 6]);
        
    });
    it('should preserve original array', function () {
        expect(arr).to.deep.equal([1,2,3]);
    });
    it('see if .map used on array', function () {
        expect(chai.spy.on(arr, 'map')).to.not.have.been.called();
    });
    it('callback called array.length times', function () {
        
        let spy1 = chai.spy(cb1);

        expect(spy1).to.have.been.called.exactly(arr.length);
    })
    
});

The test number 3 that uses spy works as intended, failing if I introduce .map method to the function.
But the test number 4 gives the same result of having 0 callbacks while expecting 3. Surely I messed something with syntax, but I couldn’t find any simple examples on the internet.

I’m using downgraded Chai version of 4.3.0, since there’s some conflict with ES6 which I don’t know how to resolve yet.

Scraping 485927rows of data from a Website problem [closed]

I am trying to scrape all data from big table: https://registru.dispozitive.amdm.gov.md/ website, but I am having some issues.

  1. URL of website does not include the nr of page you are on.
  2. If u resfresh it moves to 1st page.
  3. You cannot go to certain page automatically.
  4. You should ignore the small table with aditional data from left.
  5. The navigation html on big table is the same as navigation html on small table

I tryied couple methods of scraping:

  • Extensions from google chrome (3,4 idk) but they charge you money
  • Python scraping script, but at some point for i dont know why it’s stoping and remains stuck.

So i would love to get help on scraping the data from all pages, and save it into Excell file.

Here is the Script I was working on:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import pandas as pd
import time
import re

def get_total_pages(driver):
    soup = BeautifulSoup(driver.page_source, "html.parser")
    total_pages_element = soup.find("b", class_="dxp-lead dxp-summary")
    if total_pages_element:
        text = total_pages_element.get_text(strip=True)
        match = re.search(r'Страница (d+) из (d+)', text)
        if match:
            total_pages = int(match.group(2))
            return total_pages
    return None


def is_big_table_navigation(soup):
    page_indicator = soup.find("b", class_="dxp-lead dxp-summary")
    if page_indicator:
        text = page_indicator.get_text(strip=True)
        match = re.search(r'Страница (d+) из (d+)', text)
        if match:
            return True
    return False

def navigate_to_starting_page(driver, start_page, total_pages):
    current_page = 1
    
    from_start = start_page - 1
    from_end = total_pages - start_page

    if from_start <= from_end:
        print(f"Navigating to page {start_page} from the start...")
        while current_page < start_page:
            try:
                next_button = WebDriverWait(driver, 20).until(
                    EC.element_to_be_clickable((By.XPATH, "//a[contains(@onclick,'PBN')]"))
                )
                next_button.click()
                current_page += 1
                print(f"Moved to page {current_page}")
                WebDriverWait(driver, 20).until(
                    EC.text_to_be_present_in_element((By.XPATH, "//b[@class='dxp-lead dxp-summary']"), f"Страница {current_page}")
                )
                time.sleep(2)
            except Exception as e:
                print(f"Error navigating to page {current_page}: {e}")
                driver.save_screenshot(f"error_page_{current_page}.png")
                time.sleep(5)
                continue
    else:
        print(f"Navigating to page {start_page} from the end...")
        try:
            last_page_button = WebDriverWait(driver, 20).until(
                EC.element_to_be_clickable((By.XPATH, f"//a[@onclick and text()='{total_pages}']"))
            )
            last_page_button.click()
            current_page = total_pages
            print(f"Moved to page {current_page}")
            WebDriverWait(driver, 20).until(
                EC.text_to_be_present_in_element((By.XPATH, "//b[@class='dxp-lead dxp-summary']"), f"Страница {current_page}")
            )
            time.sleep(2)
        except Exception as e:
            print(f"Error navigating to last page: {e}")
            driver.save_screenshot("error_last_page.png")
            return
        
        while current_page > start_page:
            try:
                prev_button = WebDriverWait(driver, 20).until(
                    EC.element_to_be_clickable((By.XPATH, "//a[contains(@onclick,'PBP')]"))
                )
                prev_button.click()
                current_page -= 1
                print(f"Moved to page {current_page}")
                WebDriverWait(driver, 20).until(
                    EC.text_to_be_present_in_element((By.XPATH, "//b[@class='dxp-lead dxp-summary']"), f"Страница {current_page}")
                )
                time.sleep(2)
            except Exception as e:
                print(f"Error navigating to page {current_page}: {e}")
                driver.save_screenshot(f"error_page_{current_page}.png")
                time.sleep(5)
                continue

    print(f"Arrived at the starting page {start_page}.")

def is_valid_row(row):
    first_field = row.find("td").get_text(strip=True)
    return bool(re.match(r"^DMd+$", first_field))

def is_small_table_with_buttons(table):
    rows = table.find_all("tr")
    if len(rows) < 5:
        if table.find("div", class_="dxpLite_SoftOrange") or table.find("div", class_="dxgvPagerBottomPanel_SoftOrange"):
            return True
    return False

def scrape_table_data(start_page, end_page):
    driver = None
    all_data = []
    max_columns = 0

    try:
        driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
        driver.maximize_window()
        driver.get("https://registru.dispozitive.amdm.gov.md/")

        total_pages = get_total_pages(driver)
        if total_pages is None:
            print("Unable to determine total pages. Exiting.")
            return

        navigate_to_starting_page(driver, start_page, total_pages)

        current_page = start_page

        while current_page <= end_page:
            print(f"Scraping page {current_page}...")
            time.sleep(1)

            soup = BeautifulSoup(driver.page_source, "html.parser")
            
            if not is_big_table_navigation(soup):
                print("Ignoring small table navigation.")
                retry_count = 0
                max_retries = 5
                page_changed = False

                while retry_count < max_retries and not page_changed:
                    try:
                        next_button = WebDriverWait(driver, 20).until(
                            EC.element_to_be_clickable((By.XPATH, "//a[contains(@onclick,'PBN')]"))
                        )
                        next_button.click()
                        current_page += 1
                        print(f"Moved to page {current_page}")
                        WebDriverWait(driver, 20).until(
                            EC.text_to_be_present_in_element((By.XPATH, "//b[@class='dxp-lead dxp-summary']"), f"Страница {current_page}")
                        )
                        page_changed = True
                    except Exception as e:
                        retry_count += 1
                        print(f"Error moving to next page (attempt {retry_count}/{max_retries}): {e}")
                        driver.save_screenshot(f"error_next_page_{current_page}.png")
                        time.sleep(5)
                        continue

                if not page_changed:
                    print(f"Failed to move to next page after {max_retries} attempts. Exiting loop.")
                    break
                
                continue

            tables = soup.find_all("table", class_="dxgvTable_SoftOrange")

            for table in tables:
                rows = table.find_all("tr", id=lambda x: x and x.startswith("ctl00_ctl00_ASPxSplitter1_Content_ContentSplitter_MainContent_ASPxGridView4_DXDataRow"))
                if len(rows) < 5 or not any(is_valid_row(row) for row in rows):
                    print("Skipping a small table.")
                    continue

                for row in rows:
                    if is_valid_row(row):
                        columns = row.find_all("td")
                        data = [column.get_text(strip=True) for column in columns]
                        all_data.append(data)
                        if len(columns) > max_columns:
                            max_columns = len(columns)

            if current_page == end_page:
                print("Reached the end page.")
                break

            retry_count = 0
            max_retries = 5
            page_changed = False

            while retry_count < max_retries and not page_changed:
                try:
                    next_button = WebDriverWait(driver, 20).until(
                        EC.element_to_be_clickable((By.XPATH, "//a[contains(@onclick,'PBN')]"))
                    )
                    next_button.click()
                    current_page += 1
                    print(f"Moved to page {current_page}")
                    WebDriverWait(driver, 20).until(
                        EC.text_to_be_present_in_element((By.XPATH, "//b[@class='dxp-lead dxp-summary']"), f"Страница {current_page}")
                    )
                    page_changed = True
                except Exception as e:
                    retry_count += 1
                    print(f"Error moving to next page (attempt {retry_count}/{max_retries}): {e}")
                    driver.save_screenshot(f"error_next_page_{current_page}.png")
                    time.sleep(5)
                    continue

            if not page_changed:
                print(f"Failed to move to next page after {max_retries} attempts. Exiting loop.")
                break

    except Exception as e:
        print(f"An error occurred: {e}")
    finally:
        if driver:
            driver.quit()

        if all_data:
            for i in range(len(all_data)):
                if len(all_data[i]) < max_columns:
                    all_data[i].extend([None] * (max_columns - len(all_data[i])))

            columns = [f"Column{i+1}" for i in range(max_columns)]

            df = pd.DataFrame(all_data, columns=columns)

            file_name = f"registru.dispozitive.page{start_page}-{end_page}.xlsx"
            df.to_excel(file_name, index=False)
            print(f"Data saved to {file_name}")
        else:
            print("No data scraped to save.")

def menu():
    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
    driver.maximize_window()
    driver.get("https://registru.dispozitive.amdm.gov.md/")
    total_pages = get_total_pages(driver)
    driver.quit()
    
    if total_pages is None:
        print("Unable to determine the total number of pages. Exiting.")
        return

    print(f"Total pages: {total_pages}")

    start_page = 1
    end_page = total_pages

    while True:
        print("n1. Scrap all pages")
        print("2. Set starting page for Scrap")
        print("3. Set ending page for Scrap")
        print("4. Start Scraping")
        print("5. Quit")
        choice = input("Enter your choice (1-5): ")

        if choice == "1":
            start_page = 1
            end_page = total_pages
            print(f"Set to scrape all pages from {start_page} to {end_page}.")
        elif choice == "2":
            start_page = int(input(f"Enter the starting page (1-{total_pages}): "))
            if start_page < 1 or start_page > total_pages:
                print(f"Invalid starting page. Please enter a number between 1 and {total_pages}.")
            else:
                print(f"Starting page set to {start_page}.")
        elif choice == "3":
            end_page = int(input(f"Enter the ending page ({start_page}-{total_pages}): "))
            if end_page < start_page or end_page > total_pages:
                print(f"Invalid ending page. Please enter a number between {start_page} and {total_pages}.")
            else:
                print(f"Ending page set to {end_page}.")
        elif choice == "4":
            print(f"Scraping from page {start_page} to {end_page}.")
            scrape_table_data(start_page, end_page)
        elif choice == "5":
            print("Quitting the program.")
            break
        else:
            print("Invalid choice. Please try again.")

menu()


if __name__ == "__main__":
    menu()

How do I extend an existing Grafana Plugin? (the node graph plugin)

I want to extend the functionality of the Node Graph plugin in Grafana. Using the default plugin as a base to start from.
My current approach is to download the folder of the Node Graph plugin and start from there, but it is leading to resolving a string of dependencies.
Is there a simpler way of doing it?

I have tried downloading the Node Graph folder from https://github.com/grafana/grafana/tree/main/public/app/plugins/panel/nodeGraph and replacing it for the src folder in the new base plugin from npx create-grafan-plugin.
But the web of dependencies is very hard to resolve here.

JavaScript: Firebase Analytics don’t log my events in console

I’m newbie to Firebase Analytics Web implementation.
But here’s my code:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta
      http-equiv="Cache-control"
      content="no-cache, no-store, must-revalidate"
    />
    <meta http-equiv="Pragma" content="no-cache" />
    <meta http-equiv="Expires" content="0" />
    <title>Firebase test</title>
    <base href="./" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=0, shrink-to-fit=no"
    />
  </head>

  <body>
    Hello :)
    <button onclick="logMyEvent()">Log an event</button>
    
    <script type="module">
      import { initializeApp } from "https://www.gstatic.com/firebasejs/10.13.1/firebase-app.js";
      import { getAnalytics, logEvent } from "https://www.gstatic.com/firebasejs/10.13.1/firebase-analytics.js";
    
      const firebaseConfig = {
        apiKey: "xxx",
        authDomain: "xxx.firebaseapp.com",
        projectId: "xxx",
        storageBucket: "xxx.appspot.com",
        messagingSenderId: "000",
        appId: "xxx000",
        measurementId: "G-xxx"
      };
    
      const app = initializeApp(firebaseConfig);
      const analytics = getAnalytics(app);

      logEvent(analytics, 'test_init', { 'data': 'init' });

      window.logMyEvent = function() {
        console.log('Button was clicked!', analytics); // `analytics` is already defined.
        logEvent(analytics, 'test_button_click', { 'data': 'click' });
      };
    </script>
  </body>
</html>

As you can see this is just a basic Firebase Analytics implementation! But I don’t see my events being logged in the ‘DebugView‘ (or ‘Realtime Analytics’ page) of my Firebase project… Am I missing something?

In the ‘DebugView’, I also see this message: ‘Debug Device: 0, No devices available‘, although I have already installed ‘Google Analytics Debugger‘ Chrome extension and turned it on.

Any help is appreciated. Thanks in advance.

Java Script: window.getSelection isn’t correct on double click

I want a dictionary to open if the user clicks on any word on a webpage. It works so far:

let str = window.getSelection().toString();
window.open("https://en.thefreedictionary.com/" + str, "_blank").focus();

But now, I want that the user can select multiple words and when he double-clicks on this selection, the dictionary or a translation tool should automatically open.

On a webpage, I have the sentence “I support you as long as you need it.” If I know select “as long as” and double-click on it (say above the word long), then window.getSelection.toString() just returns “long” and not “as long as”. So why is the selection erased by the double-click. How can I fix this?

How can you automate security audits of javascript packages when using Rails importmap?

When using a package manager like npm you have the ability to run npm audit on your package.json file to check for any known vulnerabilities. You can then add this check to your CI Pipeline to continually check for issues.

However, if you use Rails 7 importmaps, you’re often referencing CDNs or packages that are downloaded to the vendor folder so what strategies are people using to automatically check JavaScript packages for known vulnerabilities in this situation?

How to run roboflow models offline in a React app?

The title. I ran Roboflow Inference Server locally on my Mac, but I’m unsure how to use the inferencejs package to do inferencing on the local inference server, more specifically in Vite.

I’ve used the deprecated (I think) Roboflowjs lib with the window.roboflow.auth etc but I’m unsure how to make them requests to the local inference server.