Using Google Identity Services to access the Photos Library API

I have been at this for a while, and even ChatGPT was lost 😛 … I am trying to write a simple “client web app” that will simply list all photos in an account. Most samples and code snippets use the API method and authenticate using the now (as of the end of March 2023) deprecated authentication. So using the new GIS method, I am able to authenticate and get a token but this token is not usable with the REST APIs.

async function handleCredentialResponse(response) {
  const credential = response.credential;
  loadPhotos(credential)
}

I have tried multiple ways of trying to access the albums using the provided token but Google is just not accepting it. Can anyone help on what is the correct use of the idToken received? Do I use the REST method ? is there a different library that I need to use?