how to remove a classList when I select another element?

When I click on an ‘imposter’ I want it to become grey with some text as you can see in my CSS with the selector .imposter.voted. But when I click another imposter I want that one to appear grey and the previous one to appear like usual again.

When I try this.classList.remove("voted") it does not work and it just makes all the imposters I select grey. The class does not get removed. I don’t see where I went wrong.

imposters = document.querySelectorAll(".imposter");

for (let i = 0; i < imposters.length; i++) {
  imposters[i].addEventListener("click", function() {
    this.classList.add("voted");
  });
}

.imposter.voted {
  background-color: gray;
  position: relative;
  opacity: 0.7;
}

.imposter.voted::after {
  content: "I voted";
  position: absolute;
  top: 5px;
  right: 5px;
  display: inline-block;
  font-family: "Comic Sans Ms";
  color: red;
}

.imposters__voting {
  text-align: right;
}

<main class="dashboard">
  <h1>Who Is The Imposter?</h1>
  <div class="imposters">
    <div class="imposter">Baba</div>
    <div class="imposter">Baraki</div>
    <div class="imposter">Garfield</div>
    <div class="imposter">Erik</div>
    <div class="imposter">GreenBlood4</div>
    <div class="imposter">Easter</div>
  </div>
</main>