Why is the done button not working properly?

Apparently I’m trying to create a todo list where I can ofcourse add and remove the tasks. Adding tasks works fine; however clicking on the Done Button works but doesn’t do what I want it to do. Basically I have a Logical Error, but I don’t know what to do to fix it.

The Code 

<!DOCTYPE html>
<html lang="en">

<head>
  <title>Document</title>
</head>

<body>
  <h1>To-Do List</h1>
  <form id="todoForm">
    <input id="todoInput" />
    <button type="button" onclick="todoList()">New</button>
    <button type="button" onclick="">Retrieve</button>
  </form>
  <ol id="todoList"></ol>
  <script>
    var todos = []; //Problem is from here
    var removed = [];

    function todoList() {
      var item = document.getElementById("todoInput").value;
      todos.push(item);

      var text = document.createTextNode(item);
      var newItem = document.createElement("li");

      newItem.innerHTML = item + ' <button id="Done">Done</button>';
      document.getElementById("todoList").appendChild(newItem);

      const donebtn = document.getElementById("Done");
      donebtn.addEventListener("click", function() {
        removetodo(newItem, item)
      });
    }


    function removetodo(item, tasktext) {
      const tasklist = document.getElementById("todoList");
      tasklist.removeChild(item);
      removed.push(tasktext);
    }
  </script>
</body>

</html>

Thing is, I tried finding solutions to it on Google and other places; yet, I still didnt know how to fix it. I dont want to just change the whole code so it could work. I specifically wanted it in the way I wrote it in