Sharp disappearing for transient div

I have created a box to toggle between appearing and disappearing via a toggle button.

The problems are:

  • The first time, the appearing height is sharp and not smooth.
  • The disappearing height is happening sharp and at the end instead of smooth.

I am not a web expert. All I have written are what I have got from different places and assembled. Please feel free to fix any other CSS/JS mistakes too. But the above issues are what I am mainly looking for.

function toggle_elem()
{
    var elem = document.getElementById('elemid');
    if(elem.style.display == 'none')
    {
        elem.style.display = 'block';
        window.setTimeout(function(){
            elem.style.opacity = 1;
            elem.style.transform = 'scale(1)';
            elem.style.height = '100px';
        },0);
    }
    else
    {
        elem.style.transform = 'scale(0)';
        window.setTimeout(function(){
            elem.style.display = 'none';
            elem.style.height = '0';
        }, 300);
    }
}
#elemid {
    border: 1px solid black;
    opacity: 0;
    transform: scale(0);
    transition: height 0.3s, width 0.3s, padding 0.3s, visibility 0.3s, 0.3s ease opacity, opacity 0.3s ease-out, 0.3s ease transform;
}
<div>
<button onclick="toggle_elem()">Toggle</button>
</div>

<div id="elemid" style="display:none">aaaaa</div>

<div>
<button >Next button</button>
</div>