How to create a flex-box container with max of 2 columns that scrolls down upon an items overflow?

I’m trying to build a specific UI that consists of a flexbox container with flex-direction: column and flex-wrap: wrap. For each flex item that is added, the item will take 100% of the container width – and upon an overflow (first items overflow), the container will wrap the items into 2 columns – but the container will not have more then 2 columns – in the next items overflow it should scroll down (overflow-y: auto)?
Is this possible?

I tried a lot of approaches, even using display: grid in a way, but the task i more complicated then it seems.