How to have both popups and side panels for a chrome extension?

I’d like to have a popup that allows the extension to be turn on/off for specific sites, as well as a side panel to display more information.

My current manifest.json is below. At the moment, clicking on the extension icon in the toolbar (top right of chrome) always opens the side panel. I have to disable the side panel before I can see the popup.

"action": {
    "default_title": "Click to open extension",
    "default_popup": "popup.html"
  },
  "host_permissions": [
    "https://developer.chrome.com/*"
  ],
  "side_panel": {
    "default_path": "sidepanel.html"
  },