Developing Home Assistant Core in a VSCode Devcontainer

Developing Home Assistant Core in a VSCode DevContainer

While the official developer documentation makes some references to VSCode devcontainers, there does not appear to be a comprehensive guide for setting it up to develop and debug HA core. I started documenting the steps that worked well for me, then realized it would be best to share back with the community in a place where it wouldn’t get lost.

I’m suspect that some (much?) of this isn’t best practice or 100% accurate. But I hope that by sharing this, it will encourage others to contribute information about their workflows and we can continue to grow the developer community.

Developing Home Assistant Core in a VSCode Devcontainer

Setting Up the Local Repository

  1. Ensure that git installed on your development wokstation
  2. Visit the Home Assistant Core repository and click Fork.
  3. Open a terminal and set up your local repository
git clone https://github.com/YOUR_GIT_USERNAME/core.git
cd core
git remote add upstream https://github.com/home-assistant/core.git
  1. Create a new feature branch that tracks the dev branch in
git checkout -b my_dev_branch --track origin/dev
  1. Open the local repository in VSCode

Configuring HomeAssistant

By default, the devcontainer will create the config directory inside the container if it doesn’t already exist. Optionally, you can override this by bind mounting a config directory from your local filesystem, which can be helpful for long-term persistence or swapping in/out different configurations to support development.

To do this, include a list of mounts inside /.devcontainer/devcontainer.json:

  "mounts": [
    // Custom configuration directory
    "source=${localEnv:HOME}/path/to/config,target=${containerWorkspaceFolder}/config,type=bind",
  ]

If these mounts are modified, then the devcontainer must to be rebuilt

  • Press F1, and enter Remote-Containers: Rebuild Container

See https://code.visualstudio.com/docs/remote/containers-advanced#_adding-another-local-file-mount for more information.

Running HomeAssistant

  1. Open the the local repository in VSCode

  2. Accept the prompt to “Reopen in Container”

  3. Start HA in the container via the menu options:

    Terminal | Run Task... | Preview

  4. Open HA in a web browser at http://localhost:8123

Keeping Code Up To Date

This is adapted from https://developers.home-assistant.io/docs/development_catching_up

The following commands will keep your local feature branch up-to-date with HomeAssistant’s official dev branch.
Reminder: upstream is the official HA repository, origin is your remote repository that was forked from it

  • Switch to a local feature branch
  • Pull the latest commits from a HA’s upstream/dev branch
  • Replay the upstream commits to the local branch, then re-apply local changes
  • Push the combined commits back to the forked repository on GitHub
git checkout my_dev_branch
git fetch upstream dev
git rebase upstream/dev
git push origin --force

Debugging HomeAssistant

Set up HomeAssistant’s Remote Python Debugger

Add the following to /config/configuration.yaml:

debugpy:
  start: true
  wait: true

See https://www.home-assistant.io/integrations/debugpy/ for more information.

Configure the VSCode Debugger

Insert the following debug configurations into /.vscode/launch.json:

{
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            // Example of attaching to local devcontainer
            "name": "HomeAssistant: Attach to Local Devcontainer",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "localhost",
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "/workspaces/${workspaceFolderBasename}"
                }
            ],
        },
        {
            // For attaching to a remote Production server
            // Update host as needed
            // HA code is located at /usr/src/homeassistant in a Supervised install
            "name": "HomeAssistant: Attach to Remote",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "homeassistant.local",
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "/usr/src/homeassistant"
                }
            ],
        }
    ]
}

Start Debugging

  1. Ensure that the HA devcontainer is running (Terminal | Run Task... | Preview)
  2. Switch to the VSCode Debugger view
  3. Select the appropriate debug configuration from the dropdown
  4. Start the debugger

Debugging Custom Components and Libraries

When developing custom components or libraries that exist in other local respoitories on your development machine, the easiest way to incorporate these into HomeAssistant is to bind mount those repositories into HA’s config directory.

To do this, include a list of mounts inside /.devcontainer/devcontainer.json, similar to this example:

  "mounts": [
    // Custom configuration directory
    "source=${localEnv:HOME}/path/to/config,target=${containerWorkspaceFolder}/config,type=bind",
    // Custom component
    "source=${localEnv:HOME}/path/to/custom_repo/custom_components/component_name,target=${containerWorkspaceFolder}/config/custom_components/component_name,type=bind",
    // Custom library
    "source=${localEnv:HOME}/path/to/custom_repo/custom_libraries/library_name,target=${containerWorkspaceFolder}/config/custom_libraries/library_name,type=bind",
  ]

Add more bind mounts as needed for your use case.

If these mounts are modified, then the devcontainer must to be rebuilt

  • Press F1, and enter Remote-Containers: Rebuild Container

Custom Components

Custom components are mounted in the container under config/custom_components, which is a HomeAssistant standard.

Custom Libraries

Custom libraries are mounted under config/custom_libraries. Note that custom_libraries is a personal naming preference, and is not prescribed in the HomeAssistant docs.

In order to develop a custom library, HA needs to be started with a custom variant of the Preview task, which invokes the --skip-pip argument as described here: https://developers.home-assistant.io/docs/api_lib_index

Add the following to .vscode/tasks.json

    // This task should be used when developing in custom_libraries.
    // Any time the container is rebuilt, first run the regular Preview task,
    // which installs all HA dependencies.  Then, swap in the custom_libraries
    // by running the following in the terminal:
    // `pip uninstall library_name
    // `pip install -e ./config/custom_libraries/library_name`
    {
      "label": "Preview with Custom Libraries",
      "type": "shell",
      "command": "hass --skip-pip -c ./config",
      "group": {
        "kind": "test",
        "isDefault": true
      },
      "presentation": {
        "reveal": "always",
        "panel": "new"
      },
      "problemMatcher": []
    },

Before starting HA with this modified Preview task, run the normal Preview task, which will install any libraries the via the normal HA boot process.

Terminal | Run Task... | Preview

If overriding a built-in library, go to the terminal and uninstall the built-in one:

pip uninstall library_name

To then enable a custom library, have pip install it from the filesystem. This should be accessible via the bind mount(s) created above.

pip install -e ./config/custom_libraries/library_name

Finally, to ensure that the custom library is used and not automatically overwritten by HomeAssistant at the next boot, stop the ‘normal’ Preview.

Terminal | Terminate Task... then select Preview

Now, to start HomeAssistant with custom libraries enabled, start the custom preview command created above:

Terminal | Run Task... | Preview with Custom Libraries

16 Likes

Useful tips!

One thing I haven’t figured out yet: How do you customize devcontainer.json to specify mounts, without having to ignore this file for git?

Not sure I completely understand…devcontainer.json is not ignored by default (best I can tell). Regardless, if it is ignored, you should be able to modify the .gitignore file to your preference.

I want to edit devcontainer.json to include mount points.
But I do not want to push those changes to GitHub (so I would gitignore).
However, I would like to pull any changes to devcontainer.json that come from upstream (which will not happen if I gitignore).

If you want hide/externalize the mount configuration outside of devcontainer.json, I think the solution might be to define mount points using environment variables in an .env file, which are documented at https://code.visualstudio.com/docs/remote/containers-advanced#_adding-environment-variables.

If you find a working solution, please report back, and we can improve the documentation.

When doing this on windows is the code in the windows partition or in WSL?

Thanks, you can help me please, config frondEnd enviroment ?

Unfortunately, I haven’t done any frontend development against the dev container. Hopefully someone else can contribute additional inputs.

On Windows, use:

  "mounts": [
    // custom component
    "source=${localEnv:USERPROFILE}\\OneDrive\\Documents\\docs\\dev\\philips-airpurifier-coap\\custom_components\\philips_airpurifier_coap,target=${containerWorkspaceFolder}/config/custom_components/philips_airpurifier_coap,type=bind",
  ],

to mount a custom component, sitting on your windows partition on, e.g., OneDrive in a sub-folder.

Thanks for this guide! It saved me so much nerves while trying to debug something in HA! :slight_smile: Seems debugging only works via debugpy. Without it the default debug config doesn’t work for me.

Glad to hear it worked for you! Strangely, I tried setting up the devcontainer for some debugging a few weeks ago, and could not the the debugger to stop on a breakpoint. Did you run into any issues like that?

Thanks for the guide, I feel that I’m missing something.

I am not certain what I actually have to do when updating to the latest tip of the dev branch to get my container working. I feel that every time there’s a new HA version, my devcontainer breaks and I waste a lot of time trying to get it to work again. Here’s what I do:

git checkout dev
git fetch upstream dev
git rebase upstream/dev

click 'Dev Container' in bottom left > rebuild container

script/setup

Then ctrl-shift-p Tasks:Run-task>Run Home Assistant Core

HA starts running, but I get lots of errors. e.g.

2022-09-13 20:23:36.560 ERROR (MainThread) [homeassistant.setup] Setup failed for analytics: Unable to import component: No module named 'sqlalchemy'
2022-09-13 20:23:39.021 INFO (MainThread) [homeassistant.setup] Setting up image
2022-09-13 20:23:39.022 INFO (MainThread) [homeassistant.setup] Setup of domain image took 0.0 seconds
2022-09-13 20:23:39.023 INFO (MainThread) [homeassistant.setup] Setting up person
2022-09-13 20:23:39.025 INFO (MainThread) [homeassistant.setup] Setup of domain person took 0.0 seconds
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of onboarding. Setup failed for dependencies: analytics
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Setup failed for onboarding: (DependencyError(...), 'Could not setup dependencies: analytics')
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of frontend. Setup failed for dependencies: onboarding
2022-09-13 20:23:39.027 ERROR (MainThread) [homeassistant.setup] Setup failed for frontend: (DependencyError(...), 'Could not setup dependencies: onboarding')
2022-09-13 20:23:39.027 INFO (MainThread) [homeassistant.bootstrap] Setting up recorder: {'recorder'}
2022-09-13 20:23:39.057 INFO (SyncWorker_0) [homeassistant.util.package] Attempting install of sqlalchemy==1.4.40
2022-09-13 20:23:42.550 INFO (SyncWorker_0) [homeassistant.util.package] Attempting install of fnvhash==0.1.0
2022-09-13 20:23:44.677 INFO (MainThread) [homeassistant.setup] Setting up recorder
2022-09-13 20:23:44.680 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //workspaces/core/config/home-assistant_v2.db was shutdown cleanly
2022-09-13 20:23:44.683 ERROR (Recorder) [homeassistant.components.recorder.util] Version 3.27.2 of SQLite is not supported; minimum supported version is 3.31.0. Starting with Home Assistant 2022.6 this prevents the recorder from starting. Please upgrade your database software
2022-09-13 20:23:44.686 INFO (MainThread) [homeassistant.setup] Setup of domain recorder took 0.0 seconds
2022-09-13 20:23:44.686 ERROR (MainThread) [homeassistant.setup] Setup failed for recorder: Integration failed to initialize.

Ok so something’s clearly wrong with some database software. But didn’t I just checkout the latest code and rebuild the dev container? So doesn’t that give me everything at the latest version? So why is it asking me to ‘Upgrade my database software’? What database software? I don’t even know how I’d check what version I have. SQLite doesn’t seem to be installed on the host machine or in the container.

Any ideas? Should what I’m doing work or am I missing a step?

Nope. No issues since I added debugpy in HA config and launching the debug config, which does the attach to a running HA instance. I am able to tldebug both custom components and HA internal code. Th breakpoints stop without any issues.

I’ve setup the dev container already a couple of time throughout the years, but when I did this on the weekend I encountered errors while launching the ‘Run’ task for the first time.
HA missed a lot of core integrations and launched in safe mode.i think I got the same errors as you.
I didn’t know what the problem was. I simply closed VS Code, reopened it again so the dev container started again and since then - no issues. So there’s something wrong with the recent dev container build in my opinion. I never had such issues before.

Oh one more thing: I do not use the git commands to rebuild my container since I didn’t know about this way :slightly_smiling_face: I follow “my way” which is rebaselining my HA fork in GitHub.com, then syncing my Windows GitHub desktop with my GitHub account, and then I click “Open in VS Code”, so when it opens it asks if it should open in dev container. When I confirm - VS Code starts rebuilding the container automatically.
I guess it’s practically the same thing as using commands.

Ok, would be interested to know what I’m doing wrong at some point. Or if not for it to be fixed.

The git commands don’t rebuild the dev container, they just update the source files on the drive.

Then you could restart vscode to get it to prompt to rebuild, or just trigger the rebuild directly (I think these have the same effect).

If it’s still not working for you then scrap it and start over. Normally it handles all the needed things automatically. Better to wait and let it do its thing than to spend hours on googling. The whole container build takes not more than 10 minutes for me

Adding my 3 cents on this topic for people seeking a solution to debugging issues in the future.
It turns out that debugging within devcontainer is enabled by default, out of the box. Normally there’s no need to enable the debugpy integration to debug the local devcontainer, developer instance.
However it didn’t work for me and since there was nothing about debugging in HA developer docs - I thought it’s not supported and some extra steps need to be taken - like the ones above.

When using the default devcontainer debugger I was getting this error: “Timed out waiting for launcher to connect.” (leaving the text for the search engines)
image

I goggled the problem, but didn’t find any solution that worked, but one thread pointed me to something that allowed my to solve/workaround this.
The trick that worked for me and allowed to use the “default” debugging way (that probably works for most of the people) is to edit and add this to your launch.json file:

 "console": "internalConsole"

So the entire config entry looks like this:
image

The effect is almost the same as enabling the debugpy integration in HA configuration, but has one advantage: you may debug even line 1 of HA startup code + it launches Home Assistant for you, no need to run the ‘Run/Preview’ task first. The debugpy way only allows debugging after Home Assistant was launched and after it loaded the debugpy integration.
After applying this line in launch.json the Home Assistant log will not be output in a new Terminal session, but instead VS Code Debug console window will be used.

The entire issue is probably related to some bugs in VS Code, but this solution works for me for real. I disabled it and debugging stopped working immediately. When I put it back it - it immediately started working again, so this really solves this.

Hope it helps someone in the future, who ran into the same issue as me or the OP.

I’ve tried scrapping it and starting over. That’s the problem, it doesn’t fix it. In my case, not talking about debug, just running HA. My current guess is that one of the following is true:

  • Something is broken in the master dev container and everyone is having to fix it
  • Something is wrong with my working copy which is causing the dev container to be wrong, or perhaps some caching I’m unaware of
  • Something is wrong with VSCode in the way that it creates dev containers
  • Something is wrong with my Host OS (Ubuntu) install, which is influencing the dev container
  • My understanding of dev containers is flawed and they are not really giving the level of containerisation that I thought they were.

For me a key point is this error:

Version 3.27.2 of SQLite is not supported; minimum supported version is 3.31.0. Starting with Home Assistant 2022.6 this prevents the recorder from starting. Please upgrade your database software

My question: Where did Version 3.27.2 of SQLite come from? I didn’t install it. So if I get this appearing straight after deploying a fresh dev container as defined on the dev branch, then doesn’t it mean the container installed the wrong version?

I’m going to investigate more, just sharing where I got to.

I think I’ve got to the bottom of it. The devcontainer is using python 3.9, and bundled with python 3.9 is sqlite3 version 3.27.2.

Demonstration: rebuilt docker container.

root ➜ /workspaces/core (dev ✗) $ python3         
Python 3.9.4 (default, Apr 10 2021, 15:31:19) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.27.2'

Then modified Docker.dev first line, replacing
FROM mcr.microsoft.com/vscode/devcontainers/python:0-3.9
with
FROM mcr.microsoft.com/vscode/devcontainers/python:0-3.10

Rebuilt container again, now I get this:

root ➜ /workspaces/core (dev ✗) $ python3 
Python 3.10.6 (main, Aug 23 2022, 08:25:41) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.34.1'

But here’s the problem, in homeassistant.components.recorder.util.py we have MIN_VERSION_SQLITE = ... "3.31.0"

So my conclusion is that the Docker.dev file is using a python version that is too old for the for the MIN_VERSION_SQLITE defined in the recorder component.

So it’s probably not just me seeing this error. It might be everyone running python 3.9.

Actually it looks like the microsoft image for python 3.9 might be what’s wrong.

According to the python 3.9.0 tag, the first sqlite3 version bundled in python 3.9.0 was 3.32.3 (which should be fine).

I have raised a ticket.