Developing Home Assistant Core in a VSCode Devcontainer

Developing Home Assistant Core in a VSCode Devcontainer

Setting Up the Local Repository

  1. Ensure that git installed on your development wokstation
  2. Visit the Home Assistant Core repository and click Fork.
  3. Open a terminal and set up your local repository
git clone https://github.com/YOUR_GIT_USERNAME/core.git Home Assistant-core
cd Home Assistant-core
git remote add upstream https://github.com/home-assistant/core.git
  1. Create a new feature branch that tracks the dev branch in
git checkout -b my_dev_branch --track origin/dev
  1. Open the local repository in VSCode

Configuring Home Assistant

By default, the devcontainer will create the config directory inside the container if it doesnā€™t already exist. Optionally, you can override this by bind mounting a config directory from your local filesystem, which can be helpful for long-term persistence or swapping in/out different configurations to support development.

To do this, include a list of mounts inside /.devcontainer/devcontainer.json:

  "mounts": [
    // Custom configuration directory
    "source=${localEnv:HOME}/path/to/config,target=${containerWorkspaceFolder}/config,type=bind",
  ]

If these mounts are modified, then the DevContainer must to be rebuilt

  • Press F1, and enter Remote-Containers: Rebuild Container

See Working with containers in Visual Studio Code for more information.

Running Home Assistant

If this the first time running Home Assistant, or the container has been rebuilt, you should use this method for starting Home Assistant. This method will install all of the out-of-the-box libraries required to run HA (including the debugger, which can then be used later).

  1. Open the the local repository in VSCode

  2. Accept the prompt to ā€œReopen in Containerā€

  3. Start HA in the container via the menu options:

    Terminal | Run Task... | Run Home Assistant Core

  4. Open HA in a web browser at http://localhost:8123

See Debugging Home Assistant for more information about debug mode.

Keeping Code Up To Date

This is adapted from Catching up with Reality | Home Assistant Developer Docs

The following commands will keep your local feature branch up-to-date with Home Assistantā€™s official dev branch.
Reminder: upstream is the official HA repository, origin is your remote repository that was forked from it

  • Switch to a local feature branch
  • Pull the latest commits from a HAā€™s upstream/dev branch
  • Replay the upstream commits to the local branch, then re-applies local changes
  • Push the combined commits back to the forked repository on GitHub
git checkout my_dev_branch
git fetch upstream dev
git rebase upstream/dev
git push origin --force

Debugging Home Assistant

Set Up Home Assistantā€™s Remote Python Debugger

Add the following to /config/configuration.yaml:

debugpy:
  start: true
  wait: false

See Remote Python Debugger - Home Assistant for more information.

Start Debugging

Using the debugger after using the Running Home Assistant startup method is not preferred for debugging, as it the debugger does not attach at the very beginning of the Home Assistant startup process. To start Home Assistant immediately in debug mode, do the following.

  1. Stop any running tasks if you used the Running Home Assistant method
    Terminal | Terminate Task... | Run Home Assistant Core
  2. Switch to the VSCode Run & Debug view
  3. Select the Home Assistant launch configuration from the dropdown
  4. Start the debugger

If You Use Run Task to Start Home Assistant

  1. Ensure that the HA devcontainer is running (`Terminal | Run Taskā€¦ | Run Home Assistant Core``)
  2. Switch to the VSCode Run & Debug view
  3. Select the Home Assistant: Attach Local launch configuration from the dropdown
  4. Start the debugger

Debugging Custom Components and Libraries

When developing custom components or libraries that exist in other local respoitories on your development machine, the easiest way to incorporate these into Home Assistant is to bind mount those repositories into HAā€™s config directory.

To do this, include a list of mounts inside /.devcontainer/devcontainer.json, similar to this example:

  "mounts": [
    // Custom configuration directory
    "source=${localEnv:HOME}/path/to/config,target=${containerWorkspaceFolder}/config,type=bind",
    // Custom component
    "source=${localEnv:HOME}/path/to/custom_repo/custom_components/component_name,target=${containerWorkspaceFolder}/config/custom_components/component_name,type=bind",
    //Custom library
    //"source=${localEnv:HOME}/path/to/custom_repo/custom_libraries/library_name,target=${containerWorkspaceFolder}/config/custom_libraries/library_name,type=bind",
  ]

Add more bind mounts as needed for your use case.

If these mounts are modified, then the devcontainer must to be rebuilt

  • Press F1, and enter Remote-Containers: Rebuild Container

Custom Components

Custom components are mounted in the container under config/custom_components, which is a Home Assistant standard.

Custom Libraries

Custom libraries are mounted under config/custom_libraries. Note that custom_libraries is a personal naming preference, and is not prescribed in the Home Assistant docs.

In order to develop a custom library, HA needs to be started with the --skip-pip argument as described here: Building a Python library for an API | Home Assistant Developer Docs

This launch config option is available in the Run & Debug view as Home Assistant (skip pip)

Prerequisites

Before starting HA in skip pip mode, start HA at least once in normal mode, which will install all out-of the-box libraries the via the normal HA boot process. This can be done via either of these methods:

  • Running Home Assistant
  • Debugging Home Assistant

Overriding an Out-Of-The-Box Library

If overriding a out-of-the-box-in library, it must first be removed from the DevContainer. Go to the VSCode terminal and enter the following:

pip uninstall library_name

Installing a Custom Library

To then enable a custom library, have pip install it from the filesystem. It should be accessible via the bind mount(s) created above. Go to the VSCode terminal and enter the following:

pip install -e ./config/custom_libraries/library_name

Using the Custom Library

Stop the running instance of Home Assistant, then restart using the Home Assistant (skip pip) debug option shown above

24 Likes

Useful tips!

One thing I havenā€™t figured out yet: How do you customize devcontainer.json to specify mounts, without having to ignore this file for git?

Not sure I completely understandā€¦devcontainer.json is not ignored by default (best I can tell). Regardless, if it is ignored, you should be able to modify the .gitignore file to your preference.

I want to edit devcontainer.json to include mount points.
But I do not want to push those changes to GitHub (so I would gitignore).
However, I would like to pull any changes to devcontainer.json that come from upstream (which will not happen if I gitignore).

If you want hide/externalize the mount configuration outside of devcontainer.json, I think the solution might be to define mount points using environment variables in an .env file, which are documented at https://code.visualstudio.com/docs/remote/containers-advanced#_adding-environment-variables.

If you find a working solution, please report back, and we can improve the documentation.

When doing this on windows is the code in the windows partition or in WSL?

Thanks, you can help me please, config frondEnd enviroment ?

Unfortunately, I havenā€™t done any frontend development against the dev container. Hopefully someone else can contribute additional inputs.

On Windows, use:

  "mounts": [
    // custom component
    "source=${localEnv:USERPROFILE}\\OneDrive\\Documents\\docs\\dev\\philips-airpurifier-coap\\custom_components\\philips_airpurifier_coap,target=${containerWorkspaceFolder}/config/custom_components/philips_airpurifier_coap,type=bind",
  ],

to mount a custom component, sitting on your windows partition on, e.g., OneDrive in a sub-folder.

Thanks for this guide! It saved me so much nerves while trying to debug something in HA! :slight_smile: Seems debugging only works via debugpy. Without it the default debug config doesnā€™t work for me.

Glad to hear it worked for you! Strangely, I tried setting up the devcontainer for some debugging a few weeks ago, and could not the the debugger to stop on a breakpoint. Did you run into any issues like that?

Thanks for the guide, I feel that Iā€™m missing something.

I am not certain what I actually have to do when updating to the latest tip of the dev branch to get my container working. I feel that every time thereā€™s a new HA version, my devcontainer breaks and I waste a lot of time trying to get it to work again. Hereā€™s what I do:

git checkout dev
git fetch upstream dev
git rebase upstream/dev

click 'Dev Container' in bottom left > rebuild container

script/setup

Then ctrl-shift-p Tasks:Run-task>Run Home Assistant Core

HA starts running, but I get lots of errors. e.g.

2022-09-13 20:23:36.560 ERROR (MainThread) [homeassistant.setup] Setup failed for analytics: Unable to import component: No module named 'sqlalchemy'
2022-09-13 20:23:39.021 INFO (MainThread) [homeassistant.setup] Setting up image
2022-09-13 20:23:39.022 INFO (MainThread) [homeassistant.setup] Setup of domain image took 0.0 seconds
2022-09-13 20:23:39.023 INFO (MainThread) [homeassistant.setup] Setting up person
2022-09-13 20:23:39.025 INFO (MainThread) [homeassistant.setup] Setup of domain person took 0.0 seconds
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of onboarding. Setup failed for dependencies: analytics
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Setup failed for onboarding: (DependencyError(...), 'Could not setup dependencies: analytics')
2022-09-13 20:23:39.026 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of frontend. Setup failed for dependencies: onboarding
2022-09-13 20:23:39.027 ERROR (MainThread) [homeassistant.setup] Setup failed for frontend: (DependencyError(...), 'Could not setup dependencies: onboarding')
2022-09-13 20:23:39.027 INFO (MainThread) [homeassistant.bootstrap] Setting up recorder: {'recorder'}
2022-09-13 20:23:39.057 INFO (SyncWorker_0) [homeassistant.util.package] Attempting install of sqlalchemy==1.4.40
2022-09-13 20:23:42.550 INFO (SyncWorker_0) [homeassistant.util.package] Attempting install of fnvhash==0.1.0
2022-09-13 20:23:44.677 INFO (MainThread) [homeassistant.setup] Setting up recorder
2022-09-13 20:23:44.680 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //workspaces/core/config/home-assistant_v2.db was shutdown cleanly
2022-09-13 20:23:44.683 ERROR (Recorder) [homeassistant.components.recorder.util] Version 3.27.2 of SQLite is not supported; minimum supported version is 3.31.0. Starting with Home Assistant 2022.6 this prevents the recorder from starting. Please upgrade your database software
2022-09-13 20:23:44.686 INFO (MainThread) [homeassistant.setup] Setup of domain recorder took 0.0 seconds
2022-09-13 20:23:44.686 ERROR (MainThread) [homeassistant.setup] Setup failed for recorder: Integration failed to initialize.

Ok so somethingā€™s clearly wrong with some database software. But didnā€™t I just checkout the latest code and rebuild the dev container? So doesnā€™t that give me everything at the latest version? So why is it asking me to ā€˜Upgrade my database softwareā€™? What database software? I donā€™t even know how Iā€™d check what version I have. SQLite doesnā€™t seem to be installed on the host machine or in the container.

Any ideas? Should what Iā€™m doing work or am I missing a step?

Nope. No issues since I added debugpy in HA config and launching the debug config, which does the attach to a running HA instance. I am able to tldebug both custom components and HA internal code. Th breakpoints stop without any issues.

Iā€™ve setup the dev container already a couple of time throughout the years, but when I did this on the weekend I encountered errors while launching the ā€˜Runā€™ task for the first time.
HA missed a lot of core integrations and launched in safe mode.i think I got the same errors as you.
I didnā€™t know what the problem was. I simply closed VS Code, reopened it again so the dev container started again and since then - no issues. So thereā€™s something wrong with the recent dev container build in my opinion. I never had such issues before.

Oh one more thing: I do not use the git commands to rebuild my container since I didnā€™t know about this way :slightly_smiling_face: I follow ā€œmy wayā€ which is rebaselining my HA fork in GitHub.com, then syncing my Windows GitHub desktop with my GitHub account, and then I click ā€œOpen in VS Codeā€, so when it opens it asks if it should open in dev container. When I confirm - VS Code starts rebuilding the container automatically.
I guess itā€™s practically the same thing as using commands.

Ok, would be interested to know what Iā€™m doing wrong at some point. Or if not for it to be fixed.

The git commands donā€™t rebuild the dev container, they just update the source files on the drive.

Then you could restart vscode to get it to prompt to rebuild, or just trigger the rebuild directly (I think these have the same effect).

If itā€™s still not working for you then scrap it and start over. Normally it handles all the needed things automatically. Better to wait and let it do its thing than to spend hours on googling. The whole container build takes not more than 10 minutes for me

Adding my 3 cents on this topic for people seeking a solution to debugging issues in the future.
It turns out that debugging within devcontainer is enabled by default, out of the box. Normally thereā€™s no need to enable the debugpy integration to debug the local devcontainer, developer instance.
However it didnā€™t work for me and since there was nothing about debugging in HA developer docs - I thought itā€™s not supported and some extra steps need to be taken - like the ones above.

When using the default devcontainer debugger I was getting this error: ā€œTimed out waiting for launcher to connect.ā€ (leaving the text for the search engines)
image

I goggled the problem, but didnā€™t find any solution that worked, but one thread pointed me to something that allowed my to solve/workaround this.
The trick that worked for me and allowed to use the ā€œdefaultā€ debugging way (that probably works for most of the people) is to edit and add this to your launch.json file:

 "console": "internalConsole"

So the entire config entry looks like this:
image

The effect is almost the same as enabling the debugpy integration in HA configuration, but has one advantage: you may debug even line 1 of HA startup code + it launches Home Assistant for you, no need to run the ā€˜Run/Previewā€™ task first. The debugpy way only allows debugging after Home Assistant was launched and after it loaded the debugpy integration.
After applying this line in launch.json the Home Assistant log will not be output in a new Terminal session, but instead VS Code Debug console window will be used.

The entire issue is probably related to some bugs in VS Code, but this solution works for me for real. I disabled it and debugging stopped working immediately. When I put it back it - it immediately started working again, so this really solves this.

Hope it helps someone in the future, who ran into the same issue as me or the OP.

Iā€™ve tried scrapping it and starting over. Thatā€™s the problem, it doesnā€™t fix it. In my case, not talking about debug, just running HA. My current guess is that one of the following is true:

  • Something is broken in the master dev container and everyone is having to fix it
  • Something is wrong with my working copy which is causing the dev container to be wrong, or perhaps some caching Iā€™m unaware of
  • Something is wrong with VSCode in the way that it creates dev containers
  • Something is wrong with my Host OS (Ubuntu) install, which is influencing the dev container
  • My understanding of dev containers is flawed and they are not really giving the level of containerisation that I thought they were.

For me a key point is this error:

Version 3.27.2 of SQLite is not supported; minimum supported version is 3.31.0. Starting with Home Assistant 2022.6 this prevents the recorder from starting. Please upgrade your database software

My question: Where did Version 3.27.2 of SQLite come from? I didnā€™t install it. So if I get this appearing straight after deploying a fresh dev container as defined on the dev branch, then doesnā€™t it mean the container installed the wrong version?

Iā€™m going to investigate more, just sharing where I got to.

I think Iā€™ve got to the bottom of it. The devcontainer is using python 3.9, and bundled with python 3.9 is sqlite3 version 3.27.2.

Demonstration: rebuilt docker container.

root āžœ /workspaces/core (dev āœ—) $ python3         
Python 3.9.4 (default, Apr 10 2021, 15:31:19) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.27.2'

Then modified Docker.dev first line, replacing
FROM mcr.microsoft.com/vscode/devcontainers/python:0-3.9
with
FROM mcr.microsoft.com/vscode/devcontainers/python:0-3.10

Rebuilt container again, now I get this:

root āžœ /workspaces/core (dev āœ—) $ python3 
Python 3.10.6 (main, Aug 23 2022, 08:25:41) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.34.1'

But hereā€™s the problem, in homeassistant.components.recorder.util.py we have MIN_VERSION_SQLITE = ... "3.31.0"

So my conclusion is that the Docker.dev file is using a python version that is too old for the for the MIN_VERSION_SQLITE defined in the recorder component.

So itā€™s probably not just me seeing this error. It might be everyone running python 3.9.

Actually it looks like the microsoft image for python 3.9 might be whatā€™s wrong.

According to the python 3.9.0 tag, the first sqlite3 version bundled in python 3.9.0 was 3.32.3 (which should be fine).

I have raised a ticket.