Moving from VM to Docker/Portainer

Installation
1

I have HA version 2023.10.5 running on a VM and I want to move it to a different server on Docker / Portainer.

My current install have Samba which allow me to map my Windows drive to HA /config folder.

Step 1:
I copied everything from /config folder to a local (Windows) folder.

Step 2:
To setup the environment in Docker / Portainer I followed this video
(taking care of the reversed volume info)

Step 3
Then I stopped the new HomeAssistant container, ssh’ed to /home/user/HomeAssistant/config and deleted all files and folders including hidden folders.

Step 4
I struggled but I believe I was able to copy over everything from step 1 to the new environment using SCP.
I first did scp -rp c:/Data/HA/. [email protected]:/home/user/HomeAssistant/config to copy the hidden folders with their respective content.
Then I did scp -rp c:/Data/HA/* [email protected]:/home/user/HomeAssistant/config to copy the rest.

Step 5
I started the container and I can see almost everything with the exception that obviously nothing is connected since all the IoT devices are still connecting to the old ip address.

The issues I have:
Issue #1:
I connected to one IoT device (SONOFF Tasmota) and from the MQTT parameters I changed the IP to the new environment but it will not connect.
I suspect my new environment is not listening tp port 1883 and I don’t know how to set this up.

Issue #2:
In the new environment it is missing ESPHome from the side menu and also missing Add-Ons from the Setting menu.
I validated the esphome folder has been copied over.

That is where I am and I am stuck there…

2

There are no add-ons in a docker installation.

Did you setup an MQTT broker already ? Docker or native ?

3

If there are no add-ons in a docker, does this mean I cannot use ESPHome?
MQTT was setup as an add-on so if no add-on is possible and no ESPHome then I guess I have no choice to stay on the curent VM

EDIT: I was under the impression I had done enough reading to migrate to Docker but the more I read, the more it gets complicated. So it looks like I need to create a container for ESPHome and another container for MQTT and I need to do a lot of reading to learn how the 3 can communicate. Is there a good link that can provide enough details? I have not found anything yet…

4

Addons are just docker containers automatically setup for you or with simplified install

Just install the docker container

Mosquitto
Esphome

If you havent discovered portainer you should look at it for container management

1 Like
5

Create a docker network >> connect containers to network>> assign containers a hostname in docker>>connect HA to container using hostname:port

macvlan

ipvlan

6

Thanks for the info, I’m off to reading the docs. By the time I get through all of this I should be a pro and manage to connect this to Cloudflare.

7

My environment:
Lenovo Thinkcenter tiny M73 with single NIC (planning to add a second NIC via mini PCIe)
Linux Debian 12.2 no desktop.
Connected to 172.16.0.124 / Mask /24 / GW 172.16.0.1
Installed Docker / Portainer
Created container: HomeAssistant / Mosquitto

I create a network name:IoT_mcvlan_cfg > driver: macvlan > type: configuration > parent: eno1 > subnet: 172.16.0.0/24 > gateway: 172.16.0.1 > Create network = Success

I create network name: IoT > driver: mcvlan > Type: creation > select: IoT_mcvlan_cfg > Create network = Success

I stoped container HomeAssistant > Select network: IoT > Join network = Success

I start HomeAssistant = Failure: Request failed with status code 500

EDIT: HomeAssistant can only start using host network.

I cannot find the logs for the network and the log for the container does not show anything about the failled attempt to start…

8

For starters, just create a single bridge network and add both HA and mosquitto to that. You can use portainer defaults.

Expose 8123 for home assistant and you should be OK.

Macvlan creation is complicated and I honestly need to google examples since i dont create this often. In the end it is not required to get HA going. Also the macvlan IP should exist on your network.

9

I created a Bridge, changed nothing and assigned to HomeAssistant succesfully but when I try to start the container I get Failure: Request failed with status code 500
Same thing with the default Bridge, it will only start if I join host network.

10

I can assign the bridge network I created to Mosquitto with no problem so I am thinking there is a problem within the Home Assistant configuration. As per noted at the beginning of this post I created the environment and then replaced all files with those from an other installation residing on a VM (not Docker)

I am trying to avoid recreating everything from scratch as I have a mixture of about 50 IoT (Shelly, Tasmotta, ESPHome) and a bunch of automation.

is there any file I can check? there nothing about the network in configuration.yaml

11

500? Not network related. Is it able to download image zfrom github, not dockerhub? You must set github up as repository in portainer to download github images.

12

Actually, 500 may be related to port in use error. Make sure another container is not using same ports you have used exposed and you may need to reboot host. Sometimes containers do not shutdown properly and cause this.

13

I am sooo confused…
First off, the only choice I have in the select box next to Registry is “Docker Hub (anonymous)”
On the image section if I click Search it takes me to Docker Hub.
Cap1

I did a test, I stopped the homeassistant container and I created another from the stack using this code from github
When I started this container, it detected all of my ESPHome devices although I didn’t install ESPHome. !!!

Anyways, this latest setup is also showing like the previous “Docker Hub (anonymous)” and it installed as host and I am getting the same 500 error if I try to join the bridge network.

At this point, I can’t start both HA containers because both are using port 8123 and this is a fresh Docker/Portainer install from yesterday (maybe the day before, I lost track) and I have not defined anywhere any ports other than that stack image and I have these as containers and I did a restart this morning:

Cap2
Cap21387×340 41.7 KB

14

Exactly the point.

In portainer, select Registries >> Github >> I think you just need to add a name for it and select [add registry]
You dont need to add personal access token or any other details. I forget exactly but to download from github in portainer you need it.

The most current image for Homeassistant is ghcr.io/ghcr.io/home-assistant/home-assistant:2023.11.0

I expect the Homeassistant container works

The Homeassistant_V2 container uses linuxserver image but I would use official image I posted above. The IP I believe will cause issues as well. macvlan you should create IP on your network but for now I would avoid that until all is working then take than as seperate project. Post the compose.yml you are using. I will place one below that I made. Pretty sure it will work as long as it can download the HA image from github. Run that without you config. After you verify running, add your config and test. As its currently set it will create the network.

services:
          
##########################################
#           HOMEASSISTANT                #
##########################################
  hass:
    container_name: homeassistant
    hostname: homeassistant
    privileged: false
    restart: unless-stopped
    image: ghcr.io/ghcr.io/home-assistant/home-assistant:2023.11.0
    volumes:
      - "/locationonyourserver/config:/config"
      - "/etc/localtime:/etc/localtime:ro"
    ports:
      - "8123:8123/tcp" # HA UI
      - "20165:21065/tcp" #
      - "5353:5353/udp" # mDNS
      - "51837:51827/udp" # homekit
    networks:
      dockerlocal:
  
##########################################
#                MQTT                    #
##########################################
  mosquitto:
    image: eclipse-mosquitto:2.0.18
    container_name: mosquitto
    hostname: mosquitto
    restart: "unless-stopped"
    volumes:
      - "/locationonyourserver/mosquitto_config:/mosquitto/config"
      - "/locationonyourserver/mosquitto_data:/mosquitto/data"
      - "/locationonyourserver/mosquitto_log:/mosquitto/log"
      - "/etc/localtime:/etc/localtime:ro"
#    ports:
#      - "1883:1883/tcp" #no ports exposed, communicates using dockerlocal network
#      - "9001:9001/tcp" #no ports exposed, communicates using dockerlocal network
    networks:
      - dockerlocal
     
##########################################
#              NETWORKS                  #
##########################################      
networks:
  dockerlocal:
    driver: bridge
15

Great! I see progress…
I replaced /locationonyourserver/ with the different paths I previously created and deployed the stack.

This created a container for homeassistant and mosquitto and a network but the IP for homeassistant was not on my network. So I cleared the bridge network and joinned the mcvlan I had previously created and was able to access the homeassistant web GUI.
The only thing is, I did the same for mosquitto but it is not taking an IP:

Cap3
Cap31263×214 29.8 KB

16

I see that the ports for mosquitto were commented. I uncommented and updated the stack but then I get this:
failed to deploy a stack: service “mosquitto” refers to undefined volume 1883: invalid compose project

EDIT: nevermind I had forgotten to uncomment the header “ports”. I get an IP now

17

Actually, the IP is assigned but then goes away and in the log there is this:

Error: Unable to open config file /mosquitto/config/mosquitto.conf

On the stack file I had modified the paths for MQTT like this:

/home/myuser/mqtt/mosquitto_config:/mosquitto/config
/home/myuser/mqtt/mosquitto_data:/mosquitto/data
/home/myuser/mqtt/mosquitto_log:/mosquitto/log

and I had previously created those directories:

/home/myuser/mqtt/mosquitto_config
/home/myuser/mqtt/mosquitto_data
/home/myuser/mqtt/mosquitto_log

However there is nothing in them.

18

You may need to add a default mosquitto config. It may not have.

19

Yes I manually created it and it is good now.

Thank you for your help.

20

I have been playing, testing, reinstalling and almost everything is working now.
The only issue I have which is not a big thing but would be nice if it could work, All ESPHome devices are showing offline on the dashboard. I tried adding to the ESPHome stack file:

environment:
  ESPHOME_DASHBOARD_USE_PING =true

but that did not work.

I tried installing avahi-daemon on the server but that took my homeassistant down due to conflicting port 5353 so I commented out this port on the stack image, homeassistant was back but still have the same problem.

Anything else I can try?