Update plan python3.8

fortwally · November 16, 2020, 10:56pm

Two questions

If I want to save all my setup, most done with the GUI front end where are the files that hold the info? The automatons are in the automatons.yaml, but where are the cards setup?
If HA won’t support python3.5 after Dec 2020 and the Raspberry does not have a install for python3.8.
You have to do a build from source and updating a venv to a new version of python
is not easy. What is the plan? That is why question 1. So I can save all my setup and then build from scratch and reload my setup.

tjntomas · November 16, 2020, 11:56pm

Just backup your entire config folder which usually is in /home/homeassistant/.homeassistant, the same folder that contains your configuration.yaml file. It contains everything you need to restore your config.

You can make a backup using

 sudo tar -zcvf ha_backup.tar.gz /home/homeassistant/.homeassistant

You might want to look into running HA in docker instead. That way you never need to worry about python versions and dependencies since the docker image contains the relevant python version.

inkblotadmirer · November 25, 2020, 8:28pm

Thread: update-python-to-the-new-version-3-9

Generally, as the prior response indicates: if you don’t want to deal with this once a year, go with a Docker install. Otherwise the linked thread has a few tips on how to upgrade python, use separate venvs for alternate python/home assistant installs, etc.

Another good tip from the thread is to limit the python version to the latest being used in the Docker builds (which as of right now is NOT 3.9 as requested in the linked thread).

As to your first question, I have my home assistant setup in /home/homeassistant and my venvs are set up in /srv/venv/homeassistant_(python_version). I point systemd to the correct venv and config, so other than backing up prior to any major changes (always wise) you don’t need to reinstall any config.

dave_t_uk · December 3, 2020, 11:37pm

I see lots of threads with this sort of question.

Is there a good guide on how to migrate from venv to Docker or even a migration script? If so (and I think there is unanimous agreement that docker is a better option), then we should point people there, as it solves the problem for next year too. Maybe even in the release notes.

My first experience of this sort of thing in the past was more or less:

Python:
1. Install the latest version!
2. Yes, I can compile from source - let’s do this!
3. Wow, this is slow on an RPi1. Now it’s not happy about something to do with bigint.
4. Oh dear, now apt complains a lot when I do anything.
5. Why have I still got python 2.7? Maybe uninstalling that will help.
6. Now everything is completely broken. Wipe SD card and start again.
7. Wish I’d spent the last 2h with my kids.
8. Be more cautious next time and only install from source when I need to.
Docker!
1. Sounds great! Whales are awesome!
2. Download and install from docker.com! Oh no, there’s also docker-io and docker-ce? Did I pick the wrong one?
3. Docker one-liners work great, but where are my files, what do I need to back up and how do I make it run automatically at startup?
4. My USB devices and GPIO aren’t working any more.
5. Help!

Hopefully someone can reassure me that this sort of thing doesn’t just happen to me!

Don’t get me wrong, I enjoy the journey. But there are a lot of hurdles that could be avoided.

So how do we make the experience friendly and awesome for the approx 13% of HA users who will face this Christmas present at the end of a tough year ?

dave_t_uk · December 16, 2020, 11:42pm

Ok, so I made a start on this, but actually can’t get it fully working. Feeling a bit stupid to be honest.

Assuming the best bet is to move to docker, my hope was to create some instructions for how someone could migrate from venv to docker. I wanted to copy my old HA config from the venv method into new docker, so I did the following:

$ sudo systemctl stop [email protected] 
$ sudo systemctl disable [email protected] 
$ sudo apt-get install docker.io docker-compose
$ cd
$ cp -r .homeassistant .homeassistant_docker
$ mkdir dockercfg
$ cd dockercfg
$ nano docker-compose.yml

Type in this:

version: '3'
services:
  homeassistant:
    container_name: home-assistant
    image: homeassistant/raspberrypi4-homeassistant:stable
    volumes:
      - /home/homeassistant/.homeassistant_docker:/config 
      - /data/media-local:/data/media-local
      - /dev/ttyUSB0:/dev/ttyUSB0
      - /dev/ttyUSB1:/dev/ttyUSB1
      - /dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0:/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
    environment:
      - TZ=Europe/London
    restart: always
    network_mode: host

then run

$ docker-compose up -d

HA runs and I can connect to the web url from another machine on port 8123 but my ZHA and some other devices aren’t working. My old method of viewing HA output doesn’t work any more but I found I can do

docker logs -f --tail 20 home-assistant

to see what HA is doing. When I do I get:

2020-12-16 23:08:44 ERROR (MainThread) [zigpy.application] Couldn't start application
2020-12-16 23:08:44 ERROR (MainThread) [homeassistant.components.zha.core.gateway] Couldn't start ZNP = Texas Instruments Z-Stack ZNP protocol: CC253x, CC26x2, CC13x2 coordinator
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/serial/serialposix.py", line 265, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
PermissionError: [Errno 1] Operation not permitted: '/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0'

I see that the serial device exists within the docker container:

$ docker exec -it home-assistant /bin/bash
bash-5.0# ls -la /dev/tty*
crw-rw-rw-    1 root     root        5,   0 Dec 16 23:30 /dev/tty
crw-rw----    1 root     dialout   188,   0 Dec 16 22:36 /dev/ttyUSB0
crw-rw----    1 root     dialout   188,   1 Dec 16 22:36 /dev/ttyUSB1
bash-5.0# ls -la /dev/serial/by-id/*
crw-rw----    1 root     dialout   188,   1 Dec 16 22:36 /dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
bash-5.0# exit

But it’s readwrite by ‘dialout’ and ‘root’.

What user is HA running as within my docker container? It seems to be root. So maybe the permissions issue is with the user running docker. But that’s also root. So why can’t they access the USB port?

So yeah, I’m stuck now because that’s a bit beyond my docker knowledge so far.
For the record, I think this is already well beyond what non-technical users would have patience for.

…

What’s kind of good is that I can do

$ docker-compose stop

then

$ systemctl start [email protected]

To go back to my old method of running HA. So at least I’ve not burned any bridges/WAF points.

But can anyone help me?

fortwally · December 17, 2020, 5:58pm

I used Docker and not docker-compose.
To get access to the wyze usb I found I needed to add this to the docker run command.
–device=/dev/hidraw0:/dev/hidraw0
Don’t know what the equivalent for docker-compose is.

This may also be needed.
If you would like to use Docker as a non-root user, you should now consider
adding your user to the “docker” group with something like:

sudo usermod -aG docker pi

dave_t_uk · December 17, 2020, 11:27pm

ok, I got it working eventually. thanks for the hint @fortwally

here is my final docker-compose.yml:

version: '3'
services:
  homeassistant:
    container_name: home-assistant
    image: homeassistant/raspberrypi4-homeassistant:stable
    volumes:
      - /home/homeassistant/.homeassistant_docker:/config 
      - /data/media-local:/data/media-local
    devices:
      - /dev/ttyUSB0:/dev/ttyUSB0
      - /dev/ttyUSB1:/dev/ttyUSB1
      - /dev/ttyUSB1:/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
      - /dev/mem
    restart: always
    network_mode: host
    privileged: true
    cap_add:
      - SYS_RAWIO

followed by

$ docker-compose up -d

Now my GPIO, USB, ZHA are all working from within docker.