Start Again? Hassbian, Python upgrade on Rpi results in multiple issues

Hello to you all!

Much searching here and other places has improved the situation but my Home Assistant installation is still a right mess.

A bit of history. Raspberry Pi and Hassbian were running happily for about a year - using headless system with Sandisk USB drive instead of the SD card. It is mainly used to monitor ground source heat pump (GSHP) with a proprietary interface card. As I wanted to store a lot of metrics I use MariaDB (MySQL) as database. MQTT is used as a transport between the interface card and HA. Another Rpi also uses it with the BME680 chip for environment monitoring.

A few weeks ago the RPi died a death and I suspect that that USB stick was to blame. I did have a backup taken about a month previously (via Etcher) so It thought I could rebuild from that. However, both the original Etcher and the new Balena Etcher would not kick off from “Starting”. Also, my Samsung laptop was crashing regularly with Windows Explorer. In desparation I contacted Samsung who pointed out that I should not have update the laptop to Win10! I suggested that perhaps the laptop should have realised it was not compatible and therefore not suggest the upgrade!

But I digress.

After much hassle, I recovered the original data from the backup and attempted to update HA. It informed me that it was now dependent on a later version of Python and so I updated that too. (3.7).

Whilst attempting to do the rebuild and boot from USB, I noticed that I cannot get it running “headerless” unless I plug in a keyboard and mouse! (I have found similar issues reported but all those fixes do not work here.)

I cannot get any response from HA unless I kick off Hass manually. In the confusion I seem to have configuration.yaml in three different directories too.

For the life of me I cannot understand why I am getting this when I have installed MySQL:

2019-10-03 16:02:52 ERROR (Recorder) [homeassistant.components.recorder] Error during connection setup: No module named '_sqlite3' (retrying in 3 seconds)

When executed automatically I get this (sorry about the format - nearly forty years of OpenVMS support is not much use with Linux’s utilities!):

Oct  3 14:24:03 hassbian systemd[1]: Started Permit User Sessions.
Oct  3 14:24:03 hassbian systemd[1]: Started OpenBSD Secure Shell server.
Oct  3 14:24:05 hassbian mysqld[602]: 2019-10-03 14:24:05 1988481024 [Note] /usr/sbin/mysqld (mysqld 10.1.23-MariaDB-9+deb9u1) starting as process 602 ...
Oct  3 14:24:07 hassbian systemd[1]: Started MariaDB database server.
Oct  3 14:24:09 hassbian hass[503]: 2019-10-03 14:24:09 INFO (SyncWorker_1) [homeassistant.util.package] Attempting install of pyotp==2.2.7
Oct  3 14:24:15 hassbian hass[503]: 2019-10-03 14:24:15 ERROR (SyncWorker_1) [homeassistant.util.package] Unable to install package pyotp==2.2.7: ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: 'METADATA'
Oct  3 14:24:15 hassbian hass[503]: Check the permissions.

Much searching has got me nowhere with the above error!

I am reluctant to start from absolute scratch as interfacing the code from the GSHP via MQTT took a considerable amount of time. I suspect I have got confused with installation within a virtual environment but never had any hassle before now with it.

(The new version of Python even broke the GSHP code.)

Some lessons learnt for me:

  1. Do not depend on the USB sticks as they seem to break when used for frequent I/O;
  2. Sort out an automated backup system - somehow;
  3. Document steps which work - and especially useful Linux commands!

I still see confusion regarding the various upgrade methods for Home Assistant too. I think a Noddy Guide is needed for the likes of me!

(Please accept my profound apologise for the size of this post!)

And here is the recorder section:

   db_url: mysql://hass:Password@localhost/homeassistant?charset=utf8
   purge_interval: 1
   purge_keep_days: 200
       - sensor
       - binary_sensor
       - sensor.met_office_weather
       - sensor.met_office_temperature
       - sensor.met_office_wind_speed

After a great deal of farting about most things are working - but did need a syntax change on some Python source file due to a suspect tightening of syntax rules on the latest version.

I understand the concept of simplifying HA to allow pure GUI config work and I suspect that users like me are going to be quite rare - quite a bit of development and troubleshooting experience but not on the specific technologies. (I had to learn quick to sort this out though!)

The sqlite3 problem was due to some missing library/module or whatever. (Oh, and I now understand why a virtual environment is needed.)

I have created a dog’s breakfast by following various instructions from various sources as I do not consider I have enough years left on the planet to understand Linux to the depth which is needed to sort this out properly. I suspect that the lack of definitive instructions to overcome simple config issues has not helped. (As has my lack of knowledge of Linux! Like how to find where a file lives!)

The heat pump interface still does not work but that is my next challenge.

Thanks to all who at least read the entry.