Hello Gurus!! I’m hoping someone can help me out, I’m sure I’m not the only one to have this issue but can’t seem to find any solutions from multiple days of googling.
I recently bought a SFF pc that I wanted to migrate my docker containers to - Z2M, mosquitto, node-red, esphome, etc - from a Raspberry Pi. Setup went just fine, was able to set up and connect my zigbee coordinator in Z2M with minimal issues. Updated my MQTT configuration inside HA. All my Tasmota devices worked once remapped to the new MQTT server, and some of my zigbee buttons and sensors started working again.
My issue is with my lights. They show as “Unavailable” in HA, but can access them without issue in Z2M (change color, toggle on/off, etc). Checked all the outgoing MQTT messages and everything seems to be fine. HA is “seeing” these messages if I inspect the “MQTT Info” link on the device page. The only thing I can find is in home-assistant.log - there are a series of Exception errors: Exception in async_discover when dispatching 'mqtt_discovery_new_binary_sensor_mqtt': ({'availability': [{'topic': 'zigbee2mqtt/bridge/state', 'value_template': '{{ value_json.state }}'}, and ending with voluptuous.error.MultipleInvalid: string value is None for dictionary value @ data['name'] .
What’s strange (just seeing this) is that this particular line is for an IKEA Tradfri light, but shows “new binary sensor”. The original errors I was inspecting was for mqtt_discovery_new_light_mqtt for some Hue bulbs. Seeing the outgoing messages from Z2M there is a value for 'name' - but the quoted mqtt message in the error shows the value as null.
I’ve tried deleting these lights from HA and Z2M and reimporting them, seems to work fine but the light. entity doesn’t show up in HA, only get a “Power On Behavior” entity for this device.
Sorry for the long winded post - I’m at a breaking point here and don’t have any hair left to pull out. Can post any logs or other info needed that would be helpful. For reference, plugging the zigbee coordinator back into my old Raspberry Pi sets everything back to working order.
Not sure what a ‘SFF pc’ is, however I do not thing it is the source of your problem. That said, do be more expressive in your posts, there are lots of smart folks here that can help. However, it is difficult to ‘peer’ into someone else’s configuration both hardware and software (not to mention someone else’s goals) without a verbose statement of your setup and configs, folks will ‘guess’ and send you down rabbit holes without such.
I am a bit old school and have a healthy skepticism for Home Assistants ability to ‘auto discover’. I would turn this off, I think it is in Zigbee2MQTT configuration, and first try and manually create a HA light via YAML or GUI for a single one of the your lights. Get this working and see what JSON values are working to get a ‘working’ light in HA.
JSON data can be a bit of a frustration to get setup and working. However, spending the time to learn how you can manually setup MQTT and JSON sensors and devices will be worth the effort.
I am guessing here, however it looks like the names you have given your devices in Zigbee2MQTT might be causing some issues. Again, in MQTT, go back to the most simple config (use defaults for names and all else), and get that working and see what is coming across via MQTT. Once you have a handle on that, change one thing at time (ONLY!). I am pretty sure you can get back to a auto discovery setup in HA from Zigbee2MQTT, but again go slow and evaluate each change.
Thanks for the thoughtful response. I do have a few questions as follow up:
Your suggestion to create a HA light via YAML/GUI - this would be direct in HA? Or Zigbee2MQTT? Looking at the configurations in Z2M they’re definitely messy, but reverting back to the original file copied from my old system doesn’t work either.
a) I assume should be in the HA’s configuration.yaml file? Something like the various examples listed here MQTT Light - Home Assistant ??
Not sure why the names now would be giving me issues. I didn’t do anything when migrating other than changing to the new computer, and updating IP address for the MQTT server. Light names were the same before and worked just fine.
Probably have a ton of other questions, but like you said I’m going to start simple.
For reference, by SFF PC I was referring to a 1 liter mini pc (HP Prodesk 600 G3 Mini in my case). Agree this isn’t the issue. Just wasn’t able to find any guides about migrating from one host to another, the one I found about restoring from backups failed on my end since was reading the backup as ARM (from Pi) to trying to restore to a AMD64 based system.
I too used a HP intel mini desktop I picked up for under USD 100, beefed up memory and SSD with some low cost upgrades and had a very good low power low cost reliable Linux docker device. I put a LTS version of Ubuntu on it, installed docker and ran these basic home automation containers:
a) Mosquitto MQTT server
b) Zigbee2MQTT
c) Postgresql database server for Home Assistant database and other databases
c) Home Assistant
This is a solid stack that ran reliable for me, I never need to upgrade the base LTS Ubuntu OS.
Example of how I create a new docker image of Home Assistant is below, with host diectory mappings for config directory. Again ‘old school docker command’. My based directory on the host Ubuntu system for HA is : /home/user/homeassistant this need to exist before running the docker command.
I could stop, start, upgrade each of the docker container with no interdependicies between each. Clearly you cannot stop Home Assistant’s database without first stopping HA. However, not having these servers run with Home Assistant is the most reliable way to build a solid home automation server IMHO. I ran Raspberry Pi 3 with similar config for a number of years successfully, however the added power and software standards of a Intel based system (especially a reliable low power unit such as the HP mini desktops is a better direction than a ARM Raspberry IMHO. Yes, what I describe above is more complex than as the ‘all in one’ Home Assistant OS Raspberry Pi image you can install, however not much, and IMHO worth your effort (it does seem like you are already down this rabbit hole some what )
I would get one or more low cost external SSD drives and do a Clonezilla off line full image backup of the host boot and home directory HDs every quarter or so. These backups are good to have in your quiver as a CYA.
Not sure what kind of zigbee bulbs you are working with, differences in abilities here can be a source of issues. My example below is for a pretty simple Cree Zigbee blub.
Not sure about your migration question, I’ve always done the docker image setup for Home Assistant and I map HA’s config directory to a directory on the host Ubuntu system. I can just zip up this complete directory and it’s sub directories (important to do this zip as root, as some directories may be owned by root and others not). I have used this zip file to completely move my HA docker system from one machine to another and also to completely restore HA back to a point in time (however, do be careful if you do an external database as I do, rather that HA’s internal SQLite db, up HA upgrades can modify the database to an extend that you can not go back without a database restore the the same version of HA you are go back to, or delete the external HA database and let HA recreate (losing your history however)).
I would download a MQTT viewing tool so you can see MQTT JSON data. I use MQTT explorer, link below.
So I am not minimizing that manually coding your MQTT devices, lights and sensors is a PIA. I stick with the manual method for my Zigbee2MQTT devices, just mostly due to my stubbornness I understand the ‘auto discovery and setup’ seems to work well for most folks. However as I said, I still think it is a good idea understand the MQTT messages and have an understand how to manually setup these MQTT devices. An example of old Cree Zigbee light that has been working for years with the config shown is below.
Again ‘old school me’ I continue to use the raw zigbee, bluetooth or wifi MAC address that comes thru (usually as the default ‘name’ of a device) via MQTT. Yes, you can create ‘friendly names’ that you will be the base topic in MQTT and then as the HA name of the device, but I prefer to anchor the device reference to it MAC address and then add a ‘nice’ name in HA via the ‘name:’ or similar attribute in HA’s MQTT device setup as shown below.
So to you point on the ‘names’ (item 2), I was just guessing. If you can, using the MQTT explorer tool publish the JSON data that Zigbee2MQTT is publishing for the device, first picture below. And in second picture below, make sure that you are getting for the source (in this case Zigbee2MQTT) a properly formatted JSON MQTT message.
I point about the ‘names’ that HA uses from Zigbee2MQTT is that these seem to be going under some revisions with the most recent version of both HA and Zigbee2MQTT. Again, jsut a wild ass guess by me. Link below is related to the naming changes that seem to be going on. Give specific attention to points about zigbee2mqtt versions and these changes:
)
