OS upgrade causing connectivity dropouts?

HugosDad · December 30, 2022, 11:30am

Running HA Yellow for a few weeks now and am seeing regular drop outs in terms of integrations being able to reach web services etc.
Notably, HA frequently can’t see my heat pump and is causing the hot water routine to fail which is obviously causing wider annoyance across the household!

This seems to have started happening when I took the latest OS upgrade, not HA core.
I don’t really understand what I’m seeing in the logs other than the multiple (but not all) integrations that are failing to connect to their devices.

It’s not an internet connectivity problem, it’s the Yellow itself…

The gaps in this chart give an idea of how frequently I’m seeing this

I’ve looked at the logs and can see lots of errors, I suspect it’s a web socket problem but have looked at this for so long I just can’t see the wood for the trees.

If anyone could advise where to start with this lot, then I can post the relevant full log extract from anything suspicious:

Thanks for any assistance with this! Am losing my mind!!

PeteRage · December 30, 2022, 12:31pm

It’d be helpful to have a full text log as a lot of those entries are truncated in your screen shot,

It looks like a network problem. Are the connections that are failing cloud connections (like the co2signal ) or local connections (using an IP in your LAN) or both?

Power cycle is a good idea.

HugosDad · December 30, 2022, 12:34pm

Hi Pete, the log full file is 30mb! Is there a specific bit I can extract that would be helpful?

It seems to be both local and cloud, which leads me to suspect it’s the HA Yellow’s connectivity rather than anything else. I’ll power cycle it and see what it comes back up with.

Here’s the log since power cycling the Yellow:

2022-12-30 12:38:17.111 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration nibe which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.115 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration powercalc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.117 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.120 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration solcast_solar which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.123 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration apple_airplayer which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.126 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration feedparser which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.129 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration comfoconnect which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.132 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration fullykiosk which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.134 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration aarlo which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.137 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration adaptive_lighting which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.139 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration shelly which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:17.142 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration meross_lan which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-12-30 12:38:32.451 ERROR (MainThread) [snitun.client.client_peer] Challenge/Response error with SniTun server (0 bytes read on a total of 32 expected bytes)
2022-12-30 12:38:32.460 ERROR (MainThread) [hass_nabucasa.remote] Connection problem to snitun server
2022-12-30 12:39:21.525 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: hassio
2022-12-30 12:39:49.630 WARNING (ArloBackgroundWorker) [pyaarlo] request-error=ReadTimeout
2022-12-30 12:40:08.672 WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecated `async_get_registry` to access entity registry, use async_get instead. Please report issue to the custom integration author for shelly using this method at custom_components/shelly/__init__.py, line 356: await self.hass.helpers.entity_registry.async_get_registry()
2022-12-30 12:40:22.054 WARNING (MainThread) [homeassistant.components.media_player] Setup of media_player platform webostv is taking over 10 seconds.
2022-12-30 12:40:22.055 WARNING (MainThread) [homeassistant.components.media_player] Setup of media_player platform webostv is taking over 10 seconds.
2022-12-30 12:40:22.107 ERROR (MainThread) [pyatv.protocols.airplay] Failed to set up remote control channel
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 390, in send_and_receive
    await event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 389, in send_and_receive
    async with async_timeout.timeout(4):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/__init__.py", line 258, in _connect_rc
    await control.start(str(core.config.address), control_port, credentials)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/remote_control.py", line 56, in start
    self.verifier = await verify_connection(credentials, self.connection)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/auth/__init__.py", line 103, in verify_connection
    has_encryption_keys = await verifier.verify_credentials()
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/auth/hap_transient.py", line 66, in verify_credentials
    await self.http.post(
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 360, in post
    return await self.send_and_receive(
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 393, in send_and_receive
    raise TimeoutError(f"no response to {method} {uri} ({protocol})") from ex
TimeoutError: no response to POST /pair-setup (HTTP/1.1)
2022-12-30 12:40:22.166 ERROR (MainThread) [pyatv.protocols.airplay] Failed to set up remote control channel
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 390, in send_and_receive
    await event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 389, in send_and_receive
    async with async_timeout.timeout(4):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/__init__.py", line 258, in _connect_rc
    await control.start(str(core.config.address), control_port, credentials)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/remote_control.py", line 56, in start
    self.verifier = await verify_connection(credentials, self.connection)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/auth/__init__.py", line 103, in verify_connection
    has_encryption_keys = await verifier.verify_credentials()
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/airplay/auth/hap_transient.py", line 50, in verify_credentials
    resp = await self.http.post(
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 360, in post
    return await self.send_and_receive(
  File "/usr/local/lib/python3.10/site-packages/pyatv/support/http.py", line 393, in send_and_receive
    raise TimeoutError(f"no response to {method} {uri} ({protocol})") from ex
TimeoutError: no response to POST /pair-setup (HTTP/1.1)
2022-12-30 12:40:22.247 ERROR (MainThread) [homeassistant.components.tado.device_tracker] Cannot load Tado data
2022-12-30 12:40:22.309 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform forecast_solar is taking over 10 seconds.
2022-12-30 12:40:22.417 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform speedtestdotnet is taking over 10 seconds.
2022-12-30 12:40:23.017 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Hue Bridge' for hue integration not ready yet: Error connecting to the Hue bridge at 192.168.9.117; Retrying in background
2022-12-30 12:40:23.052 ERROR (MainThread) [homeassistant.components.device_tracker] Error setting up platform legacy tado
2022-12-30 12:40:25.735 ERROR (MainThread) [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_sensor_mqtt': ({'name': 'Battery Status', 'state_topic': 'solis2mqtt/battery_status', 'unique_id': 'solis2mqtt/battery_status', 'device_class': None, 'state_class': 'measurement', 'unit_of_measurement': None, 'device': {'name': 'solis2mqtt', 'model': 'solis2mqtt', 'manufacturer': 'Ginlong Technologies', 'identifiers': 'solis2mqtt', 'sw_version': 'solis2mqtt 0.7'}, 'platform': 'mqtt'},)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/mixins.py", line 328, in async_discover
    config: DiscoveryInfoType = discovery_schema(discovery_payload)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
    return self._exec((Schema(val) for val in self.validators), v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
    raise e if self.msg is None else AllInvalid(self.msg, path=path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
    v = func(v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: expected SensorDeviceClass or one of 'date', 'duration', 'timestamp', 'apparent_power', 'aqi', 'battery', 'carbon_monoxide', 'carbon_dioxide', 'current', 'distance', 'energy', 'frequency', 'gas', 'humidity', 'illuminance', 'moisture', 'monetary', 'nitrogen_dioxide', 'nitrogen_monoxide', 'nitrous_oxide', 'ozone', 'pm1', 'pm10', 'pm25', 'power_factor', 'power', 'precipitation', 'precipitation_intensity', 'pressure', 'reactive_power', 'signal_strength', 'speed', 'sulphur_dioxide', 'temperature', 'volatile_organic_compounds', 'voltage', 'volume', 'water', 'weight', 'wind_speed' for dictionary value @ data['device_class']

2022-12-30 12:40:25.744 ERROR (MainThread) [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_sensor_mqtt': ({'name': 'Storage Mode', 'state_topic': 'solis2mqtt/storage_mode', 'unique_id': 'solis2mqtt/storage_mode', 'device_class': None, 'state_class': None, 'unit_of_measurement': None, 'device': {'name': 'solis2mqtt', 'model': 'solis2mqtt', 'manufacturer': 'Ginlong Technologies', 'identifiers': 'solis2mqtt', 'sw_version': 'solis2mqtt 0.7'}, 'platform': 'mqtt'},)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/mixins.py", line 328, in async_discover
    config: DiscoveryInfoType = discovery_schema(discovery_payload)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
    return self._exec((Schema(val) for val in self.validators), v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
    raise e if self.msg is None else AllInvalid(self.msg, path=path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
    v = func(v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: expected SensorDeviceClass or one of 'date', 'duration', 'timestamp', 'apparent_power', 'aqi', 'battery', 'carbon_monoxide', 'carbon_dioxide', 'current', 'distance', 'energy', 'frequency', 'gas', 'humidity', 'illuminance', 'moisture', 'monetary', 'nitrogen_dioxide', 'nitrogen_monoxide', 'nitrous_oxide', 'ozone', 'pm1', 'pm10', 'pm25', 'power_factor', 'power', 'precipitation', 'precipitation_intensity', 'pressure', 'reactive_power', 'signal_strength', 'speed', 'sulphur_dioxide', 'temperature', 'volatile_organic_compounds', 'voltage', 'volume', 'water', 'weight', 'wind_speed' for dictionary value @ data['device_class']

2022-12-30 12:40:25.752 ERROR (MainThread) [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_sensor_mqtt': ({'name': 'Inverter Status', 'state_topic': 'solis2mqtt/inverter_status', 'unique_id': 'solis2mqtt/inverter_status', 'device_class': None, 'state_class': None, 'unit_of_measurement': None, 'device': {'name': 'solis2mqtt', 'model': 'solis2mqtt', 'manufacturer': 'Ginlong Technologies', 'identifiers': 'solis2mqtt', 'sw_version': 'solis2mqtt 0.7'}, 'platform': 'mqtt'},)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/mixins.py", line 328, in async_discover
    config: DiscoveryInfoType = discovery_schema(discovery_payload)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
    return self._exec((Schema(val) for val in self.validators), v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
    raise e if self.msg is None else AllInvalid(self.msg, path=path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
    v = func(v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: expected SensorDeviceClass or one of 'date', 'duration', 'timestamp', 'apparent_power', 'aqi', 'battery', 'carbon_monoxide', 'carbon_dioxide', 'current', 'distance', 'energy', 'frequency', 'gas', 'humidity', 'illuminance', 'moisture', 'monetary', 'nitrogen_dioxide', 'nitrogen_monoxide', 'nitrous_oxide', 'ozone', 'pm1', 'pm10', 'pm25', 'power_factor', 'power', 'precipitation', 'precipitation_intensity', 'pressure', 'reactive_power', 'signal_strength', 'speed', 'sulphur_dioxide', 'temperature', 'volatile_organic_compounds', 'voltage', 'volume', 'water', 'weight', 'wind_speed' for dictionary value @ data['device_class']

2022-12-30 12:40:25.759 ERROR (MainThread) [homeassistant.util.logging] Exception in async_discover when dispatching 'mqtt_discovery_new_sensor_mqtt': ({'name': 'Battery Two Voltage', 'state_topic': 'solis2mqtt/batt2_voltage', 'unique_id': 'solis2mqtt/batt2_voltage', 'device_class': None, 'state_class': None, 'unit_of_measurement': 'V', 'device': {'name': 'solis2mqtt', 'model': 'solis2mqtt', 'manufacturer': 'Ginlong Technologies', 'identifiers': 'solis2mqtt', 'sw_version': 'solis2mqtt 0.7'}, 'platform': 'mqtt'},)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/mqtt/mixins.py", line 328, in async_discover
    config: DiscoveryInfoType = discovery_schema(discovery_payload)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 232, in __call__
    return self._exec((Schema(val) for val in self.validators), v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 355, in _exec
    raise e if self.msg is None else AllInvalid(self.msg, path=path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 351, in _exec
    v = func(v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: expected SensorDeviceClass or one of 'date', 'duration', 'timestamp', 'apparent_power', 'aqi', 'battery', 'carbon_monoxide', 'carbon_dioxide', 'current', 'distance', 'energy', 'frequency', 'gas', 'humidity', 'illuminance', 'moisture', 'monetary', 'nitrogen_dioxide', 'nitrogen_monoxide', 'nitrous_oxide', 'ozone', 'pm1', 'pm10', 'pm25', 'power_factor', 'power', 'precipitation', 'precipitation_intensity', 'pressure', 'reactive_power', 'signal_strength', 'speed', 'sulphur_dioxide', 'temperature', 'volatile_organic_compounds', 'voltage', 'volume', 'water', 'weight', 'wind_speed' for dictionary value @ data['device_class']

2022-12-30 12:40:26.051 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback MQTT._mqtt_handle_message(<paho.mqtt.cl... 0x7f7911c430>)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/client.py", line 695, in _mqtt_handle_message
    self._mqtt_data.state_write_requests.process_write_state_requests()
  File "/usr/src/homeassistant/homeassistant/components/mqtt/models.py", line 264, in process_write_state_requests
    entity.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 545, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 583, in _async_write_ha_state
    state = self._stringify_state(available)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 551, in _stringify_state
    if (state := self.state) is None:
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 690, in state
    raise ValueError(
ValueError: Invalid datetime: sensor.system_datetime provides state '2022-12-30 11:42:04', which is missing timezone information
2022-12-30 12:40:26.079 ERROR (MainThread) [homeassistant.components.sensor] Error adding entities for domain sensor with platform mqtt
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 428, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 692, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 789, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 545, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 583, in _async_write_ha_state
    state = self._stringify_state(available)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 551, in _stringify_state
    if (state := self.state) is None:
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 690, in state
    raise ValueError(
ValueError: Invalid datetime: sensor.system_datetime provides state '2022-12-30 11:42:04', which is missing timezone information
2022-12-30 12:40:28.301 ERROR (MainThread) [homeassistant.components.apple_tv] Failed to connect
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 274, in _receive
    await semaphore.acquire()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 390, in acquire
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 222, in connect_once
    await self._connect(conf, raise_missing_credentials)
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 323, in _connect
    self.atv = await connect(conf, self.hass.loop, session=session)
  File "/usr/local/lib/python3.10/site-packages/pyatv/__init__.py", line 135, in connect
    await atv.connect()
  File "/usr/local/lib/python3.10/site-packages/pyatv/core/facade.py", line 575, in connect
    if await setup_data.connect():
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/__init__.py", line 941, in _connect
    await protocol.start()
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 142, in start
    self.device_info = await self.send_and_receive(
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 261, in send_and_receive
    return await self._receive(identifier, timeout)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 273, in _receive
    async with async_timeout.timeout(timeout):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
2022-12-30 12:40:28.311 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Bedroom' for apple_tv integration not ready yet: Not found at 192.168.9.32, waiting for discovery; Retrying in background
2022-12-30 12:40:28.318 ERROR (MainThread) [homeassistant.components.apple_tv] Failed to connect
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 274, in _receive
    await semaphore.acquire()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 390, in acquire
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 222, in connect_once
    await self._connect(conf, raise_missing_credentials)
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 323, in _connect
    self.atv = await connect(conf, self.hass.loop, session=session)
  File "/usr/local/lib/python3.10/site-packages/pyatv/__init__.py", line 135, in connect
    await atv.connect()
  File "/usr/local/lib/python3.10/site-packages/pyatv/core/facade.py", line 575, in connect
    if await setup_data.connect():
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/__init__.py", line 941, in _connect
    await protocol.start()
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 142, in start
    self.device_info = await self.send_and_receive(
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 261, in send_and_receive
    return await self._receive(identifier, timeout)
  File "/usr/local/lib/python3.10/site-packages/pyatv/protocols/mrp/protocol.py", line 273, in _receive
    async with async_timeout.timeout(timeout):
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
2022-12-30 12:40:28.323 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Kitchen' for apple_tv integration not ready yet: Not found at 192.168.9.38, waiting for discovery; Retrying in background
2022-12-30 12:40:36.665 ERROR (Update loop) [pyShelly] Timeout connecting to http://192.168.9.6/shelly [shellyplus1pm-a8032ab9d2e0.localdomain]
2022-12-30 12:40:37.366 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: climate.nibe, switch.nibe, sensor.nibe, water_heater.nibe, fan.nibe
2022-12-30 12:40:42.991 WARNING (MainThread) [homeassistant.components.climate] Setup of climate platform nibe is taking over 10 seconds.
2022-12-30 12:40:42.994 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform nibe is taking over 10 seconds.
2022-12-30 12:40:43.000 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform nibe is taking over 10 seconds.
2022-12-30 12:40:43.006 WARNING (MainThread) [homeassistant.components.water_heater] Setup of water_heater platform nibe is taking over 10 seconds.
2022-12-30 12:40:43.008 WARNING (MainThread) [homeassistant.components.fan] Setup of fan platform nibe is taking over 10 seconds.
2022-12-30 12:41:15.293 WARNING (SyncWorker_9) [homeassistant.components.waze_travel_time.sensor] Error on retrieving data: empty response
2022-12-30 12:41:21.191 ERROR (MainThread) [homeassistant.components.apple_tv] Failed to connect
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 222, in connect_once
    await self._connect(conf, raise_missing_credentials)
  File "/usr/src/homeassistant/homeassistant/components/apple_tv/__init__.py", line 311, in _connect
    raise ConfigEntryNotReady(
homeassistant.exceptions.ConfigEntryNotReady: Protocol(s) RAOP not yet found for Bedroom, waiting for discovery.
2022-12-30 12:41:24.799 ERROR (SyncWorker_0) [homeassistant] Error doing job: Unclosed client session

Thanks!

PeteRage · December 30, 2022, 4:28pm

Are you able to connect to the yellow across your LAN? If so at least some network connectivity works.

I don’t know much about hass-os.

If it is just cloud, i would suggest looking at firewalls, etc; but that appears not to be the issue.

HugosDad · December 30, 2022, 5:43pm

Yep, it’s connected all the time it’s just showing regular drop outs from local and cloud based integrations… Wall tablet has become unusable without a manual refresh too.

HugosDad · December 30, 2022, 8:25pm

So, looking closer at this I’m not sure it is a connectivity issue. I’ve been watching the logbook and seeing all the problematic devices / integrations going unavailable / available repeatedly…

What would be the best approach to troubleshoot this behaviour?

PeteRage · December 31, 2022, 1:52pm

I’d start with power cycling all the network equipment. Routers, switches. Then power cycle all of the devices. You may want to also replace the Ethernet cable.

We want to determine if it’s a problem with your network or a problem with the HASS box.

If you can establish ping sensors on HASS to some of the problematic local equipment and run the same ping commands repeatedly from your laptop, you may be able to see the pings fail from both locations or just one.

Since this did start with upgrading HASS-so, you could also try reimaging the HASS box from scratch.

HugosDad · December 31, 2022, 2:15pm

Thanks Pete, I’m 99% sure the problem is on the HA Yellow…

When HA is reporting ‘unavailable’ for the devices, those with their own admin page at their IP address can still be accessed via their IP.

Their doesn’t seem to be any logic in terms of timing or duration or even what errors appear in the log first.

I think the issue is likely web socket / MQTT, but it seems to be bringing down quite a lot of the system. It’s a large install here so would really not like to re-image!

This looks suspicious to me:

Logger: homeassistant
Source: components/sensor/__init__.py:690 
First occurred: 13:11:41 (1 occurrences) 
Last logged: 13:11:41

Error doing job: Exception in callback MQTT._mqtt_handle_message(<paho.mqtt.cl... 0x7f7fa1d460>)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/components/mqtt/client.py", line 695, in _mqtt_handle_message
    self._mqtt_data.state_write_requests.process_write_state_requests()
  File "/usr/src/homeassistant/homeassistant/components/mqtt/models.py", line 264, in process_write_state_requests
    entity.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 545, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 583, in _async_write_ha_state
    state = self._stringify_state(available)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 551, in _stringify_state
    if (state := self.state) is None:
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 690, in state
    raise ValueError(
ValueError: Invalid datetime: sensor.system_datetime provides state '2022-12-30 11:42:04', which is missing timezone information