@tom_l further investigation has revealed that I may have caused this mornings crash by inadvertently having the google drive backup occur at the same time as the vm snapshot I’ve subsequently changed the google drive backup to another time.
HA was stable in its restricted/deactivated state from 1300 yesterday until the backup/snapshot snafu this morning. It seems stable “ish” now.
I’m going to trigger a manual backup and see if it succeeds
UDPATE:
the backup succeeded memory free went down as far as 4.6Gb and is slowly recovering to similar levels to before the backup.
i did not know that you can hover over the graph and see a RAM value in the system page.
using 328 of 642.5Mb total available RAM
this is the supervisor log for the backup
22-05-02 11:41:47 INFO (MainThread) [supervisor.api.middleware.security] /core/info access from cebe7a76_hassio_google_drive_backup
22-05-02 11:41:47 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-05-02 11:41:47 INFO (MainThread) [supervisor.api.middleware.security] /backups/new/full access from cebe7a76_hassio_google_drive_backup
22-05-02 11:41:47 INFO (MainThread) [supervisor.backups.manager] Creating new full backup with slug 7442ffae
22-05-02 11:41:47 INFO (MainThread) [supervisor.backups.manager] Backing up 7442ffae store Add-ons
22-05-02 11:41:47 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_ssh
22-05-02 11:41:47 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_ssh
22-05-02 11:41:47 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_vscode
22-05-02 11:42:02 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_vscode
22-05-02 11:42:02 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on core_samba
22-05-02 11:42:02 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon core_samba
22-05-02 11:42:02 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on core_mariadb
22-05-02 11:42:06 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-05-02 11:46:49 WARNING (MainThread) [supervisor.api.ingress] No valid ingress session 41db089ed633364b3eb2069418405af1a6557ede00b9aa15be77d6fc9a096d2d2ba3533366b52df0f82da4ec7b5f00f734b65f81ef422f5ec57e07bace88d44e
22-05-02 11:46:49 WARNING (MainThread) [supervisor.api.ingress] No valid ingress session e9f0e5671bfc3bd8da088ffec3456c841098a75aa2644842fd0bd8951b2e573438f4a0c1a5ee7c46682e783f19c7315501031d61d5ae12107f5266fe95df9218
22-05-02 11:46:49 ERROR (MainThread) [asyncio] Task exception was never retrieved
future: <Task finished name='Task-46344' coro=<_websocket_forward() done, defined at /usr/src/supervisor/supervisor/api/ingress.py:280> exception=ConnectionResetError('Cannot write to closing transport')>
Traceback (most recent call last):
File "/usr/src/supervisor/supervisor/api/ingress.py", line 287, in _websocket_forward
await ws_to.send_bytes(msg.data)
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_ws.py", line 315, in send_bytes
await self._writer.send(data, binary=True, compress=compress)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 688, in send
await self._send_frame(message, WSMsgType.BINARY, compress)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 653, in _send_frame
self._write(header + message)
File "/usr/local/lib/python3.9/site-packages/aiohttp/http_websocket.py", line 663, in _write
raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport
22-05-02 11:46:49 INFO (MainThread) [supervisor.api.proxy] Home Assistant WebSocket API connection is closed
22-05-02 11:46:54 INFO (MainThread) [supervisor.api.proxy] Home Assistant WebSocket API request initialize
22-05-02 11:46:54 INFO (MainThread) [supervisor.api.proxy] WebSocket access from a0d7b954_vscode
22-05-02 11:46:54 INFO (MainThread) [supervisor.api.proxy] Home Assistant WebSocket API request running
22-05-02 11:50:01 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon core_mariadb
22-05-02 11:50:01 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_nodered
22-05-02 11:50:01 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_nodered
22-05-02 11:50:01 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_nginxproxymanager
22-05-02 11:50:02 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_nginxproxymanager
22-05-02 11:50:02 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on cebe7a76_hassio_google_drive_backup
22-05-02 11:50:02 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon cebe7a76_hassio_google_drive_backup
22-05-02 11:50:02 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_influxdb
22-05-02 11:54:05 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-05-02 11:54:05 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-05-02 12:12:10 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_influxdb
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on a0d7b954_grafana
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon a0d7b954_grafana
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on 39bd2704_promtail
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon 39bd2704_promtail
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Building backup for add-on 5c53de3b_esphome
22-05-02 12:17:47 INFO (MainThread) [supervisor.addons.addon] Finish backup for addon 5c53de3b_esphome
22-05-02 12:17:47 INFO (MainThread) [supervisor.homeassistant.module] Backing up Home Assistant Core config folder
22-05-02 12:17:55 INFO (MainThread) [supervisor.homeassistant.module] Backup Home Assistant Core config folder done
22-05-02 12:17:55 INFO (MainThread) [supervisor.backups.manager] Backing up 7442ffae store folders
22-05-02 12:17:55 INFO (SyncWorker_6) [supervisor.backups.backup] Backing up folder share
22-05-02 12:17:55 INFO (SyncWorker_6) [supervisor.backups.backup] Backup folder share done
22-05-02 12:17:55 INFO (SyncWorker_7) [supervisor.backups.backup] Backing up folder addons/local
22-05-02 12:17:55 INFO (SyncWorker_7) [supervisor.backups.backup] Backup folder addons/local done
22-05-02 12:17:55 INFO (SyncWorker_5) [supervisor.backups.backup] Backing up folder ssl
22-05-02 12:17:57 INFO (SyncWorker_5) [supervisor.backups.backup] Backup folder ssl done
22-05-02 12:17:57 INFO (SyncWorker_1) [supervisor.backups.backup] Backing up folder media
22-05-02 12:17:57 INFO (SyncWorker_1) [supervisor.backups.backup] Backup folder media done
22-05-02 12:18:04 INFO (MainThread) [supervisor.backups.manager] Creating full backup with slug 7442ffae completed
22-05-02 12:18:04 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-05-02 12:18:04 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-05-02 12:18:04 INFO (MainThread) [supervisor.api.middleware.security] /backups/7442ffae/info access from cebe7a76_hassio_google_drive_backup
22-05-02 12:18:05 INFO (MainThread) [supervisor.api.middleware.security] /backups/020d401d access from cebe7a76_hassio_google_drive_backup
22-05-02 12:18:05 INFO (MainThread) [supervisor.backups.manager] Removed backup file 020d401d
22-05-02 12:18:05 INFO (MainThread) [supervisor.api.middleware.security] /backups/7442ffae/download access from cebe7a76_hassio_google_drive_backup
22-05-02 12:18:05 INFO (MainThread) [supervisor.api.backups] Downloading backup 7442ffae
22-05-02 12:22:36 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-05-02 12:22:36 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-05-02 12:22:37 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-05-02 12:22:37 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-05-02 12:41:12 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-05-02 12:42:14 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
core log for the same time
2022-05-02 11:46:21 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
resp = await request_handler(request)
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
resp = await handler(request)
File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 219, in auth_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle
result = await result
File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 1069, in get
data, content_type = await player.async_get_media_image()
File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 520, in async_get_media_image
return await self._async_fetch_image_from_cache(url)
File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 992, in _async_fetch_image_from_cache
(content, content_type) = await self._async_fetch_image(url)
File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 1003, in _async_fetch_image
return await async_fetch_image(_LOGGER, self.hass, url)
File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 1202, in async_fetch_image
content = await response.read()
File "/usr/local/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 1036, in read
self._body = await self.content.read()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 375, in read
block = await self.readany()
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 397, in readany
await self._wait("readany")
File "/usr/local/lib/python3.9/site-packages/aiohttp/streams.py", line 304, in _wait
await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed
2022-05-02 11:46:49 ERROR (MainThread) [homeassistant.components.hassio.handler] /ingress/validate_session return code 401
2022-05-02 11:46:49 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /ingress/validate_session -
2022-05-02 11:46:49 ERROR (MainThread) [homeassistant.components.hassio.handler] /ingress/validate_session return code 401
2022-05-02 11:46:49 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /ingress/validate_session -
2022-05-02 12:00:00 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities switch.socket_dyson_charger
2022-05-02 12:01:00 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities switch.socket_dyson_charger
2022-05-02 12:29:48 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities button.kingpad_z10_load_start_url
2022-05-02 12:38:46 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities button.kingpad_z10_load_start_url
2022-05-02 12:47:52 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities button.kingpad_z10_load_start_url
2022-05-02 12:50:35 WARNING (MainThread) [homeassistant.components.sonos.speaker] Subscription renewal failed for Office: Request timed out
2022-05-02 13:00:00 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities switch.socket_dyson_charger
2022-05-02 13:01:00 WARNING (MainThread) [homeassistant.helpers.service] Unable to find referenced entities switch.socket_dyson_charger