InfluxDB Writes Lagging For High Polling Sensors

I’ve setup a InfluxDB V2 with the ultimate goal of creating a Graphana dashboard back in HA. Very briefly, I’m running HA Docker (version 2024.5.5) with the InfluxDB running on same Ubuntu Server that hosts the docker container. The Ubuntu Server is a VM on a XCP-NG pool that has been allocated 2 CPU cores and 8 GBs of RAM. Storage device that all of the data is saved to is a run of the mill M.2 NVME.

My configuration between HA and InfluxDB works as data is showing up in the InfluxDB Data Explorer interface. However, high polling rate devices (specifically Shelly Power Plugs) are presenting a real problem. The first minute of writes to the InfluxDB is reasonably in real time, but after that writes slow to a crawl. Eventually the writes fall behind real time by 5 or more minutes. The longer I let it run the worse it gets. I’m guessing that this is also affecting other sensors which are attempting to be written into the InfluxDB. This also affects the HA instance during reboot as the processes that are writing to the database halt the shutdown and throw the associated errors included at the bottom of this post.

Looking at various system resource measures through htop, iostats, etc… doesn’t show that the system is overtaxed. Writes speeds during operation peak at ~0.3MB/s, cpu utilization is around 1-2%, and there’s ample head room in memory.

Looking around the web I’ve not seen any other clear mention of this issue, but I’m hoping someone else may have run into the same issue. Figured I’d start here before putting in a bug report. Any advice would be much appreciated.

Thanks
Matt

Error from the home-assistant.log during shutdown

2024-06-01 20:26:18.187 WARNING (Thread-13 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_9] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker
    work_item.run()
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/influxdb/__init__.py", line 503, in shutdown
    influx.close()
  File "/usr/src/homeassistant/homeassistant/components/influxdb/__init__.py", line 386, in close_v2
    influx.close()
  File "/usr/local/lib/python3.12/site-packages/influxdb_client/client/influxdb_client.py", line 349, in close
    self.__del__()
  File "/usr/local/lib/python3.12/site-packages/influxdb_client/client/influxdb_client.py", line 354, in __del__
    self.api_client.__del__()
  File "/usr/local/lib/python3.12/site-packages/influxdb_client/api_client.py", line 86, in __del__
    self._pool.join()
  File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 665, in join
    self._worker_handler.join()
  File "/usr/local/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):