0.36 major performance issues

I upgraded to 0.36 last night, and as soon as I did I’ve been noticing major performance issues – motion/light triggers take upwards of 10+ seconds to activate, when manually triggering a scene it takes up to 20 seconds for all the lights to turn on (and they “trickle” on one by one), or they just never trigger at all.

My setup mostly consists of MQTT “lights” which are actually z-wave switches proxied through a SmartThings hub. Trigger the lights from SmartThings results in instantaneous response. Watching MQTT messages on my network through MQTTlens definitely suggests the lag is happening in Home Assistant.

Checking home-assistant.log (I’m on a Raspberry Pi 3 with the AIO installer), I see this:

OSError: [Errno 24] Too many open files
17-01-16 15:15:40 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:41 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:41 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:42 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:43 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:43 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:44 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:45 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:45 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:46 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files
17-01-16 15:15:46 homeassistant.core: Error doing job: socket.accept() out of system resource
Traceback (most recent call last):
  File "/usr/lib/python3.4/asyncio/selector_events.py", line 137, in _accept_connection
    conn, addr = sock.accept()
  File "/usr/lib/python3.4/socket.py", line 184, in accept
    fd, addr = self._accept()
OSError: [Errno 24] Too many open files

top shows the hass and homebridge processes consistently eating most of the CPU cycles:

13754 hass      20   0  201656  60044   8260 S  40.4  6.3   1:55.12 hass                                                                                                                                          
31506 hass      20   0  169604  47772   7976 S  23.2  5.0 119:47.59 homebridge                                                                                                                                    
  132 root      20   0   10296   4280   3940 S  22.2  0.5 125:55.62 systemd-journal                                                                                                                               
  458 root      20   0   32144   2608    848 S   6.0  0.3  36:58.88 rsyslogd                                                                                                                                      
   19 root      20   0       0      0      0 S   0.3  0.0   0:33.62 ksoftirqd/3                                                                                                                                   
  475 mosquit+  20   0    5564   1844   1520 S   0.3  0.2  22:03.35 mosquitto                                                                                                                                     
13820 pi        20   0    5304   2520   2020 R   0.3  0.3   0:00.15 top                                                                                                                                           
    1 root      20   0   22864   2488   1600 S   0.0  0.3   1:12.93 systemd    

Not sure what to do at this point. Any suggestions?

“Too many open files” seems to be the issue. Try to raise it with ulimit for your hass user. I guess it will resolve the problem.
Cheers Hannes

Thanks. I think the size of my log file (over 1.6GB) wasn’t helping. I’ve transitioned event logging to MySQL and things seem to be running much better now.

This thread is tracking this issue too: