Too many open files with Unifi device tracker and new LIFX lights components

[mihalski (Michal Dobrzynski)]

I’ve been fighting a major recent problem I’ve been having with HA that results in “too many open files” errors on a MASSIVE scale… Still working to figure it out, but this seems relevant…

pi@homepi:~ $ sudo lsof | wc -l
21795
pi@homepi:~ $ sudo lsof |grep homeassistant |wc -l
15445
pi@homepi:~ $ sudo lsof |grep homeassistant |grep -i lifx |wc -l
4998
pi@homepi:~ $ sudo lsof |grep homeassistant |grep -i ubnt |wc -l
8942
pi@homepi:~ $ sudo lsof |grep homeassistant |grep -vi ubnt | grep -vi lifx |wc -l
1537

Here’s a graph of “sudo lsof | wc -l” and “sudo -u homeassistant | wc -l” :

The large cliffs are restarts of HA. Is anyone else experiencing this with either of these components? It makes using HA almost impossible without frequent restarts.

[gpbenton (Graham)]

According to this post, WebOS TV has a similar problem leaking files

Components that can break whole setup

Had problems configuring HA for some time. All of a sudden either postgres or mysql (tried both) would stop logging and break up with no warnings in the log. Sometimes an error like connection timed out was reported. My HA couldn’t run for more than 3 days in a row without restarting. Freezes or breaking out of other components (nmap, ping and others were failing with one error or another). Than all of a sudden i have seen a post on github HA issues about Web OS Tv opening too many connections…

[mihalski (Michal Dobrzynski)]

Both of these components were not having this issue in their previous incarnation… Chasing up the Lifx one first… Will see about the Unifi component next.

[mihalski (Michal Dobrzynski)]

If this is helpful to anyone here is my lsof log at the time when there are already too many open files.

lsof.log.gz

[mihalski (Michal Dobrzynski)]

So other people are happy to tolerate a home automation system that can only run for a maximum of about 20 hours at a time? Nobody else is bothered? Or affected?

[mihalski (Michal Dobrzynski)]

Could this be related to the PyUnifi2.0 update one month ago? @finish06

[Vertikar]

Think I’m running into the same issue and I’m guessing its caused by lifx:

(hass) hass@home-pvr:~/.homeassistant$ lsof | grep 56700 | wc -l
12870

[cbulock (Cameron Bulock)]

I hadn’t noticed any errors from this, but I ran across this thread and was curious myself

Looking at the open files, I visually noticed that a lot of them were TCP connections to Chromecast devices. They all have chrome in the hostname so I was able to filter them out.

pi@ha:~ $ lsof | grep 2595 | wc -l
13838
pi@ha:~ $ lsof | grep 2595 | grep chrome | wc -l
4810

Seems excessive

[mihalski (Michal Dobrzynski)]

If you use the lifx component, download the latest version, rename it as lifxtest.py custom component in the appropriate directory (i.e. .homeassistant/custom_components/light/lifxtest.py) and then add:

light:
  - platform: lifxtest

to your configuration.yaml

This not only fixed the lifx component for me, but also fixed the totally unrelated Unifi device tracker which was leaking files twice as badly. don’t ask me why as I have NO idea.

[Vertikar]

Looks like it might be fixed in 0.42.3 anyway… https://github.com/home-assistant/home-assistant/issues/6985