Yep, the Recorder parameters changed. I will update the doc to not confuse the peeps. Thanks…
Regarding your previous comment about disabling swap altogether, I don’t have enough knowledge to say something meaningful about it. Some say swap space is a good thing, others say no. Actually the same guy says no in another article. My take on this is to play it safe - limit the server’s affinity to use swap space (swappiness), like actually don’t use it, but to have swap space available as backdoor when its needed. But this is case specific, a guy running a 8GB RPi 4 with lots of memory available, may think different about it than a 1GB guy.
Would be great if some swap guru could chime in…
Actually I referred to a recommendation to disable swap only if zram is used, not in general. After a lot of research I’ve also come to the conclusion that Linux needs a kind of swap, even if it is ultimately not used.
I would also add regarding Database, that the best solution is to put it in memory all together (after the suggested optimizations in Recorder). And use InfluxDB for long-term data in case there is a system shutdown/restart etc.
Further on, in point C.4. the correct syntax for WPA log level is sudo wpa_cli log_level error, not sudo wpa_cli log _level=ERROR . The latter did not work for me.
I think, if log2ram is used (which I installed and seems great!), points D. and E. are redundant in my mind if there is enough RAM in the system. Since all logs of var/log are written to RAM with log2ram, there is little to worry about/tinker. Of course I’ll check the memory more often from now on.
I haven’t installed ZRAM too, since I see that there is a lot of headroom regarding memory. I have the Pi4 4GB version and only 1GB is used for now. I’ll see how things go on from now and react accordingly.
Again, many congratulations for the recommendations!
Is there a chance to store the data on a NAS in LAN ?
That is running 247 or on an external USB drive where size does not matter ?
I am asking cause relying on Pi OS and Docker Image I want to have small SD Cards of which I create IMGs that can easily be restored. But the data monster are grabbing 50 MB or more each day and I do not want to go for a 32 GB SD Card which takes ages to backup and the same amount of time to copy back to a PC and restore that image to a SD.
I had a similar experience with my first SSD, a cheapie from KingSpec. It lasted me a full three months before it crapped out on me. Advice here is to avoid it as the pest. I’m currently using a Kingston A400 (with a UGREEN housing) and it’s running reliably for over a year now.
If you’re using a Raspberry Pi 4, for SSD’s and adapters that work with the RPi 4 see this guide by James Chambers.
I am currently looking into all your recommendations and got thinking about the recorder. I opened a feature request in case someone is interested in discussing pros and cons of a split database (remote + local).
I’m curious: has anybody checked the impact of the MQTT Broker? Even when you filter the messages and only track and store a defined amount of sensors, are not all incoming payloads processed and stored because most have a retain flag (not in the database, but maybe in a temp file or cache)?
Hello everyone, I would like to add some sensors that perform very frequent readings related to the use of the cpu, ram, disk occupation, etc. If I exclude them from Recorder, History and Logbook is it sufficient or is there a risk that they compromise the life of the SSD anyway?
I’m not sure if any of the developers will be interested but I have logged a feature request here to add some options to HA that might reduce disk writes/extend SSD life expectancy. I would welcome any inputs from you guys there. I’m visualizing some sort of array of checkboxes that might individually contribute to reduction in disk writes, each of which the user could turn on / off as they wish, accepting whatever caveats may come with each option.
Especially for those of using HAOS which makes it that bit more difficult to manually make the excellent suggestions on this page.
This might be of your all interest, as this post has inspired me to get a step further down the road using anything sync daemon in order to prevent almost all IO to the disk: Wiki: reducing IO in home assistant (supervised)
I edited the syslog instructions in order to clean up the syslog, in order to get mainly warnings and errors:
:msg, startswith, " s6-rc: info: " stop
if ( $programname startswith "addon_" or $programname startswith "hassio_" ) and ( $msg contains "[info]" or $msg contains " info: " or $msg contains " INFO: " or $msg contains " [INFO]" or $msg contains " INFO " or $msg startswith " #033[34m" ) then { stop }
if $programname == "hassio_audio" and ( not ( $msg contains " E: " ) or ( $msg contains " E: [alsa-sink-Generic Analog] alsa-sink.c: " ) ) then { stop }
if $programname == "addon_core_mosquitto" and ( $msg contains "New connection from " or $msg contains " closed its connection." ) then { stop }
if $programname == "NetworkManager" and ( $msg contains "<info>" ) then { stop }
if $programname == "hassio_observer" and ( $msg contains " done." or $msg contains "exited 0." ) then { stop }
if $programname == "kernel" and ( $msg contains "hassio: port " or $msg contains "device veth" ) and $msg contains " entered " then { stop }
That’s fantastic. Do you think it would take much to have this as an official feature in the likes of HAOS, that could be toggled maybe in some advanced/system menu or even via the configuration.yaml?
It feels so close and could help all of us HAOS users if it was somehow added there as a supported option to reduce disk writes.
Essentially it’s really peace of cake to do so, but apparently reducing IO in order to lower the wear Home Assistant is imposing on hardware isn’t really a top priority.
Actually I think all journaling services even could be disabled. Since as far I can see, they aren’t accessible using HA interfaces like the GUI or HA CLI. See for example: https://community.home-assistant.io/t/wth-is-there-no-home-assistant-logs-to-remote-syslog-integration/473949/10. Besides all logging is being done at least twice, because the log files in add-on overview haven’t changed, even after applying these rules (and no logging to syslog).
However, I think keeping things as long as reasonably possible in memory is quite more interesting as it really pays more of > logging is really small potatoes compared to the IO-traffic of the database for example, besides also for logging counts that one sequential write imposes a lot less wear on the hardware than many random writes.
I am surprised that no one suggested using a micro-SD card with wear-leveling, and use the largest SD you can. So if your program only needs 8GB and you use a 128GB Wear-leveling SD card, then you have extended your SD card’s life by 16X.
FWIW, I have a Raspberry Pi that is my MQTT broker has been running 24/7 for the past four years without a hiccup.
Yeah, might be an option for sure. Although, if you read my other thread, you would have seen that Home Assistant is writing about 60GB/day to my SSD which I reduced to 40GB with exclusion of entities from the logger I’m not interested in in the long run. This with a database of about 1GB at the moment, means that a lot of the traffic is being caused by the rewriting / remoddeling of the database.
This amount of IO traffic is also gonna wear out a wear-leveling SD-card. As for my 4 months old SSD I already have a TBW of 7.2TB.
For an 128GB SSD -which have wear leveling by default- the guaranteed TBW is in the range 30-120TB, which would mean I would have expected to see failure within one year up till 5 years from now. For an micro-SD card, these expected TBW’s are usually much less (except for better brands like SanDisk Max Endurance or Samsung Pro Endurance; which have 400-800TBW stated).
Awesome guide! To clarify, if I want to use recorder: {db_url: 'sqlite:///:memory:'} do I need to create an in-memory SQLite database first or will one be created automatically?
It should work automatically, there is nothing to do – “in memory” here does not mean something like a RAM-disk, but literally within the memory of your HA instance. With the documented effect that you lose everything on a restart. See In-Memory Databases for more technical details.
I will update the doc to not confuse the peeps. Thanks…