Is my SD card full on my raspberry pi?

revz · August 9, 2017, 11:02pm

what kind / size do you recommend? and why is that? better read / write capability?

zarthan · August 9, 2017, 11:04pm

The forums are having trouble. The person looking after the forums is on vacation so inexperienced folks are chipping in.
Just look to see if you have a /etc/logrotate.d/rsyslog

ls /etc/logrotate.d
If you have rsyslog in that folder
nano /etc/logrotate.d/rsyslog
If you have syslog
nano /etc/logrotate.d/syslog
If you don’t have either of those you can likely just edit /etc/logrotate.conf

My /etc/logrotate.d/rsyslog

/var/log/syslog
{
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript

You might want to consider changing the lower list from
rotate 4
weekly
to
rotate 2
daily

firstof9 · August 9, 2017, 11:06pm

Tons of read/write to the disk and a couple power blips and I was out 2 SDcards, so I switched to a USB drive to avoid the complications in the future.

revz · August 9, 2017, 11:06pm

very helpful, thank you

zarthan · August 9, 2017, 11:17pm

I have thought about making changes to my Pi installs to write logfiles to a ramdisk. The home-assistant.log and the OZW log do add up as well.

revz · August 9, 2017, 11:25pm

People also export data to cloud services or external servers right, using mysql or equivalent?

So, what brought this issue to my attention in the first place (i think) was that my config file shat the bed

along with history, recorder, and logbook, mqtt stopped working too. Is this related to the full SD card issue, and should I ask you guys questions related to that, or should I open a new thread about my configuration.yaml file no longer working?

thx again, -revz

firstof9 · August 9, 2017, 11:31pm

Does df -h still show / as 100% usage?

zarthan · August 9, 2017, 11:32pm

I would stop HA then delete the home-assistant_v2.db file It would likely be corrupt. It will be recreated when you restart HA. I don’t use mqtt so I don’t know what files it might create to store information, but they will likely be corrupt. Running out of disk space is a bitch.

revz · August 9, 2017, 11:40pm

ok, will check back in a bit

revz · August 10, 2017, 1:27am

after deleting syslog.1 roughly half my SD card is free, the majority of which is occupied by daemon.log.

pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G   16G   13G  56% /
devtmpfs        427M     0  427M   0% /dev
tmpfs           432M     0  432M   0% /dev/shm
tmpfs           432M  5.9M  426M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           432M     0  432M   0% /sys/fs/cgroup
/dev/mmcblk0p1   63M   21M   43M  33% /boot
pi@raspberrypi:~ $ df -i
Filesystem      Inodes IUsed   IFree IUse% Mounted on
/dev/root      1870848 50105 1820743    3% /
devtmpfs        109264   354  108910    1% /dev
tmpfs           110347     1  110346    1% /dev/shm
tmpfs           110347   438  109909    1% /run
tmpfs           110347     4  110343    1% /run/lock
tmpfs           110347    10  110337    1% /sys/fs/cgroup
/dev/mmcblk0p1       0     0       0     - /boot
pi@raspberrypi:~ $

@zarthan Stopped home assistant, deleted home-assistant_v2.db, and restarted. MQTT still not working, everything else seems to be though. Here’s the session log:

The following errors have been logged this session:

2017-08-10 01:43:26 ERROR (MainThread) [homeassistant.setup] Error during setup of component mqtt
Traceback (most recent call last):
  File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/setup.py", line 185, in _async_setup_component
	result = yield from component.async_setup(hass, processed_config)
  File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/mqtt/__init__.py", line 380, in async_setup
	success = yield from hass.data[DATA_MQTT].async_connect()
  File "/srv/homeassistant/homeassistant_venv/lib/python3.4/site-packages/homeassistant/components/mqtt/__init__.py", line 490, in async_connect
	None, self._mqttc.connect, self.broker, self.port, self.keepalive)
  File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__
	yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup
	value = future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
	raise self._exception
  File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run
	result = self.fn(*self.args, **self.kwargs)
  File "/home/pi/.homeassistant/deps/paho/mqtt/client.py", line 700, in connect
	return self.reconnect()
  File "/home/pi/.homeassistant/deps/paho/mqtt/client.py", line 822, in reconnect
	sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
  File "/usr/lib/python3.4/socket.py", line 509, in create_connection
	raise err
  File "/usr/lib/python3.4/socket.py", line 500, in create_connection
	sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

firstof9 · August 10, 2017, 1:52am

Issue the command: sudo service mosquitto start it should start up but if it doesn’t, check /var/log/syslog for the reason why

revz · August 10, 2017, 2:04am

restarting mosquitto did not throw an error, but did not solve the problem. here are last lines of syslog file:

Aug 10 01:54:24 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:24 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:24 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:24 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:24 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:28 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:30 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:30 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, new_state=<state sun.sun=below_horizon; elevation=-18.98, friendly_name=S$
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:36 raspberrypi avahi-daemon[414]: Received packet from invalid interface.
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.components.discovery] Found new service: philips_hue {'host': '192.168.1.30', 'model_name': 'Philips hue bridge 2015', 'model_number': '$
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.loader] Loaded light from homeassistant.components.light#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.setup] Setting up light#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=turn_on, domain=light>#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=turn_off, domain=light>#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=toggle, domain=light>#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=light>#033[0m
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.core] Bus:Handling <Event platform_discovered[L]: service=load_platform.light, discovered=host=192.168.1.30, model_name=Philips hue brid$
Aug 10 01:54:46 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:46 INFO (MainThread) [homeassistant.loader] Loaded light.hue from homeassistant.components.light.hue#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (MainThread) [homeassistant.components.light] Setting up light.hue#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (Thread-11) [phue] Attempting to connect to the bridge...#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (Thread-11) [phue] Using ip: 192.168.1.30#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (Thread-11) [phue] Using username from config: 7w7JvfNxKDHmjxUdX5Y-5EAnk-nzxRrCuhNJxyPC#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=hue_activate_scene, domain=light>#033[0m
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.kitchen_nook, new_state=<state light.kitchen_nook=on; effect_list=['colorloo$
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.master_bath, new_state=<state light.master_bath=off; supported_features=127,$
Aug 10 01:54:47 raspberrypi hass[1060]: #033[32m2017-08-10 01:54:47 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, new_state=<state group.all_lights=on; friendly_name=all lights, $
Aug 10 01:55:30 raspberrypi hass[1060]: #033[32m2017-08-10 01:55:30 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, new_state=<state sun.sun=below_horizon; elevation=-19.12, friendly_name=S$
Aug 10 01:55:30 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Thu Aug 10 01:57:00 2017 [try http://www.rsyslog.com/e/2007 ]
Aug 10 01:56:30 raspberrypi hass[1060]: #033[32m2017-08-10 01:56:30 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, new_state=<state sun.sun=below_horizon; elevation=-19.26, friendly_name=S$

edit However I just reinstalled mosquitto and it seems to be back up and running:

pi@raspberrypi:~ $ sudo systemctl status mosquitto.service
● mosquitto.service - Mosquitto MQTT Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled)
   Active: active (running) since Thu 2017-08-10 12:21:45 UTC; 7min ago
	 Docs: man:mosquitto(8)
		   https://mosquitto.org/
 Main PID: 397 (mosquitto)
   CGroup: /system.slice/mosquitto.service
		   └─397 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Aug 10 12:21:45 raspberrypi systemd[1]: Started Mosquitto MQTT Broker.
pi@raspberrypi:~ $

meanwhile my database has grown by 1GB overnight…

pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G   15G   13G  54% /
devtmpfs        427M     0  427M   0% /dev
tmpfs           432M     0  432M   0% /dev/shm
tmpfs           432M   12M  420M   3% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           432M     0  432M   0% /sys/fs/cgroup
/dev/mmcblk0p1   63M   21M   43M  33% /boot

firstof9 · August 10, 2017, 12:43pm

This is part of why I go the USB thumb drive route, constant r/w to the disk. I also run a MySQL server on a separate machine for the database.

gpbenton · August 10, 2017, 4:22pm

The db does grow quickly, but this seems excessive. Mine is purged after 3days, but it is currently sitting at 192MB.

What components are you using?

revz · August 10, 2017, 6:48pm

Not sure how to get HA to list all components, but, in a nutshell:

Lutron Caseta (about 8 dimmers), Phillips Hue (LED strip and a light), 3 or 4 DIY multisensors, Netatmo (which is currently not loading), and a DIY addressable LED strip.

heres a screenshot of front end (w/o the Netatmo):

Happy to post .yaml file too if nec

firstof9 · August 10, 2017, 7:54pm

If you don’t need the history of specific items you can exclude them.

example:

recorder:
  include:
    domains:
      - sensor
      - switch

logbook:
  include:
    domains:
      - sensor
      - switch

history:
  include:
    domains:
      - sensor
      - switch

revz · August 10, 2017, 8:12pm

Ah, ok, this is in configuration.yaml?

firstof9 · August 10, 2017, 8:26pm

Correct it is

revz · August 16, 2017, 4:23pm

So, it occurs to me that I have some learning to do with regard to using Linux, otherwise I wouldn’t be going around asking such questions as ‘Is my SD card full’. I really did’t even know what it was before using HA. Now I realize there is a lot there to learn! Its very cool and I am excited to do so. In In the meantime though, the SD card is full again:

Last login: Mon Aug 14 12:00:14 2017 from 192.168.1.8
pi@raspberrypi:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        29G   29G     0 100% /
devtmpfs        427M     0  427M   0% /dev
tmpfs           432M     0  432M   0% /dev/shm
tmpfs           432M  5.8M  426M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           432M     0  432M   0% /sys/fs/cgroup
/dev/mmcblk0p1   63M   21M   43M  33% /boot

Even though it represents a fair amount of labor, I am considering a complete reinstall. But I am curious why this is happening. Can anyone tell me a basic way to diagnose what is going on?

Tinkerer · August 16, 2017, 4:33pm

You need to find out what the large files are. Likely it’s either the HA database, again, or log files.

A quick check for large files would be:

sudo du -cshx /*
sudo find / -type f -size +1G -ls

The first will display the sizes of all the directories under /, showing you where the space is being used. The second will display all the files that are above 1GB in size. Both may take some time to run.