Sqlite3 data base could not be validated

I have just a about eliminated all the errors I was getting from the cameras with exception of one camera and I think I might be onto that as well.
The solution I have come up with to meet my needs was.

  1. Delete all the HIKVision cameras from the ONVIF Integration. This also deletes all related entities.

  2. Install the HIKVision Integration and install my x6 HIKVision cameras here. The HIKVision integration was needed to get all the logical line triggers I had in these cameras. Some of these triggers were used to turn on lights after sunset when triggered.

  3. I still needed ONVIF as I had one Sonoff camera that was only supported on ONVIF.

  4. The HIKVision Cameras needed a special user with operator privileges and special setup in the Advanced Network integration protocol. The CGI Authentication with digest/basic.

Anyway that seems to have got rid of all those streaming errors FOR NOW!!
Database size

You seem to have got a good grip on how your entities affect the database. Another week or so and it should stabilise in size as the housekeeping tasks do their thing, and the error log entries get purged.

Try a different special character such as a underscore or hyphen that wonโ€™t choke parameter passing/parsing, but still satisfies the over-strict password criteria.

Is your system stable now? No crashes?

Make sure you have done the latest Spook 5.2.0 update, as it has fixed a couple of errors, including one that we attempted to use to purge orphan entities in the database that actually didnโ€™t do anything. It should be working now, and may make a big difference if you repeat some of the earlier cleanup activities after you have updated it and restarted your system. Let me know if the pesky database validation errors in the startup process are still there in the system log once it is back up again.

Check all the entities each camera has created and disable the ones not needed so it doesnโ€™t fill up the database. Purge the history of those you have no need to store. The automations in the latest version of Spook allow you to select which entities to target in the Tools/Actions screen, so you may not need to do much typing or cut-n-pasting, just select from dropdown lists.

Keep us posted on progress.

System is stable with โ€œuptimeโ€ of 188hrs or about 8 days. I have deliberately refrained from doing any updates or anything that would require a restart and monitored the logs. I still have a few oddities to chase down but again itโ€™s camera based and not super important.

I did see the Spook update and will do it this week as being a HACS it will require a restart.

Camera configuration is strange. I am now using an identical configurations on all cameras and 5 are giving no errors but 1 is consistently giving a 401 error on profile 2. I have all the latest firmware.
I have tried many things to remedy this one but itโ€™s stubborn. The only thing I have not tried is a restart.
Anyway the system is stable and not crashing so thatโ€™s a blessing.
I will keep you posted on things.

Another version update of Spook hit about an hour ago. Maybe wait till the dust settles before updating.

RESTART>>>

After 8 days HA restarted itself. This is from the supervisors log.
ERROR 134 what ever that is

2026-08-31 18:00:19.274 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/a0d7b954 repository from https://github.com/hassio-addons/repository
2026-08-31 18:00:19.276 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/5c53de3b repository from https://github.com/esphome/home-assistant-addon
2026-08-31 18:00:19.277 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/d5369777 repository from https://github.com/music-assistant/home-assistant-addon
2026-08-31 18:00:19.287 INFO (MainThread) [supervisor.store.git] Update app /data/apps/core repository from https://github.com/home-assistant/addons
2026-08-31 18:00:19.302 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/ba124dcc repository from https://github.com/lmagyar/homeassistant-addon-samba-interface
2026-08-31 18:28:39.679 WARNING (MainThread) [supervisor.homeassistant.websocket] Connection is closed
2026-08-31 18:28:43.008 WARNING (MainThread) [supervisor.homeassistant.core] Watchdog found Home Assistant exited with code 134, restarting...
2026-08-31 18:28:43.024 INFO (MainThread) [supervisor.docker.manager] Starting homeassistant
2026-08-31 18:28:43.162 INFO (MainThread) [supervisor.homeassistant.core] Wait until Home Assistant is ready
2026-08-31 18:28:47.494 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state running
2026-08-31 18:28:47.507 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete

That was at 18:28 last night. There have been no camera errors in core log since restart so thatโ€™s a plus. However I am none the wiser. SAMBA was the last thing closed??

It fell over again about the same timeโ€ฆ

2026-09-04 18:04:06.028 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/a0d7b954 repository from https://github.com/hassio-addons/repository
2026-09-04 18:04:06.029 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/5c53de3b repository from https://github.com/esphome/home-assistant-addon
2026-09-04 18:04:06.032 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/ba124dcc repository from https://github.com/lmagyar/homeassistant-addon-samba-interface
2026-09-04 18:04:06.036 INFO (MainThread) [supervisor.store.git] Update app /data/apps/core repository from https://github.com/home-assistant/addons
2026-09-04 18:04:06.047 INFO (MainThread) [supervisor.store.git] Update app /data/apps/git/d5369777 repository from https://github.com/music-assistant/home-assistant-addon
2026-09-04 18:47:21.352 WARNING (MainThread) [supervisor.homeassistant.core] Watchdog found Home Assistant exited with code 1, restarting...
2026-09-04 18:47:21.365 INFO (MainThread) [supervisor.docker.manager] Starting homeassistant

No real explanation but same music assistant add on. I donโ€™t use music assistant as far as I know?

This morning I loaded the latest update 2026.9.1 and latest Spook. I fixed spook repairs.
Did a purge and repack.
This is DB size graph

Have you installed the latest two September core updates? Not hit the delete everything button that had unintended consequences people didnโ€™t anticipate? Grafana still there?

Youโ€™ve cleaned up pretty much everything in the current recorder database that could be purged, and the long term statistics arenโ€™t there for any length of time as you started with a new database only recently. There are other issues that are currently being looked at.

Step 22: For curiosity to get file sizes, go to the CLI (see step 9) and tell us what running this command does.

ls -lh /config/.storage/core.device_registry /config/.storage/core.entity_registry /config/home-assistant_v2.db*

You may as well grab the whole screen, including any errors showing above your prompt, that may offer clues. Donโ€™t forget to post these with forum </> formatting for readability please.

Note: The first line is the device registry size, the second the entity registry size, the third the main SQLite database size, the fourth is the .db-wal (Write-Ahead Log) and fifth is the .db-shm (SHared Memory) files.

The last two grow and ebb as the database checkpoint is written to, and should be automatically purged when it is closed properly during maintenance and system shutdown procedures, and the dirty bit flag is reset. Getting a file size snapshot just before and after a successful database maintenance operation should prove interesting.

When your system is starting, the presence and contents of the wal and shm checkpoint files indicate something went wrong before, and the database integrity is suspect. Too much activity, especially on a resource constrained system (CPU, available memory, drive speed, disk space, cache resources, paging), can make these checkpoint files bloom excessively and consume added resources trying to write to the database, often exhausting them and things crashing. Manually deleting these checkpoint files will result in probable database corruption, as they are required for rollback to repair the database - donโ€™t do it, or close down your database and delete all three if starting from fresh (the system will notice and start a new one).

The probable underlying problem in your case, too many inbound entries, is the one you have recently addressed by two methods: pruning which entities the recorder queues for writing, and the recent fix to the go2rtc problem not generating excessive errors any more. There may be more issues, and this seems to have been highlighted by recent events that are still being worked on.

While we have the CLI screen open from step 22, we may want to do step 10 again at the same time and report on the current top 30 recorder database statistics too. This will give us an updated picture of what is current to compare against the past.

sqlite3 /config/home-assistant_v2.db \ "SELECT sm.entity_id, COUNT(*) AS n FROM states s JOIN states_meta sm ON s.metadata_id = sm.metadata_id GROUP BY sm.entity_id ORDER BY n DESC LIMIT 30;"`

Post those results here too.

Some interesting background reading that may be relevant to the above two steps:

Purge unreachable deleted devices on device registry load by emontnemery ยท Pull Request #181207 ยท home-assistant/core ยท GitHub
Core becomes unresponsive and Supervisor watchdog restart takes about four minutes, without OOM or high memory usage ยท Issue #181318 ยท home-assistant/core ยท GitHub << analysis stage
2026.9 Update Makes system unstable - Large Memory usage spike ยท Issue #181120 ยท home-assistant/core ยท GitHub
Trim cached orjson fragments kept by registries by emontnemery ยท Pull Request #181240 ยท home-assistant/core ยท GitHub << pending
Speed up lookup of colliding orphans in device registry by arturpragacz ยท Pull Request #181244 ยท home-assistant/core ยท GitHub
Revert orjson to 3.11.9 by emontnemery ยท Pull Request #181245 ยท home-assistant/core ยท GitHub << rollback of the underlying memory spike issue
Device registry deleted_devices grows unbounded with orphaned entries (no automatic cleanup) ยท Issue #181347 ยท home-assistant/core ยท GitHub

Patiently waiting while the underlying issue is determined and fixed may be the path forward at this stage.

Post those two screens from the last post and this one in the meantime so we can check if anything else is glaringly obvious.

Yes I have installed the latest September updates

ls -lh /config/.storage/core.device_registry /config/.storage/core.entity_registry /config/home-assistant_v2.db*
-rw-r--r--    1 root     root      134.5K Sep  6 11:26 /config/.storage/core.device_registry
-rw-r--r--    1 root     root        4.9M Sep  6 19:12 /config/.storage/core.entity_registry
-rw-r--r--    1 root     root      644.1M Sep  7 06:03 /config/home-assistant_v2.db
-rw-r--r--    1 root     root        1.1M Sep  7 06:03 /config/home-assistant_v2.db-shm
-rw-r--r--    1 root     root        5.2M Sep  7 06:03 /config/home-assistant_v2.db-wal

Sqlite3 command not found again
Loaded sqlite
Got this

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚                      entity_id                      โ”‚   n   โ”‚
โ•žโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ชโ•โ•โ•โ•โ•โ•โ•โ•ก
โ”‚ sensor.sonoff_10012393db_power                      โ”‚ 87710 โ”‚
โ”‚ sensor.sonoff_1001361f45_power                      โ”‚ 87054 โ”‚
โ”‚ sensor.sonoff_1000cd7a20_power                      โ”‚ 84639 โ”‚
โ”‚ sensor.sonoff_1001361b68_power                      โ”‚ 82587 โ”‚
โ”‚ sensor.sonoff_100123ae52_power                      โ”‚ 81383 โ”‚
โ”‚ sensor.sonoff_10012393db_voltage                    โ”‚ 76017 โ”‚
โ”‚ sensor.sonoff_1001361b68_voltage                    โ”‚ 74817 โ”‚
โ”‚ sensor.sonoff_1001361f45_voltage                    โ”‚ 73610 โ”‚
โ”‚ sensor.sonoff_100123ae52_voltage                    โ”‚ 72001 โ”‚
โ”‚ sensor.sonoff_1000cd7a20_voltage                    โ”‚ 71688 โ”‚
โ”‚ sensor.sonoff_10012393d2_voltage                    โ”‚ 68508 โ”‚
โ”‚ sensor.importing                                    โ”‚ 61551 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_1_power        โ”‚ 57798 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_2_power        โ”‚ 57536 โ”‚
โ”‚ sensor.solarpower                                   โ”‚ 57533 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_1_voltage      โ”‚ 56303 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_2_voltage      โ”‚ 56303 โ”‚
โ”‚ sensor.house_total_energy                           โ”‚ 55160 โ”‚
โ”‚ sensor.house_power_consumption                      โ”‚ 55153 โ”‚
โ”‚ sensor.house_daily_energy                           โ”‚ 55109 โ”‚
โ”‚ sensor.ergon_grid_import                            โ”‚ 55102 โ”‚
โ”‚ sensor.sonoff_1001361f45_current                    โ”‚ 44198 โ”‚
โ”‚ sensor.tz3210_cehuw1lw_ts011f_current_2             โ”‚ 41751 โ”‚
โ”‚ sensor.sonoff_10012393db_current                    โ”‚ 41599 โ”‚
โ”‚ sensor.exporting                                    โ”‚ 39958 โ”‚
โ”‚ sensor.sonoff_1001361b68_current                    โ”‚ 35164 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_1_power_factor โ”‚ 32284 โ”‚
โ”‚ sensor.sonoff_100123ae52_current                    โ”‚ 31454 โ”‚
โ”‚ sensor.shed_cons_power                              โ”‚ 30252 โ”‚
โ”‚ sensor.shed_cons_chl_2_voltage                      โ”‚ 30037 โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

My system at the moment

Installation method Home Assistant OS
Core 2026.9.1
Supervisor 2026.08.0
Operating System 18.2
Frontend 20260826.6

Need to go for medical issues

Well after spending the day in the ED I am back.
Yes interesting reading and it makes me pose the question " Where and when did the Wheels fall off??" Once moved from a RPi to a NUC with lots of RAM ,and 256GB NVMe SSD, my HA was very stable and never fell over. My observation is that it was not until the focus shifted to this new world of voice and AI that things started to get complicated and QC was not as good..

Anyway I shall be persistent as the ability to control my lights, fans, A/C and so on from my phone is absolutely great for me with my wonky legs disability.

I hope you had the best of care and (just to annoy the 'Muricans), free! No payment, and usually not even sign anything on the way out. Just busy, polite professionals doing their best.

Iโ€™m looking at your sensors you are storing data for.

โ”‚ sensor.sonoff_10012393db_voltage                    โ”‚ 76017 โ”‚
โ”‚ sensor.sonoff_1001361b68_voltage                    โ”‚ 74817 โ”‚
โ”‚ sensor.sonoff_1001361f45_voltage                    โ”‚ 73610 โ”‚
โ”‚ sensor.sonoff_100123ae52_voltage                    โ”‚ 72001 โ”‚
โ”‚ sensor.sonoff_1000cd7a20_voltage                    โ”‚ 71688 โ”‚
โ”‚ sensor.sonoff_10012393d2_voltage                    โ”‚ 68508 โ”‚
--
โ”‚ sensor.shellyem_34945470fae8_channel_1_voltage      โ”‚ 56303 โ”‚
โ”‚ sensor.shellyem_34945470fae8_channel_2_voltage      โ”‚ 56303 โ”‚

Pull up graphs of these over an hour, day, week, year and see by how much they vary.

Does your voltage vary at each sensor point on each sonoff? By much? The same amount always? Are they in sync, following each other up and down? Once you confirm it is close to normal 230/240/250v (depending on your location in respect of the power station and street wiring which you have little control over, and local load, which you do), is it important to keep those statistics longer term? Unless you are having issues you need to discuss with your power company, maybe these donโ€™t need to be stored, just monitored for normality, and a possible trigger alerts if they fall outside the nominal Australian standard +/-5% range, maybe triggering above 265 volts (surge) and below 210 (brown-out)? Yes you have to collect voltage to calculate wattage/power from multiplying current and voltage, but do you need to store it longer term?

Maybe store the ones from the Shelly EM (which should closely match all the other ones). Yes, keep the power consumed, measure in current (amps), wattage (volts x amps), consumption, energy, but you could probably cull those voltage sensors from being stored long term, and prune even more from your database.

Your previous reference to MusicAssistant as well as voice input has piqued my interest. Any add-ons, apps, or voice assisted things that rely on that installed?

When an individual brain cannot store and recall every single small detail, you need assistance to be able to manage a project effectively. Things have become so complicated and interwoven that even teams of individuals that used to be thrown at a complex issue is not good enough any more. HomeAssistant has grown to be a collaboration of many individuals, Open Source notwithstanding, and some of the add-ons, integrations and apps are done by persons of varying skills and ability, increasingly vibe coded, all coexisting and expected to work together in an orderly way. By the very relentless pace of developments in the computer industry, it has to weave and change to keep up, add functionality, and accommodate vendor changes, some done for improvement, and sometimes for spite, the endless pursuit of money being the dominant driver, and the noble altruistic efforts of some in the Open Source sector being the balancing counterpart.

The ability of AI is increasingly to be able to digest enormous amounts of information and find patterns, inconsistencies, analyse things that are the same and different, and with correct prompting from a skilled individual that still knows what the ultimate goal is, recognising and redirecting AI slop and hallucinations, to provide similar consistent error free code that was previously done for less complex projects. Like a wheelchair or walking frame as you age, the tools are to assist you, to achieve the same goals as before, to get to the pub before closing time, and get home again later!

Voice input and discussion has been the Holy Grail for computing for decades. The grunt behind the scenes to be able to do this unobtrusively, naturally, is not quite here but not far away. Yes, hobbyists are driving the technology hard, identifying the weak spots, and the industry is pouring billions into AI, Data Centers, and software to string it all together, just so you can have a conversation.

The day you can hobble into ER, the robot nurse bathe and change a dressing, all the while speaking to you about the weather as well as uniquely adapting to your medical condition that nobody else in the building has at that very moment, will be the day that you know the industry has arrived. In the meantime, we struggle, we make miztakes, and we enjoy whatever life has to offer, the alternate not being so exciting. A caring individual support person will not become unemployed for quite a few decades I suspect โ€“ empathy being something that is extremely hard to train a computer to do well.

Keep us posted on how your system is performing and how stable it now is, and whether the original problem that started this thread still exists.

Yes the Medical care was good but there is just not enough of them and the demand is high. Infrastructure has not kept up with population growth due to migration from southern states and overseas. Politicians not doing their job and have lots of money for war.

You are correct in assuming that I donโ€™t need to store Voltage as a Long Term Statistic (LTS) but I donโ€™t know how to stop that for the Sonoff devices and not the Shellyem? The Shellyem is what monitor my switchboard and the Sonoff devices are used for individual power consumption.
So can you enlighten me on how to do that please?

OK I just had a thoughtโ€ฆ
If I disabled the voltage here for sensors I donโ€™t want voltage, would that work?

ORโ€ฆ
AI suggested add this to the customize.yalm

homeassistant:
  customize:
    sensor.your_device_voltage:
      state_class: none

I have no โ€œadd onโ€™sโ€ or Voice assistant things that use this as far as I know. For Voice I use either Google or Alexa at this stage as I have Google display hubs and Amazon speakers. that do my voice (apart from phone).

What is this REPIAR all about? I didnโ€™t authorise this??

App InfluxDB has been removed from the repository it was installed from. This means it will not get updates, and backups may not be restored correctly as the Home Assistant Supervisor may not be able to build/download the resources required.

Selecting Submit will uninstall the app and permanently delete everything in its private data folder, including any databases, credentials and other internal state it kept there. Create a backup first if you might need that data later.

Alternatively, you can check Home Assistant help and the community forum for alternatives to migrate to.

Can you please advise me on this.
Thank you

Sadly, you have joined the hordes that have had it abruptly pulled out from under them when you did your update. There are other threads on these forums bitterly complaining.
Read and weep - ask for your money back. Oh waitโ€ฆ

Youโ€™ve got backups, right? Maybe checks those threads before you do anything rash.

As much as Queenslanders pooh them, I see some VictoriaMetrics may come into your life.

Itโ€™s looking for something from MusicAssistant. Check if you have any related App in HACS as well as in settings.

I found a Mini Media player in HACS. I have removed it.

Yes I have backups but itโ€™s eventually got to happen so I will have to go with your suggestion of the VictoriaMetrics. Do I really need Grafana? I had a use for it but I donโ€™t use it a lot now so maybe itโ€™s something I can do without?
I will have to study up on how to go to VictoriaMetrics. Is there a guide somewhere?

Have a read through this (warning - itโ€™s a long thread). TLDR: Donโ€™t hit submit on that popup & simply close it via the X button until you decide on a way forward.