2021.4: For our advanced users ❤️

I have a similar error, but not running Dwain’s UI.

Testing configuration at /tmp/config
Fatal error while loading config: Secrets not supported in this YAML file
Failed config
  General Errors: 
    - Secrets not supported in this YAML file

Is Lovelace_gen a custom component or what I used when I let HA control the UI?

As a heads up, I added Ids to all my automations, restarted and still have my formatting and all my comments. Im guessing if you actually edit an automation using the UI editor, all bets are off.

I also don’t have all my automations in a single file, they are in a single folder in multiple files using automation: !include_dir_merge_list automation/

Now to wait a bit for some automations to run and see if the tracing works.

2021-04-08 14:32:25 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns event_data in table events. Note: this can take several minutes on large databases and slow computers. Please be patient!
......
2021-04-08 15:01:25 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder
2021-04-08 15:01:55 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns attributes in table states. Note: this can take several minutes on large databases and slow computers. Please be patient!
2021-04-08 15:02:25 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder

It really seems like database upgrades should be done in the background, Home Assistant was perfectly capable of continuing to run in the past when the database was unavailable. So, it doesn’t store any information in the period of time that the database is upgrading, that’s still preferably to having no web interface or working system (my heating is now unavailable).

Obviously nothing can be done about it now, but perhaps something to consider for the future.

Also I would echo comments from others, some sort of progress meter so we have at least a very rough estimate as to how long this will take. As it stands I have no idea, whether I am on the last step of the upgrade or if there are another 100 columns it is going to need to alter at 30mins each time…

EDIT:
For people looking for estimated times.
Database Size:

+----------------+-----------+
| Table          | Size (MB) |
+----------------+-----------+
| states         |   2853.03 |
| events         |   1448.81 |
| recorder_runs  |      0.03 |
| schema_changes |      0.02 |
+----------------+-----------+

Stage1: “Modifying columns event_data in table events.” ~ 30mins
Stage2: “Modifying columns attributes in table states” ~ 16mins
Stage3: “Modifying columns time_fired, created in table events” ~ 30mins
Stage4: “Modifying columns last_changed, last_updated, created in table states” ~ 20mins

both of the integrations that cause the error the others posted are custom. Your error does not match their errors.

I’m having a massive issue with the upgrade. Somehow it keeps crashing on my MariaDB add-on. On restart, Home Assistant is not reachable for more than 30 minutes and then comes online with:

The MariaDB add-on logs then shows the DB might be corrupted and need repairs. Repairs it and after a restart of the add-on everything seems fine:

210408 16:29:48 mysqld_safe Logging to '/data/databases/mariadb.err'.
210408 16:29:49 mysqld_safe Starting mysqld daemon with databases from /data/databases
2021-04-08 16:29:49 0 [Note] InnoDB: Using Linux native AIO
2021-04-08 16:29:49 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-04-08 16:29:49 0 [Note] InnoDB: Uses event mutexes
2021-04-08 16:29:49 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-08 16:29:49 0 [Note] InnoDB: Number of pools: 1
2021-04-08 16:29:49 0 [Note] InnoDB: Using SSE2 crc32 instructions
2021-04-08 16:29:49 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-04-08 16:29:49 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-04-08 16:29:49 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-08 16:29:49 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-08 16:29:49 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2021-04-08 16:29:49 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-08 16:29:49 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-08 16:29:49 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-08 16:29:49 0 [Note] InnoDB: Waiting for purge to start
2021-04-08 16:29:49 0 [Note] InnoDB: 10.4.17 started; log sequence number 222990187316; transaction id 15365475
2021-04-08 16:29:49 0 [Note] InnoDB: Loading buffer pool(s) from /data/databases/ib_buffer_pool
2021-04-08 16:29:49 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-08 16:29:49 0 [Note] Server socket created on IP: '::'.
2021-04-08 16:29:49 0 [Note] Reading of all Master_info entries succeeded
2021-04-08 16:29:49 0 [Note] Added new Master_info '' to hash table
2021-04-08 16:29:49 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.4.17-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 3306  MariaDB Server
2021-04-08 16:29:49 0 [Note] InnoDB: Buffer pool(s) load completed at 210408 16:29:49
[16:29:49] INFO: Check data integrity and fix corruptions
mysql.column_stats                                 OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.index_stats                                  OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
homeassistant.events                               OK
homeassistant.recorder_runs                        OK
homeassistant.schema_changes                       OK
homeassistant.states                               OK
nginxproxymanager.access_list                      OK
nginxproxymanager.access_list_auth                 OK
nginxproxymanager.access_list_client               OK
nginxproxymanager.audit_log                        OK
nginxproxymanager.auth                             OK
nginxproxymanager.certificate                      OK
nginxproxymanager.dead_host                        OK
nginxproxymanager.migrations                       OK
nginxproxymanager.migrations_lock                  OK
nginxproxymanager.proxy_host                       OK
nginxproxymanager.redirection_host                 OK
nginxproxymanager.setting                          OK
nginxproxymanager.stream                           OK
nginxproxymanager.user                             OK
nginxproxymanager.user_permission                  OK
[16:29:50] INFO: Ensuring internal database upgrades are performed
This installation of MariaDB is already upgraded to 10.4.17-MariaDB, use --force if you still need to run mysql_upgrade
[16:29:50] INFO: Ensure databases exists
[16:29:50] INFO: Create database homeassistant
[16:29:50] INFO: Ensure users exists and are updated
[16:29:50] INFO: Update user hass
[16:29:50] INFO: Init/Update rights
[16:29:50] INFO: Granting all privileges to hass on homeassistant
[16:29:50] INFO: Successfully send service information to Home Assistant.
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
2021-04-08 16:33:55 0 [Note] /usr/bin/mysqld (initiated by: root[root] @ localhost []): Normal shutdown
2021-04-08 16:33:55 0 [Note] Event Scheduler: Purging the queue. 0 events
2021-04-08 16:33:55 0 [Note] InnoDB: FTS optimize thread exiting.
2021-04-08 16:33:55 0 [Note] InnoDB: Starting shutdown...
2021-04-08 16:33:55 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool
2021-04-08 16:33:55 0 [Note] InnoDB: Buffer pool(s) dump completed at 210408 16:33:55
2021-04-08 16:33:57 0 [Note] InnoDB: Shutdown completed; log sequence number 222990187325; transaction id 15365495
2021-04-08 16:33:57 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-04-08 16:33:57 0 [Note] /usr/bin/mysqld: Shutdown complete
s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening
[s6-finish] sending all processes the TERM signal.

However, restarting Home Assistant, the story is also beginning all over. 30 minutes wait… then error.

believed to have had al my templates fixed, until I restarted on a view with this

  - type: custom:auto-entities
    card:
      type: entities
      title: Philips light level raw sensors
      show_header_toggle: false
    filter:
      include:
        - entity_id: '*_sensor_light_level'
          options:
            type: custom:template-entity-row
            name: >
              {% if state_attr(config.entity,'friendly_name') is defined %}
              {{state_attr(config.entity,'friendly_name').split(' sensor light level')[0]}}
              {% else %} Tbd
              {% endif %}
            state: >
              {{state_attr(config.entity,'lightlevel')}} lm
            secondary: >
              {% set light_level = state_attr(config.entity,'lightlevel')|int %}
              {% if light_level < 1 %} dark
              {% elif light_level < 3000 %} bright moonlight
              {% elif light_level < 10000 %} night light
              {% elif light_level < 17000 %} dimmed light
              {% elif light_level < 22000 %} 'cosy' living room
              {% elif light_level < 25500 %} 'normal' non-task light
              {% elif light_level < 28500 %} working / reading
              {% elif light_level < 33000 %} inside daylight
              {% elif light_level < 40000 %} maximum to avoid glare
              {% elif light_level < 51000 %} clear daylight
              {% else %} direct sunlight
              {% endif %}

or this catch

{% if config.entity is defined %}

unfortunately this is not preventing the following

2021-04-08 16:44:32 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'None' has no attribute 'split' when rendering '{% if state_attr(config.entity,'friendly_name') is defined %} {{state_attr(config.entity,'friendly_name').split(' sensor light level')[0]}} {% else %} Tbd {% endif %}'

how can we catch this?

after restart is finished and all sensors loaded, this behaves fine, its just that I happened to check the log while on this view…
thanks for having a look

Maybe repacking before upgrading solves the issue.
Developers tool —> Services —> Recorder: Purge —> repack

So restore snapshot, repack and try again right? Because recorder intergration is now not loaded on boot.

what are you doing the split on? so what should you be checking for? How else can you check if something is None?

Hi,
I get an error with the notify.event integration:

'str' object has no attribute 'trim'

on both HAOS (or however this is named nowadays) and a core installation.

Notifications are not send - they worked on Core 2021.3.4

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 359, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 559, in _async_call_service_step
    await service_task
  File "/usr/src/homeassistant/homeassistant/core.py", line 1480, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1515, in _execute_service
    await handler.job.target(service_call)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 160, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/notify/__init__.py", line 137, in async_send_message
    await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/notify_events/notify.py", line 124, in send_message
    if data.get(ATTR_TOKEN, "").trim():
AttributeError: 'str' object has no attribute 'trim'

Shouldn’t it be .strip() ?

In case it matters, it’s a standard automation:

alias: Haustuer unten wurde geoeffnet
description: ''
trigger:
  - platform: state
    entity_id: switch.shelly_shsw_1_10e0dc
    from: 'off'
    to: 'on'
condition: []
action:
  - service: notify.events
    data:
      message: Haustuer geoeffnet
      title: Haustuer geoeffnet
mode: single

How big is/was your database? I’m on a similar set up but using MySQL on my synology, and sitting at 4GB of database, wondering if I should go get a drink…

that’s just the thing, the split is on state_attr(config.entity,'friendly_name') , hence the
{% if state_attr(config.entity,'friendly_name') is defined %}

and, even when I also check for the config.entity to be defined this still leads to

2021-04-08 16:56:35 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'None' has no attribute 'split' when rendering '{% if config.entity is defined and state_attr(config.entity,'friendly_name') is defined %} {{state_attr(config.entity,'friendly_name').split(' sensor light level')[0]}} {% else %} Tbd {% endif %}'

and doesn’t select the else, which it should?

You know plenty other ways to check for none…

Same here.

SpeedTest integration > Options: “Error. Config flow could not be loaded.”

Also:
Error handling request
10:42:47 AM – (ERROR) components/speedtestdotnet/config_flow.py -

1 Like

got it. its saying “WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: recorder”…its been 3 hours since i started the update :frowning:

Hi,
since my upgrade to v2021.4.0 my Speedtest.Net integration does not work anymore and I get the following log-entries:

2021-04-08 14:56:13 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry SpeedTest for speedtestdotnet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 249, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/components/speedtestdotnet/__init__.py", line 75, in async_setup_entry
    await coordinator.async_setup()
  File "/usr/src/homeassistant/homeassistant/components/speedtestdotnet/__init__.py", line 194, in async_setup
    self.api = await self.hass.async_add_executor_job(speedtest.Speedtest)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/speedtest.py", line 1091, in __init__
    self.get_config()
  File "/usr/local/lib/python3.8/site-packages/speedtest.py", line 1173, in get_config
    ignore_servers = list(
ValueError: invalid literal for int() with base 10: ''

Reinstalling the integration did not help.

Otherwise: I LOVE the tracing function of the automations. GEAT JOB. THANKS!!!

Ah, sorry, I meant 42 million records :blush:
The size is actually about 20GB.

Solution here!

Unrelated to the new HA version (the timing just by accident coincided). Was an issue in the upstream library that was fixed an hour ago. PR to use the new upstream library version is already created for HA.

3 Likes

I’ve gone through the files in /config and /custom_components and replaced secrets with their true values, but continue to get the error with Config Check. Any guesses?