Automations.yaml corruption?

Configuration
1

I’ve been attempting to use the UI for as much of my Hass work as possible, eschewing editing files in a text editor where possible. (I’m stubborn that way. Not sure why. :wink: )

On more than one occasion I’ve had my automations.yaml file get corrupted. In a few cases it looked like the file was half written out on top of an old version of the file (one statement ending halfway through a clause, for example). In other cases, an automation was cloned on top of another automation with the same trigger but different actions.

This leads me to conclude the automations editor is buggy. Perhaps it has some fault conditions it is not handling on the write path, or deals poorly with multiple web browser tabs being open to lovelace at once. But I’m wondering – is it currently at a stage of development where bug reports on things like this is helpful, or is it more in the stage of development of “try it at your own risk, we’re not done yet”?

Chris

2

HA has not reached release 1.0 yet. For most of its existence (as I understand it), most people edited the config files directly, and not much attention was paid to the UI editors. That seems to have changed recently, but that’s not to say it’s to the point where they want it, or that there aren’t any bugs. If you find something that doesn’t work, by all means you should open an issue. See:

3

Ok, cool. Next time I hit this, I’ll capture relevant logs/etc and file a bug. In the meantime, I’m now tracking my config in github so it will be less work to recover from corruption. :wink:

1 Like
4

You mentioned you had multiple tabs open with lovelace, was that with the same automation editor open, if so that could be the problem

5

Yeah, often when I’m testing things out I start off working on my desktop, then forget to close the tab and go work on my laptop as I’m triggering sensors and the like.

If that causes problems with non-atomic writes to the automations.yaml file, then that is certainly a bug – the home assistant server should be using some form of locking so it never leaves that file in an inconsistent (a.k.a. half-written) state.

6

It’d be neat if this would get fixed. I just got the ‘500’ error. After reconstructing several automations to add to my last backup it happened again.

Here’s the log:

2020-11-06 16:10:37 ERROR (SyncWorker_8) [homeassistant.util.yaml.loader] unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0
2020-11-06 16:10:37 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 61, in load_yaml
    return yaml.load(conf_file, Loader=SafeLineLoader) or OrderedDict()
  File "/usr/local/lib/python3.8/site-packages/yaml/__init__.py", line 112, in load
    loader = Loader(stream)
  File "/usr/local/lib/python3.8/site-packages/yaml/loader.py", line 34, in __init__
    Reader.__init__(self, stream)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 85, in __init__
    self.determine_encoding()
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 135, in determine_encoding
    self.update(1)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 169, in update
    self.check_printable(data)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 143, in check_printable
    raise ReaderError(self.name, position, ord(character),
yaml.reader.ReaderError: unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 18, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 72, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 129, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 124, in get
    current = await self.read_config(hass)
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 192, in read_config
    current = await hass.async_add_executor_job(_read, hass.config.path(self.path))
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 252, in _read
    return load_yaml(path)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 64, in load_yaml
    raise HomeAssistantError(exc) from exc
homeassistant.exceptions.HomeAssistantError: unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0
2020-11-06 16:11:44 ERROR (SyncWorker_8) [homeassistant.util.yaml.loader] unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0
2020-11-06 16:11:44 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 61, in load_yaml
    return yaml.load(conf_file, Loader=SafeLineLoader) or OrderedDict()
  File "/usr/local/lib/python3.8/site-packages/yaml/__init__.py", line 112, in load
    loader = Loader(stream)
  File "/usr/local/lib/python3.8/site-packages/yaml/loader.py", line 34, in __init__
    Reader.__init__(self, stream)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 85, in __init__
    self.determine_encoding()
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 135, in determine_encoding
    self.update(1)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 169, in update
    self.check_printable(data)
  File "/usr/local/lib/python3.8/site-packages/yaml/reader.py", line 143, in check_printable
    raise ReaderError(self.name, position, ord(character),
yaml.reader.ReaderError: unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 18, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 72, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 129, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 124, in get
    current = await self.read_config(hass)
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 192, in read_config
    current = await hass.async_add_executor_job(_read, hass.config.path(self.path))
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/config/__init__.py", line 252, in _read
    return load_yaml(path)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 64, in load_yaml
    raise HomeAssistantError(exc) from exc
homeassistant.exceptions.HomeAssistantError: unacceptable character #x0000: special characters are not allowed
  in "/config/automations.yaml", position 0

The (corrupted) automations.yaml can be found here: https://gofile.io/d/ocCoZx

So the automation it crashed on was for lighting based on a movement sensor, basically involving ‘device’ triggers, ‘sun’ and ‘time’ conditions and device light turn on actions. Everything selected in the UI.

I’ve also edited the automations name before saving, that might have something to do with it.