Recorder database migration errors after installing 2023.2.3

Hello,

Fairly new to HA and hoping someone can assist with this problem.

I’m running Home Assistant OS with MariaDB and upgraded the database from 2.5.1 to 2.5.2 (10.6.2). There were no issues after this upgrade. I followed this with updating to the 2023.2.3 release from 2023.2.1. Immediately after the restart my logbook stopped populating data and I got the error notification

" recorder_database_migration
The database migration failed, check [the logs]. Database Migration Failed"

The error log for that specific error is below. Any suggestions on how to fix this?

Database error during schema migration
Traceback (most recent call last):
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1900, in _execute_context
self.dialect.do_execute(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py”, line 736, in do_execute
cursor.execute(statement, parameters)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 206, in execute
res = self._query(query)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 319, in _query
db.query(q)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py”, line 254, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1030, ‘Got error 106 “Socket is connected” from storage engine InnoDB’)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/recorder/core.py”, line 742, in _migrate_schema_and_setup_run
migration.migrate_schema(
File “/usr/src/homeassistant/homeassistant/components/recorder/migration.py”, line 154, in migrate_schema
_apply_update(hass, engine, session_maker, new_version, current_version)
File “/usr/src/homeassistant/homeassistant/components/recorder/migration.py”, line 854, in _apply_update
_create_index(session_maker, “states”, “ix_states_entity_id_last_updated_ts”)
File “/usr/src/homeassistant/homeassistant/components/recorder/migration.py”, line 204, in _create_index
raise_if_exception_missing_str(err, [“already exists”, “duplicate”])
File “/usr/src/homeassistant/homeassistant/components/recorder/migration.py”, line 65, in raise_if_exception_missing_str
raise ex
File “/usr/src/homeassistant/homeassistant/components/recorder/migration.py”, line 202, in _create_index
index.create(connection)
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/schema.py”, line 4353, in create
bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 2211, in _run_ddl_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py”, line 524, in traverse_single
return meth(obj, **kw)
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py”, line 938, in visit_index
self.connection.execute(CreateIndex(index))
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/future/engine.py”, line 280, in execute
return self._execute_20(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py”, line 80, in _execute_on_connection
return connection._execute_ddl(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1472, in _execute_ddl
ret = self._execute_context(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1943, in _execute_context
self.handle_dbapi_exception(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 2124, in handle_dbapi_exception
util.raise
(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/util/compat.py”, line 211, in raise

raise exception
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1900, in _execute_context
self.dialect.do_execute(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py”, line 736, in do_execute
cursor.execute(statement, parameters)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 206, in execute
res = self._query(query)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 319, in _query
db.query(q)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py”, line 254, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (1030, ‘Got error 106 “Socket is connected” from storage engine InnoDB’)
[SQL: CREATE INDEX ix_states_entity_id_last_updated_ts ON states (entity_id, last_updated_ts)]
(Background on this error at: Error Messages — SQLAlchemy 1.4 Documentation)

1 Like

No one had a similar issue with similar recorder database issues in the past? Just looking for a little help. A suggestion, a tip, a theory, anything. :grin: I’m a bit stuck.

I was able to resolve this by uninstalling and reinstalling the MariaDB addon.

Hi,
did you purge your database or were you able to import the old statistic data?

I get the following error after upgrading:

Logger: homeassistant.components.recorder.core
Source: components/recorder/migration.py:202
Integration: Recorder (documentation, issues)
First occurred: 10:43:20 (1 occurrences)
Last logged: 10:43:20

Database error during schema migration
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb.OperationalError: (1071, 'Specified key was too long; max key length is 1000 bytes')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 742, in _migrate_schema_and_setup_run
    migration.migrate_schema(
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 154, in migrate_schema
    _apply_update(hass, engine, session_maker, new_version, current_version)
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 854, in _apply_update
    _create_index(session_maker, "states", "ix_states_entity_id_last_updated_ts")
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 204, in _create_index
    raise_if_exception_missing_str(err, ["already exists", "duplicate"])
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 65, in raise_if_exception_missing_str
    raise ex
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 202, in _create_index
    index.create(connection)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 4353, in create
    bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2211, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 938, in visit_index
    self.connection.execute(CreateIndex(index))
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/future/engine.py", line 280, in execute
    return self._execute_20(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
    return connection._execute_ddl(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1472, in _execute_ddl
    ret = self._execute_context(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (1071, 'Specified key was too long; max key length is 1000 bytes')
[SQL: CREATE INDEX ix_states_entity_id_last_updated_ts ON states (entity_id, last_updated_ts)]
(Background on this error at: https://sqlalche.me/e/14/e3q8

Got this when updating to 2023.3.1


When restarting HA it tries to update again, but still fails with some deadlock error…
Error executing query: (MySQLdb.OperationalError) (1205, ‘Lock wait timeout exceeded; try restarting transaction’) [SQL: UPDATE statistics set start_ts=IF(start is NULL,0,UNIX_TIMESTAMP(start) ), created_ts=UNIX_TIMESTAMP(created), last_reset_ts=UNIX_TIMESTAMP(last_reset) where start_ts is NULL LIMIT 250000;] (Background on this error at: Error Messages — SQLAlchemy 2.0 Documentation)
Traceback (most recent call last):
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1964, in _exec_single_context
self.dialect.do_execute(
File “/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py”, line 747, in do_execute
cursor.execute(statement, parameters)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 206, in execute
res = self._query(query)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py”, line 319, in _query
db.query(q)
File “/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py”, line 254, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1205, ‘Lock wait timeout exceeded; try restarting transaction’)

The above exception was the direct cause of the following exception:

Have had it twice! But no solution so far :frowning:
Once I lost my states table and now the statistics.
BTW, I am not running the add-on. Have MariaDB on another machine.

Similar issue. Just adding to the reports.

Logger: homeassistant.components.recorder.util
Source: components/recorder/migration.py:211
Integration: Recorder (documentation, issues)
First occurred: 8:55:56 AM (1 occurrences)
Last logged: 8:55:56 AM

Error executing query: (MySQLdb.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: CREATE UNIQUE INDEX ix_statistics_statistic_id_start_ts ON statistics (metadata_id, start_ts)] (Background on this error at: https://sqlalche.me/e/20/e3q8)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 748, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during query')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/recorder/util.py", line 123, in session_scope
    yield session
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 213, in _create_index
    raise_if_exception_missing_str(err, ["already exists", "duplicate"])
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 72, in raise_if_exception_missing_str
    raise ex
  File "/usr/src/homeassistant/homeassistant/components/recorder/migration.py", line 211, in _create_index
    index.create(connection)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 5049, in create
    bind._run_ddl_visitor(ddl.SchemaGenerator, self, checkfirst=checkfirst)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2430, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 670, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 1004, in visit_index
    CreateIndex(index)._invoke_with(self.connection)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 315, in _invoke_with
    return bind.execute(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1414, in execute
    return meth(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 181, in _execute_on_connection
    return connection._execute_ddl(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1526, in _execute_ddl
    ret = self._execute_context(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
    return self._exec_single_context(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2326, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 748, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2013, 'Lost connection to MySQL server during query')
[SQL: CREATE UNIQUE INDEX ix_statistics_statistic_id_start_ts ON statistics (metadata_id, start_ts)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Similar thing here. Happened twice already - originally thought it was connected to MySQL backup but times are inconsistent.

Logger: homeassistant.components.recorder.core
Source: components/recorder/core.py:716
Integration: Recorder (documentation, issues)
First occurred: 00:50:18 (1 occurrences)
Last logged: 00:50:18

Error in database connectivity during commit: Error executing query: (MySQLdb.OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') [SQL: INSERT INTO events (event_type, event_data, origin, origin_idx, time_fired, time_fired_ts, context_id, context_user_id, context_parent_id, data_id, context_id_bin, context_user_id_bin, context_parent_id_bin, event_type_id) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)] [parameters: (None, None, None, 0, None, 1681429800.073139, None, None, None, 272349, b'\x01\x87}\x06\xa4\x88\x17\xcf\xa7\xe3f\x80\xd5D\x1cS', None, None, 5)] (Background on this error at: https://sqlalche.me/e/20/e3q8). (retrying in 3 seconds)

Was anyone able to resolve this issue ? With MariaDB on other machine ?