Yeah, either the existing values and the imported values have to fit perfectly to each other, or you have to reimport everything.
For now, there is not really a simple way. As said, I am working on something, but that will only help if you have correct data after some specific date/time, and import the values before. Everything else will always be “not easy”, I do not even have an idea how to solve this. OK, maybe it could help to import hourly values and calculate the sum from them,
You could provide me a kind of specification containing:
what do you have currently in the database
which data you would like to import
how the result should look like
I cannot promise anyhting, but some input would help
Thanks man, its not a simple problem cause of the way they are stored.
So I had my DB crash March last year. So any energy data, Solar + Grid (Import + Export) and Battery (Charge + Discharge) is basically missing from Jan 23 to March 23. After is good
I have the data from my solar app and provider. Solar data (Battery inc) is good as it is already hourly. My data from provider is every 30mins but I used a pivot table to make it hourly.
I’m new to home assistant. I wanted to first knowing how to import statistics from an other older system. I struggled a bit with the documentation. Not finding Developer tools / Services.
Would be nice to update the README.md as there is no more service tab:
I thought it wasn’t useful to open an issue on github for that.
I’m succeeded to install this promising custom integration. I had to replace the “custom_components/homeassistant-statistics” mentioned in the readme with “custom_components/import_statistics” to get it working. But the service “import_from_file” is not registered. The UI says that the service should be added manually.
I’m running HA 2024.10 on docker. You can see my folder structure and the integration details below.
Can anyone help?
Hi Klaus, great integration, thanks a lot!
I had problems with the import of my water data, too. I also used m³ as the measuring unit. When I switched to Liter, everything worked fine.
At least I found out, that I had to switch the codepage for the csv file to Unicode UTF-8.
Perhaps this is helpful, if a similar error occures somewhere else.
Best regards - and “Guten Rutsch”,
Andreas
Readings cover various non-contiguous dates from february to october 2024 (not every day because they are readings from my provider, but I managed to “simulate” 24 rows for each day, although I don’t know if that helps).
The first month shows a really high power consumption I guess because I don’t have any readings from before. Can I do something to use the first reading as a baseline and start measuring from there?
December shows a negative value which is exactly the same as february but with a minus sign. The reason why that is really escapes me.
Do I need to calculate a proper “sum” value? Can’t I use the same value as “state”? And if not, why?
Did you read the paragraph “whereas counters are more complex” from the readme? Could be that “You have to align the imported values with the first current value in your database” is violated.
i’m trying to import this file, but i’m getting an error in logs, but “unknown error” on the gui.
i’m using container home assistant 2024.12.5
and these are the logs
Logger: homeassistant.helpers.script.websocket_api_script
Source: helpers/script.py:526
First occurred: 2:17:21 PM (3 occurrences)
Last logged: 2:38:55 PM
websocket_api script: Error executing script. Unexpected error for call_service at pos 1: expected string or bytes-like object, got 'NoneType'
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 526, in _async_step
await getattr(self, handler)()
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 764, in _async_call_service_step
response_data = await self._async_run_long_action(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
)
^
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 727, in _async_run_long_action
return await long_task
^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 2802, in async_call
response_data = await coro
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 2852, in _execute_service
return await self._hass.async_add_executor_job(target, service_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/import_statistics/__init__.py", line 42, in handle_import_from_file
stats = _prepare_data_to_import(file_path, call)
File "/config/custom_components/import_statistics/__init__.py", line 83, in _prepare_data_to_import
stats = _handle_dataframe(df, timezone_identifier)
File "/config/custom_components/import_statistics/__init__.py", line 151, in _handle_dataframe
source = _get_source(statistic_id)
File "/config/custom_components/import_statistics/__init__.py", line 182, in _get_source
if valid_entity_id(statistic_id):
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 240, in valid_entity_id
return VALID_ENTITY_ID.match(entity_id) is not None
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'
Hm. Can you please try to remove the first two columns (although I do not think that should be a problem). Then there should be a state column (values can be equal to sum for now, but I am also not sure if sum only should work as well.
If that does not help, please create a bug on the repo, including the file you try to import (not only a screenshot), and a screenshot of the HA GUI where you call the function.
The main issue was that when I started measuring, the meter was already greater than zero (previous owner energy consumption), so I could not use a sum value equal to the state value.
It would be nice to have options in the import process to fill gaps of data like simple “lerp” or “repeat-previous”. From what I understand I need to do that myself in an external tool to make sure the tsv is spaced hourly to the current timestamp.
Hi Klaus,
Just switched to HomeAssistant and well impressed, with your custom integration!
I also have a similar issue: imported 2024 gas usage data. All seems to be fine except a few days in this year: 01/01/2025, 02/01/2025 & 07/01/2025.
The history seems to be fine; the line is climbing pretty smoothly.
The Gas Usage graph (and this months totals) indicate different:
