I am currently trying to recreate the entire setup because some of the differences we are seeing here. This is a complicated puzzle which will take me some time to figure out. Once I have a new setup and have everything the way I want I will update this thread with that.
Thank you so much, @Remko. Quite impressive how detailed you analyze the things to find the best possible setup. We all benefit from this.
I am looking forward to the new/updated setup instructions. Maybe you could thereby double check if there is room to polish/simplify/modularize the config files (e.g. warning message “Property ‘sensors’ is not allowed”, don’t mix daily and yearly even if it makes no difference, don’t clutter config in multiple files, …). In a perfect world I would have just one “energy.yaml” or the like which I put under “packages” and that’s it. But again, all this would just be the “cherry on the cake”.
Another thing which I (and maybe some others) would be interested in, is the suggested DB setup for all this. For example, I am using the SE modbus integration with a 30s update interval and HA running with the default DB (at least I did not change anything) in a VM on a Synology NAS (720+) with a raid of 2 HDDs (not SSDs). Should I expect problems? Where can I improve? MariaDB? I don’t intend to buy SSDs for my NAS.
Cheers!
Thanks for the inputs…agree I have also found the initial setup a bit cluttered. It evolved over time…
I have started from scratch completely and will probably use a lot less sensors. The beginning is there already.
As to the DB…I think it is a bit personal…I like to have almost live updates, so I use a 3 second interval. This could of course cause some issues with the default DB on an SD Card, so I did do that differently as well. I use a Raspberry with an SSD connected to it and run MariaDB. I also excluded all sensors which I do not need from the recorder, to keep the DB increase within limits.
When my new config is done, I will also post that information. It may take some time, since it is not an easy puzzle and I am currently in the phase of data gathering and analyzing exactly what happens when charging or not etc. I am also trying to figure out when and where losses occur. I did already find a way of figuring it out a bit…losses when going from DC to AC and also the losses when discharging the battery, but have not found how I exactly want to see it.
There is another effect of minor fluctuations in some of the sensors (around 50 to 60W) which can cause big differences over time. I am quite sure the orginal SolarEdge monitoring ignores some of these effects, so I will most likely do that too, just need to figure out when to exactly do this.
So…it may take a week or 2…but…working on it…
1 Like
@stephanschuster Thanks for the tip on the packages. Never really looked into this, I did now based on your comment. I was also annoyed with having to edit multiple files, so I started reading up on this and…success… In the future you only need to add energy.yaml to your config folder and add this to the configuration.yaml:
homeassistant:
packages:
energy: !include energy.yaml
ooh nice…it is gonna be cleaner and better documented when I get it done…me like…
2 Likes
One more question: Would it be possible to have sensors which enable the two gauges shown here: https://echtsolar.de/photovoltaik-rechner/eigenverbrauch/
I mean the “Power Distribution Card” already shows autarky and ratio as a single bar for today (as I understand). What I would like to have is the same 3-fold visualization per graph and also for different time ranges: daily, weekly, monthly, yearly. Do you think this would be possible?
@stephanschuster : I am close to releasing the updated version of the integration, just a couple of days of verifying left. I integrated all of your requests and it looks really good now. I will probably create a new thread for this and close this one.
I will have to look at the statistics you would like to see, but I am pretty sure that can be done as well. Maybe I can find some time before releasing to implement this as well, if not, it will be an update to that. So please… Hold on for a day or 2 and we’ll take it from there. I like the idea though and will get that implemented…
2 Likes
@Remko : This sounds so damn good. I am really looking forward to all this. Thanks for being so kind and checking all our problems and feature requests. Really great!
First post updated with latest instructions and configurations. Hope this works good for everyone now…
Excited to find this thread. My solaredge system is scheduled for install on Thursday. I am expecting a 6.6kw system with a 5kw energy hub inverter. No battery for now.
Keen to enable TCP Modbus right away and start using the energy dashboard.
Thanks for your efforts, it looks great.
@Remko : Thank you! I saw your post a few hours ago and did the update as described in the first post. I definitely have to wait a few days to see how everything works out and how the values compare. Here is some first feedback. I’ll keep you updated. Feel free to answer or ignore me in case I bother you.
- I switched to MariaDB before the update. Would you mind sharing your “recorder config” so I can have a rough idea about your entity filters?
- I would like to stick to the default 30s update interval for now in order to reduce load on my HDD (HA on Synology NAS). Should I expect any noticable downsides?
- My VS Code still gives me the warning “Property sensors is not allowed.” on “template: - sensons:”. To my understanding this is because you’re using the legacy sensor configuration format which is no longer recommended: Template - Home Assistant.
- The “friendly_name” of “solar_battery_in_w” should probably be “Solar Battery In W”.
- The “name” of “solar_panel_to_house_daily” should probably be “Solar Panel To House Daily”.
- There are 13 matches of “solar_panel_…” and 3 matches of “solar_panels_…”. I suggest to stick to either one of them but not using both.
- I have 2 strange visual artifacts (see yellow markers in screenshot) which I did not have before and also cannot see on your screenshots. Most probably they are not related to your update. Still I wanted to ask if you have any idea about the root cause.
- I meanwhile have seen several temporary occurrences of “0” for grid and home power (marked red in screenshot). I did not look into this yet and need to track it further. Just wanted to mention it upfront.
Again, thank you so much.
Thanks for the feedback @stephanschuster .
Some comments:
- I switched to MariaDB before the update. Would you mind sharing your “recorder config” so I can have a rough idea about your entity filters?
Config below - I would like to stick to the default 30s update interval for now in order to reduce load on my HDD (HA on Synology NAS). Should I expect any noticable downsides?
Should not give any downsides, just less accurate readings and updates I think. -
My VS Code still gives me the warning “Property sensors is not allowed.” on “template: - sensons:”. To my understanding this is because you’re using the legacy sensor configuration format which is no longer recommended: Template - Home Assistant.
Done…first post… -
The “friendly_name” of “solar_battery_in_w” should probably be “Solar Battery In W”.
Changed in the first post -
The “name” of “solar_panel_to_house_daily” should probably be “Solar Panel To House Daily”.
Changed in the first post
6. There are 13 matches of “solar_panel_…” and 3 matches of “solar_panels_…”. I suggest to stick to either one of them but not using both.
Changed in the first post - I have 2 strange visual artifacts (see yellow markers in screenshot) which I did not have before and also cannot see on your screenshots. Most probably they are not related to your update. Still I wanted to ask if you have any idea about the root cause.
The line not being aligned is due to the Tesla Style Card. There is a fork of the repository which shows the lines with an offset as in my screenshot. I will need to find which fork it exactly is. I think it was this one:
https://github.com/mrgadget/tesla-style-solar-power-card
The blue bar being that big is a bit strange too. This bar represents the self consumption ratio and somehow it looks to be wrong. Don’t quite understand that yet. - I meanwhile have seen several temporary occurrences of “0” for grid and home power (marked red in screenshot). I did not look into this yet and need to track it further. Just wanted to mention it upfront.
Sometimes when the modbus connection drops for a second the value would be “unknown”. To counter that and follow up issues, I choose to set it to 0 at that moment. Maybe that is the reason for that.
To you second post…I do see some effects every now and then where there briefly are some “strange” flows. I suspect that is caused by a “nacheil” effect (don’t know the right English word actually). The other thing I do find strange is that your graph shows negative values. I do not get any negative values. I am wondering if there may be differences between certain inverters on how the values are reported. If that is the case my calculations could be completely wrong of course.
You could create some history graphs of the following sensors:
- solaredge_ac_power
- solaredge_dc_power
- solaredge_battery1_power
- solaredge_m1_ac_power
And mark in those graphs what happens, stuff like: producing and charging, discharging, exporting to grid etc. I could compare those with mine and see if there may be an issue there.
As to my recorder configuration:
recorder:
purge_keep_days: 60
include:
entities:
- sensor.solaredge_ac_energy_kwh
- sensor.solaredge_m1_exported_kwh
- sensor.solaredge_m1_imported_kwh
- sensor.solaredge_m1_imported_kwh_cost
- sensor.solaredge_m1_exported_kwh_compensation
- sensor.solaredge_battery1_state_of_charge
- sensor.solaredge_ac_power
- sensor.solaredge_dc_power
- sensor.solaredge_m1_ac_power
- sensor.solar_panel_production_daily
- sensor.solar_battery_in_daily
- sensor.solar_battery_out_daily
- sensor.solar_house_consumption_w
- sensor.solar_house_consumption_daily
- sensor.solar_imported_power_daily
- sensor.solar_exported_power_daily
- sensor.solar_inverter_effectiveness
- sensor.solar_battery_effectiveness
- sensor.solar_panel_to_house_daily
@stephanschuster Updated the sensor definitions to the latest Template standards. See first post for the update
@Remko Grazie! 
- Thanks for your “recorder” config. I am new to this. But if I understand it correctly with your config you will never be able to see the history of other sensor (e.g. light) than the ones you included, right? Is this intended? Also if you purge your MariaDB after 60 days you will never see yearly statistics in the energy dashboard, right? I guess I misunderstood something. I thought “auto_purge: false” would be the way to go and basically resemble the default behavior without MariaDB and recorder. I need to research here a bit. But maybe you can justify your 60 days upfront.
- Understood. Was also my expectation.
- Thanks for the update. Warnings are gone now.
- Thanks
- Thanks
- Perfect
- Ah, the fork explains the spacing between lines in your screenshot. I was already wondering which config option you used. Nevertheless, I am basically sure that yesterday I did not see the line being offset to the right in the tesla card. In fact, I could swear there was not even a line from grid to battery. So I cannot explain what happened here.
The huge blue bar in the power distribution card probably results from a self consumption ratio > 100% which in turn results from my data being not yet valid. Need to wait for a new day. - Understood. I might see these 0s more often than you due to my 30s update interval. I’ll track that.
Regarding “second post”:
- Never heard of “nacheil” effect. Do you mean “Nachteil” or “Nachhall”? Anyways, I don’t know what you mean.
- I thought the negative values in the tesla card are actually correct and just indicate the direction. Is it possible that negative values only appear when using the “show_w_not_kw: 1” config option of the tesla card like I did? If not (will check that during the next days) I can of course post the requested history graphs.
New question:
- For the config of the energy dashboard you use the raw “M1” sensors of the modbus integration. Wouldn’t it be more consistent with to use the appropriate utility meter sensors (…_power_daily) like with the rest of the energy dashboard config. Is “M1” better than “utility_meter”? I currently cannot compare them since I have not yet enough data since I switched the DB. Maybe you can elaborate here a bit.
- Thanks for your “recorder” config. I am new to this. But if I understand it correctly with your config you will never be able to see the history of other sensor (e.g. light) than the ones you included, right? Is this intended? Also if you purge your MariaDB after 60 days you will never see yearly statistics in the energy dashboard, right? I guess I misunderstood something. I thought “auto_purge: false” would be the way to go and basically resemble the default behavior without MariaDB and recorder. I need to research here a bit. But maybe you can justify your 60 days upfront.
This is not my full recorder config. I also use some entity_globs likeswitch_*to record other entities. But when I started reading up on it, I figured I actually do not need the history of each and every element, so I started excluding and including only what I really want.
*The purging of 60 days is just my personal preference. I have noticed over the years I never really go back that far to look at things, so why keep those records…? *
Since the energy dashboard got implemented HASS now also records the “long term statistics” which are not influenced by the purging of the DB. So my long term information (which is the energy dashboard) is kept in the DB permanently. - Understood. Was also my expectation.
- Thanks for the update. Warnings are gone now.
- Thanks
- Thanks
- Perfect
- Ah, the fork explains the spacing between lines in your screenshot. I was already wondering which config option you used. Nevertheless, I am basically sure that yesterday I did not see the line being offset to the right in the tesla card. In fact, I could swear there was not even a line from grid to battery. So I cannot explain what happened here.
Correct, in my previous config I did not have the “grid_to_battery” configured at all, so the line was never there. If you use the fork I linked, you should have the offsetted lines. If you don’t want to use that, you’ll just have to wait for an update from the author. The issue is reported and know, and there will be a fix, I just don’t know when and I actually do like the small offset better.
The huge blue bar in the power distribution card probably results from a self consumption ratio > 100% which in turn results from my data being not yet valid. Need to wait for a new day.
I hope it is just that and the data is correct and it is just an effect of the first day…fingers crossed, if not, we’ll have to analyze your data more carefully. - Understood. I might see these 0s more often than you due to my 30s update interval. I’ll track that.
Regarding “second post”:
- Never heard of “nacheil” effect. Do you mean “Nachteil” or “Nachhall”? Anyways, I don’t know what you mean.
I have been living in Germany for 20 years now, but I keep inventing words when I need them…
What I actually mean could be explained by thinking of a garden hose. When you open the water tap at the beginning, it takes some time for the full water flow to reach the end. Same when you turn it of, there is still water coming out even though the tap is closed. I think something similar happens with power; when I turn on my hot water (Duurchlauferhitzer) I draw a lot of power from battery and grid and when I turn it off it takes a small amount of time before the current settles down again. The battery is not on and off instantly, but ramps up and down in it’s delivery. So I suspect it is this effect we see, the power needs to go somewhere and using these almost instant modbus values actually show this effect happening. I therefore think it is actually correct what we are seeing. - I thought the negative values in the tesla card are actually correct and just indicate the direction. Is it possible that negative values only appear when using the “show_w_not_kw: 1” config option of the tesla card like I did? If not (will check that during the next days) I can of course post the requested history graphs.
I do not have that option set, so probably that is exactly the reason you are seeing negative values, which are correct of course. I will try that out.
EDIT…tried it and yes…negative because of that option. Actually quite nice, using that too now
New question:
- For the config of the energy dashboard you use the raw “M1” sensors of the modbus integration. Wouldn’t it be more consistent with to use the appropriate utility meter sensors (…_power_daily) like with the rest of the energy dashboard config. Is “M1” better than “utility_meter”? I currently cannot compare them since I have not yet enough data since I switched the DB. Maybe you can elaborate here a bit.
There is no real reason for this, besides the fact I already had data stored for these sensors, including the cost and compensation values, which I wanted to keep. Since “my” sensors give the exact same values (all based on solaredge_m1_ac_power) I just kept it as it was. Maybe I should change it out of consistency purposes, but the results are the same so I just couldn’t be bothered…
To the word “Nacheil” Effekt. It is actually a german word and used in AC power circuits… So I was not too far off…
If you want to read the details…be my guest, I stopped reading this article after I found the word…
Recorder config (1): Ah, okay, this is not your full recorder config. Then it makes sense. However, I don’t get the part with long term statistics. Are you saying that the whole energy dashboard and also the energy cards I use on my own dashboard are NOT stored in MariaDB? If not where then? Do you have some links where I can read an understand what is store where so I know what I can purge when.
Lines and bars (7): Ah, the tesla line previously was not configured at all. I should have seen this. Understood. And yes, I am sure the blue bar is just a temporary issue. Nevertheless, we could limit the sensor values of autraky and ratio to max. 100%. But this is surely an edge case.
Nacheileffekt: Ah, like “nacheilen”. 
Now I got it. Thanks for this nice explanation. 
M1 vs utility_meter: Understood. Then I will keep the utility meter. This makes things more consistent.
@Remko: Did I understand you correct that at some point in time you might look into the two gauges I mentioned (https://echtsolar.de/photovoltaik-rechner/eigenverbrauch/) and in case you add the needed sensors, you will let us know here?
To the recorder, no they ARE stored in the DB (MariaDB in our case) but are not purged out of the DB when it gets purged. They simply stay there, regardless of the purge setting.
The limiting to 100%…hmm…-yes maybe that is something I can add as well.
And yes, to the other statistics, I will look into this. Started it already and looking at which cards could be used for this. I will of course update this thread as soon as I got something. It actually should not be difficult, just a matter of creating more sensors and some formulas. Again…this may take a few days. I am now firstly enjoying how far I got already…
Last but not least: This is a bit off-topic here, but in case someone ever finds a way to prevent the line-breaks in the “chartTooltip” of the energy dashboard (e.g. by increasing witdh from 200px to 300px), please let me know. 
