Enphase Envoy with Energy Dashboard

Hi,
Love the work @del13r .
I’m sorry to have to do this. I have spend the last 3 night after work trying to nut this out.
From ready over this thread many times, I suspect it is my config.
I am only getting the Energy IMPORT available in the Energy integration. The Energy Export and Solar Corrected do not come up as entities for me to select.

Could anyone please help me know where I am going wrong to get this going.

template:
  - sensor:
        name: Grid Import Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.oakmont_solar_current_energy_consumption') | int - states('sensor.oakmont_solar_current_energy_production') | int ] | max }}

  - sensor:
        name: Grid Export Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.oakmont_solar_current_energy_production') | int - states('sensor.oakmont_solar_current_energy_consumption') | int ] | max }}

  - sensor:
        name: Solar Power Corrected
        state_class: measurement
        icon: mdi:solar-panel
        unit_of_measurement: W
        device_class: power
        state: >
            {% set value = states('sensor.oakmont_solar_current_energy_production') | int %}
            {% if value  <= 5 %}
              0
            {% elif is_state("sun.sun","below_horizon") %} 
              0
            {%- else -%}
              {{ value }}
            {%- endif %}

sensor:
  - platform: template
    sensors:
     envoy_current_exporting:
        friendly_name: "Current Energy Exporting"
        value_template: "{{ [0, (states('sensor.oakmont_solar_current_energy_production') | int - states('sensor.oakmont_solar_current_energy_consumption') | int)] | max }}"
        unit_of_measurement: 'W'
        icon_template: 'mdi:flash'
     envoy_current_importing:
        friendly_name: "Current Energy Importing"
        value_template: "{{ [0, (states('sensor.oakmont_solar_current_energy_consumption') | int - states('sensor.oakmont_solar_current_energy_production') | int)] | max }}"
        unit_of_measurement: 'W'
        icon_template: 'mdi:flash'

  - platform: integration
    name: Grid Import Energy
    source: sensor.grid_import_power
    unit_prefix: k
    unit_time: h
    device_class: energy
    
  - platform: integration
    name: Grid Export Energy
    source: sensor.grid_export_power
    unit_prefix: k
    unit_time: h
    device_class: energy
    
  - platform: integration
    name: Solar Energy Corrected
    source: sensor.solar_power_corrected
    unit_prefix: k
    unit_time: h
    device_class: energy

type or paste code here

Really appreciate any assistance. Its doing my head in.

Thanks.

@hooha_mc I had to manually change one or two of the energy entities ‘device_class’ attributes to energy in the customisation section in settings/configuration before they showed up, even though I defined them in my yams. You may have the same issue, suggest double checking that attribute is being set first, else they won’t show

I used a free online diff checker to compare our code

It seems you have tried to add device class energy to the (not template) integration sensor. This is not allowed in this part of the config for the integration platform and should not be needed if the source power sensors are providing the correct format. I suspect the presence of these device_class lines is causing platform: integration sensors to fail to load. If you are eagle eyed, you might have even noticed Home Assistant trying to warn you when you saved the file with a yellow light next to notifications. Can also go into configuration / logs and see what it had a problem with in the past as well. Please remove those device_class lines from the platform: integration sensors and restore the sensor: line above it

I might also add that if you are doing this at night after work, export won’t show up in the list until the sun starts shining and providing more power than what your house needs.

This page explains what variables are allowed to be used for platform: integration. device_class is not in the list and neither is state_class. You can only ever use what is in the list. Integration - Riemann sum integral - Home Assistant

If you want to force device_class and state_class onto the integration sensors, this is done in the customize.yaml but is optional not required.

customize.yaml

sensor.grid_export_energy:
  device_class: energy
  unit_of_measurement: kWh

sensor.grid_import_energy:
  device_class: energy
  unit_of_measurement: kWh

I noticed my enphase sensors have a slightly different name since the developer recently updated the enphase sensor names from energy to power to more accurately show what type of device class it is just in the sensor name. Please check your sensor names match what is actually in your dev tools / states list just in case the oakmount dev has done the same.

Also, not really sure what you are trying to do with these other sensors as they seem redundant because none of the integration sensors list these as a source for converting into energy. That is all the platform: integration sensors do is convert a power figure in W to an energy figure in kWh. I’d you are not feeding correctly formatted power figures into an equivalent integration sensor, I would recommend removing these from the config as it’s just confusing. If you are planning to try to use them anyway I did notice they are missing both the device_class: power and state_class: measurement
You are allowed to add those line to template sensors. You cannot add those lines to the integration sensors


sensor:
  - platform: template
    sensors:
     envoy_current_exporting:
        friendly_name: "Current Energy Exporting"
        value_template: "{{ [0, (states('sensor.oakmont_solar_current_energy_production') | int - states('sensor.oakmont_solar_current_energy_consumption') | int)] | max }}"
        unit_of_measurement: 'W'
        icon_template: 'mdi:flash'
     envoy_current_importing:
        friendly_name: "Current Energy Importing"
        value_template: "{{ [0, (states('sensor.oakmont_solar_current_energy_consumption') | int - states('sensor.oakmont_solar_current_energy_production') | int)] | max }}"
        unit_of_measurement: 'W'
        icon_template: 'mdi:flash'

1 Like

After a few days testing with the method: left added to the integration sensors I’m much happier with the accuracy.

There’s a small difference between import and export figures, but that’s due the sampling frequency being different and possibly a difference in how Enphase estimate the export but all in its more accurate than the default method.

1 Like

Hi,

I dont know what an oakmont is or wether it even suffers from the overnight production issue. You might only need 2 power sensors and 2 energy sensors and might not need to create the “corrected” sensor.

If you go into dev tools / states and search the page for _current you should see the correct sensor name and the unit_of_measurement should be W and state_class should be measurement in the right column.
Please pay very close attention to the name and note if it says energy or power as some older integrations use energy and new integrations use power in the sensor name.

I re-wrote your code assuming that
sensor.oakmont_solar_current_energy_production
and
sensor.oakmont_solar_current_energy_consumption
are correctly named as such in your system and also assuming that oakmont also shows erroneous overnight production.

homeassistant:
  customize: !include customize.yaml

template:
  - sensor:
        name: Solar Power Corrected
        state_class: measurement
        icon: mdi:solar-panel
        unit_of_measurement: W
        device_class: power
        state: >
            {% set value = states('sensor.oakmont_solar_current_energy_production') | int %}
            {% if value  <= 5 -%}
              0
            {% elif is_state('sun.sun','below_horizon') %}
              0
            {%- else -%}
              {{ value }}
            {%- endif %}

  - sensor:
        name: Grid Import Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.oakmont_solar_current_energy_consumption') | int - states('sensor.solar_power_corrected') | int ] | max }}

  - sensor:
        name: Grid Export Power
        state_class: measurement
        icon: mdi:transmission-tower
        unit_of_measurement: W
        device_class: power
        state: >
            {{ [0, states('sensor.solar_power_corrected') | int - states('sensor.oakmont_solar_current_energy_consumption') | int ] | max }}
            
sensor:
  - platform: integration
    name: Grid Import Energy
    source: sensor.grid_import_power
    unit_prefix: k
    unit_time: h
    
  - platform: integration
    name: Grid Export Energy
    source: sensor.grid_export_power
    unit_prefix: k
    unit_time: h

  - platform: integration
    name: Solar Energy Corrected
    source: sensor.solar_power_corrected
    unit_prefix: k
    unit_time: h

then in customize.yaml (once again, this step is not necessesary. It just forces the format in the absence of a positive number being fed into the sensor)

sensor.grid_export_energy:
  device_class: energy
  unit_of_measurement: kWh

sensor.grid_import_energy:
  device_class: energy
  unit_of_measurement: kWh
1 Like

I suspect that Enphase only samples every 5 minutes back to My Enlighten (sometimes with a really long lag too), where as home assistant samples every 1 minute. That being said, the energy dashboard is also 1 - 2 hours behind as well so lag is a problem for both dashboards. At least the home assistant sensors are rapidly recording every minute.
I assume that home assistant is able to capture more granular detail about what happens within that 5 minute sample. That could explain the slight discrepancy we are seeing on the grid in/out figures. Home assistant is just able to see more variations over that 5 minute sample.
The myenlighten produced and consumed figures will allways be accurate as they just go up like an odometer and they arent being calculated on the fly like grid in/out has to be.

mate @del13r - you went above and beyond. I want to donate a lockdown beer.
Used your code and works a treat. Thank you very much mate.

2 Likes

Hi, thanks for explaining the integration ‘integration’, that’s clear

Regarding the sensors: since i have values for all three newly created non template variables (‘import energy’ ‘export energy’ & ‘production energy’), i conclude there is no problem with defining them before the template sensors. Also, ‘import energy’ & ‘export energy’ are picked up by HA Energy and i was able to link them with the import and export fields of Ha energy. Unfortunatly, ‘production energy’ does however NOT show up in the HA energy config drop down.

The big difference between the newly created ‘export energy’ and ‘import energy’ variables with the newly created ‘production energy’ variable is that HA recognises the first two automaticly as device_class: energy, while ‘production energy’ is not recognised as such (it has no device class in the Tools overview).

Any ideas on how that automatic recognition is done by HA?

Thanks for your help & taughts!

I had another look at your code

Then in your follow up you say that all 3 integration sensors have values

This is good so far.

My next question is, what is the device class and unit of measurement of your source sensor named sensor.my_solar_energy_current_power_production look like?

This will directly impact how the integration sensor processes the source data.

Here is what my inverter power sensor looks like

In my case, I am not feeding this sensor directly into an integration sensor - because I don’t need to as I already have lifetime_energy_production sensor.
If i was to do that, I would probably use customize.yaml and add
device_class: power
to help guide things along as it appears to have everything else needed except that attribute.

Here is what my configuration.yaml would look like

homeassistant:
  customize: !include customize.yaml

and here is what my customize.yaml would look like

sensor.envoy_SERIALNUMBER_current_power_production:
  friendly_name: Production Now
  device_class: power

This might explain why your grid in/out sensors are selectable in energy dashboard and the production energy sensor is not.
I say this because you have created those 2 x import/export power sensors yourself and you were able to define these 3 key attributes yourself.

        state_class: measurement
        unit_of_measurement: W
        device_class: power

As you are using a 3rd ‘power’ sensor that you didnt create yourself as a source for the 3rd integration sensor, you need to ensure that 3rd ‘power’ sensor has the 3 necessesary attributes. If any of these attributes are missing, then adding the missing attributes using customize.yaml to an existing power sensor might get you going.

I just had a look at configuration / customizations. Seems to do the same thing as customize.yaml with better explanation.

Hi man,

thank you very much. I figured out it on myself yesterday but i’m happy to read that my solution is not some kind of ‘hack’ :slight_smile:

I also used configuration / customization (it is actually just an interface to change customize.yaml) and i’ve added device_class:energy to my non template Energy Production variable.
After i figured out that you also have to “!include customize.yaml” in configuration.yaml :slight_smile: , it works like a breeze

On a side note: very strange thing but after intending my template sensors more (like you said) they didn’t work any more. I’ve got them back in the lay out that I’ve posted here some posts upwards, and it works again. Very strange but i’m going to leave it like that :wink:

Thanks a lot for all your help.

1 Like

Glad you got it sorted.

I think these links pretty much sums up what I have been trying to say in most posts of this topic.

Which then links to

And then this link says you can customise any sensor that doesn’t already have what is needed

The type of data a sensor returns impacts how it is displayed in the frontend. This is controlled by the sensor’s device class designation. Built-in sensors and many created from an integration will have this designation predefined. Those can be modified in the customize section. When manually creating a new sensor the device class may be optionally assigned.

These links need to be a lot easier to find, aggregated into 1 simple link and should be promoted more regarding issues with energy dashboard.

Beyond that, I feel the energy dashboard should be updated in the next version to be able to accept current_power_production and current_power_consumption sensors and do the calculations for us instead of having to do everything mentioned in my first post

1 Like

Hi all - I did some google research today and found a website https://thecomputerperson.wordpress.com/2016/08/03/enphase-envoy-s-data-scraping that had lots of interesting and unpublished urls within our Envoys that display instantaneous values for power, reactive power, apparent power, voltage, current , power factor, lifetime power, last 7 days power and heaps of others for Consumption, Production and Net Consumption. It’s a nicely formatted json file which is updated with current values every time you call this url:

http://envoy.local/production.json

Here’s an example from my system:

{"production":[{"type":"inverters","activeCount":18,"readingTime":1629011546,"wNow":48,"whLifetime":15040561},{"type":"eim","activeCount":1,"measurementType":"production","readingTime":1629011749,"wNow":23.678,"whLifetime":15003847.261,"varhLeadLifetime":0.002,"varhLagLifetime":4480231.873,"vahLifetime":17614612.48,"rmsCurrent":1.289,"rmsVoltage":241.538,"reactPwr":305.709,"apprntPwr":311.372,"pwrFactor":0.07,"whToday":24605.261,"whLastSevenDays":154546.261,"vahToday":26747.48,"varhLeadToday":0.002,"varhLagToday":4836.873}],"consumption":[{"type":"eim","activeCount":1,"measurementType":"total-consumption","readingTime":1629011749,"wNow":667.827,"whLifetime":13996625.56,"varhLeadLifetime":9752177.355,"varhLagLifetime":4501845.22,"vahLifetime":24310859.886,"rmsCurrent":5.589,"rmsVoltage":241.653,"reactPwr":-961.189,"apprntPwr":1350.597,"pwrFactor":0.49,"whToday":14845.56,"whLastSevenDays":169982.56,"vahToday":25662.886,"varhLeadToday":11539.355,"varhLagToday":4837.22},{"type":"eim","activeCount":1,"measurementType":"net-consumption","readingTime":1629011749,"wNow":644.15,"whLifetime":9080345.931,"varhLeadLifetime":9752177.353,"varhLagLifetime":21613.347,"vahLifetime":24310859.886,"rmsCurrent":4.3,"rmsVoltage":241.768,"reactPwr":-655.479,"apprntPwr":1040.031,"pwrFactor":0.64,"whToday":0,"whLastSevenDays":0,"vahToday":0,"varhLeadToday":0,"varhLagToday":0}],"storage":[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

There is another url that displays similar information on a continuous streaming basis, however there are a at least 3 points to be aware of:

  1. You require the installer password for this - obtain this by running a simple python script from here GitHub - sarnau/EnphaseEnergy: Various collected information about Enphase Energy solar system
  2. The data is json formatted, except that the first curly bracket is missing from the “data” key, so you need to add that bracket and the quotation marks before the JSON can be parsed
  3. The data includes consumption/production details for power, current, power factor etc for three phases. If your system is single phase, the non-present phases read zero
http://installer:"installer password"@envoy.local/stream/meter

Here’s an example from my system which updates once per second:

data: {"production":{"ph-a":{"p":-2.24,"q":236.363,"s":239.815,"v":243.884,"i":0.984,"pf":0.0,"f":50.06},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"net-consumption":{"ph-a":{"p":785.708,"q":-648.654,"s":1144.564,"v":244.111,"i":4.69,"pf":0.68,"f":50.12},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"total-consumption":{"ph-a":{"p":783.468,"q":-885.017,"s":1384.502,"v":243.997,"i":5.674,"pf":0.57,"f":50.09},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}}}

data: {"production":{"ph-a":{"p":-2.127,"q":237.415,"s":240.205,"v":244.042,"i":0.985,"pf":0.0,"f":50.12},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"net-consumption":{"ph-a":{"p":787.869,"q":-649.53,"s":1147.731,"v":244.266,"i":4.707,"pf":0.68,"f":50.06},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"total-consumption":{"ph-a":{"p":785.742,"q":-886.946,"s":1389.779,"v":244.154,"i":5.692,"pf":0.57,"f":50.09},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}}}

data: {"production":{"ph-a":{"p":-1.949,"q":236.617,"s":240.536,"v":244.283,"i":0.985,"pf":0.0,"f":50.12},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"net-consumption":{"ph-a":{"p":781.671,"q":-649.608,"s":1144.144,"v":244.487,"i":4.674,"pf":0.68,"f":50.12},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}},"total-consumption":{"ph-a":{"p":779.722,"q":-886.225,"s":1382.896,"v":244.385,"i":5.659,"pf":0.56,"f":50.12},"ph-b":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0},"ph-c":{"p":0.0,"q":0.0,"s":0.0,"v":0.0,"i":0.0,"pf":0.0,"f":0.0}}}

Note:

It’s night here and you can see the phantom negative power values being generated.

Also note:

  • s = Apparent power
  • p = Real power (this is the one we are interested in)
  • q = Reactive power

I’m not very well versed with Home Assistant - I’d like to create 3 x sensors from either of these urls to display real-time Consumption, Production and Exported power. Does anyone know how to create a sensor from json data?

2 Likes

So back to this - I can now compare a full day of use with the AGL portal (which is the stuff they bill me on, so very accurate)

AGL stats

Energy Stats

image

Align almost exactly (just a .1kWh difference on import). Overall, pretty happy with the accuracy of the sensors now.

3 Likes

Ok a noob question here. You say it’s okay if my sensor says energy

  1. Does that mean I change the code in the yaml to “energy” where it says “power” or is it okay to leave it as power?

  2. Also my sensors are “sensors.enphase…” I assume I change the entry from “sensors.envoy…” to “sensors.enphase”?

And one more beginner question please.

  1. In the code where you have “sensor.envoy_SERIALNUMBER_current_power_consumption”, do I replace the SERIALNUMBER with the serial number of my Envoy or is that just calling the serial number from the already integrated system?

Yeah, when I originally started this project, as I had added my enphase envoy to Home Assistant in June 2021, mine were named …_current_energy_consumption and …_current_energy_production.
Someone mentioned the names had changed in late July 2021 however my names did not update (I assume this is only applied to newly added devices to the updated integration so it wont break anything on existing devices).
I then decided to test this and removed my enphase integration and re-added it and the names changed to …_current_power_consumption and …_current_power_production

In your case all you have to do is do a find/replace on my code whereever it says _current_power and replace with _current_energy
Alternatively, you could go into configuration / integrations / enphase / … / delete and re-setup your enphase envoy.

Yes, change the sensor names in the code to exactly match whatever your sensor names in your home assistant are. Go to developer tools / states / and confirm and make note of the sensor names containing _current

I only put SERIALNUMBER in the code as my sensor names actually had my physical envoy serial number in the sensor name. I wasnt comfortable broadcasting my envoy serial number here so I replaced my 12 digit serial number like 123456789012 with SERIALNUMBER. Go to developer tools / states / and confirm and make note of the sensor names containing _current. Your sensors may or may not have the serial number in the name.

1 Like

I have accessed this http://envoy.local/production.json file before but I never felt the need to fully understand how I would extract data from it until I read your post.

Here is some of the output when I view the .JSON file from my envoy.

Screen Shot 2021-08-16 at 11.23.41 pm

Tonight, I decided to read up on JSON format and tried to teach myself something new.

This is what I came up with

sensor:
  - platform: rest
    resource: http://envoy.local/production.json
    name: "Consumption Voltage"
    value_template: '{{ value_json.consumption[0].rmsVoltage }}'
    device_class: voltage
    unit_of_measurement: V

  - platform: rest
    resource: http://envoy.local/production.json
    name: "Consumption Power Factor"
    value_template: '{{ value_json.consumption[0].pwrFactor }}'
    device_class: power_factor
    unit_of_measurement: '%'

and this was the end result

Screen Shot 2021-08-16 at 11.20.51 pm

Screen Shot 2021-08-17 at 12.07.08 am

I’m getting a little lost here on what I ought to be doing with my Envoy-C model that’s integrated into HA. This is an older model that doesn’t have consumption data, and doesn’t expose envoy.local/production.json.

With an Envoy-C I have the following sensors:

  • sensor.envoy_{serial number}_current_energy_production
  • sensor.envoy_{serial number}_today_s_energy_production
  • sensor.envoy_{serial number}_last_seven_days_energy_production
  • sensor.envoy_{serial number}_lifetime_energy_production

Is the expectation that HA’s Enphase integration will expose these sensors with Energy in a future update?

If that is the case, am I correct that I would need some other device to monitor consumption and export?

Yes thats correct - you need a newer Envoy to allow monitor consumption or use a separate device. I have an Envoy that records consumption and production, but I also have a second way of getting this data from a device called a Flukso that counts pulses from two power meters

My envoy-s is mounted on the side of my house next to the switchboard. My Envoy-S has a CT clamp clamped around the mains wire coming out of my meter to measure consumption which then leads into the envoy-s which measures the consumption data coming from this clamp.

Envoy-S stating it does consumption

CT Clamp
image

The Envoy C appears to be a little more portable, only meant for indoors and I assume it only talks to the microinverters at the back of your solar panels.

Envoy-C not mentioning anything about the ability to read consumption.

Only the Envoy-S metered version has the Consumption Monitoring capabilty.

I found this link that confirms this.
https://www4.enphase.com/en-uk/products-and-services/envoy/family