Enphase Envoy - D7 firmware with JWT - A Different Approach

Thanks for taking the time to explain that. I have the following sensor:

        {% set n_month = now().month %}
        {% set n_hour = now().hour %}
        {% if now().weekday() in (5,6) and n_hour > 11 and n_hour < 14 %}
          {% set tou_period = 'Free' %}
        {% else %}
          {% set tou_period = 'Shoulder' %}
          {% set is_summer = (n_month <= 3 or n_month >= 11) %}
          {% set is_winter = (6 <= n_month <= 8 ) %}
          {% if n_hour >= 22 or n_hour < 7 %}
            {% set tou_period = 'Offpeak' %}
          {% elif ((is_summer and (14 <= n_hour <= 19))
            or (is_winter and (14 <= n_hour <= 19)))
            and (is_state("binary_sensor.workday_sensor", "on")) %}
            {% set tou_period = 'Peak' %}
          {% endif %}
        {% endif %}
        {{tou_period}}

I get two free hours of electricity on Saturdays amd Sundays courtesy of Red Energy’s EV Plan, hence the extra logic. I don’t have seasonal prices, but left your logic in there in case I ever do (but I changed the TOU period to be the same for summer and winter).

When I plug this into Developer Tools, it says the sensor updates every minute. That’s OK I guess, but I thought to do this (inside my templates.yaml file):

- trigger:
    - platform: time_pattern
      minutes: 0
    # update every hour on the hour
    - platform: homeassistant
      event: start
  sensor:
    # Australian Ausgrid NSW Peak-shoulder-offpeak sensor defined
    # https://www.ausgrid.com.au/Your-energy-use/Meters/Time-of-use-pricing
    # Peak: 2pm - 8pm on working weekdays 1 November - 31 March;
    # Peak: 5pm - 9pm on working weekdays 1 June - 31 August
    # Off-peak: 10pm - 7am
    # Shoulder: all other times
    - name: TOU Period
      icon: mdi:clock-time-three-outline
      state: >
        {% set n_month = now().month %}
        {% set n_hour = now().hour %}
        {% if now().weekday() in (5,6) and n_hour > 11 and n_hour < 14 %}
          {% set tou_period = 'Free' %}
        {% else %}
          {% set tou_period = 'Shoulder' %}
          {% set is_summer = (n_month <= 3 or n_month >= 11) %}
          {% set is_winter = (6 <= n_month <= 8 ) %}
          {% if n_hour >= 22 or n_hour < 7 %}
            {% set tou_period = 'Offpeak' %}
          {% elif ((is_summer and (14 <= n_hour <= 19))
            or (is_winter and (14 <= n_hour <= 19)))
            and (is_state("binary_sensor.workday_sensor", "on")) %}
            {% set tou_period = 'Peak' %}
          {% endif %}
        {% endif %}
        {{tou_period}}

    - name: "Electricity Tariff"
      state: >-
        {% if is_state('sensor.tou_period', 'Free') %}
          0
        {% else %}
          {% if is_state('sensor.tou_period', 'Peak') %}
            .484
          {% elif is_state('sensor.tou_period', 'Shoulder') %}
            .33
          {% else %}
            .22
          {% endif %}
        {% endif %}
      unique_id: electricity_tariff
      unit_of_measurement: AUD/kWh

I think this means the TOU and tariff sensors only update on the hour, and on HA startup. Reckon this is right? I didn’t bother setting up input_numbers for the tariff, and just coded the tariffs directly in the sensor. Do I need to do anything else in there, or is just coding the numbers directly like that sufficient (maybe with regard to it being a float or something)?

I’m not sure if you still use 3 automations to change the tariff, but I’m using a single one to update my two Utility Meters, like this:

alias: Tariff Utility Meters Update
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.tou_period
condition: []
action:
  - service: select.select_option
    data:
      option: "{{ states ('sensor.tou_period') }}"
    target:
      entity_id:
        - select.daily_import
  - service: select.select_option
    data:
      option: "{{ states ('sensor.tou_period') }}"
    target:
      entity_id:
        - select.monthly_import
mode: single

Just a bit simpler! Thanks again for your invaluable help…

so tried implementing this code…
I have the selector at the bottom and I can also display apexcharts as you did in a previous post, but I’m getting an error with this post…

That statement only applies to the developer tools page. This does not apply to sensors or if statements which are instantly updated as soon as a state changes.

@del13r
I was previously using the Enphase DEV integretion… definitely like this one for the local resources (quicker)

one thing I see different is this sensor missing.
wondering where in the API this could be found to create it.

image

Total overkill.
In my experience, the if statement works flawlessly without it.

Here is my current setup for the enphase style view.

Setup the helpers in configuration.yaml

input_select:
  statistics_chart_range:
    initial: day
    options:
      - day
      - week
      - month
      - year

input_datetime:
  statistics_chart_date:
    has_date: true
    has_time: false

Verified they were setup
image

Then setup the template sensors in configuration.yaml

template:

  - sensor:
      - name: statistics_chart_grouping
        unique_id: statistics_chart_grouping
        state: |
          {% set grouping={
            'day':'h',
            'week':'d',
            'month':'d',
            'year':'d'}
          %}
          {% set range = states('input_select.statistics_chart_range') %}
          {{ grouping[range] }}
      - name: statistics_chart_span
        unique_id: statistics_chart_span
        state: |
          {% macro last_date_in_range(date_str,range) -%}
            {% set date=as_timestamp(date_str)|timestamp_local()|as_datetime() -%}
            {% if range=='day' -%}
              {% set end_date=date -%}
            {% elif range=='week' -%}
              {% set end_date=date + timedelta(days=(6-date.weekday())) -%}
            {% elif range=='month' -%}
              {% set month=(date.month % 12) + 1 -%}
              {% set year=date.year+iif(date.month==12,1,0) -%}
              {% set end_date=strptime(year|string + '-' + month|string + '-01', '%Y-%m-%d')-timedelta(days=1) -%}
            {% elif range=='year' -%}
              {% set end_date=strptime(date.year|string + '-12-31', '%Y-%m-%d') -%}
            {% endif -%}
            {{ as_timestamp(end_date)|timestamp_local() -}}
          {% endmacro -%}
          {% set range = states('input_select.statistics_chart_range') %}
          {% set date_str = states('input_datetime.statistics_chart_date') %}
          {% set spans={
            'day':'24',
            'week':'7',
            'month':(last_date_in_range (strptime(date_str, '%Y-%m-%d'),range)|as_datetime()).day,
            'year':last_date_in_range (strptime(date_str, '%Y-%m-%d'),range)|as_timestamp(0)|timestamp_custom('%j',0)|int +1}
          %}
          {{ spans[range] }}
      - name: statistics_chart_period
        unique_id: statistics_chart_period
        state: |
          {% set periods={
            'day':'hour',
            'week':'day',
            'month':'day',
            'year':'day'}
          %}
          {% set range = states('input_select.statistics_chart_range') %}
          {{ periods[range] }}
          
  - sensor:
      - name: statistics_chart_offset
        unique_id: statistics_chart_offset
        state: |
          {% macro last_date_in_range(date_str,range) -%}
            {% set date=as_timestamp(date_str)|timestamp_local()|as_datetime() -%}
            {% if range=='day' -%}
              {% set end_date = date + timedelta(days=1) -%}
            {% elif range=='week' -%}
              {% set end_date=date + timedelta(days=(6-date.weekday())) -%}
            {% elif range=='month' -%}
              {% set month=(date.month % 12) + 1 -%}
              {% set year=date.year+iif(date.month==12,1,0) -%}
              {% set end_date=strptime(year|string + '-' + month|string + '-01', '%Y-%m-%d')-timedelta(days=1) -%}
            {% elif range=='year' -%}
              {% set end_date=strptime(date.year|string + '-12-31', '%Y-%m-%d') -%}
            {% endif -%}
            {{ as_timestamp(end_date) -}}
          {% endmacro -%}
          {% set range = states('input_select.statistics_chart_range') %}
          {% set date_str = states('input_datetime.statistics_chart_date') %}
          {% set offset = last_date_in_range(strptime(date_str, '%Y-%m-%d'),range)|int - now()|as_timestamp(0) -%}
          {% if states('sensor.statistics_chart_grouping') == 'h' %}
            {% set offset=(offset/60/60)|round(method='ceil') %}
          {% else %}
            {% set offset=(offset/60/60/24)|round(method='ceil') + 1 %}
          {% endif %}
          {{ iif(offset >= 0,'+','') }}{{ offset }}{{ states('sensor.statistics_chart_grouping') }}

Verified they were setup
image

Then added this to scripts.yaml

statistics_chart_date_browse:
  alias: statistics_chart_date_browse
  sequence:
  - service: input_datetime.set_datetime
    target:
      entity_id: input_datetime.statistics_chart_date
    data:
      datetime: "{% set range = states('input_select.statistics_chart_range') %}\n{%
        set date = states('input_datetime.statistics_chart_date')|as_datetime %}\n{%
        set direction = direction %}\n{% if range == 'day' %}\n  {% if direction >
        0 %}\n    {% set target = date + timedelta(days=1) %}\n  {% else %}\n    {%
        set target = date - timedelta(days=1) %}\n  {% endif %}\n{% elif range ==
        'week' -%}\n  {% if direction > 0 %}\n    {% set target = date + timedelta(days=7)
        %}\n  {% else %}\n    {% set target = date - timedelta(days=7) %}\n  {% endif
        %}\n{% elif range == 'month' -%}\n  {% if direction > 0 %}\n    {% set month=((date.month)
        % 12) + 1 %}\n    {% set year=date.year+iif(date.month==12,1,0) %}\n    {%
        set target=strptime(year|string + '-' + month|string + '-01', '%Y-%m-%d')
        -%}\n  {% else %}\n    {% set month=((date.month - 2) % 12) + 1 %}\n    {%
        set year=date.year+iif(date.month==1,-1,0) %}\n    {% set target=strptime(year|string
        + '-' + month|string + '-01', '%Y-%m-%d') -%}\n  {% endif %}\n{% elif range=='year'
        -%}\n  {% if direction > 0 %}\n    {% set target=strptime((date.year|int +
        1)|string + '-01-01', '%Y-%m-%d') -%}\n  {% else %}\n    {% set target=strptime((date.year|int
        - 1)|string + '-01-01', '%Y-%m-%d') -%}\n  {% endif %}\n{% endif -%}\n{% if
        direction == 0 %}\n    {% set target = now() %}\n{% endif -%}\n{{ target }}\n"
  mode: single
  icon: mdi:arrow-right-bold-outline

Verified that was setup

I made an extra automation to rollover the graph to todays date at 12:01AM

alias: Update day of apexchart at midnight
description: ""
trigger:
  - platform: time
    at: "00:01:00"
condition: []
action:
  - service: script.statistics_chart_date_browse
    data:
      direction: 0
mode: single

Then created a dashboard view

  - theme: Backend-selected
    title: Enphase
    path: enphase
    icon: mdi:solar-power
    type: sidebar
    badges: []
    cards:
      - type: custom:config-template-card
        variables:
          offset: states['sensor.statistics_chart_offset'].state
          span: states['sensor.statistics_chart_span'].state
          period: states['sensor.statistics_chart_period'].state
          grouping: states['sensor.statistics_chart_grouping'].state
        entities:
          - sensor.statistics_chart_offset
          - sensor.statistics_chart_span
          - sensor.statistics_chart_period
          - sensor.statistics_chart_grouping
        card:
          type: custom:apexcharts-card
          graph_span: 1d
          span:
            offset: ${offset}
          stacked: true
          header:
            show: false
            title: Total Power
          series:
            - entity: sensor.power_production
              type: column
              name: Produced
              color: '#01B4DE'
              group_by:
                func: avg
                duration: 5min
            - entity: sensor.power_consumption
              transform: return x *-1 ;
              type: column
              name: Consumed
              color: '#F37320'
              group_by:
                func: avg
                duration: 5min
            - entity: sensor.power_net
              type: column
              name: Imported(+)/Exported(-)
              color: '#545456'
              group_by:
                func: avg
                duration: 5min
          yaxis:
            - min: -7000
              max: 7000
              decimals: 0
      - type: custom:mod-card
        card:
          type: horizontal-stack
          cards:
            - type: entities
              entities:
                - entity: input_datetime.statistics_chart_date
                  name: date
            - type: glance
              show_name: false
              show_state: false
              entities:
                - entity: script.statistics_chart_date_browse
                  icon: mdi:chevron-left
                  tap_action:
                    action: call-service
                    service: script.statistics_chart_date_browse
                    data:
                      direction: -1
                - entity: script.statistics_chart_date_browse
                  icon: mdi:calendar-today
                  tap_action:
                    action: call-service
                    service: script.statistics_chart_date_browse
                    data:
                      direction: 0
                - entity: script.statistics_chart_date_browse
                  icon: mdi:chevron-right
                  tap_action:
                    action: call-service
                    service: script.statistics_chart_date_browse
                    data:
                      direction: 1

Verified that was good

Alternatively, if all of this is too complex, you can just have a simple card that just shows the last 24 hours using some of my older original instructions here.
https://community.home-assistant.io/t/enphase-envoy-with-energy-dashboard/328668/618?u=del13r

Hi @fireheadman,Which specific DEV integration are you referring to?
There are many to choose from:

briancmpbll/home_assistant_custom_envoy
posixx/home_assistant_custom_envoy (archived on Jul 14, 2023)
jrutski/home_assistant_envoy_d7_fw
DanBeard/enphase_envoy
jesserizzo/envoy_reader
vincentwolsink/home_assistant_enphase_envoy_installer

Unfortunately they are all referred to as DEV, so i need more specifics so I can try look at the code and work out what the source of binary_sensor.grid_status is.

I had a look at the code for the most popular DEV integration and found the following

I searched my https://envoy.local/home.json file for both enpower and grid_status but unfortunately I dont have batteries, so that is as far as I can test/investigate.

I did find this post where someone else had the same idea.

Great. Thank you for your insight.

1 Like

It was the campbell integration that I used… and good gosh!! Last I only knew of 5 custom enphase integrations, they are like gremlins now!

As for batteries, I do not have them either. He was able to pull out the grid status aside from batteries, but this might just be a tail chase, so not a big deal right now. I’m just happy to be able to pull data locally.

Thanks for the step by step guide, I will re-attempt this tomorrow.

1 Like

@del13r
I got it working… had to make a couple adjustments to get my values from getting chopped off. I have a 15.59kW system.

How can I reduce the size? I have to scroll to get to the selector, if I try “control -” to zoom out, it dynamically readjusts on me
Nevermind… found my issue, didn’t have type: sidebar, was using 1-Panel, not it works better. You ROCK! This is very awesome.

BTW: Is it normal behavior that when I have this card opened on my computer and at the same time view it on my iPhone, whenever I select another day on my phone, it updates the screen to that same view on my computer?

Only thing left to figure out is the tariffs. We have a funky deal between 4p-8p where the energy company takes the peak of each day out of 30 days to charge you $2/kWh. So if I decided to run the oven + dryer + sauna at the same time, then I get smacked with $2/kWh for that day between 4p-8p.

1 Like

Yes. The button calls the script to change a sensor related to what day you are viewing.

Hello everyone!

Thank you for all this valuable information! :muscle:t2: When I read the previous post and the post here, I still have a hard time understanding if there is a simple solution to integrate the Envoy (consumption, net energy, and production) in the “Energy” dashboard?

If not, why aren’t your developments or improvements directly committed to the main project?

It’s important for me to understand the context before getting into this I don’t want to wast my time with code or committing… errors :smile:

Thanks in advance for your help! :blush:

Hi @ALh4

Having initially struggled myself with energy dashboard when it was first released years ago, I can understand what you are getting at. Here is my attempt to answer your question.

When I wrote this, I had a very narrow list of requirements that were only meant to fix my own issues with my own hardware setup.
That list was:

  • I was originally only interested in getting data from 2 enphase sensors. power consumption and power production. From there I could use template sensors to calculate power import and power export and then use Integration - Riemann sum integral to record energy over time needed for energy dashboard. This is all I was interested in when I had D5 firmware.
  • get around the D7 firmware with cloud token not being supported by the native integration at the time.
  • poll the device faster than the once per 60 seconds I was previously getting with the native integration.
  • as a bonus, try to find a way to avoid using the Integration - Riemann sum integral which was used to convert power to energy over time.

The simplest solution right now is to use the native enphase integration.

Note: With all Enphase integrations, you will always need to setup template sensors to calculate power import and power export and then use Integration - Riemann sum integral to record energy over time needed for energy dashboard as the envoy production.json file that the Enphase Integrations use does not provide this information.

Why do I suggest this?
At that moment in time when I wrote this topic, the native integration did not support D7 firmware.

D7 firmware is now supported and if you are looking for the easiest and lowest maintenance solution, the native integration is the way to go because it will:

  • periodically refresh your enphase cloud token for you
  • support the increasing number of different models of enphase device (envoy/gateway/batteries)
  • work with either type of consumption CT clamp position/setup
  • get data on individual micro-inverters

A few reasons.

  • I lack any real development skills. Imagine someone who can read a language but cannot write it.

  • What I found via the replies I was getting to this topic, was that there are 2 types of consumption ct clamp position/setting. The position/setting of the consumption ct clamp was the functional difference between whether my solution needed the Integration - Riemann sum integral or not.
    This fragmentation across the installed enphase systems out there is the main reason I have not taken this further.

  • All of the other enphase related integrations that I am aware of, they all appear to rely on scraping data from the production.json file because every time that production.json file is requested, the envoy performs calculations to generate a standardised output regardless of the ct clamp setup. I can see how this standardised output would be attractive to someone creating an integration as it makes the ct clamp position one less condition to cater for.

The only real downside of using the production.json is that the envoy is slower to provide this production.json file each time it is requested. This lag only really matters when you are attempting to get data more often than the standard every 60 seconds from the envoy.

This production.json file lag/delay is what prompted me to look for other faster performing local enphase api endpoints so I could refresh the data more often and have the data arrive much faster with lower latency.
The alternative endpoint which I found that is more efficient, and still use today myself, is mainly what prompted me to write this topic so I could document my findings in the hope that someone with better development skills might see the benefit and take it further.

1 Like

A huge thank you for all this valuable information! :pray:t2: In fact I now understand my main mistake, I thought it was a “live” dashboard, but in fact it’s only a daily dashboard?

1 Like

This is not true. I use a HACS custom version based on production.json, that has the daily total and full total, as well as per panel production. I do not have clamps, and get grid import export from a P1 energy meter. I do not need any templates or Riemann.

The below integration also has the option to use the faster endpoints, but you will indeed lose the totals then. For Riemann to be accurate, you will really need the faster updates too ( and make sure you use left Riemann), but as you do not need Riemann, the slower updates are just fine.

1 Like

Energy dashboard is a daily energy dashboard which uses kWh

… made with hourly energy bars.

You can optionally make your own live power dashboard using data from an enphase device like I have which uses W

You can also graph the historical power data which also uses W

When I make a statement on this topic, it is implied that we are trying to obtain the following measurements from the enphase device:

  • energy import (grid consumption)
  • energy export (return to grid)
  • energy production (solar production)

These 3 measurements are what energy dashboard requires as input.

Sure, you can buy another device to fill in the gaps created by your choice to not buy an Enphase consumption ct clamp, however what I am trying to achieve here is allowing people to obtain all the required information exclusively from an Enphase device.

If people have already paid for an Enphase consumption ct clamp to be installed, then why would someone want spend more money on another device?

I have to assume that most people interested in setting up an enphase device on energy dashboard would understand they need an enphase consumption ct clamp to achieve this.

1 Like

I am also getting that information from production.json, however the only useful measurement for energy dashboard is production today which can be used to satisfy (solar production).

Consumption today is not useful as that only shows what your house consumed. Consumption today does not differentiate how much energy was imported and exported from the grid, nor does it show how much energy was consumed exclusively from the panels.

You still need either an enphase ct clamp or a p1 energy meter to provide energy export and energy import. Without either of those, you will only be able to see energy production in the energy dashboard. The rest will not be visible.

1 Like

The help is really appreciated, it’s given me some good starting points to get ready with Home Assistant. Thank you so much! :blush:

1 Like