Enphase Envoy - D7 firmware with JWT - A Different Approach

Share your Projects!
62

I have deleted all lines that don’t interest me and made the sample production data more interesting than -0 (you obviously captured this at night)

Here is an example of what I came up with for power sensors

Power being exported exemple

Source: /ivp/meters/readings Description W kW
value_json[0].activePower Power Production 5000.00 5.00
template sensor calculation Power Net (calculated) (consumption-production) -3662.61 -3.66
value_json[1].activePower Power Consumption 1337.40 1.34
template sensor calculation Power Export (Calculated) (production-consumption) 3662.61 3.66
template sensor calculation Power Import (Calculated) (consumption-production) 0.00 0.00

Power imported example

Source: /ivp/meters/readings Description W kW
value_json[0].activePower Power Production 400.00 0.40
template sensor calculation Power Net (calculated) (consumption-production) 937.40 0.94
value_json[1].activePower Power Consumption 1337.40 1.34
template sensor calculation Power Export (Calculated) (production-consumption) 0.00 0.00
template sensor calculation Power Import (Calculated) (consumption-production) 937.40 0.94

Ok, so I think we have power sensors covered (code for load only users to follow soon)

Unfortunately I am unable to figure out how to calculate energy sensors when load only ct clamp setting is in use, given the data and comments you have provided as they are conflicting for me.

Particularly this comment really confuses me:

This behaviour would indicate it’s recording grid export which doesn’t make sense.

If I were you, I would consider:

  • Having another look at eid": 704643584” which is value_json[1].actEnergyDlvd and double check how it behaves when exporting in the day and importing at night.
  • Setup Riemann sum integration on the power import and power exports to give you equivalent energy export and energy import sensors.
  • changing the “load only” clamp setting/location to “load with solar production”
1 Like
63

Nice find. I am load only, so this shows as: "measurementType": "total-consumption"

1 Like
64

I captured a full 24 hours of this data for 704643584 and dumped it into a spreadsheet for easy comparison. I compared .actEnergyDlvd to the next reading and made sure that the next was bigger. Take a look at increasing energy? column on the tab called 704643584. This confirms my quick analysis from prior, it’s always increasing.

I’ve setup a couple of these sensors and this seems like the best way to proceed from what I can tell so far.

Unfortunately, my energy company required that the consumption ct be installed on the utility side of the meter, so moving it is not an option.

Thanks again for all the help!

65

This post was fantastic.

I currently have been using Envoy (Dev) from HACs but this is great!

Do you know of any way to determine how much energy is going to charge the batteries? The problem with house consumption when you have batteries is that it basically does produced - exported = consumption which in the case of having batteries is not correct. Enphase obviously has some way to know this because their website graphs this:

image

BLUE = produced
Orange = Consumed
Green = Charged/Discharged

Ultimately this is what I am trying to reproduce inside Home Assistant. Any ideas which rocks to over turn?

1 Like
66

For anyone trying to understand the Enphase API I suggest you look at this github repository https://github.com/Matthew1471/Enphase-API where Mathew is collating all the information available.

There are also many existing integrations to look at.

1 Like
67

this statement is confusing as “load only” would mean the ct clamp is further away from the utility (grid).

It sounds like the ct clamp is probably installed in the correct location, but perhaps just the setting on the envoy is incorrect?

  • Load with solar production = net consumption = grid import/grid export
  • Load only = total consumption = current consumption only

In the above image, I understand the transmission tower icon to represent the utility and the lightbulb to represent your home.

68

I would start with visiting the urls I have been using and see if there is a section for storage/batteries within it.
https://envoy.local/ivp/meters
https://envoy.local/ivp/meters/readings
https://envoy.local/ivp/meters/reports

As I don’t have batteries myself, there is nothing battery specific showing up for me.
Your results may vary.

69

It will all depend on what type of battery you have installed and the controller type.

For newer installs look here https://github.com/Matthew1471/Enphase-API/blob/4e3342b8f87726678add650ca16f9b8056b1dbc1/Documentation/IQ Gateway API/IVP/Ensemble/Inventory.adoc

70

Thanks,

I did indeed go there, I also looked at the production.json, neither of which have battery information.

The main envoy page has batteries on it:

image
image922×241 16.7 KB

however, interacting with these is “restricted” and you get kicked out and have to log in again

@dlmcpaul answer got me closer, unfortunately my batteries are not exposing the real_power_w key which is what I am looking for.

I appreciate the answers! I have an Emporia Vue that I flashed with ESP Home, I am now getting current power from that by using a template to sum them together. Below is a stripped down version of my template incase others are interested

  - trigger:
    - platform: time_pattern
      seconds: '/7'
    sensor:
      - name: Total House Power Draw
        unit_of_measurement: "W"
        device_class: power
        state: >
            {{ [ states('sensor.10gbe_energy_power'),
            states('sensor.cook_top_8'),
            states('sensor.oven_16'),
            states('sensor.dish_washer_1'),
            states('sensor.kids_bedroom_p_8_c_15'),
            states('sensor.3d_printer_p_10_c_7'),
            states('sensor.furnace_p_11_c_6'),
            states('sensor.garage_freezer_treadmil_p_12_c_5'),
            states('sensor.fridge_dining_room_outlets_p_13_c_4'),
            states('sensor.microwave_p_14_c_3'),
            states('sensor.kitchen_outlets_p_15_c_2'),
            states('sensor.bar_fridge_bar_outlets_p_16_c_1')]
            | map('float') | sum |round(0) }}
        availability: >
          {{ not 'unavailable' in
            [ states('sensor.10gbe_energy_power'),
            states('sensor.cook_top_8'),
            states('sensor.oven_16'),
            states('sensor.dish_washer_1'),
            states('sensor.kids_bedroom_p_8_c_15'),
            states('sensor.3d_printer_p_10_c_7'),
            states('sensor.furnace_p_11_c_6'),
            states('sensor.garage_freezer_treadmil_p_12_c_5'),
            states('sensor.fridge_dining_room_outlets_p_13_c_4'),
            states('sensor.microwave_p_14_c_3'),
            states('sensor.kitchen_outlets_p_15_c_2'),
            states('sensor.bar_fridge_bar_outlets_p_16_c_1')
            ] }}

It would be more ideal to be able to get this information from Enphase itself, the website obviously gets this info from somewhere, I guess they just don’t expose it to me locally with the types of batteries I have

71

I’m curious, what is shown here for you?

curl -s -k -H $TOKEN "https://envoy.local/ivp/livedata/status" | jq -r '.meters.storage'
{
  "agg_p_mw": 0,
  "agg_s_mva": 0,
  "agg_p_ph_a_mw": 0,
  "agg_p_ph_b_mw": 0,
  "agg_p_ph_c_mw": 0,
  "agg_s_ph_a_mva": 0,
  "agg_s_ph_b_mva": 0,
  "agg_s_ph_c_mva": 0
}
72

Some interesting stuff in this endpoint, I’ll think about what I can do to understand it.


{
    "connection": {
        "mqtt_state": "connected",
        "prov_state": "configured",
        "auth_state": "ok",
        "sc_stream": "disabled",
        "sc_debug": "disabled"
    },
    "meters": {
        "last_update": 1690661898,
        "soc": 98,
        "main_relay_state": 1,
        "gen_relay_state": 5,
        "backup_bat_mode": 1,
        "backup_soc": 30,
        "is_split_phase": 1,
        "phase_count": 0,
        "enc_agg_soc": 98,
        "enc_agg_energy": 19958,
        "acb_agg_soc": 0,
        "acb_agg_energy": 0,
        "pv": {
            "agg_p_mw": 6549982,
            "agg_s_mva": 6995410,
            "agg_p_ph_a_mw": 6549982,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": 6995410,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        },
        "storage": {
            "agg_p_mw": -6000,
            "agg_s_mva": -6000,
            "agg_p_ph_a_mw": -6000,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": -6000,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        },
        "grid": {
            "agg_p_mw": -5388088,
            "agg_s_mva": -6607469,
            "agg_p_ph_a_mw": -5388088,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": -6607469,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        },
        "load": {
            "agg_p_mw": 1155894,
            "agg_s_mva": 381941,
            "agg_p_ph_a_mw": 1155894,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": 381941,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        },
        "generator": {
            "agg_p_mw": 0,
            "agg_s_mva": 0,
            "agg_p_ph_a_mw": 0,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": 0,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        }
    },
    "tasks": {
        "task_id": -2006921067,
        "timestamp": 1690660736
    },
    "counters": {
        "main_CfgLoad": 1,
        "main_CfgChanged": 1,
        "main_taskUpdate": 23,
        "main_sigHUP": 2,
        "MqttClient_publish": 3741,
        "MqttClient_live_debug": 540,
        "MqttClient_respond": 72,
        "MqttClient_msgarrvd": 36,
        "MqttClient_reconnect": 2,
        "MqttClient_create": 6,
        "MqttClient_setCallbacks": 6,
        "MqttClient_connect": 6,
        "MqttClient_subscribe": 6,
        "SSL_Keys_Create": 6,
        "sc_hdlDataPub": 3716,
        "sc_SendStreamCtrl": 17,
        "sc_SendDemandRspCtrl": 1,
        "rest_Status": 4568
    }
}
73

For anyone wondering what this MQTT and stream stuff means:

MQTT usually runs on TCP PORT 8883.
It appears to be used to transmit the live status information to enlighten when you press the “LIVE STATUS” button.

URL
https://enlighten.enphaseenergy.com/mobile/YOURSITEID/today/graph/hours?v=3.4.0

image

Once the button is pressed, a 15 minute counter starts

image

Here is my NAT table from my router when I press “LIVE STATUS”.

Proto NATed Address Destination Address State
icmp 192.168.1.114 8.8.8.8
tcp 192.168.1.114:40728 52.207.46.158:8883 ESTABLISHED

This NAT table shows the envoy pinging dns.google and sending mqtt data on port 8883 to ec2-52-207-46-158.compute-1.amazonaws.com (52.207.46.158)

1 Like
74

I was having trouble retrieving the 408 digit authentication access code via the python code. It kept returning:

.py", line 2, in
import requests
ModuleNotFoundError: No module named ‘requests’

Maybe due to me running this via terminal on a Mac?

However, the Enphase Access Code generator site assisted here.

Once logged in, the next question asks to “Select System” and then “Gateway”.
The answer to “Select System” is your system’s name, which is available in the Enlighten app or website. To find it, open Enlighten, click on “System” then “Site Details.” The name of your system is in the first box, just below “Site Details” and above “Site ID: xxxxx.” It will be something like your name and/or address. The Gateway will then auto-populate the drop-down menu.

Hope this helps.

Thank you for your fantastic work on this instruction @del13r

1 Like
75

you could also just run pip3 install requests - that should take care of the error as well.

76

Thanks for that @Klaneos
I’ve just added another method to the guide as well.
I find this one better as some people might have trouble knowing/finding their system name.

77

Would be brilliant if Enphase would allow a permanent local mqtt connection to Home Assistant or other system. Considering they have this already that wouldn’t be too difficult.

2 Likes
78

For batteries, have you tried this url?
https://envoy.local/ivp/ensemble/inventory

79

I dont see what I am looking for in here:

"type": "ENCHARGE",
        "devices": [
            {
                "part_num": "REDACTED",
                "installed": 1690448752,
                "serial_num": "REDACTED",
                "device_status": [
                    "envoy.global.ok",
                    "prop.done"
                ],
                "last_rpt_date": 1691066866,
                "admin_state": 6,
                "admin_state_str": "ENCHG_STATE_READY",
                "created_date": 1690448752,
                "img_load_date": 1690448752,
                "img_pnum_running": "2.0.5441_rel/22.10",
                "zigbee_dongle_fw_version": "100D",
                "bmu_fw_version": "2.1.27",
                "operating": true,
                "communicating": true,
                "sleep_enabled": false,
                "percentFull": 34,
                "temperature": 30,
                "maxCellTemp": 30,
                "comm_level_sub_ghz": 3,
                "comm_level_2_4_ghz": 3,
                "led_status": 12,
                "dc_switch_off": false,
                "encharge_rev": 2,
                "encharge_capacity": 3360
            },
{
        "type": "ENPOWER",
        "devices": [
            {
                "part_num": "REDACTED",
                "installed": 1690349829,
                "serial_num": "REDACTED",
                "device_status": [
                    "envoy.global.ok",
                    "prop.done"
                ],
                "last_rpt_date": 1691066774,
                "admin_state": 24,
                "admin_state_str": "ENPWR_STATE_OPER_CLOSED",
                "created_date": 1690349829,
                "img_load_date": 1690349829,
                "img_pnum_running": "2.0.4780_rel/22.10",
                "zigbee_dongle_fw_version": "100D",
                "operating": true,
                "communicating": true,
                "temperature": 81,
                "comm_level_sub_ghz": 5,
                "comm_level_2_4_ghz": 5,
                "mains_admin_state": "closed",
                "mains_oper_state": "closed",
                "Enpwr_grid_mode": "multimode-ongrid",
                "Enchg_grid_mode": "multimode-ongrid",
                "Enpwr_relay_state_bm": 496,
                "Enpwr_curr_state_id": 16
            }
80

have you tried?
https://envoy.local/ivp/ensemble/dataraw/

81

I get a 401 page before being kicked back out and having to log in again