Correct, only for realtime dashboard at the moment.
Ive turned off recorder for the mqtt sensors as I assume they would accumulate too much data too quickly.
I also just found that when my database file hits 3gb, problems start happening so I am in the process of identifying what’s important and what’s not for historical data.
Great, thank you. Would you mind posting your sensor and template yaml for the mqtt implementation? From your dashboard card examples, it looks like you use different names to the ones at https://github.com/vk2him/Enphase-Envoy-mqtt-json.
Oh, and how do you find out the database size? What sort of problems do you start to see at that size?
sensor:
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_production"
unit_of_measurement: W
value_template: >
{% set value = value_json["production"]["ph-a"]["p"] | int %}
{% if value <= 5 -%}
0
{% elif is_state('sun.sun','below_horizon') %}
0
{%- else -%}
{{ value }}
{%- endif %}
state_class: measurement
device_class: power
- platform: mqtt
state_topic: "/envoy/json"
value_template: '{{ value_json["total-consumption"]["ph-a"]["p"] | int }}'
name: "mqtt_consumption"
unit_of_measurement: W
state_class: measurement
device_class: power
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_power_factor"
unit_of_measurement: '%'
value_template: '{{ value_json["total-consumption"]["ph-a"]["pf"] }}'
state_class: measurement
device_class: power_factor
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_voltage"
unit_of_measurement: V
value_template: '{{ value_json["total-consumption"]["ph-a"]["v"] | int }}'
state_class: measurement
device_class: voltage
recorder:
exclude:
entity_globs:
- sensor.mqtt_*
template:
- sensor:
name: MQTT_Import_Power
state_class: measurement
icon: mdi:transmission-tower
unit_of_measurement: W
device_class: power
state: >
{% set production = states('sensor.mqtt_production') | int %}
{% set consumption = states('sensor.mqtt_consumption') | int %}
{{ [0, (consumption - production) ] | max }}
- sensor:
name: MQTT_Export_Power
state_class: measurement
icon: mdi:transmission-tower
unit_of_measurement: W
device_class: power
state: >
{% set production = states('sensor.mqtt_production') | int %}
{% set consumption = states('sensor.mqtt_consumption') | int %}
{{ [0, (production - consumption) ] | max }}
views:
- path: default_view
title: Power
type: sidebar
badges: []
cards:
- type: horizontal-stack
cards:
- type: gauge
entity: sensor.mqtt_consumption
min: 0
max: 7500
severity:
green: 0
yellow: 0
red: 0
- type: gauge
min: 0
max: 7500
severity:
green: 1
yellow: 0
red: 0
entity: sensor.mqtt_production
- type: gauge
max: 5
entity: sensor.today_energy_total_cost
min: -1
severity:
green: 0
yellow: 2
red: 4
- type: horizontal-stack
cards:
- type: gauge
min: 0
max: 7500
entity: sensor.mqtt_import_power
severity:
green: 0
yellow: 0
red: 0
- type: gauge
entity: sensor.mqtt_export_power
min: 0
max: 7500
severity:
green: 1
yellow: 0
red: 0
- type: gauge
severity:
green: 0
yellow: 0.2
red: 0.4
max: 0.5
entity: sensor.mqtt_instant_cost_per_hour
min: -1
- type: entities
entities:
- entity: sensor.mqtt_voltage
- entity: sensor.mqtt_power_factor
I got bored with voltage and power factor so replaced them with costs instead and moved them further down.
as for the 3gb database limit, I just found that the history tab wouldnt load anymore.
To find the size, I already had sqlite web addon installed
Also, I used customize.yaml to rename the sensors
sensor.mqtt_consumption:
friendly_name: House Consumption
sensor.mqtt_production:
friendly_name: Solar Production
sensor.mqtt_export_power:
friendly_name: to the Grid
sensor.mqtt_import_power:
friendly_name: from the Grid
sensor.mqtt_instant_cost_per_hour:
friendly_name: Cost Per Hour
1 Like
Amazingly helpful, as always! Why did you use customize.yaml, rather than naming the sensors like that in the first place?
I’m guessing there must be more to sensor.mqtt_instant_cost_per_hour. Have you used the mqtt data to improve your previously posted cost sensors in some way? I may have missed a post somewhere if you’ve already explained that bit.
I have 3 phases on my system. I’d appreciate any help (from anyone) as to how to change the sensors to take that into account. I’m very new to all this!
My script at GitHub - vk2him/Enphase-Envoy-mqtt-json: Takes real time stream from Enphase Envoy and publishes to mqtt broker produces mqqt data for all 3 phases - by default it displays Phase A. You just need to expand the sensors to include the other two Phases.
So this sensor
sensor:
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_production"
unit_of_measurement: W
value_template: >
{% set value = value_json["production"]["ph-a"]["p"] | int %}
{% if value <= 5 -%}
0
{% elif is_state('sun.sun','below_horizon') %}
0
{%- else -%}
{{ value }}
{%- endif %}
state_class: measurement
device_class: power
Just Copy and change ph-a to ph-b and another to ph-c
sensor:
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_production_phaseb"
unit_of_measurement: W
value_template: >
{% set value = value_json["production"]["ph-b"]["p"] | int %}
{% if value <= 5 -%}
0
{% elif is_state('sun.sun','below_horizon') %}
0
{%- else -%}
{{ value }}
{%- endif %}
state_class: measurement
device_class: power
And Phase c
sensor:
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_production_phaseC"
unit_of_measurement: W
value_template: >
{% set value = value_json["production"]["ph-c"]["p"] | int %}
{% if value <= 5 -%}
0
{% elif is_state('sun.sun','below_horizon') %}
0
{%- else -%}
{{ value }}
{%- endif %}
state_class: measurement
device_class: power
Yeah, I used customise for many reasons:
I only decided to change them yesterday.
It’s instantaneous for changing how the name displays in the dashboard and doesn’t require a reboot.
I don’t have to think about the knock on effects of changing the actual sensor name.
Changing the name of an already established sensor will start a new history record and orphan the existing history records.
The logic of the exclude filter I setup in recorder will no longer work if I choose a name that doesn’t start with mqtt_
As for the cost per hour, I already have a sensor that calculates and graphs that per minute, however, I wanted the cost calculations shown in the dashboard to be directly related to the grid import/export watts shown at this very second. I did that by adding another sensor that started with mqtt_ so it wouldn’t be recorded either.
Here is the code for my costs per hour that update every second based on the mqtt sensors.
template:
- sensor:
name: mqtt_Instant Cost Per Hour
state_class: measurement
unit_of_measurement: AUD
device_class: monetary
state: >
{% set hourly = 0.7400 / 24 %}
{% set gridout = states('sensor.MQTT_Export_Power') | int %}
{% set gridin = states('sensor.MQTT_Import_Power') | int %}
{% if gridout > 0 %}
{{ ((gridout/1000) * states('sensor.tariff_price_export') | float + hourly) | round(2) }}
{% else %}
{{ ((gridin/1000) * states('sensor.tariff_price') | float + hourly) | round(2) }}
{% endif %}
Funnily enough, I only just changed power plans to a new retailer which has an even more slightly complicated solar export reimbursement structure. How I achieved that is worthy of its own seperate topic.
First of all, thank you for your amazing script. It really fills out what others have done to integrate the Envoy.
I’m wondering whether it would be more useful for the production and consumption sensors to be the summation of all the phases, and could be a useful thing to add to your Github examples too. What do you think? (Yes, I’m fishing for you to come up with the template code to do that - I don’t trust myself with that yet!).
Safe to say you’ve knocked that one on the head then! I presume the use of Customize to rename the sensors in this way doesn’t mean those new sensor names will have their history recorded, only the original names?
Beautiful - thank you. By “a sensor that calculates and graphs that per minute”, do you mean one of the sensors you’ve already documented up at the top of thread? My next task is to implement the costings.
I am very fortunate that my current supplier (Momentum) has a scheme that is very similar to your previous supplier, with only some minor adjustments to the times. My previous supplier has different tariffs depending on the time of year as well, and would be a lot more difficult to code for!
Thanks again.
By design, the script simply takes the JSON output from the Envoy and reformats to mqtt - no further calculations or manipulation is done in the script as the logical place to do that is in the system that is using the mqtt data (because there’s an infinite number of possibilities) .
It’s quite simple to sum the phases using a value template - here’s one for you to try and hopefully for you to learn.
sensor:
- platform: mqtt
state_topic: "/envoy/json"
name: "mqtt_production"
unit_of_measurement: W
value_template: >
{% set value = value_json["production"]["ph-a"]["p"] | int + value_json["production"]["ph-b"]["p"] | int + value_json["production"]["ph-c"]["p"] | int %}
{% if value <= 5 -%}
0
{% elif is_state('sun.sun','below_horizon') %}
0
{%- else -%}
{{ value }}
{%- endif %}
state_class: measurement
device_class: power
If you wanted Consumption , change the occurrences of “production” to “consumption”
Thank you! That is perfect. I will do that for production and consumption (and learn from the example).
I have started another topic for the costings as it is very large and still a work in progress
see
Wow - great work! You have a real knack in explaining things logically and as simply as possible. I’ll have to decide whether to go the whole hog with your new stuff for more future-proofing, or use the existing stuff in this thread (because it is very similar to my current plan).
Any chance you could address the other two minor queries in my previous reply to you?
1 Like
Hi @Sddawson, Thanks for the compliment. I guess we have all been there when you need to solve a problem and the available information is vague. I try to not be so vague in order to help people 
Was it these questions you meant?
The customize.yaml just changes how its displayed to the user.
Behind the scenes, its all still based on the actual name.
This way, the exclusions in recorder are still respected and any other sensors that reference the original name are not upset.
Correct, its the same formula for the per minute sensor as it is for the per second sensor. The only difference is that im not recording and graphing the per second results. I am purely using the per second results for real time dashboard.
For reference here is my per minute sensor that is recorded and graphed per minute
- sensor:
name: Instant Cost Per Hour
state_class: measurement
unit_of_measurement: AUD
device_class: monetary
state: >
{% set hourly = 0.7400 / 24 %}
{% set gridout = states('sensor.grid_export_power') | int %}
{% set gridin = states('sensor.grid_import_power') | int %}
{% if gridout > 0 %}
{{ ((gridout/1000) * states('sensor.tariff_price_export') | float + hourly) | round(2) }}
{% else %}
{{ ((gridin/1000) * states('sensor.tariff_price') | float + hourly) | round(2) }}
{% endif %}
Here is my per second sensor that is not recorded or graphed at all
- sensor:
name: mqtt_Instant Cost Per Hour
state_class: measurement
unit_of_measurement: AUD
device_class: monetary
state: >
{% set hourly = 0.7400 / 24 %}
{% set gridout = states('sensor.MQTT_Export_Power') | int %}
{% set gridin = states('sensor.MQTT_Import_Power') | int %}
{% if gridout > 0 %}
{{ ((gridout/1000) * states('sensor.tariff_price_export') | float + hourly) | round(2) }}
{% else %}
{{ ((gridin/1000) * states('sensor.tariff_price') | float + hourly) | round(2) }}
{% endif %}
Not sure if it was mentioned. I just set this up and I had to use | abs to get absolute values on my consumption, otherwise it added consumption and production together with this:
{{ [0, states('sensor.envoy_xxxx_current_power_production') | int - states('sensor.envoy_xxxx_current_power_consumption') | int ] | max }}
Lets break down the formulas to their most basic point.
You will notice the minus symbol in the middle of the 3 lines of template code in the images below.
This is meant to subtract production from consumption to calculate grid export.
This positive result on the right shows that I am producing 727 W more than what I am consuming and therefore exporting 727 W of excess power TO the grid.
This negative result on the right shows that I am consuming more than I am producing and therefore pulling 23 W of power FROM the grid.
I don’t want this negative number to be recorded in my grid export sensor as I have a different grid import sensor to record that.
The images above have the same formula in each screenshot.
{{
states('sensor.envoy_SERIALNUMBER_current_power_production') | int
-
states('sensor.envoy_SERIALNUMBER_current_power_consumption') | int
}}
All I have done is added this extra formula below to ignore any negative number below 0.
{{
[0,
states('sensor.envoy_SERIALNUMBER_current_power_production') | int
-
states('sensor.envoy_SERIALNUMBER_current_power_consumption') | int
] | max
}}
In plain english, the first bit of the code is saying here are 2 numbers [0 and 727] for example.
The last bit of the code is saying only record the bigger number between [0 and 727].
If the number is -23 for example, then it will only record 0 instead as that is the bigger number (max).
This code is what make it possible for me to have 1 sensor for grid export and another separate sensor for grid import.
If you want ‘consumption’ you are focusing on the wrong sensor.
This is the grid import ONLY (excess consumption) sensor.
Only this sensor will show grid consumption.
The code you have posted …
… is from my export ONLY (excess production) and therefore you will see 0 instead of the actual consumption amount. This is by design and intentional in order to separate grid import and grid export into 2 different sensors.
I’m just saying, in my envoy installation, current power consumption is a negative number, so you are subtracting a negative number, minus and minus is plus, so it got added. Which is why I had to take the absolute value to make the subtraction work.
template:
- sensor:
name: Grid Import Power
state_class: measurement
icon: mdi:transmission-tower
unit_of_measurement: W
device_class: power
state: >
{{ [0, states('sensor.envoy_xxx_current_power_consumption') | int | abs - states('sensor.envoy_xxx_current_power_production') | int | abs ] | max }}
- sensor:
name: Grid Export Power
state_class: measurement
icon: mdi:transmission-tower
unit_of_measurement: W
device_class: power
state: >
{{ [0, states('sensor.envoy_xxx_current_power_production') | int | abs - states('sensor.envoy_xxx_current_power_consumption') | int | abs ] | max }}
Interesting, I’ve been following this thread for a bit, and just got my enphase envoy activated this week.
When using the provided templates + sensors as-is from #1, I always get 0 values.
In debugging further, it seems the raw data I am getting from the envoy is incorrect, as my consumption and production values are always identical, which explains the zeroing out values.
Template Editor
{% set production_life = states('sensor.envoy_SERIAL_lifetime_energy_production') | int %}
{% set consumption_life = states('sensor.envoy_SERIAL_lifetime_energy_consumption') | int %}
Production Life = {{ production_life }}
Consumption Life = {{ consumption_life }}
Net Life = {{ production_life - consumption_life }}
{% set production_current = states('sensor.envoy_SERIAL_current_power_production') | int %}
{% set consumption_current = states('sensor.envoy_SERIAL_current_power_consumption') | int %}
Production Current = {{ production_current }}
Consumption Current = {{ consumption_current }}
Net Current = {{ production_current - consumption_current }}
{% set production_today = states('sensor.envoy_SERIAL_today_s_power_production') | int %}
{% set consumption_today = states('sensor.envoy_SERIAL_today__power_consumption') | int %}
Production Today = {{ production_today }}
Consumption Today = {{ consumption_today }}
Net Today = {{ production_today - consumption_today }}
Results:
Production Life = 101059
Consumption Life = 101059
Net Life = 0
Production Current = 1010
Consumption Current = 1010
Net Current = 0
Production Today = 0
Consumption Today = 0
Net Today = 0
I’m now wondering if our system was installed incorrectly. Also when browsing http://envoy.local I do see a warning that consumption metering is not enabled (only production metering is enabled) but I called and asked my solar installer and they said to ignore that. Perhaps I need to reach out to enphase support?
Edit - sorry my bad Consumption is negative and production positive …