@dydx, @Han_Solo see above – as new user, I cannot tag more than two people in a post…
Thanks @jgruen . I’ve actually done something very similar but doing it on the device itself at high resolution. No HA required. I can share details if interested
I’m HIGHLY interested in your solution. I’m running jgruen’s code, but I’m about a year behind you guys and just now starting to try to figure out why net metering doesn’t work.
PG&E w/ 12kw of solar and an Aclara I210+ meter.
I also have a service ticket in to Emporia asking for the protocol documentation for the serial link between the Zigbee transceiver and the ESP32. Never hurts to ask nicely.
@jgruen I’ll take a look at what you came up with. Mine still accumulates errors in either direction with how much I export. Have you tested and compared with what your utility reports?
Here is my yaml:
sensor:
- platform: uptime
name: Uptime Sensor
- platform: custom
lambda: |-
auto vue = new EmporiaVueUtility(id(emporia_uart));
App.register_component(vue);
return {vue->kWh_consumed, vue->kWh_returned, vue->W, vue->kWh_net};
sensors:
- id: kWh_consumed_raw
internal: true
- id: kWh_returned_raw
internal: true
- id: watts_raw
internal: true
- id: kWh_net_raw
internal: true
- platform: copy
id: watts_produced_raw
source_id: watts_raw
internal: true
filters:
- lambda: 'return max(-x, 0.0f);'
# - multiply: 0.001
unit_of_measurement: W
- platform: copy
id: watts_consumed_raw
source_id: watts_raw
internal: true
filters:
- lambda: 'return max(x, 0.0f);'
unit_of_measurement: W
- platform: total_daily_energy
id: kWh_returned
power_id: watts_produced_raw
name: "kWh Returned"
unit_of_measurement: "kWh"
accuracy_decimals: 3
state_class: total_increasing
device_class: energy
# Reduce the rate of reporting the value to
# once every 5 minutes and/or when 0.1 kwh
# have been returned, unless the fast_reporting
# button has been pushed
filters:
- multiply: 0.001
- or:
- throttle: 5min
- delta: 0.1 # <- kWh
- lambda: |-
if (id(fast_reporting)) return(x);
return {};
on_raw_value:
then:
lambda: |-
ESP_LOGI("Vue", "Wh = %0.3f", x);
- platform: copy
id: kWh_consumed
source_id: kWh_consumed_raw
name: "kWh Consumed"
unit_of_measurement: "kWh"
accuracy_decimals: 3
state_class: total_increasing
device_class: energy
# Reduce the rate of reporting the value to
# once every 5 minutes and/or when 0.1 kwh
# have been consumed, unless the fast_reporting
# button has been pushed
filters:
- or:
- throttle: 5min
- delta: 0.1 # <- kWh
- lambda: |-
if (id(fast_reporting)) return(x);
return {};
on_raw_value:
then:
lambda: |-
ESP_LOGI("Vue", "kWh = %0.3f", x);
- platform: copy
name: "Watts Consumed"
id: watts_consumed
source_id: watts_consumed_raw
unit_of_measurement: "W"
accuracy_decimals: 0
state_class: measurement
device_class: power
# Report every 5 minutes or when +/- 20 watts
filters:
- or:
- throttle: 5min
- delta: 20 # <- watts
- lambda: |-
if (id(fast_reporting)) return(x);
return {};
on_raw_value:
then:
lambda: |-
ESP_LOGI("Vue", "Watts = %0.3f", x);
- platform: copy
name: "Watts Returned"
id: watts_returned
source_id: watts_produced_raw
unit_of_measurement: "W"
accuracy_decimals: 0
state_class: measurement
device_class: power
# Report every 5 minutes or when +/- 20 watts
filters:
- or:
- throttle: 5min
- delta: 20 # <- watts
- lambda: |-
if (id(fast_reporting)) return(x);
return {};
on_raw_value:
then:
lambda: |-
ESP_LOGI("Vue", "Watts = %0.3f", x);
- platform: copy
name: "kWh Net"
id: kWh_net
source_id: kWh_net_raw
unit_of_measurement: "kWh"
accuracy_decimals: 3
device_class: energy
# Reduce the rate of reporting the value to
# once every 5 minutes and/or when 0.1 kwh
# have been consumed or returned, unless the fast_reporting
# button has been pushed
filters:
- or:
- throttle: 5min
- delta: 0.1 # <- kWh
- lambda: |-
if (id(fast_reporting)) return(x);
return {};
on_raw_value:
then:
lambda: |-
ESP_LOGI("Vue", "kWh = %0.3f", x);
As you can see, I am getting all the values first as internal “raw” values. The energy consumed I can directly copy (with some filters) because it is already correct. But for kWh_returned I use total_daily_energy to calculate it from watts_produced_raw directly. watts_produced_raw is just defined as return max(-x, 0.0f); where x is the raw watts value. That’s because the Vue correctly sets power negative when pushing back to the grid (it just doesn’t decrease energy). So now the produced (sent back to grid) power is just the absolute value if it is negative or zero otherwise (it watts>0, we are pulling from the grid). This positive value can now be integrated.
Note I am really working with raw values at the maximum native speed directly on the ESP. This means it is as accurate as it can be given the circumstances. No extra interpolation happens and integration does not happen in HA, where it would get inaccurate if HA dies etc.
Embarrasingly, I’m completely unable to get this yaml config to even compile. I suspect it has to do with indentation of the yaml: I’ve tried a doen ways to line things up, but my command of the “sensors” component of esphome and the way lambdas integrate isn’t sufficient to make it work.
Is there any way you can paste or link to a correctly formatted file?
Thanks!
John
I need to vent. This Zigbee Smartmeter crap is the most frustrating experience ever!
I had it running for nearly a year. Here and there I had to call PG&E to re-provision. Then in August it went downhill. The meter was just reporting the exact same number.
I spent tens of calls, hours on the phone with PG&E. They replaced the meter. New meter first did not connect at all. Now it’s connecting and reporting. Emporia sent me TWO new devices. PG&E is blaming Emporia. Emporia is blaming PG&E. I had PG&E even replace the meter but apart from repreovision for thousand times nothing. And Emporia, apart from sending me yet a FOURTH device, no insight whatsoever. And zero opportunity to debug myself because of this closed proprietary crap. Fuck this.
I had a like problem but because it was using the esphone firmware I had access to the logs showing it was trying to connect and need to be reprovisioned. Send the log to support and the next day it magically started working
Good luck
As others mentioned, kWh Returned sensor is not updated with the solar production and is always zero.
It’s possible to calculate the imported/exported power (W) and imported/exported energy (kWh) just from one sensor value (Watts or sensor.vue_utility_watts) with the following code in your HA configuration.yaml file (restart needed):
template:
- sensor:
- name: "Grid Imported Power (ESPHome)"
unit_of_measurement: "W"
state: >
{{ max(states.sensor.vue_utility_watts.state | int, 0) | default(0)}}
device_class: power
- name: "Grid Exported Power (ESPHome)"
unit_of_measurement: "W"
state: >
{{ min(states.sensor.vue_utility_watts.state | int, 0) | abs | default(0)}}
device_class: power
sensor:
- platform: integration
source: sensor.grid_imported_power_esphome
name: Grid Imported Energy (ESPHome)
unit_prefix: k
unit_time: h
method: left
round: 2
- platform: integration
source: sensor.grid_exported_power_esphome
name: Grid Exported Energy (ESPHome)
unit_prefix: k
unit_time: h
method: left
round: 2
See Integration - Riemann sum integral - Home Assistant for more details.
For real-time display, you can use GitHub - flixlix/power-flow-card-plus: A power distribution card inspired by the official Energy Distribution card for Home Assistant and the Watts sensor provided by ESPHome integration. If you also have a sensor for your solar, it will automatically display the total consumption of your house (solar + grid).
For the Energy dashboard, use the Grid Imported Energy (ESPHome) and Grid Exported Energy (ESPHome) sensors.
I’ve been using this method with the Cloud Emporia Vue integration and 1-minute sensor. It should work the same (or better) with the ESPHome version.
Just flashed my device with the ESPHome firmware as the official API has been down for a whole day. Hope it will be more stable.
Flashing was tricky with my specific USB-UART adapter and only worked with --before usb_reset esptool option for some reason.
I used the test hook clips to avoid soldering. Needs some tinkering to put inside the holes, but if done properly, stays there and has a reliable connection.
I dumped the original firmware with the following command just in case:
esptool --before usb_reset --port COM3 --baud 460800 read_flash 0 0x400000 vue-backup.bin
After building the firmware with esphome compile vue-utility-solar.yaml, I also had to upload it from the command line and --before usb_reset option:
esptool --before usb_reset --after hard_reset --baud 115200 --port COM3 --chip esp32 write_flash -z --flash_size detect 0x10000 "c:/projects/esphome-emporia-vue-utility/src/.esphome/build/vue-utility/.pioenvs/vue-utility/firmware.bin" 0x1000 "C:/projects/esphome-emporia-vue-utility/src/.esphome/build/vue-utility/.pioenvs/vue-utility/bootloader.bin" 0x8000 "c:/projects/esphome-emporia-vue-utility/src/.esphome/build/vue-utility/.pioenvs/vue-utility/partitions.bin" 0xe000 "C:/Users/serge/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin"
There is probably an easier way, but that is what worked for me. Hope it will help someone else.
3 Likes
I flashed my vue utility with esphome after getting frustrated with the spottiness of emproia’s API. After the flash it’s not able to connect to the meter. In my logs, I see this:
[16:46:54][I][Vue:503]: MGM Firmware Version: 4
[16:46:54][I][Vue:504]: MGM Mac Address: removed_by_me
[16:46:54][I][Vue:505]: MGM Install Code: removed_by_me (secret)
[16:46:54][I][Vue:506]: Trying to re-join the meter. If you continue to see this message
[16:46:54][I][Vue:507]: you may need to move the device closer to your power meter or
[16:46:54][I][Vue:508]: contact your utililty and ask them to reprovision the device.
[16:46:54][I][Vue:509]: Also confirm that the above mac address & install code match
[16:46:54][I][Vue:510]: what is printed on your device.
[16:46:54][E][Vue:511]: You can also file a bug at
[16:46:54][E][Vue:512]: https://forms.gle/duMdU2i7wWHdbK5TA
Did you folks have to call the utility to repair the meter after flashing esphome? I was impatient and did not take a backup of the stock firmware, so I have no way no of going back.
It’s not uncommon to have to have the utility co rejoin the unit. I use the ESP solution and had to do it a number of times.
For me, it worked out of the box after moving from the stock firmware. Not sure how long it will last before I need to call to re-join it.
Does the issue persist if you move the dongle closer to the meter?
The vue-utility is in the same location as before. It’s maybe 10-15 ft from the meter, separated by an outside wall. It worked out of the box with the stock firmware after a call to PGE, I guess I’ll wait till business hours tomorrow and have them attempt to reprovision the device again.
SDGE uses an on-line site (energy.ceiva.com) where I can remove/add devices and see their status and MAC address. My Vue device is still listed there, and I’m happy I don’t have to call anywhere. If it stops working, I should be able to add it again on that site. I’m surprised PGE doesn’t have a similar service.
PGE does have an online way to do this with a service they call Stream My Data. There are caveats with it - it does not work if you have grid tied solar or your primary provider is a reseller of PGE. I happen to fit both these boxes, so I called in to see if they can help. They were helpful and tried to repair the device twice, but my vue does not pick up the meter. I left the vue on since the call in the morning and it still hasn’t picked up anything.
Looking through the source code, I see that handle_resp_meter_join is the function that logs “Got meter join response”. This function is called only when the serial communication gets a meter request, so I assume the esphome code works. It’s probably something broken on the meter side, or some other unexplainable error. Either way - this was broken before with the stock firmware and continues to be broken with esphome.
If someone here can share their dump of the original emporia firmware, that would be great. I really wish I took the extra minute to backup the firmware before wiping it.
Sadly, SDGE is stopping the HAN devices program:
It will work until the Vue device dies or until the meter is replaced. Then we’ll have to switch to the clamp-based smart meters 
This is crazy. I thought the utilities were using the HAN functionality to remotely log the meter usage. The tech is out there and is relatively maintenance free, I don’t see why SDGE wants to shut down the program.
Posted elsewhere, but it doesn’t appear that CPUC has rescinded the requirement that the IOUs support these HAN devices (entirely possible I missed something.)
Given that, it would be appropriate to file a complaint with the CPUC: Utility Complaint
Worst case, the CPUC will clarify that this requirement has been dropped, best case, SDG&E will be told that they must continue to support HAN devices.
The HAN radio is completely separate from the upstream data radio. As a meter manufacturer, we did the absolute bare minimum amount of work on the HAN side; the majority of our development and testing efforts went into upstream network connections (and the flip-side, delivering commands like remote disconnect TO the meter).
Most of the time, the HAN radio was a chip off on the side of the board with a limited serial connection to the main CPU, and it could query certain memory registers in the CPU.
OTOH, none of my code in in any of the meters in SCE, PG&E, or SDG&E, so I can’t say for sure. If you live in Georgia, I can be a little more definitive.