Yep, the MGM111 only responds to queries, it doesn’t send unsolicited data. Not a problem though, you still have the serial port you used to program the device, and since you can flash it over wifi from now on, you don’t need the port.
Change your config to:
logger:
baud_rate: 0 # Disable logging over serial
uart:
id: emporia_uart
rx_pin: GPIO34
tx_pin: GPIO35
baud_rate: 115200
Then connect P3 pin 4 to P5 pin 2 and P3 pin 5 to P5 pin 1 and you should be back in business.
hey @jrouvier thanks for that awesome lateral thinking idea! I actually solved my issue with a bit of flux and a soldering iron! and now im getting data!
[11:43:22][D][Vue:487]: Sending request for meter reading
[11:43:22][D][uart_debug:114]: >>> 24:72:0D
[11:43:27][D][Vue:487]: Sending request for meter reading
[11:43:27][D][uart_debug:114]: >>> 24:72:0D
[11:43:30][VV][api.service:470]: on_ping_request: PingRequest {}
[11:43:30][VV][api.service:043]: send_ping_response: PingResponse {}
[11:43:32][D][Vue:487]: Sending request for meter reading
[11:43:32][D][uart_debug:114]: >>> 24:72:0D
[11:43:37][I][Vue:494]: MGM Firmware Version: 2
[11:43:37][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:24:F7
[11:43:37][I][Vue:496]: MGM Install Code: #########(secret)
[11:43:37][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[11:43:37][I][Vue:498]: you may need to move the device closer to your power meter or
[11:43:37][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[11:43:37][I][Vue:500]: Also confirm that the above mac address & install code match
[11:43:37][I][Vue:501]: what is printed on your device.
[11:43:37][E][Vue:502]: You can also file a bug at
[11:43:37][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[11:43:37][V][component:200]: Component <unknown> took a long time for an operation (0.07 s).
[11:43:37][V][component:201]: Components should block for at most 20-30ms.
[11:43:37][D][uart_debug:114]: >>> 24:6A:0D
[11:43:37][D][Vue:436]: Got meter join response
[11:43:37][D][uart_debug:114]: <<< 24:01:6A:01:01:0D
[11:43:42][D][Vue:487]: Sending request for meter reading
[11:43:42][D][uart_debug:114]: >>> 24:72:0D
I am having trouble getting it to join my meter, my utility has a web interface that allows me to initiate the pairing of HAN devices. I’ve put the correct MAC and install code but having no luck. The meter is only a couple of metres away through a stud wall
If you have had it previously connected to a different meter, you may need to just call the utility to get them to help. I had previously connected mine at this house, using the web interface, but after disconnecting it once it would not reconnect via the utility’s web interface. I had to call to get it connected.
Hi @jrouvier I am finally back home after a month so I could try this.
Here is the output:
Found multiple options, please choose one:
[1] /dev/ttyACM0 (F5521gw - F5521gw Mobile Broadband Modem)
[2] /dev/ttyACM1 (F5521gw - F5521gw Mobile Broadband Data Modem)
[3] /dev/ttyACM2 (F5521gw - F5521gw Mobile Broadband GPS Port)
[4] MQTT (10.227.79.10)
(number): [15:21:13][D][Vue:282]: Meter Response Bytes 148 to 151: 29 24 05 ae
[15:21:14][D][Vue:487]: Sending request for meter reading
[15:21:14][I][Vue:072]: kWh = 101.272
[15:21:14][I][Vue:094]: kWh = 0.024
[15:21:14][I][Vue:134]: kWh = 116164.516
[15:21:14][I][Vue:113]: Watts = -650.000
[15:21:14][D][Vue:264]: Meter Divisor: 1
[15:21:14][D][Vue:265]: Meter Cost Unit: 1000
[15:21:14][D][Vue:266]: Meter Flags: fb fb
[15:21:14][D][Vue:267]: Meter Energy Flags: 06
[15:21:14][D][Vue:268]: Meter Power Flags: 00
[15:21:14][D][Vue:270]: Meter Timestamp: 2919578
[15:21:14][D][Vue:271]: Meter Energy: 116164.512kWh
[15:21:14][D][Vue:272]: Meter Power: -650W
[15:21:14][D][Vue:282]: Meter Response Bytes 4 to 7: 06 ec 87 a2
[15:21:14][D][Vue:282]: Meter Response Bytes 44 to 47: 00 00 00 01
[15:21:14][D][Vue:282]: Meter Response Bytes 48 to 51: 00 00 03 e8
[15:21:14][D][Vue:282]: Meter Response Bytes 52 to 55: fb fb 00 00
[15:21:14][D][Vue:282]: Meter Response Bytes 56 to 59: 00 ff fd 75
[15:21:14][D][Vue:282]: Meter Response Bytes 148 to 151: 87 37 05 ae
[15:21:19][D][Vue:487]: Sending request for meter reading
[15:21:21][I][Vue:072]: kWh = 101.272
[15:21:21][I][Vue:094]: kWh = 0.024
[15:21:21][I][Vue:134]: kWh = 116164.516
[15:21:21][I][Vue:113]: Watts = -687.000
[15:21:21][D][sensor:127]: 'Watts': Sending state -687.00000 W with 0 decimals of accuracy
[15:21:21][D][Vue:264]: Meter Divisor: 1
[15:21:21][D][Vue:265]: Meter Cost Unit: 1000
[15:21:21][D][Vue:266]: Meter Flags: fb fb
[15:21:21][D][Vue:267]: Meter Energy Flags: 06
[15:21:21][D][Vue:268]: Meter Power Flags: 00
[15:21:21][D][Vue:270]: Meter Timestamp: 2919585
[15:21:21][D][Vue:271]: Meter Energy: 116164.512kWh
[15:21:21][D][Vue:272]: Meter Power: -687W
[15:21:21][D][Vue:282]: Meter Response Bytes 4 to 7: 06 ec 87 a2
[15:21:21][D][Vue:282]: Meter Response Bytes 44 to 47: 00 00 00 01
[15:21:21][D][Vue:282]: Meter Response Bytes 48 to 51: 00 00 03 e8
[15:21:21][D][Vue:282]: Meter Response Bytes 52 to 55: fb fb 00 00
[15:21:21][D][Vue:282]: Meter Response Bytes 56 to 59: 00 ff fd 50
[15:21:21][D][Vue:282]: Meter Response Bytes 148 to 151: 46 53 05 ae
[15:21:24][D][Vue:487]: Sending request for meter reading
[15:21:29][D][Vue:487]: Sending request for meter reading
[15:21:34][D][Vue:487]: Sending request for meter reading
[15:21:39][D][Vue:487]: Sending request for meter reading
[15:21:44][D][Vue:487]: Sending request for meter reading
[15:21:49][D][Vue:487]: Sending request for meter reading
[15:21:54][I][Vue:494]: MGM Firmware Version: 2
[15:21:54][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:21:54][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:21:54][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:21:54][I][Vue:498]: you may need to move the device closer to your power meter or
[15:21:54][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:21:54][I][Vue:500]: Also confirm that the above mac address & install code match
[15:21:54][I][Vue:501]: what is printed on your device.
[15:21:54][E][Vue:502]: You can also file a bug at
[15:21:54][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:21:54][D][Vue:436]: Got meter join response
[15:21:59][D][Vue:487]: Sending request for meter reading
[15:22:04][D][Vue:487]: Sending request for meter reading
[15:22:09][D][Vue:487]: Sending request for meter reading
[15:22:14][D][Vue:487]: Sending request for meter reading
[15:22:19][D][Vue:487]: Sending request for meter reading
[15:22:24][D][Vue:487]: Sending request for meter reading
[15:22:29][I][Vue:494]: MGM Firmware Version: 2
[15:22:29][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:22:30][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:22:30][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:22:30][I][Vue:498]: you may need to move the device closer to your power meter or
[15:22:30][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:22:30][I][Vue:500]: Also confirm that the above mac address & install code match
[15:22:30][I][Vue:501]: what is printed on your device.
[15:22:30][E][Vue:502]: You can also file a bug at
[15:22:30][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:22:30][D][Vue:436]: Got meter join response
[15:22:34][D][Vue:487]: Sending request for meter reading
[15:22:39][D][Vue:487]: Sending request for meter reading
[15:22:44][D][Vue:487]: Sending request for meter reading
[15:22:49][D][Vue:487]: Sending request for meter reading
[15:22:52][I][Vue:072]: kWh = 101.272
[15:22:52][I][Vue:094]: kWh = 0.024
[15:22:52][I][Vue:134]: kWh = 116164.516
[15:22:52][I][Vue:113]: Watts = -690.000
[15:22:52][D][Vue:264]: Meter Divisor: 1
[15:22:52][D][Vue:265]: Meter Cost Unit: 1000
[15:22:52][D][Vue:266]: Meter Flags: fb fb
[15:22:52][D][Vue:267]: Meter Energy Flags: 06
[15:22:52][D][Vue:268]: Meter Power Flags: 00
[15:22:52][D][Vue:270]: Meter Timestamp: 2919672
[15:22:52][D][Vue:271]: Meter Energy: 116164.512kWh
[15:22:52][D][Vue:272]: Meter Power: -690W
[15:22:52][D][Vue:282]: Meter Response Bytes 4 to 7: 06 ec 87 a2
[15:22:52][D][Vue:282]: Meter Response Bytes 44 to 47: 00 00 00 01
[15:22:52][D][Vue:282]: Meter Response Bytes 48 to 51: 00 00 03 e8
[15:22:52][D][Vue:282]: Meter Response Bytes 52 to 55: fb fb 00 00
[15:22:52][D][Vue:282]: Meter Response Bytes 56 to 59: 00 ff fd 4d
[15:22:52][D][Vue:282]: Meter Response Bytes 148 to 151: 5a a7 06 ae
[15:22:54][D][Vue:487]: Sending request for meter reading
[15:22:59][D][Vue:487]: Sending request for meter reading
[15:23:04][D][Vue:487]: Sending request for meter reading
[15:23:09][D][Vue:487]: Sending request for meter reading
[15:23:14][D][Vue:487]: Sending request for meter reading
[15:23:19][D][Vue:487]: Sending request for meter reading
[15:23:24][I][Vue:494]: MGM Firmware Version: 2
[15:23:25][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:23:26][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:23:26][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:23:26][I][Vue:498]: you may need to move the device closer to your power meter or
[15:23:26][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:23:26][I][Vue:500]: Also confirm that the above mac address & install code match
[15:23:26][I][Vue:501]: what is printed on your device.
[15:23:26][E][Vue:502]: You can also file a bug at
[15:23:26][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:23:26][D][Vue:436]: Got meter join response
[15:23:29][D][Vue:487]: Sending request for meter reading
[15:23:34][D][Vue:487]: Sending request for meter reading
[15:23:39][D][Vue:487]: Sending request for meter reading
[15:23:44][D][Vue:487]: Sending request for meter reading
[15:23:49][D][Vue:487]: Sending request for meter reading
[15:23:54][D][Vue:487]: Sending request for meter reading
[15:23:59][I][Vue:494]: MGM Firmware Version: 2
[15:24:00][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:24:00][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:24:00][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:24:00][I][Vue:498]: you may need to move the device closer to your power meter or
[15:24:00][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:24:00][I][Vue:500]: Also confirm that the above mac address & install code match
[15:24:00][I][Vue:501]: what is printed on your device.
[15:24:00][E][Vue:502]: You can also file a bug at
[15:24:00][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:24:00][D][Vue:436]: Got meter join response
[15:24:04][D][Vue:487]: Sending request for meter reading
[15:24:09][D][Vue:487]: Sending request for meter reading
[15:24:14][D][Vue:487]: Sending request for meter reading
[15:24:19][D][Vue:487]: Sending request for meter reading
[15:24:24][D][Vue:487]: Sending request for meter reading
[15:24:29][D][Vue:487]: Sending request for meter reading
[15:24:34][I][Vue:494]: MGM Firmware Version: 2
[15:24:34][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:24:34][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:24:34][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:24:34][I][Vue:498]: you may need to move the device closer to your power meter or
[15:24:34][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:24:34][I][Vue:500]: Also confirm that the above mac address & install code match
[15:24:34][I][Vue:501]: what is printed on your device.
[15:24:34][E][Vue:502]: You can also file a bug at
[15:24:34][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:24:34][D][Vue:436]: Got meter join response
[15:24:39][D][Vue:487]: Sending request for meter reading
[15:24:44][D][Vue:487]: Sending request for meter reading
[15:24:49][D][Vue:487]: Sending request for meter reading
[15:24:54][D][Vue:487]: Sending request for meter reading
[15:24:59][D][Vue:487]: Sending request for meter reading
[15:25:04][D][Vue:487]: Sending request for meter reading
[15:25:09][I][Vue:494]: MGM Firmware Version: 2
[15:25:09][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:2D:A0
[15:25:09][I][Vue:496]: MGM Install Code: XX:XX:XX:XX:XX:XX:XX:XX (secret)
[15:25:09][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[15:25:09][I][Vue:498]: you may need to move the device closer to your power meter or
[15:25:09][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[15:25:09][I][Vue:500]: Also confirm that the above mac address & install code match
[15:25:09][I][Vue:501]: what is printed on your device.
[15:25:09][E][Vue:502]: You can also file a bug at
[15:25:09][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[15:25:09][D][Vue:436]: Got meter join response
[15:25:14][D][Vue:487]: Sending request for meter reading
[15:25:19][D][Vue:487]: Sending request for meter reading
[15:25:24][D][Vue:487]: Sending request for meter reading
It seems that while power val is negative and changing, EnergyVal (bytes 4-7) indeed stay the same.
Hence kwh_net can’t go down.
Based the raw response bytes from the meter, it’s not decreasing kWh when you send power back to the grid. There also isn’t some other magic field that’s this is being reported in. As a workaround I could have the code keep a running total of negative meter watts over time and decrease the reported kWh by that amount as long as the meter-reported kWh never decreases. It’s not going to be as accurate as a meter reported value though, and it’s going to reset to zero when the device reboots, causing huge spikes in your home assistant graphs.
edit: actually, the reset-to-zero on reboot will only be a problem if you use kWh Net. If you just use kWh Consumed and kWh Returned you’ll be fine.
thanks for the tip @doshea I called the utility and had a look at it but they couldnt find an issue
any ideas @jrouvier for why it might not be connecting?
This is actually my first question: Is this supposed to be the case? And if yes, what could be the reason that it is not the case?
I am not sure if you have solar yourself but based on your knowledge, does it always go down with solar/NEM? Maybe it depends on the electricity utility provider? Do you know anyone with solar and PG&E?
Having this option would be a really great workaround, yes.
I understand this can’t be as accurate because we effectively have to calculate the integral over the watts. This naturally causes inaccuracies. Especially since the meter does not provide the data in very dense time stamps. Sad, but would be a good workaround.
Just to make sure, kWh_Consumed would be unchanged and kWh_Returned you would just set as follows:
Meter Response Bytes: No unexpected non-zero bytes.
Utility: PG&E + solar
I also have:
Meter: Landis+Gyr FOCUS AXR-SD
Questions:
Does someone have vue-utility-solar working?
If so, could you please share the following?
MGM Firmware Version
Meter Flags
Meter make & model
Utility
@jrouvier, do you have any tips for debugging? I have solar and some reverse engineering experience. Perhaps it would be useful to flash the emporia firmware and observe data on the MGM11 UART? Looks like P4 or P1 might provide convenient access. Though, unfortunately, I can’t find my backup of the emporia firmware
I also wanted to give a huge thank you to @jrouvier for your amazing work here. The quality is excellent. I’ve been using it for >6 months and have saved a ton of energy waste as a result. I especially love the documentation in the repo. Thank you so much for sharing this
I am trying to debug this further and I found the following: The kWh value that’s shown directly on the meter does not match the value reported by Vue.
Example: Right now, the meter shows 15823kWh. This matches roughly with my PG&E bill (which showed a reading of 16071kWh on the last bill). Also, maybe 30min ago, the meter shows 15824kWh. This means the net metering on the meter itself works (and also on the PG&E bill).
As you can see from the logs, the number reported by the firmware is around 116164.512kWh. This is far away from the number that’s on the meter itself.
Why do they not match? This really seems fishy. Is there a way to completely reset the Vue and start from scratch? Maybe something got stuck?
(EDIT:: Actually, @ccd , is your kWh shown on the LCD somewhat consistent with the data sent back via the Vue?)
Welcome and happy to hear that I am not the only person with this issue. I meanwhile found someone else who also has Solar with NEM, PG&E and the Vue Utility Connect (but original firmware) and for him, the net energy is reported correctly.
Seems to be the same as mine.
Have you tried this by any chance? I think I have saved the original firmware bin somewhere but I am not sure how to flash it back…
Thanks again @jrouvier for your work here. Im tryinng to debug why my emporia isn’t connecting to the meter my logs were saying that I was getting a meter join response but I was never receiving any data.
So I changed emporia_vue_utility.h in the loop to include a logging line of the message:
case 'j': // Meter reading
handle_resp_meter_join();
ESP_LOG_BUFFER_HEXDUMP(TAG, input_buffer.data, msg_len, ESP_LOG_ERROR);
led_wifi(true);
if (startup_step == 3) {
send_meter_request();
startup_step++;
}
break;
So now my log output looks like this:
[17:58:33][D][Vue:487]: Sending request for meter reading
[17:58:38][I][Vue:494]: MGM Firmware Version: 2
[17:58:38][I][Vue:495]: MGM Mac Address: 70:B3:D5:EB:80:00:24:F7
[17:58:38][I][Vue:496]: MGM Install Code: (secret)
[17:58:38][I][Vue:497]: Trying to re-join the meter. If you continue to see this message
[17:58:38][I][Vue:498]: you may need to move the device closer to your power meter or
[17:58:38][I][Vue:499]: contact your utililty and ask them to reprovision the device.
[17:58:38][I][Vue:500]: Also confirm that the above mac address & install code match
[17:58:38][I][Vue:501]: what is printed on your device.
[17:58:38][E][Vue:502]: You can also file a bug at
[17:58:38][E][Vue:503]: https://forms.gle/duMdU2i7wWHdbK5TA
[17:58:38][D][Vue:436]: Got meter join response
[17:58:38][D][esp-idf:000]: E (19960411) Vue: 0x3ffb2de0 24 01 6a 01 01 0d |$.j...|
[17:58:43][D][Vue:487]: Sending request for meter reading
is that all i should be seeing from your experience?
@Han_Solo and @something-is-wrong you both seem to be Solar/NEM users. PLEASE let us know if the returned energy value (not power value) is a net value or not. I am struggling with this for months and I’m not even sure what the intended behavior is.
What I mean: does your energy value DECREASE when you send energy back to the grid or does it stay constant? In other words, if you plot sensor.kWh_net, do you get a monotonic curve or not?
@dydx No, the unfortunately the Emporia does not report decreasing energy value when I’m exporting. It reports a counter that only goes up when importing and stays constant while exporting, in other words, an import meter. It’s been a year of exporting to the grid and it is now quite different than what the meter displays, whereas it read the same value before my net metering was commissioned. The display on my meter, does as we would expect: it goes down as I’m exporting. There must be separate import and export counters in the meter and the display is showing the difference between them. I have a Landis+GYR Focus AXR-SD meter with PGE. As mentioned in this thread there doesn’t seem to be anything resembling an export counter in the payload that we’re reading with the Emporia, however the stock firmware seems to be able to read it somehow.
I have implemented a workaround that calculates exported energy over a the meter reading time interval based on the reported negative POWER that the Emporia does report. I have coded it so that this calculation only occurs if the current power is negative. HASS grabs this and accumulates it to keep track of total exported. Otherwise, when I’m producing the code will report zero exported and the import just accumulates in the meter value that Emporia grabs (what I call it the import meter).
Unfortunately, after 1 year of using code, it still underestimates the exported energy when I compare with the data that PGE has. It’s possible I have an error as I wrote it myself and I’m not a professional, I’ve been meaning to compare it with what jrouvier eventually deployed, or just download his and see.
The fork does effectively what you suggested: breaks out “Watts returned” and “Watts consumed” so that you can create Integration - Riemann sum integral sensor Helpers in Home Assistant, which can then be used in the Energy component. I called my helpers Energy consumed from grid and Energy returned to grid.
My fork also fixes timing issues, so you can e.g. enable the ESPHome Time Source in your device’s ESPHome configuration YAML.
Thanks @jgruen . I’ve actually done something very similar but doing it on the device itself at high resolution. No HA required. I can share details if interested
I’m HIGHLY interested in your solution. I’m running jgruen’s code, but I’m about a year behind you guys and just now starting to try to figure out why net metering doesn’t work.
PG&E w/ 12kw of solar and an Aclara I210+ meter.
I also have a service ticket in to Emporia asking for the protocol documentation for the serial link between the Zigbee transceiver and the ESP32. Never hurts to ask nicely.
and happy to hear that I am not the only person with this issue. I meanwhile found someone else who also has Solar with NEM, PG&E and the Vue Utility Connect (but original firmware) and for him, the net energy is reported correctly.