Duke Energy Gateway - Realtime Smart Meter Readings

I’m creating this thread to help facilitate a means of integrating Duke Energy’s Gateway with real-time meter readings into Home Assistant

Duke Energy Gateway Integration by MJMeli:

Unboxing and setup:

The device (DK-GW001-A01-C001):
https://fccid.io/2AUYW-A010001/

Internal photos:
https://fccid.io/2AUYW-A010001/Internal-Photos/EUT-Internal-Photo-4695782

Hardware Summary:
Intel Atom e3805 1.33Ghz CPU

16GB eMMC storage
4GB DDR3 RAM
USB(3?) port
PCIe mini slot (empty)
SATA port (only solder pad, no connector)
Sim Card slot (empty)
WiFi11ac (2.4Ghz+5Ghz), Bluetooth 4.1, ZigBee, Z-wave, Ethernet connectivity
Gateway average energy usage, 3.5-5watts

This is currently a pilot program, and the gateway is free if you qualify. Apply for the pilot program here:

Thanks to Alfiro for the information on this program:

3 Likes

I got my gateway recently, and did a quick port scan before pairing the device with the app. Ports 22, and 9330 were open. After pairing I couldn’t find any open ports.

I wasn’t able to boot off the the USB port with a bootable USB stick. The eMMC on the gateway is soldered in, so reading directly from it will take some work:
https://www.riverloopsecurity.com/blog/2020/03/hw-101-emmc/

I too got one of these recently after a lone email from Duke Energy asking if I wanted to participate. I am both intrigued and bothered by this thing. First, it looks as if it has wifi, bluetooth, z-wave, and zigbee compatability, in addition to the ethernet. And, it packs a significant amount of horsepower for what it is supposed to be doing (Only tracking my energy usage?). Why so robust a device? That is what bothers me. Now, I am speculating here, but could they be using this thing to monitor my usage as well as some of my neighbors? I am not sure how they bill now, but I am assuming that they use some kind of mesh network to feed their servers, and that all the meters around me talk to each other. I put a monitor on this thing and it uses about 3.5-5w of power. Which in my neck of the woods would cost me about $5 annually. I also did a packet capture on this and it is hitting a bunch of AWS servers, but also a server in Japan (University of Tsukuba,Tsukuba, Ibaraki, Japan) every so often. That’s all I have so far.

Yeah, enabling WiFi is one of the main reasons I want to get into this thing. This device is decked out with antennas and connectivity options. 3-15w is a bit much for a device that only gives me real-time energy stats. I don’t know if this is the final design or just a very capable prototype that gives them extra flexibility in trying things out.

I think the Japanese server is because the Duke IT guys are using SoftEther VPN (Open Source out of University of Tsukuba) to tunnel into the devices.
https://www.softether.org/9-about

I opened my gateway up and was sad to see that the SATA connector that is next to the ethernet port in the FCC photo, is missing. The pad is there but the connector was not soldered in.

I was able to more clearly see the underside though. There is a PCIe Mini interface (8+18pin), along side a sim card slot. This is probably for cellular board addon.

After reading more about the Intel Atom, I see that it uses a 32bit EFI to boot even though the processor is 64bit. So I tried making a bootable USB with Kali Live (32bit iso) with SSH server enabled. The gateway still failed to boot from the USB. Disconnecting the CMOS battery to clear the bios settings did nothing, still no USB boot.

One option might be to get a SATA connector and solder it onto the pads in hopes that SATA takes boot priority.

EDIT: after talking it over, I corrected my post to say it’s an ePCI mini slot not mSATA (identical connectors).

1 Like

I started poking around today to try and see if there was a way to pull the data into HA. It looks like the real time usage data comes through an MQTT websockets connection (looks like it is via AWS IoT). Unfortunately, I didn’t have any luck in getting a client subscribed to the broker due to some SSL related issues.

I did also note that there is a more traditional REST API endpoint that returns minute-by-minute kWh usage (which is what is displayed in the minute graph in the app). This is easily retrievable and might be a good first step. I’m working through writing something up now, probably a HACS integration. We’ll see how it goes…

1 Like

Awesome, I’m glad to hear a way forward :slight_smile:

I noticed today that when the gateway is booting up ports 22 and 9330 will be open for about 10seconds. I tried SSH and got a connection but couldn’t login without a username/password. The easiest way to test this is to continuously ping the device on power up. As soon as you get a successful ping (~100s after power on) ports 22 and 9330 will briefly open and close

The device also responds to a keyboard attached to the USB port. Holding down ‘delete’ (may open the BIOS) on startup prevents the device from booting up. When the device is up and running pressing different key combinations causes the meter connection to break or the device to reboot.

I have a early version up and running. The repo is here and can be added as a custom repository in HACS: GitHub - mjmeli/ha-duke-energy-gateway: Uses the Duke Energy Gateway to import near real time energy u

I’ve been working through a few bugs but it seems to be pretty stable now. When you configure it with your email/password, it will attempt to find your meter automatically. Hopefully it does.

It creates an entity that tracks the current day’s total energy usage. That data comes from summing the minute-by-minute data since midnight local time. The meter will reset at midnight.
image

There’s one big issue with the API though. Unfortunately while the data is technically at the minute level, it looks like they are caching the data for 15 minutes so there will be delays of 15 minutes between readings :(. That means data could appear attributed to the wrong hour (e.g. if a reading was at 6:55, we won’t get the next reading until 7:10, so the 5 minutes from 6:55-7:00 will be attributed to the 7:00 hour). The same issue exists at midnight and means losing data.

However overall this is a big improvement over the options before as there was no way to get “today’s” data at all. The hour-by-hour data in the energy dashboard is pretty close to what I see in the app.

And obviously this API is completely unofficial so I’d suspect some brittleness.

Also note there’s a breaking change coming in 2021.9 with how meters are tracked. I’ll have to put a fix together before updating. That probably means issues if you are on beta.

4 Likes

Mine is on it’s way, so I should be able to help out with the integration shortly.

2 Likes

I got mine yesterday. Setup was easy with the app. It shows the gateway and data in the Duke App.

When I setup everything in Home Assistant it doesn’t seem to find it. I get the following:

2021-09-04 19:14:17 ERROR (MainThread) [custom_components.duke_energy_gateway] Could not identify a smart meter on your account with gateway access.

Any thoughts?

What version of HA Core and what version of the Duke Energy integration are you using?

I wasn’t able to get v0.0.8 of the Duke Energy integration to work until I upgraded the core to the September release. If you are on a pre September release, try v0.0.7

Yeah Home Assistant has a breaking change in the 2021.9 update for how the measurements are tracked and the adjustments are in v0.0.8, so v0.0.8 is only compatible with 2021.9.

As for the issue finding the smart meter, it probably means there is something different with your account or the data coming back than is expected at the moment. To help debug through this in the past I’ve recommended getting the debug logs. You can do that by adding this to your configuration.yaml

logger:
  default: info
  logs:
    custom_components.duke_energy_gateway: debug

Then in your full home-assistant.log file you should see some more verbose logging.

Is it possible to pull the kw/h cost from the app? I’m entering it in manually, but it would be nice to pull it in automatically so that it’s adjusted whenever there is a rate change.

I couldn’t find the rates anywhere in the app. I had to pull my rate plan from a bill and then find a document with the rates.

I did create a template sensor for it as the rate is different for half the year. But if they ever change the rates themselves I’ll have to update manually.

template:
  # Rate is higher from July 1 - Oct 31.
  - sensor:
    - name: "Duke Energy Charge per kWh"
      unique_id: duke_energy_charge_per_kwh
      unit_of_measurement: USD/kWh
      icon: mdi:currency-usd
      device_class: monetary
      state: >
        {{ 0.10772 if now().month >= 7 or now().month <= 10 else 0.10271 }}
2 Likes

Thank you, that template will work nicely!

It took me a little while to get back to this. I updated to the latest 2021.9. Added the debug and got the following:

2021-09-08 05:21:37 DEBUG (MainThread) [custom_components.duke_energy_gateway] Failed to find meter on account 6143399001: Request failed with unexpected error [https://cust-pilot-api.duke-energy.com/smartmeter/v1/auth]: 500, message=‘Internal Server Error’, url=URL(‘https://cust-pilot-api.duke-energy.com/smartmeter/v1/auth’)

As I said before I can see the Gateway in the app. Unsure why it can’t find the meter.

Any thoughts?

I pushed out a new version this morning based on a similar issue report, 0.0.9. Try updating to that version and see if that fixes your issue.

That seems to have fixed it. Now to give it some time. Thanks.

I have this sensor working now. It is reporting 0.10772 as it should.

Under Configuration, Energy…

I selected “Use an entity with current price” and selected this sensor. No errors.

The cost has been setting at zero for days on the Energy page even though I do have kWh.

Got any thoughts? Bug? Or am I missing something?