Powerpal smart energy monitor

Feature Requests
312

I have updated to the latest ESPHome release and the code doesn’t compile anymore.

src/esphome/components/powerpal_ble/powerpal_ble.cpp: In member function 'void esphome::powerpal_ble::Powerpal::parse_measurement_(const uint8_t*, uint16_t)':
src/esphome/components/powerpal_ble/powerpal_ble.cpp:107:13: error: 'ESPTime' is not a member of 'esphome::time'
       time::ESPTime date_of_measurement = time_->now();
             ^~~~~~~
src/esphome/components/powerpal_ble/powerpal_ble.cpp:107:13: note: suggested alternative:
In file included from src/esphome/components/time/real_time_clock.h:8,
                 from src/esphome/components/homeassistant/time/homeassistant_time.h:4,
                 from src/esphome.h:32,
                 from src/esphome/components/powerpal_ble/powerpal_ble.h:8,
                 from src/esphome/components/powerpal_ble/powerpal_ble.cpp:1:
src/esphome/core/time.h:16:8: note:   'esphome::ESPTime'
 struct ESPTime {
        ^~~~~~~
src/esphome/components/powerpal_ble/powerpal_ble.cpp:108:11: error: 'date_of_measurement' was not declared in this scope
       if (date_of_measurement.is_valid()) {
           ^~~~~~~~~~~~~~~~~~~
src/esphome/components/powerpal_ble/powerpal_ble.cpp:108:11: note: suggested alternative: 'parse_measurement_'
       if (date_of_measurement.is_valid()) {
           ^~~~~~~~~~~~~~~~~~~
           parse_measurement_
*** [/data/esphome-web-36d5b8/.pioenvs/esphome-web-36d5b8/src/esphome/components/powerpal_ble/powerpal_ble.cpp.o] Error 1
========================= [FAILED] Took 27.06 seconds =========================

I saw there is a breaking change

image
image831×308 28.1 KB

but looking at the GitHub issue I’m not sure how to fix this…

Have you experienced this?

313

Just fixed it. You can now replace the external_components section of yaml file to:

external_components:
  - source:
     type: git
     url: https://github.com/muneeb1990/esphome
     ref: powerpal_ble
    components: [ powerpal_ble ]

This will always pull the most up-to-date code without having to store a local copy of powerpal_ble folder.

5 Likes
314

Thanks. Works a treat now :slight_smile:

315

Hi @muneeb1990 - thanks for your code update. I did a bunch of testing yesterday and whilst I have success in reading the battery, firmware, …… but I can’t actually retrieve the “energy values” from the Powerpal (that being the 59DAxxxx) I believe. All looks correct (including “Writing pairing code to Powerpal”)

I have factory reset the Powerpal and retried directly, also reconnected it to the phone app - that worked.

From within nRF Connect I can’t read those values either (assume that is due to the pairing code)
But can see all the fields

IMG_0322
IMG_03221179×1093 213 KB

316

I will need to see the logs from ESP32 to figure out what’s happening. Are you sure you’re not testing it during day time where power consumption may be 0W due to solar production?

317

For those who are Emerald users, I’ve recently been able to get local bluetooth working fully using only yaml and lambda (i.e no custom components) on the latest version of Home Assistant. Thanks to @WeekendWarrior1 for his great skills on reverse engineering, it really helped me out. If there is interest, let me know and will post my yaml.

318

Good to know! Are you using host Bluetooth to connect directly to Emerald?

319

Not sure if by host bluetooth you mean on the HA server itself? If so no, I am instead using bluetooth on an ESP32C3 running esphome talking directly to the Emerald device, but only using standard components. Here is my yaml. Hope it helps others.

esphome:
  name: "power-meter-bridge"
  friendly_name: power-meter-bridge
  on_boot: #this is to avoid slow startup time because of BT and WiFi when using a single core ESP32. I am using the C3. 
    priority: 250
    then:
      - wait_until:
          wifi.connected
      - delay: 3s
      - lambda: |- 
          id(ble_tracker).set_scan_continuous(true); 
          id(ble_tracker).start_scan();

esp32:
  board: esp32-c3-devkitm-1
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_BT_BLE_50_FEATURES_SUPPORTED: y
      CONFIG_BT_BLE_42_FEATURES_SUPPORTED: y
      CONFIG_ESP_TASK_WDT: y
      CONFIG_ESP_TASK_WDT_TIMEOUT_S: "10"

logger:

preferences:
    flash_write_interval: 10min

globals:
  - id: total_pulses_global
    type: uint64_t
    restore_value: yes
    initial_value: '0'      
  - id: daily_pulses_global
    type: uint64_t
    restore_value: yes
    initial_value: '0' 
  - id: day_of_last_measurement_global
    type: uint16_t
    restore_value: yes
    initial_value: '0' 

time:
  - platform: homeassistant
    id: esptime  

esp32_ble_tracker:
  id: ble_tracker
  scan_parameters:
    continuous: false

ble_client:
  mac_address: aa:bb:cc:dd:ee:ff
  id: emerald
  on_passkey_request:
      then:
        - ble_client.passkey_reply:
            id: emerald
            passkey: 123123
  on_connect:
    - delay: 5s
    - ble_client.ble_write:
          id: emerald
          service_uuid: 00001910-0000-1000-8000-00805f9b34fb
          characteristic_uuid: 00002b11-0000-1000-8000-00805f9b34fb
          # List of bytes to write to enable auto-upload
          value: [0x00, 0x01, 0x02, 0x0b, 0x01, 0x01]
    - delay: 1s
    - lambda: |-
          id(ble_tracker).set_scan_continuous(false); 
          id(ble_tracker).stop_scan();
  on_disconnect:
    - delay: 5s
    - lambda: |-
          id(ble_tracker).set_scan_continuous(true); 
          id(ble_tracker).start_scan();

api:

ota:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  id: wifi_us

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esphome-Web-AP"
    password: "abcdef"
      

sensor:
  - platform: uptime
    name: Uptime Sensor
    id: uptime_sensor
  - platform: template
    name: "Emerald Power"
    id: emerald_power
    unit_of_measurement: kW
    device_class: power
    state_class: measurement
    accuracy_decimals: 2 
  - platform: template
    name: "Emerald Energy"
    id: emerald_energy
    unit_of_measurement: kWh
    device_class: energy
    state_class: total
    accuracy_decimals: 2
  - platform: template
    name: "Emerald Daily Energy"
    id: emerald_daily_energy
    unit_of_measurement: kWh
    device_class: energy
    state_class: total_increasing
    accuracy_decimals: 2
  - platform: ble_client
    type: characteristic
    ble_client_id: emerald
    name: "Emerald battery level"
    service_uuid: '180f'
    characteristic_uuid: '2a19'
    icon: 'mdi:battery'
    unit_of_measurement: '%'
    update_interval: 24h
  - platform: ble_client
    type: characteristic
    ble_client_id: emerald
    name: "Data Read"
    internal: true
    service_uuid: '00001910-0000-1000-8000-00805f9b34fb'
    characteristic_uuid: '00002b10-0000-1000-8000-00805f9b34fb'
    notify: true
    update_interval: never
    lambda: |-
      ESP_LOGI("emerald", "Notification received from Emerald device");
      
      static uint32_t return30sPowerConsumptionCmd = 0x0001020a06;
      static float pulses_per_kw = 1000;
      static float pulse_multiplier = (2*60.0) / pulses_per_kw;
      uint16_t length = x.size();
     
      uint32_t command_header = 0;
      for (int i = 0;  i < 5; i++) {
          command_header += (x[i] << (8*(4-i)));
      }

      if (command_header == return30sPowerConsumptionCmd){
        if (length == 11) {
          uint16_t pulses_within_interval = x[9] << 8;
          pulses_within_interval += + x[10];

          float avg_watts_within_interval = pulses_within_interval * pulse_multiplier;
          float energy = (id(total_pulses_global) += pulses_within_interval) / pulses_per_kw;
          float daily_energy = (id(daily_pulses_global) += pulses_within_interval) / pulses_per_kw;

          id(emerald_power).publish_state(avg_watts_within_interval);         
          id(emerald_energy).publish_state(energy);
          id(emerald_daily_energy).publish_state(daily_energy);

        } else {
            ESP_LOGD("emerald", "Data length != 11, bad data received");
        }
      }
      return 1;

interval:
  - interval: 1min
    then:
      lambda: |-
        ESPTime currentTime = id(esptime).now();
        if (currentTime.is_valid()) {
          if (id(day_of_last_measurement_global) == 0) { 
            id(day_of_last_measurement_global) = currentTime.day_of_year; 
          }
          else if (id(day_of_last_measurement_global) != currentTime.day_of_year) {
            id(daily_pulses_global) = 0;
            id(day_of_last_measurement_global) = currentTime.day_of_year;
          }
        }
1 Like
320

Is anyone else having this problem: since the update to ESPHome 2023.7.0 I am getting the following messages in my logs for my Powerpal device:

[10:11:34][W][component:204]: Component esp32_ble took a long time for an operation (0.05 s).
[10:11:34][W][component:205]: Components should block for at most 20-30ms.

This is happening every minute (when the Powerpal data is collected), not once as per the discussion on Github #3000 (https://github.com/esphome/issues/issues/3000). I am still getting readings from Powerpal but I do not know whether the accuracy is affected.
I am using the latest version of powerpal_ble as per the post from @muneeb1990 on 5 July.

321

I’m seeing the same symptoms. Recently updated my PowerPal firmware, so I think there may be an incompatibility with this new firmware version.

Update: Ahem. :man_facepalming: The firmware update had nothing to do with it. This is the symptom you’ll see when you entered the incorrect pairing code: You’ll only receive the battery status. All working now after I updated the pairing code.

322

Yeah mine is dead in the water :frowning:

323

Man, this sounded so elegant and simple when I first stumbled across it, but there’s so much chatter bouncing around with various issues, that I can’t even work out what source code I can reply upon… There’s at least a bunch of credible forks where someone has successfully attacked apparent issues, and would send a sane man crazy.

Now someone please correct me if I’m wrong, but as of this time, WeekenWarriors Yaml - inclusive of external_components, in addition to an installation of core esphome, that a properly configured setup and hardware behaving nicely, everything works inclusive of Web upload?? EXCEPT for Dual-Tarrif support which is not supported… And which seems like it wasn’t really pursued a great deal, or was it conclusively eliminated from being able to get working?

Would it be possible to establish these things, and move on from this thread being used as a means for everyone person who’s code fails compilation, this doesn’t get dragged through the sludge over and and over. This thread then contains so many HTML links that I actually ended up reading two different posts in the timeline at the same time. If it’s working, this thread is seriously detrimental to those looking for a solution, that’s just as elegant as when I first stumbled across it, when someone posted their own thread with their experiences, linking directly to the same place that hours of bouncing around the internet in this thread might hopefully take you.

324

I’ve found this to be the latest external_components, but keep receiving Error 133 on connecting, even though i’ve compiled on computer and not through HA itself as recommended in this thread. not sure if it’s broken completely with newer Powerpals maybe?

325

Thanks @rya ! I’ll give that a(nother) shot - I see it’s still being actively maintained in some fashion by the dev of that fork. He was active on this thread, but kinda fizzled out - and I understand why, as this isn’t the place for specific 3rd party integration support, merely a feature request.

Before I get into my essay - there’s 2 versions of the Powerpal? What do you mean by compiled on your own machine?

I would suggest, trying to contact him via git/post on the git project, or via HA forums (if you scroll back to find his username) I’m going to do the same, once I’ve reached your point essentially, because I gave up along my way to getting to where you’re at because I wasn’t sure if it was even going to work in the end.

Would I be able to reach out to you for a hand on getting further along with my setup? If yes, just message me on here, then we’ll work out what’s easiest for you.

Because it’s such a small part of the HA community that may require this - specifically “Victorian users with Smart Meters AND using a Powerpal” - I don’t think we’ll ever see anything official into the HA release with PowerPal. Buuuuuuut…

I then plan to provide some clear concise documentation, once it’s working for me and spoken to WeekendWarrior, because currently, there’s tidbits of information all over the place. If we can get Powerpal working smoothly, along with detailed HA format documentation we MIGHT be able to get it into Integrations

Though it would be nice if someone did an Integration for United Energy website, as it does from a number of other providers (OVO Energy, EasyEnergy, Energy Zero, Opower, SRP Energy et Al.) at present.
They’re the distributors, thus would necessitate creating individual energy companies. This would also have a wider user footprint(650k+ users) and thus draw greater need/reliance/attention to it.
Besides from my point of view, it provides Net_Energy_Consumption, Energy_Consumption & Solar_Exported which is handy on it’s own and I could live without Powerpal (won’t stop me trying to if it’s working!).
There was talk over 2yrs years ago about an API from UE, so I might follow that up and see if that eventuated because that would make things much easier!

Regardless there’s still plenty of use-case for Powerpal to run alongside the UE Integration. PowerPal, with the addition of some simple maths, would then provide the instantaneous Utility_Consumption, Solar_Output, Solar_Utilization & Net_EnergyTotal, all independent of the cloud. Then the cloud dependant United Energy Integration data would allow for comparison against their bills. That would be the perfect Utility_Meter integration for me, and if it takes a little bit of work to get there, then that’s fine by me!

I’ll post back a update if I make it any further!

1 Like
326

You can compile the yaml and deploy via your computer - Installing ESPHome Manually - ESPHome, and I tried as someone above posted:

I’ve honestly given up on this for now - using this Custom Component instead.

Feel free to! I tried heaps but the Custom Component was seamless honestly.

327

Aaaah, I see… Yeh got no issues there, I did it on my computer. Everything documented is quite a bit behind the software, ESPHome can be installed in a few clicks on HA now.

Thanks for that information, that’s certainly a quick and easy way to make it happen… That said, it’s not quite enough… I’d have to run another phone closer to the Powerpal, and I’m not ready to give up on raw data yet, the United Energy API is still a possibility, I’m willing to take a second run at getting the original going again - the one you linked to may provide some assistance/clues in its code.

How does it handle being disconnected from the device?

I fell over at the Bluetooth connection/communication stage, and I actually saw something that I missed just a few posts above that remind me that I’d completely missed something out.

328

What is Error 133? Can you provide more details as I haven’t encountered this code before?

329

Hello @muneeb1990, I recall you from earlier posts as being very familiar with the process of getting this going. Is the current method, that which I spoke of a few posts ago still working?

Can I ask, regarding the BLE receiver, must this be physically plugged into the USB port, or just connected to the network?

Re: the 133 error, just in case you weren’t aware, note that he is using a different method to connect that this

330

ESP32 works via wifi so it just needs to be powered.

331

Right, so I take it that everything is working fine as per the current build?