Powerpal smart energy monitor

Mine only has a sticker with model (PPL-002), manufacturer (PowerPal), serial number and patent number.

Hi Michael,

My only advice then on troubleshooting steps is to triple check that ALL smart devices (eg. including a partnerā€™s device that has also once been configured to use the powerpal) have their bluetooth completely disabled/turned off, and then trying again using the esp32_ble_tracker.

Youā€™re right!!! I had killed the apps, thinking that would be enough, but turning off bluetooth on my iPhone and iPad caused it to show up (no partner :slight_smile: ). It might help others to add this to your readme.

Hi Michael, thanks for your feedback and glad you found the MAC address.
I have added a little note about disabling bluetooth in the READMEs.

1 Like

Is there a problem with the current build? Iā€™m getting the error below, whereas a few hours ago I was not.

src/esphome/components/powerpal_ble/powerpal_ble.cpp: In member function 'virtual void esphome::powerpal_ble::Powerpal::dump_config()':
src/esphome/components/powerpal_ble/powerpal_ble.cpp:18:61: error: 'class esphome::powerpal_ble::Powerpal' has no member named 'time_'
   if (this->daily_energy_sensor_ != nullptr && !(bool)this->time_) {
                                                             ^~~~~
*** [/data/m5stamp-c3u-1/.pioenvs/m5stamp-c3u-1/src/esphome/components/powerpal_ble/powerpal_ble.cpp.o] Error 1

Thanks for the error message, should be fixed now.
Depending on how you are running ESPHome to upload the compile/upload the code, you may need to delete the .build folder to force a fresh pull from github to get the fix, else I think ESPHome may only check for updates on a github repo hourly(?).

That fixed it thanks. I struggled to find the right folder to delete (Iā€™m running under HA), so I just renamed ā€œ/config/esphome/.esphomeā€. However, I suspect I could have just deleted the following folder:

/config/esphome/.esphome/external_components/ca346dbe/esphome/components/powerpal_ble

Iā€™m not getting the daily energy. How do I force a new version to download from github? During the install, Iā€™m sure Iā€™ve seen a ā€œdownloading from githubā€ type of message. I have renamed ā€œ.esphomeā€, and done a ā€œClean Buildā€, but when I install, Iā€™m not seeing another download message, and not getting daily energy in Home Assistant. However, if I look into the powerpal_ble external component folder (mentioned in my previous post), I can see it has code relating to daily energy.

EDIT: I also used Portainer to connect to the container and renamed /data/cache just in case. Still no dice getting daily energy.

Thanks @WeekendWarrior1 - my ESP32 arrived today from ebay ($10AUD locally delivered) and tried it out the first time with your code. Took a little fiddling to work out what I was doing but was mostly hassle-free :slight_smile:

Any idea how we can keep the PowerPal app on our phone but stop it from hijacking the connection from the ESP32? App seems to run in the background (at least on my Android phone) and it was causing the ESP to throw the ā€˜133ā€™ error message.

Is there any issues with leaving logging on? Does it persist the logs on the ESP and will it therefore kill the storage/wear it out?

Hi Michael, please make sure you have the daily energy sensor set up in your yaml:

sensor:
  - platform: powerpal_ble
    ble_client_id: powerpal
    power:
      id: my_powerpal
      name: "Powerpal Power"
    energy:
      name: "Powerpal Total Energy"
    daily_energy: # are you sure you have these 2 lines?
      name: "Powerpal Daily Energy"
    battery_level:
      name: "Powerpal Battery"
    pairing_code: 123123
    notification_interval: 1
    pulses_per_kwh: 800

Hi JJW.AU, glad to hear your experience was mostly hassle free.

If you donā€™t rely on Bluetooth, disabling bluetooth should be enough. If you do, perhaps you could try forgetting the Powerpal device from the android bluetooth known devices menu?
I am unsure if that will affect your ability to see the data within the Powerpal, something you will have to test.

None at all :slight_smile:

If I may ask, did you enable the cloud uploader functionality, and if so, has it been working?
Itā€™s something that I tested with a local API endpoint, but not against the Powerpal cloud API, so would love to know if it works or if there are any errors.

I did enable the cloud upload but how do I actually check it? Presume reinstall the PowerPal app and browse history? But this would probably trigger syncing of data from the PowerPal at the same time?

Hmmm, totally unsure.
You can check the logs of your esphome device to confirm that it is uploading without failure (should send data every 15 measurements received).

I believe the Powerpal app protects you from ā€œdouble syncingā€ your data, since it only requests measurements from the Powerpal that donā€™t exist on the cloud, but I really have no clue if you can actually look at your device graphs without bluetooth enabled / being connected to the Powerpal device.

UPDATE ===>
Looking at sensor.py, it has the following code for total energy at line 150, but no equivalent for daily energy. Is this a problem?

    if CONF_ENERGY in config:
        sens = await sensor.new_sensor(config[CONF_ENERGY])
        cg.add(var.set_energy_sensor(sens))

If I include the following code after this, I do start getting daily energy in Home Assistant:

    if CONF_DAILY_ENERGY in config:
        sens = await sensor.new_sensor(config[CONF_DAILY_ENERGY])
        cg.add(var.set_daily_energy_sensor(sens))

<===

It goes to show that you should always include your yaml. I do have daily energy set up, but itā€™s still not coming through.

substitutions:
  name: m5core2-1
  id: m5core2_1

packages:
  wifi: !include include/wifi.yaml
  device_base: !include include/m5core2-base.yaml

esp32_ble_tracker:

logger:
  level: DEBUG
  logs:
    esp32_ble_tracker: INFO

external_components:
# - <<: !include include/external_components/ping.yaml
  - source: github://WeekendWarrior1/esphome@powerpal_ble
    components: [ ble_client, powerpal_ble ]

ble_client:
  - mac_address: !secret powerpal_mac
    id: powerpal

binary_sensor:
# - <<: !include include/binary_sensor/connection-status.yaml

time:
  - platform: homeassistant
    id: ha_time
    
http_request:
  id: powerpal_cloud_uploader

sensor:
# - <<: !include include/sensor/uptime.yaml
# - <<: !include include/sensor/wifi-signal.yaml
- platform: powerpal_ble
  ble_client_id: powerpal
  power:
    name: "${name} Powerpal Power"
  energy:
    name: "${name} Powerpal Total Energy"
  daily_energy:
    name: "${name} Powerpal Daily Energy"
  battery_level:
    name: "${name} Powerpal Battery"
  pairing_code: !secret powerpal_pairing_code
  notification_interval: 1 # every 1 minute
  pulses_per_kwh: 1000
  http_request_id: powerpal_cloud_uploader
  cost_per_kwh: 0.23452 #dollars per kWh
  time_id: ha_time
  # powerpal_device_id: 0000abcd #optional, component will retrieve from your Powerpal if not set
  # powerpal_apikey: 4a89e298-b17b-43e7-a0c1-fcd1412e98ef #optional, component will retrieve from your Powerpal if not set

text_sensor:
# - <<: !include include/text_sensor/uptime-text.yaml

I also have the uploader configured, but itā€™s not updating. From the logger Iā€™m getting an 8 digit number after ā€œPowerpal apikey:ā€, not a UUID as shown in the documentation. Iā€™ve tried all the combinations I can think of in the curl below, and always get ā€œAuthentication failedā€.

curl -H "Authorization: <YOUR_API_KEY>" https://readings.powerpal.net/api/v1/device/<YOUR_DEVICE_ID>

Here is the log Iā€™m getting:

[15:01:57][I][ble_client:085]: Attempting BLE connection to fd:fb:4f:00:00:00
[15:02:00][I][ble_client:166]: Service UUID: 0x1800
[15:02:00][I][ble_client:167]:   start_handle: 0x1  end_handle: 0x9
[15:02:00][I][ble_client:380]:  characteristic 0x2A00, handle 0x3, properties 0xa
[15:02:00][I][ble_client:380]:  characteristic 0x2A01, handle 0x5, properties 0x2
[15:02:00][I][ble_client:380]:  characteristic 0x2A04, handle 0x7, properties 0x2
[15:02:00][I][ble_client:380]:  characteristic 0x2AA6, handle 0x9, properties 0x2
[15:02:01][I][ble_client:166]: Service UUID: 0x1801
[15:02:01][I][ble_client:167]:   start_handle: 0xa  end_handle: 0xd
[15:02:01][I][ble_client:380]:  characteristic 0x2A05, handle 0xc, properties 0x20
[15:02:01][I][ble_client:220]: auth complete. remote BD_ADDR: fdfb4f000000
[15:02:01][I][powerpal_ble:471]: [fd:fb:4f:00:00:00] Writing pairing code to Powerpal
[15:02:01][D][powerpal_ble:360]: [fd:fb:4f:00:00:00] ESP_GATTC_WRITE_CHAR_EVT (Write confirmed)
[15:02:02][D][powerpal_ble:265]: [fd:fb:4f:00:00:00] ESP_GATTC_READ_CHAR_EVT (Received READ)
[15:02:02][D][powerpal_ble:272]: Recieved reading_batch_size read event
[15:02:02][D][powerpal_ble:062]: DEC(4): 0x01000000
[15:02:02][D][powerpal_ble:265]: [fd:fb:4f:00:00:00] ESP_GATTC_READ_CHAR_EVT (Received READ)
[15:02:02][I][powerpal_ble:322]: Recieved uuid read event
[15:02:02][I][powerpal_ble:324]: Powerpal device id: 0000000000000000000000000000
[15:02:02][D][powerpal_ble:265]: [fd:fb:4f:00:00:00] ESP_GATTC_READ_CHAR_EVT (Received READ)
[15:02:02][I][powerpal_ble:334]: Recieved serial_number read event
[15:02:02][I][powerpal_ble:336]: Powerpal apikey: 00000000
[15:02:02][D][powerpal_ble:265]: [fd:fb:4f:00:00:00] ESP_GATTC_READ_CHAR_EVT (Received READ)
[15:02:02][D][powerpal_ble:301]: Recieved battery read event
[15:02:02][D][powerpal_ble:066]: Battery: DEC(1): 0x5f
[15:02:02][D][sensor:125]: 'm5core2-1 Powerpal Battery': Sending state 95.00000 % with 0 decimals of accuracy
[15:02:02][D][powerpal_ble:265]: [fd:fb:4f:00:00:00] ESP_GATTC_READ_CHAR_EVT (Received READ)
[15:02:02][D][powerpal_ble:308]: Recieved firmware read event
[15:02:02][D][powerpal_ble:062]: DEC(5): 0x312e302e30
[15:02:07][D][powerpal_ble:444]: [fd:fb:4f:00:00:00] Received Notification
[15:02:07][D][powerpal_ble:455]: Recieved measurement notify event
[15:02:07][D][powerpal_ble:073]: Meaurement: DEC(20): 0x8cac9162090066e7b6fdff52541044dbedcbff00
[15:02:08][I][powerpal_ble:087]: Timestamp: 1653714060, Pulses: 9, Average Watts within interval: 540.000000 W
[15:02:08][D][sensor:125]: 'm5core2-1 Powerpal Power': Sending state 540.00000 W with 5 decimals of accuracy
[15:02:08][D][sensor:125]: 'm5core2-1 Powerpal Total Energy': Sending state 0.00900 kWh with 5 decimals of accuracy

Also, you can see updated graphs when youā€™re not connected in the app. I have the app on my phone and iPad, and only one can connect. If you go into the other, you normally see ā€œnot connectedā€, but the graphs are updated. However, I have not had any uploads for the last 3 days while figuring it out with the esp32.

When I re-paired with the phone, it downloaded the last 3 days of usage info from the device and then uploaded that to the cloud. By implication there might be a better way to get the info - i.e. all data since last successful retrieval. At the moment, every time you restart the esp32 it starts from zero again, and any down-time means lost data.

Did anybody read all the way down here? Itā€™s a bit wordy, sorry.

Hi Michael, youā€™re totally right (and thanks so much for your help, I donā€™t have access to a Powerpal anymore so itā€™s really hard to make sure that the new features are working correctly)
I somehow managed to commit without adding the Powerpal daily energy sensor codegen builder and I have to apologise that you spent so much time personally debugging it. This has now been pushed.

Hmm, okay, the 8 digits are your device ID, not your API key.
Do you get anymore characteristics/services printed out than the services you posted?
When I had a powerpal I was getting:

[03:56:46][I][ble_client:166]: Service UUID: 0x1800
[03:56:46][I][ble_client:167]:   start_handle: 0x1  end_handle: 0x9
[03:56:46][I][ble_client:380]:  characteristic 0x2A00, handle 0x3, properties 0xa
[03:56:46][I][ble_client:380]:  characteristic 0x2A01, handle 0x5, properties 0x2
[03:56:46][I][ble_client:380]:  characteristic 0x2A04, handle 0x7, properties 0x2
[03:56:46][I][ble_client:380]:  characteristic 0x2AA6, handle 0x9, properties 0x2
[03:56:46][I][ble_client:166]: Service UUID: 0x1801
[03:56:46][I][ble_client:167]:   start_handle: 0xa  end_handle: 0xd
[03:56:46][I][ble_client:380]:  characteristic 0x2A05, handle 0xc, properties 0x20
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0xd
[03:56:46][I][ble_client:166]: Service UUID: 0x180F
[03:56:46][I][ble_client:167]:   start_handle: 0xe  end_handle: 0x11
[03:56:46][I][ble_client:380]:  characteristic 0x2A19, handle 0x10, properties 0x12
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x11
[03:56:46][I][ble_client:166]: Service UUID: 59DAABCD-12F4-25A6-7D4F-55961DCE4205
[03:56:46][I][ble_client:167]:   start_handle: 0x12  end_handle: 0x34
[03:56:46][I][ble_client:380]:  characteristic 59DA0001-12F4-25A6-7D4F-55961DCE4205, handle 0x14, properties 0x1a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x15
[03:56:46][I][ble_client:380]:  characteristic 59DA0002-12F4-25A6-7D4F-55961DCE4205, handle 0x17, properties 0x28
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x18
[03:56:46][I][ble_client:380]:  characteristic 59DA0003-12F4-25A6-7D4F-55961DCE4205, handle 0x1a, properties 0x12
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x1b
[03:56:46][I][ble_client:380]:  characteristic 59DA0004-12F4-25A6-7D4F-55961DCE4205, handle 0x1d, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x1e
[03:56:46][I][ble_client:380]:  characteristic 59DA0005-12F4-25A6-7D4F-55961DCE4205, handle 0x20, properties 0xa
[03:56:46][I][ble_client:380]:  characteristic 59DA0007-12F4-25A6-7D4F-55961DCE4205, handle 0x22, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x23
[03:56:46][I][ble_client:380]:  characteristic 59DA0008-12F4-25A6-7D4F-55961DCE4205, handle 0x25, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x26
[03:56:46][I][ble_client:380]:  characteristic 59DA0009-12F4-25A6-7D4F-55961DCE4205, handle 0x28, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x29
[03:56:46][I][ble_client:380]:  characteristic 59DA0010-12F4-25A6-7D4F-55961DCE4205, handle 0x2b, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x2c
[03:56:46][I][ble_client:380]:  characteristic 59DA0011-12F4-25A6-7D4F-55961DCE4205, handle 0x2e, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x2f
[03:56:46][I][ble_client:380]:  characteristic 59DA0012-12F4-25A6-7D4F-55961DCE4205, handle 0x31, properties 0x2
[03:56:46][I][ble_client:380]:  characteristic 59DA0013-12F4-25A6-7D4F-55961DCE4205, handle 0x33, properties 0x3a
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x34
[03:56:46][I][ble_client:166]: Service UUID: 0x180A
[03:56:46][I][ble_client:167]:   start_handle: 0x35  end_handle: 0x3b
[03:56:46][I][ble_client:380]:  characteristic 0x2A29, handle 0x37, properties 0x2
[03:56:46][I][ble_client:380]:  characteristic 0x2A25, handle 0x39, properties 0x2
[03:56:46][I][ble_client:380]:  characteristic 0x2A26, handle 0x3b, properties 0x2
[03:56:46][I][ble_client:166]: Service UUID: 0xFE59
[03:56:46][I][ble_client:167]:   start_handle: 0x3c  end_handle: 0xffff
[03:56:46][I][ble_client:380]:  characteristic 8EC90003-F315-4F60-9FB8-838830DAEA50, handle 0x3e, properties 0x28
[03:56:46][V][ble_client:415]:    descriptor 0x2902, handle 0x3f

If I canā€™t be certain that the handle matches the characteristic 100% of the time, I will add dynamic code to handle tomorrow.

Really happy to see this, as this was my vision of the best case - people could use the esphome component to upload to the cloud (when we get that working), but still use the app to view the data if they wanted.

Will be fixed via esphome flash sensor persistence, just a few lines I need to add.

1 Like

Should now be fixed, please try the newest commit.

Hey, youā€™ve done all the work here. Iā€™m glad I can help, given Iā€™m getting the benefit. Iā€™ve been trying to understand how youā€™ve reverse engineered it, and used nRF Connect to see the services it provides, but thatā€™s about as far as Iā€™ve got. This is my first attempt and digging into bluetooth.

I can confirm that Iā€™m getting an API key that looks like a UUID now. Thanks. I will update if it uploads the data.

A combination of:

  1. nRF Connect for initial service/characteristic discovery, and eventually for testing authentication/handshaking and the data received from notifications
  2. Android Bluetooth HCI debug logs opened in wireshark - Used to see the exact pairing process, but then also how and what data is sent around (at this stage the data has very little context)
  3. Decompiling the Powerpal/Emerald Android apps using: https://apklab.surendrajat.xyz/ and then spending a fair bit of time stepping through the decompiled code. (the Powerpal app was a little more involved because it is a React Native application so inside the decompiled Java app is a bundle of compiled/obfuscated JavaScript (ended up being over 1100 seperate obfuscated JS files to dig through lol)). Main focus was how the apps communicated with the BLE devices and how they decoded the data received from the BLE devices. (as well as what characteristics mapped to what functionality)
  4. Wading through the android logs of each app to get insight into how the applications operated normally
  5. Specifically for the Emerald app I set up an android MITM certificate on my phone so I could collect the web traffic and document the Emerald API (this had already been done by other people for the Powerpal (although step 3 held extra Powerpal API secrets)), mainly just for completeness sake.

And then testing everything I had learnt using Arduino BLE on the esp32 before moving over to building the ESPHome components.

2 Likes

Hi [WeekendWarrior1],
Thanks for your incredible work.
Unfortunately Iā€™m have some troubles connecting with esphome and arduino frameworks.
If I use you sketch, when itā€™s compiling I get a few warnings:

D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino:11:45: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
 static char *BLE_address("d5:d7:12:2c:68:75"); // lowercase only or else will fail to match
                                             ^
D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino: In function 'uint8_t* powerpal_reverse_uint32(uint32_t)':
D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino:72:16: warning: narrowing conversion of '(input & 255)' from 'uint32_t' {aka 'unsigned int'} to 'uint8_t' {aka 'unsigned char'} inside { } [-Wnarrowing]
         (input & 0x000000FF),
         ~~~~~~~^~~~~~~~~~~~~
D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino:73:31: warning: narrowing conversion of '((input & 65280) >> 8)' from 'uint32_t' {aka 'unsigned int'} to 'uint8_t' {aka 'unsigned char'} inside { } [-Wnarrowing]
         ((input & 0x0000FF00) >> 8),
         ~~~~~~~~~~~~~~~~~~~~~~^~~~~
D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino:74:31: warning: narrowing conversion of '((input & 16711680) >> 16)' from 'uint32_t' {aka 'unsigned int'} to 'uint8_t' {aka 'unsigned char'} inside { } [-Wnarrowing]
         ((input & 0x00FF0000) >> 16),
         ~~~~~~~~~~~~~~~~~~~~~~^~~~~~
D:\Temp\.arduinoIDE-unsaved2022430-17500-1q11xkh.bw2ej\sketch_may30a\sketch_may30a.ino:75:31: warning: narrowing conversion of '((input & 4278190080) >> 24)' from 'uint32_t' {aka 'unsigned int'} to 'uint8_t' {aka 'unsigned char'} inside { } [-Wnarrowing]
         ((input & 0xFF000000) >> 24),
         ~~~~~~~~~~~~~~~~~~~~~~^~~~~~
Sketch uses 924233 bytes (70%) of program storage space. Maximum is 1310720 bytes.
Global variables use 26796 bytes (8%) of dynamic memory, leaving 300884 bytes for local variables. Maximum is 327680 bytes.

then when I monitor the serial connection, it seems to connect and then hang when trying to get battery levels.

configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
load:0x40078000,len:13516
load:0x40080400,len:3604
entry 0x400805f0
Starting Arduino BLE Client application...
BLE Advertised Device found: Name: ATC_0304B2, Address: a4:c1:38:03:04:b2
Skipping, isn't correct device
BLE Advertised Device found: Name: ATC_733A43, Address: a4:c1:38:73:3a:43
Skipping, isn't correct device
BLE Advertised Device found: Name: Powerpal 0000d32e, Address: d5:d7:12:2c:68:75, serviceUUID: 59daabcd-12f4-25a6-7d4f-55961dce4205
Attempting to connect to device...
Found our device
Forming a connection to d5:d7:12:2c:68:75
 - Created client
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
 - Connected to server
Attempting to get battery service...

Any advice would be appreciated.

Hi pencilhead,

Looks like the ESP32 is failing to read the main Powerpal service, but also none of the security callbacks are being triggered.
As always, please confirm that you have the correct MAC + Pairing code and have them properly entered in the sketch, and all other smart devices that can connect to the Powerpal have had their bluetooth disabled.
You can also try the sketch from this commit:

Which is prior to when I added the authentication retrieval, a feature I didnā€™t get to test 100%.

Also of interest is what version of the Arduino ESP32 core youā€™re using, it is likely you need >2.0.0 (on my computer I can see my arduino core at ~/.arduino15/packages/esp32/hardware/esp32/).

And finally, what errors did you see when using the esphome component?
And out of curioisty, is there a reason youā€™re using the Arduino sketch? For troubleshooting? Or maybe just to play around with the Powerpal?

Thanks, I did manage to get a connection, but seems very random. Like 1 in100 tries. Iā€™ve double checked MAC / pass key and bluetooth is disabled on phone.
I tried earlier version, you provided and could connect with that but rare and random.
Connection info:

MTU:23
 - Connected to server
Attempting to get battery service...
Attempting to get powerpal service...
We are now connected to the BLE Server.
onAuthenticationComplete
auth_cmpl.success
Sending pairingcode to char...
Reading measurements every: 15 minute(s)
Setting to every 1 minute
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  176 129 152 98 24 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654161840, Pulses: 24, Power: 1.44 kW

and another attempt:

AuthenticationComplete
auth_cmpl.success
Attempting to get powerpal service...
We are now connected to the BLE Server.
Sending pairingcode to char...
Reading measurements every: 15 minute(s)
Setting to every 1 minute
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  236 114 152 98 67 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158060, Pulses: 67, Power: 4.02 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  40 115 152 98 60 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158120, Pulses: 60, Power: 3.60 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  100 115 152 98 57 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158180, Pulses: 57, Power: 3.42 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  160 115 152 98 55 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  220 115 152 98 53 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158300, Pulses: 53, Power: 3.18 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  24 116 152 98 51 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158360, Pulses: 51, Power: 3.06 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  84 116 152 98 47 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158420, Pulses: 47, Power: 2.82 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  144 116 152 98 43 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158480, Pulses: 43, Power: 2.58 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  204 116 152 98 39 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158540, Pulses: 39, Power: 2.34 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  8 117 152 98 38 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158600, Pulses: 38, Power: 2.28 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  68 117 152 98 38 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158660, Pulses: 38, Power: 2.28 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  128 117 152 98 38 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158720, Pulses: 38, Power: 2.28 kW
Notify callback for battery characteristic 00002a19-0000-1000-8000-00805f9b34fb of data length 1
Battery:  93%
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  188 117 152 98 36 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158780, Pulses: 36, Power: 2.16 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  248 117 152 98 36 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158840, Pulses: 36, Power: 2.16 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  52 118 152 98 31 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158900, Pulses: 31, Power: 1.86 kW
Notify callback for characteristic 59da0001-12f4-25a6-7d4f-55961dce4205 of data length 20
data:  112 118 152 98 11 0 116 173 191 115 184 14 87 52 135 109 92 200 97 73
Time: 1654158960, Pulses: 11, Power: 0.66 kW
Notify callback for battery characteristic 00002a19-0000-1000-8000-00805f9b34fb of data length 1
Battery:  92%
ets Jun  8 2016 00:22:57

It seems to fail (hang after Connected to server) when I get serial message:

Forming a connection to d5:d7:12:2c:68:75
 - Created client
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
 - Connected to server

With debugging turned on:

Starting Arduino BLE Client application...
[  1104][D][BLEAdvertisedDevice.cpp:472] setRSSI(): - setRSSI(): rssi: -89
[  1104][D][BLEAdvertisedDevice.cpp:292] parseAdvertisement(): Type: 0x01 (), length: 1, data: 05
[  1108][D][BLEAdvertisedDevice.cpp:292] parseAdvertisement(): Type: 0x07 (), length: 16, data: 0542ce1d96554f7da625f412cdabda59
[  1119][D][BLEAdvertisedDevice.cpp:500] setServiceUUID(): - addServiceUUID(): serviceUUID: 59daabcd-12f4-25a6-7d4f-55961dce4205
[  1131][D][BLEAdvertisedDevice.cpp:292] parseAdvertisement(): Type: 0x09 (), length: 17, data: 506f77657270616c203030303064333265
[  1142][D][BLEAdvertisedDevice.cpp:461] setName(): - setName(): name: Powerpal 0000d32e
BLE Advertised Device found: Name: Powerpal 0000d32e, Address: d5:d7:12:2c:68:75, serviceUUID: 59daabcd-12f4-25a6-7d4f-55961dce4205
Attempting to connect to device...
Found our device
Forming a connection to d5:d7:12:2c:68:75
 - Created client
[  4172][I][BLEDevice.cpp:622] addPeerDevice(): add conn_id: 0, GATT role: client
[  4174][D][BLEDevice.cpp:148] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[  4183][D][BLEClient.cpp:178] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
[ 23644][D][BLEDevice.cpp:148] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[ 23645][D][BLEClient.cpp:178] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[ 23655][D][BLEDevice.cpp:148] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[ 23664][D][BLEClient.cpp:178] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
[ 23674][E][BLEClient.cpp:239] gattClientEventHandler(): Failed to connect, status=Unknown ESP_ERR error
[ 23683][I][BLEDevice.cpp:633] removePeerDevice(): remove: 0, GATT role client
 - Connected to server
[ 23690][D][BLEDevice.cpp:148] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... Unknown
Attempting to get battery service...

When I try and connect using ESPHOME. It doesnā€™t compile:

INFO Reading configuration /config/esphome/wroomtest.yaml...
INFO Detected timezone 'Australia/Sydney'
INFO Generating C++ source...
INFO Compiling app...
Processing powerpalble (board: esp32dev; framework: arduino; platform: platformio/espressif32 @ 3.5.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
Dependency Graph
|-- <WiFi> 1.0
|-- <ESPmDNS> 1.0
|-- <Update> 1.0
Compiling /data/powerpalble/.pioenvs/powerpalble/src/esphome/components/ble_client/ble_client.cpp.o
Compiling /data/powerpalble/.pioenvs/powerpalble/src/esphome/components/powerpal_ble/powerpal_ble.cpp.o
Compiling /data/powerpalble/.pioenvs/powerpalble/src/esphome/components/socket/bsd_sockets_impl.cpp.o
Compiling /data/powerpalble/.pioenvs/powerpalble/src/esphome/components/socket/lwip_raw_tcp_impl.cpp.o
In file included from src/esphome/components/ble_client/ble_client.cpp:5:0:
src/esphome/components/ble_client/ble_client.h:91:8: error: 'void esphome::ble_client::BLEClient::gap_event_handler(esp_gap_ble_cb_event_t, esp_ble_gap_cb_param_t*)' marked 'override', but does not override
   void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override;
        ^
In file included from src/esphome/components/powerpal_ble/powerpal_ble.h:4:0,
                 from src/esphome/components/powerpal_ble/powerpal_ble.cpp:1:
src/esphome/components/ble_client/ble_client.h:91:8: error: 'void esphome::ble_client::BLEClient::gap_event_handler(esp_gap_ble_cb_event_t, esp_ble_gap_cb_param_t*)' marked 'override', but does not override
   void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override;
        ^
*** [/data/powerpalble/.pioenvs/powerpalble/src/esphome/components/ble_client/ble_client.cpp.o] Error 1
*** [/data/powerpalble/.pioenvs/powerpalble/src/esphome/components/powerpal_ble/powerpal_ble.cpp.o] Error 1
========================== [FAILED] Took 1.99 seconds ==========================

My Yaml file:

substitutions:
  device_name: powerpalble
  friendly_name: "powerpalble"
  device_ip: 192.168.1.214

#################################

esphome:
  name: ${device_name}
#  includes:
#    - powerpal_ble.cpp
#    - powerpal_ble.h  
    
esp32:
  board: esp32dev
  framework:
    type: arduino

external_components:
  - source: github://WeekendWarrior1/esphome@powerpal_ble
    # requires ble_client because I had to add some small features to authenticate properly
    components: [ ble_client, powerpal_ble ]
    
wifi:
  ssid: !secret ssid
  password: !secret wifi-password
  fast_connect: on 
  manual_ip:
    static_ip: ${device_ip}
    gateway: 192.168.1.1
    subnet: 255.255.255.0
  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "powerpalble"
    password: !secret ap-password
    
api:
  password: !secret api-password

ota:
  password: !secret ota-password

logger:



    
# optional requirement to enable powerpal cloud uploading
#http_request:
#  id: powerpal_cloud_uploader

# optional requirement used with daily energy sensor
time:
  - platform: homeassistant
    id: homeassistant_time

esp32_ble_tracker:

ble_client:
  - mac_address: d5:d7:12:2c:68:75
    id: powerpal

sensor:
  - platform: powerpal_ble
    ble_client_id: powerpal
    power:
      name: "Powerpal Power"
    daily_energy:
      name: "Powerpal Daily Energy"
    energy:
      name: "Powerpal Total Energy"
    battery_level:
      name: "Powerpal Battery"
    pairing_code: 667124
    notification_interval: 1
    pulses_per_kwh: 800
    time_id: homeassistant_time 

# daily energy still works without a time_id, but recommended to include one to properly handle daylight savings, etc.
#    http_request_id: powerpal_cloud_uploader
#    cost_per_kwh: 0.1872 #dollars per kWh
#    powerpal_device_id: 0000abcd #optional, component will retrieve from your Powerpal if not set
#    powerpal_apikey: 4a89e298-b17b-43e7-a0c1-fcd1412e98ef #optional, component will retrieve from your Powerpal if not set

I tried downloading and including

  includes:
    - powerpal_ble.cpp
    - powerpal_ble.h  

I get error when trying to compile:

INFO Reading configuration /config/esphome/wroomtest.yaml...
INFO Detected timezone 'Australia/Sydney'
Failed config

sensor.powerpal_ble: [source /config/esphome/wroomtest.yaml:64]
  
  Platform not found: 'sensor.powerpal_ble'.
  platform: powerpal_ble

Not sure what to do with sensor.py

I just managed to get retrieve_api_key.py on laptop working. This confirms MAC address and pairing code :slight_smile: :grinning_face_with_smiling_eyes:

I would prefer to get it going esphome. I was using Arduino for testing as I had no idea if I was even able to connect.