How to count pulse frequency accurately with ESPHome?

depen · January 6, 2021, 8:38pm

I´m also trying your custom component. It validates just fine with Esphome v1.15.3, but when I compile it I get some errors:

In file included from src/main.cpp:44:0:
src/pulse_meter_sensor.h:11:7: error: redefinition of 'class esphome::pulse_meter::PulseMeterSensor'
 class PulseMeterSensor : public sensor::Sensor, public Component {
       ^
In file included from src/esphome.h:23:0,
                 from src/main.cpp:3:
src/esphome/components/pulse_meter/pulse_meter_sensor.h:11:7: error: previous definition of 'class esphome::pulse_meter::PulseMeterSensor'
 class PulseMeterSensor : public sensor::Sensor, public Component {
       ^
*** [/data/esp32_01/.pioenvs/esp32_01/src/main.cpp.o] Error 1

I have put pulse_meter_sensor.h, pulse_meter_sensor.cpp and sensor.py in
the directory: config/esphome/custom_components/pulse_meter/

Code snippet:

esphome:
  name: ${esp_name}
  platform: ESP32
  board: mhetesp32devkit
  includes:
    - custom_components/pulse_meter/pulse_meter_sensor.h

  - platform: pulse_meter
    name: "main_power_usage"
    pin: GPIO12
    unit_of_measurement: "kW"
    internal_filter: 100ms
    accuracy_decimals: 3
    timeout: 2 min
    filters:
      # Filter outliers
      - median:
          window_size: 3
          send_every: 1
          send_first_at: 1
      # Convert pulses/min (Wh/min) to kW
      - multiply: 0.06
    total:
      name: "main_energy_usage"
      unit_of_measurement: "kWh"
      accuracy_decimals: 3
      filters:
        - multiply: 0.001

What am I doing wrong ??

stevebaxter · January 6, 2021, 10:40pm

Can you check you definitely haven’t got pulse_meter_sensor.h anywhere else in your config directory? It looks like two versions are being included from different directories.

depen · January 7, 2021, 5:35pm

Yes, the three files are only located in
config/esphome/custom_components/pulse_meter/

I made a clean build but it didn´t help. I run the esphome add-on from the HA community, but it is in sync with the Esphome repository and shouldn´t matter…?

stevebaxter · January 7, 2021, 8:11pm

Ah the problem is the:

  includes:
    - custom_components/pulse_meter/pulse_meter_sensor.h

This is already in the custom_components folder, so it will just work without that line. Delete it and you should be good!

depen · January 7, 2021, 9:13pm

Yes, Thanks! Unfortunately I got a new error

Linking /data/esp32_01/.pioenvs/esp32_01/firmware.elf
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `esphome::pulse_meter::PulseMeterSensor::gpio_intr(esphome::pulse_meter::PulseMeterSensor*)':
/config/esphome/esp32_01/src/pulse_meter_sensor.cpp:58: multiple definition of `esphome::pulse_meter::PulseMeterSensor::gpio_intr(esphome::pulse_meter::PulseMeterSensor*)'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:/config/esphome/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp:58: first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `esphome::pulse_meter::PulseMeterSensor::setup()':
pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor5setupEv+0x0): multiple definition of `esphome::pulse_meter::PulseMeterSensor::setup()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor5setupEv+0x0): first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::setup()':
pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor5setupEv+0x0): multiple definition of `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::setup()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor5setupEv+0x0): first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `esphome::pulse_meter::PulseMeterSensor::dump_config()':
pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor11dump_configEv+0x0): multiple definition of `esphome::pulse_meter::PulseMeterSensor::dump_config()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor11dump_configEv+0x0): first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::dump_config()':
pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor11dump_configEv+0x0): multiple definition of `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::dump_config()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor11dump_configEv+0x0): first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `esphome::pulse_meter::PulseMeterSensor::loop()':
pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor4loopEv+0x0): multiple definition of `esphome::pulse_meter::PulseMeterSensor::loop()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZN7esphome11pulse_meter16PulseMeterSensor4loopEv+0x0): first defined here
/data/esp32_01/.pioenvs/esp32_01/src/pulse_meter_sensor.cpp.o: In function `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::loop()':
pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor4loopEv+0x0): multiple definition of `non-virtual thunk to esphome::pulse_meter::PulseMeterSensor::loop()'
/data/esp32_01/.pioenvs/esp32_01/src/esphome/components/pulse_meter/pulse_meter_sensor.cpp.o:pulse_meter_sensor.cpp:(.text._ZThn160_N7esphome11pulse_meter16PulseMeterSensor4loopEv+0x0): first defined here
collect2: error: ld returned 1 exit status
*** [/data/esp32_01/.pioenvs/esp32_01/firmware.elf] Error 1

stevebaxter · January 7, 2021, 9:54pm

Try cleaning the build files (… menu and “Clean build files”). Things might be a bit messed up.

DarthMob · January 8, 2021, 7:23pm

For reference if interesting for others:

I wanted to show the daily value for the power usage (with daily reset).

I tried doing this with the Total Daily Energy Sensor from ESPHome (basically integration of kW over time)

This however results in ~400 Wh more on a day where i used 6 kWh. (see blue graph).

I then used the HA Utility Meter on the Pulse/ kWh sensor (red graph).
This then returns the correct value.

mhooghiemstra · January 26, 2021, 11:54am

@stevebaxter First of all thanks for the code and your explanations in this thread. It helped met to setup a puls meter for my Solar Panels.
I was wondering if you could help me with one issue I have. Sometimes I get big spikes.
I already set my internal_filter to 500ms which helps a little. When I set a median the spikes are gone but then the out ut never reaches 0, it hovers around 36W.

Could you help me remove the spikes but still get the output to 0. What am I missing?

massive · January 27, 2021, 4:39pm

Just chipping in here to thank @stevebaxter! I found your pulse meter quite accurate. I’ve built a power monitor using M5Stick and accompanied LDR that reads impulses from a led flashes our electricity meter generates.

It is surprising that pulse meter was so much more accurate than built in pulse counter. In fact, pulse counter reported values that were off over 30% from the actual numbers.

I’m comparing the numbers to my electricity distributor numbers that I can check from their app a few days after the fact.

So just saying that I appreciate your work, and as it seems to work so good, ask if it could be merged to the esphome main line?

I had a few issues here and there. I also got huge spikes which I had to filter by tweaking the median filter and changing input pin to inverted. Additionally, I’m seeing some jitter in the power numbers, but I suspect that it’s probably some underfloor heating system going on and off. This needs to be verified.

stevebaxter · January 30, 2021, 1:17pm

How big is your median filter? I saw spikes as well, I’m really not sure of the source, but most likely noise on the input (this wouldn’t be visible with the existing pulse_counter because that essentially averages everything over 1 min or whatever you set as your interval). I found that a very short median filter of window_size: 3 fixed this for me. My latest config for reference:

sensor:
  - platform: pulse_meter
    name: "Electricity"
    pin: D5
    unit_of_measurement: "kW"
    internal_filter: 100ms
    accuracy_decimals: 3
    timeout: 2 min
    filters:
      # Filter outliers
      - median:
          window_size: 3
          send_every: 1
          send_first_at: 1
      # Convert pulses/min (Wh/min) to kW
      - multiply: 0.06
    total:
      name: "Electricity Meter"
      unit_of_measurement: "kWh"
      accuracy_decimals: 3
      filters:
        - multiply: 0.001

  - platform: pulse_meter
    name: "Gas"
    pin: D6
    unit_of_measurement: "kW"
    internal_filter: 4s
    accuracy_decimals: 3
    timeout: 2 min
    filters:
      # Convert 0.01 m³ (one pulse) to kWH/min using:
      #
      # 1.022640 * 39.2 / 3.6 / 100 = 0.1113541333
      #
      # Then kWH/min to kW by multiplying by 60 = 6.6812
      - multiply: 6.6812
    total:
      name: "Gas Meter"
      unit_of_measurement: "m³"
      accuracy_decimals: 2
      filters:
        # One revolution of the small dial is 10l
        - multiply: 0.01

500ms as the internal filter might be a bit long - you may under-read if a real signal involves more than two pulses per second.

If the reading isn’t reaching 0, check your timeout value. This is used to decide when we’re at 0 (we can only update the output when we see a pulse, so if we never get a pulse we “stick” on the last reading, the timeout value gets around that). Also, are you maybe really seeing 36W? For a system that has one flash per Wh, 36W is a flash every 100 seconds, which is quite a long time. You can look at the absolute counter value to see if you are getting counts at a very low rate.

Can you post your “sensor” config?

stevebaxter · January 30, 2021, 1:27pm

Thanks!

pulse_meter should be much more accurate than pulse_counter, particularly for the “absolute” count. I’d love for this to be merged, there’s a pull request here awaiting review from the ESPHome maintainers:

If you could add a voice asking for it to be merged it might help!

I think some jitter in the power numbers is inevitable (and might be real). Devices don’t consume a constant level of power, it will vary from second to second depending on load (even a fridge will vary its load when the compressor is running I think). Readings will also be sensitive to any jitter in the timing and interrupt handling on the board - I think this is likely to be small though, if I have my maths right, 1 ms jitter (quite a lot) would only make about 10 mW of difference for a typical meter that outputs 1 pulse per Wh.

mhooghiemstra · January 31, 2021, 9:50am

Hi Steve,
Thanks for your reply.
I first had my settings like you. Just copied your settings how that would work. Worked good only I still got some spikes therefore tried below.

sensor:
  - platform: pulse_meter
    name: "Solar Panels"
    pin: D1
    unit_of_measurement: "kW"
    internal_filter: 500ms
    accuracy_decimals: 3
    timeout: 1 min
    filters:
      # Filter outliers
    #  - median:
    #      window_size: 7
     #     send_every: 4
     #     send_first_at: 1
      # Convert pulses/min (Wh/min) to kW
      - multiply: 0.06

I did comment out the median because this also doesn’t show 0w.
It should definitely read 0 because it is measuring my solar panels and at night there is no sun .
Hope you can point me in the right direction.

stevebaxter · January 31, 2021, 3:25pm

Window size of 7 is probably a bit big - can you try 3?

I’d still really check you aren’t seeing any counts at night. I guess you may still get small amounts of power generated from moonlight, or possibly a reverse flow back into the panels (that may show up on the meter as pulses even if it is counting backwards).

seb888 · February 4, 2021, 7:33pm

Hi @stevebaxter ,
Very good job !!
The sensor works perfectly in my installation.
I have a question in order to simplify my board, is-it possible to activate the pull-up fonction on the GPIO ?
Thanks !

stevebaxter · February 5, 2021, 11:52am

Hi, I think you can do that in the pin config, so something like:

sensor:
  - platform: pulse_meter
    name: "Electricity"
    pin: 
      number: D2
      mode: INPUT_PULLUP
    unit_of_measurement: "kW"
.
.
.

This is standard behaviour for any pin config I think. See https://esphome.io/guides/configuration-types.html#config-pin-schema

stevebaxter · February 5, 2021, 11:54am

I wonder if that is the cause of some of the spikiness we’re all seeing…

seb888 · February 5, 2021, 12:04pm

actually it isn’t possible (you have a yaml error) because pulse_meter herit from sensor (input_pullup is not implemented) and not gpio_binary_sensor (where pullup is implemented)

stevebaxter · February 5, 2021, 12:19pm

Ah interesting, let me try and see if this can be fixed.

orange-assistant · February 6, 2021, 2:48am

Or just counting the total which was introduced with esphome 1.16



# Example configuration entry
sensor:
- platform: pulse_counter
    pin: 12
    unit_of_measurement: 'kW'
    name: 'Power Meter House'
    filters:
      - multiply: 0.06

# Highly accurate using total
    total:
      unit_of_measurement: 'kWh'
      name: 'Energy Meter House'
      filters:
        - multiply: 0.001

stevebaxter · February 6, 2021, 1:35pm

Or use pulse_meter which gives a total and more accurate data .