How to count pulse frequency accurately with ESPHome?

ESPHome
1

Hi,

I’ve got a light sensor on the flashing led of my electric meter, and using the pulse counter in ESPHome, I’ve got it working fairly well, and it gives a ballpark figure in KW every 60 seconds. The problem is the pulse meter sensor in ESPHome appears to be inaccurate, in that it tells you how many times it’s flashed in a given period, and thus is confined to give whole numbers. For example, if the led on my meter is flashing once every 50 seconds, and I want the reading every minute, ESPHome alternates between reporting 1 or 2 pulses per minute. I’d like accurate data, so I’m trying to work out how to get it report the correct 1.2 pulses per minute.

Here’s my code:

  - platform: adc
    pin: GPIO35
    id: adc_sensor
    attenuation: 11db
  - platform: pulse_counter
    pin: GPIO23
    unit_of_measurement: 'kW'
    name: 'Power to yard'
    update_interval: 60s
    filters:
      - multiply: 0.06
2

The only way to get more accurate data is to count over a longer interval.

3

That seems like a massive design flaw of the system, in that it has no choice but to give a rounding error.

Does this mean if I want an accurate power readout, I’ll have no choice but to determine the time between pulses somehow, and then once every minute poll what that number currently is?

4

So…?
You have a photoresistor that reads the led and that hooked up to the analog input?

You could also use a digital input and just pass it to a counter (or something) in HA.
That way the LED “wakes” up the ESP.

Or maybe I don’t understand…

EDIT: maybe I understand now. Can’t you use a statistics sensor to smooth out the data?

5

The design flaw is your meter’s low pulse rate.

Set the update interval for 5 minutes. Capture 5 times as many pulses. Get at least a fifth of the quantisation error.

6

This isn’t an acceptable solution to me as it means you have to wait long periods of time to improve accuracy (and never achieve it). This seems a daft approach as after just two pulses all the information required has been gathered, and is exact. Waiting longer AND having less accuracy isn’t what programming is about in my opinion.

Anyway, I’ve solved it, and thought I’d share my code incase it helps others, or ESPHome want to use it to improve their https://esphome.io/cookbook/power_meter.html example.

  • platform: pulse_width
    pin:
    number: GPIO23
    inverted: False
    name: Pulse Width Sensor yard
    update_interval: 5s
    filters:
    • lambda: return 3600 / x;
      unit_of_measurement: “W”

The above code uses the pulse width sensor to measure the time between pulses in seconds and then divides 3600 by it to give the watts being consumed between the last two pulses.

At the moment there is a problem of the reading only changing when a new pulse is received. So if the house was consuming 1000w and then went to a consumption of 0w, the value would stay at 1000w forever. I’ll have another cup of tea and a think, but I guess my only solution would be to have the sensor detect when a certain time had passed and then drop the output.

Hope this helps someone, and if anyone can think of a solution to the value adjusting with no pulses, that’d be cool.

1 Like
7

I think you have come full circle here. You want to use the ‘accuracy’ of the second method pulse to pulse, but the ability to see the behavior minute by minute. I’d suggest using either a wider the interval as already suggested, or shorten the interval of measurement and use a moving average filter. Say measure pulses every 10s, but average over a few minutes.

See:

8

For me, accuracy trumps frequency, so I guess I’ll just have to stick with what I’ve got. It’s only drawback is that very low values can “stick” but that’s only for 5 minutes at 12 watts. So I’ll probably leave it as the chance of my house drawing less than that are pretty slim.

9

None of us can help you make your power meter better. The data is the data. If you want more data, try a current meter.

10

I’m just going round this loop as well. I have a NodeMCU with two inputs, one from a hall effect sensor on my gas meter (which outputs one pulse per litre), and one from a photoresistor on the electricity meter (which outputs 1000 pulses per kWh, so 1 per Wh).

The problem is that using pulse_counter, with an update every minute, the resolution of the electricity meter is limited to 60W. To get to 1W of power resolution I would need to integrate over an hour which would give very poor temporal resolution.

Ideally, I think I want to do two things:

  1. Measure the pulse width (both sensors are active low, so we can measure the width of the “high” state to give the time between pulses). This needs to output on every pulse, not at a fixed time interval (because we know the power at every pulse).
  2. Count the total pulses - this gives an accurate total usage figure, e.g. for echoing the meter reading

Unfortunately, neither pulse_width or pulse_counter quite do what I need. pulse_width doesn’t seem to output on every pulse, pulse_counter doesn’t actually count the total pulses (though I see that https://github.com/esphome/esphome/pull/1173 has been merged but not yet released which should do this). Also, using pulse_counter and pulse_width on the same pin doesn’t seem to work (if you use pulse_width, the counter doesn’t see any pulses).

I’m thinking of writing a new sensor called pulse_meter that will give both an instantaneous measurement (by measuring the time between edges rather than the pulse width and outputting on every pulse) and also count the total pulses. Before I start tinkering with this, is there another way of doing this that I might have missed?

1 Like
11

Here is what I did:

globals:
# https://esphome.io/guides/automations.html#bonus-2-global-variables
- id: us_previousTriggerTime
  type: time_t
  restore_value: no
  initial_value: '0'
- id: us_pulsetime
  type: float
  restore_value: no
  initial_value: '0'
- id: us_power
  type: float
  restore_value: no
  initial_value: '0'

binary_sensor:
  - platform: gpio
    name: "Elecmeters US - Pulse"
    id: us_pulse
    pin:
      number: 27
      mode: INPUT_PULLUP
      inverted: yes
    # https://community.home-assistant.io/t/publish-timestamp-into-text-sensor/122531/10
    # 1 'unit'/hr = 3600 / pulsewith (seconds)
    # 1000 pulses = 1kWh
    # 1 pulse = 1 Wh
    # 1 pulse per 'unit' (Wh)
    # 3600/1 = 3600
    # Max power 17250W (25A)
    on_press:
      then:
        - sensor.template.publish:
            id: template_us_electricity_power
            state: !lambda |-
              auto currTime = millis();
              id(us_pulsetime) = (float) (currTime - id(us_previousTriggerTime));
              id(us_previousTriggerTime) = currTime;
              id(us_power) = 3600000.0 / id(us_pulsetime);
              if (id(us_power) < 18000) return id(us_power);
              else return {};

(Code borrowed and adapted from esp_easy forums).

I used to have a service call in this code block to increase a counter as well, keeping track of consumption (kWh).

However, it turns out that apparently the gpio platform does not utilise interrupts, cause I was seeing a discrepancy between my counter and the meter in the electricity cabinet.
I can even observe this with the kW sensor, with high loads (10kW) it sometimes drops to 5kW briefly, indicting a missed pulse.
I opened an issue on the ESPHome repo, but it received 0 replies…

So I added the pulsecounter platform to count pulses (kWh) and accepted the fact that the power sensor sometimes misses a pulse.

Sidenote 1:
Using a service call directly to HAS avoids loosing pulses when the ESP reboots (I want to avoid writing to memory in ESPhome, prolonging flash life), but now I have the problem I’m loosing pulses when HAS reboots. I am building an automation which at the end of the day compares daily pulses from HAS and ESP, and increases the counter with the difference.
EDIT: another downside of directly incrementing a HAS counter is that during WiFi disconnects (a rather common occurrence on ESPs) we miss pulses as well.

Sidenote 2:

Also, using pulse_counter and pulse_width on the same pin doesn’t seem to work (if you use pulse_width , the counter doesn’t see any pulses).

I have noticed this behavior on an ESP8266 as well when using the gpio and pulsecounter platform for the same pin. On an ESP32 this works.
A jumper wire solves this problem, but sacrifices an input pin (which are scarce on the 8266).

12

Thanks for the info! The GPIO component just uses a loop to monitor the input so is pretty much guaranteed to miss some pulses, particularly if they are short. We would need to use an interrupt (like the pulse counter component) to ensure that the pulses are detected correctly (along with the debouncing that the pulse counter provides).

I will hopefully have time to look at the C++ code over the next few days and put something together.

13

That would be great! Are you a ESPHome contributor?

14

Not yet! I will submit this back to the project though.

I have this working, just testing a bit and then I’ll post it here for people to play with.

15

OK, the attached code implements a new sensor called pulse_meter. This has a very similar setup to pulse_counter, with two key differences:

  1. It measures the time between the pulses and outputs a measurement for every pulse rather than a measurement every minute. This gives much better resolution.
  2. It has an additional “total” sensor that outputs the total number of pulses (this is the same as the current development branch of pulse_counter).

These two together allow you to get the best possible temporal resolution (a measurement for every pulse) and accurate totals (as we can really count the pulses rather than trying to integrate the frequency).

Here’s an example config for my electricity meter that outputs 1000 pulses per kWh:

sensor:
  - platform: pulse_meter
    name: "Electricity Meter"
    pin: D5
    unit_of_measurement: "kW"
    internal_filter: 5us
    accuracy_decimals: 3
    timeout: 5min
    total:
      name: "Electricity Meter Total Count"
    filters:
      - multiply: 0.06

The setup is much the same as pulse_counter with a couple of changes:

  1. You can’t specify rising or falling edges, we always look for rising edges (you can invert the pin if you want falling edges for some reason). In particular we can’t count both rising and falling, that doesn’t make sense for this sensor.
  2. I’ve added a timeout value, this defaults to 5 mins. If we don’t see a pulse for this amount of time we assume 0 pulses/min. If we don’t do this then we get “stuck” on the previous measured value if the pulses stop (because we normally update the value on each pulse).

Here is some output showing pulse_meter vs pulse_counter:

Screenshot 2020-12-27 at 15.59.28
Screenshot 2020-12-27 at 15.59.282129×410 39.3 KB

You can see that the new sensor (blue) tracks the usage much more quickly than the pulse counter (red). In particular it shows spikes that the pulse counter will miss, and doesn’t hunt up and down (see the period from 14:39 to 15:09 as a good example). It does however track the old sensor pretty well which is reassuring.

You can download the new sensor here:

Unpack it into your esphome directory (this will be in the config directory for hassio).

6 Likes
16

Thanks @stevebaxter for the improvement on the counter!

Sadly it does not work on my side.
I also have a counter 1 pulse = 1 Wh
If i integrate the sensor 1:1 copy of our senor code i have 2 issues:

  1. the “Electricity Meter” continuously shows 0
  2. the “Electricity Meter Total Count” increases much too fast (i only have like 1 kW running)

image

any idea what is happening here?

Thanks!

17

Hmm, that’s odd! Can you share the output from the ESPHome device log? That will give me the full config and allow me to see exactly what is happening.

One thing I had to do was to use much longer values for the “internal filter” (around half the minimum pulse width), as noise on the pulses will cause very high readings. Even then I had a problem with occasional spikes, I used a median filter to remove that. That doesn’t explain what you’re seeing though.

This is my full sensor config for reference:

sensor:
  - platform: pulse_meter
    name: "Electricity"
    pin: D5
    unit_of_measurement: "kW"
    internal_filter: 100ms
    accuracy_decimals: 3
    timeout: 2 min
    filters:
      # Filter outliers
      - median:
          window_size: 3
          send_every: 1
          send_first_at: 1
      # Convert pulses/min (Wh/min) to kW
      - multiply: 0.06
    total:
      name: "Electricity Meter"
      unit_of_measurement: "kWh"
      accuracy_decimals: 3
      filters:
        - multiply: 0.001

  - platform: pulse_meter
    name: "Gas"
    pin: D6
    unit_of_measurement: "kW"
    internal_filter: 4s
    accuracy_decimals: 3
    timeout: 2 min
    filters:
      # Convert litres (one pulse) to kWH/min using:
      #
      # 1.022640 * 39.2 / 3.6 / 1000 = 0.01113541333
      #
      # Then kWH/min to kW by multiplying by 60 = 0.66812
      - multiply: 0.66812
    total:
      name: "Gas Meter"
      unit_of_measurement: "m³"
      accuracy_decimals: 3
      filters:
        - multiply: 0.001

For the electricity meter my filter is 100ms (5 pulses a second is 18kW which is way more than our house would ever use). For gas it’s even longer (4 seconds).

I’ve updated the custom components archive with the latest code (there were some changes which were only cosmetic but maybe I missed something). You can find it at the same URL (https://www.dropbox.com/s/6tn8th7mmtvwlup/custom_components.zip?dl=0).

1 Like
18

Actually, are you sure your multipliers are right in the filter? Looking at your graph, is that over 5 seconds or 5 mins? If it’s 5 mins there appear to be 14-15 steps each min, that’s 840-900/hour or nearly 1 kW which sounds about right. The raw data from the log will tell us though!

19

Thank you @stevebaxter!

With the new code it is working (don’t ask me why :wink: )

I have this energy counter (sorry only found it in German) - 1000 imp/kWh
SDM72D

Log output for reference:

[19:33:28][C][logger:185]: Logger:
[19:33:28][C][logger:186]:   Level: DEBUG
[19:33:28][C][logger:187]:   Log Baud Rate: 115200
[19:33:28][C][logger:188]:   Hardware UART: UART0
[19:33:28][C][pulse_meter:052]: Pulse Meter 'Electricity'
[19:33:28][C][pulse_meter:052]:   Unit of Measurement: 'kW'
[19:33:28][C][pulse_meter:052]:   Accuracy Decimals: 3
[19:33:28][C][pulse_meter:052]:   Icon: 'mdi:pulse'
[19:33:28][C][pulse_meter:053]:   Pin: GPIO12 (Mode: INPUT)
[19:33:28][C][pulse_meter:054]:   Filtering pulses shorter than 100000 µs
[19:33:28][C][pulse_meter:055]:   Assuming 0 pulses/min after not receiving a pulse for 120s
[19:33:28][C][homeassistant.time:010]: Home Assistant Time:
[19:33:28][C][homeassistant.time:011]:   Timezone: 'CET-1CEST-2,M3.4.0/2,M10.5.0/3'
[19:33:28][C][captive_portal:169]: Captive Portal:
[19:33:28][C][ota:029]: Over-The-Air Updates:
[19:33:28][C][ota:030]:   Address: stromzaehler.local:8266
[19:33:28][C][api:095]: API Server:
[19:33:28][C][api:096]:   Address: stromzaehler.local:6053
[19:33:31][D][sensor:092]: 'Electricity Meter': Sending state 0.00100 kWh with 3 decimals of accuracy
[19:33:31][D][api.connection:604]: Client 'Home Assistant 2020.12.2 (192.168.123.60)' connected successfully!
[19:33:31][D][time:040]: Synchronized time: Tue Jan  5 19:33:31 2021
[19:33:38][D][sensor:092]: 'Electricity': Sending state 0.24951 kW with 3 decimals of accuracy
[19:33:38][D][sensor:092]: 'Electricity Meter': Sending state 0.00200 kWh with 3 decimals of accuracy
[19:33:45][D][sensor:092]: 'Electricity': Sending state 0.49903 kW with 3 decimals of accuracy
[19:33:45][D][sensor:092]: 'Electricity Meter': Sending state 0.00300 kWh with 3 decimals of accuracy
[19:33:52][D][sensor:092]: 'Electricity': Sending state 0.50293 kW with 3 decimals of accuracy
[19:33:52][D][sensor:092]: 'Electricity Meter': Sending state 0.00400 kWh with 3 decimals of accuracy
[19:33:59][D][sensor:092]: 'Electricity': Sending state 0.50293 kW with 3 decimals of accuracy
[19:33:59][D][sensor:092]: 'Electricity Meter': Sending state 0.00500 kWh with 3 decimals of accuracy
[19:34:07][D][sensor:092]: 'Electricity': Sending state 0.49924 kW with 3 decimals of accuracy
[19:34:07][D][sensor:092]: 'Electricity Meter': Sending state 0.00600 kWh with 3 decimals of accuracy
[19:34:13][D][sensor:092]: 'Electricity': Sending state 0.49924 kW with 3 decimals of accuracy
[19:34:13][D][sensor:092]: 'Electricity Meter': Sending state 0.00700 kWh with 3 decimals of accuracy
[19:34:19][D][sensor:092]: 'Electricity': Sending state 0.54836 kW with 3 decimals of accuracy
[19:34:19][D][sensor:092]: 'Electricity Meter': Sending state 0.00800 kWh with 3 decimals of accuracy
[19:34:26][D][sensor:092]: 'Electricity': Sending state 0.57398 kW with 3 decimals of accuracy
[19:34:26][D][sensor:092]: 'Electricity Meter': Sending state 0.00900 kWh with 3 decimals of accuracy
[19:34:32][D][sensor:092]: 'Electricity': Sending state 0.57398 kW with 3 decimals of accuracy
[19:34:32][D][sensor:092]: 'Electricity Meter': Sending state 0.01000 kWh with 3 decimals of accuracy
[19:34:38][D][sensor:092]: 'Electricity': Sending state 0.57361 kW with 3 decimals of accuracy
[19:34:38][D][sensor:092]: 'Electricity Meter': Sending state 0.01100 kWh with 3 decimals of accuracy

Thanks again for the code - looking forward to using this module for energy counting :partying_face:

20

Thanks for replying! Possibly there was some bug in the original code, apologies for that. Glad it’s working!