Pulse_meter counting too many pulses

[prajuriz]

Hello all,

I have connected a reed switch to two different GPIO to have both a binary_sensor and a pulse_meter, as I wanted to troubleshoot strange pulse/min nb I was getting.

The original issue is resolved (although unclear what happened, I didn’t change anything) but I discovered a new, smaller one: pulse_meter is counting slightly more pulses than binary_sensor.

binary_sensor is perfectly correct after 10000 pulses, when comparing with the meter.

pulse_meter has 13 pulses too much. That’s 0.1% error, so it’s no big deal, but since the binary_sensor is perfectly correct, I feel like there’s an easy software solution to this and I want to research it.

Currently my main suspect is the fact that the meter wheel sometimes leaves the reed switch closed for a longer period of time when the meter wheel stopped with the magnet facing the reed switch.

Any reason this could cause the pulse_meter to count two pulses instead of one?

I checked, there was roughly 20 instances where this happened within these 10000 pulses. This doesn’t really align with the 13 additional pulses unfortunately.

Any other idea what could cause the pulse_meter to count one pulse too much?

[mekaneck (Rick Auch)]

You might find this post useful for explaining the difference between the methods. I’m guessing you’ve set up your binary_sensor to mimic a pulse_counter but it’s hard to know without seeing your code.

The pulse_meter has a number of options that could affect how many pulses you back-calculated from it. Seeing the code for this sensor also would be useful.

[prajuriz]

Thanks for the link.

Please find below the yaml.

total_count of binary_sensor is perfectly accurate, while the total of pulse_meter has a couple of pulses too many.

What I don’t understand is how it can be that there are too many pulses. Somewhere the pulse_meter is inventing a pulse that the binary_sensor is not seeing…

external_components:
  - source: github://syssi/esphome-total-count@main

total_count:
  binary_sensor_id: "binary_reed"
  restore: true
  min_save_interval: 300s
  initial_value: 0
  step: 1

binary_sensor:
  - platform: gpio
    name: "binary_reed"
    id: "binary_reed"
    pin:
      number: GPIO26
      inverted: true
      mode: INPUT_PULLUP
    icon: 'mdi:fire'
    device_class: gas
  
sensor:
  - platform: total_count
    total_count:
      name: "gasmeter binary_sensor total count"

  - platform: pulse_meter
    name: "gas consumption"
    id: sensor_pulse_meter_gas
    pin:
      number: GPIO27
      inverted: true
    device_class: gas
    state_class: measurement
    accuracy_decimals: 0
    internal_filter_mode: EDGE
    internal_filter: 3400ms
    timeout: 20s
    icon: 'mdi:fire'
    total:
      name: "total gas consumption"
      state_class: total_increasing
      device_class: gas
      state_class: total_increasing
      accuracy_decimals: 0
      filters:
        - debounce: 200ms

[prajuriz]

It appears that the pulse_meter is counting one more pulse at the beginning or at the end of a “pulse block” as I call it.

Here is how my pulses look like from the binary_sensor point of view (reed switch is connected to two GPIO, reminder), it’s a serie of relatively regular on/off events, which I subsequently refer to as a “pulse block”.

I have graphed the count of pulses for both the binary_sensor and the pulse_meter and this is what I see.

Example at the beginning of a “pulse block”. Blue is the pulse_meter, you can see it jumps one pulse earlier than the binary_sensor:

This is what the binary_sensor shows at this time: there is only two pulses before 22:55:30, not three pulses as the pulse_meter is counting.

I have also observed the same issue at the end of a “pulse block”. Here the pulse_meter total is already one more than the binary_sensor count at the beginning, but at the end, it is visible it is then 2 pulses above, it gains again a pulse at the very end of the pulse block:

While the binary sensor looks like this at the end of the block, clean cut pulses only, and a single pulse after 01:12:20, not two like the pulse_meter total is seeing:

It feels like a bug in the pulse_meter total code but I really don’t see what the error in the code could be…

[mekaneck (Rick Auch)]

How do you know the binary_sensor is correct and not the pulse_meter? Having the pulse_meter trigger early seems impossible unless you’re suggesting that a bug in the software allows it to predict the future.

[prajuriz]

I know, it sounds crazy, but the total of binary_sensor has the exact correct number of pulses after 10000+ pulses, compared to what I read on the meter, whereas pulse_meter is off by a few. I can’t see how binary_sensor can be wrong here then.

I can’t explain either how pulse_meter could be triggering early, unless the total of pulse_meter is calculated somehow retroactively and there’s some form of error or time shift there…

[mekaneck (Rick Auch)]

I’m not good at reading code, so I won’t be able to give a conclusive answer to your question. Hopefully you can get someone else to chime in.

I’m interested what your pulse signal actually looks like. Is it a 50% duty square wave that transitions when 50% of one increment of gas consumed, or do you get very short duration pulses whenever one increment of gas has been consumed? And since you have invert: true configured, is the ESP only seeing logic low for a very short time?

What my uneducated eye sees in the code is that the pulse meter total count is incremented on a falling edge (as long as the logic high duration was sufficiently long). And the binary sensor appears to be incremented on a rising edge (ignoring subsequent rising edges that happen in the filtered time period).

So that would make me believe that, when the signal is logic high for a long time, and then finally falls to logic low, the pulse meter will increment the count but the binary sensor will not.

I haven’t checked out the code for the binary sensor and the debounce settings, but there’s probably cases where you are getting noise on your signal and the binary sensor is debouncing it with the long debounce you have configured (3.4 seconds) but the pulse meter isn’t.

I’m just throwing darts here, that’s about as good as I can do.

[prajuriz]

@mekaneck

I’m interested what your pulse signal actually looks like

You mean the analog signal/voltage? I need to research how to graph this. If you have a pointer, let me know. I expect the anolog signal of a reed switch to be very blocky: the meter has a wheel with a magnet, when the magnet is close enough, the reed switch closes and the voltage on the GPIO jumps, causing a raising edge. Same when the wheel turns further, switch opens again and voltage goes back to zero. It might be there is noise, but the 3400ms filter is taking care of this.

@mekaneck

And since you have invert: true configured, is the ESP only seeing logic low for a very short time?

No, that’s because I connected the reed to GND, hence voltage is negative when when the switch closes. This is described here:

if you are connecting the reed switch from GPIO to GND, LOW will mean the door is closed so you won’t need to invert the pin.

@mekaneck

the pulse meter total count is incremented on a falling edge

Correct, but now that you are saying this I notice something: the function PulseMeterSensor::loop() (line 44) is not happening at the same time as the edge detection in the code (line 57).

I wonder if that could be the problem.

In any case, the fact the pulse_meter is incremented on falling edge means the total should not increment at the same time as the total of the binary_sensor, it should be delayed, which is not at all what I’m seeing in the graph. That would align with the fact that the total is incremented before the falling edge, which is what I pointed to earlier.

@mekaneck

there’s probably cases where you are getting noise on your signal and the binary sensor is debouncing it with the long debounce you have configured (3.4 seconds) but the pulse meter isn’t.

The 3400ms is for the pulse_meter, not for the binary_sensor. Hence if anything is being debounced, pulse_meter is the one that should end up with less pulses. There is no debouncing on the binary_sensor because I wanted to be certain to see everything, even the noise.

Since then I actually added this to binary_sensor to see if debouncing would bring down the binary_sensor to the level of the pulse_meter, but it did not.

    filters:
      - delayed_on_off:
          time_on: 200ms
          time_off: 200ms

In summary, I wonder if there isn’t a logic issue in the pulse_meter code whereby the total is incremented twice for one pulse when certain conditions are met.

[prajuriz]

Something is very off with pulse_meter somehow, I really cannot understand what is happening:

Increasing both on rising and falling edge:

Extraneous pulse at the end:

Random pulses out of nowhere:

[prajuriz]

Update:

I changed internal_filter_mode from EDGE to PULSE (and changed value from 3400ms to 200ms) and added a debounce: 400ms filter to the total of the pulse_meter, and now I’m getting too little pulses.

Now I see things like this: pulse_meter total is missing a pulse which is completely normal according to binary_sensor total count. It’s above 900ms wide, hence I don’t see how it could be mistakenly debounced, it’s the same size as all pulses before and after. It gets more and more confusing (at least for me), it seems random, which I would then attribute to the reed switch, but the binary_sensor is doing just fine and counting pulses very accurately, so that cannot be it.

My current impression of pulse_meter is that it’s unreliable, simply put.

[BigG]

I am using pulse_meter to capture electric usage off the flashing led on my electric meter and the usage is ever so slightly over what the smart meter readings from my supplier say.
I have tried a number of different things with exactly the same result.
I will now try using a binary sensor and compare.

[prajuriz]

After adding the PR 6126, I am now down to 0.3% deviation (too many pulses) with pulse_meter. Still a mystery why.

[BigG]

My readings are 0.5% over, what is PR 6126 ?

[prajuriz]

Sorry, I’m on mobile.

This one: Check for missed pulse_meter ISRs in the main loop by TrentHouliston · Pull Request #6126 · esphome/esphome · GitHub

You can add it like this:

external_components:
  - source: github://pr#6126
    components: [ pulse_meter ]

before adding it I add more like 1% missing pulses, now it’s 0.3% too much. So it’s better, but still a mystery to me how it can fail, because the pulses are very clean and super regular in my case…

[BigG]

Thanks for the reply, I’ll add that in and see how it goes.