[SOLVED] Bug report - Error in calculation

In somehow calculations on HA is broken. I am getting the local price of electrical power in Euros per MWh. To find the cost of Euros per kWh I am dividing the value by 1000. Instead of just returning a shift on the price dot, its gets changed. See below.

Can someone advise what to do? I have never seen a computer doing wrong calculations…

Data received at MQTT:

Data on Lovelace card after the calculation:

HA version:

Sorry… Below the sensor definition:

- name: OMIE
  state_topic: "OMIE"
  value_template: "{{ value_json.omie | float / 1000 }}"
  unit_of_measurement: "€/kWh"

- name: OMIE DAY
  state_topic: "OMIE"
  value_template: "{{ value_json.data }}"

It’s not a bug. Computers don’t have infinite precision. Just like in decimal how you can express 1/9 as an infinite repeating expression 0.11111111, some floating point numbers cannot be expressed exactly in binary. If you truncate 0.11111111 at some point (like 8 decimals) you don’t get back to exactly 1/9.

Solution: apply round.

{{ (value_json.omie | float / 1000) | round(6) }}

Tip: If you’re going to do more calculations and you want the maximum precision, work with the original values.

If you made this sensor to use kWh for presentation purposes, not that you can select the presented precision from the UI.

I am using a scrapper to get this value. I will fix the problem doing a left shit on the original value dot and publish it to MQTT with the kWh value.

Sorry but rouding a division by 1, 10, 100, 1000, etc. does not make sense. You may be right but I will ask this question on a computer science forum as, no matter what, if you divide 1.30 by 1000, then the result must be 0.0013 and never a possible repeating expression.

There’s no need for that:

  1. I have a degree in comp sci.
  2. I’ve written a software FPU according to IEEE 754 — twice.
  3. 15. Floating Point Arithmetic: Issues and Limitations — Python 3.12.3 documentation

Just remember, even though the printed result looks like the exact value of 1/10, the actual stored value is the nearest representable binary fraction.

Bit shifting isn’t going to help you. It has nothing to do with the division. It’s how that specific value can be presented in binary.

2 Likes

It is just great to know that I am talking to an expert. So, before ahead, thank you for spending your time with me.

This said, my scrapper runs in bash as a once a day crontab job. To fix the problem I will send the kWh price instead of MWh. I have used Linux bc for this. For the sake of testing, I have asked bc a “scale” of 50 (50 decimals places). Look the code below.

Just for the sake of learning, can you tell me why bc does the right calculation and Python doesn’t?

bc <<< 'scale=50; 9.21/1000'
.00921000000000000000000000000000000000000000000000

Please don’t bother responding the above question. I went to ChatGPT and put the question.

For those who got curious like I was, I let below the explanation.