Experiment - Heat Pump vs Gas Boiler - the ultimate comparison in efficiency and operation cost

With the increasing energy prices here in Belgium (and the rest of Europe), I decided to do a (rather expensive) experiment which hopefully pays off: we’ve installed a heat pump next to our existing gas boiler. This means we currently have a hybrid setup where I can choose to either heat our home via the heat pump or via the gas boiler.

As I noticed that even my HVAC specialist does not have a lot of experience in these hybrid setups (nor in heat pumps too for that matter), I wanted to share my experience so other people can hopefully make a better informed decision.

The setup is as follows (thanks to ha-floorplanner):

  • House renovated and insulated in 2018 (completely controlled by HA :blush: )
  • Variotherm floor heating everywhere (this is not standard floor heating and responds much faster)
  • Heat Pump: Itho Daalderop HP S 95
  • Gas Boiler: Itho Daalderop Cube One
  • Boiler / Tank (only for storage): ACV Smart E Plus 240 (combined tank with tapwater heated inside a smaller tank within the bigger tank)
  • Average gas usage for the last years:
    • 2019: 8823 kWh
    • 2020: 9539 kWh
    • 2021: 11715 kWh (worked a lot from home and my office is the least insulated room)
    • 2022: 2006 kWh (not a cold winter, decided not to heat/use some rooms anymore including my office and lowered thermostat to 18 °C)

As I’m interested in the actual efficiency of the heat pump (COP / SCOP) and to have a realistic comparison, my intention is to heat our home using the gas boiler on the even days and the heat pump on the uneven days for a limited amount of time (probably 2 weeks).

During this period, every 5 minutes data will be appended to a CSV log file:

  • date & time
  • whether it’s a heat pump day or a gas boiler day
  • outside temperature
  • living room temperature
  • boiler hot tap water temperature
  • total m3 gas consumption
  • daily m3 gas consumption
  • total kWh heat pump consumption
  • daily kWh heat pump consumption

I think this will allow me to calculate everything we need to make a good comparison (if you think something is missing, do not hesitate to reach out). On top of that, home assistant already calculates the degree days and more:

Currently, we still have net metering and solar panels - that’s why I want to limit the time of the experiment. But, I also want our family to already prepare for when net metering here in Belgium ends and that’s why I decided to see my house as a battery that will be heated ( = charged) when electricity prices are low (somewhere between 01:00 and 06:00) to 21 °C.

Probably the heating will need to turn on once more during the day. Below the graph for our living room temperature today where heating started at 3:20 until 6:00 and 17:00 to 18:30 using the heat pump.

The experiment starts now (yes, that’s Friday the 13th) unless someone can pinpoint me on a reasoning mistake and this topic will be updated with my findings. Off-course, I’m more than interested in feedback / ideas from the community but may not respond fast due to work or sleep deprivation :wink: .


I have a hybrid setup too- my house has boiler baseboard heat, and I wanted to add air conditioning, so I installed mini split air conditioners for cooling. I never thought about using them for heating initially, but for a few dollars more, went with the heating option, mainly as a backup if the boiler goes out.

I live in the US near Niagara Falls- it gets very cold here in the winter, and our design temp for hvac here is 7F. The heat pump can work down to -5 farenheit, but it occasionally gets colder.

I’ve wondered with the increasing price of natural gas, can the heatpump be cheaper? From research I’ve read, there are so many variables, it’s hard to tell. I also have an outdoor reset on the boiler, which changes the water temp in the system as the outdoor temp changes. My gas usage has gone down significantly since installing the reset, but its yet another variable.

One of the hardest things to account for is that the efficiency of a heat pump changes as temperatures change. On a mild winter day of 40 degrees, a heat pump can actually be over 300% efficient, but on a 5 degree day, that efficiency plummets to under 100%, which is worse then just electric resistive baseboard. I’ve heard a rule of thumb is use the heat pump when Temps are above freezing, and gas below freezing. I’m sure home assistant can be used to automate these changes, but haven’t done them yet

I’d be interested in your experiments results, but I think you would have to account for the changing outdoor temperature somehow. If on an odd day outdoor temp is 20 degrees, and an even day it warms to 40, I could see your results being skewed by the changing efficiency of the heat pump.


That’s why I want to run the experiment for 2 weeks so these temperature differences per day also are taken into account in the end result.
I also plan to calculate the Degree Days which takes the outside temperature into account when calculating performance of heating systems.

Very interesting experiment! I will follow this with great interest.

One remark: a heat pump is less efficient in very cold temperatures, but never goes below 100%. An hvac might be different, but a normal hesting air/water heat pump will usually still have a COP of 2 at -5 to -10 and worst case it will enable it’s resistive Hester addon if it can’t provide enough heat on its own. That would drive efficiency down, but it should still be at or above 100%

Mine unfortunately doesnt have resistive backup. Combined with 50 feet of line set outside, and I’m not sure if it remains over 100% at the minimum operating temperature.

According to this article, at -18c, heat pump cop can fall under 1, so i wouldnt say “never” falls under 100%. Now, that’s a very cold trmp, and my current model wouldn’t even work that low, but we do see temperatures that low where I live from time to time.

Also, you can lose half the rated btu performance at very low temperatures

I think heat pumps have a place and can save money in the right conditions, but I would still want a gas or resistive backup at very low temperatures.

Extreme temperatures like -18°C of course require a powerful heat pump and often need resistive heating backup. That’s why it is so extremely important to have the heat pump planned well. Also, there are very strict rules (at least here in Germany) on how long your outdoor pipes are allowed to be. Ideally it would be far less than 1m of pipe outside the thermal envelope of the building.
And here in Germany as far as I am aware all hest pumps come with resistive heating backup for temps below the efficient operating value.

Small update: off-course, Friday the 13th did not do any good :grinning_face_with_smiling_eyes: .

I had some faulty cabling which resulted in the gas boiler not starting. This was luckily easy to to resolve.

But, on top of that, I think that the installation manual of the ACV Smart E Plus is not that clear as we’re currently unable to warm the house via the gas boiler. Reason for this is that the water that departs to the floor heating is completely at the bottom whereas the return water from the gas boiler is higher inside the ACV boiler (you can see that in the first picture I posted). Because of this, there are 2 different water currents inside the boiler which are not crossing one-another. This results in the top part of the boiler going easily to 60 °C but the floor heating departure temperature never goes above 21.5 °C. With the heat pump connected on the same height as the floor heating, which is connected the floor heating departure temperature can go up to 35 °C.

Long story short: next Wednesday we’re going to have a look at the piping again. It’s really an experiment (for both me and my HVAC guy)!

Piping these systems can be tricky. I would checkout some hvac forums for posts or to ask for some help.

1 Like

Am wondering @rdehuyss; what is now the status of the experiment? Eager to learn from your efforts :slight_smile:

Hi @fversteegen - well, as the piping was wrong, this will only be fixed somewhere in April. So, I’m afraid I cannot do any tests this winter :frowning_face: .

Is your piping fixed and ready for the next winter?
Can you also show me how you have configured your DD sensors?

Hi @rdehuyss,
Have you been able to run the experiment ? :slight_smile:
I just renovated a house (100yo) but could not put insulation on the walls (only floor and roof). The old gas boiler is dying at the moment and I still did not decide what I’ll do regarding heating. So I also have much interest in your results (although I don’t have underfloor heating).

Many thanks !

Piping is fixed, still need to replace a battery sensor in the outdoor temperature sensor and the experiment can start.


I changed from oil (no gas here) to Ecodan 14Kw ASHP about 10 years ago in a 40 year old house and wonder if I did the right thing?

The main issue is that an ASHP works best in a well insulated home plumbed with underfloor heating, manifolds and large pipework to provide the necessary flow rates for the much lower temperature circulating water e.g. 35degC from ASHP, 77degC from gas or oil boiler.

The installers fitted some larger radiators but didn’t change any of the pipework so the ASHP can’t achieve the necessary flow rates through the old 10mm microbore pipe to most of the rads and they connected up 2 loops of underfloor heating to 15mm pipe where an old rad was, rather than back to a manifold connected direct to ASHP. As a result, my ASHP is always cycling on and off because it can’t dump the heat quick enough…

So IMHO I would not advise replacing an oil or gas boiler with an ASHP unless you also replumb the (resized) radiators and/or UFH with bigger pipework, manifolds etc.

Finally, the battery from the outdoor temperature sensor is replaced… this means I am all set for the experiment!


  • on uneven days the gas boiler will be used
  • on even days the heat pump will be used
  • each 5 minutes, the following data is logged:
    • date & time
    • whether it’s a heat pump day or a gas boiler day
    • outside temperature
    • the degree day (16.5 - outside temperature)
    • living room illuminance (as we’ve got a lot of south facing windows, this impacts the overall picture quite a lot)
    • living room temperature
    • boiler hot tap water temperature
    • total m3 gas consumption
    • daily m3 gas consumption
    • m3 gas / degree day (triggered at 23:59:50 / uses yesterdays daily consumption and average temperature so will be fixed per day)
    • kWh gas / degree day (triggered at 23:59:50 / uses yesterdays daily consumption and average temperature so will be fixed per day)
    • total kWh heat pump consumption
    • daily kWh heat pump consumption
    • kWh heat pump / degree day (triggered at 23:59:50 / uses yesterdays daily consumption and average temperature so will be fixed per day)

If anybody thinks I’m missing something, feel free to report now!

1 Like

Hi Pete - you’re completely correct. In hindsight, even I find that the size of the different pipes to the collectors where my floor heating starts in each room was chosen too small.

Absolutely awesome project!

You need to factor in the amount of hot tapwater used. If you use a bath or long shower the will skew your results as well.

Edit: grafana will enable you some beautiful graphs, aggregations and math!

You’re correct, I hope that by alternating each day from heat pump to gas we will get good “sample” data and this is thus averaged out (we do not have fixed bathing days :rofl:)

1 Like

I find the low ambient temperature efficiency figures for ASHP worrying. At 100% and with electricity costing 4 times as much per kWh as electricity, on a cold day like it is today in the South West UK (2oC) we will use as much as 50kWh of gas keeping our home warm enough for my 80 year old mother. The running cost for a heat pump would be far higher than our gas boiler.

Good luck with your experiment but I am already a long way from convinced that these heat pumps are a realistic option for many on limited incomes.

Your worry is understandable, however, while the coldest times of the year might be more expensive, it should still be cheaper most of the time. Temperatures around or below 0 are statistically quite rare.
Also… never forget that prices for oil and gas are unlikely to stay low in future, so unless electricity prices keep climbing as well, the cost benefit of heat pumps should increase over time.
Since nobody can forsee the future though… it’s always going to be a bit of a gamble, no matter what you do.