[Energy] Read IEC-62056 based electricity meter (Elster, Landis&Gyr etc..) and send data using Lorawan, WebHook and Template Sensors

Since the new awesome “Energy Dashboard” was released in Home-Assistent in 2021-08 I immediately felt the urge to use it. As I live on the third floor of an Appartement Building and my meter is in the basement I can’t use a Wi-Fi based approach to read my meter since there is no connectivity. That is where Lora comes in handy. I ordered a Heltec Lora Esp32 V2 Dev Board to play around with and was lucky to have a decent connectivity in my basement (RSSI: ~109 dBm / SNR: ~0 dB - using Akenza as an advisable LoRaWAN connectivity Integrator for Switzerland). Initially I built a similar approach as Home-Assitant Glow where the light pulses of my meter are being count, sent to the TTN and used the Riemann sum integration to calculate the kWh. I didn’t like it as it wanted to make my project battery powered (in the future) and when my household consumes less than 50 watt it runs quite a while to count for at least 2 light pulses. Also, I know that my meter already sums up my energy consumption (kwh) and many other OBIS values, thus I really wanted to get to that data using an optical reading head. As my meter requires an initial sequence to be sent to receive data, I needed a reading head (sometimes also referred to as optical probe), that has a photodiode (for RX) and infrared led (for TX) and use a serial interface (TTL) to interact with it using Arduino.

Project

  • Periodically read current power, energy consumption and other OBIS-values from an electricity meter that support the IEC 62056-21 protocol.
  • Use LoRa/TTN to send the data to a (public or private) gateway
  • The gateway POST’s the data to a Home-Assistant WebHook where the data is applied to several template sensors
  • I’m an absolute beginner with Arduino, Platformio and C++ so please have mercy

State

  • Project is still work in progress (feel free to contribute / integrated your smart-meter)
  • In the beginning I was experimenting arround with the Heltec lora esp32 microcontroller. That microcontroller is fully Arduino / Platformio compatible and work fine as long as you don’t power it by a battery as it consumes a lot of energy even in deep-sleep. Thus I switched to the Heltec Cubecell AB02 series that is specialized for battery operated LoRa use-cases.
  • I ran a long dozen of times up and down the staircase (third floor) down to the basement for this project - and yes, I got a sore muscle :sweat_smile:

My Hardware

Optical Probes:

Source:

Home-Assistant

Template Sensors

:warning: The strucure of trigger.json.data depends on your gateway/provider POST request

- trigger:
    - platform: webhook
      webhook_id: lorawan-data
      id: lorawan-data
  unique_id: smart_meter
  sensor:
    - name: "Smart Meter Power"
      unique_id: smart_meter_power
      icon: mdi:flash-outline
      unit_of_measurement: W
      device_class: power
      state_class: measurement
      state: >-
        {% set payloadHex = trigger.json.data.payload_hex | default(none) %}
        {% set value = payloadHex[0:4] %}
        {% if value == '0000' %}
          {{ none }}
        {% else %}
          {{ value | int(value,16) }}
        {% endif %}

    - name: "Smart Meter Kwh"
      unique_id: smart_meter_kwh
      icon: mdi:chart-histogram
      unit_of_measurement: "kWh"
      state_class: measurement
      device_class: energy
      #last_reset_type: auto
      state: >-
        {% set payloadHex = trigger.json.data.payload_hex | default(none) %}
        {% set value = payloadHex[4:12] %}
        {% if value == '00000000' %}
          {{ none }}
        {% else %}
          {{ value | int(value,16)/100|float }}
        {% endif %}

Utility Meter

Needed to integrate the values into the energy dashboard


utility_meter:
  hourly_energy:
    name: "Hourly Energy Consumption"
    source: sensor.smart_meter_kwh
    cycle: hourly
  daily_energy:
    name: "Daily Energy Consumption"
    source: sensor.smart_meter_kwh
    cycle: daily

Pictures

Electricity Meter and Optical Reading Head:

Home-Assistant Sensors:

Energy Dashboard:

5 Likes

Hello @pavax, great project man!
I am currently having the same problem as you have, my energy meter is in a cellar which is quite far away from my flat.
How is the project going along, is it working as you want it to be?

Unfortunately I never coded something for embedded devices, otherwise I’d love to help you with that. Maybe you need another tester :D?

Was there an improvment with the new Heltec LoRa Chip? Also, I guess you are based in Switzerland, but the Protocol of the D0 / SML “Bus” should be the same in germany if I got that correctly.
Is there a reason why you don’t put your data into Volkszähler and integrate that to HA?
Best regards from Berlin
Joe

1 Like

Hello @OhNoItSJoe, thanks for your interest.

The project is going very well. After a fiddled around with the heltec lora esp32 I was able to switch to a more energy efficient board, the Heltec CubeCell AB02-A. I’ve been operating it now for 2 months using a 1/2AA battery where the data is read every 20minutes. So far, the battery is still fully charged (~3.6v).

I updated my project and recommend using the CubeCell based board.

If the SML messages are send following the iec-62056-21 standard (see here) you should be able to use my project to read them. But there are some differences depending on the manufacture. I tried to extract configuration differences into variables that need to be adjusted depending on the type of smart meter. What is the exact type of your smart meter that you want to integrate?

It’s my understanding, that Volkszähler middleware is based on a raspberry pi. I didn’t want to install a raspberry pi in the cellar (my neighbours must already think that I’m crazy, but they would freak out/get paranoid if there was a mini-computer installed beneath their apartment) - Also I couldn’t power the raspi down there. Another option to use the volkszaehler project would have been to measure the data using the cubecell microcontroller, then send a LoraWAN message to the Lora-Gateway, then pass the data through to the raspberry pi volkszaehler middleware and then integrate that somhow into home-assistant. But that’s too complex, time-consuming and error-prune.

2 Likes

Interesting!
How did you enter the pin of the Zähler?

Yeah, that was my plan, to give Volkszähler the data via a script running on my server.

I will order the stuff and tell you if it works for me as soon as I get my new counter. Gonna be in a few months, but I want to be ready. Gotta talk to my landlady, but she is quite chill.

1 Like

Some smart meters (like the Elster AS1440) are sometimes protected with a password or pin (in my case there was no pin necessary). In that case, the pin must be provided in the start-sequence. I extracted another variable START_SEQUENCE that can be adjusted for that purpose. If that was the pin you were referring to.

Cool I’m really looking forward and hope you’ll succeed integrating your smart meter. Don’t hesitate to contact me should you need any help!

1 Like

Hello @pavax,
Sorry for the late reply, was quite busy.
Nice, that sounds perfect. Will still take a while until I get the Zähler, 2 months approx.
But I already want to try if the Signal even gets through, so I’ll order the heltec board.
What Gateway are you using?

Best regards
Joe

1 Like

I’m using the LoRaWAN network infrastructure from swisscom that covers 97% of the housing areas in Switzerland. Thus, I didn’t need to install/use my own gateway

Interesting!
Already sorry for all the questions, still trying to understand the whole LoRa stuff.

Do you have more Details about the webhook config? Didn’t find anything in your docu regarding that. Or maybe I’m just blind.

If I’d want to go for my own Gateway I guess I can still use your Arduino code and then just either let the gateway pass the data to HA via a webhook or just use MQTT I guess.

Still thinking about if I want to go with TTN or just build my own LoRa Network.

No Problem!

If I’d want to go for my own Gateway I guess I can still use your Arduino code and then just either let the gateway pass the data to HA via a webhook or just use MQTT I guess.

Exactly.

The Helltec Cubcell and thus my Arduino Code doesn’t have any Home-Assistan or MQTT related configuration. As it only sends out data (bytes) to the next available Gateway. The Lora Gateway receives the data and passes the payload (in my case, my gateway sends the payload in a field called payload_hex) through a POST http-request (webhook) to my home-assistant instance. Then the Template-Sensors that I defined (see my initial post) parse the HTTP-Body payload_hex field and parse the hex values according to the data structure that I defined in the code:

Still thinking about if I want to go with TTN or just build my own LoRa Network.

If you are lucky enough to have decent connectivity in your basement using the TTN then use that one. It’s the exact same setup as I have, except that I don’t use the public TTN but a commercial lora-network. The Thing Network Console also supports invoking HTTP-Request (Webhooks). Should you need to setup an own LoRa Network due to bad connectivity, there shouldn’t be any problem since all gateways supports to pass data through a webhook.

Do you have any recommendations for a cheap LoRa Gatetway?

I’m sorry but since I don’t have an own LoRa Gateway I can’t give you a recommendation…

Hi Pavax

Well done on creating this great project, I’m very interested in using it for my smart meter.

I have a similar use case as you except that I do not require LoRa as my smart meter is only 5m from my WiFi AP.

I was wondering if it would be possible to add WiFi as an option to your project?

Also can you suggest an ESP32 board that would work for this?

Many thanks

Dan

2 Likes

Hi @Subzero unfortunately I don’t have the time to work on a Wi-Fi version now… But my project is heavily based on the following project that uses the MQTT protocol to send data over Wi-Fi.

It works with any ESP8266 Board (such as a Wemos D1) - It might also work with an Esp32 though a bit over dimensioned

1 Like

Thanks, that’s a great place to start.

did you create any public solution for esp ieC 62056 reader?

@pavax this is exactly what i was looking for. Thank you. What is the reason you moved away from platform.io for Cubecells? I have a different Heltec Cubecell laying arround which I plan to use. Would probably be easy to do with pio :wink: But first I need all the HW parts. I’m struggling to find a way to get a cheap optical head in Switzerland. How and where did you get yours?

Thank you for this project and your work!
I have no LoRaWAN Network in my area (I do not get a connection) and would like to avoid setting up my own.

Has anyone already tried this with a plain LoRa peer to peer connection?

Line 90 in heltec-cubecell.ino looks to me like this should be possible:
//+LORAWAN=1 LoRaWAN 1, LoRa 0

Hello,

Unfortunately I have problems reading my meter.
It is an Itron 3.HZ-BC-D1-A1-12.

I have deactivated the PIN and activated the extended information. The data should be readable without sending an initialisation.

According to this table here (in German), the meter uses 9600 baud and the character set “1_8D_0P_N_1S”.

So if I understand correctly, it uses SML binary, 9600/8N1.

This is different to the Elster AS1440 as used in this project, which uses 300 baud and the character set “1_7D_1P_E_1S” with the protocol “D0, Modus C”.

I have therefore set/changed the configuration values as follows:

#define STRIP_UNIT
//#define SKIP_CHECKSUM_CHECK
#define MODE_OVERRIDE '0' 
#define PARITY_SETTING SERIAL_8N1 
#define START_SEQUENCE ""
#define METER_IDENTIFIER ""
#define OBIS_VALUE_POWER "16.7.0"
#define OBIS_VALUE_TOTAL_ENERGY "1.8.0"     
#define INITIAL_BAUD_RATE 9600

It does not read the values and it looks like the issue could be the starting sequence and/or the meter identification.

Despite having set “start_sequence” to an empty string, it seems to send something for the initialisation.

In this configuration it also throws an error, that ident is too short. I did not find an appropriate setting, but if I understand it correctly, in “meter.cpp” in line 94 it is compared to a length of 6 haracters.
I tried setting this to 0 instead of 6. The error goes away, but it still can’t read the data.

I also tried a lot of other combinations, without success.

Did someone already successfully use this tool with a meter with character set 1_8D_0P_N_1S?

Thank you very much in advance!