Getting Meter Readings from UK Smart Meter

Hi

I fancied a challenge of trying to get the data from my smart meter into HA which I have successfully achieved in a bit of a hack

I have one of those in home displays provided to me by my supplier SSE. It just so happened it has a USB socket on it. Unfortunately it isn’t USB compliant but it turned out that the port was producing serial data rs232 when I probed it.

I then spent a day capturing serial packets and working out where the readings were in the packet for day use, instant use and the meter reading itself. There’s other data in the packet but I haven’t worked that out yet, hoping power factor and supply voltage are there.

Having done this I wrote some really crap code that surprisingly works and captures the data in the packets and sends it over mqtt as a value in watts.

I then incorporated this in the energy dashboard and now I can be confident that me and the DNO are seeing the same data. My previous current clamp meter was giving me different results. In fact it was reading higher usage than the Smart Meter.


3 Likes

Just got one of those this week. Would love to see how you did it in detail to replicate!

Sure

I can’t say of it would work on any other IHD display but if its the same as mine in the picture I’ll post the code here for you if you want

Thank you

1 Like

@tekelectron I’ve recently had smart meters installed and was given a Smartview2 monitor which looks like the one you’ve got in the photo. I’d be interested to know what hardware you used to get the serial data from the USB port and your code.

Cheers.

I’ve picked up a serial to USB adaptor and hooked up the smart meter. I’m getting a serial output, but it’s all grabbled. Not sure what the settings (baud rate, etc.) are to pick up the data. Any direction would be much appreciated.

I’ve got a Scottish Power one - never thought of plugging into the back of it! Would love to see some wiring diagrams and connection settings.

Very interested in this. I have the same one. Not sure what SSE would think of this though :slight_smile:
So how are using this on day to day basis ? Where is the socket, i don’t remember it having one. Is it on the board itself ?

Would love to see pics and the code.

Hi

I use it to get the meter readings and then I use the energy integration in HA. it is 100% accurate so everything in terms of readings and cost is same on the display and HA.

I use the “Current Use” function live on HA dashboard so I can monitor real time usage, this is the same as what is on the display.

Its a USB socket on the back, that outputs this data in serial but its very disorganised and difficult to capture. I then send these values as MQTT message payloads.

What pictures would u like?

Code is attached. Code is ugly, unformatted and messy but does work. Its hard to capture the data because the beginning of the hex strings spewing out of the IHD often change and jump around and so do their start positions. Someone who can code better than me could make it work im sure.

The USB cable has the serial lines on it (rs232) in a serial plug form.

You’d have to change ur MQTT server IP and WiFi etc



//byte byteOne = 0xFF;
//byte byteTwo = 0xFF;
//byte byteThree = 0xFF;
//byte byteFour = 0xFF;



#include <ESP8266WiFi.h>
#include <PubSubClient.h>

const char* ssid = "xxxxxx";
const char* password = "xxxxxxx";
const char* mqtt_server = "192.168.0.31";

WiFiClient espClient;
PubSubClient client(espClient);

long now = millis();
long lastMeasure = 0;
long readingFilter = 999999999;
long lastdayUsage = 0;
long lastmeterReadingone = 0;
byte byteCheck[150];
byte mreadingData[8];
byte dailyData [4];
byte currentData[4];

char msg1[50];
char msg2[50];
char msg3[50];


// Don't change the function below. This functions connects your ESP8266 to your router
void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("WiFi connected - ESP IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected;
  while (!client.connected()) {
    Serial.print("MQTT connect..");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str())) {
     
      Serial.print("connected");
       Serial.print("connected");
       Serial.print("connected");
        Serial.print("connected");
      // Once connected, publish an announcement...
     // client.publish("energy/meterread", "hello world");
      // ... and resubscribe
      //client.subscribe("inTopic");
    } else {
      
      Serial.print("failed. Retry.");
      //Serial.print(client.state());
      //Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}







void setup() {
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial.println("This is a test");
}

void loop() {
  // put your main code here, to run repeatedly:
     if (!client.connected()) {
    reconnect();
  }
  client.loop();
  
  if (Serial.available() > 0) {
    for (int i = 0; i < 150; i++) {      //Read
     
      byteCheck[i] = Serial.read();

      delay(10);
      Serial.println(byteCheck[i], HEX);
      Serial.println(i);
 //     Serial.println("Beginning");

    }


    
    // Meter Reading
    if (byteCheck[90] == 0x00 && byteCheck[91] == 0x00 && byteCheck[92] == 0x25) {

      Serial.print("Got Meter Reading Header");
//      Serial.println(byteCheck[29], HEX);
//      Serial.println(byteCheck[30], HEX);
//      Serial.println(byteCheck[31], HEX);
//      Serial.println(byteCheck[32], HEX);
//      Serial.println(byteCheck[33], HEX);
      long meterReadingone = (byteCheck[96] << 24) + (byteCheck[95] << 16) + (byteCheck[94] << 8) + (byteCheck[93]);
      Serial.println(meterReadingone, HEX);
      Serial.println(meterReadingone, DEC);
      Serial.flush();
      if((meterReadingone <= readingFilter) && (meterReadingone > lastmeterReadingone)){
      sprintf(msg1,"%u",(meterReadingone));
      
      client.publish("energy/meterread", msg1);
      lastmeterReadingone = meterReadingone;
    }
    }
//
       if (byteCheck[89] == 0x00 && byteCheck[90] == 0x00 && byteCheck[91] == 0x25) {

      Serial.print("Got Meter Reading Header");
//      Serial.println(byteCheck[29], HEX);
//      Serial.println(byteCheck[30], HEX);
//      Serial.println(byteCheck[31], HEX);
//      Serial.println(byteCheck[32], HEX);
//      Serial.println(byteCheck[33], HEX);
      long meterReadingone = (byteCheck[95] << 24) + (byteCheck[94] << 16) + (byteCheck[93] << 8) + (byteCheck[92]);
      Serial.println(meterReadingone, HEX);
      Serial.println(meterReadingone, DEC);
      Serial.flush();
if((meterReadingone <= readingFilter) && (meterReadingone > lastmeterReadingone)){
      sprintf(msg1,"%u",(meterReadingone));
      
      client.publish("energy/meterread", msg1);
      lastmeterReadingone = meterReadingone;
    }
       }
    //    // Daily Use
    if (byteCheck[100] == 0x00 && byteCheck[101] == 0x22){
      Serial.println("Got Daily Usage");
//      Serial.println(byteCheck[6], HEX);
//      Serial.println(byteCheck[7], HEX);
      long dayUsage = (byteCheck[103] << 8) + (byteCheck[102]);
      Serial.println(byteCheck[103], HEX);
      Serial.println(byteCheck[102], HEX);
      Serial.println(dayUsage, HEX);
      Serial.println(dayUsage, DEC);
      Serial.flush();
      if((dayUsage >= lastdayUsage) || (dayUsage <= 100)){
       sprintf(msg2,"%u",(dayUsage));
      client.publish("energy/dayuse", msg2);
      lastdayUsage = dayUsage;
    }
    }
       if (byteCheck[101] == 0x00 && byteCheck[102] == 0x22){
      Serial.println("Got Daily Usage");
//      Serial.println(byteCheck[6], HEX);
//      Serial.println(byteCheck[7], HEX);
      long dayUsage = (byteCheck[104] << 8) + (byteCheck[103]);
      Serial.println(byteCheck[103], HEX);
      Serial.println(byteCheck[102], HEX);
      Serial.println(dayUsage, HEX);
      Serial.println(dayUsage, DEC);
      Serial.flush();
      if((dayUsage >= lastdayUsage) || (dayUsage <= 100)){
       sprintf(msg2,"%u",(dayUsage));
      client.publish("energy/dayuse", msg2);
      lastdayUsage = dayUsage;
    }
//             sprintf(msg2,"%u",(dayUsage));
//      client.publish("energy/dayuse", msg2);
    }
    // Current Use
    if (byteCheck[27] == 0x00 && byteCheck[28] == 0x2a) {
      //      for(int j = 24; j < 27; j++){
      //        currentData[j] = Serial.read();
      //      }
      Serial.println("Got Current Usage");
      //      Serial.println(byteCheck[24], HEX);
      //      Serial.println(byteCheck[25], HEX);
      //      Serial.println(byteCheck[26], HEX);
      //      Serial.println(byteCheck[27], HEX);
      //      Serial.println(byteCheck[28], HEX);
      //      Serial.println(byteCheck[27]+byteCheck[28], HEX);
      //      Serial.println(byteCheck[27]+byteCheck[28], DEC);
      long currentUsedec = (byteCheck[30] << 8) + byteCheck[29];
//      Serial.println(byteCheck[30]);
//      Serial.println(byteCheck[29]);
      Serial.println(currentUsedec);
      Serial.flush();
      if(currentUsedec > 0){
            sprintf(msg3,"%u",(currentUsedec));
            
      client.publish("energy/current", msg3);
    }
    }
  }
//Serial.flush();
  
}

Here are some pictures



Many thanks. Would you be so kind as to show me a picture of the back, and what cables go where and what ESP device you have hooked up to it ?

Where and how would i compile the code ? Was hoping to get some nice ESPhome yaml template :wink:

Thanks,

Hi

Its an esp8266 coded in Arduino IDE with code attached no frills just the module mounted on a PCB

No nothing fancy like templates its a bit more hands on than that im afraid. You have to add the mqtt published topics to ur yaml file manually to get the values into the dashboard for example.

At the back of the display is the power cable, the power cable is in fact a USB cable which can be removed. Standard USB colours are red black green and white. Use a spare cable, insert end into HID and cut the plug off the other end. White is TX from the HID so connect this to the RX pin on your ESP along with the ground wire for reference.

Its quite hands on I wouldnt recommend tackling it unless you have some experience with arduino etc

1 Like

Hi Tekelectron,

Thanks for the code. I’ve managed to get it running on an ESP32 board. Connects to MQTT broker and I can see the test message. In the Arduino software, I can also see the serial output, but it doesn’t pick up the meter reading headers.

How did you workout what the meter reading headers are? I’m relatively new to using serial communication, any reference would be helpful. I’m wondering if the software version on mine is different and therefore the headers are different… Also, I’ve got a smart gas meter, so keen to get that data into HA as well.

One further question. Once I get the code sorted, to power the smart meter device, I’m presuming that the red and black will need to be connected for power separately? Is it ok to use the green data cable for reference ground on the GND pin on the ESP? I’ve tried that and it seem to work.

Thanks

Hi all,

I’ve managed to adapt the code to work on an ESP32.

#include <WiFi.h>
#include <PubSubClient.h>

// Replace with your network credentials (STATION)
const char* ssid = "wifi network name";
const char* password = "wifi network password";
const char* mqtt_server = "IP address of MQTT server";
const char* mqtt_user = "mqtt user name";
const char* mqtt_password = "mqtt password";

WiFiClient espClient;
PubSubClient client(espClient);

long now = millis();
long lastMeasure = 0;
long readingFilter = 999999999;
long lastdayUsage = 0;
long lastmeterReadingone = 0;
byte byteCheck[150];
byte mreadingData[8];
byte dailyData [4];
byte currentData[4];

char msg1[50];
char msg2[50];
char msg3[50];

void initWiFi() {
  delay (100);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println();
  Serial.print("Connecting to WiFi...");
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print('.');
    delay(1000);
  }
  Serial.println();
  Serial.print("WiFi connected to: ");
  Serial.print(ssid);
  Serial.println();
  Serial.print("ESP IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected;
  while (!client.connected()) {
    Serial.print("MQTT connecting...");
    // Create a random client ID
    String clientId = "ESP32Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str(), "mqtt", "test")) {
     
      Serial.print("connected");
      Serial.println();
      // Once connected, publish an announcement...
      client.publish("energy/meterread", "hello world");
      // ... and resubscribe
      //client.subscribe("inTopic");
    } else {
      
      Serial.print("failed. Retry.");
      //Serial.print(client.state());
      //Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}


void setup() {
  Serial.begin(115200);
  initWiFi();
  Serial.print("RSSI: ");
  Serial.println(WiFi.RSSI());
  Serial.println("Connecting to MQTT server");
  client.setServer(mqtt_server, 1883);
  Serial.println("Connected to MQTT server");
  Serial.println();
}

void loop() {
  // put your main code here, to run repeatedly:
  // put your main code here, to run repeatedly:
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
  
  if (Serial.available() > 0) {
    for (int i = 0; i < 150; i++) {      //Read
     
      byteCheck[i] = Serial.read();

      delay(10);
      Serial.println(byteCheck[i], HEX);
      Serial.println(i);
 //     Serial.println("Beginning");

    }


    
    // Meter Reading
    if (byteCheck[90] == 0x00 && byteCheck[91] == 0x00 && byteCheck[92] == 0x25) {

      Serial.print("Got Meter Reading Header");
//      Serial.println(byteCheck[29], HEX);
//      Serial.println(byteCheck[30], HEX);
//      Serial.println(byteCheck[31], HEX);
//      Serial.println(byteCheck[32], HEX);
//      Serial.println(byteCheck[33], HEX);
      long meterReadingone = (byteCheck[96] << 24) + (byteCheck[95] << 16) + (byteCheck[94] << 8) + (byteCheck[93]);
      Serial.println(meterReadingone, HEX);
      Serial.println(meterReadingone, DEC);
      Serial.flush();
      if((meterReadingone <= readingFilter) && (meterReadingone > lastmeterReadingone)){
      sprintf(msg1,"%u",(meterReadingone));
      
      client.publish("energy/meterread", msg1);
      lastmeterReadingone = meterReadingone;
      }
    }
//
    if (byteCheck[89] == 0x00 && byteCheck[90] == 0x00 && byteCheck[91] == 0x25) {

      Serial.print("Got Meter Reading Header");
//      Serial.println(byteCheck[29], HEX);
//      Serial.println(byteCheck[30], HEX);
//      Serial.println(byteCheck[31], HEX);
//      Serial.println(byteCheck[32], HEX);
//      Serial.println(byteCheck[33], HEX);
      long meterReadingone = (byteCheck[95] << 24) + (byteCheck[94] << 16) + (byteCheck[93] << 8) + (byteCheck[92]);
      Serial.println(meterReadingone, HEX);
      Serial.println(meterReadingone, DEC);
      Serial.flush();
        if((meterReadingone <= readingFilter) && (meterReadingone > lastmeterReadingone)){
          sprintf(msg1,"%u",(meterReadingone));
          client.publish("energy/meterread", msg1);
          lastmeterReadingone = meterReadingone;
        }
    }
    //    // Daily Use
    if (byteCheck[100] == 0x00 && byteCheck[101] == 0x22){
      Serial.println("Got Daily Usage");
//      Serial.println(byteCheck[6], HEX);
//      Serial.println(byteCheck[7], HEX);
      long dayUsage = (byteCheck[103] << 8) + (byteCheck[102]);
      Serial.println(byteCheck[103], HEX);
      Serial.println(byteCheck[102], HEX);
      Serial.println(dayUsage, HEX);
      Serial.println(dayUsage, DEC);
      Serial.flush();
      if((dayUsage >= lastdayUsage) || (dayUsage <= 100)){
        sprintf(msg2,"%u",(dayUsage));
        client.publish("energy/dayuse", msg2);
        lastdayUsage = dayUsage;
      }
    }
    if (byteCheck[101] == 0x00 && byteCheck[102] == 0x22){
      Serial.println("Got Daily Usage");
//      Serial.println(byteCheck[6], HEX);
//      Serial.println(byteCheck[7], HEX);
      long dayUsage = (byteCheck[104] << 8) + (byteCheck[103]);
      Serial.println(byteCheck[103], HEX);
      Serial.println(byteCheck[102], HEX);
      Serial.println(dayUsage, HEX);
      Serial.println(dayUsage, DEC);
      Serial.flush();
      if((dayUsage >= lastdayUsage) || (dayUsage <= 100)){
        sprintf(msg2,"%u",(dayUsage));
        client.publish("energy/dayuse", msg2);
        lastdayUsage = dayUsage;
      }
//             sprintf(msg2,"%u",(dayUsage));
//      client.publish("energy/dayuse", msg2);
    }
    // Current Use
    if (byteCheck[27] == 0x00 && byteCheck[28] == 0x2a) {
      //      for(int j = 24; j < 27; j++){
      //        currentData[j] = Serial.read();
      //      }
      Serial.println("Got Current Usage");
      //      Serial.println(byteCheck[24], HEX);
      //      Serial.println(byteCheck[25], HEX);
      //      Serial.println(byteCheck[26], HEX);
      //      Serial.println(byteCheck[27], HEX);
      //      Serial.println(byteCheck[28], HEX);
      //      Serial.println(byteCheck[27]+byteCheck[28], HEX);
      //      Serial.println(byteCheck[27]+byteCheck[28], DEC);
      long currentUsedec = (byteCheck[30] << 8) + byteCheck[29];
//      Serial.println(byteCheck[30]);
//      Serial.println(byteCheck[29]);
      Serial.println(currentUsedec);
      Serial.flush();
      if(currentUsedec > 0){
        sprintf(msg3,"%u",(currentUsedec));    
        client.publish("energy/current", msg3);
      }
    }
  }
//Serial.flush();
  
}

I’ve managed to get some data from the smart meter; meter reading and energy usage for the week (which is picked up instead of day use). Current energy use is not being picked up. The readings are pretty sporadic and sometimes aren’t picked up for a few hours.

Still trying to workout how to workout how to detect current energy use and gas energy use…

I took a different approach with regards to SSE in the UK. The reader they provided works in tandem with another app GeoTogether. I used a modified python script to poll the account in the cloud and return my watts usage at that time. It runs every minute. So far works pretty good. I also use the same method for the gas usage. I can pick up kettles being boiled or Irons in use along with the steady run rate. If anybody is interested i’ll put the code on a GIT.

image