Power monitoring with an XTM18S and MQTT


#3

Thanks @gpbenton
Although I’m still thinking what will be the definitive use of this device, I have placed it inside a cupboard to measure the electric consumption of the TV. On one side I will satisfy my curiosity of how much electricity is consumed by my TV and, on the other side, I could check how much time and when it is switched on. So my current setup is as follows:

  • There is an extension cord going from the wall mains to a cupboard.
  • Inside the cupboard I have plenty of room to plug a phone charger to the extension cord powering the Wemos.
  • I have attached in and out cables to the XTM18S as shown in my first post. However, instead of having a hair-dryer, it is the TV that is plugged into the outlet of the XTM18S.

A picture of this setup is shown below.

I hope now it is more clear how I power the Wemos.
Anyway, there must be something wrong in my code since “Watts” do not go to zero when pulses do not increase, as you can detect in the figure below (TV was switched on only 5 minutes). So I have to double-check my code. Fun has just began!


#4

From what I can see, watt doesn’t change unless you get a pulse, so if no more pulses, it stays at the previous level.


#5

Yes, you are right. Thank you!
So I decided to change the philosophy in the pulses counting code.
As previously, the Wemos reports every SEND_FREQUENCY miliseconds (2 minutes in my case) messages to 3 different MQTT topics:

  • (1)- Energy (in kWh) consumed in the last SEND_FREQUENCY/1000 seconds
  • (2)- Power (in W) consumed in the last SEND_FREQUENCY/1000 seconds
  • (3)- Energy (in kWh) consumed since the device was switched on

There could be errors in (1) and (2) since pulse counting and information sending events do not take place at the same time. (3), if enough time has passed by, is as accurate as the one from the measuring device

UPDATED INO FILE

/**
 * This script is based on these sources:
 *
 * To count pulses using interruptions:  https://github.com/mysensors/MySensors/blob/master/examples/EnergyMeterPulseSensor/EnergyMeterPulseSensor.ino
 * To connect to Wifi and publish MQTT messages:  https://github.com/knolleary/pubsubclient/blob/master/examples/mqtt_esp8266/mqtt_esp8266.ino
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * DESCRIPTION
 * Use this sensor to measure kWh and Watt of your house meter.
 * You need to set the correct pulsefactor of your meter (pulses per kWh).
 * You have to update the information related to the MQTT server
 * Reports to different MQTT topics every SEND_FREQUENCY miliseconds:
 * (1)- Energy (in kWh) consumed in the last SEND_FREQUENCY/1000 seconds
 * (2)- Power (in W) consumed in the last SEND_FREQUENCY/1000 seconds
 * (3)- Energy (in kWh) consumed since the device was switched on
 * There could be errors in (1) and (2) since pulse counting and information sending events
 * do not take place at the same time. (3), if enough time has passed by, is as accurate as
 * the one from the measuring device
 */

#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.

/************************* WiFi Access Point *********************************/

const char* ssid = "your-essid";
const char* password = "your-essid-password";

/**************************** MQTT Broker ************************************/

const char* mqtt_server = "your-mqtt-server-ip";
const char* mqtt_username = "your-mqtt-user";
const char* mqtt_password = "your-mqtt-password";
const char* mqtt_topic_watt = "ESP-energy-01/watt";
const char* mqtt_topic_kwh = "ESP-energy-01/kwh";
const char* mqtt_topic_pulse = "ESP-energy-01/pulse";

#define DIGITAL_INPUT_SENSOR 12 // The digital input you attached S0+ D6 in Wemos D1 mini
#define PULSE_FACTOR 1000       // Nummber of pulses per kWh of your meeter

unsigned long SEND_FREQUENCY = 120000; // Minimum time between send (in milliseconds)
volatile unsigned long pulseCount = 0;
volatile unsigned long lastBlink = 0;
double kwh;
unsigned long lastSend;

WiFiClient espClient;
PubSubClient client(espClient);

long lastMsg = 0;
char msg[50];
char wattString[7];
char kwhString[7];
char kwhaccumString[7];

void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

// Setup a MQTT subscription
void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP8266Client", mqtt_username, mqtt_password)) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup()
{
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
  // Use the internal pullup to be able to hook up this sketch directly to an energy meter with S0 output
  // If no pullup is used, the reported usage will be too high because of the floating pin
  pinMode(DIGITAL_INPUT_SENSOR,INPUT_PULLUP);
  attachInterrupt(digitalPinToInterrupt(DIGITAL_INPUT_SENSOR), onPulse, RISING);
  // Initialization of variables
  kwh = 0;
  lastSend=millis();
}

void loop()
{
    if (!client.connected()) {
        reconnect();
    }
    client.loop();

    unsigned long now = millis();
    // Only send values at a maximum frequency
    bool sendTime = now - lastSend > SEND_FREQUENCY;
    if (sendTime) {
        // convert to a string with 2 digits before the comma and 2 digits for precision
        dtostrf(kwh, 2, 4, kwhString);
        client.publish(mqtt_topic_kwh,kwhString);  // Publish kwh to MQTT topic
        lastSend = now;  // once every thing is published we update the send time
        // We calculate the power using the energy
        double watt = kwh * 1000.0 * 3600.0 / (double)SEND_FREQUENCY * 1000.0;
        dtostrf(watt, 4, 2, wattString);
        client.publish(mqtt_topic_watt,wattString);  // Publish watt to MQTT topic
        Serial.print("Watt:");
        Serial.println(wattString);
        // We calculate the accumulated energy since the begining using pulses count
        double kwhaccum = ((double)pulseCount/((double)PULSE_FACTOR));
        dtostrf(kwhaccum, 3, 3, kwhaccumString);
        client.publish(mqtt_topic_pulse,kwhaccumString);  // Publish pulses to MQTT topic
        kwh = 0;
    }

}

void onPulse()
{
    unsigned long newBlink = micros();
    unsigned long interval = newBlink-lastBlink;
    if (interval<10000L) { // Sometimes we get interrupt on RISING
            return;
    }
    // Every time there is a pulse, the energy consumption is 1 [pulse] / PULSE_FACTOR [pulses/kWh]
    // We also want to accumulate the energy (it will be initialized again once MQTT message is sent)
    kwh += 1.0 / (double)PULSE_FACTOR;
    lastBlink = newBlink;
    pulseCount++;
}

I’ve been using this setup for several days and I think it’s working nicely. These are the results in the HA frontend:

  • With TV switched on:

  • With TV switched off:

PS: Now I see that I need to change the icons.


#6

Hi, exelent project!
Can you share your final sensor setup on home assistant? (electricity.yaml)
I do not understand how you get the value “Accum energy”


#7

Hi @cosmok , thank you! I’m so sorry for the long silence; my raspi also takes holidays in August :slight_smile:
Maybe you already solved your questions but just in case…
This is my updated /home/homeassistant/.homeassistant/includes/sensors/electricity.yaml file:

- platform: mqtt
  name: "Inst power 01"
  state_topic: "ESP-energy-01/watt" 
  unit_of_measurement: "W"
- platform: mqtt
  name: "Inst energy 01"
  state_topic: "ESP-energy-01/kwh" 
  unit_of_measurement: "kWh"
- platform: mqtt
  name: "Accum energy 01"
  state_topic: "ESP-energy-01/pulse" 
  unit_of_measurement: "kWh"

and my updated /home/homeassistant/.homeassistant/includes/groups/electricity.yaml file:

electricity:
  name: 'Electricity'
  view: yes
  entities:
    - group.xtm18s_01

xtm18s_01:
  name: 'TV consumption'
  entities:
    - sensor.inst_power_01
    - sensor.inst_energy_01
    - sensor.accum_energy_01

Well, the accumulated energy (i.e. the total energy consumed by the electric device) can be easily calculated by knowing the pulses counted by the XTM18S. In the INO file it is done here (around line 150):

        // We calculate the accumulated energy since the begining using pulses count
        double kwhaccum = ((double)pulseCount/((double)PULSE_FACTOR));

Sorry again for the delayed response.


#8

Thanks Timseebeck.
I have mounted the system following your instructions but with an LCD counter XTM18SA and a SONOFF BASIC (it has power included and the port GPIO14 accessible)
Everything seems to work correctly


#9

Hi @timseebeck,
I’ve made a couple of changes to your code to make it connect with fixed IP to the router, send IP and MAC address via MQTT, and make the internal NodeMCU LED blink.

After a few hours the kWh counter returns to 0 even without NodeMCU being reset or powered off… What could be the problem?

You can see the code here: https://gist.github.com/jorgeassuncao/40956bbe4ac8db9fdb8f0526a4e0c0fe

Another question, how to modify the code to make this: Calculate energy price based on time of day and daylight savings time

Thank you very much for any help!


#10

Hi @j.assuncao ,

I’m not really an expert on this area so I have no clue about the behavior you mentioned with your NodeMCU. I’ve checked the differences between your code and mine and I don’t see why yours may lead to failure. Maybe you could try the following:
a) Increase the time between events sending: I have SEND_FREQUENCY = 120000 while yours is 15000, which maybe puts more stress to your MCU
b) Remove the part related to the LED blinking
c) Try on another MCU

While I’m not using this device right now, I have to say that I used it for 5 consecutive days without issues.

As for the other question, I’m going to answer in that thread.
Good luck!


#11

Hi @timseebeck,

I’ll be trying what you suggested this week. I suspect that the problem is the blinking LED. It stopped working right after i added the changes as you can see in this image.

What did you use to flash the NodeMCU, Arduino IDE or Platform .io? If you used Arduino what settings did you use for “flash mode” and “flash size”?

Thank you for your time.


#12

I used the Arduino IDE. My board selection was a “WeMos D1 R2 & mini” with CPU Frequency: “80 MHz”, Flash Size: “4M (3M SPIFFS)” and Upload Speed: “115200”. I don’t know anything about flash mode. Sorry.
You’re welcome. Good luck!


#13

Hi @timseebeck,

I removed the LED blinking part of the code and now is working fine, on the last 24h there was no reset. I’ll keep an eye on the next few days just to be sure. Thanks for the tip!

One more thing, is there a way to keep the accumulated kWh value between reboots/power-off of the device?


#14

I see two approaches here:
1.- Storing the value in the EEPROM of the microcontroller. I’m not 100% sure of what I’m going to say but it seems that the NodeMCU does not have an EEPROM. However, you could still use its flash memory. Be warned that too much writings/readings may wear the flash memory.
2.- Maybe the kWh value could be stored in HA and transferred to the NodeMCU somehow (just like it is done with the states of the relays).

In any case, I don’t have any clue about how any of the mentioned approaches could be implemented. And maybe there are other solutions. Let’s wait for someone smarter…

Offtopic: can you please tell me what you are using to make the plots? Is it Graphana? Thanks.


#15

Hi @timseebeck,

Tried EEPROM and SPIFFS methods today with no luck but i’m a total noob programming in Arduino… If anyone out there would like to help, just say something.

Using HA to store a variable is something that i don’t know how to do either. I’ll Google it and try on the next few days. Can we get someone smarter to help us!? Please!?

Offtopic: can you please tell me what you are using to make the plots? Is it Graphana? Thanks.

Yes, it’s InfluxDb server + Grafana addon on Hass.io.

Let’s wait for someone smarter. I really like you project and i’m planning to add a non-invasive current sensor to it.


#16

Hi again @timseebeck,

I think i’ve found the solution to keep the accumulated kWh value between reboots/power-off of the device!

The solution was in front of me all the time an i didn’t see it. Retaining the topic of kwhaccumString and then read it from broker after reboot/reset.


#17

That’s great!

So there was a third approach and it involves retaining the topic. Thanks for telling! Always learning from this community.


#18

Yes, and also solved the problem with the blinking LED. I’ll publish the code on Github soon and it will have a reference to you because it was based on you work.

You’re welcome! Sometimes we learn, sometimes we teach…

Thanks for sharing your work to!


#19

Hi Jorge.

I’m very interested in your code. When are you going to publish it on Github? :hugs:


#20

Hi @Pharizna. Sorry for the late reply…

You can get the code here. If you need any help or explanation just say!


#22

Thank you a lot!
I’m not an expert on this subject (as you will see in the future) and probably I’ll need a lot of your help

:wink:


#23

Don’t worry about that, that’s watch this forum is about!