Energy dashboard not populating

[sender]

Indeed, but how?

[petro]

well, start by posting your configuration for that esp device

[sender]

I have now added some code I found on github. this “skips” the first 20 readings after ESP boot. But I doubt this will work because the ESP is already running when the inverter is starting up in the morning.

Sensor Filter: Ignore the first values after ESP boot

the current code:

esphome:

  name: growatt_modbus

  platform: ESP8266

  board: d1_mini

  on_boot:

    # set a default value for the skip counter, otherwise it's nan

    - sensor.template.publish:

        id: ${my_sensor}_skips

        state: 0

substitutions:

  esp_name: Growattmodbus

  my_sensor: my_sensor

wifi:

  ssid: "SSID"

  password: !secret wifi_key

  use_address: 192.168.6.42

  domain: !secret domain

  # Enable fallback hotspot (captive portal) in case wifi connection fails

  ap:

    ssid: "${esp_name} fb Hotspot"

    password: !secret Fallback_Hotspot

captive_portal:

# Enable logging

logger:

#  level: VERBOSE

#  baud_rate: 0

# Enable Home Assistant API

api:

ota:

time:

  - platform: homeassistant

    id: homeassistant_time

    on_time:

      # Every morning on weekdays reset ID todaygen

      - seconds: 0

        minutes: 0

        hours: 0

        #days_of_week: MON-SUN

        then:

          - lambda: id(todaygen).publish_state(0);

uart:

  - id: uart1

    baud_rate: 9600

    tx_pin: D1

    rx_pin: D2

#  stop_bits: 1

modbus:

  uart_id: uart1

 

web_server:

  port: 80

sensor:

  - platform: wifi_signal

    name: "${esp_name} - ESP WiFi Signal"

    update_interval: 60s

   

  - platform: uptime

    name: "${esp_name} - ESP Uptime"

    icon: mdi:clock-outline

    update_interval: 60s

   

  - platform: growatt_solar

    update_interval: 3s

    protocol_version: RTU2

    inverter_status:

      name: "${esp_name} - Status Code"

      id: inverter_status

    phase_a:

      voltage:

          name: "${esp_name} - AC Voltage"

      current:

          name: "${esp_name} - AC Current"

      active_power:

          name: "${esp_name} - AC Power"

          filters:

            - filter_out: NaN

    pv1:

      voltage:

          name: "${esp_name} - PV1 Voltage"

      current:

          name: "${esp_name} - PV1 Current"

      active_power:

          name: "${esp_name} - PV1 Power"

    pv2:

      voltage:

          name: "${esp_name} - PV2 Voltage"

      current:

          name: "${esp_name} - PV2 Current"

      active_power:

          name: "${esp_name} - PV2 Power"

    active_power:

      state_class: "measurement"

      name: "${esp_name} - Output Power"

      filters:

       - lambda: |-

           if (x <= 5500 && x >= 0) return x;

           else return 0;

       

       #- lambda: |-

       #    if (x <= 5500) return x;

       #    else return {};

    pv_active_power:

      name: "${esp_name} - Input Power"

    frequency:

      name: "${esp_name} - Grid Frequency"

    energy_production_day:

      name: "${esp_name} - Today Gen"

      id: todaygen

      filters:

      # increment a skip counter, this skips the first two measurements

      - lambda: auto s = id(${my_sensor}_skips); if(s->state > 20) { return x; } else { s->publish_state(s->state+1); return {}; }

      #filters:

      # - lambda: |-

      #     if (x >= 0) return x;

      #     else return 0;

         #lambda: |-

         #   if (id(inverter_status).state != 0) {

         #       return x;

         #   }

         #   return {};

      #  - lambda: "if (x - id(todaygen).state) > 0.5) return {}; return x;"

     

      # https://github.com/esphome/feature-requests/issues/607

      #- lambda: auto s = id(${todaygen}_skips); if(s->state > 100) { return x; } else { s->publish_state(s->state+1); return {}; }

    total_energy_production:

      name: "${esp_name} - Total Gen"

      #state_class: "measurement"

      #device_class: energy

      accuracy_decimals: 1

    inverter_module_temp:

      name: "${esp_name} - Temperature"

  # create a skip counter

  - platform: template

    id: ${my_sensor}_skips

    internal: true

    accuracy_decimals: 0

This is the original “problematic” code:

esphome:
  name: growatt_modbus
  platform: ESP8266
  board: d1_mini

substitutions:
  esp_name: Growattmodbus

wifi:
  ssid: "SSID"
  password: !secret wifi_key
  use_address: 192.168.6.42
  domain: !secret domain
  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "${esp_name} fb Hotspot"
    password: !secret Fallback_Hotspot

captive_portal:

# Enable logging
logger:
#  level: VERBOSE
#  baud_rate: 0

# Enable Home Assistant API
api:

ota:

time:
  - platform: homeassistant
    id: homeassistant_time
    on_time:
      # Every morning on weekdays reset ID todaygen
      - seconds: 0
        minutes: 0
        hours: 0
        #days_of_week: MON-SUN
        then:
          - lambda: id(todaygen).publish_state(0);

uart:
  - id: uart1
    baud_rate: 9600
    tx_pin: D1
    rx_pin: D2
#  stop_bits: 1

modbus:
  uart_id: uart1
  
web_server:
  port: 80

sensor:
  - platform: wifi_signal
    name: "${esp_name} - ESP WiFi Signal"
    update_interval: 60s
    
  - platform: uptime
    name: "${esp_name} - ESP Uptime"
    icon: mdi:clock-outline
    update_interval: 60s
    
  - platform: growatt_solar
    update_interval: 3s
    protocol_version: RTU2

    inverter_status:
      name: "${esp_name} - Status Code"
      id: inverter_status

    phase_a:
      voltage:
          name: "${esp_name} - AC Voltage"
      current:
          name: "${esp_name} - AC Current"
      active_power:
          name: "${esp_name} - AC Power"
          filters:
            - filter_out: NaN
    pv1:
      voltage:
          name: "${esp_name} - PV1 Voltage"
      current:
          name: "${esp_name} - PV1 Current"
      active_power:
          name: "${esp_name} - PV1 Power"

    pv2:
      voltage:
          name: "${esp_name} - PV2 Voltage"
      current:
          name: "${esp_name} - PV2 Current"
      active_power:
          name: "${esp_name} - PV2 Power"

    active_power:
      state_class: "measurement"
      name: "${esp_name} - Output Power"
      filters:
       - lambda: |-
           if (x <= 5500 && x >= 0) return x;
           else return 0;
       
       #- lambda: |-
       #    if (x <= 5500) return x;
       #    else return {};

    pv_active_power:
      name: "${esp_name} - Input Power"

    frequency:
      name: "${esp_name} - Grid Frequency"

    energy_production_day:
      name: "${esp_name} - Today Gen"
      id: todaygen
      filters:
       - lambda: |-
           if (x >= 0) return x;
           else return 0;
         #lambda: |-
         #   if (id(inverter_status).state != 0) {
         #       return x;
         #   }
         #   return {};

      #  - lambda: "if (x - id(todaygen).state) > 0.5) return {}; return x;"
      
      # https://github.com/esphome/feature-requests/issues/607
      #- lambda: auto s = id(${todaygen}_skips); if(s->state > 100) { return x; } else { s->publish_state(s->state+1); return {}; }

      #state_class: total
      #state_class: total_increasing # was empty
      # https://developers.home-assistant.io/docs/core/entity/sensor/#how-to-choose-state_class-and-last_reset

    total_energy_production:
      name: "${esp_name} - Total Gen"
      #state_class: "measurement"
      #device_class: energy
      accuracy_decimals: 1

    inverter_module_temp:
      name: "${esp_name} - Temperature"

[petro]

which sensor is outputting 0 or NaN on startup that’s causing the issues?

[sender]

@sender

energy_production_day:

But its outputing:

@sender

This happens when the solar converter starts up every morning:

[petro]

You’re most likely running into a timing issue. The last calculation of the day is slightly past midnight by the time it reaches HA, so HA adds it to the next days values. Your only option is to control the timing of your data points, by forcing the values to specific intervals.

Or have HA calculate the energy from your power entities.

[sender]

Ok, but how can I solve that? Get the first bugus readings out every day?

[petro]

You can try adjusting your update interval to an odd value so that it doesn’t occur on the minute when crossing midnight.

Keep in mind, that you think your problem is getting 0 at midnight. Your actual problem is that you’re getting the previous day’s value at midnight. So sending an extra 0 or whatever at midnight isn’t going to solve the problem. And it doesn’t look like lambdas can get the time? Unless I’m wrong. If you can get the current time in your lambda, you can output zero if it’s >= midnight.

EDIT: You can get the time in lambdas. So all you need to do is filter out values between 23:59 and 00:01 and you’ll be good.

Time Component

[sender]

The problem is not at midnight, but is at the starting up of the inverter. So every morning on sunrise.
I need the value near-real-time due to some automations, so update interval of a minute is not feasible.

I need something to filter out the values every morning before the actual counting starts…

[petro]

Then you need to get inventive and figure out what’s happening. If the ESP device is turning off, then don’t have the device turn off.

Your other option is to have HA do the calculations and you can use an automation to ignore data around sunrise.

Either way, the problem is in your device and you have to work around it. To me, it doesn’t make sense that your device would be outputting yesterdays values on startup at sunrise. This would be something should occur at midnight every day.

[sender]

The ESP device is always on. It’s the inverter going on and off (on sunrise) and sending the day gen of the day before a few times before it completely starts.

At midnight the inverter is really off (dead).

[sender]

The best solution would be to “filter” out the first readings every start of the day.

But how do I do that?

[petro]

it’s not at the start of every day. It’s at sunrise when it turns on. You need to understand that those have different meanings.

I recommend using HA’s energy calculations instead of the devices as the devices is not correct.

The last resort is that you can wrap a template sensor around your data and filter out the results that way if you are within 10 minutes of sunrise when the device is going to turn on.

Either way, you can’t do it on the esp side as I don’t think you can get sunrise/sunset times. Then again, I’m not 100% versed in ESPHome syntax.

[sender]

@petro

I recommend using HA’s energy calculations instead of the devices as the devices is not correct.

Semantics but you’re right. It’s at the start of every dawn.

I prefer clean input in my sensors and prefer to use the value from the esp. Everything works fine with it (all my sensors are calculated correctly) but the energy dashboard.

I would really be happy with a filter that filters out every bogus reading every dawn…

[petro]

well, a simple search on the forums found that yes, you can get sunrise in lambds

ESPHOME and Sun Components

My solution. substitutions: #Shift sunset and sunrisen on minute shift_sunset: '-22' shift_sunrise: '17' time: - platform: sntp id: sntp_time timezone: Europe/Prague sun: latitude: !secret latitude_data longitude: !secret longitude_data id: sun_sun switch: - platform: gpio pin: D4 name: "Relay" id: relay inverted: yes text_sensor: - platform: template name: "Sunset" id: sunset_shift lambda: |- auto sunset = id(sun_sun).sunset()…

so you should be able to filter out startup values around sunrise.

[sender]

Sun

would this component be an option to include in a lambda and if the sun if rised only send the readings?

(I don’t know how to lambda-code that)…

[sender]

@petro

you

Sorry, I don’t know how to do
that in lambda

[petro]

Sorry, I’ve never used lambdas, I don’t know the syntax and I’d have to learn it. It seems straight forward and looks like JS with a twist. If you know JS you should be able to pick it up quickly. Anyways, this seems like an adventure you’ll have to take. Might be an opportunity to create a new posts explicitly asking to filter out data at a specific time in the ESPHome section.

[sender]

For the archive:

Lambda Filter out bogus values during dawn

Hi all, for a long time I am looking for a solution to filter out some bogus data every morning at sunrise when my inverter is starting up and sending bogus total gen of the day before. This messes up my energy dashboard with peaks. I have tried multiple things but do not have a solution for it yet. In another topic I was advised to open a new topic in esphome to ask for it. My question is if I can get help creating a working lambda filter that filters out the bogus data every morning on sunr…

[tom_l]

@petro

looks like JS with a twist

It’s basically C++

Which is foreign to me. I can usually muddle through simple things by adapting examples but this is beyond me.

Sender, how to filter 10 minutes either side of sunrise with a lambda would best be asked on the ESPHome discord. The ESPHome devs hang out there and are usually very helpful.