Creating a sensor from a json file where the data comes in late

Configuration
1

Hello!

I’ve written a simple Puppeteer-driven application that fetches my energy consumption from my power company once every 24 hours. In other words, I will never have any data available on what my current power consumption is, I will only have it after the fact. The power company doesn’t actually show me the data until the day after, so I couldn’t query this faster even if I wanted to.

I want to use this to diff the consumptions of all of my power consumers that I’m measuring with the total consumption, to figure out how much of my total consumption that is coming from sources that I’m not measuring, and also to display historical power consumption for recent dates on a dashboard.

It seems to me like what I want to do is fundamentally misaligned with what a “sensor” in HA does, and that this won’t work since my sensor never has a “current value”, but only historical data.

I’d like to know if I’m correct that this won’t work, and also hear some suggestions on how I can accomplish my goals within HA.

Thanks!

Example data (abbreviated to fit in the post) from my output (but I’m in control of the code for this, so I can reformat it however I want):

[
  {
    "startDate": "2024-01-12T00:00:00",
    "endDate": "2024-01-18T11:17:46.0919075+00:00",
    "delta": null,
    "unit": "kWh",
    "aggregationInterval": "Hourly",
    "consumption": [
      {
        "date": "2024-01-12T00:00:00",
        "week": null,
        "consumption": 1.363,
        "status": null
      },
      {
        "date": "2024-01-12T01:00:00",
        "week": null,
        "consumption": 0.653,
        "status": null
      },
      {
        "date": "2024-01-12T02:00:00",
        "week": null,
        "consumption": 0.956,
        "status": null
      },
      {
        "date": "2024-01-15T23:00:00",
        "week": null,
        "consumption": 1.11,
        "status": null
      }
    ],
    "highLoadConsumption": null,
    "lowLoadConsumption": null,
    "highLoad": null,
    "monthlyLoad": null
  },
  {
    "startDate": "2024-01-16T00:00:00",
    "endDate": "2024-01-26T10:48:01.6381498+00:00",
    "delta": null,
    "unit": "kWh",
    "aggregationInterval": "Hourly",
    "consumption": [
      {
        "date": "2024-01-16T00:00:00",
        "week": null,
        "consumption": 1.009,
        "status": null
      },
      {
        "date": "2024-01-16T01:00:00",
        "week": null,
        "consumption": 0.918,
        "status": null
      },
      {
        "date": "2024-01-16T02:00:00",
        "week": null,
        "consumption": 0.99,
        "status": null
      },
      {
        "date": "2024-01-16T03:00:00",
        "week": null,
        "consumption": 1.03,
        "status": null
      },
      {
        "date": "2024-01-16T04:00:00",
        "week": null,
        "consumption": 1.07,
        "status": null
      },
      {
        "date": "2024-01-16T05:00:00",
        "week": null,
        "consumption": 1.124,
        "status": null
      },
      {
        "date": "2024-01-16T06:00:00",
        "week": null,
        "consumption": 1.384,
        "status": null
      },
      {
        "date": "2024-01-16T07:00:00",
        "week": null,
        "consumption": 1.517,
        "status": null
      },
      {
        "date": "2024-01-16T08:00:00",
        "week": null,
        "consumption": 1.479,
        "status": null
      },
      {
        "date": "2024-01-16T09:00:00",
        "week": null,
        "consumption": 1.382,
        "status": null
      },
      {
        "date": "2024-01-16T10:00:00",
        "week": null,
        "consumption": 1.221,
        "status": null
      },
      {
        "date": "2024-01-21T01:00:00",
        "week": null,
        "consumption": 0.945,
        "status": null
      }
    ],
    "highLoadConsumption": null,
    "lowLoadConsumption": null,
    "highLoad": null,
    "monthlyLoad": null
  }
]
2

It’s not impossible but if you have to ask it’s probably beyond you. Would involve directly writing to the database, and would probably not play nicely with the energy dashboard.

The easiest way is to measure power consumption directly. Options for that are a current clamp on your meter tail, or something that interfaces with your electricity meter.

I use an ESP8266 device with a photodiode, counting the 1/Wh flashing light on my electricity meter, and that data comes straight into HA:

image

3

I wouldn’t be concerned with tampering with the database per se, but this opens up a pretty big can of worms in that I don’t know what cascade effects this could have down the line, or that it could break in a future update. My intent is to accomplish this in a way which is in line with the intended way of doing things in HA, and it seems to me like sensors must by definition be able to provide an answer for “what is your reading RIGHT NOW” and that HA then collects those values and provides all the historical persistence for that. I think I will abandond this plan as it would end up being nothing more than a hard-to-maintain hack.

That leaves me with either purchasing a power meter for my grid connection, or setting up a similar photodiode device counting flashes (my power meter also does this), OR pulling out the data from HA and graphing the stuff I’m interested in on a completely HA-free dashboard. I think I like the latter solution the most for now. Might purchase a power meter down the line but that’ll probably have to wait until I take the plunge and get solar panels for the house.

Thanks for the input, Troon.