JSON REST sensor State unknown BeeClear

Hi, I’m totally new to JSON so I’m doing something wrong here. I can’t figure it it out.

My goal:
To create different sensors from a GET request using the rest sensor from a device called BeeClear.

My JSON:

{
  "starttime": 1628462820,
  "endtime": 1628462820,
  "version": "49.10_NL",
  "type": "elekw",
  "serial": [
    {
      "serial": "elekw-serial",
      "start": 1462893260,
      "end": 1484596470
    },
    {
      "serial": "E0009110013394714",
      "start": 1484596880,
      "end": 1628462820
    }
  ],
  "meetwaarden": [
    {
      "serial": "E0009110013394714",
      "val": [
        {
          "time": 1628462820,
          "verbruik": 497,
          "leveren": 0
        }
      ]
    }
  ],
  "setting": {
    "landcode": "NL",
    "user": "beeclear",
    "auth": "admin",
    "metertype": false,
    "mijnmeter": true,
    "showgas": false,
    "gasUseElekTime": false,
    "dubbeltariefmeter": true,
    "levering": true,
    "testfirmware": false,
    "enableHttps": false,
    "enableMqtt": true,
    "driefaseMeting": false,
    "rawlogging": false,
    "dsmrtime": false,
    "certUse": false,
    "tarief": {
      "gas": 0.6,
      "elekHoog": 0.19512,
      "elekLaag": 0.17982,
      "gasvast24h": 0,
      "elekvast24h": 0
    },
    "starttime": {
      "gas": 1462896000,
      "elek": 1462896000,
      "elekw": 1462893260
    }
  }
}

Examples of what is not working for me:
Returns Unknown

sensor:
  - platform: rest
    resource: http://192.168.2.144/bc_getVal?type=elekw
    name: BCserial
    value_template: '{{ value_json.serial[1]["serial"]}}'
    scan_interval: 10

Returns empty (nothing)

sensor:
  - platform: rest
    resource: http://192.168.2.144/bc_getVal?type=elekw
    name: BCversion
    value_template: '{{ value_json.version}}'
    scan_interval: 10

I’d also like to create sensors from “verbruik” & “leveren” from “meetwaarden”, but I do not understand how to get there because of all the nesting. If anybody could help me out that would be awesome!

It would be this:

'{{ value_json.meetwaarden[0].val[0].verbruik }}'

There’s a useful online tool called JSON Pathfinder to help determine the correct path to a desired key.

However, your other two templates are technically correct but don’t return values so whatever is causing that failure will also affect the template I posted.

1 Like

Thanks! That really helps :grin:

Unfortunately this also returns ‘unknown’, just as you said.
afbeelding

Does anybody know what might be causing this problem? :sweat_smile:

Are you certain the URL is returning data? An entity’s state value is unknown when it has no data.

The URL returns the full JSON as seen above. I though that this was enough.

This simple template should have returned the version:

    value_template: '{{ value_json.version}}'

The fact that it returned nothing is why I asked if Home Assistant is actually receiving anything via the URL.

Is there anything relevant reported in Configuration > Logs?

Okay so it seems to be an authentication problem. I tried the URL on my phone instead of my PC and it returned:

{"info":"notAuthenticated"}

So now my next question becomes: How do I authenticate to get to the data? I tried giving the password & username variables like this (didn’t work):

sensor:
  - platform: rest
    resource: http://192.168.2.144/bc_getVal?type=elekw
    name: BCserial
    value_template: '{{ value_json.serial[1]["serial"]}}'
    scan_interval: 10
    username: beeclear
    password: energie

This is some info about the device’s API. I don’t see any key or token that I must use.

That explains why the entity’s value is unknown because, as suspected, the URL isn’t acquiring the desired data.

I skimmed the document and the only reference to a passwd is on the ninth page and it appears to be for configuring the device’s WiFi connection. There are no examples for URL authentication.

It worked on your PC but not your phone so what’s the difference (if any) between the two with regards to how they are networked to beeclear?

Cookie it seems. On my PC I logged into the webUI (http://192.168.2.144), on my phone I did not. Even though I was logged out of that webUI, I still had access to: http://192.168.2.144/bc_getVal?type=elekw

After clearing my cookies I got the same result as on my phone: {“info”:“notAuthenticated”}

If you first have to access beeclear via its web UI in order to acquire a cookie, that presents a challenge for using a REST sensor.

If the cookie contains a Bearer token, maybe you could copy it and use it in the REST sensor’s configuration as shown in the documentation. However, I am making a huge assumption that whatever is in the cookie can be used that way.

Checked the cookie:

bctokpass af1a637

This changes after every login, so that seems to be useless unfortunately.

Also, I found this (it’s in Dutch sorry):
https://forum.beeclear.nl/announcements.php?aid=3

After running the following URL: http://192.168.2.144/bc_security?set=off, I was able to receive the JSON on my phone (without ever logging in) on http://192.168.2.144/bc_getVal?type=elekw.

afbeelding

And that worked! :partying_face:

Kinda useless security if you can just disable it with one command :sweat_smile: However it’s only on my local network so this seems fine for me. Thanks for all the help @123 :grin:

Allright. So I’ve got the data that I want (for now):
afbeelding

This data comes from the following sensor:

rest:
  - scan_interval: 10
    resource: http://192.168.2.144/bc_getVal?type=elekw
    sensor:
      - name: "BCleveren"
        value_template: '{{ value_json.meetwaarden[0].val[0].leveren }}'
        unit_of_measurement: W
        device_class: power
        force_update: true
      - name: "BCverbruik"
        value_template: '{{ value_json.meetwaarden[0].val[0].verbruik }}'
        unit_of_measurement: W
        device_class: power
        force_update: true
      - name: "BCleverenWh"
        value_template: '{{ value_json.meetwaarden[0].val[0].leveren | multiply(0.001)}}'
        unit_of_measurement: kWh
        device_class: energy
        force_update: true
      - name: "BCverbruikWh"
        value_template: '{{ value_json.meetwaarden[0].val[0].verbruik | multiply(0.001)}}'
        unit_of_measurement: kWh
        device_class: energy
        force_update: true
      - name: "BCLaagTarief"
        value_template: '{{ value_json.setting.tarief.elekLaag }}'
        unit_of_measurement: "€"
        device_class: monetary
      - name: "BCHoogTarief"
        value_template: '{{ value_json.setting.tarief.elekHoog }}'
        unit_of_measurement: "€"
        device_class: monetary

I’d like to use this data in the new Energy section of Home assistant. However, I am not able to use my current sensor setup for that:

It seems I need to let Home Assistant know that both my kWh sensors are ‘Long-term Statistics’?
Sensor Entity | Home Assistant Developer Docs (home-assistant.io)

I do not understand how to do this :sweat_smile:

Add this to customize.yaml (or its equivalent via Configuration > Customizations):

sensor.bcleverenwh:
  state_class: measurement
  last_reset: '1970-01-01T00:00:00+00:00'
sensor.bcverbruikwh:
  state_class: measurement
  last_reset: '1970-01-01T00:00:00+00:00'

Restart and they should become accessible to the Energy integration.

1 Like

That worked wonderfully! That made my day!

My next challenge is to set the current rate in euro’s. So between 23:00 and 7:00 (lets call it nighttime) it should return the value_json.setting.tarief.elekLaag, else value_json.setting.tarief.elekHoog.

I’ve currently got the following:

      - name: "BCHuidigTarief"
        value_template: >-
          {%  if (states.sensor.time.state > '23:00') and (states.sensor.time.state < '07:00') %}
          {{ value_json.setting.tarief.elekLaag }}
          {% else %}
          {{ value_json.setting.tarief.elekHoog }}
          {% endif %}
        unit_of_measurement: "€"
        device_class: monetary

Unfortunately this returns unknown :sob:
I also tried it without the {{ }} or just single {}, however both still result in unknown.

Try this version:

      - name: "BCHuidigTarief"
        value_template: >-
          {%  if now().hour == 23 or now().hour < 7 %}
          {{ value_json.setting.tarief.elekLaag }}
          {% else %}
          {{ value_json.setting.tarief.elekHoog }}
          {% endif %}
        unit_of_measurement: "€"
        device_class: monetary
1 Like

Worked perfectly once again!

I tried adding weekdays since Saturdays and Sundays are always on low rates (regardless of the time). So the times only matter for workingdays (mon-fri):

      - name: "BCHuidigTariefTest"
        value_template: >-
          {%  if now().weekday >=5 or now().hour == 23 or now().hour < 7  %}
          {{ value_json.setting.tarief.elekLaag }}
          {% else %}
          {{ value_json.setting.tarief.elekHoog }}
          {% endif %}
        device_class: monetary
        unit_of_measurement: "€"

However, this time the entity doesn’t even appear in HA at all :thinking:

When something fails so badly, the first thing you should do to debug the problem is inspect Logs for relevant error messages.

It may contain at least one because now().weekday is invalid.

Change it to:
now().weekday()

1 Like

That fixed it! That was my simple but stupid mistake :grin:
This seemed to have appeared in the logs:

2021-08-11 00:27:26 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: list object has no element 0 when rendering '{{ value_json.meetwaarden[0].val[0].leveren }}'
2021-08-11 00:27:26 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: list object has no element 0 when rendering '{{ value_json.meetwaarden[0].val[0].verbruik }}'
2021-08-11 00:27:26 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: list object has no element 0 when rendering '{{ value_json.meetwaarden[0].val[0].leveren | multiply(0.001)}}'
2021-08-11 00:27:26 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: list object has no element 0 when rendering '{{ value_json.meetwaarden[0].val[0].verbruik | multiply(0.001)}}'

This results in the following:

afbeelding

Edit:
Okay, so the above errors sometimes happen after a restart. It doesn’t fix itself. After a restart this got fixed and everything works perfectly again. However, this has now happened, like, 3/4 times now while I was testing. Is there anything I can do to prevent this from happening?

To prevent it, you have to know why it’s happening. The challenge of determining why it’s happening is greater if it doesn’t happen after every restart, only sometimes.

You could make the template “defensive” like this:

{{ value_json.meetwaarden[0].val[0].leveren if value_json.meetwaarden is defined else 0 }}
1 Like

Since i was looking for this integration I ended up here. This integration will not work for multiple reasons.

  • You cannot divide Watt by 1000 and call it KWh, that’s not how it works.
  • The URL get requests in the config often gets NO output response in the measurements array which will be resolved when using the history parameters.

So the configuration above is a starting point but does not work. I spend some time researching and getting it to work so: Here is the fully working configuration for BeeClear and Home assistant running in HASS 7.5 hoping to save people some time and effort.

First step: Disable the API security on the beeclear since their API works but the authentication mechanism is unsuitable for stateless integration. Request the following URL in your browser to disable it.

http://beeclear.local/bc_security?set=off

If your DNS is not set up properly, replace beeclear.local for the IP address of the beeclear device.

Next up the config for the rest polling. I’m still playing with the timing. 10 seconds interval seems excessive so you might change the scan interval and play with the duration get parameter here. In the end the wattage will be converted to KWh anyway with a formula.

rest:
  - scan_interval: 10
    resource: "http://beeclear.local/bc_getVal?type=elekw&duration=10"
    sensor:
      - name: "BCleveren"
        value_template: '{{ value_json.meetwaarden[0].val[0].leveren }}'
        unit_of_measurement: W
        state_class: 'measurement'
        device_class: power
        force_update: true
      - name: "BCverbruik"
        value_template: '{{ value_json.meetwaarden[0].val[0].verbruik }}'
        unit_of_measurement: W
        device_class: power
        state_class: 'measurement'
        force_update: true
      - name: "BCLaagTarief"
        value_template: '{{ value_json.setting.tarief.elekLaag }}'
        unit_of_measurement: "€"
        device_class: monetary
      - name: "BCHoogTarief"
        value_template: '{{ value_json.setting.tarief.elekHoog }}'
        unit_of_measurement: "€"
        device_class: monetary
  - resource: "http://beeclear.local/bc_getVal?type=gas&duration=3599&period=hour"
    sensor:
      - name: "BCGasVerbruik"
        value_template: '{{ value_json.meetwaarden[0].val[0].gas }}'
        unit_of_measurement: 'l'
        device_class: gas
        force_update: true
      - name: "BCGasVerbruikM3"
        value_template: '{{ value_json.meetwaarden[0].val[0].gas | multiply(0.001) }}'
        unit_of_measurement: 'm³'
        device_class: gas
        force_update: true

Since beeclear submits W and not KWh we need to use the home assistant virtual sensors to do the conversion. Add the following sensors.

sensor:
  - platform: integration
    source: sensor.bcleveren
    name: bcleverenKWH
    unit_prefix: k
    round: 2
    
  - platform: integration
    source: sensor.bcverbruik
    name: bcverbruikKWH
    unit_prefix: k
    round: 2

Only thing i didn’t get working is the gas consumption without customizing it. Add the following to your customization.yaml. If anyone has a better solution, let me know.

sensor.bcgasverbruikm3:
  state_class: total_increasing
  last_reset: '1970-01-01T00:00:00+00:00'

Next configure these sensors in your energy dashboard and you are done.

Small update

For some reason beeclear after a while returns invalid json for me. It is random but it shows as the following entries in the logfile

value_json is undefined when rendering

The reason is the json returned by Beeclear is invalid and cannot be parsed due to some kind of unicode character in the ‘user’ field due to some kind of session timeout or something. Clearing the cache and restarting will fix this.

"user":"",

Since the beeclear project seems dead I tried fixing it and found a way to avoid this. Just add a dummy resource to your configuration

  - scan_interval: 10
    resource: "http://beeclear.local/bc_login"
    sensor:
      - name: "BCLoggedIn"
        value_template: '{{ value_json.status }}'

This will return an error message but also fixes the next call to beclear to just return in the json

user:""

My final working config in the configuration.yaml file is

rest:
  - scan_interval: 60
    resource: "http://beeclear.local/bc_getVal?type=elekw&duration=60"
    sensor:
      - name: "BCleveren"
        value_template: '{{ value_json["meetwaarden"][0].val[0]["leveren"] }}'
        unit_of_measurement: W
        state_class: 'measurement'
        device_class: power
        force_update: true
      - name: "BCverbruik"
        value_template: '{{ value_json["meetwaarden"][0].val[0]["verbruik"] }}'
        unit_of_measurement: W
        device_class: power
        state_class: 'measurement'
        force_update: true
  - scan_interval: 60
    resource: "http://beeclear.local/bc_getVal?type=gas&duration=3599&period=hour"
    sensor:
      - name: "BCGasVerbruik"
        value_template: '{{ value_json.meetwaarden[0].val[0].gas }}'
        unit_of_measurement: 'l'
        device_class: gas
        force_update: true
      - name: "BCGasVerbruikM3"
        value_template: '{{ value_json.meetwaarden[0].val[0].gas | multiply(0.001) }}'
        unit_of_measurement: 'm³'
        device_class: gas
        force_update: true
  - scan_interval: 120
    resource: "http://beeclear.local/bc_login"
    sensor:
      - name: "BCLoggedIn"
        value_template: '{{ value_json.status }}'