How to decode special JSON ? Ex => "conso_hc" : 006699756,

Neuvidor · December 7, 2020, 9:10pm

Dear all the HA community,

I’m sure you can help me regarding my issue.

I’m trying to get data from my LEGRAND EcoCompteur through the REST platform.
I did something working well into the configuration.yaml

  - platform: rest
    name: "House heating power"
    resource: http://192.168.1.92/inst.json
    value_template: '{{ value_json.data1 }}'
    device_class: power
    unit_of_measurement: "W"

Raw data

{
    "data1":1181.000000,
    "data2":289.000000,
    "data3":53.000000,
    "data4":51.000000,
    "data5":0.000000,
    "data6":0.000000,
    "data6m3":0.000000,
    "data7":0.000000,
    "data7m3":0.000000,
    "heure":21,
    "minute":44,
    "CIR1_Nrj":0.000000,
    "CIR1_Vol":0.000000,
    "CIR2_Nrj":0.000000,
    "CIR2_Vol":0.000000,
    "CIR3_Nrj":0.000000,
    "CIR3_Vol":0.000000,
    "CIR4_Nrj":0.000000,
    "CIR4_Vol":0.000000,
    "Date_Time":1607377448
}

Unfortunately it’s more difficult than planned regarding the second request below :

http://192.168.1.92/data.json

The problem is that the data looks like Json but finally they don’t respect the Json rules regarding conso_hc and conso_hp values. The problem seems that the value start with a zero value.

	"conso_hc"   : 006699756,    =>  "conso_hc"   : 6699756,  
	"conso_hp"   : 014039296,    =>  "conso_hp"   : 14039296,

Question :

How to get “conso_hc” and “conso_hp” values into HA ?

####################################################

http complete raw answer of the second request :

{
	"option_tarifaire" : 1,
	"tarif_courant" : 2,
	"isousc" : 60,
	
	"conso_base" : 0,
	"conso_hc"   : 006699756,
	"conso_hp"   : 014039296,
	"conso_hc_b" : 0,
	"conso_hp_b" : 0,
	"conso_hc_w" : 0,
	"conso_hp_w" : 0,
	"conso_hc_r" : 0,
	"conso_hp_r" : 0,
	
	"type_imp_0" : 0,
	"type_imp_1" : 0,
	"type_imp_2" : 1,
	"type_imp_3" : 1,
	"type_imp_4" : 1,
	"type_imp_5" : 1,

	"label_entree1" : "Chauffage",
	"label_entree2" : "Eau chaude",
	"label_entree3" : "Eclairage           ",
	"label_entree4" : "Cuisine             ",
	"label_entree5" : "Seches serviettes   ",
	
	"label_entree_imp0" : "Eau froide adoucie",
	"label_entree_imp1" : "Eau chaude",
	"label_entree_imp2" : "Eau",
	"label_entree_imp3" : "Eau",
	"label_entree_imp4" : "Eau",
	"label_entree_imp5" : "Eau",
	
	"entree_imp0_disabled" : 1,
	"entree_imp1_disabled" : 1,
	"entree_imp2_disabled" : 1,
	"entree_imp3_disabled" : 1,
	"entree_imp4_disabled" : 1,
	"entree_imp5_disabled" : 1
}

VDRainer · December 7, 2020, 9:39pm

Copying the json in Dev tools/templates works here:

{% set value_json = {
	"option_tarifaire" : 1,
	"tarif_courant" : 2,
	"isousc" : 60,
	
	"conso_base" : 0,
	"conso_hc"   : 006699756,
	"conso_hp"   : 014039296,
	...
	
	"entree_imp0_disabled" : 1,
	"entree_imp1_disabled" : 1,
	"entree_imp2_disabled" : 1,
	"entree_imp3_disabled" : 1,
	"entree_imp4_disabled" : 1,
	"entree_imp5_disabled" : 1
} %}
{{ value_json.conso_hc }}

returns
Auswahl_411

Hellis81 · December 7, 2020, 9:41pm

It seems the values are 9 characters long?
If so you could use this method to get the last 9 characters of a string that is padded with zeros on the left (and right since I could not get the last character by some reason).

{% set s = '123456' %} # your json value
{% set s = '000000000' + s + '0' %}
{{ s[-10:-1] }} # 000123456

Hellis81 · December 7, 2020, 9:42pm

Probably because the template tool reads it as a string (because it is) but the json parser takes it as an integer.

Neuvidor · December 7, 2020, 10:06pm

Thank you @VDRainer for this check.

I did the same check and yes it works but I got an unknown value in the overview tab.

My configuration.yaml

  - platform: rest
    name: "conso_hc"
    resource: http://192.168.1.92/data.json
    value_template: '{{ value_json.conso_hc }}'
    device_class: energy
    unit_of_measurement: "kWh"

Result on overview tab :

Developer tools check

Neuvidor · December 7, 2020, 10:11pm

@Hellis81,

Your proposal seems great, if we can read right to left and stop when a zero appear, we will be able to get the value.

Could you please help me to put your code into my HA ?
How to put your code in configuration.yaml ?

Hellis81 · December 7, 2020, 10:29pm

Something like:

value_template: 
               {% set s = value_json.conso_hc %} # your json value
               {% set s = '000000000' + s + '0' %}
               {{ s[-10:-1] }}

I can’t remember if you must have a > sign on the value_template row or not.

But there won’t be a zero in your json if I understand correctly?
What you need is the last nine digits of the string.
But since you may get a string with less characters then if we pad with zeros and get the last nine then it should be correct.

Like:
Json = 1234
We pad it to
0000000001234
And extract the last nine:
000001234

Right?

Neuvidor · December 7, 2020, 10:50pm

My problem is that the raw value is not JSON.

Try to copy the http raw answer below to the website https://jsonformatter.curiousconcept.com/#

{
	"option_tarifaire" : 1,
	"tarif_courant" : 2,
	"isousc" : 60,
	
	"conso_base" : 0,
	"conso_hc"   : 006699756,
	"conso_hp"   : 014039296,
	"conso_hc_b" : 0,
	"conso_hp_b" : 0,
	"conso_hc_w" : 0,
	"conso_hp_w" : 0,
	"conso_hc_r" : 0,
	"conso_hp_r" : 0,
	
	"type_imp_0" : 0,
	"type_imp_1" : 0,
	"type_imp_2" : 1,
	"type_imp_3" : 1,
	"type_imp_4" : 1,
	"type_imp_5" : 1,

	"label_entree1" : "Chauffage",
	"label_entree2" : "Eau chaude",
	"label_entree3" : "Eclairage           ",
	"label_entree4" : "Cuisine             ",
	"label_entree5" : "Seches serviettes   ",
	
	"label_entree_imp0" : "Eau froide adoucie",
	"label_entree_imp1" : "Eau chaude",
	"label_entree_imp2" : "Eau",
	"label_entree_imp3" : "Eau",
	"label_entree_imp4" : "Eau",
	"label_entree_imp5" : "Eau",
	
	"entree_imp0_disabled" : 1,
	"entree_imp1_disabled" : 1,
	"entree_imp2_disabled" : 1,
	"entree_imp3_disabled" : 1,
	"entree_imp4_disabled" : 1,
	"entree_imp5_disabled" : 1
}

The problem is that the answer is not JSON => Invalid number
My new question is : How to get value from a not Json answer ?

Hellis81 · December 8, 2020, 5:39am

If that is interpreted as a plain string then you can perhaps use regex

{{ str | regex_findall_index('conso_hc\s+:\s+(\d+),', ignorecase=True)}}
{{ str | regex_findall_index('conso_hp\s+:\s+(\d+),', ignorecase=True)}}

Neuvidor · December 8, 2020, 7:32am

Thanks for your support, I really appreciate !
I have implemented your proposal like below but I got error into the yaml, could you have a look ?

  - platform: rest
    name: "conso_hc"
    resource: http://192.168.1.92/data.json
    value_template: "{{ str | regex_findall_index('conso_hc\s+:\s+(\d+),',ignorecase=True)}}"
    device_class: energy
    unit_of_measurement: "kWh"

Error I get

Neuvidor · December 8, 2020, 7:33am

I found a similar issue on the forum HERE, the problem is that my competencies are not enough to make the parrallel

Hellis81 · December 8, 2020, 7:50am

I used str since I don’t have your json data.
I assigned a text version of the json data to str as you can see in the image.

You need to either replace str with value_json or assign str to value_json. (Or whatever the string is called that you get)

Neuvidor · December 8, 2020, 8:27am

it doesn’t work from my side in dev tools.
I think the reason is that you have deleted the double quote in the text, so this is different.
I tried to adapt the regex_findall_index command without success.

code into dev tools

{% set str = {
	"option_tarifaire" : 1,
	"tarif_courant" : 2,
	"isousc" : 60,
	
	"conso_base" : 0,
	"conso_hc"   : 006699756,
	"conso_hp"   : 014039296,
	"conso_hc_b" : 0,
	"conso_hp_b" : 0,
	"conso_hc_w" : 0,
	"conso_hp_w" : 0,
	"conso_hc_r" : 0,
	"conso_hp_r" : 0,
	
	"entree_imp0_disabled" : 1,
	"entree_imp1_disabled" : 1,
	"entree_imp2_disabled" : 1,
	"entree_imp3_disabled" : 1,
	"entree_imp4_disabled" : 1,
	"entree_imp5_disabled" : 1
} %}
{{ str | regex_findall_index('conso_hc\s+:\s+(\d+),', ignorecase=True)}}
{{ str | regex_findall_index('conso_hp\s+:\s+(\d+),', ignorecase=True)}}

result

Neuvidor · December 8, 2020, 9:01am

There is really strange behavior in HA.
Json data reading is working well into the dev tools but doesn’t work into configuration.yaml

Do I made any mistake into my comfiguration.yaml file ?


# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml


sensor:
  - platform: systemmonitor
    resources:
      - type: last_boot
      - type: ipv4_address
        arg: eth0
      - type: processor_temperature
      - type: processor_use
      - type: processor_temperature
      - type: memory_use_percent
      - type: disk_use_percent
      
  - platform: rest
    name: "House heating power"
    resource: http://192.168.1.92/inst.json
    value_template: '{{ value_json.data1 }}'
    device_class: power
    unit_of_measurement: "W"
    
  - platform: rest
    name: "Water heating power"
    resource: http://192.168.1.92/inst.json
    value_template: '{{ value_json.data2 }}'
    device_class: power
    unit_of_measurement: "W"
    
  - platform: rest
    name: "Litgh power"
    resource: http://192.168.1.92/inst.json
    value_template: '{{ value_json.data3 }}'
    device_class: power
    unit_of_measurement: "W"
    
  - platform: rest
    name: "Kitchen power"
    resource: http://192.168.1.92/inst.json
    value_template: '{{ value_json.data4 }}'
    device_class: power
    unit_of_measurement: "W"
    
  - platform: rest
    name: "conso_hc"
    resource: http://192.168.1.92/data.json
    value_template: '{{ value_json.conso_hc }}'
    device_class: energy
    unit_of_measurement: "kWh"

Neuvidor · December 9, 2020, 12:50pm

Hello @Hellis81,

I tried your code but unfortunately HA does not accept your value_template proposal.
The system restart into safe mode after the reboot.

Do you have any idea ?

Hellis81 · December 9, 2020, 1:35pm

no…
I would probably do something about the source.
You could perhaps use something to parse the incorrect data and feed HA with correct json.

Neuvidor · December 9, 2020, 8:33pm

@Hellis81,
No problem, thank you for your time !

Hellis81 · December 9, 2020, 9:56pm

You could probably do this with node red.
It has JavaScript in the function nodes. That is probably a good way to parse the data.

Just use an inject node to start the sequence timed and have it populate a input text with the value.

Neuvidor · December 9, 2020, 10:01pm

My brother will try to help me tomorrow evening (french time).
If no sucess, If somebody have time to help me, I can organize a teamviewer session

Hellis81 · December 9, 2020, 10:09pm

Sorry won’t have time for that.
I have two kids that need my attention.

But a node red sequence is not very hard to set up.

Take inject node, http node and debug node to start with.
Connect the three and set debug to output complete message.
Add the URL of the json in the http node and press the inject node and see what the debug console shows.

If that gives you the json(-ish) then it’s fairly simple to do the rest