Parse json with special characters coming from Ubibot

parabag · June 11, 2020, 8:07am

Hello guys, I am integrating a sensor from Ubibot (WS1) to check temperature, humidity and most of all outside brightness. I received the info through a GET call at api.ubibot.com/channels?account_key=blablablablablablablablabla.

The json received is in a pretty complicated format:

{
   "result":"success",
   "server_time":"2020-06-10T11:20:26Z",
   "channels":[
      {
         "channel_id":"123456",
         "field1":"Temperature",
         "field2":"Humidity",
         "field3":"Light",
         "field4":"Voltage",
         "field5":"WIFI RSSI",
         "field6":"Vibration Index",
         "field7":"Knocks",
         "field8":"External Temperature Probe",
         "field9":"",
         "field10":null,
         "latitude":"45.05",
         "longitude":"7.6667",
         "name":"C-123456",
         "public_flag":"false",
         "tags":null,
         "url":null,
         "metadata":"{\"fn_th\":300,\"fn_light\":300,\"fn_mag\":0,\"fn_mag_int\":0,\"fn_acc_tap1\":0,\"fn_acc_tap2\":0,\"fn_acc_act\":0,\"fn_acc_min\":10,\"fn_bt\":0,\"fn_ext_t\":300,\"fn_battery\":7200,\"fn_dp\":900,\"cg_data_led\":1, \"wifi_mode\": 1, \"no_net_fn\": 1}",
         "description":null,
         "traffic_out":"8384",
         "traffic_in":"9203",
         "status":"{\"ssid\":\"xxxxxxxxxxx\",\"status\":\"mac=MACADDRESS,usb=0\",\"usb\":\"0\",\"mac\":\"MACADDRESS\"}",
         "timezone":null,
         "created_at":"2020-06-10T08:31:46Z",
         "updated_at":"2020-06-10T08:31:46Z",
         "usage":"25438",
         "last_entry_id":"128",
         "last_entry_date":"2020-06-10T11:19:35Z",
         "product_id":"ubibot-ws1",
         "device_id":"MYDVICEID1234",
         "channel_icon":null,
         "last_ip":"47.53.15.102",
         "attached_at":"2020-06-10T08:31:46Z",
         "firmware":"ws1_v2.6.9",
         "full_dump":"0",
         "activated_at":"1591778067",
         "serial":"AKG***WS1",
         "mac_address":"MACADDRESS",
         "full_dump_limit":"3",
         "cali":null,
         "size_out":"1073741824",
         "size_storage":"209715200",
         "plan_code":"ubibot_free",
         "plan_start":"2020-06-10T08:31:46Z",
         "plan_end":null,
         "bill_start":"2020-06-10T08:31:46Z",
         "bill_end":"2020-07-10T08:31:46Z",
         "last_values":"{\"field1\":{\"value\":20.041199,\"created_at\":\"2020-06-10T11:19:29Z\",\"net\":\"1\"},\"field3\":{\"value\":808.320007,\"created_at\":\"2020-06-10T11:19:29Z\"},\"field2\":{\"value\":51,\"created_at\":\"2020-06-10T11:19:29Z\",\"net\":\"1\"},\"field4\":{\"value\":3.165038,\"created_at\":\"2020-06-10T10:34:29Z\"},\"field5\":{\"value\":-63,\"created_at\":\"2020-06-10T11:19:32Z\"},\"wifi\":{\"value\":\"MACADDRESS,-74;MACADDRESS,-83;MACADDRESS,-70;MACADDRESS,-63;MACADDRESS:ec,-81\",\"created_at\":\"2020-06-10T11:19:32Z\"}}",
         "vconfig":"{\"field1\":{\"h\":\"0\",\"u\":\"1\"},\"field2\":{\"h\":\"0\",\"u\":\"3\"},\"field3\":{\"h\":\"0\",\"u\":\"4\"},\"field4\":{\"h\":\"0\",\"u\":\"5\"},\"field5\":{\"h\":\"0\",\"u\":\"6\"},\"field6\":{\"h\":\"0\",\"u\":\"7\"},\"field7\":{\"h\":\"0\",\"u\":\"8\"},\"field8\":{\"h\":\"0\",\"u\":\"1\"},\"field9\":{\"h\":\"0\",\"u\":\"9\"}}",
         "vpref":"{\"fields\":[{\"key\":\"field1\",\"index\":0,\"visible\":true},{\"key\":\"field2\",\"index\":1,\"visible\":true},{\"key\":\"field3\",\"index\":2,\"visible\":true},{\"key\":\"field4\",\"index\":3,\"visible\":true},{\"key\":\"field5\",\"index\":4,\"visible\":true},{\"key\":\"field6\",\"index\":5,\"visible\":true},{\"key\":\"field7\",\"index\":6,\"visible\":true},{\"key\":\"field8\",\"index\":7,\"visible\":true}],\"v\":1}",
         "battery":null,
         "vpref_from":"owner",
         "net":"1",
         "c_icon_base":null,
         "full_serial":"XXXXXXXXX",
         "triggering_rules":null
      }
   ]
}

I want to extract the info related to: “last_values”:"{“field1”:{“value”:20.041199,“created_at”:“2020-06-10T11:19:29Z”,“net”:“1”},“field3”:{“value”:808.320007,“created_at”:“2020-06-10T11:19:29Z”}…

But due to the usage of special characters (") I cannot find a way out… I am stuck to this…

# Integration with UBIBOT
  - platform: rest
    resource: https://api.ubibot.io/channels?account_key=blablablablablablablablabla
    name: UBIBOT
    value_template: '{{ value_json.channels[0].last_value.field1 }}'

But it is absolutely NOT WORKING
Can somebody help me?

thanks very much!

Luca

pnbruckner · June 11, 2020, 4:42pm

So the problem is there is double JSON encoding going on here for many of these items, including last_values. Try this, e.g.:

    value_template: '{{ (value_json.channels[0].last_values|from_json).field1 }}'

parabag · June 11, 2020, 4:58pm

Thanks Phil… you are a star… solved! I did tried as well with from_json command but I was not sure about the syntax.
Finally since I wanted to extract the “value” I justed added it at the end and it works:

value_template: '{{ (value_json.channels[0].last_values|from_json).field1.value }}'

thanks again

Luca

IceAge · June 14, 2020, 4:55am

Hi parabag, I just got a WS1 today, and HA yesterday. Is there any chance you could either post exactly what you did, or maybe PM me or something? I’m reasonably good with tech, but I’m lacking experience with HA and especially POST/GET with respect to how to make use of them. If you choose to help, please pretend like you’re talking to someone who has little idea of what they’re doing.

What my goal is, is to have temperature, humidity, and lux display in HA, and later on, to trigger a Google Assistant routine to say, “turn on the living room heater”. The reason for this, is that my VOCOlinc smart plugs don’t have an integration in HA, but they do in Google, and I’m not sure yet if the firmware can be replaced with something friendlier.

I’d be really happy to get three gauges to show the three data points in HA for starters, but I’m not really sure how to start.

One thing I noticed is that UbiBot has an HTTP data forwarding (beta) feature that appears to send a copy of the JSON data from the WS1 to whatever URL you want. So I’m going to start there with getting DuckDNS going alongside HTTPS, and then after that it probably shouldn’t be too tough to set up HA to receive the POST and “do something” with it.

UbiBot also appears to have local integration where you can completely cut them out and have no need for outside internet access, but they claim there’s a one time fee which I haven’t been able to locate as yet. I might pursue that option if it doesn’t cost a ridiculous amount.

Thank you in advance.

EDIT: Spelling error

parabag · June 15, 2020, 11:07am

It is very easy…
go in your configuration.yaml file and put this:

# Integration with UBIBOT
  - platform: rest
    resource: https://api.ubibot.io/channels?account_key=xxxxxxxxxxxxxxxxxxxxxxx
    name: UBIBOT
    value_template: '{{ (value_json.channels[0].last_values|from_json).field3.value|round()|float(value) }}'

Once you did this you will have your integration done (go in integration section of HA) and you can set gauges, trends… etc as you like for the lux. filed1 and field2 I think are for Temperature and humidity if I’m not wrong and it is the same procedure…
For what concerns Google Assistant I cannot help you very much because I do not have it.

IceAge · June 16, 2020, 2:24am

I figured it probably had to be pretty easy. I’ve got the remote access with HTTPS working now, so I should probably be good from here.

In the future I think I’m going to just get an Arduino instead of the Ubibot. Overall it seems like a way better route to go, especially considering there’s practically no end to the types of sensors you can add. I don’t like how you have to rely on UbiBot for the device to function. You can buy a developer membership for a mere $5000 (LOL) to get a local server and become independent.

I’ll figure out the Google thing. I imagine there’s a way to do what I’m going for, and if not I’ll just buy Hassio friendly smart plugs and call it “cost of learning”.

Thank you VERY VERY MUCH for your assistance, I really appreciate it.

IceAge · June 20, 2020, 11:38pm

Hi parabag, or anyone else who reads this; I put the above code into configuration.yaml exactly as shown, except with my own Ubibot “account key” which I got from the Ubisoft website under settings/security, but I keep getting an error in Hassio that states: “bad indentation of a mapping entry at line 22, column 2:
- platform: rest”

I can’t seem to figure out what’s going wrong. Here’s the COMPLETE code I used:

Integration with UBIBOT

 -  platform: rest
    resource: https://api.ubibot.io/channels?account_key=my_account_key
    name: "UBIBOT"
    value_template: '{{ (value_json.channels[0].last_values|from_json).field1.value|round()|float(value) }}'

I’ve checked the YAML syntax with a few websites, and it’s apparently correct. Whatever I’m missing surely can’t be all that major.

How do you get Ubibot to send the data to the Hassio endpoint? It looks like you’ve used a GET request, which I’m having trouble working out how to do. Currently I’m just trying to use the Ubibot data forwarding option, which does appear to work, as I’ve tested it at RequestBin.

Thank you in advance.

UPDATE: Looks like I fixed it. Sure enough, it was simple. I amended the code as follows:

sensor flower1
 -  platform: rest
     resource: https://api.ubibot.io/channels?account_key=my_account_key
     name: "UBIBOT"
     value_template: '{{ (value_json.channels[0].last_values|from_json).field1.value|round()|float(value)}}'

Now I’m able to add a gage and interact with this webhook as a sensor! It appears lack of experience was the problem.

pvanleen · June 24, 2020, 6:18am

Hi All,

I too have issues with reading and writing to a device that uses .XML to read from and write to the device. It is a Xytronics ‘ControlByWeb’ X410 PLC. I would like to be able to connect to it via HA. The data received is as follows-

<datavalues>
<bmsplcDigitalInput1>0</bmsplcDigitalInput1>
<bmsplcDigitalInput2>0</bmsplcDigitalInput2>
<bmsplcDigitalInput3>0</bmsplcDigitalInput3>
<bmsplcDigitalInput4>0</bmsplcDigitalInput4>
<alarmSounder>0</alarmSounder>
<bmsplcRelay2>0</bmsplcRelay2>
......
<mainBedroomTemp>20.0</mainBedroomTemp>
<utcTime>1592979110</utcTime>
<timezoneOffset>28800</timezoneOffset>
<serialNumber>00:0C:C8:03:E1:A1</serialNumber>
</datavalues>

I know roughly how but having issues with correct syntax to get the data.

Any help appreciated.
Phil

pvanleen · June 24, 2020, 7:11am

Sorry, I should have said the data is also read and written in json. IE:

{
"bmsplcDigitalInput1":"0",
"bmsplcDigitalInput2":"0",
"bmsplcDigitalInput3":"0",
"bmsplcDigitalInput4":"0",
"alarmSounder":"0",
"bmsplcRelay2":"0",
"bmsplcRelay3":"0",
"bmsplcRelay4":"0",
.......
"utcTime":"1592982036",
"timezoneOffset":"28800",
"serialNumber":"00:0C:C8:03:E1:A1"
}

jlanglois · December 19, 2020, 2:57am

I have this working for 1 Ubibot, now that I have added a second, I am not sure how to setup the different channel.

Is that formatted correct?

I have 2 ubibots looking at it they are on channels 17577 and 22692.

sensor:
  - platform: rest
    resource: https://api.ubibot.io/channels/17577?account_key=PRIVATE  
    name: Temperature_Kitchen_ubibot
    value_template: '{{ (value_json.channels[0].last_values|from_json).field1.value|float(value) }}'```