How to process REST request?

jriker1 · April 21, 2020, 12:44pm

I have a monitoring device on one of my computers that I want to pull data from and show it in HA. It’s basically a rest call thru a webservice. So basically I can call it like:

curl -s -k -u admin:<password> https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:

The results are:

{"command":"check_drivesize","lines":[{"message":"OK All 1 drive(s) are ok","perf":{"c: used":{"critical":199.99687156639993,"maximum":222.21874618530273,"minimum":0.00000000000000000,"unit":"GB","value":68.534137725830078,"warning":177.77499694749713},"c: used %":{"critical":90.000000000000000,"maximum":100.00000000000000,"minimum":0.00000000000000000,"unit":"%","value":31.000000000000000,"warning":80.000000000000000}}}],"result":0}

In theory then to break it out:

{
   "command":"check_drivesize",
   "lines":[
      {
         "message":"OK All 1 drive(s) are ok",
         "perf":{
            "c: used":{
               "critical":199.99687156639993,
               "maximum":222.21874618530273,
               "minimum":0.00000000000000000,
               "unit":"GB",
               "value":67.551097869873047,
               "warning":177.77499694749713
            },
            "c: used %":{
               "critical":90.000000000000000,
               "maximum":100.00000000000000,
               "minimum":0.00000000000000000,
               "unit":"%",
               "value":30.000000000000000,
               "warning":80.000000000000000
            }
         }
      }
   ],
   "result":0
}

So thinking two things I’d want to do if I could figure it out:

Create a Gauge that shows how full or empty the drive is based on maximum, minimum and thinking that 68.434 is current used. Also be nice to have the color of the gauge change based on the defined critical values or percentages. Any help with doing this? Thinking if I can understand how to pull the data into HA for this I will be able to figure out all the rest I want to pull in.
Get the 0 value at the end. That in theory should be the “OK” that everything is cool with this drive.

Thanks.

JR

VDRainer · April 21, 2020, 9:30pm

Copy and paste the whole result in Dev Tools/templates like shown here and play with it.

This template you can use with the rest sensor.

jriker1 · April 21, 2020, 11:19pm

Ahhh, interesting. With a call like the curl command I have above work in HA? I know i tried to use it with the Restful Sensor and it didn’t like the “?drive=c:” Would error when I tested and would at least pass the test when I removed it.

For reference when I do put the above into the resource I get:

Error loading /config/configuration.yaml: mapping values are not allowed here
in “/config/entities/sensor/sensor_rest.yaml”, line 2, column 95

Tried slashing out things but didn’t help like

resource: https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute\?drive\=c:

Thanks.

JR

VDRainer · April 22, 2020, 6:57am

Maybe try it with quotes around te url?

jriker1 · April 22, 2020, 1:24pm

I did single quotes and that seemed to get past that hump. Now on the value template having challenges because it already has quotes in the template:

value_template: {{ value_json.lines[0].perf['c: used'].value }}

Tried a variety of quotes like below, including double quotes but nothing returning just showing the value template for the value.

- platform: rest
  resource: 'https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:\'
  username: admin
  password: test1
  name: C_Drive_Used
  authentication: basic
  verify_ssl: false
  value_template: '{{ value_json.lines[0].perf[''c: used''].value }}'
  unit_of_measurement: '{{ value_json.lines[0].perf[''c: used''].unit }}'

Should also note with the above password i get no errors in the logs and it’s not the right password.

Note now switched to the below quotes but no difference:

- platform: rest
  resource: 'https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:\'
  username: admin
  password: test1
  name: C_Drive_Used
  authentication: basic
  verify_ssl: false
  value_template: "{{ value_json.lines[0].perf['c: used'].value }}"
  unit_of_measurement: "{{ value_json.lines[0].perf['c: used'].unit }}"

jriker1 · April 23, 2020, 11:59am

So here’s where I’m at. It appears HA doesn’t EVER like colons at the end of a line unless it’s like a section that everything under it is indented under it. So I put a slash at the end for the resource and then for good measure a double slash. I did that because when you do a curl call with one slash it returns the results and then sits there. Double slash takes it back to the cursor so figured I would do that:

- platform: rest
  resource: https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:\\
  username: admin
  password: <rest_call_password>
  name: C_Drive_Used
  authentication: basic
  verify_ssl: false
  json_attributes:
    - value
    - unit
  value_template: "{{ value_json.lines[0].perf['c: used'].value }}"
  unit_of_measurement: "{{ value_json.lines[0].perf['c: used'].unit }}"

I’m not getting any errors in the logs but no results either. Basically all I get is:

or the full length version:

VDRainer · April 23, 2020, 7:06pm

The unit_of_measurement is not templatable, but that is not the problem, i think.
If the rest sensor doesn’t work with this kind of url, maybe try it with the command_line sensor and your already working curl command.

jriker1 · April 23, 2020, 9:16pm

Curious how we officially know like is there some logging that would return what it’s seeing or not? Below is where I’m at now with no effect:

- platform: rest
  resource: 'https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:\\'
  method: GET
  username: admin
  password: test1
  name: C_Drive_Used
  authentication: basic
  verify_ssl: false
  value_template: "{{ value_json.lines[0].perf['c: used'].value }}"
  unit_of_measurement: '%'

Going to try the command line but not a fan of that path.

CentralCommand · April 23, 2020, 9:28pm

Did you try encoding the value of that query string parameter? I ran it through encodeURIComponent, what if you tried this:
https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c%3A%5C%5C

[EDIT] Ok hang on I think I misunderstood. So the actual URL doesn’t have any \'s, it looks like this right?
https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:

So first of all you could try just URL escaping the colon
https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c%3A

Alternatively if it has to be a colon and that’s definitely an issue, report a bug since that seems like one. But maybe as a workaround, does your API care if you provide additional query string parameters or would it just ignore them? like what if you just did this:
https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:&foo=bar

jriker1 · April 23, 2020, 10:45pm

I will try and check but the command line was no better. Did:

platform: command_line
name: C Drive Used
command: “curl -s -k -u admin:<admin_pass> https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c:\”
unit_of_measurement: “%”
value_template: “{{ value_json.lines[0].perf[‘c: used’].value }}”

Got the same unknown value.

Just tried and apparently the colon isn’t required. but will see if that matters.

firstof9 · April 23, 2020, 10:59pm

If you set the REST component to debug it’ll likely help you with the rest sensor issue:

CentralCommand · April 23, 2020, 11:33pm

I just realized your URL is https but its a local IP address. I’m guessing your using a self-signed cert? The REST sensor has a verify_ssl option which defaults to true when not specified. Try adding:

  verify_ssl: false

Also like @firstof9 said, definitely turn on debug. Then it’ll tell you what is going wrong more specifically

jriker1 · April 23, 2020, 11:37pm

I’ll try verify_ssl had that on the “gui” version but took it out on the command line version. Then if that doesn’t work will enable debugging per firstof9

EDIT: verify_ssl says a non valid option for the command line version I’m playing with.

EDIT EDIT: When I try typing in logger.set_level or just get to logger nothing shows up. Note see I need to add logger: to my configuration.yaml so doing that now.

JR

jriker1 · April 23, 2020, 11:57pm

OK that may have been helpful. I see an error 403:

2020-04-23 18:53:59 DEBUG (SyncWorker_7) [homeassistant.components.rest.sensor] Data fetched from resource: 403 Your not allowed

I can reproduce this by running the call thru curl and NOT passing the username and password.

So from what I can tell the username and password are not being sent along or something. Not sure if that’s normal also but never see the username and password in the debug logs.

mrand · April 24, 2020, 4:20am

Not sure I follow. Are you saying you see the curl command, but don’t see the arguments?

I’ve been working on a set of command_line curl commands for a week now… HA has some weird behaviors when it comes to quotes… especially single quotes. But I don’t see that in your sensor. Regardless, the debug log shows the full command with arguments - so if stuff is missing, then you definitely need to focus on resolving that.

BTW, speaking of the full command with arguments, putting the password the command line is not ideal for security: it could show up if another user did a “ps” while that command was running. It’s preferable to use a .netrc file with curl.

Here’s what I’ve got working, if it helps you: https://github.com/mrand/smart_meter_texas/tree/master/home-assistant

jriker1 · April 24, 2020, 1:18pm

Scratch what I just wrote. So what the heck happened? So I have this:

- platform: rest
  resource: 'https://192.168.0.1:9999/api/v1/queries/check_drivesize/commands/execute?drive=c'
  method: GET
  username: admin
  password: <password>
  name: C_Drive_Used
  authentication: basic
  verify_ssl: false
  value_template: "{{ value_json.lines[0].perf['c: used'].value }}"
  unit_of_measurement: '%'

Which I don’t think is different than something I had before and now it’s returning results. Hope having logger on isn’t making it work which would be silly.

jriker1 · April 24, 2020, 1:39pm

So now that I got over that painful process with no idea why or how, what is the best way to represent this. So I have per above in the call:

{% set value_json = {
   "command":"check_drivesize",
   "lines":[
      {
         "message":"OK All 1 drive(s) are ok",
         "perf":{
            "c: used":{
               "critical":199.99687156639993,
               "maximum":222.21874618530273,
               "minimum":0.00000000000000000,
               "unit":"GB",
               "value":69.114101409912109,
               "warning":177.77499694749713
            },
            "c: used %":{
               "critical":90.000000000000000,
               "maximum":100.00000000000000,
               "minimum":0.00000000000000000,
               "unit":"%",
               "value":31.000000000000000,
               "warning":80.000000000000000
            }
         }
      }
   ],
   "result":0
} %}

I was thinking of using a glance card but it only allows for one value and the maximum, and warning values are manual entry. In theory with the above I could define them all dynamically. Any thoughts on how to do that using just more rest calls based on the above?

firstof9 · April 24, 2020, 1:54pm

Try adding to your rest sensor:

    json_attributes_path: "$.[0].perf['c: used']"
    json_attributes:
      - critical
      - maximum
      - minimum

This would add them as attributes to the sensor, to which you could use a template sensor to break them out into their own entities.

jriker1 · April 24, 2020, 2:15pm

Thanks firstof9. Would I need to remove the value_template? Oh and will the 255 character limit cause this to crap out with to much info?

firstof9 · April 24, 2020, 2:15pm

Nope leave that in