Sensor state data in Restful sensor

jjpeet · April 13, 2017, 2:50am

Hi,

I’m trying to use sensor state data in in one sensor to populate the setting of another sensor (Rest). See the last line - Authorization:

  - platform: rest
    resource: https://command.aglsolar.com.au/api/v2/live_read?isUpdate=true&lastSuccessfulReadTimestampInUtc={{ utcnow().strftime("%Y%m%d%H%M") }}
    method: GET
    unit_of_measurement: 'W'
    value_template: '{{ value_json[0].energy_consumed|round }}'
    name: Power Use
    headers:
      Referer: https://command.aglsolar.com.au/
      Accept-Encoding: gzip, deflate, sdch, br
      Accept: application/json, text/plain, */*
      Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
      Authorization: Bearer {{  states.sensor.agl_auth.state }}

It doesn’t work. Should it work at all? Can I use a template in this context?

Thx
JP

jjpeet · April 13, 2017, 9:52pm

Just to be clear, it works fine as long as I replace the token with a new one every day. I Paste the token directly into the sensor.
Thx
JP

jjpeet · May 14, 2017, 6:14am

Hi @balloob , do you have any thoughts on this one?
Nobody seems to be able to help me.

I need a security token that is refreshed every 24 hours - so I grab it using the command line sensor (setting that sensor’s stage to the token value), and then want to push that data into the setting of the Rest sensor.

Any other way you’d recommend because it doesn’t look like the rest sensor settings supports templates…

At the moment, I’m manually editing the config.yaml and restarting HA every day to keep my power sensor working.

Thx
James

#AGL Authentication token - renewed every three hours
  - platform: command_line
    name: AGL Auth
    command: "/home/hass/get-auth"
    scan_interval: 10800
#AGL Power usage
  - platform: rest
    resource: https://command.aglsolar.com.au/api/v2/live_read/3de94b39-2a3a-4a48-a302-44be6f5e9c44?isUpdate=true&lastSuccessfulReadTimestampInUtc={{ utcnow().strftime("%Y%m%d%H%M") }}
    method: GET
    unit_of_measurement: 'W'
    value_template: '{{ value_json[0].energy_consumed|round }}'
    name: Power Use
    headers:
      Referer: https://command.aglsolar.com.au/
      Accept-Encoding: gzip, deflate, sdch, br
      Accept: application/json, text/plain, */*
      Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
      Authorization: Bearer {{ states.sensor.agl_auth.state }}

ecko · August 23, 2018, 4:38am

Hi jjpeet,

I’m having a bit of trouble getting the token from AGL, do you have a copy of your get-auth script that I could look at?

jjpeet · August 23, 2018, 10:23am

Hi,

I’ve now moved it to a Node-Red flow (a HASS.IO Addon) which I have attached below. Just copy this JSON to the IMPORT CLIPBOARD function within NODE-RED.

If you would prefer to use a bash script, I’ve included that below the JSON, and below that, I’ve included the sensor config from my HA YAML.

Basically I’m using the script/Node-Red flow to grab the tocken and put it into a secrets file - which then is referenced by the Sensor config YAML.

Hope that helps - give me a yell if you need a hand
JP

JSON for Node Red

[{"id":"3a50faf1.e07736","type":"tab","label":"AGL Data","disabled":false,"info":""},{"id":"30ab0e41.2627b2","type":"inject","z":"3a50faf1.e07736","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"00 01 * * *","once":false,"onceDelay":0.1,"x":146.5,"y":141,"wires":[["132e9ec8.6f54a1","f819c787.182738"]]},{"id":"7a8fe06e.c41a3","type":"www-request","z":"3a50faf1.e07736","name":"Get token","method":"POST","ret":"obj","url":"https://command.aglsolar.com.au/api/v2/Account/LoginUser","follow-redirects":true,"tls":"","x":547.5,"y":141,"wires":[["98ed0682.2bde3"]]},{"id":"132e9ec8.6f54a1","type":"function","z":"3a50faf1.e07736","name":"Inject Headers","func":"msg.headers = {\n \"Content-type\" : \"application/json\",\n \"Accept\": \"application/json, text/plain, */*\"\n };\nmsg.payload = {\n \"email\":\"[email protected]\",\n \"password\":\"Barraagld!\"\n};\nreturn msg;","outputs":1,"noerr":0,"x":345.5,"y":141,"wires":[["7a8fe06e.c41a3"]]},{"id":"dac950d4.88632","type":"file","z":"3a50faf1.e07736","name":"Create secrets.yaml","filename":"/config/sensors/secrets.yaml","appendNewline":false,"createDir":false,"overwriteFile":"true","x":668.5,"y":236,"wires":[[]]},{"id":"f02d371e.52471","type":"function","z":"3a50faf1.e07736","name":"Create Text File","func":"files = \"# Created \" + msg.payload.date + \"\\n\";\nfiles = files + \"agl_token: Bearer \" + msg.payload.access_token + \"\\n\";\nfiles = files + \"agl-graph-reference: https://command.aglsolar.com.au/api/v2/graph/3de94b39-2a3a-4a48-a302-44be6f5e9c44?endDate=\" + msg.payload.date + \"&granularity=Minute&metrics=read&units=W&startDate=\" + msg.payload.date + \"\\n\";\nmsg.payload = files;\nreturn msg;","outputs":1,"noerr":0,"x":419.5,"y":236,"wires":[["dac950d4.88632","3e2629b3.2140ee"]]},{"id":"98ed0682.2bde3","type":"function","z":"3a50faf1.e07736","name":"Date string","func":"today = new Date();\ndd = today.getDate();\nmm = today.getMonth()+1; //January is 0!\nyyyy = today.getFullYear();\n\nif(dd<10) {\n dd = '0'+dd\n} \n\nif(mm<10) {\n mm = '0'+mm\n} \n\ndateString = yyyy + '-' + mm + '-' + dd;\nmsg.payload.date = dateString.toString()\nreturn msg;","outputs":1,"noerr":0,"x":214.5,"y":236,"wires":[["f02d371e.52471"]]},{"id":"f819c787.182738","type":"file","z":"3a50faf1.e07736","name":"Delete secrets.yaml","filename":"/config/sensors/secrets.yaml","appendNewline":false,"createDir":false,"overwriteFile":"delete","x":344,"y":79,"wires":[[]]},{"id":"3e2629b3.2140ee","type":"api-call-service","z":"3a50faf1.e07736","name":"Restart HASS","server":"c299f3c.0d3871","service_domain":"hassio","service":"host_reboot","data":"","mergecontext":"","x":652.5,"y":290,"wires":[[]]},{"id":"c299f3c.0d3871","type":"server","z":"","name":"Home Assistant","url":"http://hassio/homeassistant","pass":"d7cf7e7eb88b4731bee6d35a6c296929"}]

Bash Script
This relies upon you loading the JQ JSON processor

#! /bin/bash
cd /home/james/.homeassistant
wget --header='Content-Type: application/json' \
--header='Accept: application/json, text/plain, */*' \
--post-data='{"email":"YOUR EMAIL ADDRESS","password":"YOUR PASSWORD"}' \
--no-check-certificate \
https://command.aglsolar.com.au/api/v2/Account/LoginUser
token=$(jq .access_token LoginUser)
token2=${token#'"'}
token3=${token2%'"'}
echo agl_token: Bearer $token3

reference1='https://command.aglsolar.com.au/api/v2/graph/3de94b39-2a3a-4a48-a302-44be6f5e9c44?endDate='
reference2='&amp;granularity=Minute&amp;metrics=read&amp;units=W&amp;startDate='
datestring="$(date +'%Y-%m-%d')"
cp secrets_template.yaml sensors/secrets.yaml
echo "# Token last updated: $(date +'%Y-%m-%d')" &gt;&gt; sensors/secrets.yaml
echo agl_token: Bearer $token3 &gt;&gt; sensors/secrets.yaml
echo agl-graph-reference: $reference1$datestring$reference2$datestring &gt;&gt; sensors/secrets.yaml
systemctl restart hass

rm LoginUser

The various sensors I setup based on the AGL API Data

   sensor:
    #AGL Energy Consumed (W)
      - platform: rest
        resource: https://command.aglsolar.com.au/api/v2/live_read/3de94b39-2a3a-4a48-a302-44be6f5e9c44?isUpdate=true&lastSuccessfulReadTimestampInUtc={{ utcnow().strftime("%Y%m%d%H%M") }}
        method: GET
        unit_of_measurement: 'W'
        value_template: '{{ value_json[0].energy_consumed|round }}'
        name: Power Use
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

    #AGL Energy Produced Statistics (W)
      - platform: statistics
        entity_id: sensor.power_use
        name: Power Use Stats
        max_age:
          minutes: 10
    #AGL Energy Produced Mean (W)
      - platform: template
        sensors:
          power_use_stats_median:
            entity_id: sensor.power_use_stats_mean
            # name: Power Use Median
            unit_of_measurement: "W"
            value_template: "{{ states.sensor.power_use_stats_mean.attributes.median | round }}"


          #AGL Energy Produced (W)
      - platform: rest
        resource: https://command.aglsolar.com.au/api/v2/live_read/3de94b39-2a3a-4a48-a302-44be6f5e9c44?isUpdate=true&lastSuccessfulReadTimestampInUtc={{ utcnow().strftime("%Y%m%d%H%M") }}
        method: GET
        unit_of_measurement: 'W'
        value_template: '{{ value_json[0].energy_generated|round }}'
        name: Solar
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

    # #AGL Power Produced (kWh)
      - platform: rest
        resource: !secret agl-graph-reference
        method: GET
        unit_of_measurement: 'kWh'
        value_template: '{{ (float(value_json.reads.energyGenerated)/1000) | round(1) }}'
        name: Solar Today
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

    #AGL Power Consumed (kWh)
      - platform: rest
        resource: !secret agl-graph-reference
        method: GET
        unit_of_measurement: 'kWh'
        value_template: '{{ (float(value_json.reads.energyConsumed)/1000) | round(1) }}'
        name: Power Use Today
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

    # #AGL Power Consumed (kWh)
      - platform: rest
        resource: !secret agl-graph-reference
        method: GET
        unit_of_measurement: 'kWh'
        value_template: '{{ (float(value_json.reads.energyExported)/1000) | round(1) }}'
        name: Power Exported Today
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

    # AGL Power Imported (kWh)
      - platform: rest
        resource: !secret agl-graph-reference
        method: GET
        unit_of_measurement: 'kWh'
        value_template: '{{ (float(value_json.reads.energyImported)/1000) | round(1) }}'
        name: Power Imported Today
        headers:
          Referer: https://command.aglsolar.com.au/
          Accept-Encoding: gzip, deflate, sdch, br
          Accept: application/json, text/plain, */*
          Cookie: _gat_UA-55822146-2=1; _ga=GA1.4.1412009925.1482291643
          Authorization: !secret agl_token

vlaraort · December 6, 2018, 11:19am

Hi @jjpeet, I am having the same issue as you, I need to do this:

Authorization: Bearer {{ states.sensor.my_token_sensor.state }}

Did you managed how to get it work?

jjpeet · December 6, 2018, 11:42am

Hi,
I never was able to make the authorisation line anything other than static data or point to a secrets fle. As per my previous comments, I used a bash script for a while (which refreshed the secrets file and did a restart for HASS to pick it up at 1am each day). Later, I moved to a Node-Red flow which did essentially the same thing but a bit more elegant and worked within Hass.io.

See script and node-red flow above. Let me know if you need more basic info.

JP

el-Salmon · January 30, 2019, 10:49am

It should be a solution that not need Node-Red The Oauth 2.0 protocol is very common nowdays for the REST APIs.

RaceLuke · January 30, 2019, 10:58am

I just opened a feature request for the same reason. I need it to integrate bticino smarther thermostat, that requires an authorization token that expires every hour (i can’t even think to restar hassio every hour, it would be a real trouble).

In my case is even worse because my authorization token is longer than 1000characters, so i need to use an attribute and not a state.

vlaraort · March 12, 2019, 6:13pm

I’m still in the same situation. @RaceLuke can you post a link of your issue so we can make more noise?

jjpeet · March 12, 2019, 7:14pm

I don’t use the AGL power sensor any more so don’t have the challenge of token refresh, but I’ve done a few other things over the year that would make me rethink doing what I posted above.

I think the simplest way to do this is the following;

Install Node-Red & MQTT. If you’re using Hass.io they are just add-ons. 1 minutes work.
Setup a Node-red flow - similar to the one above; but extend it to not only get and renew the token, but to also get the sensor data from the API.
Once the node-red flow has received the data, write it to an MQTT sensor that you’ve setup in Homeassistant.

If you use this approach, no reboots are required.

There are plenty of examples of config for an MQTT sensor, and you can use the beginnings of my node red flow above to get you going with the flow if you haven’t used it anymore.

Hope that helps.
JP

vlaraort · March 12, 2019, 11:12pm

Thank you, but by now I prefer to try to make this solution, as I am not running hassio, just plain home assistant, and install node-red can be a problem, also overloading my system, and adding another possible fail point, so I prefer to avoid it by now

icaballero · June 18, 2020, 1:00pm

I was trying to do the same thing discussed here. Instead of using “Bearer {{state…}}”, I was looking to the docs the rest sensor, and it should allow you using “resource_template”:

  - platform: rest
    name: Title
    resource_template: https://www.oneurl.com?access_token={{states.sensor.my_token.state}}
    method: GET
    value_template: '{{ value_json[0].data  }}'

I have manage to make it working.
Note: when you restart HA make sure to give some time to the sensor to have value before the rest does his job.