I had a feeling it might not work because the value in x would no longer be understood to be a JSON list but an ordinary string.
Thanks for the URL. I’ll experiment with it later and see if I can get some useful results.
It’s me who is thanking 
I created this REST Sensor:
- platform: rest
name: "Dagrenovation"
resource: https://herlev.renoweb.dk/Legacy/JService.asmx/GetAffaldsplanMateriel_mitAffald
method: POST
payload: '{"adrid": 24750,"common": false}'
value_template: >-
{{value | truncate(250)}}
Here’s the value it reports:
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
<meta charset="UTF-8">
<title>RenoWeb offline</title>
<!-- Bootstrap -->
<link rel="stylesheet"...
That’s definitely not in JSON format. Can you confirm the URL is correct? Or is the service unavailable?
ohh i am sorry, i forgot to add above in the example, if it doesn’t have the Content-Type it will throw the error you see. thank you for trying to help me
headers:
Content-Type: application/json
Well now, that was an educational experience ending in failure. 
I managed to produce a template that corrects the invalid JSON lists received from the website. You can’t store more than 255 characters in the sensor’s state, so I just took the first 125 characters and the last 125 characters. The point was just to confirm that the beginning and the end of the JSON list was now properly formatted.
- platform: rest
name: "Dagrenovation"
resource: https://herlev.renoweb.dk/Legacy/JService.asmx/GetAffaldsplanMateriel_mitAffald
method: POST
payload: '{"adrid": 24750,"common": false}'
headers:
Content-Type: application/json
value_template: >-
{% set x = ((value.replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','') %}
{{x[:125]}} .. {{x[-125:]}}
The result is this:
{"d":{"list":[{"id":27445,"materielnavn":"140 l beholder - ugetømning (1 stk.)","ordningnavn":"Dagrenovation","toemningsdage" .. "mattypeid":390,"antal":1,"vejnavn":"Agerledet","modulsort":3,"adrid":24750,"fractionid":7,"viskalender":true,"modulId":3}]}}
Both ends have been purged of the extra double-quotes and there are no \ characters. Looks good!
Now for the real test. I modified value_template to display the value in ordningnavn.
value_template: >-
{% set x = ((value.replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','') %}
{{ x['d']['list'][0]['ordningnavn'] }}
The result is unknown.
The reason is because the template produced a nice clean string in JSON format. What it didn’t produce is a JSON list. Therefore this approach, attempting to fix the broken JSON list, doesn’t work.
The second approach is to simply use regex_findall_index to search through the string and find all instances of ordningnavn and report their values.
Yeah that’s back to square one 
I am also looking whether python scripts can be used for this but it doesn’t seem so I need to go to AppDaemon but that seems overkill.
You could use a Command Line Sensor instead and have it use an external script to process the data and return what you want as a proper JSON list.
The external script can use whatever is best to get the job done (shell script, python, etc). A combination of curl and grep might work but you’d probably have more flexibility with python.
Yes i was looking at that, i just can’t seem to find an example how to set a value of the sensor with output from the python script file. I will search further, thank you for trying to help.
It needs to write to ‘standard output’. In other words, if you are at the Linux command line and execute your script, it’s output should appear on the command line (‘standard output’).
Here’s an example of a command line sensor calling a python script:
Here’s an example using shell script but it’s the same principal.
Ok so i have made this :
#!/usr/bin/python2
import requests
import json
from datetime import datetime
import locale
import sys
sensor = sys.argv[1]
body = {"adrid": 24750,"common": False}
headers = {'content-type': 'application/json'}
url = 'https://herlev.renoweb.dk/Legacy/JService.asmx/GetAffaldsplanMateriel_mitAffald'
response = requests.post(url,data=json.dumps(body),headers=headers)
cleaned = ((response.content.replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','')
y = json.loads(cleaned)
locale.setlocale(locale.LC_ALL, 'da_DK.UTF-8')
if sensor == "Dagrenovation":
datetime_object = datetime.strptime(y['d']['list'][0]['toemningsdato'], '%A den %d-%m-%Y')
print(datetime_object.date())
elif sensor == "MadamSkrald":
datetime_object = datetime.strptime(y['d']['list'][2]['toemningsdato'], '%A den %d-%m-%Y')
print(datetime_object.date())
elif sensor == "Storskrald":
datetime_object = datetime.strptime(y['d']['list'][3]['toemningsdato'], '%A den %d-%m-%Y')
print(datetime_object.date())
elif sensor == "Haveaffald":
datetime_object = datetime.strptime(y['d']['list'][5]['toemningsdato'], '%A den %d-%m-%Y')
print(datetime_object.date())
else:
print("unknown")
Now i have to test it in HASS to see if it works.
of course it didn’t work… 
Log Details (ERROR)
Sat May 04 2019 22:26:43 GMT+0200 (Central European Summer Time)
Command failed: python3 /config/herlev_affald.py 'Haveaffald'
and the sensor is unknown so for some reason the argument is not passed correctly
I think that message means Home Assistant’s was either unable to find or to execute the python file.
Are you using hass.io?
Yes i am, it can find the the script. i chmod 755. I made another script with just print("hello") and that worked so i started to add code until it failed and as far as i can see as soon as i try to use response.content the script fails but if i do print(response) i get 200 so i don’t know what’s going on
When I run this from the command-line:
python3 ./herlev_affald.py 'Haveaffald'
I get this response:
Traceback (most recent call last):
File "./herlev_affald.py", line 14, in <module>
cleaned = ((response.content.replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','')
TypeError: a bytes-like object is required, not 'str'
Ouch…i only tested with python 2.7…hmmm i guess then it’s python 3 issue, well spotted, let me upgrade on my mac python to ver 3
So i changed the line, it works locally with python3 but still doesn’t work in HASS
cleaned = ((response.content.decode('utf-8').replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','')
I’ve encountered a hurdle. I have trouble making the script work for me because my locale is not Danish. If I comment out the line that sets the locale, strptime fails to perform the date conversion. In my locale (English Canada), %A and Fredag are incompatible (no surprise there).
All I want to do is try running the script from within Home Assistant to see if I get the same error message as you. Later I will try simplifying the python script just enough so it works for my locale.
Yes you are on to something, i also nailed it down to
datetime_object = datetime.strptime(y['d']['list'][5]['toemningsdato'], '%A den %d-%m-%Y')
maybe because the locale i set isn’t working so strptime is failing…i am too tired now so i can’t concentrate anymore. I will hit the bed…damn it…
I modified the code and this works for me. I now know on which days you can put out garden waste, refuse, bulk items, etc. 
#!/usr/bin/python3
import requests
import json
from datetime import datetime
import sys
if len(sys.argv) == 2:
days = {'Mandag':'Monday', 'Tirsdag':'Tuesday', 'Onsdag':'Wednesday', 'Torsdag':'Thursday', 'Fredag':'Friday'}
types = {'Dagrenovation':0, 'MadamSkrald':2, 'Storskrald':3, 'Haveaffald':5 }
body = {'adrid': 24750,'common': False}
headers = {'content-type': 'application/json'}
url = 'https://herlev.renoweb.dk/Legacy/JService.asmx/GetAffaldsplanMateriel_mitAffald'
response = requests.post(url,data=json.dumps(body),headers=headers)
cleaned = ((response.content.decode('utf-8').replace('"d":"{', '"d":{')).replace('}]}"', '}]}')).replace('\\','')
j = json.loads(cleaned)
t = types.get(sys.argv[1])
if t != None:
d = j['d']['list'][t]['toemningsdato']
for k, v in days.items():
d = d.replace(k, v)
datetime_object = datetime.strptime(d, '%A den %d-%m-%Y')
print(datetime_object.date())
else:
print('unknown')
else:
print('error')
Sensor definition (my python file is located in a sub-directory):
- platform: command_line
name: garden waste
command: "python3 /config/python_scripts/test.py Haveaffald"
1 Like
@123 Thank you very much, indeed this a solution which solves the problem and works. Weird the one with setting the locale didn’t work. Now i will continue to make a binary sensor that can check the date towards today so i can make an automation to send notification.
Thanks again for your effort
1 Like