I guess alternatively you could invoke a wget / curl command from a nodered exec node and get the content that way. I suspect its something to do with the version of the http-parser in node, and the server you’re calling returning duff headers that cause the parser to choke.
What colour would you display for a black bin?
I guess whatever color you want to substitute?
I have yellow/red/green lidded bins so it works fine for me.
It’s a bugger for us we have green and blue, no problem.
But then we’ve got black and grey too …
I wonder where I can find an old multi-selector ??? Or maybe I should build my own… Hmmm.! I feel another project coming on.
Hi, That is an idea, I have a webserver running node red too. I could try on that. How did you get the data to show on the HA though ?
For Wakefield (UK) Bin Collections (copied mostly from previous posts)
In node red I used the following flow sending it via mqtt to the mosquitto broker in Home Assistant:-
Replace the URL in the http request with a valid URL:
[{"id":"b3b9112.7186e7","type":"tab","label":"Flow 4","disabled":false,"info":""},{"id":"48ddbf6c.583e58","type":"inject","z":"b3b9112.7186e7","name":"","topic":"","payload":"","payloadType":"date","repeat":"21600","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":40,"wires":[["4425f225.f6690c"]]},{"id":"c2f5a643.386ba","type":"debug","z":"b3b9112.7186e7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":450,"y":120,"wires":[]},{"id":"4425f225.f6690c","type":"http request","z":"b3b9112.7186e7","name":"","method":"GET","ret":"txt","url":"https://www.wakefield.gov.uk/site/Where-I-Live-Results?uprn=1234567","tls":"","x":170,"y":120,"wires":[["c2f5a643.386ba","671d937b.284aec","5c77a43b.21cc9c"]]},{"id":"671d937b.284aec","type":"html","z":"b3b9112.7186e7","name":"","property":"payload","outproperty":"payload","tag":"table[class=\"mb10 wilWasteContent gardenFutureData\"]","ret":"html","as":"single","x":490,"y":80,"wires":[["13fcb302.8626e5"]]},{"id":"13fcb302.8626e5","type":"debug","z":"b3b9112.7186e7","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":830,"y":80,"wires":[]},{"id":"5c77a43b.21cc9c","type":"html","z":"b3b9112.7186e7","name":"","property":"payload","outproperty":"payload","tag":"div[class=\"mb10 ind-waste-wrapper\"]","ret":"html","as":"single","x":410,"y":240,"wires":[["c9ec1ce5.516198","4f09f215.41d8fc","6ef6538e.bb042c","f3b60799.89885"]]},{"id":"c9ec1ce5.516198","type":"debug","z":"b3b9112.7186e7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":650,"y":420,"wires":[]},{"id":"4f09f215.41d8fc","type":"html","z":"b3b9112.7186e7","name":"","property":"payload[2]","outproperty":"payload","tag":"div","ret":"html","as":"single","x":650,"y":360,"wires":[["b70c77b8.826be","73e044.a322bfbc"]]},{"id":"b70c77b8.826be","type":"debug","z":"b3b9112.7186e7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload[6]","x":800,"y":400,"wires":[]},{"id":"6ef6538e.bb042c","type":"html","z":"b3b9112.7186e7","name":"","property":"payload[1]","outproperty":"payload","tag":"div","ret":"html","as":"single","x":650,"y":260,"wires":[["9c45239f.285f18","4cfeb625.c0efc"]]},{"id":"9c45239f.285f18","type":"debug","z":"b3b9112.7186e7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload[6]","x":820,"y":260,"wires":[]},{"id":"f3b60799.89885","type":"html","z":"b3b9112.7186e7","name":"","property":"payload[0]","outproperty":"payload","tag":"div","ret":"html","as":"single","x":650,"y":220,"wires":[["a63a00a1.759e1","f95e2b63.b5eb9"]]},{"id":"a63a00a1.759e1","type":"debug","z":"b3b9112.7186e7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload[6]","x":820,"y":160,"wires":[]},{"id":"e695a1c5.9861","type":"mqtt out","z":"b3b9112.7186e7","name":"","topic":"home/outside/recycling","qos":"2","retain":"true","broker":"6ef3cbf0.00da34","x":970,"y":300,"wires":[]},{"id":"6ee6ef90.bfeac","type":"mqtt out","z":"b3b9112.7186e7","name":"","topic":"home/outside/garden","qos":"2","retain":"true","broker":"6ef3cbf0.00da34","x":940,"y":360,"wires":[]},{"id":"a27dec0b.c5ee8","type":"mqtt out","z":"b3b9112.7186e7","name":"","topic":"home/outside/waste","qos":"2","retain":"true","broker":"6ef3cbf0.00da34","x":960,"y":200,"wires":[]},{"id":"f95e2b63.b5eb9","type":"function","z":"b3b9112.7186e7","name":"","func":"msg.payload = msg.payload[6];\nreturn msg;","outputs":1,"noerr":0,"x":790,"y":200,"wires":[["a27dec0b.c5ee8"]]},{"id":"4cfeb625.c0efc","type":"function","z":"b3b9112.7186e7","name":"","func":"msg.payload = msg.payload[6];\nreturn msg;","outputs":1,"noerr":0,"x":790,"y":300,"wires":[["e695a1c5.9861"]]},{"id":"73e044.a322bfbc","type":"function","z":"b3b9112.7186e7","name":"","func":"msg.payload = msg.payload[6];\nreturn msg;","outputs":1,"noerr":0,"x":770,"y":360,"wires":[["6ee6ef90.bfeac"]]},{"id":"6ef3cbf0.00da34","type":"mqtt-broker","z":"","name":"Hassio","broker":"192.168.0.100","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]
Then in home assistant in the sensor.yaml file I added the following:
- platform: mqtt
state_topic: 'home/outside/waste'
name: 'Waste'
- platform: mqtt
state_topic: 'home/outside/garden'
name: 'Garden'
- platform: mqtt
state_topic: 'home/outside/recycling'
name: 'Recycling'
2 Likes
Got Bolton working thanks to some help here: Help getting JSON into Restful / Template Sensor
Thanks guys!
1 Like
Okay, I have been inspired by some of the efforts on this thread and it appeals to the lazy B4574rd in me (okay doing this is more work but it’s fun)
Given that the colour light wouldn’t work for me I was thinking of building a rotary indicator.
This way whenever I’m in the kitchen I could just look up and see what the next collection was.
I could have it ‘between’ colours when not due and have it select the colour on the day before and on the day of collection.
I was looking for a mains powered solenoid to operate a ratchet that I’d cut into an MDF disc.
This seemed like a lot of time and effort so …
I was walking round ikea and they have some very cheap, very naff clocks
Hows about me buying one of these clocks, getting rid of the hour hand, painting the face into sections, getting a cheap phone charger (lots lying around) a buck converter, use a HA compatible power switch, and sticking them together.
we have a four week cycle (but you could adapt for 3 week or whatever)
I’d run the clock 7.5 min after the collection to take it to a border, then 7.5 mins the day before etc. You could run it at say 03:00 or something (no one should ever see it move !)
The switch brings the psu on, feeds the buck converter to supply 3v (2 off AAA’s equivalent)
And you have a steady state coloured indicator, no need to look at a screen. Wife friendly 
I could stick a more ‘visible’ pointer over the hand if it was too thin
What do you think ?
Any alternate suggestions ?
Love the creativity! Why not take the clock idea but setup a stepper motor? You could control the steps from a nodemcu
I just get my amazon echo to announce every hour from 6pm the day before what bin needs to be put out
1 Like
Whatever works best for you, but a simple switch should be within reach of everyone, a stepper motor may be a bit advanced AND you won’t get a stepper motor for $10
That would bug me let alone the wife who would kick me out of the house 
1 Like
Hi all, trying to get something configured on my Home Assistant, but pretty new at all this so I’m not sure what is required.
Can anyone have a look at https://www.n-kesteven.org.uk/bins/display?uprn=10006503138 (fake address) to see if the data can be scraped easily, and how I would go about this. We have FOUR bins collected on various dates.
They also show bin events, for example, bin emptied, or bin not out etc would be nice to integrate this into the card as well?
My Python is awful - but try this as a Python script
#!/usr/bin/env python3
from urllib.request import Request, urlopen
import json
from bs4 import BeautifulSoup
user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)'
headers = {'User-Agent': user_agent}
req = Request('https://www.n-kesteven.org.uk/bins/display?uprn=10006503138')
req.add_header('User-Agent', user_agent)
fp = urlopen(req).read()
page = fp.decode("utf8")
soup = BeautifulSoup(page, features="html.parser")
soup.prettify()
data = {}
for bins in soup.findAll("div", {"class" : lambda L: L and L.startswith('bg-')}):
binType = bins.h3.text
binCollection = bins.find_all("strong")[-1].text
data[f'{binType}'] = f'{binCollection}'
json_data = json.dumps(data)
print(json_data)
It produces the following ouput
{"Black (Domestic)": "Next Collection: 9 days from now on Tuesday, 14 January 2020", "Green (Recycling)": "Next Collection: 16 days from now on Tuesday, 21 January 2020", "Purple (Paper/Card)": "Next Collection: 2 days from now on Tuesday, 7 January 2020", "Brown (Garden Waste)": "Next Collection: 26 days from now on Friday, 31 January 2020"}
You should run this to produce a file - eg
get_bins.py > binInfo.json
Then use a file sensor in HA to parse the resulting JSON
- platform: file
file_path: /home/<youruser>/scripts/binInfo.json
name: Black Bin
value_template: '{{ value_json["Black (Domestic)"] }}'
1 Like
Hi Rob, Looks great and I can confirm that the py script works perfectly when ran outside Home Assistant. One issue though, I have never used Python scripts before inside of Home Assistant, so I’m not sure how to run the file?? Any help appreciated! I am running Hass.io HassOS 2.12 on a Raspberry Pi
Ah ok - I’m not a user of hass.io
Are you able to ssh into the raspberry pi and run from there? If so I’d be setting up a cronjob weekly to create the output file and let HA parse that
Ermm I don’t think you can SSH into Hass.io, Think I’ve been there before unfortunately, but will have to read up on the documentation again!
Is there a way to use scrape to achieve the same thing as the .py script does?
Potentially - but your looking at 4 scrapes and a lot of complexity.
Or your looking at one scrape and a ton of templating (Im not even sure if its possible this way)
The link I posted mentions how to have a script run from a trigger - you could potentially also have a shell command that runs the py script - common location for the output and then the file sensor approach
I think I can just call the service as an automation to run every day to reduce the load but when triggering manually, I get this error show up in the logs.
Looks like I may need to somehow install the dependencies maybe? Or should I be calling the service with some service data?
0-01-05 17:40:39 WARNING (SyncWorker_13) [homeassistant.components.python_script] Warning loading script get_bins.py: Line None: Prints, but never reads ‘printed’ variable.
2020-01-05 17:40:39 ERROR (SyncWorker_13) [homeassistant.components.python_script.get_bins.py] Error executing script: import not found
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/python_script/init.py”, line 194, in execute
exec(compiled.code, restricted_globals, local)
File “get_bins.py”, line 2, in
ImportError: import not found