That would bug me let alone the wife who would kick me out of the house 
1 Like
Hi all, trying to get something configured on my Home Assistant, but pretty new at all this so I’m not sure what is required.
Can anyone have a look at https://www.n-kesteven.org.uk/bins/display?uprn=10006503138 (fake address) to see if the data can be scraped easily, and how I would go about this. We have FOUR bins collected on various dates.
They also show bin events, for example, bin emptied, or bin not out etc would be nice to integrate this into the card as well?
My Python is awful - but try this as a Python script
#!/usr/bin/env python3
from urllib.request import Request, urlopen
import json
from bs4 import BeautifulSoup
user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)'
headers = {'User-Agent': user_agent}
req = Request('https://www.n-kesteven.org.uk/bins/display?uprn=10006503138')
req.add_header('User-Agent', user_agent)
fp = urlopen(req).read()
page = fp.decode("utf8")
soup = BeautifulSoup(page, features="html.parser")
soup.prettify()
data = {}
for bins in soup.findAll("div", {"class" : lambda L: L and L.startswith('bg-')}):
binType = bins.h3.text
binCollection = bins.find_all("strong")[-1].text
data[f'{binType}'] = f'{binCollection}'
json_data = json.dumps(data)
print(json_data)
It produces the following ouput
{"Black (Domestic)": "Next Collection: 9 days from now on Tuesday, 14 January 2020", "Green (Recycling)": "Next Collection: 16 days from now on Tuesday, 21 January 2020", "Purple (Paper/Card)": "Next Collection: 2 days from now on Tuesday, 7 January 2020", "Brown (Garden Waste)": "Next Collection: 26 days from now on Friday, 31 January 2020"}
You should run this to produce a file - eg
get_bins.py > binInfo.json
Then use a file sensor in HA to parse the resulting JSON
- platform: file
file_path: /home/<youruser>/scripts/binInfo.json
name: Black Bin
value_template: '{{ value_json["Black (Domestic)"] }}'
1 Like
Hi Rob, Looks great and I can confirm that the py script works perfectly when ran outside Home Assistant. One issue though, I have never used Python scripts before inside of Home Assistant, so I’m not sure how to run the file?? Any help appreciated! I am running Hass.io HassOS 2.12 on a Raspberry Pi
Ah ok - I’m not a user of hass.io
Are you able to ssh into the raspberry pi and run from there? If so I’d be setting up a cronjob weekly to create the output file and let HA parse that
Ermm I don’t think you can SSH into Hass.io, Think I’ve been there before unfortunately, but will have to read up on the documentation again!
Is there a way to use scrape to achieve the same thing as the .py script does?
Potentially - but your looking at 4 scrapes and a lot of complexity.
Or your looking at one scrape and a ton of templating (Im not even sure if its possible this way)
The link I posted mentions how to have a script run from a trigger - you could potentially also have a shell command that runs the py script - common location for the output and then the file sensor approach
I think I can just call the service as an automation to run every day to reduce the load but when triggering manually, I get this error show up in the logs.
Looks like I may need to somehow install the dependencies maybe? Or should I be calling the service with some service data?
0-01-05 17:40:39 WARNING (SyncWorker_13) [homeassistant.components.python_script] Warning loading script get_bins.py: Line None: Prints, but never reads ‘printed’ variable.
2020-01-05 17:40:39 ERROR (SyncWorker_13) [homeassistant.components.python_script.get_bins.py] Error executing script: import not found
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/python_script/init.py”, line 194, in execute
exec(compiled.code, restricted_globals, local)
File “get_bins.py”, line 2, in
ImportError: import not found
What does your automation look like?
Also check https://www.home-assistant.io/integrations/python_script/
action:
- service: python_script.getBinData
1 Like
So an Update on the Waste Collection Indicator I discussed implementing further up the thread.
We have weekly collections and they rotate on a 4 week basis (you may need to adjust to suit yours)
So we have : -
- Blue - Paper/Cardboard
- Grey - General Waste
- Black - Glass/Steel/Aluminium/Plastic
- Grey - General Waste
Green Waste (for composting) is collected every week (so the input select also shows green)
Then the cycle starts again. I just have an input select with the options above and use input_select.select_next triggered once a week
But the wall display : -
which is powered by : -
And the whole action is : -
So if it’s on the border before, then the next collection is … but not imminent, when it goes into the segment - “Pull Your Finger Out And Get The Bins Ready”
I bought a clock (£8 from Ikea) with a second hand (this was important to calibrate the unit).
I switch the switch on and it takes 1 sec or so for the voltage to sit up. I then switch off and the power supplies run on for about another 5 sec. The PSU uses a gutted z wave switch, feeding a 250vac to 5vdc (fixed) buck converter, feeding another buck converter to bring the voltage down to a single AAA cell voltage. (couldn’t get a 240vac to 1.6vdc direct and if I’d built one myself it would have been a LOT bigger).
So I pointed the virgin clock to 12 (ALL hands) then I ran the clock for 7.5 minutes, every 10 minutes for a week (I ran a counter too, to make sure how many times it had run)
So 6 times an hour, 24 hours a day for 7 days = 1008 increments
Divide by 8 is the number of revolutions = 126 hours so it “should” have returned to hour hand on 6 and both minute and second hands on 12 - It didn’t.
So how many minutes and seconds off was it ? Then divide that number of seconds by 1008 and you get a single cycle offset. mine wasn’t exact but I can’t do a 4.2 second offset (maybe you MQTT guys can) So my cycle is 7.5 mins minus an offset of 4 secs (yours may vary).
Here Is my code : -
sensor:
- platform: template
sensors:
bins_to_collect:
friendly_name: Bins To Collect
entity_id: input_select.is_bin_day_bins
value_template: "{{ states('input_select.is_bin_day_bins') }}"
icon_template: "{{ 'mdi:trash-can-outline' }}"
bin_day:
friendly_name: Bin Collection Day
entity_id: input_select.is_bin_day
value_template: "{{ states('input_select.is_bin_day') }}"
icon_template: "{{ 'mdi:trash-can-outline' }}"
input_number:
in_switch_bin_clock_run:
name: Bin Run Timer (mins)
#initial: 7.5
min: 0.5
max: 60
step: 0.5
mode: box
icon: mdi:trash-can-outline
input_datetime:
id_switch_bin_clock_on_time:
name: Bin Clock Update Time
has_time: true
#initial: "03:30:00"
icon: mdi:clock-start
input_select:
## when bin to be collected
is_bin_day:
name: Bin Collection Day
options:
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
- Sun
icon: mdi:trash-can-outline
#initial: Mon
## when bin notification is indicated
is_bin_day_notification:
name: Bin Day Notification
options:
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
- Sun
icon: mdi:trash-can-outline
#initial: Sun
## when bin notification is reset
is_bin_day_reset:
name: Bin Day Reset
options:
- Mon
- Tue
- Wed
- Thu
- Fri
- Sat
- Sun
icon: mdi:trash-can-outline
#initial: Tue
## when bin notification is reset
is_bin_day_bins:
name: Bin Day Reset
options:
- Grey & Green
- Blue & Green
- Grey & Green
- Black & Green
icon: mdi:trash-can-outline
automation:
## name: Switch Bin Clock Trigger
- alias: au_switch_bin_trigger
trigger:
- platform: template
value_template: "{{ states('sensor.time') == states('input_datetime.id_switch_bin_clock_on_time') [0:5] }}"
condition:
- condition: template
value_template: >
{% set today = now().strftime('%a') %}
{{ today == states('input_select.is_bin_day_notification') or today == states('input_select.is_bin_day_reset') }}
action:
- service: script.sc_switch_bin_clock_timer
- condition: template
value_template: "{{ now().strftime('%a') == states('input_select.is_bin_day_reset') }}"
- service: input_select.select_next
entity_id: input_select.is_bin_day_bins
script:
sc_switch_bin_clock_timer:
alias: Switch Bin Clock Timer Script
sequence:
- service: switch.turn_on
entity_id: switch.switch_fibaro_bin_clock
- delay: "00:00:{{ (((states('input_number.in_switch_bin_clock_run') | float) *60) -4) | int }}"
- service: switch.turn_off
entity_id: switch.switch_fibaro_bin_clock
This is a ‘package’ file.
I have yaml for lovelace too. The first is just the display, the second is for the maintenance/adjustments.
This clock should run 13 hours per year so I’m not worried about it drifting too much though I won’t object if I have to get up and adjust the clock 1/yr.
## Bin Collection Status
- type: entities
title: Bin Collection Status
show_header_toggle: false
entities:
- entity: sensor.bin_day
- entity: sensor.bins_to_collect
## Bin Collection Settings
- type: entities
title: Bin Collection Settings
show_header_toggle: false
entities:
- entity: input_select.is_bin_day
- entity: input_select.is_bin_day_bins
- entity: input_select.is_bin_day_notification
- entity: input_select.is_bin_day_reset
- entity: input_datetime.id_switch_bin_clock_on_time
- entity: input_number.in_switch_bin_clock_run
What does bug me (so beware) I cut the face from a grey floor tile (save on paint) I drew the circle, cut the circle and measured exactly to ensure that I had equal quarters and drilling the hole in the centre.
What I didn’t allow for was gluing the face onto the clock.
It moved a bit so it’s not bang in the centre, so though I lined up the 12 O’Clock position the others are a ‘bit’ off - but all is right at 12 next time (it just bugs me !)
My bins are collected on a Monday So I have the bin status updated at 02:00 on a Tuesday - Then It moves to ACTION on a Friday so I have the weekend to do the bins.
Choose to suit yourselves.
Requirements : -
- Access to eBay (clock, buck converters, wire, box, heat shrink (or just a lot of insulation tape) )
- Tools (scissors, soldering iron, screwdriver)
- An HA controllable switch (I used a z wave, Fibaro smart switch, but virtually any type would do)
4 Likes
Wow that’s amazing engineering- well done!
Product Name : B-indicator ?
Thanks, not sure I’d call it engineering though (I am an engineer) it’s more a detailed guide for any semi-serious hobbyist.
Part of my use for HA is to reduce my energy bills (and thus reduce my damage on the environment) I don’t think in my wildest dreams I could justify this project under those terms but its a damn sight less damaging than a bulb indicator would be 
Hey @KablammoNick,
I am also in Australia and our local council collection is pretty simple (garbage every week, alternate weeks for Green Waste and Recyclables) - This does not change with public holidays etc.
I just used your sample and changed it slightly:
- platform: template
sensors:
binday:
friendly_name: Next Bin Day
value_template: '{% if (((now().strftime("%W") | int) % 2) == 0)%}Green{%else%}Yellow{%endif%}'
Then my card is a Picture Entity Card.
entity: sensor.binday
state_image:
Green: /local/bins/green.png
Yellow: /local/bins/yellow.png
type: picture-entity
which looks like:
I haven’t ran it for a week yet to see that it all changes correctly but i cant really see why it wouldn’t, the only thing you may need to consider is adding some days to when sensor changes state. reading the example provided it should work fine for me as my bin day is Monday and the python week starts Monday
Cheers,
Aaron
1 Like
@afalzon nice! It’s been a while since I implemented that, and I reckon Lovelace will make it a lot easier and tidier now - think I ended up with like three sensors and a template to get it working with a little image/icon.
1 Like
Hello all, just found this project yesterday and have been all over it like a bad rash! Thanks OP for starting it and all the posts, and also everybody else for your valuable input.
I’ve got the scrape working but unfortunately it’s 1 scrape per bin (so that’s 7 hits to the website). There’s also a “following collection date” so that makes an update 14 hits using scrape 
I’m trying to get the scripting working (as per OP’s post), however after parsing the JSON, I have the below, which results in the error log stating: "Invalid state encountered for entity id: sensor.bincollectiontype. State max length is 255 characters."
“Type of Collection”
“Last collection date”
“Next Collection Date”
“Following collection date”
“Black Bin”
“Saturday 11/01/2020”
“Friday 24/01/2020”
“Friday 07/02/2020”
“Brown Bin”
“Friday 17/01/2020”
“Friday 31/01/2020”
“Friday 14/02/2020”
“Card Sack”
“Saturday 11/01/2020”
“Friday 24/01/2020”
“Friday 07/02/2020”
“Food Waste External Bin”
“Friday 17/01/2020”
“Friday 24/01/2020”
“Friday 31/01/2020”
“Green Box”
“Saturday 11/01/2020”
“Friday 24/01/2020”
“Friday 07/02/2020”
“Paper Sack”
“Friday 17/01/2020”
“Friday 31/01/2020”
“Friday 14/02/2020”
“Plastic Sacks”
“Friday 17/01/2020”
“Friday 31/01/2020”
“Friday 14/02/2020”
Any ideas how to get around this?
I don’t need “Last collection date” but it’s just there in the pulled data - and even removing last week’s collections still exceeds the 255 character limit.
Thanks
1 Like
I’m a doofus, I forgot about the ‘grep’ after the parsing of the json… still the “grep date -d 'Friday' '+%d-%b-%Y' -A1” returns empty. Urrgh
Alright just went with grep “bin” -A3 and will have to play with the Sacks later.
Feels like .py would be easier but I’m a total nub with that
If you send me the url - I’ll do you a python beautiful soup version
One scrape to produce a json file then use the rest sensor to hit your local file.
Similar to : Bin / Waste Collection
Are you practicing your python? I totally need to get into that a lot more.
Here’s a sample house: https://mychelmsford.secure.force.com/WasteServices/WM_WasteViewProperty?id=a02240000054H2rAAE
Current command_line for parseBinCollection.sh below, if any of it is recyclable:
cat /config/scripts/binCollection.json | jq ‘…|.“text”? | select (. !=null)’ | tr -d ‘"’ | grep Bin -A3
Thanks