I know, but it is also good to be cynical sometimes!
1 Like
I’m as salty as they come, for me this was also a little bit about learning how to do stuff with hassio other than preinstalled plugins. I figured that most councils in Australia are similar (week about for the green/recycling) so it could easily be shared once done.
As it turns out, I also asked on the Discord and mihalski came up with a solution:
sensor:
- platform: template
sensors:
binday:
value_template: '{% if (((now().strftime("%W") | int) % 2) == 0)%}Even{%else%}Odd{%endif%}'
Just testing it now…
1 Like
My wife calls me a grumpy old man 
1 Like
So I tried to write something similar, turns out my local council’s waste collection page is nowhere near as easy to scrape data from. After some investigation with my browser’s dev tools, it looks like the API calls that it makes under the covers are very arcane and seem to rely on unique IDs generated at runtime in order to return any data. On top of that the whole page uses AJAX so it’s not like I can link directly to a result page for my address.
Our government makes a lot of noise about providing open, easily accessible data and APIs for public services, it’s just a shame that none of it is standardized and each council seems to have its own implementation!
Hi @Robbrad, you helped me with this on another thread curl 'http://ci.draftserver.com/penrith/webservice/location/search' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' -H 'Origin: http://ci.draftserver.com' -H 'Upgrade-Insecure-Requests: 1' -H 'Content-Type: application/x-www-form-urlencoded' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://ci.draftserver.com/penrith/webservice/location/search?st_number=38&suburb=penrith&address=mulgoa+road&offset=' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' --data 'st_number=38&suburb=penrith&address=mulgoa+road&offset=' --compressed
which gives me {"code":"0001","msg":"Success","offset":1000,"total":"1","results":[{"unit_number":"0","house_number":"38","street":"Mulgoa","street_type":"Road","suburb":"PENRITH","postcode":"2750","collection_day":"TuesA2","bin_type":"3","dwelling":"single","date_time":"0000-00-00 00:00:00","current_week_type":"EVEN","bin_colors":"1,2,3","collection_frequencies":"2,1,2","area":"2","item_garbage":"2018-06-26","item_recycling":"2018-06-19","item_organic":"2018-06-19"}]}
What’s the best way to get this running and import the data into HA?
I did a bit of work on this last night. Couldn’t get any API calls or a screen scrape from my council’s website to work, so I went with something a bit simpler.
To calculate the next collection date:
import datetime
import holidays
import json
today = datetime.date.today()
# Normal collection day is Thursday, day 3 of a zero-indexed week.
if today.weekday() > 3:
# If this Thursday has passed, we only care about next week.
today = today + datetime.timedelta((0 - today.weekday()) % 7)
this_week = dates = [today + datetime.timedelta(days=i) for i in range(0 - today.weekday(), 7 - today.weekday())]
uk_holidays = holidays.CountryHoliday('UK')
is_bank_holiday = False
for weekday in this_week:
if weekday in uk_holidays:
is_bank_holiday = True
# Set the collection date to next Thursday.
next_collection = today + datetime.timedelta((3 - today.weekday()) % 7)
if is_bank_holiday:
# Rubbish is collected a day later on weeks with a bank holiday.
next_collection = next_collection + timedelta(days=1)
print(next_collection)
And then, to work out what the next type of collection is:
import datetime
today = datetime.date.today()
if today.weekday() > 3:
# If this Thursday has passed, we only care about next week.
today = today + datetime.timedelta((0 - today.weekday()) % 7)
# Get the ISO week number (1-52~53)
week_number = today.isocalendar()[1]
if (week_number % 2) == 0:
# Even weeks are for Landfill waste
collection_type = 'Landfill'
else:
# Odd weeks are for Recycling and Garden Waste.
collection_type = 'Recycling & Garden Waste'
print(collection_type)
Finally, hook up some sensors:
- platform: command_line
name: Next Bin Collection
command: "python3 /config/Scripts/waste_collection_date.py"
- platform: command_line
name: Bin Collection Type
command: "python3 /config/Scripts/waste_collection_type.py"
And voila!
7 Likes
Could you try using the https://www.home-assistant.io/components/sensor.rest/
This sensor can read your data - (JSON)
Thanks @Robbrad, I finally got something working it’s a bit of a messy solution but it’s natively getting the data so I can eventually remove the google calendar.
I’m using hass.io so I tried to use the commandline tool to run the curl script, but that didn’t work so in the end I converted the curl string to a python script. This runs on a Windows machine running python and apache. The rest sensor then reads the data and imports to hass.io.
Any suggestions on how I could just run the python script in hass.io and the rest sensor reading it from there?
For now I can display the information but no automation. Which is the next new thing I’m trying to work out. Currently I’m using google calendar which I have entries for eg: 2018-07-09 00:00:00, I then compare this with the current date time but I trim the time and the output will be the current date with 00:00:00 as the time (as_timestamp / 86400).
Is there an easier way to just compare the date and get the value? 2018-07-09 - 2018-07-03? Otherwise is it possible to combine the date that I have from my sensor and attach 00:00:00 to it?
Have a look how I achieved it just using bash. See my original post above.
My script just returns a value based on parsing the JSON
Should be no need for a python script.
Finally! After a few attempts I have the rest sensor natively doing the work. Thanks @Robbrad and @lolouk44 for sending me over to this thread and the help 
Now I need to rework my date calculation and I should be able to do the automation, will be using LIFX to indicate the bin colour that I need to take out the day before bin day.
4 Likes
Nice work man
That is a great idea!!
1 Like
Yeah, I’m stuck again. Lol. Need to learn how to join the date that I get (yyyy-mm-dd) from the JSON response/value/attribute with time 00:00:00 as this is the only way I can calculate two dates and just fake the time of both being 00:00:00. Also noticed that the new collection date doesn’t rollover once this weeks collection is done.
Is it possible to just compare/subtract two dates with value being this? yyyy-mm-dd instead of having to use yyyy-mm-dd 00:00:00? Or how I can combine yyyy-mm-dd with 00:00:00?
Dates are stored internally as seconds from the epoch, so you should be able to.
Thanks buddy. Works a treat!!! Sorry it’s taken a while for me to reply. I’ve had a few family issues happen latley, so been busy sorting them out, and only just looked at this tonight.
The output is all the code for the page, so I’ll need to save that to a file and scrape out just the info that I need or work out a way to save just what I need.
1 Like
My local council is nice enough to provide PDFs for east and west sides broken down day by day.
Similar to mentioned above, you have to enter your postcode and house name/number to get the right PDF.
My thinking is of scraping the page somehow and OCRing the PDF to get the relevant days and bins … weather that works or not is a different matter 
1 Like
Can you share what you have done already - e.g. what you have to date
Currently I’ve reverted back to using my google calendar with 3 repeating entries for my green, yellow and red bin. They get imported into home assistant. I then configured a template sensor to calculate the days until bin collection. I then use this in my automation to determine which bins need to go out the day before. Notification is done via LIFX bulb.
An issue I’ve found is my template sensors dont seem to update unless home assistant is restarted. The code works fine and updates when I test it in the developer/templates section of home assistant.
I’ll post the code I have in place once I have access to my hass.
Using the rest sensor and getting the data from the council servers I’ve discovered that they don’t update the information until the actual collection day.
Example:
- item_garbage: 2018-07-09
- item_organic: 2018-07-09
- item_recycling: 2018-07-16
Organic is weekly so I could ignore the item_organic date and just use item_garbage and item_recycling to determine what bin is due the following week. With that in mind I can only get 2018-07-16.
To my knowledge the only to calculate two dates is by converting it to timestamp and for that I need YYYY-MM-DD 00:00:00.
So how would I use something like states.sensor.penrith_waste_info.attributes.results.0.item_recycling and combine it with 00:00:00. So I can then use that within as_timestamp(2018-07-16 00:00:00) / 86400. There’s obviously more to that, but that’s all I remember without looking at my configuration.yaml.
Option 1: Stick with my google calendar, but I need to resolve why hass is not updating the template sensor.
Option 2: Use the info from the council so I don’t need to use the google calendar, but I need to figure out how to process the data and also not sure if I’ll still have an issue with the template sensor.
I would simplify where possible, you have a rest sensor that gets data / present that sensor - anything where logic is required then opt for a different solution - I did mine using bash which worked out the next thus date and grep’ed for that in the JSON (that filtered the day I needed)
Your saying the council only give the info you need on the day of collection - there is no further in the future information? then its going to make the logic difficult - do we have the right url here to cURL - I noticed there is an offset parameter.
If you were doing this from a PC manually to find out the bins - which site would you go to to get the data?