Bin / Waste Collection

q00dee · July 3, 2018, 12:47am

Thanks @Robbrad, I finally got something working it’s a bit of a messy solution but it’s natively getting the data so I can eventually remove the google calendar.

I’m using hass.io so I tried to use the commandline tool to run the curl script, but that didn’t work so in the end I converted the curl string to a python script. This runs on a Windows machine running python and apache. The rest sensor then reads the data and imports to hass.io.

Any suggestions on how I could just run the python script in hass.io and the rest sensor reading it from there?

For now I can display the information but no automation. Which is the next new thing I’m trying to work out. Currently I’m using google calendar which I have entries for eg: 2018-07-09 00:00:00, I then compare this with the current date time but I trim the time and the output will be the current date with 00:00:00 as the time (as_timestamp / 86400).

Is there an easier way to just compare the date and get the value? 2018-07-09 - 2018-07-03? Otherwise is it possible to combine the date that I have from my sensor and attach 00:00:00 to it?

Robbrad · July 3, 2018, 11:00am

Have a look how I achieved it just using bash. See my original post above.

My script just returns a value based on parsing the JSON

Should be no need for a python script.

q00dee · July 3, 2018, 2:19pm

Finally! After a few attempts I have the rest sensor natively doing the work. Thanks @Robbrad and @lolouk44 for sending me over to this thread and the help

Now I need to rework my date calculation and I should be able to do the automation, will be using LIFX to indicate the bin colour that I need to take out the day before bin day.

Robbrad · July 3, 2018, 2:43pm

Nice work man

nickrout · July 3, 2018, 8:19pm

That is a great idea!!

q00dee · July 4, 2018, 1:51am

Yeah, I’m stuck again. Lol. Need to learn how to join the date that I get (yyyy-mm-dd) from the JSON response/value/attribute with time 00:00:00 as this is the only way I can calculate two dates and just fake the time of both being 00:00:00. Also noticed that the new collection date doesn’t rollover once this weeks collection is done.

q00dee · July 9, 2018, 5:39am

Is it possible to just compare/subtract two dates with value being this? yyyy-mm-dd instead of having to use yyyy-mm-dd 00:00:00? Or how I can combine yyyy-mm-dd with 00:00:00?

nickrout · July 9, 2018, 5:50am

Dates are stored internally as seconds from the epoch, so you should be able to.

cjsimmons · July 10, 2018, 1:17pm

Thanks buddy. Works a treat!!! Sorry it’s taken a while for me to reply. I’ve had a few family issues happen latley, so been busy sorting them out, and only just looked at this tonight.

The output is all the code for the page, so I’ll need to save that to a file and scrape out just the info that I need or work out a way to save just what I need.

theTechnician · July 10, 2018, 1:29pm

My local council is nice enough to provide PDFs for east and west sides broken down day by day.
Similar to mentioned above, you have to enter your postcode and house name/number to get the right PDF.

My thinking is of scraping the page somehow and OCRing the PDF to get the relevant days and bins … weather that works or not is a different matter

Robbrad · July 10, 2018, 8:26pm

Can you share what you have done already - e.g. what you have to date

q00dee · July 11, 2018, 1:30am

Currently I’ve reverted back to using my google calendar with 3 repeating entries for my green, yellow and red bin. They get imported into home assistant. I then configured a template sensor to calculate the days until bin collection. I then use this in my automation to determine which bins need to go out the day before. Notification is done via LIFX bulb.

An issue I’ve found is my template sensors dont seem to update unless home assistant is restarted. The code works fine and updates when I test it in the developer/templates section of home assistant.

I’ll post the code I have in place once I have access to my hass.

Using the rest sensor and getting the data from the council servers I’ve discovered that they don’t update the information until the actual collection day.
Example:

item_garbage: 2018-07-09
item_organic: 2018-07-09
item_recycling: 2018-07-16

Organic is weekly so I could ignore the item_organic date and just use item_garbage and item_recycling to determine what bin is due the following week. With that in mind I can only get 2018-07-16.

To my knowledge the only to calculate two dates is by converting it to timestamp and for that I need YYYY-MM-DD 00:00:00.

So how would I use something like states.sensor.penrith_waste_info.attributes.results.0.item_recycling and combine it with 00:00:00. So I can then use that within as_timestamp(2018-07-16 00:00:00) / 86400. There’s obviously more to that, but that’s all I remember without looking at my configuration.yaml.

Option 1: Stick with my google calendar, but I need to resolve why hass is not updating the template sensor.
Option 2: Use the info from the council so I don’t need to use the google calendar, but I need to figure out how to process the data and also not sure if I’ll still have an issue with the template sensor.

Robbrad · July 11, 2018, 9:41am

I would simplify where possible, you have a rest sensor that gets data / present that sensor - anything where logic is required then opt for a different solution - I did mine using bash which worked out the next thus date and grep’ed for that in the JSON (that filtered the day I needed)

Your saying the council only give the info you need on the day of collection - there is no further in the future information? then its going to make the logic difficult - do we have the right url here to cURL - I noticed there is an offset parameter.

If you were doing this from a PC manually to find out the bins - which site would you go to to get the data?

q00dee · July 11, 2018, 11:17am

This is the information that I get from the rest sensor.

This is what I’ve observed. As you can see this week’s bin collection is still showing even though it has passed already. So like what I was saying I could always just ignore item_organic because this is a weekly thing and just use item_garbage and item_recycling.

{{ states.sensor.penrith_waste_info.attributes.results.0.item_garbage }}

Would give me 2018-07-16

{{ states.sensor.penrith_waste_info.attributes.results.0.item_recycling }}

Would give me 2018-07-09

How would I calculate these dates with the current date? Or is the only way to do it is something like below:

value_template: '{{ (as_timestamp(states.calendar.hass_calendar_fgo.attributes.start_time) / 86400 - as_timestamp(as_timestamp(now()) | timestamp_custom("%Y-%m-%d 00:00:00",true)) / 86400) | int(0) }}'

This is what I’m currently using to read the dates from google calendar as I have no idea how to calculate two dates to get the remaining days to use the rest sensor data. I have limited coding skills.

My google calendar method is all in place with the template sensors and the automation in place to indicate on my LIFX only problem is the template sensor doesn’t seem to update. Only updates when I reboot hass. The best would be to use the rest sensor as I don’t need to manually update the dates, but again I dont know how to solve the date calculation issue.

This is what’s in my automation. This is what I managed to get working, I’m sure it can be done a better way.

alias: 'Notification: Bin (LIFX GREEN+YELLOW)'
  trigger:
  - at: '18:00:00'
    platform: time
  - at: '19:00:00'
    platform: time
  - at: '20:00:00'
    platform: time
  - at: '21:00:00'
    platform: time
  - at: '22:00:00'
    platform: time
  - at: '23:00:00'
    platform: time
  condition:
  - below: '2'
    condition: numeric_state
    entity_id: sensor.bin_service_fgo
  - below: '2'
    condition: numeric_state
    entity_id: sensor.bin_service_recycling
  - above: '2'
    condition: numeric_state
    entity_id: sensor.bin_service_residual
  - condition: state
    entity_id: binary_sensor.people_home
    state: 'on'
  action:
  - data:
      brightness: '204'
      color_name: green
      cycles: '2'
      entity_id: light.living_room
      mode: blink
    service: light.lifx_effect_pulse
  - delay: '2'
  - data:
      brightness: '204'
      color_name: yellow
      cycles: '2'
      entity_id: light.living_room
      mode: blink
    service: light.lifx_effect_pulse

Robbrad · July 11, 2018, 1:09pm

It’s a bit crap the council don’t tell you what’s next even though the date has passed

Could you split the sensor into 3 and have a value template hide that sensor if it’s past the current date?

Which council website do you use(don’t have to tell me your address)

cjsimmons · July 12, 2018, 3:23am

Thanks for all your help @Robbrad. I finally got something up and running. The zenid was causing me grief, but you help and that python script you made works a treat. It just pretty much reads the HTML and outputs it on screen.

I added some magic with awk & sed to cut down to the table I needed and removed any extra white space, then saved it to a txt file. I then set up a command line sensor for each bin that uses sed to read a specific line fron the txt file and voila, I now have the following thanks to your help.

bins

As my rubbish is picked up weekly regardless of public holidays, I can just set the python script to update once a week, probably Monday night at midnight after collection day, and it should be good to go.

I might look at combining these at a late date somehow to have just the next collection date, and what is to be collected.

Thanks again for you help with this.

Robbrad · July 12, 2018, 5:54am

Nice work - looks good

q00dee · July 14, 2018, 3:02am

Okay, I finally have a working solution. Using rest sensor to the server and no more google calendar. Red box is just for debug and I’ll eventually remove.

Next Collection date uses the date from garbage or recycling if it’s within 1 to 7 days. The same is used for the Bin Type, it’s used to determine the icon to display and bin types.

UhtredtheBold · July 16, 2018, 9:23pm

I’ve tried to do this in the past and got nowhere. My council is Stevenage and their form is here - http://www.stevenage.gov.uk/find/

I’ve tried various commands, for example curl --form txtPostcode='postcode' --form txtNumber=555 http://www.stevenage.gov.uk/find/ but no dice. If anyone is able to point me in the right direction i’d be very grateful.

vnkr · July 17, 2018, 8:29am

Hi All,

Decided I’ll share my solution to this problem which is much lazier and despite no api available for the page doesn’t require running external scripts.

My collection page looks like that (Address is fake):
https://www.aylesburyvaledc.gov.uk/bin-collections/766313148

And I decided not to parse the table but just to use a scrape sensor to get information from the text field above the table. I only need the text in bold so it goes like:

select: ".collection-next p strong"

After that I use the template sensor to set the name of template to the bin type, value of the template to the date (by simply removing all the text before the date), and image to the image of the bin from the same website so my sensor looks like that:

- platform: template
sensors:
  waste_collection:
    friendly_name_template: >-
      {% if "GARDEN" in states('sensor.waste_collection_scrape') %}
        Garden waste
      {% elif "RECYCLING" in states('sensor.waste_collection_scrape') %}
        Recycling waste
      {% else %}
        Regular waste
      {% endif %}
    value_template: >-
      {% if "GARDEN" in states('sensor.waste_collection_scrape') %}
        {{ states('sensor.waste_collection_scrape')|replace('GARDEN WASTE',"") }}
      {% elif "RECYCLING" in states('sensor.waste_collection_scrape') %}
        {{ states('sensor.waste_collection_scrape')|replace('RECYCLING + FOOD WASTE',"") }}
      {% else %}
        {{ states('sensor.waste_collection_scrape')|replace('WASTE + FOOD WASTE',"") }}
      {% endif %}
    entity_picture_template: >-
      {% if "GARDEN" in states('sensor.waste_collection_scrape') %}
        "https://www.aylesburyvaledc.gov.uk/sites/all/themes/avdc/images/bin-garden.gif"
      {% elif "RECYCLING" in states('sensor.waste_collection_scrape') %}
        "https://www.aylesburyvaledc.gov.uk/sites/all/themes/avdc/images/bin-recycling.gif"
      {% else %}
        https://www.aylesburyvaledc.gov.uk/sites/all/themes/avdc/images/bin-refuse.gif
      {% endif %}

And the resulting card in lovelace for this sensor looks like that: