Bin / Waste Collection

@bachoo786

I clearly said it wasn’t tested and linked to the documentation - however it feels like you want me to write it for you, so with a little trial + error, perserverance and more reading of the documentation I got it working. I think you could have done the same.

The following tested example is 3 rest sensors + 6 templates (this could be improved, as it will hit the council daily x 3. but I have missed doing the kids bath / bedtime to do this, so take it as is or improve if you wish)

  - platform: rest
    name: bin_sensors_recyclables
    resource: https://www.harrow.gov.uk/site/custom_scripts/harrow/ajax/bin_collections.php?u=100021261750
    json_attributes_path: "$.last.*.RECYCLABLES"
    value_template: "OK"
    json_attributes:
      - "BinType"
      - "EventDate"
    scan_interval: 43200
  - platform: template
    sensors:
      recyclables_bintype:
        value_template: "{{ state_attr('sensor.bin_sensors_recyclables', 'BinType') }}"
      recyclables_eventdate:
        value_template: "{{ state_attr('sensor.bin_sensors_recyclables', 'EventDate') }}"

  - platform: rest
    name: bin_sensors_food
    resource: https://www.harrow.gov.uk/site/custom_scripts/harrow/ajax/bin_collections.php?u=100021261750
    json_attributes_path: "$.last.*.FOOD"
    value_template: "OK"
    json_attributes:
      - "BinType"
      - "EventDate"
    scan_interval: 43200
  - platform: template
    sensors:
      food_bintype:
        value_template: "{{ state_attr('sensor.bin_sensors_food', 'BinType') }}"
      food_eventdate:
        value_template: "{{ state_attr('sensor.bin_sensors_food', 'EventDate') }}"

  - platform: rest
    name: bin_sensors_residual
    resource: https://www.harrow.gov.uk/site/custom_scripts/harrow/ajax/bin_collections.php?u=100021261750
    json_attributes_path: "$.last.*.RESIDUAL"
    value_template: "OK"
    json_attributes:
      - "BinType"
      - "EventDate"
    scan_interval: 43200
  - platform: template
    sensors:
      residual_bintype:
        value_template: "{{ state_attr('sensor.bin_sensors_residual', 'BinType') }}"
      residual_eventdate:
        value_template: "{{ state_attr('sensor.bin_sensors_residual', 'EventDate') }}"

Thanks mate but I did have a go at it and couldn’t resolve it. God Bless :raised_hands:

1 Like

If you need it to be “next” instead of last you might need to tweak the path

Yep I will look into it thanks again mate :+1:t3:

Hi @Holdestmade

I have used your code i.e.

  - platform: template
    sensors:
      blue_bin_in_days:
        friendly_name: Blue Bin
        icon_template: 'mdi:recycle'
        value_template: >-
          {% set date_in = states.sensor.recyclables_eventdate.state|replace('\n', '') %}
          {% set bin = strptime((date_in), "%a, %d %B %Y") %}
          {% set diff = as_timestamp(bin) - as_timestamp(now()) %}
          {% set days = ((diff / 86400)+1) | int %}
          {% if days == 0 %}
            Today
          {% elif days == 1 %}
            Tomorrow
          {% elif days == 7 %}
            1 Week
          {% elif days == 14 %}
            2 Weeks
          {% else %}
            {{ days }} days
          {% endif %}

however, with the new HA release i.e. 2021.10 … I am getting these warnings:

 2021-10-11 12:22:21 WARNING (MainThread) [homeassistant.helpers.template] Template warning: 'strptime' got invalid input '2021-10-20T00:00:00' when rendering template '{% set date_in = states.sensor.recyclables_eventdate.state|replace('\n', $
  Today
{% elif days == 1 %}
  Tomorrow
{% elif days == 7 %}
  1 Week
{% elif days == 14 %}
  2 Weeks
{% else %}
  {{ days }} days
{% endif %}' but no default was specified. Currently 'strptime' will return '2021-10-20T00:00:00', however this template will fail to render in Home Assistant core 2021.12
2021-10-11 12:22:21 WARNING (MainThread) [homeassistant.helpers.template] Template warning: 'strptime' got invalid input '2021-10-20T00:00:00' when rendering template '{% set date_in = states.sensor.recyclables_eventdate.state|replace('\n', $
  Today
{% elif days == 1 %}
  Tomorrow
{% elif days == 7 %}
  1 Week
{% elif days == 14 %}
  2 Weeks
{% else %}
  {{ days }} days
{% endif %}' but no default was specified. Currently 'strptime' will return '2021-10-20T00:00:00', however this template will fail to render in Home Assistant core 2021.12

Is that something you could please help with? As my log is piling up quite quickly with the same warnings.

Its something to do with updates in the latest release, see below. Not getting any errors myself though:

1 Like

Thanks I resolved this now.

I was wondering is there a way which would tell me what bin type is for the week? So e.g. “This week’s bin is …” something along those lines?

I’m sure its possible. I’ll have a play around sometime

1 Like

First post, be kind. This is one of my favourite projects! forever forgetting which bin and when!

Is there a simple way to piggy back on top of what appears to be someone who has already done most of the work for my council ?

Thanks

Hello,
can someone help me with this issue please? I’ve managed to figure out a place we can scrape from

For Wigan Council, I have found an APi of sorts, but ran out of talent fast. Info is here in case ayone fancies a challenge

I’ve managed to get something working in Powershell and Node-Red to get the bin data from the Cardiff Council website. The security token is generated and passed back to you when you make a request to one of the API pages - so I just did that first in a web request then used the bearer token in my api request.

I’ve got to the point where I am sending the date for next collection from node-red to a sensor entity for each bin type, but I’m still working on how to display the information in a usable way in HA.

I got it working late last night, let me know if you would like me to share the node-red flow or powershell. I might have a go at converting it to python and adding to that github library mentioned above in a few days…

1 Like

I’d love to see how you got it working if you could share your code that would be amazing.

anyone having issues since 2021.12 with date formatting from the Json sensor? I am getting a date rendering error now - worked fined before. My council passes an ISO formatted date back - which i then convert ( see code below) - which no longer works:

  - platform: rest
    name: Black Bins Collection
    resource: https://citizen.westberks.gov.uk/apiserver/ajaxlibrary
    method: POST
    scan_interval: 7200
    device_class: timestamp
    payload: '{ "jsonrpc" : "2.0" , "id" : "1321455687934464654" , "method" : "veolia.wasteservices.v1.getServicesByUPRN" , "params" : { "uprn" : "xxxxxxxx" } }'
    value_template: "{{ as_timestamp(value_json.result.services[0].ServiceHeaders.ServiceHeader.Next) | timestamp_custom('%a, %b %d') }}"
    headers:
      Content-Type: application/json
      User-Agent: Home Assistant REST sensor

The json out put today is “2021-12-13T07:00:00” - so iso format - yet the sensor wont render the time

Hey Dan

Did you get any further with the Cardiff Council code?

I’ve got a Node-Red flow for anyone using Cheshire East. You will need to replace the UPRN in the http request node, you can find yours using UPRN search, address & postcode finder - FindMyAddress.

Might not be perfectly elegant, however it works for me

[{"id":"cae57bf.8eb5988","type":"inject","z":"40d22550.6e2dcc","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"43200","crontab":"","once":true,"onceDelay":"5","topic":"","payloadType":"date","x":100,"y":40,"wires":[["812b22c7.db831"]]},{"id":"812b22c7.db831","type":"http request","z":"40d22550.6e2dcc","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://online.cheshireeast.gov.uk/MyCollectionDay/SearchByAjax/GetBartecJobList?uprn=xxxxxxxxx","tls":"","persist":false,"proxy":"","authType":"","senderr":false,"x":190,"y":100,"wires":[["c61cbb09.69fd68"]]},{"id":"c61cbb09.69fd68","type":"html","z":"40d22550.6e2dcc","name":"Table","property":"payload","outproperty":"payload","tag":"div.inner-results-contains-table > div > table > tbody > tr > td ","ret":"text","as":"single","x":270,"y":160,"wires":[["f0338d6d.f83b1"]]},{"id":"f0338d6d.f83b1","type":"function","z":"40d22550.6e2dcc","name":"","func":"dayjs.extend(duration);\ndayjs.extend(customParseFormat);\n\nlet upcomingBins = [];\nlet gardenDone = false;\nlet mixedDone = false;\nlet generalDone = false;\n\nmsg.payload.forEach((i) => {\n  if (i.length > 0) {\n    const item = i.split(\"\\n\");\n    const items = item.map((s) => s.trim());\n\n    if (items.includes(\"Empty Standard Garden Waste\") && !gardenDone) {\n      gardenDone = true;\n      upcomingBins.push(populatePayload(items, \"sensor.garden_waste\"));\n    }\n    if (items.includes(\"Empty Standard Mixed Recycling\") && !mixedDone) {\n      mixedDone = true;\n      upcomingBins.push(populatePayload(items, \"sensor.recycling_waste\"));\n    }\n    if (items.includes(\"Empty Standard General Waste\") && !generalDone) {\n      generalDone = true;\n      upcomingBins.push(populatePayload(items, \"sensor.general_waste\"));\n    }\n  }\n});\n\nfunction populatePayload(items, id) {\n  let binItem = {\n    payload: {\n      data: {\n        state: null,\n        attributes: {}\n      }\n    }\n  };\n\n  const date = dayjs(items[2], \"DD/MM/YYYY\");\n  const today = dayjs();\n\n  let hours = date.diff(today, \"hours\");\n  binItem.entity_id = id;\n  binItem.payload.data.attributes.day = items[1];\n  binItem.payload.data.attributes.date = date;\n  binItem.payload.data.state = date;\n  binItem.payload.data.attributes.daysUntil = hoursToDays(hours);\n\n  return binItem;\n}\n\nfunction toString(h) {\n  if (h < 0) return \"Today\";\n  else if (h > 0 && h < 24) return \"Tomorrow\";\n  else return `${hoursToDays(h)} days time`;\n}\n\nfunction hoursToDays(h) {\n  return Math.ceil(dayjs.duration({ hours: h }).asDays());\n}\n\nconst slicedArray = upcomingBins.slice(0, 3);\nmsg.payload = slicedArray;\n\nreturn msg","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"dayjs","module":"dayjs"},{"var":"customParseFormat","module":"dayjs/plugin/customParseFormat"},{"var":"duration","module":"dayjs/plugin/duration"}],"x":340,"y":220,"wires":[["2805aa44.d14236"]]},{"id":"2805aa44.d14236","type":"split","z":"40d22550.6e2dcc","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":470,"y":260,"wires":[["14df8c02.bdabd4"]]},{"id":"14df8c02.bdabd4","type":"ha-api","z":"40d22550.6e2dcc","name":"","server":"68b1f3b6.69c6ac","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/states/{{payload.entity_id}}","data":"payload.payload.data\t","dataType":"jsonata","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":590,"y":260,"wires":[[]]},{"id":"68b1f3b6.69c6ac","type":"server","name":"Home Assistant","version":2,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":"30"}]

I then have the following template sensors to tell me how many days until collection

template:
  sensor:
    - unique_id: garden_bin
      state: >
        {% set t = states('sensor.garden_waste') | as_datetime %}
          {% if t %}
            {% set delta = t - now() %}
            {% set days = delta.days + 1 %}
            {% if days == 1 %}
              Tomorrow
            {% elif days == 0 %}
              Today
            {% else %}
              {{ days }} days time
            {% endif %}
        {% endif %}
    - unique_id: general_bin
      state: >
        {% set t = states('sensor.general_waste') | as_datetime %}
          {% if t %}
            {% set delta = t - now() %}
            {% set days = delta.days + 1 %}
            {% if days == 1 %}
              Tomorrow
            {% elif days == 0 %}
              Today
            {% else %}
              {{ days }} days time
            {% endif %}
        {% endif %}
    - unique_id: recycling_bin
      state: >
        {% set t = states('sensor.recycling_waste') | as_datetime %}
          {% if t %}
            {% set delta = t - now() %}
            {% set days = delta.days + 1 %}
            {% if days == 1 %}
              Tomorrow
            {% elif days == 0 %}
              Today
            {% else %}
              {{ days }} days time
            {% endif %} 
        {% endif %}

I might work on turning this into a web api if anyone is interested?

EDIT: Web API is now available. Repo available here: GitHub - greghesp/bin-collection-api
Temporary web API endpoint: https://bin-collection-api.vercel.app/api/council/cheshire_east?uprn=XXXXXXX

3 Likes

Hey so I’ve followed along through this thread and there’s been some really great information here.
Of specific help was the person (sorry I scrolled back through but couldn’t find the reply again) who pointed out that a lot of these sites were hitting an API server.

This lead me onto realising that for South Tyneside it was doing the same and hitting this url with a curl POST request resulted in really well formatted JSON being returned:

Real Address omitted - used the chippy round the corner obviously!

curl -X POST -H "Content-Type: application/json" -d '{"id": "1640213425122","method": "wtGetBinCollectionDates","params": {"addresscode": "S100000525991|303 ALBERT ROAD, JARROW, SOUTH TYNESIDE, TYNE AND WEAR, NE32 5JU"}}'

I’ve tried this in my configuration.yaml as follows with a value_template to pull out the date string like this:
(value set to 10 for testing, it will be set to 1/2 day on full deploy)

sensor:
  - platform: command_line
    command: 'curl -X POST -H "Content-Type: application/json" -d `{"id": "1640213425122","method": "wtGetBinCollectionDates","params": {"addresscode": "S100000525991|303 ALBERT ROAD, JARROW, SOUTH TYNESIDE, TYNE AND WEAR, NE32 5JU"}}`'
    name: BlackBinDate
    scan_interval: 10
    value_template: "{{ value_json['result']['NextHouseholdCollection']['FullDateString'] }}"

I don’t really want it saving a file to the server then having to parse the json separately.
That seems like an extra step considering how nicely formatted the returned JSON is and exactly what value_template seems to be for.

At this point I’ve run out of talent/knowledge.
Running that curl command on my machine works fine and returns json but doesn’t work in home assistant with the above sensor.

Any help greatly appreciated!

EDIT:

For anyone who might stumble across this I got it working and here’s how.
I added 3x sensors to my configuration.yaml. One for each bin:

sensor:
  - platform: rest
    name: Black Bin Collection
    resource: https://www.southtyneside.gov.uk/apiserver/ajaxlibrary
    method: POST
    scan_interval: 10
    payload: '{"jsonrpc": "2.0","id": "1640209277454","method": "wtGetBinCollectionDates","params": {"addresscode": "S100000525991|303 ALBERT ROAD, JARROW, SOUTH TYNESIDE, TYNE AND WEAR, NE32 5JU"}}'
    headers:
      Content-Type: application/json
      User-Agent: Home Assistant REST sensor
    value_template: '1'
    json_attributes_path: "$.result.NextHouseholdCollection"
    json_attributes:
      - FullDateString

  - platform: rest
    name: Blue Bin Collection
    resource: https://www.southtyneside.gov.uk/apiserver/ajaxlibrary
    method: POST
    scan_interval: 10
    payload: '{"jsonrpc": "2.0","id": "1640209277454","method": "wtGetBinCollectionDates","params": {"addresscode": "S100000525991|303 ALBERT ROAD, JARROW, SOUTH TYNESIDE, TYNE AND WEAR, NE32 5JU"}}'
    headers:
      Content-Type: application/json
      User-Agent: Home Assistant REST sensor
    value_template: '1'
    json_attributes_path: "$.result.NextRecyclingCollection"
    json_attributes:
      - FullDateString

  - platform: rest
    name: Green Bin Collection
    resource: https://www.southtyneside.gov.uk/apiserver/ajaxlibrary
    method: POST
    scan_interval: 10
    payload: '{"jsonrpc": "2.0","id": "1640209277454","method": "wtGetBinCollectionDates","params": {"addresscode": "S100000525991|303 ALBERT ROAD, JARROW, SOUTH TYNESIDE, TYNE AND WEAR, NE32 5JU"}}'
    headers:
      Content-Type: application/json
      User-Agent: Home Assistant REST sensor
    value_template: '1'
    json_attributes_path: "$.result.NextGardenCollection"
    json_attributes:
      - FullDateString

I then created an entities card with the following yaml code:

type: entities
title: Bindicator
entities:
  - type: attribute
    name: 'Black Bin:'
    entity: sensor.black_bin_collection
    attribute: FullDateString
    icon: mdi:trash-can
  - type: attribute
    name: 'Blue Bin:'
    entity: sensor.blue_bin_collection
    attribute: FullDateString
    icon: mdi:recycle
  - type: attribute
    name: 'Green Bin:'
    entity: sensor.green_bin_collection
    attribute: FullDateString
    icon: mdi:sprout

Pretty tidy solution and I’m happy with it so far.
Massive thanks to everyone in this thread, it’s a wealth of knowledge and just by reading through properly I was able to piece this solution together nicely.
If anyone can point to any improvements, as a fairly new Home Assistant user I’d be all ears.
Some ideas which would be great and could use help with:

  • colour the icons to their actual bin colour
    edit: did this by installing card-mod and using:
card_mod:
      style: |
        :host {
          --card-mod-icon: mdi:trash-can;
          --card-mod-icon-color: black
        }
  • highlight the bin which is the next date
  • Green bin is only collected at certain times of the year. If result is “” then display “No current collections”

Sorry to re-open an old thread but I’m really struggling with scraping the bin schedule off my councils website. I’ve successfully scraped other elements from the site just to test it is possible but for some reason the bin schedule isn’t shown. When I enter my postcode into the lookup, adds a string to the url which I can then copy and paste to go straight to the schedule however when I curl the same address the schedule isn’t there which makes me think there is some kind of script being ran.

The url is https://mybins.blackburn.gov.uk/calendar/MTAwMDEwNzQ5NjQx

As per other suggestions in in this thread I’ve also tried exploring the cookie to see if anything is being stored in there but I can’t see anything obvious.

Any help would be appreciated :slightly_smiling_face:

I managed to solve this myself. The problem was that the bin schedule is loaded after the page loads so scraping the url didn’t work. Instead I used phantomjs to download the page and then ha-multiscrape to scrape based on the CSS Selectors.

Hello Dan. I’m also interested in understanding how you did this. Cheers