Bin / Waste Collection

mayker · June 9, 2019, 6:07pm

Here’s scrape sensors to return the date for pink and grey days:

First “pink” day on page (don’t need an index for the first one, returns “10”):

- platform: scrape
  name: First Pink
  resource: https://apps.castlepoint.gov.uk/cpapps/index.cfm?roadID=2757&fa=wastecalendar.displayDetails
  select: ".pink"

Second “pink” day on page (indexes start at 0, returns “24”):

- platform: scrape
  name: Second Pink
  resource: https://apps.castlepoint.gov.uk/cpapps/index.cfm?roadID=2757&fa=wastecalendar.displayDetails
  select: ".pink"
  index: 1

And you can do the same for the grey days, but the class is .normal in this case (returns “3”).

- platform: scrape
  name: First Grey
  resource: https://apps.castlepoint.gov.uk/cpapps/index.cfm?roadID=2757&fa=wastecalendar.displayDetails
  select: ".normal"

The month header can be grabbed with the following (months are the second and third “h2” on the page, returns “June 2019”)

- platform: scrape
  name: First Month
  resource: https://apps.castlepoint.gov.uk/cpapps/index.cfm?roadID=2757&fa=wastecalendar.displayDetails
  select: "h2"
  index: 1

“.pink” is the css class used for the pink dates, “.normal” for the grey, and “h2” is used for the month headers. Then you just use index to select what number occurrence the element you’re looking for is, index numbers start at zero and go up from there.

dicko · June 9, 2019, 9:46pm

Hi

https://www.wakefield.gov.uk/site/Where-I-Live-Results?uprn=63121996

This is not my exact address but a random one but how would I scrape that please ?

There isnt a calendar on the page just text

Thanks in advance

Martyn

mayker · June 10, 2019, 4:09am

So here are instructions on how to accomplish this with a scrape sensor, but I will say that I was unable to get a working sensor for it as the wakefield site takes far too long to load and times out.

In chrome right click the element that you want to get and select “inspect”.
In the panel that just opened right click the highlighted element.
Select “copy” then “copy selector”.
Paste the copied selector into the select: field of a scrape sensor.

For the wakefield site, doing this on the date under “last collection” of “household waste” gives us:

#ctl00_PlaceHolderMain_Waste_output > div:nth-child(2) > div:nth-child(2) > div:nth-child(2)

Your sensor would look like this:

- platform: scrape
  resource: https://www.wakefield.gov.uk/site/Where-I-Live-Results?uprn=63121996
  select: "#ctl00_PlaceHolderMain_Waste_output > div:nth-child(2) > div:nth-child(2) > div:nth-child(2)"

denver · June 10, 2019, 6:55am

Thats amazing, looks so easy but I just couldn’t work it out. Thank you mayker for your time on this.

denver · June 10, 2019, 7:30am

Is there a way to create and test scrapes without writing them in yaml and restating Hassio each time?

DavidFW1960 · June 10, 2019, 8:54am

You could scrape the data and then set it to a variable in the template editor and play with the template to get the correct data and then make the sensor and restart. I do that all the time to check I am trying the right options.

Robbrad · June 10, 2019, 11:19am

Unfortunately not. There are browser plugins to help find out the correct selectors though. Search Google Chrome CSS selector extension

denver · June 10, 2019, 2:03pm

Thanks David and Robbrad, I will look into both methods.

dicko · June 10, 2019, 7:25pm

Thank you for explaining and possibly why I wasn’t getting anything but I will keep trying lol

Martyn

nerobot · June 12, 2019, 6:26pm

Hi,

Sorry for the delay in replying. If you use Node-red the following code will scrap the information, split it into Garden, Recycling, and General waste. Those are then sent via mqtt (which I “sense” in HA for display).

You will need to change the http request for the one you obtain when you enter your postcode, but it appears to be working for me.

denver · June 13, 2019, 9:50am

Ive run ‘go get github.com/ericchiang/pup’ on my ubuntu distribution and its installed the ‘pup’ file in /go/bin folder. Is there anything else I need to do? as it keeps coming up with

Command ‘pup’ not found, but there are 17 similar ones.

Robbrad · June 13, 2019, 11:11am

Does running /go/bin/pup work?

denver · June 13, 2019, 11:57am

/go/bin/pup didn’t but /root/go/bin/pup seem to do something, just a cursor waiting for something to be entered.

This is what I get until I press cont+c to stop.

Sorry to be so vague but I’ve searched everywhere I could think of and couldn’t get any information hence posting here.

dicko · June 13, 2019, 1:43pm

Node - Red maybe the solution for some but I could not copy that code into my Node Red. I got a couple of erros with the " not been correct and then with mb10.

Not sure if its just me but I could not import it via the clipboard to even have a lool

nerobot · June 13, 2019, 1:51pm

I just tried copying it in myself and can confirm I couldn’t copy it into my work machine’s node-red.

I’ll have a look ino it when I get home tonight.

Sorry about that.

nerobot · June 13, 2019, 1:59pm

Try the following code. It should work when imported into node-red

dicko · June 13, 2019, 9:03pm

Hi mate.

Spot on. That actually worked where the scrape didnt. So thank you.

Ive attached my love lace card for now.

Just need to get the colours changed accordingley and maybe change it to countdown days but I will be hooking it up to my Telegram warnings.

Thanks again

Martyn

denver · June 17, 2019, 6:47am

Im stumped, Ive been trying to work out how show the next date and colour of rubbish to be placed out a few days in advance.
After using this information - Bin / Waste Collection
Ive created a snippet to show the days count down as in -

{% set day = states.sensor.second_pink_sack.state %}
{% set month = states.sensor.month_current.state %}
{% set datetext = day + month %}
{% set days = strptime(datetext,’%d%B %Y’) | as_timestamp - as_timestamp(now()) %}
{{(days/86400) | round | int}} days

I was intending to make the ‘day’ in the first line a variable and populate it via a ‘if loop’ looking at the ‘now().day’ and ‘<, >, =,’ comparisons containing something like this -

{% set week1 = states.sensor.first_black_sack.state %}
{% set week2 = states.sensor.first_pink_sack.state %}
{% set week3 = states.sensor.second_black_sack.state %}
{% set week4 = states.sensor.second_pink_sack.state %}
{% set week5 = states.sensor.third_black_sack.state %}
{% set week6 = states.sensor.third_pink_sack.state %}

I also create a short snippet to test a ‘if loop’, by manually changing the {“day”:n} number to represent the date changing, it sort of worked -

{% set day = {“day”:19} %}
{{day.day}}

{% if (day.day | int <= states.sensor.first_pink_sack.state |int) and (day.day | int < states.sensor.second_pink_sack.state |int) %}1st pink

{% elif (day.day | int <= states.sensor.second_pink_sack.state |int) and (day.day | int > states.sensor.first_pink_sack.state |int)%}2nd pink

{% elif (day.day | int <= states.sensor.third_pink_sack.state |int) and (day.day | int > states.sensor.second_pink_sack.state |int)%}3nd pink

{% else %}
{%endif%}

I was intending to some how rewrite and merge the snippets together with a automation, but with my limited knowledge in this Im not even sure Im taking the right approach. any pointers on this would be much appreciated.

Martinvdm · July 8, 2019, 7:22pm

How do you find the uniqueAddressID ?

Cadster · July 10, 2019, 7:49am

If you use the componnent by @yniezink it wil get that for you.
You can also do a post request:

curl -X POST -v -d '{"companyCode":"<REPLACE BY THE COMPANYCODE OF YOUR 
 MUNICIPALTY>", "postCode":"<YOUR POSTCODE>", "houseNumber":"<YOUR HOUSENUMBER>", "houseLetter":"", "houseNumberAddition":""}' http://wasteapi2.2go-mobile.com/api/FetchAdress

Do a callender request trough you local waste-webpage in your municipality and watch the response to post messages. In for example firefox look at the pagesource and than in the networktab. you wil find a post-request FetchAdress. When you sellect that that request you can read the response wich wil give u the UniqueId u need.