Hey guys, I’ve created a custom component for the Dutch waste collector Cure Afvalbeheer. It creates sersors (for the waste fractions you specify in the configuration.yaml) like this:
If anyone wants to implement the component, this is the code:
Isn’t it always the same day of the week that Cure Afvalbeheer is collecting? I mean, it would be nice to know what time they are in my street so I can have an excuse to my collegues and leave work earlier
Most of the time it is the same day of the week, but not always! The time is sadly not shared by Cure, so i have no way of knowing what time they will be there.
I spent a few hours studying you code, I tried to modify it such that it would work with the above address.
Unfortunately I keep getting errors. I guess it all boils down to the right url, which I don’t seem to get right…
I know the Scrape sensor, have been playing with it a bit before. Unfortunately I have not managed to get a clear understanding of how to use this sensor
Is there a way to find out how the data exactly looks like that is being scraped? Once I know that, I am probably able to get it to work.
This is the important part of the website, I think, showing via View page source:
<tbody><tr data-lob="HGRIJS" data-material="MINI" class="blockWrapper RGE2Z1-16">
<th scope="row"><h2>Grijze container</h2>
<a title="Meer informatie over Grijze container" href="/afval/huisvuil-en-restafval-inleveren"><img alt="Grijze container" src="/sites/all/modules/custom/afvalwijzer/img/web_picto.minicontainer_grijs.jpg"></a><p>Wordt op de aangegeven dinsdagen<br />opgehaald. U mag de grijze container<br />tussen 6.00-8.00 uur aan de straat<br />zetten.<br /></p>
</th><td class="m-01"><ul><li>9</li><li>23</li></ul></td><td class="m-02"><ul><li>6</li><li>20</li></ul></td><td class="m-03"><ul><li>6</li><li>20</li></ul></td><td class="m-04"><ul><li>3</li><li>17</li></ul></td><td class="m-05"><ul><li>1</li><li>15</li><li>29</li></ul></td><td class="m-06"><ul><li>12</li><li>26</li></ul></td><td class="m-07"><ul><li>10</li><li>24</li></ul></td><td class="m-08"><ul><li>7</li><li>21</li></ul></td><td class="m-09"><ul><li>4</li><li>18</li></ul></td><td class="m-10"><ul><li>2</li><li>16</li><li>30</li></ul></td><td class="m-11"><ul><li>13</li><li>27</li></ul></td><td class="m-12"><ul><li>11</li><li class="afwijkende-ophaaldag">22</li> <li class="element-invisible">(afwijkende ophaaldag)</li></ul></td></tr><tr data-lob="HGRIJS" data-material="MINI" class="blockWrapper RGE2Z1-19">
<th scope="row"><p>Wordt op de aangegeven dinsdagen<br />opgehaald. U mag de grijze container<br />tussen 6.00-8.00 uur aan de straat<br />zetten.<br /></p>
</th><td class="m-01"><ul></ul></td><td class="m-02"><ul></ul></td><td class="m-03"><ul></ul></td><td class="m-04"><ul></ul></td><td class="m-05"><ul></ul></td><td class="m-06"><ul></ul></td><td class="m-07"><ul></ul></td><td class="m-08"><ul></ul></td><td class="m-09"><ul></ul></td><td class="m-10"><ul></ul></td><td class="m-11"><ul></ul></td><td class="m-12"><ul></ul></td></tr><tr data-lob="HGFT" data-material="NVT" class="blockWrapper BGO2Z1-19">
<th scope="row"><h2>Groene container</h2>
<a title="Meer informatie over Groene container" href="/afval/gft-inleveren"><img alt="Groene container" src="/sites/all/modules/custom/afvalwijzer/img/web_picto.minicontainer_bio.jpg"></a><p>Wordt op de aangegeven dinsdagen<br />opgehaald. U mag de groene container<br />tussen 6.00-8.00 uur aan de straat<br />zetten.<br /></p>
</th><td class="m-01"><ul></ul></td><td class="m-02"><ul></ul></td><td class="m-03"><ul></ul></td><td class="m-04"><ul></ul></td><td class="m-05"><ul></ul></td><td class="m-06"><ul></ul></td><td class="m-07"><ul></ul></td><td class="m-08"><ul></ul></td><td class="m-09"><ul></ul></td><td class="m-10"><ul></ul></td><td class="m-11"><ul></ul></td><td class="m-12"><ul></ul></td></tr><tr data-lob="HGFT" data-material="NVT" class="blockWrapper BGO2Z1-16">
<th scope="row"><p>Wordt op de aangegeven dinsdagen<br />opgehaald. U mag de groene container<br />tussen 6.00-8.00 uur aan de straat<br />zetten.<br /></p>
</th><td class="m-01"><ul><li>2</li><li>16</li><li>30</li></ul></td><td class="m-02"><ul><li>13</li><li>27</li></ul></td><td class="m-03"><ul><li>13</li><li>27</li></ul></td><td class="m-04"><ul><li>10</li><li>24</li></ul></td><td class="m-05"><ul><li>8</li><li>22</li></ul></td><td class="m-06"><ul><li>5</li><li>19</li></ul></td><td class="m-07"><ul><li>3</li><li>17</li><li>31</li></ul></td><td class="m-08"><ul><li>14</li><li>28</li></ul></td><td class="m-09"><ul><li>11</li><li>25</li></ul></td><td class="m-10"><ul><li>9</li><li>23</li></ul></td><td class="m-11"><ul><li>6</li><li>20</li></ul></td><td class="m-12"><ul><li>4</li><li>18</li></ul></td></tr><tr data-lob="HGFT" data-material="KERSTB" class="blockWrapper GA3K07-17">
<th scope="row"><h2>Kerstboom</h2>
<a title="Meer informatie over Kerstboom" href=""><img alt="Kerstboom" src="/sites/all/modules/custom/afvalwijzer/img/web_picto_kerstboom.jpg"></a><p>Zet de kerstboom (zonder versiering!)<br />tussen 6.00 en 08.00 uur aan de straat.<br />Niet op of in de nabijheid van een<br />ondergrondse container.<br /></p>
</th><td class="m-01"><ul></ul></td><td class="m-02"><ul></ul></td><td class="m-03"><ul></ul></td><td class="m-04"><ul></ul></td><td class="m-05"><ul></ul></td><td class="m-06"><ul></ul></td><td class="m-07"><ul></ul></td><td class="m-08"><ul></ul></td><td class="m-09"><ul></ul></td><td class="m-10"><ul></ul></td><td class="m-11"><ul></ul></td><td class="m-12"><ul></ul></td></tr><tr data-lob="TEXTL" data-material="NVT" class="blockWrapper T26">
<th scope="row"><h2>Kleding, textiel en schoenen</h2>
<a title="Meer informatie over Kleding, textiel en schoenen" href="/afval/textiel-inleveren"><img alt="Kleding, textiel en schoenen" src="/sites/all/modules/custom/afvalwijzer/img/web_picto.textiel.jpg"></a><p>Zet textiel in een plastic zak om 08.00<br />uur aan de straat. Ook kleine elektr.<br />apparaten mogen mee (in aparte plastic<br />zak). Inzamelaar: GoudGoed, tel.:<br />050-5266881<br /></p>
</th><td class="m-01"><ul><li>23</li></ul></td><td class="m-02"><ul><li>27</li></ul></td><td class="m-03"><ul><li>27</li></ul></td><td class="m-04"><ul><li>24</li></ul></td><td class="m-05"><ul><li>22</li></ul></td><td class="m-06"><ul><li>26</li></ul></td><td class="m-07"><ul></ul></td><td class="m-08"><ul></ul></td><td class="m-09"><ul><li>25</li></ul></td><td class="m-10"><ul><li>23</li></ul></td><td class="m-11"><ul><li>27</li></ul></td><td class="m-12"><ul></ul></td></tr><tr data-lob="HPAP" data-material="PAP240" class="blockWrapper P72C">
<th scope="row"><h2>Oud papier</h2>
<a title="Meer informatie over Oud papier" href=""><img alt="Oud papier" src="/sites/all/modules/custom/afvalwijzer/img/web_picto.minicontainer_papier.jpg"></a><p>Zet de container om 8.00 uur aan de<br />straat. Geen melk- en sappakken.<br />Inzamelaar:Dweilband Lekkur Anders<br />tel: 06-11181410<br /></p>
</th><td class="m-01"><ul><li>6</li></ul></td><td class="m-02"><ul><li>3</li></ul></td><td class="m-03"><ul><li>3</li></ul></td><td class="m-04"><ul><li>7</li></ul></td><td class="m-05"><ul><li>5</li></ul></td><td class="m-06"><ul><li>2</li></ul></td><td class="m-07"><ul><li>7</li></ul></td><td class="m-08"><ul></ul></td><td class="m-09"><ul><li>1</li></ul></td><td class="m-10"><ul><li>6</li></ul></td><td class="m-11"><ul><li>3</li></ul></td><td class="m-12"><ul><li>1</li></ul></td></tr><tr data-lob="KGA" data-material="ORIGIN" class="blockWrapper K26">
It shows the data for the whole year (2018) in a table.
How do I get out of this garble of info, the data I want?
I do understand the format: m-01 means January, m-02 means February, the dates that follow the m-xy data are the dates of pick up. Meaning pick up on Jan 9 and 23 of the “Grijze container”, and so on.
Sorry, I’m a hardware guy, I can manage some simple programming in HA itself, but this goes above my hat (bad translation )
This gives me the number of the month.
By varying “blockWrapper BGO2Z1-19”, “$2” and “1p” I can get out all the data that I need.
Now I need to vary only the “$2” part of awk ‘{print $2}’ . Is there an easy way to “connect” the value now().month+1 for this month and now().month+2 for next month to the awk-statement? I tried some different things but these did not work.
I will have a look. But as far as I can see, they don’t have an API. So my implementation of Groningen Afvalwijzer would be a all-in-one scraper at best.
In the end I decided to create a long list of automations, 2 for each of the dates listed on the page for my postal-code. Once written not so difficult to maintain, in principle only needed once a year.
So, there is no requirement from my side anymore
Appreciate your willingness to try and help though!
Of course, I’ll test it, during the weekend, I don’t work close to home and doing this remotely is too cumbersome.
Also, a nice opportunity for me to learn how to code such a function…
Thanks!!
For my postal code there is no entry for “Klein chemisch afval” and there are no new " Kerstboom"-dates for the rest of the year. That explains the two "Unknown"s I think?
yes indeed, you can either remove them from the config, hide them using customize.yaml or hide the unknown sensors dynamicly via Lovelace. I prefer the last one, because when there are dates for those sensors they automaticly show up
Nice work @Pippyn. Very curious to use it, however I could not get the custom_component to be recognized. I’ve placed the file in custom_components/sensor/cure_afbeheer.py as you mentioned. Added the yaml-bit to my sensor config. Restart gives this in the log:
`
2019-01-12 08:45:36 ERROR (MainThread) [homeassistant.loader] Unable to find component sensor.cure_afvalbeheer
Do I need to allow the custom_component somewhere? Another custom component (alarm) does work actually.
Next to that, I found quite some spelling issues in your script, but I hope that doesn’t influence the workings at this moment in time.
update
Good news, the component works now. The issue was indeed a typo in @Pippyn instruction
Save the file as cure_afbeheer.py in [homeassistant]/config/custom_components/sensor/