Gas Pricing Plus Oil

Share your Projects!
1

image

Seeing how gas prices are going up I decided to start pulling in the info via the Scrape integration.
https://www.home-assistant.io/integrations/scrape/

I leveraged the Gas Buddy site for fuel prices.
https://www.gasbuddy.com

If you search the site for your zip code or town you can actually select the gas station you would like in the drop down such as Costco, RaceTrac etc…

Then just leverage chromes F12 developer function. You can then click the element selector button and hover over elements in the page.

image

Hover over the item you would like to capture data from on the left and select it. The corresponding code on the right will be highlighted.

image
image1071×318 35.4 KB

Next, right click the code section on the right and select “Copy Selector”
image

This string will be what you use for the select in the Scrape sensor.

- platform: scrape
    resource: Page Url
    name: Name of your sensor that you choose
    select: Text string from the chrome selector

I also pulled oil pricing from another site. You can use whatever site you would like for that info.

9 Likes
2

Just wanted to say thank you for this. I was struggling with the scrape sensor to pull in gas prices into my dashboard, and your guide worked perfectly the first time around!

1 Like
3

Jason Walter THANK YOU!! Amazing work. What are some other links you have scraped from?

4

Thanks for this! Having issues with the select: text.

Can you give me examples of your working select?

1 Like
5

Sure,

- platform: scrape
    resource: "https://oilprice.com/oil-price-charts/#WTI-Crude"
    name: Oil Per Barrel
    select: '#pagecontent > div.siteContainer.oilprices > div.oilprices__centercolumn > table:nth-child(1) > tbody.row_holder > tr:nth-child(1) > td.last_price'
    scan_interval: 3600
1 Like
6

That’s it so far. Just haven’t had anything else to scrape. I really try to leverage API’s but did not find one for this and wanted the data quickly.

2 Likes
7

I love every part of this, thanks to OP for the thread.

1 Like
8

Everyone loves it with the price of fuel right now! :slight_smile:

9

Can someone please post the text of a complete working sensor?
I am missing something between resource and select, I believe.

Thank you

10

An example was posted by myself further up the thread.

Not sure what else you need besides that, that is a scrape sensor.

12

Thank you so much for putting this together! Really made things easy to setup.

1 Like
13

I’m getting the following error in validation when I try to add this to my configuration.yaml

Invalid config for [sensor.scrape]: string value is None for dictionary value @ data['select']. Got None. (See ?, line ?).
Invalid config for [sensor.scrape]: string value is None for dictionary value @ data['select']. Got None. (See ?, line ?).

Any thoughts?

14

I use a split config setup and do not add stuff like this to my config file so I’m not much help.

Leveraging your config file you will most likely need to add it like this.

sensor:
  - platform: scrape
    resource: "https://oilprice.com/oil-price-charts/#WTI-Crude"
    name: Oil Per Barrel
    select: '#pagecontent > div.siteContainer.oilprices > div.oilprices__centercolumn'
    scan_interval: 3600

Please read the scrape integration documentation.

15

Thanks for that demo code - my select didn’t have the ’ ’ around the content. That did the trick!

16

For you information, a specific integration has been build to get similar fuel and oil data for several countries in Europe: Belgium, France, Luxembourg, Netherlands, Italy and Germany: https://github.com/myTselection/Carbu_com
Maybe some of the setup of this integration can be re-used to build something similar for other countries. On request I may also be able to further extend that extension to cover extra countries.

17

If you have time to extend for U.S. I’m sure others would appreciate very much. I do not currently have the time to invest.

18

I could take a look into it. Any good site available with daily actual prices?

19

Only one I know of that usually stays up to date. https://www.gasbuddy.com/

20

I’ve tried to integrate it, but it seems protected with cloudflare human control now. so far, I didn’t find a workaround…

21

Hey guys, I’m having trouble getting this sensor to work. I’m not getting any errors, but the entity is not populating:

sensor:
  - platform: scrape
    resource: "https://www.gasbuddy.com/station/34888"
    name: Costco Gas
    select: '#root > div > div.Page-module__content___196kn.Page-module__padded___3hQ0U > div > div > div:nth-child(2) > div.grid__column___nhz7X.grid__desktop10___2T18R > div:nth-child(1) > div.grid__column___nhz7X.grid__desktop8___38Y4U > div:nth-child(3) > div > div.GasPriceCollection-module__collectionContainer___29Ngz > div:nth-child(2) > div:nth-child(1) > span'
    scan_interval: 3600
1 Like