I’m trying to include a sensor to show the departure time of the local transport, scraping the website www.9292.nl for a table and then showing the next departures. I use a python script, use the command_line sensor to collect the information and then template sensor to display (because command_line json cannot generate attributes.
Problem is that this script sometimes results in command failed error - but I cannot understand why. If I run the command in a terminal it generates the expected result. But maybe the python script is not ‘clean’ enough? Who can help? Python script and configuration.yaml included below.
import requests, json, re, sys
from datetime import datetime, timedelta
url = "https://9292.nl/berkel-en-rodenrijs/metrostation-berkel-westpolder"
page = requests.get(url)
from bs4 import BeautifulSoup
soup = BeautifulSoup(page.content, 'html.parser')
data = []
table = soup.find('table', attrs={'class':'table table-striped responsive-table m-t-10'})
table_body = table.find('tbody')
rows = table_body.find_all('tr')
for row in rows:
cols = row.find_all('td')
cols = [ele.text.strip() for ele in cols]
data.append([ele for ele in cols if ele]) # Get rid of empty values
# remove all fourth elements
for row in data:
destination = (row[1].split("\n"))
row.append(destination[0])
row.append(destination[1])
#check for delay in first cell
if row[0].count("\n\n"):
time = row[0].split("\n\n")
row[0] = time[0]
row.append(time[1])
try:
delay = int(re.search(r'\d+', time[1]).group())
except:
delay = 0
else:
row.append(0)
delay = 0
row[0] += ":00"
#remove unnecessary cells
del row[1]
del row[1]
del row[1]
#Departure time - to calculate minutes until departure...
s1 = row[0]
today_date = datetime.now().strftime("%d %b %Y")
scheduled_time = datetime.strptime(s1, '%H:%M:%S')
tdelta = datetime.combine(datetime.today(), scheduled_time.time()) + timedelta(0, delay * 60) - datetime.now()
row.append(str(round(tdelta.seconds / 60)) + "min")
#The relevant entry deduced from argument
selection = int( sys.argv[1] )
#Keep the relevant entry
selected_row = data[selection]
#Clear up direction
selected_row[2] = selected_row[2].replace("Richting ","")
#setting up the output
departures = {"deptime": selected_row[0] , "type": selected_row[1], "dir": selected_row[2], "tbl": selected_row[4]}
#print the output
print(json.dumps(departures))
Inclusion in configuration.yaml:
- platform: command_line
name: First_departure
command: python3 /config/python_scripts/departures.py 0
scan_interval: 60
value_template: '{{ value_json }}'
And to generate the template sensor:
- platform: template
sensors:
first_dep:
friendly_name: "First departure"
value_template: >
{% set state = states("sensor.first_departure").split(",") %}
{% set dest = state[2] %}
{% set dest = dest|replace("'","") %}
{% set dest = dest|replace("dir: ","") %}
{% set tbl = state[3]|replace("'tbl':","") %}
{% set tbl = tbl|replace("'","") %}
{% set tbl = tbl|replace("}","") %}
{% set tbl = tbl|replace(" ","") %}
{% set time = state[0]|replace("{'deptime':","") %}
{% set time = time|replace("'","") %}
{% set time = time|replace(" ","") %}
{% set time = time[0:5] %}
To{{ dest }} leaving in {{ tbl }} ({{ time }})