Brussels Public Transportation (STIB-MIVB) integration (API v2)

nxd4n · January 3, 2024, 7:39pm

Hi,
I made a new version of the stib script:

It can use both method with online api data or mixed with GTFS data which provides enhanced data.
As the data is inconsistent, we can’t get good trip details from the API and the sensor name differs for both methods.
To run this script, create the config.yaml file and use python3 sensor.py.
Let me know if this works for you.
If you uses the former script, you will probably end with some unused sensors, to get rid of them, send an empty mqtt message to the config topic.

Dan

glagla · January 5, 2024, 8:45pm

hello, thank you for the new script, it working very well
i made some modification on to make it work better for me
i m in litlle problem with stop names, when i look for a string as stop name, it will get all the stop name that contain the string
i tried to resolve it by using “=” instead of “like” on the quiery,

where_stop_names = " OR ".join(' stop_name = "' + item + '"' for item in STOP_NAMES)

but no succes !!

any idea?

nxd4n · January 5, 2024, 9:33pm

Hi,
this line constructs the query which is send to the stib api.

stop_name like "FOREST"  [or stop_name like 'SAINT-DENIS' or ...]

It adds an OR stop_name like… for each entry.

For a stop name, you get different stop ids, at least 2, one per way.
With the ‘like’, If you put just a part of the stop name, you might get a lot of stop ids. For example if I put “FOREST” I get Forest Centre, Fores Bervoets, Forest National, Chaussée de Forest… But if I put “FOREST-CENTRE” I get only the ids for this particular stop.
If we use a ‘=’, it will search for a stop_name “FOREST” which doesn’t exist, no results. So we need to be sure to put the right stop names in the list in the configuration file.

What did you put as stop name(s)?

glagla · January 8, 2024, 3:12pm

Hello,

I used “DOMAINE” as stop name
but i got “DOMAINE” as result, and also “DGHR DOMAINE MIL.” i dont even know where is it lol… so i got 4 sensors that i dont need

The solution was to tell the script to quesry only stop name “DOMAINE” and the stop name containing “DOMAINE”

i tried to query with code station, but not very strong at python

finaly, i use this, and it seem to work

    where_stop_names = ' OR '.join(f'LOWER(stop_name) = LOWER("{item}") AND LENGTH(stop_name) = {len(item)}' for item in STOP_NAMES)

thank you for all you support

glagla · January 8, 2024, 7:53pm

i add two function that keep the script running if the network is not reacheable

here is my fork

github.com

glagla1/stibgtfs2mqtt/blob/208fad5d76d9a958b6c4f1cd5aec7cdde09a9872/mqttsensor/sensor.py

import logging
from pyStib import BASE_URL, LOGGER, StibData 
import yaml
import random
import asyncio
import json
import pytz
import pygtfs
import os
from sqlalchemy.sql import text
from datetime import datetime, timedelta
import requests
import time
from paho.mqtt import client as mqtt_client

LOGGER = logging.getLogger(__name__)
with open('config.yaml', 'r') as file:
    configuration = yaml.safe_load(file)

This file has been truncated. show original

tebra · May 3, 2024, 3:58pm

Hi again @nxd4n
I try your last version of stibgtfs2mqtt but I get this error:

root@odroidhc4:~/stibgtfs2mqtt-main/mqttsensor# python3 sensor.py 
{'71': {'pointid': '3520', 'lineid': '71', 'passingtimes': [{'destination': {'fr': 'DE BROUCKERE', 'nl': 'DE BROUCKERE'}, 'expectedArrivalTime': '2024-05-03T17:52:00+02:00', 'lineId': '71'}, {'destination': {'fr': 'DE BROUCKERE', 'nl': 'DE BROUCKERE'}, 'expectedArrivalTime': '2024-05-03T17:57:00+02:00', 'lineId': '71'}]}}
{'5': {'pointid': '8232', 'lineid': '5', 'passingtimes': [{'destination': {'fr': 'HERRMANN-DEBROUX', 'nl': 'HERRMANN-DEBROUX'}, 'expectedArrivalTime': '2024-05-03T17:54:00+02:00', 'lineId': '5'}, {'destination': {'fr': 'HERRMANN-DEBROUX', 'nl': 'HERRMANN-DEBROUX'}, 'expectedArrivalTime': '2024-05-03T17:58:00+02:00', 'lineId': '5'}]}}
{'72': {'pointid': '3546', 'lineid': '72', 'passingtimes': [{'destination': {'fr': 'ADEPS', 'nl': 'ADEPS'}, 'expectedArrivalTime': '2024-05-03T18:41:00+02:00', 'lineId': '72'}, {'expectedArrivalTime': '2024-05-03T17:52:00+02:00', 'lineId': '72'}]}}
{'72': {'pointid': '3556', 'lineid': '72', 'passingtimes': [{'destination': {'fr': 'ULB', 'nl': 'ULB'}, 'expectedArrivalTime': '2024-05-03T18:16:00+02:00', 'lineId': '72'}, {'expectedArrivalTime': '2024-05-03T17:52:00+02:00', 'lineId': '72'}]}, '71': {'pointid': '3556', 'lineid': '71', 'passingtimes': [{'destination': {'fr': 'DE BROUCKERE', 'nl': 'DE BROUCKERE'}, 'expectedArrivalTime': '2024-05-03T17:55:00+02:00', 'lineId': '71'}, {'destination': {'fr': 'DE BROUCKERE', 'nl': 'DE BROUCKERE'}, 'expectedArrivalTime': '2024-05-03T18:00:00+02:00', 'lineId': '71'}]}}
{'5': {'pointid': '8231', 'lineid': '5', 'passingtimes': [{'destination': {'fr': 'ERASME', 'nl': 'ERASMUS'}, 'expectedArrivalTime': '2024-05-03T17:52:00+02:00', 'lineId': '5'}, {'destination': {'fr': 'ERASME', 'nl': 'ERASMUS'}, 'expectedArrivalTime': '2024-05-03T17:58:00+02:00', 'lineId': '5'}]}}
Retrieving STIB-MIVB realtime data
{"line_ids": ["72", "5", "71"], "waiting_times": {"3556": {"72": {"pointid": "3556", "lineid": "72", "passingtimes": [{"destination": {"fr": "ULB", "nl": "ULB"}, "expectedArrivalTime": "2024-05-03T18:16:00+02:00", "lineId": "72"}, {"expectedArrivalTime": "2024-05-03T17:52:00+02:00", "lineId": "72"}]}, "71": {"pointid": "3556", "lineid": "71", "passingtimes": [{"destination": {"fr": "DE BROUCKERE", "nl": "DE BROUCKERE"}, "expectedArrivalTime": "2024-05-03T17:55:00+02:00", "lineId": "71"}, {"destination": {"fr": "DE BROUCKERE", "nl": "DE BROUCKERE"}, "expectedArrivalTime": "2024-05-03T18:00:00+02:00", "lineId": "71"}]}}, "3546": {"72": {"pointid": "3546", "lineid": "72", "passingtimes": [{"destination": {"fr": "ADEPS", "nl": "ADEPS"}, "expectedArrivalTime": "2024-05-03T18:41:00+02:00", "lineId": "72"}, {"expectedArrivalTime": "2024-05-03T17:52:00+02:00", "lineId": "72"}]}}, "8231": {"5": {"pointid": "8231", "lineid": "5", "passingtimes": [{"destination": {"fr": "ERASME", "nl": "ERASMUS"}, "expectedArrivalTime": "2024-05-03T17:52:00+02:00", "lineId": "5"}, {"destination": {"fr": "ERASME", "nl": "ERASMUS"}, "expectedArrivalTime": "2024-05-03T17:58:00+02:00", "lineId": "5"}]}}, "3520": {"71": {"pointid": "3520", "lineid": "71", "passingtimes": [{"destination": {"fr": "DE BROUCKERE", "nl": "DE BROUCKERE"}, "expectedArrivalTime": "2024-05-03T17:52:00+02:00", "lineId": "71"}, {"destination": {"fr": "DE BROUCKERE", "nl": "DE BROUCKERE"}, "expectedArrivalTime": "2024-05-03T17:57:00+02:00", "lineId": "71"}]}}, "8232": {"5": {"pointid": "8232", "lineid": "5", "passingtimes": [{"destination": {"fr": "HERRMANN-DEBROUX", "nl": "HERRMANN-DEBROUX"}, "expectedArrivalTime": "2024-05-03T17:54:00+02:00", "lineId": "5"}, {"destination": {"fr": "HERRMANN-DEBROUX", "nl": "HERRMANN-DEBROUX"}, "expectedArrivalTime": "2024-05-03T17:58:00+02:00", "lineId": "5"}]}}}}
Traceback (most recent call last):
  File "/root/stibgtfs2mqtt-main/mqttsensor/sensor.py", line 380, in <module>
    init(clean)
  File "/root/stibgtfs2mqtt-main/mqttsensor/sensor.py", line 257, in init
    setConfig(attribute)
  File "/root/stibgtfs2mqtt-main/mqttsensor/sensor.py", line 333, in setConfig
    mqttSend(c,topic,True)
  File "/root/stibgtfs2mqtt-main/mqttsensor/sensor.py", line 361, in mqttSend
    client = connect_mqtt()
  File "/root/stibgtfs2mqtt-main/mqttsensor/sensor.py", line 357, in connect_mqtt
    client.connect(mqtt_server, int(mqtt_port))
  File "/usr/local/lib/python3.9/dist-packages/paho/mqtt/client.py", line 914, in connect
    return self.reconnect()
  File "/usr/local/lib/python3.9/dist-packages/paho/mqtt/client.py", line 1044, in reconnect
    sock = self._create_socket_connection()
  File "/usr/local/lib/python3.9/dist-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
    return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
  File "/usr/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

It works now, I was using a wrong mqtt_server adress

tebra · May 3, 2024, 5:22pm

Hi again @nxd4n
There is a small typo in the example of config.yaml file.
You have to add a ‘/’ at the end of mqtt_topic: ‘homeassistant/sensor’
If not the sensorstib1234567890 sensor is not published in /homeassistant/sensor directory and so not auto discovered by home assistant.

DaMan · November 23, 2024, 4:43pm

Thanks a lot for developing this tool. I installed it after setting up MQTT in my Home Assistant (I have a docker version running on my raspberry pi) and everything seems to be working ( the tmux suggestion was nice to be able to run the python script continuously).

I remember from an earlier version with API V1 that it was possible to get the arrival time of the 2 next tram/bus/metro at a stop. Is this not possible anymore?

I didn’t manage to automatically install the dependencies using the requirements.txt but used the pip install manually. It installed the latest version 2.1.0 of paho-mqtt and I had to update the line

client = mqtt_client.Client(client_id)

to:

client = mqtt_client.Client(mqtt_client.CallbackAPIVersion.VERSION1, client_id)

After that, the script was running and I could add the MQTT sensors to my home assistant dashboard

nxd4n · November 25, 2024, 8:43am

Hi,
Unfortunately, the new API doesn’t have the easy access to the next 2 arrivals of a line on a given stop. The way to work with the API has completely changed.
But you can find the next passing times in the attributes of the created sensor of the line of the stop.