Speedtest Integration No longer loading

As an FYI, I use a bit of a hybrid scheme. I use the built in speedtest integration and have it configured for “manual” mode, but I use the OOKLA speedtest-cli binary to do the actual speedtest. With this is some python code/shell script (that an automation can use) which calls the OOKLA speedtest-cli binary, gets the results and updates the HA speedtest sensors.

You may have to play around with it some to get it work in your setup.

Here is my script:

#!/bin/bash

# Only edit the following setup lines with correct login information

# Mosquitto Broker Details

mqttuser=*****
mqttpassword=*****
mqttbroker=x.x.x.x
mqttport=1883

# DO NOT ALTER ANYTHING BELOW THIS LINE

speedtest_result=$(sudo speedtest --format=json --accept-license)
echo "**********************************************************************************************"
echo $speedtest_result

download=$(echo "$speedtest_result" | jq '.download."bandwidth"')
upload=$(echo "$speedtest_result" | jq '.upload."bandwidth"')
ping_server=$(echo "$speedtest_result" | jq '.ping."latency"')
url_test=$(echo "$speedtest_result" | jq '.result."url"' | tr -d \")
time_run=$(echo "$speedtest_result" | jq '.timestamp' | tr -d \")
echo $download $upload $ping_server $time_run
echo $url_test

download=$(printf %.2f "$((10**3 * $download*8/10**6))e-3")
upload=$(printf %.2f "$((10**3 * $upload*8/10**6))e-3")
ping_server=$(printf %.3f "$ping_server")
echo $download $upload $ping_server $time_run
echo $url_test

speedteststring='{"state":"'"$url_test"'","attributes":{"time_run":"'"$time_run"'","ping":"'"$ping_server"'","download":"'"$download"'","upload":"'"$upload"'"}}'
echo $speedteststring
echo "**********************************************************************************************"

# Publish to MQTT Broker

mosquitto_pub -h $mqttbroker -p $mqttport -u "$mqttuser" -P "$mqttpassword" -t speedtestresult -m "$speedteststring" -r

Enter your creds at the top for the mqtt addon and ip address.
Make sure the script is executable

You first need to install speedtest.
Use these commands

sudo apt-get install gnupg1 apt-transport-https dirmngr
export INSTALL_KEY=379CE192D401AB61
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $INSTALL_KEY
echo "deb https://ookla.bintray.com/debian generic main" | sudo tee  /etc/apt/sources.list.d/speedtest.list
sudo apt-get update

Straight from Ookla site.
The owner of the script is my local debian user.
I run the script via a crontab:

sudo crontab -e

and add these lines to it:

29 * * * * /home/david/speedtest_result.sh 2>&1 >> /var/log/speedtest.log
58 * * * * /home/david/speedtest_result.sh 2>&1 >> /var/log/speedtest.log
0 0 * * * rm /var/log/speedtest.log 2>&1 > /dev/null

so it runs every 30 mins at 29 and 58 past the hour and writes to /var/log/speedtest.log so you can view the current day… it deletes the log at midnight

Then in HA I add MQTT sensors:

  - platform: mqtt
    name: "Speedtest Result"
    state_topic: "speedtestresult"
    value_template: "{{ value_json.state }}"
    json_attributes_topic: "speedtestresult"
    json_attributes_template: "{{ value_json.attributes | tojson }}"

  - platform: average
    name: "Speedtest Download Average"
    entities:
      - sensor.speedtest_download
    start: '{{ 0 }}'
    end: '{{ now() }}'

  - platform: average
    name: "Speedtest Upload Average"
    entities:
      - sensor.speedtest_upload
    start: '{{ 0 }}'
    end: '{{ now() }}'

# Speedtest and Up Time

  - platform: template
    sensors:

# Speedtest Results

      speedtest_download:
        value_template: '{{ states.sensor.speedtest_result.attributes["download"] }}'
        unique_id: speeddown
      speedtest_upload:
        value_template: '{{ states.sensor.speedtest_result.attributes["upload"] }}'
        unique_id: speedup
      speedtest_ping:
        value_template: '{{ states.sensor.speedtest_result.attributes["ping"] }}'
        unique_id: speedping
      speedtest_lastrun:
        value_template: '{{ states.sensor.speedtest_result.attributes["time_run"] }}'
        unique_id: speedtimerun

homeassistant:
  customize:
    package.node_anchors:
      common: &common
        package: "sysmonitor"
    sensor.speedtest_download_average:
      icon: mdi:speedometer
      <<: *common
    sensor.speedtest_upload_average:
      icon: mdi:speedometer
      <<: *common
    sensor.speedtest_download:
      friendly_name: Speedtest Download
      icon: mdi:speedometer
      unit_of_measurement: Mbit/s
      <<: *common
    sensor.speedtest_upload:
      friendly_name: Speedtest Upload
      icon: mdi:speedometer
      unit_of_measurement: Mbit/s
      <<: *common
    sensor.speedtest_ping:
      friendly_name: Speedtest Ping
      icon: mdi:speedometer
      unit_of_measurement: ms
      <<: *common
    sensor.speedtest_result:
      friendly_name: Speedtest Result
      icon: mdi:speedometer
      <<: *common
    sensor.speedtest_lastrun:
      friendly_name: Speedtest Last Run
      icon: mdi:clock
      device_class: timestamp
      <<: *common

I use a custom average component in HACS to give me the average

Then in Lovelace:

      - type: vertical-stack
        cards:
          - type: entities
            title: Internet
            show_header_toggle: false
            entities:
              - entity: sensor.speedtest_download
                secondary_info: last-updated
              - entity: sensor.speedtest_download_average
                secondary_info: last-updated
              - entity: sensor.speedtest_upload
                secondary_info: last-updated
              - entity: sensor.speedtest_upload_average
                secondary_info: last-updated
              - entity: sensor.speedtest_ping
                secondary_info: last-updated
              - entity: sensor.speedtest_lastrun
                format: relative
              - sensor.fritz_netmonitor
          - type: horizontal-stack
            cards:
              - type: gauge
                name: Mbit/s Download
                unit: ' '
                min: 0
                max: 50
                severity:
                  red: 0
                  yellow: 30
                  green: 40
                entity: sensor.speedtest_download
              - type: gauge
                name: Mbit/s Upload
                unit: ' '
                min: 0
                max: 20
                severity:
                  red: 0
                  yellow: 11
                  green: 15
                entity: sensor.speedtest_upload
          - type: custom:apexcharts-card
            graph_span: 24h
            header:
              show: true
              title: Speedtest
            apex_config:
              responsive:
                - breakpoint: 765
                  options:
                    chart:
                      height: 500px
                - breakpoint: 1025
                  options:
                    chart:
                      height: 400px
                - breakpoint: 10000
                  options:
                    chart:
                      height: auto
              stroke:
                curve: smooth
              chart:
                zoom:
                  enabled: true
                toolbar:
                  show: true
                  tools:
                    zoom: true
                    zoomin: true
                    zoomout: true
                    pan: true
                    reset: true
              plotOptions:
                bar:
                  columnWidth: '50%'
              legend:
                showForSingleSeries: true
            series:
              - entity: sensor.speedtest_download
                show:
                  extremas: true
                stroke_width: 3
                fill_raw: last
                name: Download
                type: line
                group_by:
                  func: raw
                  duration: 30min
              - entity: sensor.speedtest_upload
                show:
                  extremas: true
                stroke_width: 3
                fill_raw: last
                name: Upload
                type: line
                group_by:
                  func: raw
                  duration: 30min
              - entity: sensor.speedtest_ping
                show:
                  extremas: true
                stroke_width: 3
                fill_raw: last
                name: Ping
                type: column
                group_by:
                  func: raw
                  duration: 30min

Note the sensors etc are in a package /packages/sysmonitor.yaml

Note also it is possible to directly write the sensor from a bash script but I haven’t tried that and also the mqtt way with a retained message means it is always available on restarts instead of waiting up to 30 mins… I also use mqtt for other things so it was a no brainer to use that.

5 Likes

This looks perfect. I appreciate you sharing it

1 Like

I’m on 2021.4.5 and also have the problem, if I go into the integration options and manually change the server then it starts working again (for a while at least).

And after chaging the server …

My integration has been going fine until last night then it failed here are the logs

Logger: homeassistant.components.speedtestdotnet
Source: components/speedtestdotnet/__init__.py:162
Integration: Speedtest.net (documentation, issues)
First occurred: April 25, 2021, 10:00:07 PM (8 occurrences)
Last logged: 5:00:14 AM

Unexpected error fetching speedtestdotnet data: Unable to connect to servers to test latency.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/speedtest.py", line 1493, in get_best_server
    fastest = sorted(results.keys())[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 173, in _async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 143, in _async_update_data
    return await self.update_method()
  File "/usr/src/homeassistant/homeassistant/components/speedtestdotnet/__init__.py", line 174, in async_update
    return await self.hass.async_add_executor_job(self.update_data)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/speedtestdotnet/__init__.py", line 162, in update_data
    self.api.get_best_server()
  File "/usr/local/lib/python3.8/site-packages/speedtest.py", line 1495, in get_best_server
    raise SpeedtestBestServerFailure('Unable to connect to servers to '
speedtest.SpeedtestBestServerFailure: Unable to connect to servers to test latency.

To get it going again
I noticed it had stopped
went to Configuration → Intehrations → speedtest → options → Reload
Then enabled entities.
HA version is 2021.4.6

1 Like

Had the same error as listed at the top of the thread, HA would not start until this addon was disabled.

Download this program and replace the one in ~/.homeassistant/deps/lib/python3.8/site-packages
It has been updated from 2.1.2 to 2.1.3

Apologies for resurrecting this one but the standard Speedtest integration is still stopping (sometime quickly or within a day or so) [core-2021.5.5] - swapping location in config seems to restarts it ok (for a while again)

any insights on a permanent fix??

Same issue here. I need to reload the integration to have it working again (until next time…)

1 Like

No problem for more than a year, here, with the standard speedtestdotnet

anything special with your setup/config? its a daily failure event here. and not sure how to fix!

Anybody got this fixed yet?

so its now been a few days/a week - usually it dies 1 or twice a day… The change I made was to use AUTO for the integration setting for speedtest and not a specific close located site - since then its back to stability… not sure why auto vs a specific one makes a difference.

spoke too early it seems… Even with auto-detect set, today the sensors went unavailable again (but lasted much much longer).

Nope given up on this now.

So I discovered when the sensor goes unavailable and I manually trigger the speed test it works. As a work around I created an automation to manually trigger the speed test if the sensor moves to this state. Seems to have kept things running for a while now.

alias: Re-Load Spead Test
description: 'Perform a Manual Speed Test When SEnsor goes offline'
trigger:
  - platform: state
    entity_id: sensor.speedtest_download
    to: unavailable
condition: []
action:
  - service: speedtestdotnet.speedtest
mode: single
2 Likes

I’ve ended up setting the update frequency in the integration to run once a week and then created an automation to trigger the service on the frequency I actually wanted. We’ll see how stable this is (annoyingly in the integration you can’t stop it from automatically running - a value of null becomes zero then runs always!)

lol
That PR actually started as a way to tell scan_interval: 0 meant no automatic running, but was refused as such.

EDIT: Ah wait. That one has a UI, so you can disable polling in “Sytem options”.

EDIT2: It actually has also a builtin option (now redundant)
image

Perfect thanks @koying, hadn’t dug into the config options - great help, thanks!

thanks @pcwii - the automation with a small delay added before calling the action has seen it work nicely and kick start the sensor again after it goes into the unavailable state!

1 Like