Speedtest plugin wildly off?

evanrich · September 8, 2018, 7:02pm

I have symmetric 1Gbps fiber (FTTH) from AT&T. I consistantly get ~800Mbps both directions, with sub 10ms latency. However, since starting to use the speedtest component of HA, I find it’s results wildly off from what speedtest shows. See screenshot below. Speedtest.net on the left, and metrics from speedtest sensor. Both are using the same server (AT&T in san francisco) to make sure it was polling the same site

speedtest.net shows 2ms latency, 685 down (downloading a backup from B2 at the moment) and 876 Up, whereas current readings from HA sensor are 427Mbps down. 19Mbps up and 857ms latency.

silvrr · September 8, 2018, 8:34pm

What device is your ha running on? Can it handle a gigabit connection?

evanrich · September 8, 2018, 9:22pm

@silvrr
https://www.supermicro.com/products/system/2U/6027/SYS-6027TR-DTRF.cfm

2x E5-2630v2 (12C/24T total @2.6Ghz)
64GB RAM
10Gbit ISCSI mount for HA config folder.
Local storage is a 4x Intel DCS3610 400GB SSD’s in RAID5

silvrr · September 9, 2018, 2:21am

Have you tried running Speedtest manually from the same machine to see what it can pull for speed?

evanrich · September 10, 2018, 9:37pm

seems it might be the python code. I tested both directly in the container and from the host system itself:

evan@mira-a:/tmp$ ./speedtest-cli --server 5026
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.190.131)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by AT&T (San Francisco, CA) [31.10 km]: 3.756 ms
Testing download speed................................................................................
Download: 852.75 Mbit/s
Testing upload speed................................................................................................
Upload: 392.62 Mbit/s
evan@mira-a:/tmp$ ./speedtest-cli --server 5026
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.190.131)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by AT&T (San Francisco, CA) [31.10 km]: 3.663 ms
Testing download speed................................................................................
Download: 858.11 Mbit/s
Testing upload speed................................................................................................
Upload: 128.95 Mbit/s
evan@mira-a:/tmp$ ./speedtest-cli --server 5026
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.190.131)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by AT&T (San Francisco, CA) [31.10 km]: 3.723 ms
Testing download speed................................................................................
Download: 883.45 Mbit/s
Testing upload speed................................................................................................
Upload: 380.27 Mbit/s

The latency looks better (not 800ms like the GUI, but that could be a temporary thing), but the upload performance is nowhere near what “speedtest.net” is. HA is reporting sometimes 200+ ms latency and as low as 20Mbps speeds. I wonder if it’s the speedtest code itself.

efurban · March 6, 2019, 5:34pm

Have you resolved this issue ? I am experiencing the same here. Testing from another machine with command line shows the correct speed. I may gonna do just that and using MQTT to send the value over.

evanrich · March 17, 2019, 9:06pm

nope. It’s still showing like 700 Down 300 up, despite everything else saying 970/970

efurban · March 22, 2019, 12:11am

ok. I fixed my issue. The culprit was PiHole and / or AdGuard.
Stopping them did not help, I had to remove the add-ons from Hassio completely.
Everything is back to normal. I do plan to setup PiHole using a different VM.

jonmayer · March 25, 2019, 3:52pm

At least for downloading, I was able to fix this script. The problem I saw was that near the end of the test, there were a lot of 0 bit entries in the array of received data. The way the script works it starts a timer when downloading starts and stops a timer when it’s all done. The problem with a bunch of 0 bit data was that it added time to the counter while not receiving more data (essentially it’s done, but the script doesn’t recognize it yet).

YMMV, but i made the following changes to keep track of time better and ignore the time of 0 bits received:

speedtest.py (approx line 1472)

    finished = []
    times = []  # added array

    def consumer(q, request_count):
        while len(finished) < request_count:
            bitstart = timeit.default_timer()  # added timer start
            thread = q.get(True)
            while thread.isAlive():
                thread.join(timeout=0.1)
            finished.append(sum(thread.result))
            times.append(timeit.default_timer() - bitstart) # store timer length
            callback(thread.i, request_count, end=True)

    q = Queue(self.config['threads']['download'])
    prod_thread = threading.Thread(target=producer,
                                   args=(q, requests, request_count))
    cons_thread = threading.Thread(target=consumer,
                                   args=(q, request_count))
    start = timeit.default_timer()
    prod_thread.start()
    cons_thread.start()
    while prod_thread.isAlive():
        prod_thread.join(timeout=0.1)
    while cons_thread.isAlive():
        cons_thread.join(timeout=0.1)

    stop = timeit.default_timer()
    self.results.bytes_received = sum(finished)
    mytime = 0 # added variable to store timer sum
    for x in range(0, len(finished)): # cycle through finished date
        if(finished[x] > 0): # Ignore 0 bit received
            mytime = mytime + times[x]  # sum timers
    self.results.download = (
        (self.results.bytes_received / (mytime)) * 8.0  # change to new timer value for calculation
    )

evanrich · April 4, 2019, 3:31am

Didnt make much of a difference

before

evan@mira-a:/tmp$ ./speedtest-cli --server 5026
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.xxx.yyy)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by AT&T (San Francisco, CA) [30.72 km]: 29.646 ms
Testing download speed................................................................................
Download: 701.09 Mbit/s
Testing upload speed................................................................................................
Upload: 278.23 Mbit/s

after

evan@mira-a:/tmp$ ./speedtest-cli --server 5026
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.xxx.yyy)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by AT&T (San Francisco, CA) [30.72 km]: 18.535 ms
Testing download speed................................................................................
Download: 749.70 Mbit/s
Testing upload speed................................................................................................
Upload: 289.94 Mbit/s

but now im thinking it’s something with that server. When I ran it on its own, without a server:

evan@mira-a:/tmp$ ./speedtest-cli
Retrieving speedtest.net configuration...
Testing from AT&T U-verse (69.222.xxx.yyy)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by CodecCloud Limited (Fremont, CA) [16.76 km]: 19.215 ms
Testing download speed................................................................................
Download: 817.54 Mbit/s
Testing upload speed................................................................................................
Upload: 522.64 Mbit/s

This still pales in comparison though to the browser, which coincidentally goes to the same SF server I got the 200ish Mbps up:

sarkyscouser · July 2, 2019, 12:32pm

Just registering that this is also an issue for me.

I have a FTTC 70/20 connection and via a browser my speed is showing as 72/18 but in the HA sensor it’s showing 14/12.

Ping is also ~50% too high compared to speedtest.net in a browser

I have set up speedtestdotnet to use the same server as my browser to make sure.

tom_l · July 2, 2019, 12:46pm

What hardware are you running HA on?

silvrr · July 2, 2019, 1:02pm

Two good sources of info on this:

sarkyscouser · July 2, 2019, 1:03pm

I’m running it in docker on an AM1 5350 with an Intel i350-T2 card and via gigabit switch and Cat 6.

My windows pc (a new-ish Ryzen PC) via the same network consistently reports different speeds.

I’ve also just installed and run speedtest-cli on the server (outside of docker) and the reported speeds are the same as windows.

Debian Buster I should add is the host OS

tom_l · July 2, 2019, 1:28pm

So it’s the docker container limiting the bandwidth.

sarkyscouser · July 8, 2019, 1:48pm

Unless I can find a way to run speedtest-cli within the container there’s no way knowing if it’s a docker issue, or an issue with the integration.

I’m not the first to report this though so I suspect it’s probably not Docker. And why would the Docker container limit the bandwidth anyway? I’ve not configured it to do that.

max5962 · July 13, 2019, 9:55pm

Hello everyone.
I have the same issue. The culprit is adguard.

Before the fall : pihole
After the fall : adguard

markusressel · December 15, 2020, 10:38pm

I have a similar issue, with the reported speed of home assistant being way off. I run home assistant core using the docker container. I also have a PiHole running on my router which is used as the main DNS provider on all network devices. I am not using AdGuard.

This is a speed test showing the full speed available:

And this is the reported speed of the home assistant plugin using the same server:

(ignore the unavailable and missing data, I restarted a couple of times, but the phenomenon exists on other days as well).

Interestingly, in the evening hours when the internet is more congested, the difference is HUGE, showing f.ex. only 30mbit/s while the speedtest still shows more or less the full speed (250Mbit/s).

What exactly does PiHole do to cause this?
What exactly are the steps to work around this? Specify a custom DNS for Home Assistant?

EDIT:
I just installed the CLI tool by ookla on my server (located at home) as well as my desktop pc and ran some tests. On both cli instances, the numbers seem to match the ones seen in the homeassistant screenshot, beeing lower than what is actually available. However, when I run a speedtest on my desktop pc using firefox, I get the full speed and real ping displayed (the same machine shows lower numbers when using the cli!).

I disabled PiHole for 60 minutes and ran the cli tests again, but nothing changed. So… whats the deal?

EDIT2:
Strike the above. For the CLI testing I used the speedtest-cli Arch package, which is not the package provided by ookla. When using the official CLI tool I also get full speed readings, just like on the website.

So I guess the implementation of the speedtest integration is just using outdated technology then?
I also read this section about inconsistencies with the official site, but I am not convinced the things listed there can explain why I am seeing 30mbit/s instead of 250 while the website still shows 250ish.

ben.mcclure · March 27, 2023, 5:58pm

I debated resurrecting this post, but this is a very prominent result when searching for this issue so I wanted to mention what works for me so that it can hopefully help others.

I just started using this really handy SpeedTest.net add-on today and it fixed it so my speeds are now almost the same as when I run it in my browser! As a bonus, it still uses the original SpeedTest.net integration entities so I didn’t have to change any of my cards.

Actual speed as reported by a browser test: Around 970 Mbps up and down
Official integration reported speeds: Around 400 Mbps down and 200-300 Mbps up
Speeds reported by the new add-on: Around 870 Mbps up and down

I think I can chalk up that extra 100 Mbps to overhead of running it from an HA add-on container with HA running in a VirtualBox VM on my server. I am much happier with the new measurements.