Help with invalid HTTP method errors appearing in the logs once every second

Hey guys,

PROBLEM:
I’m having trouble figuring out how to deal with these HTTP errors in my logs. I see roughly one error per second when I ssh into the home assistant server and trail the logs. The error in question looks like this:

2020-08-07 13:04:30 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 275, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method

I’ve searched around and found a few threads on the subject: here’s one, here’s another, and here’s one more. It looks like most of these threads land on the idea that this errors are due to something trying to access my instances via https instead of http.

SETUP:
My setup is pretty simple. I’m running the latest 64bit version Home Assistant OS on an RPI4. I’m using Nabu Casa for external access, and internally I’m not using ssl. My instance of HA is accessible on LAN via http://homeassistant.local:8123.

For add-ons, I’m running the following:


For UI integrations:

For yaml integrations:
doods, generic camera, netgear, google calendar, google cloud tts.

For custom components:
HACS and Node-RED, along with a bunch of themes and front end stuff pulled down from HACS

WHAT I’VE DONE SO FAR:
I’ve trailed the logs via ssh during a restart of HA and have not found any pattern regarding when this error first shows up. Sometimes it first appears when zigbee is setting up, sometimes when netgear is setting up, and sometimes when i think nabu casa is setting up.

Here’s an example of a first appearance after restart with logging set to debug. It pops up when some ZHA and roomba stuff is setting up. The restart began at 11:12:10, so this is about 11 seconds after:

2020-08-07 11:12:21 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=vacuum.roomba_eve, old_state=None, new_state=<state vacuum.roomba_eve=paused; fan_speed_list=['Automatic', 'Eco', 'Performance'], battery_level=34, battery_icon=mdi:battery-30, fan_speed=Eco, software_version=v2.4.13-119, status=Charging, position=(-537, -294, 129), bin_present=True, bin_full=True, friendly_name=Roomba - Status, supported_features=13308 @ 2020-08-07T11:12:21.349621-05:00>>
2020-08-07 11:12:21 DEBUG (MainThread) [zigpy_deconz.api] Read parameter nwk_panid response: [0xfaf2]
2020-08-07 11:12:21 DEBUG (MainThread) [zigpy_deconz.api] Command Command.aps_data_indication (1, 1)
2020-08-07 11:12:21 DEBUG (MainThread) [zigpy_deconz.uart] Send: 0x170a000800010001
2020-08-07 11:12:21 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 275, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
2020-08-07 11:12:21 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.548042845680] Sending {'id': 4, 'type': 'event', 'event': <Event state_changed[L]: entity_id=vacuum.roomba_eve, old_state=None, new_state=<state vacuum.roomba_eve=paused; fan_speed_list=['Automatic', 'Eco', 'Performance'], battery_level=34, battery_icon=mdi:battery-30, fan_speed=Eco, software_version=v2.4.13-119, status=Charging, position=(-537, -294, 129), bin_present=True, bin_full=True, friendly_name=Roomba - Status, supported_features=13308 @ 2020-08-07T11:12:21.349621-05:00>>}

I’ve made sure all add-ons are configured with ssl:false or equivalent.

I don’t have any port forwards set up in my router. UPnP is off in the router.

I have disabled netgear, all generic cameras, and doods and the errors still appear. Typically one a second but sometimes less frequent.

Any help here? Do i start going nuclear and killing integrations until i find the one? This isn’t something related to nabu casa is it?

What is your internal_url: set to under homeassistant?

Good question. Internal is my http://lan_ip:8123. External is empty, per nabu casa standards?

Screen Shot 2020-08-07 at 2.01.13 PM

Ok that’s not your issue then.

i’m completely stumped. though that’s not hard to do seeing as how i don’t really know what i’m doing

The cryptic nature of the error doesn’t help.

Have you tried bumping up your log level?

Debug is the most verbose, right? The logs posted so far are with a debug log level.

I think so, but I am not sure what notset does. https://www.home-assistant.io/integrations/logger/#log-levels

Ill try later. From the docs it sounds like notset will grab even more.

Ran it again with notset logging level. Seems to be roughly right the same stuff in the logs. I do see a couple things that may be something to consider. The error references aiohttp every time and the only other reference to that in the logs is related to nabu casa. Here’s the last two things logged before a restart:

2020-08-07 20:10:04 INFO (MainThread) [snitun.utils.aiohttp_client] AioHTTP snitun client disconnected from: us-east-1.ui.nabu.casa:443
2020-08-07 20:10:04 INFO (MainThread) [snitun.utils.aiohttp_client] AioHTTP snitun client closed

A few seconds later i see it connect to nabu casa, then i see this message:

2020-08-07 20:10:24 DEBUG (MainThread) [hass_nabucasa.cloud_api] Fetched https://remote-sni-api.nabucasa.com/register_instance (200)

Then within the next few lines is the first http error:

2020-08-07 20:10:24 INFO (SyncWorker_5) [homeassistant.loader] Loaded google_cloud from homeassistant.components.google_cloud
2020-08-07 20:10:25 INFO (MainThread) [homeassistant.setup] Setting up tts
2020-08-07 20:10:25 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 275, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
2020-08-07 20:10:25 DEBUG (MainThread) [hass_nabucasa.remote] Waiting for aiohttp runner to come available
2020-08-07 20:10:25 DEBUG (MainThread) [hass_nabucasa.remote] Initializing remote backend
2020-08-07 20:10:25 INFO (MainThread) [homeassistant.setup] Setup of domain cloud took 1.5 seconds
2020-08-07 20:10:25 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=cloud>
2020-08-07 20:10:25 INFO (MainThread) [homeassistant.components.binary_sensor] Setting up binary_sensor.cloud

I’m going to try to sign out and see what happens.

Edit: signing out of nabu casa didn’t change anything. Looking at the logs and it seems a lot of them show up near roomba entries and esphome entries. Here’s an esphome one:

020-08-07 20:37:27 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 275, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: invalid HTTP method
2020-08-07 20:37:27 DEBUG (MainThread) [aioesphomeapi.connection] 192.168.1.10: Got message of type <class 'api_pb2.SensorStateResponse'>: key: 3430275550

I’ll try taking all the esphome stuff out tomorrow and see what happens.

Is it actually creating a problem with your system?

If it ain’t broken, do you need to fix it?

If it IS broken, post a github issue.

Good question. I don’t think so? At least, I don’t notice anything out of the ordinary. Automations are fairly responsive. CPU utilization stays around 5% and never really moves from that. I guess I’m here because I don’t really know if its an issue or not.

Seeing an error in the logs that has 20K hits in a day is just alarming. It got 10K hits in 11 hours last night:

Error handling request
7:05:28 AM – /usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py (ERROR) - message first occurred at August 7, 2020, 8:51:22 PM and shows up 9812 times

Started with a fresh install of the latest 32bit home assistant OS for rpi4. Before loading anything else up I checked the logs and see it filling up with the same http errors. Oh well, i’m not chasing this anymore.

1 Like

Same behavior here, after installing the nginx proxy (I needed to access without SSL from local network because of some Android devices not capable of dealing with it). Something is still accessing the 8123 port via https, but I cannot figure what. I disabled SSL of add-ons, also adjust callback addresses for Nest and Spotify integration but messages still pop up in logs. Very hard to find what is causing this.

Like I said.

Try to enable debug logging for homeassistant.components.http and homeassistant.components.webhook in your logger config.

I struggled for awhile to track down something like this for smart things where I had an invalid webhook or callback registered on their side.

Finally found the source of the problem. I made a DNS rewrite entry in the Adguard addon long time ago to resolve my duckdns.org hostname to an internal IP. Something was apparently accessing this hostname on port 8123 with SSL, which was resolved as internal_I:8123 which was accepting non-encrypted trafic only.