kingo55
December 30, 2021, 12:04pm
1
I recently upgraded Home Assistant to the latest 2021.12.x version and all of a sudden my once-stable Kodi integration stopped working.
I’ve tried everything from:
Rolling back to an earlier HA version
Removing and re-adding the Kodi integration (both via YAML and via the UI)
Toggling the settings within Kodi (such as re-enabling HTTP access, UPnP, removing the username/password etc)
I can access Kodi as usual from my phone and laptop. And I can see connections to Kodi opening for both Websocket and HTTP ports from the HASS IP.
All the integration reports is “cant_connect” during the configuration flow.
Reading the config flow’s code, it looks like it’s excepting out at CannotConnect - however it does connect in order to:
Open the port to Kodi on the remote instance
Verify my credentials are correct - when incorrect, it tells me
tom_l
December 30, 2021, 12:34pm
2
Try this:
Delete the Kodi integration.
Restart Home Assistant
Go to Configuration / Devices & Services and delete any “restored” devices or entities assosciated with Kodi (you may have to turn off filtering, top right of the lists).
Restart.
Try adding the Kodi integration again.
If that does not work you may have to dive into the .storage registry to remove any vestige Kodi entities or devices after deleting and restarting.
kingo55
December 31, 2021, 2:05am
3
Thanks. I gave that a shot but it’s still not getting past the config flow.
There aren’t any vestigial Kodi elements in my .storage/core.* files either. I’m beginning to wonder if it’s a deeper issue with the HA integration not communicating properly with Kodi and vice versa.
tom_l
December 31, 2021, 2:55am
4
It’s working fine for me at the moment (2021.12.7).
EDIT: Ah, though I am still on Kodi v18.9 (Leia).
kingo55
December 31, 2021, 11:08am
5
Maybe something’s up there. Kodi 19 has been awfully unstable with my plugins, too. I’m running:
kodi 19.3-2
I have dug a bit deeper and found that the underlying library used in Home Assistant for Kodi (pykodi) is complaining about a duplicate Content-Length HTTP header:
jsonrpc_base.jsonrpc.TransportError: ("Error calling method 'JSONRPC.Ping': Transport Error", ClientResponseError(RequestInfo(url=URL('http://192.168.0.100:8080/jsonrpc'), method='POST', headers=<CIMultiDictProxy('Host': '192.168.0.100:8080', 'Content-Type': 'application/json', 'Accept': 'application/json-rpc', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Python/3.9 aiohttp/3.8.1', 'Authorization': 'Basic ******', 'Content-Length': '104')>, real_url=URL('http://192.168.0.100:8080/jsonrpc')), (), status=400, message='Duplicate Content-Length'))
Sure enough, there are two Content-Length headers on the JSON-RPC responses:
No idea how to fix this, but it seems to be a Kodi problem. If anyone else stumbles across this problem, see if you can reproduce and report back here:
opened 08:54AM - 31 Dec 21 UTC
Triage: Needed
## Bug report
### Describe the bug
Here is a clear and concise description of … what the problem is:
Responses from the JSON-RPC include a duplicate `Content-Length` HTTP header that is causing some useragents to throw exceptions:
<img width="787" alt="Screen Shot 2021-12-31 at 19 14 58" src="https://user-images.githubusercontent.com/2361388/147811799-bcba7237-5fc6-4047-8553-678c7dcf58c9.png">
I am running Kodi 19.3-2 on both my Linux desktop (5.15.10) and Mac OS (20.6.0).
## Expected Behavior
Here is a clear and concise description of what was expected to happen:
The JSON-RPC server should only include one `Content-Length` header in responses, like tested on Mac OS:
<img width="769" alt="Screen Shot 2021-12-31 at 19 43 14" src="https://user-images.githubusercontent.com/2361388/147813031-797ae1df-011c-40be-8caa-a2750622be3c.png">
## Actual Behavior
There are two `Content-Length` headers in the HTTP response. This breaks some useragents from using the JSON-RPC data, such as Home Assistant's Kodi integration (will see if I can grab you a traceback for that too):
<img width="787" alt="Screen Shot 2021-12-31 at 19 14 58" src="https://user-images.githubusercontent.com/2361388/147811799-bcba7237-5fc6-4047-8553-678c7dcf58c9.png">
## Possible Fix
Between these two functions, do we have a double-up of `Content-Length` Header logic?:
1. https://github.com/xbmc/xbmc/blob/a80d1293b0579bbed7dd7d20304ba34a20519f78/xbmc/utils/HttpResponse.cpp#L89
2. https://github.com/xbmc/xbmc/blob/7478f983b67dc1e3b09e061acf2eaa675b0f0170/xbmc/network/WebServer.cpp#L445
### To Reproduce
Steps to reproduce the behavior:
1. Running Kodi 19.3-2 on Linux (reinstalled multiple times)
### Debuglog
### Screenshots
Here are some links or screenshots to help explain the problem:
## Additional context or screenshots (if appropriate)
Here is some additional context or explanation that might help:
```
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 898, in start
message, payload = await protocol.read() # type: ignore[union-attr]
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/streams.py", line 616, in read
await self._waiter
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_proto.py", line 213, in data_received
messages, upgraded, tail = self._parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadHttpMessage: 400, message='Duplicate Content-Length'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/jsonrpc_async/jsonrpc.py", line 33, in send_message
response = await self._request(data=message.serialize())
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client.py", line 559, in _request
await resp.start(conn)
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 900, in start
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Duplicate Content-Length', url=URL('http://192.168.0.100:8080/jsonrpc')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/pykodi/kodi.py", line 164, in ping
response = await self._server.JSONRPC.Ping()
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/jsonrpc_async/jsonrpc.py", line 35, in send_message
raise TransportError('Transport Error', message, exc)
jsonrpc_base.jsonrpc.TransportError: ("Error calling method 'JSONRPC.Ping': Transport Error", ClientResponseError(RequestInfo(url=URL('http://192.168.0.100:8080/jsonrpc'), method='POST', headers=<CIMultiDictProxy('Host': '192.168.0.100:8080', 'Content-Type': 'application/json', 'Accept': 'application/json-rpc', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Python/3.9 aiohttp/3.8.1', 'Authorization': 'Basic ******', 'Content-Length': '104')>, real_url=URL('http://192.168.0.100:8080/jsonrpc')), (), status=400, message='Duplicate Content-Length'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/test.py", line 19, in <module>
asyncio.run(test())
File "/usr/local/Cellar/[email protected] /3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/Cellar/[email protected] /3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/robertkingston/Documents/pykodi-testing/test.py", line 11, in test
await kodi.ping()
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/pykodi/kodi.py", line 170, in ping
raise CannotConnectError from error
pykodi.kodi.CannotConnectError
```
### Your Environment
Used Operating system:
- [ ] Android
- [ ] iOS
- [ ] tvOS
- [x] Linux
- [ ] OSX
- [ ] Windows
- [ ] Windows UWP
- Operating system version/name: Linux 5.15.10 / Arch
- Kodi version: 19.3-2
*note: Once the issue is made we require you to update it with new information or Kodi versions should that be required.
Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.*
Hello there!Content-Lenght HTTP Header on kodi webserver side.
kingo55
February 19, 2022, 11:37pm
7
Sorry to hear that @hexxotest - I’ve posted bug reports to both Kodi and the upstream library’s developers:
opened 05:49AM - 23 Jan 22 UTC
## Expected behaviour
If the response headers contain a `Content-Length` head… er then MHD doesn't add one.
## Actual behaviour
When a response already contains a `Content-Length` header, MHD adds another one, causing some useragents to reject the response.
For example, when a non-compliant duplicate header exists, Python's `aiohttp` library, for example, raises an exception:
```
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 898, in start
message, payload = await protocol.read() # type: ignore[union-attr]
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/streams.py", line 616, in read
await self._waiter
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_proto.py", line 213, in data_received
messages, upgraded, tail = self._parser.feed_data(data)
File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadHttpMessage: 400, message='Duplicate Content-Length'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/jsonrpc_async/jsonrpc.py", line 33, in send_message
response = await self._request(data=message.serialize())
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client.py", line 559, in _request
await resp.start(conn)
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 900, in start
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Duplicate Content-Length', url=URL('http://192.168.0.100:8080/jsonrpc')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/pykodi/kodi.py", line 164, in ping
response = await self._server.JSONRPC.Ping()
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/jsonrpc_async/jsonrpc.py", line 35, in send_message
raise TransportError('Transport Error', message, exc)
jsonrpc_base.jsonrpc.TransportError: ("Error calling method 'JSONRPC.Ping': Transport Error", ClientResponseError(RequestInfo(url=URL('http://192.168.0.100:8080/jsonrpc'), method='POST', headers=<CIMultiDictProxy('Host': '192.168.0.100:8080', 'Content-Type': 'application/json', 'Accept': 'application/json-rpc', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Python/3.9 aiohttp/3.8.1', 'Authorization': 'Basic ******', 'Content-Length': '104')>, real_url=URL('http://192.168.0.100:8080/jsonrpc')), (), status=400, message='Duplicate Content-Length'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/robertkingston/Documents/pykodi-testing/test.py", line 19, in <module>
asyncio.run(test())
File "/usr/local/Cellar/[email protected] /3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/Cellar/[email protected] /3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/robertkingston/Documents/pykodi-testing/test.py", line 11, in test
await kodi.ping()
File "/Users/robertkingston/Documents/pykodi-testing/venv/lib/python3.9/site-packages/pykodi/kodi.py", line 170, in ping
raise CannotConnectError from error
pykodi.kodi.CannotConnectError
```
## Further details
I discovered this behaviour downstream in XBMC which uses a MHD version `0x00097500` - details here: https://github.com/xbmc/xbmc/issues/20759
We think it may be due to this commit https://github.com/Karlson2k/libmicrohttpd/commit/4820d93bc91eea924dcad29e879f31d8d28649a1.
As the resolution to this issue is external to Kodi, I thought I should raise this in the correct upstream repo.
It’s been over a month and no traction.
I’m not familiar with the languages MHD or Kodi are written in to submit a fix - If more users are getting this issue, it might give the issue enough gravity for a fix from someone who knows what they’re doing.
I could also raise this issue with the aiohttp Python library - technically the exception is being raised by this library, meanwhile other useragents (e.g. Chrome/FF) are handling the duplicate Content-Length headers just fine.
kingo55
February 21, 2022, 11:52am
8
Update : With some gentle prodding, there’s a pull request now with a fix! Let’s hope it makes it for the next Kodi release…
xbmc:master ← Karlson2k:MHD_fix_03
opened 06:00PM - 20 Feb 22 UTC
I started this PR as a fix for https://github.com/xbmc/xbmc/issues/20759
Whil… e fixing duplicated `Content-Length:` headers, I discovered that Kodi uses libmicrohttpd (MHD) API incorrectly. Responses for `HEAD` requests are formed as responses with zero-length body. This is incorrect.
MHD API uses the same responses for `GET` and `HEAD` requests. When response is used to answer `GET` request, it is completely sent, including body. When the same response is used to answer `HEAD` request, response body is not sent (even if body has non-zero length).
According to HTTP RFCs, reply to `HEAD` requests should be exactly the same as reply for the `GET` requests, but reply body must not be sent.
In this PR I fixed improper use of MHD API.
This PR is fixing several things:
* Response for `HEAD` requests are handled in exactly the same way as responses for `GET` requests (as per RFC), now response headers for `HEAD` request are **exactly** the same as in response for similar `GET` request.
* Fixed wrong `Content-Length: 0` header always present in replies for `HEAD` request. This may fix a number of problems in clients that use Kodi HTTP interface.
* Fixed duplicated `Content-Length:` headers in all responses.
* Unrelated small optimisation suggested by @lrusak
1 Like
Kodi 19.4 has been released with a fix to the headers. I’ve just confirmed it’s working now. Anyone else should try upgrading if they get this issue.
1 Like
