(Digest) Authentication with scrape sensor not working since Homeassistant 0.117.0

The problem

Scrape sensor is giving error since home assistant version 0.117.0

Environment

  • Home Assistant Core release with the issue: 0.117.0
  • Last working Home Assistant Core release (if known): 0.116.4
  • Operating environment (OS/Container/Supervised/Core): Debian 10 Homeassistant Core in venv, Python 3.9
  • Integration causing this issue: scrape
  • Link to integration documentation on our website: https://www.home-assistant.io/integrations/scrape/

Problem-relevant configuration.yaml

sensor: - platform: scrape resource: “https://myinternalserverurl.local:8443/myapi.html?part=status” name: My Service verify_ssl: false username: !secret my_username password: !secret my_password authentication: digest select: “test status client[name=‘MyName’] connection”

Traceback/Error logs

2021-01-02 17:40:39 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up scrape platform for sensor
Traceback (most recent call last):
  File "/opt/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 199, in _async_setup_platform
    await asyncio.shield(task)
  File "/opt/homeassistant/lib/python3.9/site-packages/homeassistant/components/scrape/sensor.py", line 82, in async_setup_platform
    await rest.async_update()
  File "/opt/homeassistant/lib/python3.9/site-packages/homeassistant/components/rest/data.py", line 54, in async_update
    response = await self._async_client.request(
  File "/opt/homeassistant/lib/python3.9/site-packages/httpx/_client.py", line 1371, in request
    response = await self.send(
  File "/opt/homeassistant/lib/python3.9/site-packages/httpx/_client.py", line 1406, in send
    response = await self._send_handling_auth(
  File "/opt/homeassistant/lib/python3.9/site-packages/httpx/_client.py", line 1438, in _send_handling_auth
    request = await auth_flow.__anext__()
  File "/opt/homeassistant/lib/python3.9/site-packages/httpx/_auth.py", line 92, in async_auth_flow
    request = next(flow)
  File "/opt/homeassistant/lib/python3.9/site-packages/httpx/_auth.py", line 115, in auth_flow
    yield self._func(request)
  File "/opt/homeassistant/lib/python3.9/site-packages/requests/auth.py", line 292, in __call__
    r.register_hook('response', self.handle_401)
AttributeError: 'Request' object has no attribute 'register_hook'

Additional information

Tried the sensor without ssl (https) and same error appeared.

Last working homeassistant was 0.116.4, first non-working is 0.117b0, problem still there at version 2020.12.2 of homeassistant.

When removing authentication from the local webserver and removing username and password and authentication from the sensor it is working as expected.

When using below shell command the sensor is working as expected:
curl --cacert /etc/ssl/private/newca.crt -X get --digest --user myusernamehere:mypasswordhere@https://myinternalserverurl.local:8443/myapi.html?part=status"

How to debug?

Best regards,
Donald.

Also posted in:

Same issue here running on Home Assistant OS 5.11 and core version core-2021.2.3.
Did you find any workaround other than disabling authentication?
Thanks

Unfortunately not…