pycognito.exceptions.TokenVerificationException: Your 'id_token' token could not be verified

pergola.fabio · May 10, 2021, 7:52am

Just rebooted my HassOs , but its now stuck on the cloud connection?? is nabucasa down??

2021-05-10 11:48:32 ERROR (MainThread) [hass_nabucasa.iot] Unexpected error
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/iot_base.py", line 108, in connect
    await self._handle_connection()
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/iot_base.py", line 147, in _handle_connection
    await self.cloud.auth.async_check_token()
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 172, in async_check_token
    await self._async_renew_access_token()
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 199, in _async_renew_access_token
    await self.cloud.run_executor(cognito.renew_access_token)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 636, in renew_access_token
    self._set_tokens(refresh_response)
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 708, in _set_tokens
    self.verify_token(tokens["AuthenticationResult"]["IdToken"], "id_token", "id")
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 254, in verify_token
    raise TokenVerificationException(
pycognito.exceptions.TokenVerificationException: Your 'id_token' token could not be verified.
2021-05-10 11:48:33 ERROR (MainThread) [hass_nabucasa.remote] Unexpected error in Remote UI loop
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/remote.py", line 379, in _certificate_handler
    if not await self.load_backend():
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/remote.py", line 137, in load_backend
    resp = await cloud_api.async_remote_register(self.cloud)
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/cloud_api.py", line 16, in check_token
    await cloud.auth.async_check_token()
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 172, in async_check_token
    await self._async_renew_access_token()
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 199, in _async_renew_access_token
    await self.cloud.run_executor(cognito.renew_access_token)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 636, in renew_access_token
    self._set_tokens(refresh_response)
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 708, in _set_tokens
    self.verify_token(tokens["AuthenticationResult"]["IdToken"], "id_token", "id")
  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 254, in verify_token
    raise TokenVerificationException(
pycognito.exceptions.TokenVerificationException: Your 'id_token' token could not be verified.

Hellis81 · May 10, 2021, 7:58am

Works fine for me.

pergola.fabio · May 10, 2021, 8:09am

its indeed working again after a new reboot, maybe a small glitch

pergola.fabio · May 10, 2021, 12:36pm

hmm, its happening again

pergola.fabio · May 10, 2021, 7:31pm

I only have the issue it seems when I do a full HassOs reboot… Then i see the error, if i do a restart service afterwards, the error is gone…

No idea what’s causing this

sylviepipi · May 13, 2021, 6:31pm

same problem here
rebooting the host, no cloud access… when rebooting via server manager i don’t have any problems

pergola.fabio · May 13, 2021, 8:26pm

Yeah, i created issues on GitHub, but no response yet from developers

pergola.fabio · May 14, 2021, 8:26am

Hey @sylviepipi …
Can you also post your log? Is it the exact same ? Can you maybe post your full error log upon HA restart, maybe we are using another same component that is causing this issue…

Because it’s strange no more persons are reporting it

sylviepipi · May 14, 2021, 3:21pm

2021-05-15 01:10:32 ERROR (MainThread) [hass_nabucasa.iot] Unexpected error

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/iot_base.py", line 108, in connect

    await self._handle_connection()

  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/iot_base.py", line 147, in _handle_connection

    await self.cloud.auth.async_check_token()

  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 172, in async_check_token

    await self._async_renew_access_token()

  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/auth.py", line 199, in _async_renew_access_token

    await self.cloud.run_executor(cognito.renew_access_token)

  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run

    result = self.fn(*self.args, **self.kwargs)

  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 636, in renew_access_token

    self._set_tokens(refresh_response)

  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 708, in _set_tokens

    self.verify_token(tokens["AuthenticationResult"]["IdToken"], "id_token", "id")

  File "/usr/local/lib/python3.8/site-packages/pycognito/__init__.py", line 254, in verify_token

    raise TokenVerificationException(

pycognito.exceptions.TokenVerificationException: Your 'id_token' token could not be verified.```

pergola.fabio · May 14, 2021, 3:43pm

Indeed the same error

pergola.fabio · May 14, 2021, 9:08pm

Hey , can you also post your full log after a HassOs restart? Not only the error ?

sylviepipi · May 15, 2021, 5:29pm

I was able to solve my issue…

I’m running hassio on vmware ESXi. For some reason the time on the host was incorrect (12h before) and set on hassio during the reboot… I think hassio only correct the time (via NTP) after trying to connect to the nebucase. I guess the token are invalid if the date’s are to much off

pergola.fabio · May 15, 2021, 5:32pm

That could indeed be the solution, gonna test it also tomorrow… That’s why it only happens upon an HassOs boot…

I am also using esxi , probably on different time zone…

Strange that it worked on 2021.4…

Allthough there was an PR change in pycognito in hass nabucasa 0.43.0 , something with tokens…

pergola.fabio · May 15, 2021, 5:38pm

hmm, just checked, the time is correct , where did you change? did you change the timezone on HassOS or on esxi ? i am in +1 zone, i know my logs are are always wrong upon first boot, they are 1 hour behind
so i think the time is incorrect on HassOS itself

pergola.fabio · May 15, 2021, 6:09pm

ok, i think i have it fixed too, i changed the NTP servers on HassOS itself
it was synching with the ones from Google, maybe that caused the wrong timezone

now it seems correct

sylviepipi · May 16, 2021, 1:05pm

glad to hear your issue is also solved. I just configured my esxI to use its NTPclient, which i didn’t configurer previously.

pergola.fabio · May 16, 2021, 1:16pm

Indeed, thnx for this!

Ho did you figure out it was a time issue?

sylviepipi · May 16, 2021, 2:45pm

i was collecting the logs for publishing afteryour request and i noticed the time changed after the error … So i guessed something must be wrong with the host

ColinRobbins · January 8, 2022, 5:52pm

I am just starting to see this error.
Same symptoms - HA working perfectly for a long time, then on a server reboot, this has stopped working - with error as above.
I’m running HA core.

I have checked the time on my server, and it is the correct time and timezone.
I have also checked IPv6 is disabled.

Any suggestions beyond “check the time” (I’ve also raised a support ticket).

Config:

Version	core-2021.12.8
Installation Type	Home Assistant Core
Development	false
Supervisor	false
Docker	false
User	homeassistant
Virtual Environment	true
Python Version	3.9.0
Operating System Family	Linux
Operating System Version	5.10.63-v7l+
CPU Architecture	armv7l
Timezone	Europe/London

ColinRobbins · January 9, 2022, 7:34am

Debugging steps…

I copied my entire .homeassistant configuration directory to a test server, running the development version of HA (2022.2.0.dev0)- it worked fine. So in my mind this rules out HA configuration issues.
I saved my .homeassistant config on my production server, and ran HA creating a completely new config from scratch. This has the same issues connecting to the cloud.
Conclusion, it is not a HA config issue, but something changed on my server environment.
I reverted HA to 2021.11.0. Same issue, so not an HA new code issue
Modified python3.9/site-packages/pycognito/__init__.py, line 254 from

        except JWTError:
            raise TokenVerificationException(
                f"Your {id_name!r} token could not be verified."
            ) from None

to

        except JWTError as e:
            raise TokenVerificationException(
                f"Your {id_name!r} token could not be verified ({e},{token})."
            ) from None

to get extra logging.

e reports “Signature verification failed”.
Took the token returned, and pasted into https://jwt.io/ which reports signature verified.
However https://token.dev/ reports verification failed - but no error reason.

The time on my server is correct, and NTP schronised

$ timedatectl
               Local time: Sun 2022-01-09 07:32:24 GMT
           Universal time: Sun 2022-01-09 07:32:24 UTC
                 RTC time: n/a
                Time zone: Europe/London (GMT, +0000)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

Disabled NTP, and tried rolling the clock back 30 minutes and forward 30 minutes. No change.

Any suggestions as to how I can figure out why HA is failiing the signature verification.