Tradfi throws errors

Hello!

I just got a Mac Mini and moved my home assistant config to a new instance of home assistant to the Mac in Docker.
Everything is working nicely and finally its really fast but Tradfri integration is acting strange. I successfully integrated it and when I start/restart Home Assistant, everything is working for a while then suddenly it stops working. I have a lot of errors in the logs:

2019-11-01 08:33:00 ERROR (MainThread) [coap] Exception CancelledError() can not be represented as errno, setting -1.
2019-11-01 08:33:00 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Living Room remote
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 95, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 816, in _run_outer
    yield from cls._run(app_request, response, weak_observation, protocol, log, exchange_monitor_factory)
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 865, in _run
    blockresponse = yield from blockrequest.response
aiocoap.error.RequestTimedOut

I got this several times for all my Tradfri devices (remotes, switches, bulbs), and then:

2019-11-01 08:33:00 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 95, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 816, in _run_outer
    yield from cls._run(app_request, response, weak_observation, protocol, log, exchange_monitor_factory)
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 865, in _run
    blockresponse = yield from blockrequest.response
OSError: [Errno -1] No error information

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1261, in _execute_service
    await handler.func(service_call)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
    self._platforms.values(), func, call, service_name, required_features
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 348, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 370, in _handle_service_platform_call
    await getattr(entity, func)(**data)
  File "/usr/src/homeassistant/homeassistant/components/tradfri/switch.py", line 45, in async_turn_off
    await self._api(self._device_control.set_state(False))
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 155, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 146, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 105, in _get_response
    await self._reset_protocol(e)
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 78, in _reset_protocol
    await protocol.shutdown()
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 133, in shutdown
    for exchange_monitor, cancellable in self._active_exchanges.values():
AttributeError: 'NoneType' object has no attribute 'values'
2019-11-01 08:33:01 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 95, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 816, in _run_outer
    yield from cls._run(app_request, response, weak_observation, protocol, log, exchange_monitor_factory)
  File "/usr/local/lib/python3.7/site-packages/aiocoap/protocol.py", line 865, in _run
    blockresponse = yield from blockrequest.response
aiocoap.error.RequestTimedOut

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/core.py", line 1261, in _execute_service
    await handler.func(service_call)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
    self._platforms.values(), func, call, service_name, required_features
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 348, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 370, in _handle_service_platform_call
    await getattr(entity, func)(**data)
  File "/usr/src/homeassistant/homeassistant/components/tradfri/switch.py", line 49, in async_turn_on
    await self._api(self._device_control.set_state(True))
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 155, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 146, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.7/site-packages/pytradfri/api/aiocoap_api.py", line 101, in _get_response
    raise RequestTimeout('Request timed out.', e)
pytradfri.error.RequestTimeout: ('Request timed out.', RequestTimedOut())

I already tried to remove/readd the integration from the UI, restarted Tradfri gateway.
My configuration has these (gateway is on fixed ip):

tradfri: 
  host: 192.168.10.102

Do you guys have any idea what could be wrong? It was working fine on my rpi 3 with hassbian (I cannot remember the exact version of home assistant but it was around 0.98.*). My current version is the latest (0.101.1).

I also have Homekit and it works fine while home assistant integration is not responding.

Thanks!