How to call importlib.import_module in a way that doesn't block the thread?

Development
1

I do a fair amount of dynamic loading of module files in my custom component.
Users of my custom component have started opening issues about the Detected blocking call to import_module inside the event loop warning.

It is happening here: HAsmartirrigation/custom_components/smart_irrigation/helpers.py at 011688abb2dc826d680716782373a757401b6b18 · jeroenterheerdt/HAsmartirrigation · GitHub

but for the life of me I cannot figure out how to keep using this but not block the thread. Any suggestions on how to do this? I have read the async dev docs page but I wasn’t able to make it work. I don’t really want to get rid of the dynamic loading…

2

There are prob a few changes to make some of your methods async on the way but fundamentally, if you call

await hass.async_add_executor_job(
        loadModules, const.MODULE_DIR
)

in line 731 of your init file, this will run it in a thread and not block the loop. Obv, this getModuleInstanceByID method also needs to be async and the upstream modules that call it.

1 Like
3

thanks a lot, I will try to do this!

4

I have been making progress with this, thank you very much.

@msp1974 another question if I may: I have a websocket call that eventually needs to call the loadModules function. Since loadModules has to be async now, I need to await it in my websocket - but that does not seem to be the right thing to do. Any idea what to do?

5

Given that your websocket returns the result of coordinator.async_get_all_modules(), then awaiting it is fine as it needs to complete before it can.

As a note, loadModules itself does not have to be async, but must not block the event loop. As such, using the example i provided before, runs it in another thread and not the event loop, so that this wont block it.

6

thanks. I made the following changes:

@callback
async def websocket_get_all_modules(hass: HomeAssistant, connection, msg):
    """Publish all module data. This is not retrieved from the store."""
    coordinator = hass.data[const.DOMAIN]["coordinator"]
    modules = await coordinator.async_get_all_modules()
    connection.send_result(msg["id"], modules)

However, now I am getting:

2024-05-24 16:58:46.698 WARNING (MainThread) [py.warnings] /workspaces/core/homeassistant/components/websocket_api/connection.py:229: RuntimeWarning: coroutine 'websocket_get_all_modules' was never awaited
Coroutine created at (most recent call last)
  File "/workspaces/core/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware
    return await handler(request)
  File "/workspaces/core/homeassistant/components/http/request_context.py", line 26, in request_context_middleware
    return await handler(request)
  File "/workspaces/core/homeassistant/components/http/ban.py", line 85, in ban_middleware
    return await handler(request)
  File "/usr/local/lib/python3.12/site-packages/aiohttp_session/__init__.py", line 199, in factory
    response = await handler(request)
  File "/workspaces/core/homeassistant/components/http/auth.py", line 295, in auth_middleware
    return await handler(request)
  File "/workspaces/core/homeassistant/components/http/headers.py", line 32, in headers_middleware
    response = await handler(request)
  File "/workspaces/core/homeassistant/helpers/http.py", line 73, in handle
    result = await handler(request, **request.match_info)
  File "/workspaces/core/homeassistant/components/websocket_api/http.py", line 53, in get
    return await WebSocketHandler(request.app[KEY_HASS], request).async_handle()
  File "/workspaces/core/homeassistant/components/websocket_api/http.py", line 416, in async_handle
    async_handle_str(command_msg_data)
  File "/workspaces/core/homeassistant/components/websocket_api/connection.py", line 229, in async_handle
    handler(self.hass, self, schema(msg))
  handler(self.hass, self, schema(msg))

Internet search is not really helping. What should I do?

7

never mind, I was able to fix it. If anyone finds this later, I added a @async_response decoration to my method.