When one of the modbus device is disconnected ESPhome not able to connect to wifi

Hello

I have a config with 4 modbus device, and for 2 of them i using update interval 0.1s
I need 0.1s because i want to detect DoubleClick.

My main problem is when one of the modbus device is disconnected it is not able to connect to the WIFI. And my main problem is all my local automatization is not working.

I wonder if there is a way to tweak my config to work in case of one of the device is failed.

This is the relevant part of my config:

uart:
  - id: mbus_fast
    tx_pin: 17
    rx_pin: 16
    baud_rate: 115200 
    stop_bits: 1

  - id: mbus
    tx_pin: 1 
    rx_pin: 3
    baud_rate: 9600 
    stop_bits: 1


modbus:
  - id: ga_f_eloszto_modbus1
    uart_id: mbus

  - id: ga_f_eloszto_modbus2
    uart_id: mbus_fast
    
modbus_controller:
  - id: ga_f_eloszto_mbsl32di_1
    ## the Modbus device addr
    address: 10
    modbus_id: ga_f_eloszto_modbus2
    setup_priority: -10
    update_interval: 0.1s

  - id: ga_f_eloszto_mbsl32di_2
    ## the Modbus device addr
    address: 11
    modbus_id: ga_f_eloszto_modbus2
    setup_priority: -10
    update_interval: 0.1s
    #command_throttle: 100ms

  - id: ga_f_eloszto_24c3d32
    ## the Modbus device addr
    address: 30
    modbus_id: ga_f_eloszto_modbus1
    setup_priority: -20

    
  - id: ga_f_eloszto_meter_A1
    address: 50
    modbus_id: ga_f_eloszto_modbus1
    setup_priority: -10
    update_interval: 15s


Thank you

The problem was i using pin 1 and 3 for modbus and i connected the esp board to usbhub to power it. It was a mistake.