ControllerX. Bring full functionality to light and media player controllers

Anybody able to help me to get my Ikea E1744 symfonisk controller to be more responsive in controlling Philips Hue lights? I’ve followed the steps mentioned under 9. Symfonisk controller (E1744) works, but pretty laggy in the ControllerX FAQ, however the Hue Light group still lags ~2 seconds behind knob turns. It is also a bit hit or miss if the brightness will increase or decrease reagardless of which direction I turn the knob, and the changes are not very smooth.

I am using the Hue integration on it’s own hub, but triggering light changes from within the HA frontend or even via other Zigbee devices through MQTT happens without measurable delay, so I don’t think it’s something to do with Hue.

Here is what I have in apps.yaml:

symfonisk_light_controller:  # direct via mqtt topic
  module: controllerx
  class: E1744LightController
  integration:
    name: z2m
    listen_to: mqtt
  controller: symfonisk_light_controller
  light: light.dining_room
  smooth_power_on: true

Here is the relevant section from Zigbee2MQTT devices.yaml. I’ve tried playing around with the debounce and removing it entirely but I don’t notice much of a difference.

'0x680ae2fffe5c1fc9':
  friendly_name: symfonisk_light_controller
  optimistic: true
  legacy: false
  simulated_brightness: {}
  debounce: 0.5
  debounce_ignore:
    - action

And here is how I’ve added the MQTT broker in appdaemon.yaml:

secrets: /config/secrets.yaml
appdaemon:
  latitude: 52...
  longitude: 4...
  elevation: 2
  time_zone: Europe/Amsterdam
  plugins:
    HASS:
      type: hass
    MQTT:
      type: mqtt
      namespace: mqtt # This is important
      client_host: 192.168.178.44
      client_user: mqtt-user
      client_password: !secret mqtt_client_password
http:
  url: http://127.0.0.1:5050
admin:
api:
hadashboard: