Ok I’ve seen that ZHA doesn’t support the 5 Button Ikea remote out of the box. As stated in this 2 year old issue: ZHA: Tradfri 5 button Remote no press reported · Issue #22443 · home-assistant/core · GitHub
But luckily there is a Blueprint that works fine for my usecase here:
ZHA - IKEA TRADFRI - 2 Button Remote - OnOff and Brightness