Raspberry falling off wired network, automations running

So I am running latest HA OS on a raspberry 4 with wired LAN.
Suddenly I can no longer access it via web interface, also to SSH in.
Also my router no longer shows it in device list.
However LAN port LEDs are blinking and my automations are running as usual.
I have tried to connect a small HDMI LCD to check network settings, however HDMI interface seems to be off.

My network setup is:
5G CPE (Huawei H122-373) with DHCP server → 16-port PoE switch (Linksys) → raspbery with PoE shield, and other devices.
This may be an issue with my network, as one more headless linux machine is dropping as well. However it reappeared and continued roper operation when LAN cable was replugged (it is AC powered).

What may be the cause of this, and how do I debug?