Home Assistant bans docker IP instead of remote client IP

If I remember correctly this what I did.
For http component added following two lines:

    use_x_forwarded_for: true
    trusted_proxies: <NGINX IP address here, or 127.0.0.1 if hosted on the same machine>

I think I followed guide from here:

Hope this helps