High memory use is not an issue, stability is…
This is my Pi3+ 1Gb using between 80-95% all the time, and it’s fine. If your using an SD card, change to a SSD and make sure to increase the size of your swap file (How to increase the swap file size on Home Assistant OS)