Matter on Home Assistant Core

I am trying to install the Matter integration on Home Assistant Core. More specifically, I am up to the step where I install the Matter server which needs to be done manually since this is Home Assistant Core.

So I go into Python Matter Server on github. I am able to pip install the Matter server but where I am stuck is here:
The server library depends on the CHIP Core SDK which is architecture and OS specific. We build (and publish) wheels for Linux (amd64 and aarch64) to pypi but for other platforms (like Macos) you will need to build those wheels yourself using the exact same version of the SDK as we use for the clusters package.

How do I go about building those wheels - I am running on Raspberry Pi 4?