Starting from scratch would be just great, the rest of us are hacking on to our exisiting wiring etc.
I think something like KNX would be great if starting from scratch. It is compatible with HA and then HA will let you use other components for things like your music and video.
Make sure you have neutral to all switches.
For music, I am a great squeezebox enthusiast. But if I was starting from scratch I would look at the Monoprice whole house amp or similar: Monoprice Whole Home Audio controller 10761 success