Exactly. Because without using GPU at all, the power consumption wouldn’t be anywhere near the maximum you could encounter when using it as a cheap desktop PC. And the recommended power supply need to be able to handle such usage without a glitch…hence those recommended 3 Amps.
It is important to have the power supply with stable 5+ Volts though. I have bad experience even with original 2.5A power bricks for RPi 3, that after some time, I start getting undervoltage warnings even during medium load not reaching anywhere near 1.5A