Did you create the command in home assistant to change the channel?
If not then there is your problem…
I don’t believe there is a easy way to do it. You will need to create an automation or script that does it and in google home app create a routine to trigger this script/automation using a voice command of your choice.
As far as I know you can’t pass “5” from “channel 5” to HA so you will have to create everything from scratch.
But there was a feature request a last week or so asking for this feature, to capture elements of voice messages and have that passed to HA.