Just to make this clear, the Chromecast has no inputs or outputs on it except for a male HDMI plug and an optional power connection. It connects via WiFi and plugs into a HDMI input.
In my setup the Chromecast plugs into a HDMI input on the TV, the normal connection. So of course it would normally play back sound through the TV. I mute the built-in speakers and use the TV's fiber optic digital audio output to feed into the audio setup. If you have a HDMI output on the TV connected to a HDMI input on the stereo I suppose that would work as long as you can switch the HDMI input the Chromecast is plugged into to feed the HDMI output. Kinda the same thing.
The audio setup is also Bluetooth-enabled so if I don't need a picture but want to play something from a phone or tablet I use Bluetooth and skip the Chromecast.