Everything is there in terms of all possible combinations.
1) Either have sound or no sound
2) Set it to vibrate: Never, Always, when on silent,
So a completely silent option is to check as no sound and never vibrate.
I have a toggle on my homescreen that makes it easier for me. Instead of changing the setting all the time.
Cheers