I've had this happen twice so far. Still trying to figure out what conditions are causing this to happen. When it does happen a reboot isn't necessary - going in and out of airplane mode seems to get the phone working again.
Note that this is on my Galaxy Nexus (don't have a Nexus 4) so seems to be related to the OS and not the N4 specifically.
@kalaster - did you do a factory reset, and if so did that clear this up for you?
Galaxy Nexus, AT&T, Southern California USA.