No, but imagine a line from the headset to the phone. If it's going through your head, or through a major portion of your body, the phone is receiving a very weak signal and may lose it entirely from time to time. That will cause the handset to switch to itself.
If I wear a Motorola 850 in my left ear and the phone is on my right hip (in a holster on my belt), Bluetooth becomes iffy, But if I put the phone down and walk 20 feet away, with the headset and phone "seeing" each other there's no problem. The human body soaks up 2.4GHz signals like a sponge, and BT headset transmitters are VERY low powered.
Keep the headset and phone positioned so they can see each other without having to go through you.