Are you sure? I am not. Sounds to me like you installed ubuntu, something didnt quite work and you are actually booting to your "live USB" when you have the USB drive in and NOT an installed Ubuntu. Maybe you didnt even install ubuntu at all?
Again, if you still see the windows boot manager (or no boot manager at all) then grub isnt installed properly so for the 5th time, replace your MBR!!! (Unless you havent installed Ubuntui and just running Live - in which case, install ubuntu)
Did you read the guide?