Since you will not be in the operating system environment when you SBF, it doesn't make a difference if the USB debugging is on, off or purple... When you SBF, the device is being handled by the PC instead of its own internal processing.
Hope this answered your q.