Are you using a phone case? If so remove it and re-test
Does it happen if you are on speakerphone? Earpiece? Headset?
What is your carrier? Is it the unlocked version or a carrier variant?
How does the person on the other end sound to you?
So, when they complain of backgroiusn noise, are you actually in noisy envorinments ?