Neither Serial Number nor IMEI allows the phone to be hacked, either separately or together. They may be of interest to scammers, and as I mentioned could indirectly have consequences for you (e.g. if this results in your IMEI being blacklisted), but there is no magic backdoor where someone who knows your true name IMEI can take control of you your phone.
Of course if someone had access to your carrier's systems and knows your IMEI they could monitor your approximate location (the phone uses the IMEI to identify itself to the carrier when it connects - this is how a phone can be blacklisted). But that's like saying that if someone has access to your carrier's systems they can see who you call and when, who you send SMS to: of course they can, because all of that comes through their systems. But that's nothing to do with hacking your phone. If you are worried about that, making sure your Google account is secure is the most important thing, because that's the easiest way for anyone to get into your phone.