Is your phone rooted or not?
If it is rooted, don't forget that while rooting gives you more unfettered access to the installed Android operating system on your phone, it also significantly increases the risk to be compromised -- rooting gives you system-level permissions, and just as important it gives some jerk system-level permissions to install something like a remote-code exploit. Especially when you haphazardly side-load APKs.
If it's not rooted, than at least you know that APK you side-loaded is going to be easily accessible in the user data partition. You should be able to just wipe if using a typical file manager app. Or you can also just re-install that same APK file, but this time don't skip past any pop-up messages that occur during the install process.