You can use a file manager such as ES File Explorer to delete straight from the phone.
Alternatively, plug the phone in to your PC, mount as a USB disk drive and delete that way.
I'd always keep your most recent good backup though.
What recovery are you using?