Did you delete it manually from your Photos app, or did you let it automatically remove old photos to save space? It makes a huge difference, since if you delete it from Photos it tells you that it will delete the upload as well.
If you've just let it automatically remove pictures to save space I would expect it would still leave the thumbnail on your device (I don't know, because I have that option turned-off, but it's what I've seen Google Photos do on an friend's iPhone).