I had the same issue, where i had a ongoing download from my browser that I just couldn't clear from the notification area. I tried everything, including restarting the phone, clearing the cache of the Android browser, Google Chrome Browser, I tried as well to keep my finger pressed on the download, no reaction. checked in the Downloads in both browsers where it didn't appear.
No way to stop the ever ongoing download. And it apparently drained my battery quite a lot, when it kept on trying to download.
Finally i got it to stop. The reason was, that my USB storage was full. I deleted a few files, and the download completed and can now be cleared.
Just wanted to write it here in case someone else is googling this problem.