Pictures aren't stored "in the Gallery", the Gallery app shows all the pictures that the media scan function finds on the phone. Clear the cache or use an app like QuickPic.
But the more pictures you have the longer it's going to take to open all the files, regardless of what you do.