If you already have ROMs, just put them on your SD card. When you run the emulator it will ask you to select them. ROM Gripper not only finds the ROMs but allows you to download them too. I'm assuming ROM Buddy does as well, but again, you have to pay for that one, and the comments on the Market say that ROM Gripper has a larger selection anyway.
If you decide to use one of the ROM apps from the Market, they create a folder to save the ROMs too. It is defaulted to the SD card but you can change it. It creates one called /SD/ROM and then subfolders for the emulator type (GBA/NES/SNES/etc.).
Depending on the emulator you may need a BIOS file as well, such as with the GBA emulator. ROM Gripper has that as well I believe.