I have never used any call recording app myself. I know that, on some custom ROM's, it is possible to have 2 way call recording (not as an app, but as part of the ROM) and have heard mixed reviews, to be honest.
I have heard that some (not sure if it's all) call recording apps record using the phone's microphone, so the volume of the other person can be quite quiet.
With regards to whether the other party will know if you are recording them, I am not sure... I think that I read somewhere that some call recording apps have some sort of audiable tone to denote that a call is being recorded - I think that this may be to do with the fact that, in some parts of the world, it can be a criminal offence to record someone without their knowledge.