The pen definitely drags more, but since I've had my note 5 for about a year now it feels normal.
The holster clips well enough for me. I previously had an S3 in a Defender and didn't notice any difference there, but make sure the nub on the holster hasn't broken off.