The People app (on the phone) has an import function, though it looks for vCard files to import. If you really don't want these contacts in Google (I'd suggest you do so, as it will make things much easier if you ever factory reset, but you know what you want best), it does ask before the import if you want the contacts to be Google or Phone contacts.
You can try with the csv file, but you may need to convert them to a vCard (.vcf) file first.
I looked around and could not find any app that restores csv files to contacts on the phone.
