OK, I tried to connect the phone to my network, I have a droid 2 on there and have used a droid, iphones, my laptops, xbox, etc without trouble.
But this Nexus One seems to not like my security WPA WPA 2 on my router.
Because when I reset my router and have no encryp, it connects flawlessly...