So you log in to your google account using a yahoo address? That doesn't seem right. You need to have a google account in order to use Google Voice.
You're unwittingly correct about what the OPs problem is.
These days, if you create a new Google voice account
first, without creating a gmail account, google does not even let you create a normal username. GV actually forces you to enter an email address. This email address need not be a gmail address. The email address can be anything that works, and then after verification it then becomes your GV username. So yes, someone going down this path would have a GV username like "bob@yahoo.com". It works when logging into google.com/voice from a web browser.
But due to a
software defect (or limitation) in Groove IP, "bob@yahoo.com" does not work, even though it's a correct GV username. The help documentation for Groove IP states that a gmail account must be created and linked to the GV account. What the help docs fail to state is that the gmail username must be used to login.
So the solution: login to your GV account on the web (of course using your yahoo email address as a GV username), then click on the link that gives you other google services. Click on "gmail". Google will automatically sense that your GV account is not associated with any gmail account, and it will create one. You will be asked for your real name, username of choice, and security question to setup a gmail account. Yes, you need to do this, even if you have no intent of using gmail. After you complete that, Groove IP must be given your new google username, not your previous GV username that's represented as an email address.
Both usernames will continue to work on the web, but Groove IP can only handle the proper gmail username (e.g. "bob").