The people you put in your address book are not directly connected to the viewer accounts a user creates.
When you import people into your address book it does not automatically create them an account. This is because you can send someone a ving and they do not have to sign up for a viewer account.
Now if you send your vings verified or invited you are requiring that someone sign up for a viewer account. This will allow them to access their profile and inbox.
In their viewer profile, they can connect multiple phones and emails. If someone works for multiple companies they could have all their emails connected to one viewer account so they could see what all they have left to do for any ving that has been ever been invited to any of their emails.
Now in your address book, you might only have their work email and cell phone so the way you have someone in your address book might not be the same as how someone has their viewer account set up.
This also means that their names could be different, avatars could be different, etc.