How do I fix this problem?
To fix this problem you need logout of your Unix / Linux / OS X / BSD desktop session and log back in again to force the agent to forget identity. Another recommend option is to run the ssh-add command to add RSA or DSA identities to the authentication agent. This will fix your problem without login out from the current session. The syntax is as follows:
Pro tip: If your key located in different path or does not have the default filename, you will need to pass the path to ssh-add command as follows:
$ ssh-add /path/to/my/ssh_key
$ ssh-add ~/.ssh/server3_key
See ssh-add(1) command man page for more information.