SSH agent forwarding fails after updates SSH key

trellis
1

Hi!

I have recently created a new ed25519 SSH ( I was still using an RSA key previously). I figured this would be easy:

  • Create new SSH key and add to Gitlab and Github accounts

  • Add keys to remote servers by running

provision trellis --tags=users <env>
  • update my .ssh/config to use the new key for forwarding:
Host *
  AddKeysToAgent yes
  UseKeychain yes
  ForwardAgent yes
  IdentitiesOnly yes
  IdentityFile ~/.ssh/id_ed25519
  • add new ssh key to keychain ssh-add --apple-use-keychain

However my deployments fail now during the [deploy : Clone project files] step.

I think for some reason my old key is still used when trying to connect to Gitlab to clone the project but fail to understand why. I have tried loads of things:

  • remove .ssh/id_rsa & .ssh/id_rsa.pub and restart (ssh-agent & reboot computer)
  • remove rsa key from keychain
  • run a full reprovision
  • sudo reboot remote server
  • clear known_hosts both locally and on remote server
  • Double check SSH key is correct at https://gitlab.com/Abel-Sch.keys and added to remote servers known_hosts

One thing I found strange is that when I run:
ssh -T git@gitlab.com or ssh -T git@github.com locally they both work properly, but while connected to the remote server Github authenticates succesfully but Gitlab fails (this project is hosts on Gitlab).

2 Likes
2

Removing all identities by running ssh-add -D and re-adding the new ed25519 seems to have worked:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

This leds me to believe the identity with the old key was still being used, oddly enough it wasn’t shown when running ssh-add -L.

Any explanation is still very much appreciated.

6 Likes
3

Glad you solved it… Thanks for the well written post with lots of details and steps :heart:

I would have thought restarting ssh-agent would have fixed it so unfortunately I don’t have any better explanation.

2 Likes
4

After rebooting my laptop the issue was back. Removing the identities worked again. I found a persistent fix is updating my .zprofile to:

# #add SSH keys
ssh-add --apple-use-keychain &>/dev/null
3 Likes
5

This worked for me as well

6

I think ssh-add -D could be a blurb in the ssh documentation to help people with these weird cruft ssh key issues.