Thanks @fullyint, this got me there. Both settings were correct, but I found the fix way down at the very bottom of the GitHub SSH Forwarding page:
On Mac OS X, ssh-agent will “forget” this key, once it gets restarted during reboots. But you can import your SSH keys into Keychain using this command:
/usr/bin/ssh-add -K yourkey
I added the keys I have associated with GitHub using ssh-add and the next connection forwarded them correctly. My deploys are now working without manually transferring the keys.
Thanks!