I updated one of my projects to the latest Trellis version (1.3.0) and created a new project also with the latest version (1.3.0) and on both projects I start receiving this error when (re)provisioning my remote servers:
TASK [common : Retrieve SSH client IP] ******************************************************************************************************************
System info:
Ansible 2.8.0; Darwin
Trellis Head
---------------------------------------------------
No valid or no response from url https://api.ipify.org/ within 10 seconds
(timeout)
fatal: [178.62.254.13]: FAILED! => {"changed": false}
TASK [common : Fail when unable to retrieve SSH client IP] **********************************************************************************************
System info:
Ansible 2.8.0; Darwin
Trellis Head
---------------------------------------------------
External IP resolution failed. Check that your DNS servers are working. Try to disable DNSCrypt if you are using it.
fatal: [178.62.254.13]: FAILED! => {"changed": false}
So I checked my other projects and I’m getting this error on ALL my projects now?
I recently updated to Catalina 10.15.3 and to Vagrant 2.2.7 and Ansible 2.8.0.
The api.ipify.org website returns my correct IP address on my host machine, but when I curl it:
curl http://api.ipify.org
it returns my IP but with a percentage sign at the end?
xxx.xxx.xxx.xxx%
Right now, provisioning my servers only works when I follow these instructions.
Anyone else seeing this?
Does MacOS Catalina have DNSCrypt installed somewhere?
Nobody else ran in this issue recently? Hmm then it must be network or device related I guess… What kind of 3rd party software could be causing this?
Yeah those are the instructions I was referring to in my post.
Weird issue though… Are you using something like a VPN or other DNS software? Because I’m not?
I tried VPN and I tried without. Without diving deeper into it, I feel like maybe its the change from bash to zsh in 10.15. Probably wrong, but it’s a thought at the moment.
That’s what I thought for a moment, but then I realized that I already switched to zsh before upgrading to Catalina and had no issues related to this?
Ok, I just successfully re-provisioned my remote server on the exact same project with the exact same Ansible 2.8.0 & Vagrant 2.2.7 versions but on MacOS Mojave 10.14.6 without any errors.
So it’s definitely a Catalina thing I guess? Does Catalina somehow mess with the DNS lookups?
It shouldn’t… Catalina causing DNS issues would be something we’d hear about I’d imagine. Though it definitely has bugs so maybe something more specific to your setup
We both destroyed and recreated our local VM, tried to re-provision the remote and no errors on my colleague’s machine and the same error again on my machine.
The only difference in the playbook log I can detect is this warning:
[WARNING]: Could not find aptitude. Using apt-get instead
I have no VPN software installed, I don’t have any custom DNS or proxy settings in my Network preferences and I don’t have any weird things in my host file…
We can both connect to the remote server as admin and web with our own ssh keys.
What else could be causing this difference?
Could it be that my SSH key is blocked on the remote once the lookup has failed once?
My full verbose error log:
TASK [common : ipify_facts] ***************************************************************************************************************************
task path: ~/path/to/my/project/trellis/roles/common/tasks/main.yml:152
Using module file /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ansible/modules/net_tools/ipify_facts.py
Pipelining is enabled.
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: Username
<localhost> EXEC /bin/sh -c '/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 && sleep 0'
System info:
Ansible 2.8.0; Darwin
Trellis 1.2.0: October 11th, 2019
---------------------------------------------------
No valid or no response from url https://api.ipify.org/ within 10 seconds
(timeout)
fatal: [198.199.125.53]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"api_url": "https://api.ipify.org/",
"timeout": 10,
"validate_certs": true
}
}
}
TASK [common : fail] **********************************************************************************************************************************
task path: ~/path/to/my/project/trellis/roles/common/tasks/main.yml:158
System info:
Ansible 2.8.0; Darwin
Trellis 1.2.0: October 11th, 2019
---------------------------------------------------
External IP resolution failed. Check that your DNS servers are working. Try
to disable DNSCrypt if you are using it.
fatal: [198.199.125.53]: FAILED! => {
"changed": false
}
Ok I got a bit further, when I disable the validate_certs in the ipify_facts task: in /roles/common/tasks/main.yml:
- name: Retrieve SSH client IP
block:
- ipify_facts:
validate_certs: no
delegate_to: localhost
become: no
when: env != 'development' and ssh_client_ip_lookup | default(true)
tags: [fail2ban, ferm]
rescue:
- fail:
msg: "External IP resolution failed. Check that your DNS servers are working. Try to disable DNSCrypt if you are using it."
I can re-provision without any errors?
Which certificates is ipify_facts trying to validate anyway? Don’t understand…
