Ansible provisioning fails for root login disabled and different admin_user

trellis
1

Hi,

I am trying to provision a staging server at Google Cloud, after having successfully set up a dev server with Trellis (Vagrant and Virtualbox). My instance is Ubuntu 16.04 (Xenial). My local environment is Debian 9 with Ansible 2.4.0.0.

I can ssh into my instance as mclarke:

ssh 35.197.167.246

It’s not possible to ssh in as root or admin, so I have set up my users.yml as follows:

admin_user: mclarke

# Also define 'vault_users' (`group_vars/staging/vault.yml`, `group_vars/production/vault.yml`)
users:
  - name: "mclarke"
    groups:
      - sudo
    keys:
      - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
  - name: "{{ web_user }}"
    groups:
      - "{{ web_group }}"
    keys:
       - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
      # - https://github.com/username.keys
  - name: "{{ admin_user }}"
    groups:
      - sudo
    keys:
      - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
      # - https://github.com/username.keys

web_user: web
web_group: www-data
web_sudoers:
  - "/usr/sbin/service php7.1-fpm *"

The (abbreviated) output I am getting from running:

ansible-playbook server.yml -e env=staging

(full output included at the end of this message) is:

TASK [users : Ensure sudo group has sudo privileges] **********************************************************************************************************************************
changed: [35.197.167.246]

TASK [users : Fail if root login will be disabled but admin_user will not be a sudoer] ************************************************************************************************
System info:
  Ansible 2.4.0.0; Linux
  Trellis at "Normalize `apt` tasks"
---------------------------------------------------
The conditional check 'TrueTrue' failed. The error was: error while
evaluating conditional (TrueTrue): 'TrueTrue' is undefined
fatal: [35.197.167.246]: FAILED! => {"failed": true}

RUNNING HANDLER [fail2ban : restart fail2ban] *****************************************************************************************************************************************
changed: [35.197.167.246]

RUNNING HANDLER [ferm : restart ferm] *************************************************************************************************************************************************
changed: [35.197.167.246]

RUNNING HANDLER [ntp : restart ntp] ***************************************************************************************************************************************************
changed: [35.197.167.246]
        to retry, use: --limit @/home/mclarke/gitlab/australiascience.tv/trellis/server.retry

PLAY RECAP ****************************************************************************************************************************************************************************
35.197.167.246             : ok=38   changed=18   unreachable=0    failed=1   
localhost                  : ok=0    changed=0    unreachable=0    failed=0   

$

Other things I have modified from Trellis’s master branch and (I hope!) in accordance with the instructions are:

  • group_vars/all/users.yml as shown above
  • group_vars/staging/wordpress_sites.yml
  • group_vars/staging/vault.yml (new passwords and salts)
  • group_vars/all/vault.yml
  • .vault_pass (added)
  • ansible.cfg updated to include .vault_pass
  • hosts/staging updated with the new IP address
  • group_vars/all/security.yml changed sshd_permit_root_login to false

I believe that in ansible.cfg ssh agent forwarding is already enabled, so I did not modify ssh_args in there.

I have been trying to fix this for over a day, now, and am out of ideas. I have probably done or not done something stupid. Hope you can help. Anything you can suggest will be much appreciated.

Here is the full output from Ansible:

~/gitlab/australiascience.tv/trellis$ ansible-playbook server.yml -e env=staging
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for dynamic inclusions. This feature will be 
removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This 
feature will be removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [Ensure necessary variables are defined] *****************************************************************************************************************************************

TASK [Ensure environment is defined] **************************************************************************************************************************************************
skipping: [localhost]

PLAY [Test Connection and Determine Remote User] **************************************************************************************************************************************

TASK [connection : Require manual definition of remote-user] **************************************************************************************************************************
skipping: [35.197.167.246]

TASK [connection : Specify preferred HostKeyAlgorithms for unknown hosts] *************************************************************************************************************
ok: [35.197.167.246]

TASK [connection : Check whether Ansible can connect as root] *************************************************************************************************************************
The authenticity of host '35.197.167.246 (35.197.167.246)' can't be established.
ED25519 key fingerprint is SHA256:94Eibw6IyhrHJuMwnHfQPZoZiW1AizYu5g5SzJlHw2I.
Are you sure you want to continue connecting (yes/no)? yes
ok: [35.197.167.246 -> localhost]

TASK [connection : Warn about change in host keys] ************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [connection : Set remote user for each host] *************************************************************************************************************************************
ok: [35.197.167.246]

TASK [connection : Announce which user was selected] **********************************************************************************************************************************
Note: Ansible will attempt connections as user = mclarke

Note: The host `35.197.167.246` was not detected in known_hosts
so Trellis prompted the host to offer a key type that will work with
the stronger key types Trellis configures on the server. This avoids future
connection failures due to changed host keys. Trellis used this SSH option:

  -o HostKeyAlgorithms=ssh-ed25519-cert-v01@openssh.com,ssh-rsa-
cert-v01@openssh.com,ssh-ed25519,ssh-rsa

To prevent Trellis from ever using this SSH option, add this to group_vars:

  dynamic_host_key_algorithms: false
ok: [35.197.167.246]

TASK [connection : Load become password] **********************************************************************************************************************************************
ok: [35.197.167.246]

PLAY [Install prerequisites] **********************************************************************************************************************************************************

TASK [Install Python 2.x] *************************************************************************************************************************************************************
ok: [35.197.167.246]

PLAY [WordPress Server - Install LEMP Stack with PHP 7.1 and MariaDB MySQL] ***********************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************
ok: [35.197.167.246]

TASK [common : Validate wordpress_sites] **********************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Validate format of site_hosts] *****************************************************************************************************************************************
skipping: [35.197.167.246] => (item=australiascience.tv) 

TASK [common : Verify dict format for apt package component variables] ****************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Verify dict format for apt package combined variables] *****************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Validate Ubuntu version] ***********************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Check whether passlib is needed] ***************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Retrieve local SSH client's settings per host] *************************************************************************************************************************
ok: [35.197.167.246]

TASK [common : Validate compatible settings between SSH client and server] ************************************************************************************************************
ok: [35.197.167.246] => {
    "changed": false, 
    "failed": false, 
    "msg": "All assertions passed"
}

TASK [common : Checking essentials] ***************************************************************************************************************************************************
changed: [35.197.167.246] => (item=python-software-properties)
changed: [35.197.167.246] => (item=build-essential)
changed: [35.197.167.246] => (item=python-mysqldb)
changed: [35.197.167.246] => (item=libnss-myhostname)
ok: [35.197.167.246] => (item=dbus)
changed: [35.197.167.246] => (item=git-core)
ok: [35.197.167.246] => (item=python-pycurl)
ok: [35.197.167.246] => (item=curl)

TASK [common : Validate timezone variable] ********************************************************************************************************************************************
ok: [35.197.167.246]

TASK [common : Explain timezone error] ************************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Add myhostname to nsswitch.conf to ensure resolvable hostname] *********************************************************************************************************
ok: [35.197.167.246]

TASK [common : Generate SSH key for vagrant user] *************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [common : Retrieve SSH client IP] ************************************************************************************************************************************************
ok: [35.197.167.246]

TASK [swapfile : Write swapfile] ******************************************************************************************************************************************************
changed: [35.197.167.246]

TASK [swapfile : Set swapfile permissions] ********************************************************************************************************************************************
changed: [35.197.167.246]

TASK [swapfile : Create swapfile] *****************************************************************************************************************************************************
changed: [35.197.167.246]

TASK [swapfile : Enable swapfile] *****************************************************************************************************************************************************
changed: [35.197.167.246]

TASK [swapfile : Add swapfile to /etc/fstab] ******************************************************************************************************************************************
changed: [35.197.167.246]

TASK [swapfile : Configure vm.swappiness] *********************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [swapfile : Configure vm.vfs_cache_pressure] *************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [fail2ban : ensure fail2ban is installed] ****************************************************************************************************************************************
changed: [35.197.167.246]

TASK [fail2ban : ensure fail2ban is configured] ***************************************************************************************************************************************
changed: [35.197.167.246] => (item=jail.local)
changed: [35.197.167.246] => (item=fail2ban.local)

TASK [fail2ban : ensure fail2ban starts on a fresh reboot] ****************************************************************************************************************************
ok: [35.197.167.246]

TASK [ferm : ensure ferm status is in debconf] ****************************************************************************************************************************************
changed: [35.197.167.246]

TASK [ferm : ensure ferm is installed] ************************************************************************************************************************************************
changed: [35.197.167.246]

TASK [ferm : ensure configuration directories exist] **********************************************************************************************************************************
changed: [35.197.167.246] => (item=/etc/ferm/ferm.d)
changed: [35.197.167.246] => (item=/etc/ferm/filter-input.d)

TASK [ferm : ensure firewall is configured] *******************************************************************************************************************************************
changed: [35.197.167.246] => (item=etc/default/ferm)
changed: [35.197.167.246] => (item=etc/ferm/ferm.conf)

TASK [ferm : ensure iptables INPUT rules are removed] *********************************************************************************************************************************
skipping: [35.197.167.246] => (item={u'dport': [u'http', u'https'], u'type': u'dport_accept', u'filename': u'nginx_accept'}) 
skipping: [35.197.167.246] => (item={u'dport': [u'ssh'], u'type': u'dport_accept', u'saddr': [u'113.197.13.1']}) 
skipping: [35.197.167.246] => (item={u'dport': [u'ssh'], u'seconds': 300, u'hits': 20, u'type': u'dport_limit'}) 

TASK [ferm : ensure iptables INPUT rules are added] ***********************************************************************************************************************************
changed: [35.197.167.246] => (item={u'dport': [u'http', u'https'], u'type': u'dport_accept', u'filename': u'nginx_accept'})
changed: [35.197.167.246] => (item={u'dport': [u'ssh'], u'type': u'dport_accept', u'saddr': [u'113.197.13.1']})
changed: [35.197.167.246] => (item={u'dport': [u'ssh'], u'seconds': 300, u'hits': 20, u'type': u'dport_limit'})

TASK [ferm : ensure iptables rules are enabled] ***************************************************************************************************************************************
ok: [35.197.167.246]

TASK [ferm : ensure iptables rules are disabled] **************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [ntp : Include OS-specific variables.] *******************************************************************************************************************************************
ok: [35.197.167.246]

TASK [ntp : Ensure NTP-related packages are installed.] *******************************************************************************************************************************
ok: [35.197.167.246]

TASK [ntp : Ensure tzdata package is installed (Linux).] ******************************************************************************************************************************
ok: [35.197.167.246]

TASK [ntp : Check if clock file exists.] **********************************************************************************************************************************************
skipping: [35.197.167.246]

TASK [ntp : Create clock file if it doesn't exist.] ***********************************************************************************************************************************
skipping: [35.197.167.246]

TASK [ntp : Set timezone] *************************************************************************************************************************************************************
ok: [35.197.167.246]

TASK [ntp : Ensure NTP is running and enabled as configured.] *************************************************************************************************************************
ok: [35.197.167.246]

TASK [ntp : Ensure NTP is stopped and disabled as configured.] ************************************************************************************************************************
skipping: [35.197.167.246]

TASK [ntp : Generate ntp.conf file] ***************************************************************************************************************************************************
changed: [35.197.167.246]

TASK [users : Ensure requested groups are present] ************************************************************************************************************************************
ok: [35.197.167.246] => (item=sudo)
ok: [35.197.167.246] => (item=www-data)

TASK [users : Ensure sudo group has sudo privileges] **********************************************************************************************************************************
changed: [35.197.167.246]

TASK [users : Fail if root login will be disabled but admin_user will not be a sudoer] ************************************************************************************************
System info:
  Ansible 2.4.0.0; Linux
  Trellis at "Normalize `apt` tasks"
---------------------------------------------------
The conditional check 'TrueTrue' failed. The error was: error while
evaluating conditional (TrueTrue): 'TrueTrue' is undefined
fatal: [35.197.167.246]: FAILED! => {"failed": true}

RUNNING HANDLER [fail2ban : restart fail2ban] *****************************************************************************************************************************************
changed: [35.197.167.246]

RUNNING HANDLER [ferm : restart ferm] *************************************************************************************************************************************************
changed: [35.197.167.246]

RUNNING HANDLER [ntp : restart ntp] ***************************************************************************************************************************************************
changed: [35.197.167.246]
        to retry, use: --limit @/home/mclarke/gitlab/australiascience.tv/trellis/server.retry

PLAY RECAP ****************************************************************************************************************************************************************************
35.197.167.246             : ok=38   changed=18   unreachable=0    failed=1   
localhost                  : ok=0    changed=0    unreachable=0    failed=0   

~/gitlab/australiascience.tv/trellis$
1 Like
2

Thank you for the thorough info and for working through it so diligently.

Given that you have admin_user: mclarke the users list item with
name: "{{ admin_user }}" is actually the same thing as the list item with
name: "mclarke". You have two users with the same name.

I’m guessing what is happening is that these loops over the list of users normally loops once, producing only True but in your case loops twice and produces TrueTrue.

I’ll work out a fix for Trellis, but in the meantime you could just get rid of the duplicate user:

 users:
-  - name: "mclarke"
-    groups:
-      - sudo
-    keys:
-      - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
   - name: "{{ web_user }}"
     groups:
       - "{{ web_group }}"
     keys:
        - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
       # - https://github.com/username.keys
   - name: "{{ admin_user }}"
     groups:
       - sudo
     keys:
       - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
       # - https://github.com/username.keys
2 Likes
3

Thank you so much for the super-fast response! I will give this a go and let you know what happens.

4

Hi @fullyint,

Thanks again for correcting my mistake, and that worked well. I was able to finish provisioning the server, and was then able to deploy. When I ssh in I can see the site and its files at /srv/www/, as expected.

However, I am not sure what to do next. The docs say:

Trellis does not automatically install WordPress on remote servers. It's normal and expected to see the WordPress install screen the first time you deploy. It's up to you to either import an existing database or install a fresh site.

However, when I browse to the site, I don’t get the installation screen. Instead, all I get is “The connection to the server was reset while the page was loading”.

So:

  • I am browsing directly to the IP address, since we don’t yet have a domain name set up. Is that a problem? Tacking the port on the end didn’t help.
  • Am I meant to just go into the server, cd to the ‘current’ directory and run ‘composer install’? (I don’t think so because the vendor directory has already been created and populated.)

Either way, I would have thought that I’d get the nginx welcome page. Going into the site itself, nginx appears to be running, appears to be listening on port 80, but interestingly, I got this:

mclarke@instance-5:/srv/www$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
mclarke@instance-5:/srv/www$ curl -i http://127.0.0.1/nginx_status
curl: (52) Empty reply from server

and I thought it should have produced the contents of /usr/share/nginx/html/index.html.

Do you have any suggestions?

Thanks again, and apologies if this turns out to be another silly question!

5

Hi, it’s okay, I just found this: err_empty_response.

It seems that the use of the IP address is indeed the cause of the issue.

1 Like