Ansible deploy fails at "Run post_build_commands in the new_release_path"

Hi!

My ansible deploy fails at the composer install task:

TASK: [deploy | Run post_build_commands in the new_release_path] **************
failed: [preview.domain.tld] => (item={'cmd': 'composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts'}) => {"cmd": "composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts", "failed": true, "item": {"cmd": "composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts"}, "rc": 2}
msg: [Errno 2] No such file or directory

FATAL: all hosts have already failed -- aborting

If I run this command (composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts) manually (via ssh) in the new_release_path it works and composer installs all the necessary dependencies.

This staging server is on a shared hosting with ssh access.

And here is the group_vars/staging file (without the env part):

wordpress_sites:
  preview.domain.tld:
site_hosts:
  - preview.domain.tld
local_path: '../site'
repo: git@domain.tld:git/repo.git
branch: master
subtree: site
multisite:
  enabled: false
  subdomains: false
ssl:
  enabled: false
system_cron: true

I think it’s worth mentioning that the file structure in the new_release_path looks like this:

.
|-- ansible
|   |-- group_vars
|   |-- hosts
|   |-- roles
|   |   |-- common
|   |   |-- composer
|   |   |-- deploy
|   |   |-- fail2ban
|   |   |-- ferm
|   |   |-- github-ssh-keys
|   |   |-- hhvm
|   |   |-- mariadb
|   |   |-- memcached
|   |   |-- nginx
|   |   |-- php
|   |   |-- rollback
|   |   |-- sshd
|   |   |-- ssmtp
|   |   |-- sudoers
|   |   |-- users
|   |   |-- wordpress-install
|   |   |-- wordpress-setup
|   |   `-- wp-cli
|   `-- vars
|-- config
|   `-- environments
|-- scripts
|   `-- Roots
|       `-- Bedrock
|-- site
|-- vendor
|   |-- composer
|   |   `-- installers
|   |-- johnpbloch
|   |   `-- wordpress-core-installer
|   `-- vlucas
|       `-- phpdotenv
`-- web
    |-- app
    |   |-- mu-plugins
    |   |-- plugins
    |   |-- themes
    |   `-- uploads -> /home/user/public_html/preview.domain.tld/shared/uploads
    `-- wp
        |-- wp-admin
        |-- wp-content
        `-- wp-includes

Any help is appreciated!

Thanks, Gérard

Can you post your full deploy log?

Yes, sure.

➜  ansible git:(master) ./deploy.sh staging staging.example.tld

PLAY [Deploy WP site] *********************************************************

GATHERING FACTS ***************************************************************
ok: [staging.example.tld]

TASK: [deploy | Initialize] ***************************************************
ok: [staging.example.tld]

TASK: [deploy | Clone project files] ******************************************
ok: [staging.example.tld]

TASK: [deploy | write unfinished file] ****************************************
changed: [staging.example.tld]

TASK: [deploy | Copy files to new build dir] **********************************
changed: [staging.example.tld]

TASK: [deploy | Move project subtree into root folder] ************************
changed: [staging.example.tld]

TASK: [deploy | Remove unwanted files/folders from new release] ***************
changed: [staging.example.tld] => (item=.git)

TASK: [deploy | Run pre_build_commands_local on Ansible host] *****************
skipping: [staging.example.tld]

TASK: [deploy | Copy project local files] *************************************
skipping: [staging.example.tld]

TASK: [deploy | Copy project templates] ***************************************
changed: [staging.example.tld] => (item={'dest': '.env', 'src': 'roles/deploy/templates/env.j2', 'name': '.env config'})

TASK: [deploy | Run pre_build_commands in the new_release_path] ***************
skipping: [staging.example.tld]

TASK: [deploy | Check if project folders exist] *******************************
ok: [staging.example.tld] => (item=vendor)

TASK: [deploy | Copy project folders] *****************************************
changed: [staging.example.tld] => (item={'invocation': {'module_name': u'stat', 'module_args': u'path="/home/username/public_html/staging.example.tld/current/vendor"'}, 'item': 'vendor', u'stat': {u'uid': 2058, u'exists': True, u'woth': False, u'mtime': 1434702937, u'inode': 1612072701, u'isgid': False, u'size': 85, u'wgrp': False, u'isuid': False, u'isreg': False, u'pw_name': u'username', u'gid': 2053, u'ischr': False, u'wusr': True, u'xoth': True, u'rusr': True, u'nlink': 5, u'issock': False, u'rgrp': True, u'gr_name': u'username', u'path': u'/home/username/public_html/staging.example.tld/current/vendor', u'xusr': True, u'atime': 1434702937, u'isdir': True, u'ctime': 1434702937, u'isblk': False, u'xgrp': True, u'dev': 64771, u'roth': True, u'isfifo': False, u'mode': u'0755', u'islnk': False}, u'changed': False})

TASK: [deploy | Ensure shared sources are present] ****************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Ensure shared paths are absent] *******************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Create shared symlinks] ***************************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Run post_build_commands in the new_release_path] **************
skipping: [staging.example.tld]

TASK: [deploy | Finalize the deploy] ******************************************
changed: [staging.example.tld]

TASK: [deploy | Run post_finalize_commands] ***********************************
skipping: [staging.example.tld]

TASK: [deploy | debug ] *******************************************************
ok: [staging.example.tld] => {
    "msg": "master@78362e1 deployed as release 20150620111529"
}

PLAY RECAP ********************************************************************
staging.example.tld : ok=15   changed=10   unreachable=0    failed=0

For your information: I never ran ansible-playbook -i hosts/staging server.yml because it is a shared hosting.

Thanks!

Did you make changes for that deploy run? Because it skipped post_build_commands instead of failing.

Only reason it would skip is you removed that composer task from the variable.

Unrelated: but you should add this line: https://github.com/roots/roots-example-project.com/blob/06c7f6abf84d58babaeef6a583469611cdd9b6ac/ansible/group_vars/all#L15

1 Like

Yes, I’m sorry, should’ve mention that.

I made these changes to ansible/roles/deploy/defaults/main.yml for the deploy to complete successfully:

project_post_build_commands: []
# - cmd: "composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts"

# Post finalize commands are run with Ansible's `shell` module.
# These are meant primarily for service restarts such as php5-fpm or memcached.
project_post_finalize_commands: []
#  - sudo service php5-fpm reload

I executed the composer command manually (working) and the second commented out line won’t work because of sudo/shared hosting.

A deploy with project_post_build_commands enabled looks like this:

➜  ansible git:(master) ✗ ./deploy.sh staging staging.example.tld

PLAY [Deploy WP site] *********************************************************

GATHERING FACTS ***************************************************************
ok: [staging.example.tld]

TASK: [deploy | Initialize] ***************************************************
ok: [staging.example.tld]

TASK: [deploy | Clone project files] ******************************************
ok: [staging.example.tld]

TASK: [deploy | write unfinished file] ****************************************
changed: [staging.example.tld]

TASK: [deploy | Copy files to new build dir] **********************************
changed: [staging.example.tld]

TASK: [deploy | Move project subtree into root folder] ************************
changed: [staging.example.tld]

TASK: [deploy | Remove unwanted files/folders from new release] ***************
changed: [staging.example.tld] => (item=.git)
changed: [staging.example.tld] => (item=ansible)
changed: [staging.example.tld] => (item=Vagrantfile)

TASK: [deploy | Run pre_build_commands_local on Ansible host] *****************
skipping: [staging.example.tld]

TASK: [deploy | Copy project local files] *************************************
skipping: [staging.example.tld]

TASK: [deploy | Copy project templates] ***************************************
changed: [staging.example.tld] => (item={'dest': '.env', 'src': 'roles/deploy/templates/env.j2', 'name': '.env config'})

TASK: [deploy | Run pre_build_commands in the new_release_path] ***************
skipping: [staging.example.tld]

TASK: [deploy | Check if project folders exist] *******************************
ok: [staging.example.tld] => (item=vendor)

TASK: [deploy | Copy project folders] *****************************************
changed: [staging.example.tld] => (item={'invocation': {'module_name': u'stat', 'module_args': u'path="/home/username/public_html/staging.example.tld/current/vendor"'}, 'item': 'vendor', u'stat': {u'uid': 2058, u'exists': True, u'woth': False, u'mtime': 1434792347, u'inode': 2956294210, u'isgid': False, u'size': 98, u'wgrp': False, u'isuid': False, u'isreg': False, u'pw_name': u'username', u'gid': 2053, u'ischr': False, u'wusr': True, u'xoth': True, u'rusr': True, u'nlink': 6, u'issock': False, u'rgrp': True, u'gr_name': u'username', u'path': u'/home/username/public_html/staging.example.tld/current/vendor', u'xusr': True, u'atime': 1434792347, u'isdir': True, u'ctime': 1434792347, u'isblk': False, u'xgrp': True, u'dev': 64771, u'roth': True, u'isfifo': False, u'mode': u'0755', u'islnk': False}, u'changed': False})

TASK: [deploy | Ensure shared sources are present] ****************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Ensure shared paths are absent] *******************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Create shared symlinks] ***************************************
changed: [staging.example.tld] => (item={'path': 'web/app/uploads', 'src': 'uploads'})

TASK: [deploy | Run post_build_commands in the new_release_path] **************
failed: [staging.example.tld] => (item={'cmd': 'composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts'}) => {"cmd": "composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts", "failed": true, "item": {"cmd": "composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts"}, "rc": 2}
msg: [Errno 2] No such file or directory

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/Users/scherii/deploy.retry

staging.example.tld : ok=13   changed=9    unreachable=0    failed=1

Thanks for the project_unwanted_items!
Shouldn’t that be in https://github.com/roots/bedrock-ansible/blob/master/group_vars/all as well?
Or is it just because my git repo includes the whole project folder (ansible and site), like subtree: site?

I just realized that overwriting the values in roles/deploy/defaults/main.yml may not be the best practice and put all the configuration in group_vars/<staging/production>.

I tinkered around with some commands, the only one I got to work (and could check) was touch filename.
As soon as I switch to a command like wp or composer the error message appears:

msg: [Errno 2] No such file or directory

Seeeeeeems to be a shared hosting thing.
If I change the command from

`composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts``

to

php ~/bin/composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts

it works flawlessly.

Does anyone know why I have to specify the composer path?

When I execute which composer via SSH it prints the correct path to composer.

Tools like Ansible and Capistrano that issue SSH commands do so “non-interactively”. Here’s a good resource on this and the answer to your question: http://capistranorb.com/documentation/faq/why-does-something-work-in-my-ssh-session-but-not-in-capistrano/

Good job debugging :thumbsup:

2 Likes

This is great, thank you! It makes sense now.

I’m still trying to get the whole picture – but it gets clearer.

(As far as I’m concerned this topic can be closed.)