Provision fails on 'Install Dependencies with Composer Task'

trellis
1

Hi,

is there any possible way to check why the following task “Install Dependencies with Composer” fails?
(roles/wordpress-install/tasks/main.yml).

If I do

vagrant provision --reload

provision fails

TASK: [wordpress-install | Install Dependencies with Composer] **************** 
fatal: [default] => error while evaluating conditional: 'Nothing to install or update' not in composer_results.stderr

If I do:

cd ~/trellis
vagrant ssh
cd /srv/www/project.dev/current/
composer install

Everything works fine. Then

  vagrant reload --provision

The Task succeded.

I have to admit that the problem is probably that my composer.json is modified. But I would want to know why it fails on provision.

I think I will paste u my composer.json here to clarify the problem: http://pastebin.com/bUXrxNHZ

Regards Kevin

1 Like
2

@kregenrek I suspect you won’t see the error again, now that your composer seems to have sorted itself out.

My guess is that the composer install command produced some unusual output for some reason. The task registers the output in a composer_results variable. This variable typically has a stderr attribute, which is evaluated in the changed_when parameter, but I’m guessing that for some reason this attribute was missing. This means composer_results.stderr would have been undefined, causing Ansible to throw an error while trying to evaluate it in the changed_when conditional.

If you run into the error again, you could make the following edits to the task and share the debug output with us.

  - name: Install Dependencies with Composer
    command: composer install
    args:
      chdir: "{{ www_root }}/{{ item.key }}/current/"
    register: composer_results
    with_dict: wordpress_sites
-   changed_when: "'Nothing to install or update' not in composer_results.stderr"
+   # changed_when: "'Nothing to install or update' not in composer_results.stderr"

+ - debug: var=composer_results
+
3

@fullyint thank you very much for your response.

It fails either on the SSH Fingerprint prompt for bitbucket or on the composer xdebug output…

I’ve read you can prevent this question by doing something like here http://askubuntu.com/questions/123072/ssh-automatically-accept-keys

Is there any easy soluton for that?

TASK: [wordpress-install | Install Dependencies with Composer] **************** 
failed: [default] => ...
BECOME-SUCCESS-rfcykaxnhbhhsuqllpcposbsjgroumjw
The authenticity of host 'bitbucket.org (104.192.143.3)' can't be established.
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)? 
{"changed": true, "end": "2016-01-23 11:40:11.544962", "stdout": "", 
"cmd": ["composer", "install"],
"start": "2016-01-23 11:35:08.935145", "delta": "0:05:02.609817", "stderr": "You are running composer with      
xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug\nLoading composer repositories with package 
   
information\n\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\
b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b                                                      \b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b
\b\b\b\b\b\b\b\b\b\b\b\b\b\bReading composer.json of kkern/wp-aqua-resizer (master)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b
\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b                                                       \b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b
\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n                                                                                                                                                                                                            \n  [Symfony\\Component\\Process\\Exception\\ProcessTimedOutException]                                                                                                                                            \n  The process \"git clone --mirror 'git@bitbucket.org:regenrek/commercial-plugins.git' '/root/.composer/cache/vcs/git-bitbucket.org-regenrek-commercial-plugins.git/'\" exceeded the timeout of 300 seconds.  \n                                                                                                                                                                                                            \n\ninstall [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-plugins] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--ignore-platform-reqs] [--] [<packages>]...", "rc": 1, "warnings": []}
OpenSSH_6.9p1, LibreSSL 2.1.8
debug1: Reading configuration data /Users/**/.ssh/config
debug1: /Users/**/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: auto-mux: Trying existing master
debug1: mux_client_request_session: master session id: 2
Shared connection to 127.0.0.1 closed.
1 Like
4

I’m pretty sure it’s failing on the SSH connection and not that xdebug warning.

You could add a simple task to add that known host. Ansible has a module for it: http://docs.ansible.com/ansible/known_hosts_module.html

- name: Add known hosts
  known_hosts:
    name: "{{ item }}"
  with_items:
    - bitbucket.org

You could add that to a hook file for the deploy_before step. See our https://roots.io/trellis/docs/deploys/#hooks docs for instructions.

2 Likes
5

@kregenrek roots/trellis#475 proposes an option to add known_hosts. Any testing or feedback would be very welcome.

1 Like
6

@fullyint Sorry for the late reply I didn’t get the chance yet to test this out.

EDIT: It worked now. I overlooked that my sshkey wasn’t loaded ssh-add -L … worked now

I merge your known-hosts branch into the actual master branch and tried it.
So i think it will work with bitbucket and github.
But I have problems to get my own bitbucket codebase to work (stash).
I added my the public key to the known hosts file.

known_hosts:
  - name: codebase.example.com
    key: codebase.example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQ...
  - name: github.com
    key: github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAA...
  - name: bitbucket.org
    key: bitbucket.org ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAubiN81eDcafrgMeLzaFPsw2kNvEcqTKl/VqLat/MaB33pZy0y3rJZtnqwR2qOOvbwKZYKiEO1O6Vq...

But is this really necessary since there is an accept_hostkeys option:

# Whether `git clone` and `composer install` should accept host keys into known_hosts
# If `no`, manually add host keys to `group_vars/all/known_hosts.yml`
accept_hostkeys: yes

So then I had the problem with this here:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

https://github.com/Varying-Vagrant-Vagrants/VVV/issues/360
https://laracasts.com/discuss/channels/general-discussion/warning-remote-host-identification-has-changed-wtf

So i removed all keys via

vagrant ssh
and cleared the while ~/.ssh/known_hosts file.

Good again I try … now I got again

failed: [default] => ....... The authenticity of host 'codebase.example.com (x.x.x.x)' can't be established.\r\n
RSA key fingerprint is 2c:fe:f1:a9:0a:7a:8c:bc:ca:dc:85:9d:65:24:08:ce.\r\nAre you sure you want to continue
connecting (yes/no)? {\"changed\": true, \"end\": \"2016-02-22 20:24:18.551322\", \"stdout\": \"\", \"cmd\": 
[\"composer\", \"install\"], \"rc\": 1, \"start\": \"2016-02-22 20:19:17.015026\", \"stderr\": \"You are running 
composer with xdebug enabled. This has a major impact on runtime performance. See 
https://getcomposer.org/xdebug\\nLoading composer repositories with package information\\nInstalling 
dependencies (including require-dev) from lock file\\n  - Installing regenrek/commercial-plugins (dev-master 
7f62f3d)\\n    Cloning 7f62f3d4fc053eed3ac31aca7382fe15c5c176bd\\n\\n                                                                                                                                                                                                                                                                                                                                 \\n  [Symfony\\\\Component\\\\Process\\\\Exception\\\\ProcessTimedOutException]                                                                                                                                                                                                                                                                 \\n  The process \\\"git clone --no-checkout 'ssh://git@codebase.example.com/intp/commercial-plugins.git' 
'web/app/plugins/commercial-plugins/' && cd 'web/app/plugins/commercial-plugins/' && git remote add 
composer 'ssh://git@codebase.example.com/intp/commercial-plugins.git' && git fetch composer\\\" exceeded the 
timeout of 300 seconds.  \\n                                                                                                                                                                         
                                                                                                                                                    \\n\\ninstall [--prefer-source]
 [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-plugins] [--no-custom-installers] [--no-autoloader]
 [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] 
[--ignore-platform-reqs] [--] [<packages>]...\", \"delta\": \"0:05:01.536296\", \"invocation\": {\"module_args\": {\"warn\": true, \"executable\": null, \"_uses_shell\": false, \"_raw_params\": \"composer install\", \"removes\": 
null, \"creates\": null, \"chdir\": \"/srv/www/djf.dev/current/\"}}, \"warnings\": []}\r\n", "msg": "MODULE FAILURE",
 "parsed": false}

I dont know… I’m struggling with this thing a long time now… Is there an safe way to disable the composer install via the trellis provision process?

Regards Kevin

7

@kregenrek Thank you so much for testing and providing feedback. I think you were pretty close. I plan to revise roots/trellis#475 soon and hopefully it will work for you.

You could comment out this task temporarily. In doing so you would be modifying Trellis core, which you would normally want to avoid. I’ll try to post back if/when roots/trellis#475 is merged so you can uncomment those lines, merge in the new code, and try again.

8

Hi,

i found some problems:

https://github.com/fullyint/trellis/blob/known-hosts/roles/common/tasks/main.yml

- name: Add known_hosts
  known_hosts:
    name: "{{ item.name }}"
    key: "{{ item.key | default(omit) }}"
    path: "{{ item.path | default('/home/' + web_user + '/.ssh/known_hosts') }}"
    state: "{{ item.state | default('present') }}"
  with_items: known_hosts | default([])
  tags: known-hosts

If you provision the server you will get an error if the web_user doesn’t exist.
So it will be better to add this line after the roles/users Task.

I’m gonna test the deploy then.

Regards Kevin

1 Like
9

Ahh, excellent! Thank you! I think my testing included removing hostkeys and reprovisioning, but not removing users, so I already had the web_user. But as you pointed out, the web_user won’t be there on the first provision. Thank you!

As a heads up, my upcoming revision of that PR will remove the option of auto-detecting and adding hostkeys for hosts in composer.json. I judged that it wasn’t a good idea to do a lot of extra work to create a feature that makes people vulnerable to MITM attacks (while running ssh-keyscan -t to collect hostkeys during provision). Better to just stay secure. The default known_hosts list will include github and bitbucket, so the majority of users won’t have to think about it.

So, users whose composer.json includes a different git host will need to manually list hostkeys in that new known_hosts list variable (then Trellis will load them), which should still be easier than requiring users to load keys on their own.

My work on this has stalled while I create a messaging plugin for Trellis, one feature of which will give more targeted guidance when hostkeys are missing for git hosts in composer.json. These messages will make life easier for the next person encountering the original issue of this thread, giving explanation and tips on how to resolve. As the messaging plugin finalizes soon, I’ll polish up the known_hosts PR.

1 Like