Bedrock & Capistrano deploy fails

philipp · February 3, 2018, 3:25pm

Hi!

I’m using Bedrock and Capistrano in a couple of projects. But this time I’m very frustrated as I cant’t find the source of the error, which I’m receiving when doing the deploy to production server. The deploy:checkjust works fine and I’m able to clone & pull from my gitlab account from the remote server. However something fails and the log is not very helpoful here.

I would be happy if anyone has an idea:

** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
[Deprecation Notice] Future versions of Capistrano will not load the Git SCM
plugin by default. To silence this deprecation warning, add the following to
your Capfile after `require "capistrano/deploy"`:

require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git

** Invoke deploy (first_time)
** Execute deploy
** Invoke deploy:starting (first_time)
** Execute deploy:starting
** Invoke deploy:check (first_time)
** Invoke git:check (first_time)
** Invoke git:wrapper (first_time)
** Execute git:wrapper
00:00 git:wrapper
  01 mkdir -p /tmp
  01 export: Befehl nicht gefunden.
✔ 01 admin@my.server.net 0.777s
  Uploading /tmp/git-ssh-my-project-production-philipp.sh 100.0%
  02 chmod 700 /tmp/git-ssh-my-project-production-philipp.sh
  02 export: Befehl nicht gefunden.
✔ 02 admin@my.server.net 0.240s
** Execute git:check
00:01 git:check
  01 git ls-remote git@gitlab.com:faro.digital/my-project.git HEAD
  01 export: Befehl nicht gefunden.
  01 a549513177da6347e1517bf6c16b9192cc54db44       HEAD
✔ 01 admin@my.server.net 1.989s
** Execute deploy:check
** Invoke deploy:check:directories (first_time)
** Execute deploy:check:directories
00:03 deploy:check:directories
  01 mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/releases
  01 export: Befehl nicht gefunden.
✔ 01 admin@my.server.net 0.137s
** Invoke deploy:check:linked_dirs (first_time)
** Execute deploy:check:linked_dirs
00:03 deploy:check:linked_dirs
  01 mkdir -p /usr/local/www/apache24/noexec/my-project/shared/web/app/uploads
  01 export: Befehl nicht gefunden.
✔ 01 admin@my.server.net 0.244s
** Invoke deploy:check:make_linked_dirs (first_time)
** Execute deploy:check:make_linked_dirs
00:03 deploy:check:make_linked_dirs
  01 mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/shared/web
  01 export: Befehl nicht gefunden.
✔ 01 admin@my.server.net 0.242s
** Invoke deploy:check:linked_files (first_time)
** Execute deploy:check:linked_files
** Invoke deploy:set_previous_revision (first_time)
** Execute deploy:set_previous_revision
** Invoke deploy:started (first_time)
** Execute deploy:started
** Invoke deploy:updating (first_time)
** Invoke deploy:new_release_path (first_time)
** Execute deploy:new_release_path
** Invoke git:create_release (first_time)
** Invoke git:update (first_time)
** Invoke git:clone (first_time)
** Invoke git:wrapper
** Execute git:clone
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as admin@my.server.net: if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi exit status: 1
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stdout: Nothing written
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stderr: if: Ausdruck nicht korrekt.
fi: Befehl nicht gefunden.
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'

Caused by:
SSHKit::Command::Failed: if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi exit status: 1
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stdout: Nothing written
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stderr: if: Ausdruck nicht korrekt.
fi: Befehl nicht gefunden.
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/command.rb:99:in `exit_status='
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/netssh.rb:169:in `execute_command'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:141:in `block in create_command_and_execute'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:141:in `tap'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:141:in `create_command_and_execute'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:74:in `execute'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:79:in `within'
/Library/Ruby/Gems/2.3.0/gems/capistrano-3.10.1/lib/capistrano/scm/tasks/git.rake:30:in `block (3 levels) in eval_rakefile'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:29:in `instance_exec'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/backends/abstract.rb:29:in `run'
/Library/Ruby/Gems/2.3.0/gems/sshkit-1.15.1/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
Tasks: TOP => git:create_release => git:update => git:clone
The deploy has failed with an error: Exception while executing as admin@my.server.net: if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi exit status: 1
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stdout: Nothing written
if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi stderr: if: Ausdruck nicht korrekt.
fi: Befehl nicht gefunden.
** Invoke deploy:failed (first_time)
** Execute deploy:failed


** DEPLOY FAILED
** Refer to log/capistrano.log for details. Here are the last 20 lines:


  INFO [00e9f2ce] Finished in 0.242 seconds with exit status 0 (successful).

 DEBUG [4ce85782] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ] as admin@my.server.net

 DEBUG [4ce85782] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ]

 DEBUG [4ce85782] Finished in 0.238 seconds with exit status 0 (successful).

 DEBUG [d73e7feb] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/web/.htaccess ] as admin@my.server.net

 DEBUG [d73e7feb] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/web/.htaccess ]

 DEBUG [d73e7feb] Finished in 0.237 seconds with exit status 0 (successful).

 DEBUG [194fe637] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ] as admin@my.server.net

 DEBUG [194fe637] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ]

 DEBUG [194fe637] Finished in 0.236 seconds with exit status 0 (successful).

 DEBUG [5a701fa9] Running [ -f /usr/local/www/apache24/noexec/my-project/current/REVISION ] as admin@my.server.net

 DEBUG [5a701fa9] Command: [ -f /usr/local/www/apache24/noexec/my-project/current/REVISION ]

 DEBUG [5a701fa9] Finished in 0.233 seconds with exit status 1 (failed).

 DEBUG [08ac9087] Running [ -f /usr/local/www/apache24/noexec/my-project/repo/HEAD ] as admin@my.server.net

 DEBUG [08ac9087] Command: [ -f /usr/local/www/apache24/noexec/my-project/repo/HEAD ]

 DEBUG [08ac9087] Finished in 0.233 seconds with exit status 1 (failed).

 DEBUG [7d11ebd2] Running if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi as admin@my.server.net

 DEBUG [7d11ebd2] Command: if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi

 DEBUG [7d11ebd2]       if: Ausdruck nicht korrekt.

 DEBUG [7d11ebd2]       fi: Befehl nicht gefunden.

swalkinshaw · February 3, 2018, 4:59pm

That’s pretty weird.

It successfully creates directories as seen in mkdir -p /usr/local/www/apache24/noexec/my-project/shared/web/app/uploads.

Yet then complains /usr/local/www/apache24/noexec/my-project does not exist.

Can you SSH into the server and verify it’s there? Maybe that entire parent directory needs a permissions reset.

philipp · February 3, 2018, 5:06pm

Yes! I know, confusing me for two days now. I’m connected via SSH and the folder exists. Also already deleted it a couple of times and everything is set up as expected as well as the /usr/local/www/apache24/noexec/my-project folder. What is now also in the Capistrano.log:

  INFO ---------------------------------------------------------------------------
  INFO START 2018-02-03 17:58:57 +0100 cap production deploy
  INFO ---------------------------------------------------------------------------
  INFO [c78731f7] Running /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/tmp as admin@my.server
 DEBUG [c78731f7] Command: ( export WP_ENV="production" ; /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/tmp )
 DEBUG [c78731f7] 	export: Befehl nicht gefunden.
  INFO [c78731f7] Finished in 0.786 seconds with exit status 0 (successful).
 DEBUG Uploading /usr/local/www/apache24/noexec/my-project/tmp/git-ssh-my-project-production-philipp.sh 0.0%
  INFO Uploading /usr/local/www/apache24/noexec/my-project/tmp/git-ssh-my-project-production-philipp.sh 100.0%
  INFO [3301467d] Running /usr/bin/env chmod 700 /usr/local/www/apache24/noexec/my-project/tmp/git-ssh-my-project-production-philipp.sh as admin@my.server
 DEBUG [3301467d] Command: ( export WP_ENV="production" ; /usr/bin/env chmod 700 /usr/local/www/apache24/noexec/my-project/tmp/git-ssh-my-project-production-philipp.sh )
 DEBUG [3301467d] 	export: Befehl nicht gefunden.
  INFO [3301467d] Finished in 0.241 seconds with exit status 0 (successful).
  INFO [8055abfb] Running /usr/bin/env git ls-remote git@gitlab.com:faro.digital/fasino_design.git HEAD as admin@my.server
 DEBUG [8055abfb] Command: ( export WP_ENV="production" GIT_ASKPASS="/bin/echo" GIT_SSH="/usr/local/www/apache24/noexec/my-project/tmp/git-ssh-my-project-production-philipp.sh" ; /usr/bin/env git ls-remote git@gitlab.com:faro.digital/fasino_design.git HEAD )
 DEBUG [8055abfb] 	export: Befehl nicht gefunden.
  INFO [8055abfb] Finished in 2.159 seconds with exit status 0 (successful).
  INFO [f755e1ab] Running /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/releases as admin@my.server
 DEBUG [f755e1ab] Command: ( export WP_ENV="production" ; /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/releases )
 DEBUG [f755e1ab] 	export: Befehl nicht gefunden.
  INFO [f755e1ab] Finished in 0.133 seconds with exit status 0 (successful).
  INFO [f7f2093d] Running /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared/web/app/uploads as admin@my.server
 DEBUG [f7f2093d] Command: ( export WP_ENV="production" ; /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared/web/app/uploads )
 DEBUG [f7f2093d] 	export: Befehl nicht gefunden.
  INFO [f7f2093d] Finished in 0.244 seconds with exit status 0 (successful).
  INFO [496c3659] Running /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/shared/web as admin@my.server
 DEBUG [496c3659] Command: ( export WP_ENV="production" ; /usr/bin/env mkdir -p /usr/local/www/apache24/noexec/my-project/shared /usr/local/www/apache24/noexec/my-project/shared/web )
 DEBUG [496c3659] 	export: Befehl nicht gefunden.
  INFO [496c3659] Finished in 0.235 seconds with exit status 0 (successful).
 DEBUG [00a0907b] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ] as admin@my.server
 DEBUG [00a0907b] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ]
 DEBUG [00a0907b] Finished in 0.244 seconds with exit status 0 (successful).
 DEBUG [8b5478be] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/web/.htaccess ] as admin@my.server
 DEBUG [8b5478be] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/web/.htaccess ]
 DEBUG [8b5478be] Finished in 0.235 seconds with exit status 0 (successful).
 DEBUG [4af3a8e3] Running [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ] as admin@my.server
 DEBUG [4af3a8e3] Command: [ -f /usr/local/www/apache24/noexec/my-project/shared/.env ]
 DEBUG [4af3a8e3] Finished in 0.239 seconds with exit status 0 (successful).
 DEBUG [02a24f5f] Running [ -f /usr/local/www/apache24/noexec/my-project/current/REVISION ] as admin@my.server
 DEBUG [02a24f5f] Command: [ -f /usr/local/www/apache24/noexec/my-project/current/REVISION ]
 DEBUG [02a24f5f] Finished in 0.242 seconds with exit status 1 (failed).
 DEBUG [1b66492f] Running [ -f /usr/local/www/apache24/noexec/my-project/repo/HEAD ] as admin@my.server
 DEBUG [1b66492f] Command: [ -f /usr/local/www/apache24/noexec/my-project/repo/HEAD ]
 DEBUG [1b66492f] Finished in 0.233 seconds with exit status 1 (failed).
 DEBUG [32dfa07e] Running if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi as admin@my.server
 DEBUG [32dfa07e] Command: if test ! -d /usr/local/www/apache24/noexec/my-project; then echo "Directory does not exist '/usr/local/www/apache24/noexec/my-project'" 1>&2; false; fi
 DEBUG [32dfa07e] 	if: Ausdruck nicht korrekt.
 DEBUG [32dfa07e] 	fi: Befehl nicht gefunden.

The folder “repo” is missing as well “current” as symlink. The folder “releases” stays empty. It seems to abort when cloning the git, doesn’t it?

Thank you so much!

swalkinshaw · February 3, 2018, 5:12pm

Looks like it. Maybe try manually running Git commands on the remote server and see if that offers any clues.

philipp · February 3, 2018, 5:14pm

Already tried “git clone” the repository and that worked fine. Otherwise which commands can I try to follow the path of the deploy?

Best,

Philipp

swalkinshaw · February 3, 2018, 5:29pm

Every line in your log that starts with “command” is an actual bash command you can run.

Not really sure how much help I can provide beyond that unfortunately. It’s a weird error

philipp · February 3, 2018, 5:38pm

OK, thank you! I’ll try that. Any idea what the output “export: command not found.” might be related to? I’m getting it all the time and that might be the root of the issue?

swalkinshaw · February 3, 2018, 5:56pm

export is a pretty important unix command so yeah if that doesn’t work I doubt much else would. It sets an environment variable so it’s available to processes.

philipp · February 3, 2018, 6:06pm

Thank you @swalkinshaw I guess we’re getting closer. I’ll be happy to invite you for some coffee.

echo $SHELL tells me on the FreeBSD server: /bin/csh.
For this I found the following:

It looks like you are using csh. It has a different syntax
than /bin/sh. Among other things it does not use the ‘export’ command.

This is far away from the topic (Bedrock) now. But you might have an idea how I could get around that. E.g. any chance to tell Capistrano to use sh?

swalkinshaw · February 3, 2018, 6:35pm

Depends if Bash is installed at all. Try running which bash or which sh to see if it returns something.

If it does, I think you can set it via default_run_options[:shell] = '/bin/bash' (replace with the path from the which cmd).

edit: to be clear, add default_run_options[:shell] = '/path/to/bash'in your Capistrano config file near the top.

philipp · February 3, 2018, 8:24pm

Dear @swalkinshaw,

again thank you very much for your support and help. I’ll share this thread with a server administrator as I’m not able to solve it. Adding default_run_options[:shell] = '/path/to/bash'sadly doesn’t work as Capistrano directly stops execution with
cap aborted! NameError: undefined local variable or method default_run_options' for main:Object Did you mean? default_tasks

I added a new user to FreeBSD just for deploys who uses /bin/sh for the shell. This solves the issue of export: Command not found but also generates permission and operation errors when Capistrano executes /tmp/git-ssh-my-project-production-user.sh. The user is even in the same group…

However thank you very much and please let me know how I can invite you for a coffee or beer!

Philipp

swalkinshaw · February 3, 2018, 8:59pm

Oops, sorry. default_run_options was for Capistrano v2. It doesn’t exist in v3 unfortunately.

It looks like it’s not possible to specify a specific shell: Problems when remote user has tcsh or csh as default shell · Issue #1870 · capistrano/capistrano · GitHub

Looks like you might be on the right track with a user set to /bin/sh. Hopefully you can figure that out.

If you’re ever in Toronto, let me know

philipp · February 5, 2018, 9:55am

I was finally able to solve this and some more deploy issues. For later reference:

1.) Added a new user (e.g. ‘deploy’) with another default shell.

2.) Go sure that the folder to which the deploy is done has “exec” permission. In my case the public folder in which websites are saved is “noexec” so I had to deploy to the mirrored folder for which this restriction is not set. Especially for the /tmp folder which can be set:
set :tmp_dir, "/path/to/tmp"

3.) To set the path for BASH in SSHKit:
SSHKit.config.command_map[:bash] = "/usr/iports/bin/bash"

4.) To set another PHP version and/or path for composer to run (e.g. with openssl.so as extension):
SSHKit.config.command_map[:composer] = "/usr/php71/bin/php /usr/bin/composer"

5.) Learn new stereotypes: Toronto is a city with helpful people.

Thanks!