Deploying Bedrock Sites

Originally published at: https://roots.io/deploying-bedrock-sites/
Bedrock was created with Capistrano as its built-in deployment tool and its remained that way for almost 18 months. Very soon, the Capistrano specific files and configs in Bedrock will be moved to its own separate repository but still easy to use. You can follow the progress at https://github.com/roots/bedrock/pull/163 and check out the new repo.…

5 Likes

Great news, it’s awesome to simplify the project without losing much or any features.

Reminds me as well to try and keep up with PR’s and commits of these projects I use all the time, need to make sure I know what’s going on!

I’m just starting out with Bedrock and I really like it!

One question though. Let’s say that you’re working on a big site with a lot of assets and you hook gulp tasks to deployment, won’t the production environment be non functional (or at least unstyled content) during the duration of the gulp build?

No, the good part about the Ansible/Capistrano deploy process is the new deploy is only used (via symlink) one the deploy is complete.

@swalkinshaw Ok thanks! I’m not totally sure I understand. Are there any good resources (articles, screencasts) on the subject that you can recommend?

Not off the top of my head. I’m sure if you google there’s resources about it. Best bet is to search about Capistrano.

But very simply:

With this setup your site/virtual host in Nginx (or Apache) points to the current symlink. Let’s say your latest release looks like this:

current/ => releases/201504130000

So your symlink points to a release done on April 13th for example.

Now today you do another deploy. The steps look like this (roughly):

  1. Create a new directory releases/201504141648
  2. Pull latest code (through Git)
  3. Run any commands (such as gulp --production)
  4. Update current symlink to point to releases/201504141648

Step #4 only happens if #1-3 were successful. It doesn’t matter how long those steps take, the symlinking doesn’t happen until the end.

3 Likes

Ansible looks great. Time to get back on the learning curve.

I see there is an SSL branch in github, is this likely to land soon?

Thanks for all your efforts, as ever.

@treb0r very soon, it’s all working just need some documentation updates.

So I spent the afternoon getting to grips with bedrock-ansible, and it worked liked a charm.

I have a remote VM provisioned, and everything works.

I’ve hit a problem trying to run gulp on the remote server to compile the assets.

I’ve installed node with nvm on the remote VM:

web@vm:/root$ nvm ls
        v0.12.2
node -> stable (-> v0.12.2) (default)
stable -> 0.12 (-> v0.12.2) (default)

I followed your instructions from the blog post and added the following to roles -> deploy -> defaults -> main.yml (under project_post_build_commands:)

  - path: web/app/themes/csv
    cmd: npm install && gulp --production

The deploy runs fine, composer works, but then I get the following error:

failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'npm install && gulp --production'}) => {"changed": true, "cmd": ["npm", "install", "&&", "gulp", "--production"], "delta": "0:00:00.005051", "end": "2015-04-16 10:53:24.376116", "item": {"cmd": "npm install && gulp --production", "path": "web/app/themes/csv"}, "rc": 127, "start": "2015-04-16 10:53:24.371065", "warnings": []}
stderr: /usr/bin/env: node: No such file or directory

FATAL: all hosts have already failed -- aborting

When I try running npm install in the failed release directory it seems to work. The output is large but I can post it here if anyone is interested.

Just for the record, I checked that nvm is setup correctly:

web@vm:~$ nvm current
v0.12.2

It seems okay. Am I missing something?

Am I correct in thinking that the remote ‘web’ user is used for deploys?

I think Ansible is like Capistrano in that it doesn’t run in full TTY mode. Meaning running a command (even as the same user) is different than actually SSHing manually and running a command. That’s because the full Bash environment isn’t loaded.

nvm works by having your PATH altered. So I’m guessing it’s just missing.

If you log in and run which node you can add that path to your $PATH.

Or you can just change the project_post_build_commands to reference the full path to node.

Or look into overriding project_environment: https://github.com/roots/bedrock-ansible/blob/master/roles/deploy/defaults/main.yml#L62-L63

3 Likes

Okay, given that bash doesn’t seem to be loading any dotfiles how do I add that path to $path?

I think this would work:

project_environment:
  PATH: "$PATH:/home/{{ web_user }}/.nvm/v0.12.2/bin"
  WP_ENV: "{{ project.env.wp_env }}"

Assuming that path is correct.

So I tried adding the path as suggested but then composer failed to run.

I decided to install nodejs 0.12.2 using apt via nodesource, from here:

Now the deploy at least runs the npm install command.

The error:

failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'npm install && gulp --production'}) => {"changed": true, "cmd": ["npm", "install", "&&", "gulp", "--production"], "delta": "0:00:00.951442", "end": "2015-04-17 11:29:06.674056", "item": {"cmd": "npm install && gulp --production", "path": "web/app/themes/csv"}, "rc": 254, "start": "2015-04-17 11:29:05.722614", "warnings": []}
stderr: npm ERR! addLocal Could not install /srv/www/test.org.uk/releases/20150417112848/web/app/themes/csv/&&
npm ERR! Linux 3.13.0-43-generic
npm ERR! argv "node" "/usr/local/bin/npm" "install" "&&" "gulp" "--production"
npm ERR! node v0.12.2
npm ERR! npm  v2.7.6
npm ERR! path /srv/www/test.org.uk/releases/20150417112848/web/app/themes/csv/&&
npm ERR! code ENOENT
npm ERR! errno -2

npm ERR! enoent ENOENT, open '/srv/www/test.org.uk/releases/20150417112848/web/app/themes/csv/&&'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! Please include the following file with any support request:
npm ERR!     /srv/www/test.org.uk/releases/20150417112848/web/app/themes/csv/npm-debug.log

FATAL: all hosts have already failed -- aborting

The npm-debug.log: http://pastebin.com/ZAje7fVh

That path looks weird.

Try splitting up those commands:

  - path: web/app/themes/csv
    cmd: npm install
  - path: web/app/themes/csv
    cmd: gulp --production

Getting a bit further now.

TASK: [deploy | Run post_build_commands in the new_release_path] ************** 
changed: [46.101.12.18] => (item={'cmd': 'composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts'})
failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'npm install'}) => {"changed": true, "cmd": ["npm", "install"], "delta": "0:03:30.521274", "end": "2015-04-18 12:29:52.136475", "item": {"cmd": "npm install", "path": "web/app/themes/csv"}, "rc": 1, "start": "2015-04-18 12:26:21.615201", "warnings": []}
stderr: npm WARN engine img-stats@0.4.2: wanted: {"node":"~0.10"} (current: {"node":"0.12.2","npm":"2.7.6"})
npm WARN optional dep failed, continuing fsevents@0.3.5
  ✔ pngcrush pre-build test passed successfully
  ✔ jpegtran pre-build test passed successfully
stdout: 
> ws@0.5.0 install /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/engine.io/node_modules/ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/engine.io/node_modules/ws/build'
  CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
  SOLINK_MODULE(target) Release/obj.target/bufferutil.node
  SOLINK_MODULE(target) Release/obj.target/bufferutil.node: Finished
  COPY Release/bufferutil.node
  CXX(target) Release/obj.target/validation/src/validation.o
  SOLINK_MODULE(target) Release/obj.target/validation.node
  SOLINK_MODULE(target) Release/obj.target/validation.node: Finished
  COPY Release/validation.node
make: Leaving directory `/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/engine.io/node_modules/ws/build'

> ws@0.4.31 install /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/socket.io-client/node_modules/engine.io-client/node_modules/ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)

make: Entering directory `/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/socket.io-client/node_modules/engine.io-client/node_modules/ws/build'
  CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
make: Leaving directory `/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/socket.io-client/node_modules/engine.io-client/node_modules/ws/build'

> pngcrush-bin@2.2.1 postinstall /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/imagemin-pngcrush/node_modules/pngcrush-bin
> node lib/install.js


> jpegtran-bin@2.0.2 postinstall /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/gulp-imagemin/node_modules/imagemin/node_modules/imagemin-jpegtran/node_modules/jpegtran-bin
> node lib/install.js


> gifsicle@2.0.1 postinstall /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/gulp-imagemin/node_modules/imagemin/node_modules/imagemin-gifsicle/node_modules/gifsicle
> node lib/install.js
failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'gulp --production'}) => {"changed": true, "cmd": ["gulp", "--production"], "delta": "0:00:01.474344", "end": "2015-04-18 12:29:54.468991", "item": {"cmd": "gulp --production", "path": "web/app/themes/csv"}, "rc": 1, "start": "2015-04-18 12:29:52.994647", "warnings": []}
stderr: /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/asset-builder/node_modules/main-bower-files/lib/index.js:54
        throw new Error('Bower components directory does not exist at ' + bowe
              ^
Error: Bower components directory does not exist at /srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/bower_components
    at module.exports (/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/asset-builder/node_modules/main-bower-files/lib/index.js:54:15)
    at new module.exports (/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/asset-builder/lib/Manifest.js:29:15)
    at module.exports (/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/node_modules/asset-builder/index.js:17:10)
    at Object.<anonymous> (/srv/www/test.org.uk/releases/20150418122604/web/app/themes/csv/gulpfile.js:11:40)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)

FATAL: all hosts have already failed -- aborting

I tried running bower install in the theme directory (in the failed release) and it works. As does gulp --production.

I checked and bower_components and bower.json are both there as expected.

Oh that’s a simple fix which I forgot earlier:

  - path: web/app/themes/csv
    cmd: npm install
  - path: web/app/themes/csv
    cmd: bower install
  - path: web/app/themes/csv
    cmd: gulp --production

Thanks, should have been able to figure that out myself.

More errors:

TASK: [deploy | Run post_build_commands in the new_release_path] ************** 
changed: [46.101.12.18] => (item={'cmd': 'composer install --no-ansi --no-dev --no-interaction --no-progress --optimize-autoloader --no-scripts'})
failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'npm install'}) => {"changed": true, "cmd": ["npm", "install"], "delta": "0:03:32.041572", "end": "2015-04-19 11:42:10.983588", "item": {"cmd": "npm install", "path": "web/app/themes/csv"}, "rc": 1, "start": "2015-04-19 11:38:38.942016", "warnings": []}
stderr: npm WARN engine img-stats@0.4.2: wanted: {"node":"~0.10"} (current: {"node":"0.12.2","npm":"2.7.6"})
npm WARN optional dep failed, continuing fsevents@0.3.5
stdout: 
> ws@0.5.0 install /srv/www/test.org.uk/releases/20150419113821/web/app/themes/csv/node_modules/browser-sync/node_modules/socket.io/node_modules/engine.io/node_modules/ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)
changed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'bower install'})
failed: [46.101.12.18] => (item={'path': 'web/app/themes/csv', 'cmd': 'gulp --production'}) => {"changed": true, "cmd": ["gulp", "--production"], "delta": "0:00:04.965341", "end": "2015-04-19 11:42:21.193063", "item": {"cmd": "gulp --production", "path": "web/app/themes/csv"}, "rc": 1, "start": "2015-04-19 11:42:16.227722", "warnings": []}
stderr: 
events.js:85
      throw er; // Unhandled 'error' event
            ^
Error: spawn /srv/www/test.org.uk/releases/20150419113821/web/app/themes/csv/node_modules/gulp-imagemin/node_modules/imagemin/node_modules/imagemin-optipng/node_modules/optipng-bin/vendor/optipng ENOENT
    at exports._errnoException (util.js:746:11)
    at Process.ChildProcess._handle.onexit (child_process.js:1053:32)
    at child_process.js:1144:20
    at process._tickCallback (node.js:355:11)
stdout: [11:42:17] Using gulpfile /srv/www/test.org.uk/releases/20150419113821/web/app/themes/csv/gulpfile.js
[11:42:17] Starting 'clean'...
[11:42:17] Finished 'clean' after 7.29 ms
[11:42:17] Starting 'default'...
[11:42:17] Starting 'wiredep'...
[11:42:18] Starting 'jshint'...
[11:42:18] Starting 'fonts'...
[11:42:18] Starting 'images'...
[11:42:21] Finished 'default' after 3.1 s

FATAL: all hosts have already failed -- aborting

So I ran the deploy again, and it almost completely worked this time. The site deployed and the assets got compiled and the symlink got updated. The error:

TASK: [deploy | Finalize the deploy] ****************************************** 
failed: [46.101.12.18] => {"failed": true}
msg: rmtree failed: [Errno 13] Permission denied: '/srv/www/test.org.uk/releases/20150419113821/web/app/themes/csv/dist/fonts/glyphicons-halflings-regular.eot'

FATAL: all hosts have already failed -- aborting

One observation is that npm install seems a bit flaky, and it takes a long time to run (~3.5 minutes on a $10 / month DO droplet).

I’m wondering if I may be better off compiling the assets locally and uploading like I’ve previously done with Capistrano.

Yes compiling locally is the better method. We’re going to be officially recommending that.

We’re in the process of setting up an example project to show all Bedrock + Ansible + Sage all working and it’s already working with local compiles.

See https://github.com/roots/roots-example-project.com