Using Bedrock + Sage to deploy with Capistrano (theme Gulp dist files?)

1

I’m able to deploy with Capistrano and all is mostly well. However, my theme (built on Sage) doesn’t have the dist folder and all of the assets etc on the server when I deploy. I could add dist to the git repo and then it would be deployed t o the server that way, but it seems like I should probably be using Capistrano to generate these during deployment or something?

How do I configure Capistrano to run gulp --production and deploy the dist directory and all of its assets to my server during a Capistrano deploy?

Or maybe I’m looking at this the wrong way and there’s something obvious I am overlooking. What I want is to be able to run bundle exec cap staging(/production) deploy and have my Sage theme dist files included and on the server. What’s the best way to go about doing this?

There are some great resources for Bedrock, and some great resources for Sage. Are there any resources for using these two together?

3 Likes
2

This needs updating for Gulp but you should be able to figure it out.

3

thanks. this looks like what i’m going to want.

The main thing I’m going for here is to have the production server contain only the necessary files for the production environment. I don’t want it to include anything it doesn’t need to. Ideally, I’d rather it didn’t include composer.json in the site root (I’m using bedrock), though I suppose at least this isn’t in the web root.

When I read your comment last night (on my phone, sorry for the late reply), it got me thinking about what would be the ideal way to deploy.

I feel like I should have a “build” command on my development server that will compile everything, optimize it and remove the unnecessary “scaffolding” files like composer.json, package.json etc, and will zip everything up and upload that to github/bitbucket as a “release” archive. The server would then pull that and unpack it on the server, so I would end up with the server containing only the necessary files to run.

For example, looking at just the Sage theme, you’d rather not publish all of the .scss/.less files to your production server, and would rather include only the compiled/concatenated/minified css/js files.

However, as it stands with a Bedrock and Capistrano deploy, it appears as though it pulls the code from the repo itself on bitbucket (I’m using bitbucket, not github), and then actually runs composer update on the server.

Is there a better way? I feel like there must be. Do I want to look into CI servers or something? Sorry if this is all a bit off topic and/or rambling.

edit

Something like this: http://stackoverflow.com/a/21722205/3972095

The guy basically says, you don’t want to install dependencies on the server, only production files and stuff. How can I do this?

4

Don’t worry about composer, it’s central to the power of Bedrock.

There is a great article I read recently that sets out the various options for this stuff, but I can’t find it just now.

The roots guys are planning to switch to ansible based deploys and I think this could involve ci and will give you the solution that I think you are looking for.

Just have Capistrano compile the assets and copy them to production for now.

I find that if you just follow what roots are doing you should be in a good place that keeps on getting better all the time.

5

The solution to this problem is just to compile your assets locally.

Here’s a Gist of a deploy.rb: https://gist.github.com/swalkinshaw/9e4556d4b0c402eba29a

Note: haven’t tested this but it’s pretty straightforward.

It’s just the default Bedrock deploy.rb with an extra task to compile and copy assets.

6

Thanks for the reply. I tried integrating the stuff from your Gist, but it’s giving me a pretty strange error:

INFO[7c53ef33] Running /usr/bin/env composer install --no-dev --prefer-dist --no-interaction --quiet --optimize-autoloader on theproject.com

INFO[7c53ef33] Finished in 4.270 seconds with exit status 0 (successful).

cap aborted!

SSHKit::Command::Failed: if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi exit status: 256

if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi stdout: Nothing written

if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi stderr: ! was unexpected at this time.

config/deploy.rb:71:in `block (3 levels) in <top (required)>'

config/deploy.rb:70:in `block (2 levels) in <top (required)>'

Tasks: TOP => deploy:updated => assets:deploy => assets:compile

(See full trace by running task with --trace)

The deploy has failed with an error: #<SSHKit::Command::Failed: if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi exit status: 256

if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi stdout: Nothing written

if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; then echo "Directory does not exist 'D:/Projects/theproject.com/web/app/themes/thetheme'" 1>&2; false; fi stderr: ! was unexpected at this time.

>

PS D:\Projects\theproject.com>

I added an extra space between each line to (hopefully) make it easier to read. It’s a pretty odd error, because I don’t see anything about an “if test” in the deploy.rb file, so I can only assume it’s including something that is making that test? Hell, I really don’t even know lol…

It looks like it’s echoing the source code or something.

7

When Capistrano sees a within fetch(:local_theme_path) line, it first tests if the directory exists.

So it’s running if test ! -d D:/Projects/theproject.com/web/app/themes/thetheme; (reads as “if NOT a directory”) and then it’s failing with "Directory does not exist ".

Obviously if that directory doesn’t exist it can’t go on so it just fails. So make sure those paths are correct but it could be a Windows problem.

Only thing I can suggest to try is this:

set :theme_path, Pathname.new(File.join('web', 'app', 'themes', 'thetheme'))
set :local_app_path, Pathname.new(File.join('D:', 'Projects', 'theproject.com'))
set :local_theme_path, fetch(:local_app_path).join(fetch(:theme_path))

edit:

Or try backslashes:

set :theme_path, Pathname.new('web\app\themes\thetheme')
set :local_app_path, Pathname.new('D:\Projects\theproject.com')
set :local_theme_path, fetch(:local_app_path).join(fetch(:theme_path))
8

For anyone falling on this thread, @swalkinshaw has provided an example he hasn’t yet tested.

Needs a correction invoke 'deploy:compile_assets is not a native task.

Theres this thread (Bedrock & Roots: Copying Production Assets with Capistrano - #2 by bigsweater) where the discussion is around using a manifest within the deploy.rb / see https://gist.github.com/nateroling/22b51c0cfbe210b00698

Just remove invoke 'deploy:compile_assets on this line https://gist.github.com/swalkinshaw/9e4556d4b0c402eba29a#file-deploy-rb-L63 and it will run.

To be safe and since I don’t have roles setup, I changed https://gist.github.com/nateroling/22b51c0cfbe210b00698#file-deploy-addendum-rb-L50 to on roles(:app) do

Also please note that bash shortcuts don’t work as per @bigsweater suggested on that thread, so the paths to local files need to be absolute.

“Figured it out. Using the ~ homedir shortcut doesn’t work in Capistrano, at least for the upload! method, because SCP don’t play that. ~ is a Bash shortcut, not SCP, so SCP wasn’t finding the directory, naturally.”

1 Like
9

Thanks! That was left behind from another old Gist for Grunt that I based it off of. I updated my gist.

10

To close this out @swalkinshaw and @brett , adding some modularity to the deployment method so that anyone that is interested won’t have to hardcode local paths to their deploy script.

Find the gist below. It’s been tested and verified with Cap 3.2.0

Done.

You can verify that the command will run during deploy by running bundle exec cap staging assets:copy --trace

This task will run automatically when you run cap staging deploy

Here is the Gist

1 Like
11

@rodrigo this gist seems to be so close to working on my first attempt to get sage deployed on a bedrock wp install via capistrano to digitalocean, but… for some reason, even though I can see the compile and upload taking place successfully while the deploy runs, the dist folder simply never appears on the server within the theme.

Any idea why that might be happening?? It’s really perplexing me.

Here’s the output gleaned from the commit (server/site names changed): http://tny.cz/4d137413

All of the paths look bang-on, but the dist folder never makes it into the theme… and can’t be found anywhere else either.

12

You can see what’s happening right in the log:

INFOYour local distribution path: web/app/themes/faketheme2015/dist
INFOBoom!!! Your remote distribution path: /var/www/mysitename/current/web/app/themes/faketheme2015/dist

And later on:

INFO[fc7b431c] Running /usr/bin/env rm -rf /var/www/mysitename/current on fakeserver.com
DEBUG[fc7b431c] Command: ( WP_ENV=staging /usr/bin/env rm -rf /var/www/mysitename/current )
INFO[fc7b431c] Finished in 0.084 seconds with exit status 0 (successful).
INFO[e59bb167] Running /usr/bin/env ln -s /var/www/mysitename/releases/20150424133950 /var/www/mysitename/current on fakeserver.com
DEBUG[e59bb167] Command: ( WP_ENV=staging /usr/bin/env ln -s /var/www/mysitename/releases/20150424133950 /var/www/mysitename/current )
  1. Assets uploaded to /var/www/mysitename/current/web/app/themes/faketheme2015/dist
  2. /var/www/mysitename/current deleted
  3. /var/www/mysitename/current re-linked to newest release

Step #1 is what’s wrong because it should be uploading them to the release path and not hte current symlink.

Try changing:

set :remote_theme_path, release_path.join(fetch(:theme_path))

to

set :remote_theme_path, -> { release_path.join(fetch(:theme_path)) }
13

Ah, but seeing and interpreting… big difference!

Anyway — I tried implementing that modification, and no joy — problem persisted.

Also, I have a new problem: starting from a perfectly clean server state with no deploys (ie. all deploys removed from server deploy destination), I’m getting the following error as soon as gulp is invoked during the deploy:

INFO[53571b0f] Finished in 13.256 seconds with exit status 0 (successful).
INFOYour local distribution path: web/app/themes/faketheme2015/dist
INFOBoom!!! Your remote distribution path: /var/www/fakesite/current/web/app/themes/faketheme2015/dist
INFOUploading files to remote
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host fakeserver.com: scp: /var/www/fakesite/current/web/app/themes/faketheme2015/dist: No such file or directory

It’s not happy about there not being a preexisting /dist/ folder in the theme path

This would be easier if I knew anything at all about ruby syntax :frowning:

14

Welp — after many, many hours of bashing my face into the keyboard, I figured out the problem: the asset build/deploy seems to have been happening on the wrong deployment hook.

  • I could find no way to modify the gist from @rodrigo to get the upload happening to the specific release folder and not the current symlink. I supposed it’s probably possible and I’m simply too much of a ruby/capistrano newb to suss out the correct way to do that, but the deploy script kept trying to upload to /dist/ within the /current/ path, and either failling if dist didn’t already exist, or dumping the files into the previous release if it did.
  • the hook running the gulp + upload routine was running before the new deploy was created (ie. before the new release was symlinked properly), so hooking into the deploy:published hook instead allowed the current path to work properly, and resolved the problem…
# Remote Paths
# set :remote_theme_path, release_path.join(fetch(:theme_path))
set :remote_theme_path, -> { release_path.join(fetch(:theme_path)) }
set :remote_dist_path, fetch(:remote_theme_path).join('dist')

namespace :assets do
  task :compile do
    run_locally do
      within fetch(:local_theme_path) do
        # execute :gulp, '--production'
        execute :gulp, ''
      end
    end
  end

  task :copy do
    on roles(:web) do
      ld = fetch(:local_dist_path)
      rd = fetch(:remote_dist_path)
      info " Your local distribution path: #{ld} "
      info " Boom!!! Your remote distribution path: #{rd} "
      info " Uploading files to remote "
      # execute "rm -r #{fetch(:remote_theme_path).to_s}/dist"
      # execute "mkdir  #{rd} "

      upload! fetch(:local_dist_path).to_s, fetch(:remote_dist_path), recursive: true

    end
  end

  task deploy: %w(compile copy)
end

after 'deploy:published', 'assets:deploy'

Please advise if that’s not a technically sound way of running the task, but… it is working. Hopefully that’s useful to someone.

15

@timichango

I totally dropped the ball on updating you guys on this. Its still WIP

The simple fix for making it “work” is on the last line. I did resolve it locally on my implementation but forgot to make the changes to the Gist, exactly in the same way.

Basically, it waits till the remote is published (symlink activated) then uploads the file.

after 'deploy:published', 'assets:deploy'

NOTE: There is one problem with this though (for production). Your assets will not be present from the time the symlink is published to the moment your assets are uploaded so I am only running it for staging and won’t be coming up with a solution until we move to production.

We need to use the latest_release variable = release path or the current release depending on if the current symlink is valid

And hook to happen before publishing (after deploy:updated),

deploy:updated
    [before]
      deploy:bundle
    [after]
      deploy:migrate
      deploy:compile_assets
      deploy:normalize_assets

See
http://capistranorb.com/documentation/getting-started/flow/

Anyways, I updated my Gist to reflect the non production fix/change even though its WIP

2 Likes
16

Thanks @rodrigo for sharing your deployment configuration. I’ve spent some time working with it to get the uploading to take place before the new site goes live with the symlink update.

The key was to lazy-load the release_path variable only after the deploy task has initiated.

Doing it this way ensures that all of the files are uploaded and ready to serve when the new deploy goes live.

3 Likes
17

@jaywilliams I just tested your latest gist and it worked for me! The only weird thing is that it also added a dist folder in my dist folder and also copied the files in there. So now I have all my files in /dist but also in /dist/dist.

Has this issue occured with you as well?

18

No, I’m afraid not. I just ran a deploy to test it, and everything was created and uploaded properly.

I’m curious though, were you seeing the additional /dist/ folder on the server or your local computer? And if so, are you sure your paths are configured correctly?

As a side note, on my personal Capistrano config, I added a callback to run grunt again without the --production flag, to recompile the dev assets on my local machine.

19

Hey all, thanks for the assistance on this. I’m using the deploy.rb that @jaywilliams uploaded (https://gist.github.com/jaywilliams/c0abbec89ef6bc81cb49) and, while the assets:copy task seems to be working, the assets:compile task doesn’t want to. It looks like it’s having an issue with the local theme directory? Here are the errors I get:
https://gist.github.com/BrettLefty/5823f6707509ebcefbb7

You’ll notice my local directory is absolute and not relative. I first assumed this was the issue, so I made a small change to the set :local_app_path line, and changed it from:
set :local_app_path, Pathname.new(File.dirname(__FILE__)).join('../')
…to…
set :local_app_path, Pathname.new(Dir.pwd)

Here’s my full deploy.rb file:
https://gist.github.com/BrettLefty/a0025d7d2bfcec102595

I’m running these commands in PowerShell (I know right?), but I have tried them in Git Bash.

20

Hello,
Just a suggestion,
Make sure the theme folder exists before using:

upload! fetch(:local_dist_path).to_s, fetch(:remote_dist_path), recursive: true

Before that line, add something like:

remote_theme_path = fetch(:remote_theme_path)
execute 'mkdir -p #{remote_theme_path}'

And add :remote_theme_path as Global (after :theme_path):

set :remote_theme_path, Pathname.new(fetch(:deploy_to) + '/current/web/app/themes').join(fetch(:theme_name))

Or something like that…

:wink:

1 Like