Roots Discourse

Backing Up Trellis Sites to an S3 Bucket


#1

Originally published at: https://roots.io/guides/backing-up-trellis-sites-to-an-s3-bucket/

Add backup shell script Create site/scripts/backup-to-s3.sh with the following contents: #!/bin/bash eval $(cat …/.env | sed ‘s/^/export /’) export AWS_CONFIG_FILE="/home/web/.aws/config" SITE="${DB_USER//_/.}" ENVIRONMENT="$WP_ENV" TIMESTAMP=env TZ=America/Denver date +%Y-%m-%d-%H%M ARCHIVE_PATH=/tmp/$SITE-$ENVIRONMENT-$TIMESTAMP ARCHIVE_FILENAME=$SITE-$ENVIRONMENT-$TIMESTAMP.tar.gz mkdir -p $ARCHIVE_PATH && cd /srv/www/$SITE/current && wp db export $ARCHIVE_PATH/db.sql && rsync -kavzP --exclude web/wp/ --exclude web/wp-config.php /srv/www/$SITE/current/web $ARCHIVE_PATH && rsync -kavzP /srv/www/$SITE/shared/uploads $ARCHIVE_PATH/web/app && tar…


#2

Thanks, @benword. You rock.

Parameters

I believe that the trellis/group_vars/all/vault.yml parameters should be:

aws_access_key_id: xxxxxxx
aws_secret_access_key: "xxxxxxx"

This way the dstil aws-cli template will be able to grab them:

vendor/roles/aws-cli/aws_cli_config.js:

[default]
output = {{ aws_output_format }}
region = {{ aws_region }}
aws_access_key_id = {{ aws_access_key_id }}
aws_secret_access_key = {{ aws_secret_access_key }}

(I’m not sure if region matters, and not sure where the first two parameters would belong, perhaps as group_vars/production/wordpress_sites.yml env variables?)

Shell Script

Name of script referenced in cron job is backup-to-s3.sh, not backup.sh.

Can we note that the s3 bucket referenced in the third line of the script must be modified to match an s3 bucket that has been:

  1. Created manually by the user either via the AWS interface or some other tool like awscli or s3cmd.
  2. Will need to be unique among all s3 buckets as they all share the same namespace.

It would also be nice to remind folks that if their server is already provisioned and they want to save time they can just run the modified tasks:

ansible-playbook server.yml -e env=production --tags "wordpress-setup, aws-cli"

#3

Two more steps:

  1. add one parameter to group_vars/all/users.yml: aws_cli_user: web

Otherwise by default, aws-cli credentials are set for admin-user, while cron script (backup-to-aws.sh) owned and run by web:www-data.

  1. Permissions on backup-to-s3.sh need to be 755 aka -rwxr-xr-x, aka chmod +x backup-to-s3.sh and not 644 or /backup-to-s3.sh won’t work.

The following task, also added to wordpress-setup/tasks/main.yml:

- name: Update 'backup-to-s3.sh' permissions
  file:
    path: "{{ www_root }}/{{ item.key }}/{{ item.value.current_path | default('current') }}/scripts/backup-to-s3.sh"
    owner: "{{ web_user }}"
    group: "{{ web_group }}"
    mode: 0755
  with_dict: "{{ wordpress_sites }}"

#4

Troubleshooting

On the production server:

  • Read the output from the ansible provisioning process
  • Confirm that the credentials from group_vars/all/vault.yml exist in /home/web/.aws/config
  • As user web, run aws s3 ls s3://your-unique-namespace-site-backups
  • As user web, run aws s3 cp some_arbitrary_file s3://your-unique-namespace-site-backups
  • As user web, run bash /srv/www/example.com/current/scripts/backup-to-s3.sh manually
  • Confirm that file /etc/cron.d/backup-nightly-example_com exists and contains

Contents should be:

0 12 * * * web cd /srv/www/example.com/current/scripts && ./backup-to-s3.sh > /dev/null 2>&1


#5

To debug script, run ./backup-to-s3.sh from its directory:

cd /srv/www/my-site.com/current/scripts && ./backup-to-s3.sh

instead

bash /srv/www/my-site.com/current/scripts/backup-to-s3.sh

Otherwise the line 2 throw an error:

$ bash /srv/www/my-site.com/current/scripts/backup-to-s3.sh 
cat: ../.env: No such file or directory
/srv/www/my-site.com/current/scripts/backup-to-s3.sh: line 10: cd: /srv/www//current: No such file or directory

Also, I suggest to add this in the document guide:

Change vendor/roles/aws-cli/defaults/main.yml

aws_access_key_id: 'YOUR_ACCESS_KEY_ID'
aws_secret_access_key: 'YOUR_SECRET_ACCESS_KEY'

to

aws_access_key_id: '{{ vault_aws_access_key_id }}'
aws_secret_access_key: '{{ vault_aws_secret_access_key }}'

Backup to s3 not working