Trellis-Backup, an ansible role for local backups

trellis
12

Hello, @guilro!

It looks like the issue is with python-openssl.

When provisioning fails (gist above) and I vagrant ssh into my development server, running pip --version produces the following:

$ pip --version
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 13, in <module>
    from pip.exceptions import InstallationError, CommandError, PipError
  File "/usr/lib/python2.7/dist-packages/pip/exceptions.py", line 6, in <module>
    from pip._vendor.six import iteritems
  File "/usr/lib/python2.7/dist-packages/pip/_vendor/__init__.py", line 64, in <module>
    vendored("cachecontrol")
  File "/usr/lib/python2.7/dist-packages/pip/_vendor/__init__.py", line 36, in vendored
    __import__(modulename, globals(), locals(), level=0)
  File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/__init__.py", line 9, in <module>
  File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/wrapper.py", line 1, in <module>
  File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py", line 3, in <module>
  File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/__init__.py", line 53, in <module>
  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/contrib/pyopenssl.py", line 54, in <module>
  File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 118, in <module>
    SSL_ST_INIT = _lib.SSL_ST_INIT
AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

Reinstalling python-openssl fixes the problem:

sudo apt-get remove --purge python-openssl
sudo apt-get install python-openssl

This gives expected output:

$ pip --version
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)

After this, reprovisioning works.

Now to get Trellis to install a working python-openssl out of the box :slightly_smiling_face:

1 Like
13

…Huh, I experienced the same issue with python-openssl, too, except when I wiped the py-openssl directory, pip croaked and died to the point where I’m going to blow the machine away and start again.

I really want it working on a dev server. Being able to build locally and then just import remotely is a pretty big win for me.

1 Like
14

I’m having success pre-installing python-openssl by adding it to trellis/roles/common/defaults/main.yml

apt_packages_custom:
    - python-openssl # For trellis-backup-role to work

Hope this helps you.

I’m currently having a challenge setting up the usernames and passwords for the backups. The README.md is unclear on how to define those.

1 Like
15

That does help – thanks!!

As to your issue, I left a comment in the Github thread but I’ll reprise it here:

Yeah, it’s not super well documented yet, but you need to set up some backup information within wordpress_sites.yml or vault.yml. The above error is basically just saying you don’t have a backup_target_user key.

What you need is something like this in your vault:

vault_wordpress_sites:`
  <site domain>:
    env:
      db_password: #...
      # ...
     
      backup_target_user: ""
      backup_target_pass: ""

If you’re using S3, you need to specify your access key as the user and secret as the pass.

I’ll try to do a PR with these documentation details soon.

1 Like
16

Thank you, @robyurkowski!

Your recommendations work for me in development. In production, I’d like to use S3. If you have configuration notes on how to set that up, I’d appreciate those.

Thanks in advance.

17

Happy to share. It’s pretty simple unless you’re using a non-US bucket. In the case that your bucket only has V4 Signatures, you’ll have to use the code in my fork until the pull request is accepted.

If you do need the V4, the code you’d want looks something like this (and please forgive me because I’m doing this from memory — I may mis-key something.)

# group_vars/production/wordpress_sites.yml

wordpress_sites:
  <site domain>:
    backup:
      # ...
      target: "s3://<region-identifier>.amazonaws.com/<bucket>/<subfolder>"
      params:
        - 'export S3_USE_SIGV4="True"'

If you have a bucket that accepts V2 signatures, you don’t need the params field.

# group_vars/production/vault.yml

vault_wordpress_sites:
  <site domain>:
    env:
      # ...
      backup_target_user: "<AWS ACCESS KEY>"
      backup_target_pass: "<AWS SECRET KEY>"

There are a couple extra things I noted:

  1. Make sure your bucket doesn’t have a period in it. Duplicity doesn’t like that.
  2. I had issues when I used s3.ca-central-1 as my region identifier, but not s3-ca-central-1. However, I was also debugging the S3_USE_SIGV4 issue at that time, so that might not be important.
  3. Make sure you have a bucket policy that allows basically everything inside of the bucket.

As a side note, I’m still having issues with the backups. I checked the logs last night (/var/log/duply/) and it turns out that it’s not connecting to MySQL and the purge operation isn’t working. So more debugging needed. (EDIT: Looks like there’s a PR to fix that too.)

PS: @guilro, if you’ve got some time, we could use some PRs reviewed and merged! Otherwise I’m sure one of us would be happy to hop on as a maintainer!

3 Likes
18

Hi everyone, thanks for your PRs, most of them have been merged.

I normally have time to maintain the role (we use it in production at my company), I was just spending a week off with my family.

I will check the python-openssl problem, it should be sufficient to add the package in the role.

1 Like
19

It should have been fixed in version 1.1.0, at least on the Ubuntu image my provider is working with.

If you still meet problems, can you tell me the providers you are using so I can reproduce ?

1 Like
20

Thank you, @guilro and @robyurkowski. The latest updates work as advertised for me from the docs in the new README.md to the S3 backups. Thanks!

21

Thanks, @guilro! Hope you had a great week off. :slight_smile:

I’ll verify that everything is working tonight.

22

I would like to see the ability to redirect this backup to the same Git repo that the site is located at, but in a separate branch.

23

Hello, @guilro!

I have been having challenges with trellis-backup-role since updating to the Ansible 2.4 compatible version of Trellis.

Help? :slight_smile:

24

Hello @silumesii and everyone.

I am working on a new major version with a fork of Stouts.backup, it should be released today or tommorow :slight_smile:

1 Like
25

Hello everyone.

I released version 2.0.2 yesterday.

  • it works from scratch without creating a new mysql user, using the auth_socket plugin
  • paramiko depdency is removed (there is no support for scp by default)
4 Likes
26

Hi @guilro, your solution seems really a good one, thanks for the work (I think should thanks @MWDelaney too).

I have a question about where I’m supposed to insert roles in the server.yml file.

Here it says:

Run ansible-galaxy install -r requirements.yml to install the new roles.

Then, add the roles to the server.yml:

roles:
… other Trellis roles …

  • { role: trellis-backup, tags: [backup] }

Is the … other Trellis roles … are these in this part of the server.yml:

- name: WordPress Server - Install LEMP Stack with PHP 7.2 and MariaDB MySQL
  hosts: web:&{{ env }}
  become: yes
  roles:
     ... all list of roles ...
     - { role: wordpress-setup, tags: [wordpress, wordpress-setup, letsencrypt] }

Should I put the new role for backup at the end after worpress-setup (if the file wasn’t modify at this part of course)?

Thank you in advance

P.S. (surpris de voir le nom franceinsoumise comme dépendance… j’ai dû vérifier que c’était bien une dépendance et non un exemple de mise en situation de code)

27

Exactly ! Using the backup tag is optional :slight_smile:

La France insoumise est mon employeur.

28

Ok Thanks

Dommage pour les élections, c’était un bon programme, mes amis français ici au Québec avaient fondés beaucoup d’espoir en ce parti et le mouvement. Gardez espoir.

29

Oh I forgot to ask, can we configure the purge to keep a number of backup, or it’s just true or false, I didn’t found information in Duply doc.

30

What Trellis-backup does is that it sets backup_mysql_user, backup_mysql_pass, and backup_profiles to correct values for trellis, and then runs https://github.com/lafranceinsoumise/ansible-backup which does all the job.

If you want to use other options of lafranceinsoumise.backup, you can set backup_max_age, backup_max_full_backups, backup_full_max_age in the vars of your playbook, they will be applied. Just check https://github.com/lafranceinsoumise/ansible-backup to see what settings fits your need.

31

Thanks a lot for your answer.

Ok, I checked ansible-backup doc, but something I’m not sure is: should I put vagrant as user or create a specific one?

Because I put a new user name and there is nothing in the backup folder where it supposed to drop the backup (local in file:///srv/www/website.com/current/web/app/trellis-backup).

Is there a way to see if it’s work correctly, I didn’t found any Duply folder or log in the VM?

Thanks again