Issue deploying older project

trellis
1

Trying to deploy an older project which I’ve updated to a more recent version of Trellis to be able to use PHP 8.1.

It seems like there’s some SSH issue, I haven’t seen this issue before. I can ssh into the machine just fine from my local machine.

TASK [deploy : Check if deploy_initialize_before scripts exist] ****************
task path: /Users/user/WWW/domain.tld/trellis/roles/deploy/tasks/initialize.yml:2

TASK [deploy : include_tasks] **************************************************
task path: /Users/user/WWW/domain.tld/trellis/roles/deploy/tasks/initialize.yml:9

TASK [deploy : Initialize] *****************************************************
task path: /Users/user/WWW/domain.tld/trellis/roles/deploy/tasks/initialize.yml:16
redirecting (type: modules) ansible.builtin.deploy_helper to community.general.deploy_helper
Using module file /Users/user/WWW/domain.tld/trellis/.trellis/virtualenv/lib/python3.9/site-packages/ansible_collections/community/general/plugins/modules/deploy_helper.py
Pipelining is enabled.
<x.x.x.x> ESTABLISH SSH CONNECTION FOR USER: username
<x.x.x.x> SSH: EXEC ssh -vvv -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o HostKeyAlgorithms=ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,ssh-rsa -o StrictHostKeyChecking=no -o Port=22 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="username"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/Users/user/.ansible/cp/a2d951abbc x.x.x.x '/bin/sh -c '"'"'/usr/bin/python3 && sleep 0'"'"''
<x.x.x.x> (1, b'', b'OpenSSH_9.0p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/user/.ssh/config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/a.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/a.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/b.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/b.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/domain.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/domain.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/c.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/c.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/testinglima.com/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/testinglima.com/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/d.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/d.tld/ssh.config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname x.x.x.x is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/Users/user/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/Users/user/.ssh/known_hosts2\'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 47758\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\nTraceback (most recent call last):\n  File "<stdin>", line 102, in <module>\n  File "<stdin>", line 94, in _ansiballz_main\n  File "<stdin>", line 40, in invoke_module\n  File "/usr/lib/python3.8/runpy.py", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code\n    exec(code, run_globals)\n  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 524, in <module>\n  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 485, in main\n  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 347, in create_path\n  File "/usr/lib/python3.8/os.py", line 213, in makedirs\n    makedirs(head, exist_ok=exist_ok)\n  File "/usr/lib/python3.8/os.py", line 223, in makedirs\n    mkdir(name, mode)\nPermissionError: [Errno 13] Permission denied: \'/sites/domain.tld\'\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\n')
<x.x.x.x> Failed to connect to the host via ssh: OpenSSH_9.0p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/user/.ssh/config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/a.tld/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/a.tld/ssh.config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/b.tld/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/b.tld/ssh.config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/domain.tld/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/domain.tld/ssh.config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/c.tld/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/c.tld/ssh.config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/testinglima.com/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/testinglima.com/ssh.config
debug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/d.tld/ssh.config depth 0
debug1: Reading configuration data /Users/user/.lima/d.tld/ssh.config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug2: resolve_canonicalize: hostname x.x.x.x is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/user/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 47758
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
Traceback (most recent call last):
  File "<stdin>", line 102, in <module>
  File "<stdin>", line 94, in _ansiballz_main
  File "<stdin>", line 40, in invoke_module
  File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 524, in <module>
  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 485, in main
  File "/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py", line 347, in create_path
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/sites/domain.tld'
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
fatal: [x.x.x.x]: FAILED! => {
    "changed": false,
    "module_stderr": "OpenSSH_9.0p1, LibreSSL 3.3.6\r\ndebug1: Reading configuration data /Users/user/.ssh/config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/a.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/a.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/b.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/b.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/domain.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/domain.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/c.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/c.tld/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/testinglima.com/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/testinglima.com/ssh.config\r\ndebug3: /Users/user/.ssh/config line 1: Including file /Users/user/.lima/d.tld/ssh.config depth 0\r\ndebug1: Reading configuration data /Users/user/.lima/d.tld/ssh.config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files\r\ndebug1: /etc/ssh/ssh_config line 54: Applying options for *\r\ndebug2: resolve_canonicalize: hostname x.x.x.x is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/user/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/user/.ssh/known_hosts2'\r\ndebug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 47758\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\nTraceback (most recent call last):\n  File \"<stdin>\", line 102, in <module>\n  File \"<stdin>\", line 94, in _ansiballz_main\n  File \"<stdin>\", line 40, in invoke_module\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 524, in <module>\n  File \"/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 485, in main\n  File \"/tmp/ansible_deploy_helper_payload_j0gzp3p0/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 347, in create_path\n  File \"/usr/lib/python3.8/os.py\", line 213, in makedirs\n    makedirs(head, exist_ok=exist_ok)\n  File \"/usr/lib/python3.8/os.py\", line 223, in makedirs\n    mkdir(name, mode)\nPermissionError: [Errno 13] Permission denied: '/sites/domain.tld'\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

PLAY RECAP *********************************************************************
x.x.x.x             : ok=2    changed=0    unreachable=0    failed=1    skipped=11   rescued=0    ignored=0
localhost                  : ok=0    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
2

I noticed that the server is on Python 3.8.10 while my local is 3.12.3 . Could that be an issue here?

3

It looks like you’re trying to bring up your development environment here. Take a look at this thread and see if it helps:

4

Hey! Thanks for the reply!

This I’ve managed to get the VM up and running, it’s the deploy that fails. I pulled Trellis 1.21.0 into my project before I ran trellis vm start so IP etc should already be fixed :thinking:

5

I commented out these lines in roles/deploy/tasks/initialize.yml:

# - name: Initialize
#   deploy_helper:
#     current_path: "{{ project_current_path }}"
#     path: "{{ project_root }}"
#     state: present

Which gets me a bit further, but ultimately stops at:

TASK [deploy : Clone project files] ********************************************
fatal: [207.154.236.94]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
...ignoring

TASK [deploy : Failed connection to remote repo] *******************************
fatal: [207.154.236.94]: FAILED! => {"changed": false, "msg": "Git repo git@github.com:user/domain.tld.git on branch develop cannot be accessed. Please verify the repository/branch are correct and you have SSH forwarding set up correctly.\nMore info:\n> https://roots.io/trellis/docs/deploys/#ssh-keys\n> https://roots.io/trellis/docs/ssh-keys/#cloning-remote-repo-using-ssh-agent-forwarding\n\nError:\nfatal: could not create leading directories of '/sites/xxx.tld/files/shared/source': Permission denied\n"}

Edit: I can clone the repo just fine with the web user from within the server. And create folders etc.

6

Tried to setup a fresh project (trellis new), adding all necessary details and then deploying, this notice before getting the same error:

virtualenv ❯ trellis deploy staging
Running command => ansible-playbook deploy.yml -e env=staging -e site=domain.tld
[WARNING]: Skipping plugin
(/Users/user/WWW/domain.tld/trellis/lib/trellis/plugins/vars/version.py),
cannot load: No module named 'distutils'
/Users/user/WWW/domain.tld/trellis/lib/trellis/plugins/callback/vars.py:50: SyntaxWarning: invalid escape sequence '\.'
  patterns = [re.sub(r'\*', '(.)*', re.sub(r'\.', '\.', var)) for var in raw_vars if var.split('.')[0] in hostvars]
/Users/user/WWW/domain.tld/trellis/lib/trellis/plugins/callback/vars.py:51: SyntaxWarning: invalid escape sequence '\.'
  keys = set(pattern.split('\.')[0] for pattern in patterns)
7

This might indeed be a python environment issue. See this post as Stack Overflow:

8

A real head scratcher this one.

I deleted the project once more, downgraded Python to 3.11.6 which works fine for another project, on the same server. When deploying, the previous warning does not appear, but the very same error stops me once again.

TASK [deploy : Initialize] *****************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: PermissionError: [Errno 13] Permission denied: '/sites/domain.tld'
fatal: [x.x.x.x]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 107, in <module>\n  File \"<stdin>\", line 99, in _ansiballz_main\n  File \"<stdin>\", line 47, in invoke_module\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_deploy_helper_payload_6z9xiuku/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 536, in <module>\n  File \"/tmp/ansible_deploy_helper_payload_6z9xiuku/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 499, in main\n  File \"/tmp/ansible_deploy_helper_payload_6z9xiuku/ansible_deploy_helper_payload.zip/ansible_collections/community/general/plugins/modules/deploy_helper.py\", line 357, in create_path\n  File \"/usr/lib/python3.8/os.py\", line 213, in makedirs\n    makedirs(head, exist_ok=exist_ok)\n  File \"/usr/lib/python3.8/os.py\", line 223, in makedirs\n    mkdir(name, mode)\nPermissionError: [Errno 13] Permission denied: '/sites/domain.tld'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
9

This is a guess, and maybe try this on staging first, but have you reprovisioned the server since creating your new Trellis?

10

The server is not provisioned by Trellis, it’s a SpinupWP managed DO droplet that hosts a bunch of Bedrock sites for staging purposes.

Running out of ideas on what to try. The server is running 20.04. But I’m deploying other, newer Trellis (1.21.0 atleast) projects to this server so it shouldn’t because of this.

11

I think my issues were due to me having some canary release of trellis-cli that introduced some weirdness. I’m back on the v1.11.1 again and everything seems fine!