Hi. One of our regular projects started throwing an error when onboarding a new dev. Apparently when running trellis init, it now installs python 3.9, instead of the previously supported 3.8, and that breaks deploys.
Trying to figure out where we can specify a min/max python version for the corresponding versions of trellis/ansible that we’re running.
trellis init doesn’t actually install Python, it just creates a virtualenv using whatever version of Python the local system has under the python3 command.
So I think the best solution is to use a tool like pyenv which will let you control the python version by committing a .python-version file in your project.
Then I assume trellis init would respect pyenv’s activated python version within that project correctly.
Hi Scott, thank you for replying. I assumed trellis init installs python because when running ansible --version from within the virtualenv, it reports python 3.9 and the path is inside the .trellis subfolder, whereas, if I delete the .trellis generated by trellis init and re-run ansible --version, the system Python version is 3.8, and then I can proceed with deploys.
So it sounds like a dependency within the project (bedrock?) makes trellis-cli install python 3.9, then?
It sounds like that person’s computer might have two version of Python installed already?
They could confirm by running these outside of a Trellis project:
python3 --versionpython --versionI’d also run which python3 and which python to see where they are coming from. My guess is that ansible is defaulting to python outside of the virtualenv.
One other note: I’m surprised deploys would fail between 3.8 and 3.9
You might want to post another thread or issue if you think there’s a bug in Trellis between those versions.
I ran the checks for the various versions (having run into the issue again), so I have two versions of python3 installed at the system level, and python version 3.9.10 installed by trellis-cli in the virtualenv.
As the which command points out, it seems to be running the python in the virtualenv.
I do suspect there’s a bug in trellis in-between versions, but what I’m uncertain on is how to change the python version when setting up the virtualenv.
❯ python3 --version
Python 3.9.10
❯ cd …
[trellis] deactivated env
❯ python3 --version
Python 3.9.10
❯ python --version
Python 3.10.4
❯ cd example.com
[trellis] activated env
❯ python3 --version
Python 3.9.10
❯ python --version
Python 3.9.10
❯ which python
/home/username/work/example.com/trellis/.trellis/virtualenv/bin/python
❯ which python3
/home/username/work/example.com/trellis/.trellis/virtualenv/bin/python3
❯ cd …
[trellis] deactivated env
❯ which python
/usr/bin/python
❯ which python3
/home/linuxbrew/.linuxbrew/bin/python3
This is a bit weird and not really ideal since you have two separate versions of Python 3 installed. One looks like the system version (from Ubuntu?), and the other is from Homebrew?
As mentioned before, trellis-cli uses python3 to create the virtualenv so that’s where the version comes from. Regardless, you can use pyenv to lock to specific Python versions if your project requires it.
Hi @swalkinshaw. I hadn’t understood what you meant until I realized that trellis-cli uses the system-installed python(3) version to create the virtualenv. I downgraded my homebrew-installed python3 3.10 to 3.9, and now trellis init reports python 3.9.13, which is a good step. However, trellis init still fails with [✘] Error installing dependencies
Do you know of any log I can check that can help me track down the source of the issue? This started occurring after I updated my main system from Ubuntu 20.04 to 22.04, but the issue is not happening on my laptop, which I also updated from 20.04 to 22.04.
You can manually run the underlying command to get more information:
trellis exec pip install -r requirements.txt
That results in Error running pip: permission denied. Problem with my system install of python?
Dove a little deeper, tried running just pip install -r requirements.txt which executes successfully.
Then tried trellis init, it failed again. Tried to deploy, got:
ERROR! Unexpected Exception, this is probably a bug: cannot import name 'AnsibleCollectionLoader' from 'ansible.utils.collection_loader' (/home/username/.local/lib/python3.10/site-packages/ansible/utils/collection_loader/__init__.py)
So somewhere in the mix, the unsupported python version is still acting up.
I checked python binaries from within the project:
which python
/home/blackthorne/work/thethirdwave.co/trellis/.trellis/virtualenv/bin/python
❯ which python3
/home/blackthorne/work/thethirdwave.co/trellis/.trellis/virtualenv/bin/python3
❯ python --version
Python 3.9.13
❯ python3 --version
Python 3.9.13
All good there. Then checked from outside the project folder:
/usr/bin/python
❯ which python3
/home/linuxbrew/.linuxbrew/bin/python3
❯ python --version
Python 3.10.4
❯ python3 --version
Python 3.9.13
Well, crap. the regular python command is installed as a dependency of my system packages, I worry that downgrading it will break my system, and, I don’t understand why it’s being utilized in the first place. Shouldn’t the virtualenv install of Python be used in this case? Is this a trellis-cli bug?
UPDATE:
Tried running ansible-playbook deploy.yml -vvvv -e "site=example.com env=staging", I still get the same ansible error referencing python 3.10. Downgrading this is likely to break my other software dependencies, so I feel a biiiiit stuck.
UPDATE 2:
Tried copying over the .trellis folder with the correct dependencies installed from my laptop computer. Upon running a trellis deploy command, my error message is now:
fork/exec /home/username/work/example.com/trellis/.trellis/virtualenv/bin/ansible-playbook: permission denied, which is reminescent of the trellis-cli error with the trellis exec pip install -r requirements.txt command suggested above.
UPDATE 3: I also installed pyenv and used it to install python 3.9.13, and set it locally using pyenv local 3.9.13 in the trellis folder. I checked:
3.9.13 (set by /home/username/work/example.com/trellis/.python-version)
No change in terms of behavior.
UPDATE 4: I got a bit more creative, navigated to /home/linunxbrew/.linuxbrew/bin , ran pyenv local 3.9.13 and successfully created a new trellis project using trellis new in that folder.
Trying to execute something similar elsewhere, including by going to my projects folder(s) and running pyenv exec trellis init, for example, still fails.
@swalkinshaw I still can’t tell if this is a bug in trellis-cli or not. It doesn’t help that running ansible-playbook directly, in the project folder, with pyenv local 3.9.13 set, still uses the system-installed Python 3.10, which points to the answer being not, but without any insight into why trellis obeys the pyenv restriction from within the folder its binary is located in, but not outside of that folder.
Oh, before anyone takes my idiocy seriously, my laptop isn’t at 22.04, it’s at 21.10, which is why it can still deploy/provision/init.
Off-topic, we really should change the title of this thread, as I had misunderstood what was happening. It really should say “System-wide Python 3.10+ install breaks Trellis/Ansible compatibility” or something.
I too am currently having this same issue.
Existing projects, Manjaro, Python 3.10
Will be giving pyenv a try to see if I can workaround this.
Sorry I missed your detailed reply earlier @sangemaru
I’ll have to dig in a bit more when I’m home (travelling now), but here’s a few general notes:
python3 command on your local computer. It makes no assumption about where/what that Python install is; it could be a system one or one from pyenv.python3 binary that it used to create the virtualenv (from the point above)trellis init --force will re-create the venv as if from scratchwhich python3), make sure you’re outside a project dir or disable that feature temporarilyGeneral reminder: with trellis-cli, there’s 3 ways to run ansible commands:
trellis command like trellis provision
trellis exec
ansible-playbook directlyThe first two are the recommended ones because trellis ensures that the virtualenv is properly loaded. The 3rd option can work but only if you have trellis-cli’s shell integration properly working.
@sangemaru (and maybe @codepuncher ?) I think you might be misinterpreting that original Ansible error about AnsibleCollectionLoader. That error has come up before in Python 3.8 and 3.9. Usually it means there’s some mismatch between the Ansible version and the Python version.
In this case, it might mean that Python 3.10 (or even 3.9) is only supported by a newer version of Ansible than you’re using.