Provision hangs on "Setup composer authentications" on older project if using Trellis CLI (but works with old process)

gilad · August 25, 2022, 8:05pm

trellis up and trellis provision development hang on the wordpress-install : Setup composer authentications task.

trellis up --debug yields no new information; the process hangs and I see nothing useful in the output that follows ctrl c.

This same project is indeed provisionable (in Dev, thus far), when not using Trellis CLI (vagrant up, vagrant provision, etc).

Background:

Project originally built on Trellis 1.8.0 and Bedrock 1.15.4 without Trellis CLI (using the old manual process)
Trellis CLI enabled later on the original Dev machine (macOS Monterey) and ran as expected (no errors, warnings, etc)
Monterey machine crashed, and I’m now attempting to rebuild on an older Mac running Catalina (trellis init and trellis init --force complete as expected without any error or warning)
Vagrant = 2.2.18
VirtualBox = 6.1.36

Strictly speaking, this is not a critical break (as far as I know at this time), because I can work. Still, I’d love to get this project working with Trellis CLI again.

Any of this ring any bells for anyone?

swalkinshaw · August 26, 2022, 3:03am

Did you override the composer_authentications variable to set any? If not, the task should just skip entirely.

It would be helpful for you to include the output for that task as well.

You have a few options:

Remove the no_log option for more output: trellis/composer-authentications.yml at 4a5332e299ce6ed4803d6b8a49d31832b7e456eb · roots/trellis · GitHub
feel free to remove the entire task if you don’t care about composer authentications

gilad · September 1, 2022, 1:43pm

I’m using composer_authentications to enable pulling premium plugins from our own Composer server (SatisPress).

As mentioned, I didn’t see any useful output during trellis up --debug (with no_log set to false), but I can certainly try again. I suppose I could have missed something in the rush.

swalkinshaw · September 1, 2022, 3:10pm

Oh! In that case can you share your composer_authentications config (with any secret values changed of course).

I’d also try the golden rule of Trellis troubleshooting which is running the command manually. In this case, you want to SSH in (vagrant ssh), go into the site directory and run something like this:

composer config  --auth http-basic.[HOSTNAME] [USERNAME] [PASSWORD]

and replace those values with whatever you’ve configured in composer_authentications

gilad · September 18, 2022, 5:32pm

Apologies for the late response (travel, apocalyptic virus, new machine, other fun…).

The custom composer source in my vault is:

vault_mysql_root_password: password-here

vault_wordpress_sites:
  clientsite.com:
    admin_password: password-here
    env:
      db_password: password-here

    # Custom Composer Source:
    composer_authentications:
      - { hostname: satispress.company.com, username: username-here, password: password-here }

Running the manual composer config command results in no error (no feedback at all, in fact).

Running composer config --list does indeed display the config details for this custom composer source.

After the above steps, I also went back to project root and attempted another trellis provision development run: Still stalling at wordpress-install : Setup composer authentications.

gilad · September 22, 2022, 3:02am

Solved! Finally.

I changed composer_authentications, as in the above vault.yml sample…

To: composer_authentication (plural to singular)!

The project now creates a fresh VM and reprovisions an existing VM, as expected.

swalkinshaw · September 22, 2022, 2:19pm

Something is still wrong here. By changing the variable name to composer_authentication, you’ve you bypassed that feature entirely.

The feature works by using composer_authentications so now no authentications are enabled at all. This is probably fine for now since you did it manually before but if you (or anyone else) ever creates a new VM/server, it won’t exist.

I’m happy to help continue debugging this but it’s quite hard to do remotely. Going back to my suggestion before, it would be helpful to get the actual output related to the composer authentications task when no_log is removed: Provision hangs on "Setup composer authentications" on older project if using Trellis CLI (but works with old process) - #3 by gilad

gilad · September 22, 2022, 2:31pm

I’m actually happy to have this particular feedback because I’m encountering “new” issues this morning that are probably not new at all (but rather caused by my change, given your explanation).

Regarding the output you’re looking for, is my report in Provision hangs on "Setup composer authentications" on older project if using Trellis CLI (but works with old process) - #5 by gilad not what you were looking for (no error or output from the manual command)?

swalkinshaw · September 22, 2022, 5:58pm

restore the proper variable name composer_authentications
remove no_log from trellis/composer-authentications.yml at 4a5332e299ce6ed4803d6b8a49d31832b7e456eb · roots/trellis · GitHub
run trellis provision --tags=wordpress-install development (or vagrant provision)
Paste entire output here

gilad · September 22, 2022, 9:08pm

Oops, apologies, I misunderstood Step 3.

Still, as mentioned earlier, I did previously provision with no_log (albeit in a complete provision), so here’s what I get back from trellis provision --tags=wordpress-install development (still no output, and no_log is commented out both in trellis/roles/wordpress-install/tasks/composer-authentications.yml and in trellis/roles/wordpress-install/tasks/main.yml):

Here are the last few lines of output while trellis uping (with --debug) after vagrant destroy:

composer-authentications-break-c

In both cases, the process stopped just as appears in the image(s). I killed the first one after 40 minutes. I didn’t wait quite as long the second time.

Here’s what I have for repositories in composer.json:

  "repositories": [
    {
      "type": "composer",
      "url": "https://wpackagist.org",
      "only": ["wpackagist-plugin/*", "wpackagist-theme/*"]
    },
    {
      "type": "composer",
      "url": "https://satispress.company.com/satispress/"
    },
    {
      "type": "vcs",
      "url": "git@github.com:retlehs/kinsta-mu-plugins"
    }
  ],

Here’s what I have in my auth.json (password and username copied carefully anew):

{
    "http-basic": {
        "satispress.company.com": {
            "username": "username-here",
            "password": "password-here"
        }
    }
}

And, I should mention again that this same project did in fact work correctly on another machine until that machine crashed. The crashed machine is back from the shop now, so I’ll be attempting this project there again shortly.

gilad · September 23, 2022, 3:20pm

Update: The hung process apparently terminated (with feedback) sometime overnight:

I’m still looking into possibilities, but if this looks familiar to anyone, please do share.

The SatisPress server to which this composer_authentications config is attempting to connect does indeed work well with other projects, including this very project: I’m getting this error when attempting to provision development, but I can deploy to one of the project’s previously provisioned remotes (and I have verified that composer dependencies are indeed installed on the remote every time).

swalkinshaw · September 24, 2022, 8:11pm

Ah sorry, I didn’t realize before when you said it “hangs” that there’s literally no helpful output at all So no_log isn’t actually hiding any output in this.

Honestly I don’t really know what to suggest next. The process hanging implies two things to me:

a network connection that’s being blocked
the process is waiting for input

Considering it hangs for 40 minutes, I’d be very skeptical it’s a network connection. Almost any client/server would have lower timeouts.

So I wonder if the composer command that Ansible ends up running is someone causing composer to wait for input.

There’s been some changes/improvements made this composer authentications feature. Maybe you can apply the patch/commit and see if that helps? Improve composer authentications password default by swalkinshaw · Pull Request #1412 · roots/trellis · GitHub

gilad · September 25, 2022, 12:24am

Thank you, yes, of course, I’ll apply the patch you linked and report back.

Also, I see a fair amount of chatter online about key_exchange_identification errors, and in a fairly wide variety of circumstances. I’ll continue researching that angle for a while, too, given what I shared at Provision hangs on "Setup composer authentications" on older project if using Trellis CLI (but works with old process) - #11 by gilad.

Several other projects I have are now giving me a hard time (starting last week or so) with errors at [wordpress-install : Setup composer authentications - example.com] that output Do not run Composer as root/super user! See https://getcomposer.org/root for details Aborting as no plugin should be loaded if running as super user is not explicitly allowed.

I was going to open a separate thread on this… But before I do, any chance that behavior is a known issue that might be impacting me a little differently on this project?

swalkinshaw · September 25, 2022, 10:21pm

I think that might be a bug in Trellis looking at it again For the composer install task, become: no is set (meaning do NOT use sudo): trellis/main.yml at 82a7893e2836c6898a7e73033df291411177d633 · roots/trellis · GitHub

But we don’t set that for composer authentication. I’ll fix that now.

You can manually add become: no for that task to fix it as well.

gilad · September 25, 2022, 10:41pm

Wow, easy fix indeed!

I’m about to go offline for a couple of days, and this is a great sendoff! Thanks!!

sangemaru · January 19, 2023, 12:08pm

Apologies if I’m dropping in an unrelated comment. I provisioned a new server using trellis new and trellis droplet create, deployed to it fine. Next day, I’m trying to redeploy to it, and it fails task deploy : Setup composer authentications (HTTP Basic)].

Here’s the rub: I didn’t even authenticate at this point, I was using wpackagist only. I tried adding my own auth.json and defining a private repo later, but at one point, the file just stopped working. I tried setting no_log: false but it still shows "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
I don’t understand why this is happening.

dmw · July 23, 2023, 11:22pm

I have recently started seeing this behaviour too. I don’t use any custom composer authentications, but trellis deploy is failing at this step. It’s as though it’s failing to skip it when it should be.

I have currently worked around it by commenting out the Setup composer authentications steps in build-after.yml

Twansparant · June 14, 2024, 1:19pm

I’m experiencing the same problem too now with my colleague, we have the exact same setup, same repo. We have no composer_authentications entry in development/vault?

The wordpress-install task just fails with the message:

fatal: [default]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}

Setting no_log: false doesn’t provide any useful information.

Disabling all tasks in composer-authentications.yml is the only thing that works to get past the failed task.

We both have a auth.json file in ~/.composer and we have some premium plugins defined in the site/composer.json file which we are both able to install without errors.

When I run trellis provision --tags=wordpress-install development, it runs the composer-authentications tasks without any problems, but when he runs it, it fails…

Any idea what might be causing this?

strarsis · June 14, 2024, 2:18pm

Just some guess, maybe I am totally wrong here: But could have it something to do with SSH Agent Forwarding and some composer repositories using SSH and SSH authentication.

Twansparant · June 14, 2024, 2:26pm

Yeah could be!
I also compared both our ~/.ssh/config and they’re also identical, so not sure what else I need to check? Running composer diagnose also doesn’t return any errors?