Can't update a custom private plugin with ansible deploy (The .git directory is missing from ...)

maroluke · March 7, 2022, 8:08pm

Recently i’ve updated my private custom plugin successfully.
Today i made a customization to that plugin so i updated github repo and taged the plugin with a new version:
git tag 1.3.4
git push --tags

Then i runed
ansible-playbook deploy.yml -e "site=domain.com env=production"
to deploy the website.

Since today i get an error and i can’t deploy anymore.

Installing dependencies from lock file Verifying lock file contents can be
installed on current platform. Package operations: 19 installs, 3 updates, 0
removals - Syncing ghuser/menu-to-pdf (1.3.5) into cache - Downloading
ghuser/ab-xy (1.0.11) - Downloading ghuser/woocommerce-
tipping (1.2) Failed to download ghuser/woocommerce-tipping from dist: The
"https://api.github.com/repos/ghuser/woocommerce-
tipping/zipball/4d365443837dc7f2x61fea52458923d6ed57da61" file could not be
downloaded (HTTP/2 404 ): {"message":"Not Found","documentation_url":"https:/
/docs.github.com/rest/reference/repos#download-a-repository-archive"} Now
trying to download from source - Syncing ghuser/woocommerce-tipping (1.2)
into cache Failed to download ghuser/ab-xy from dist: The
"https://api.github.com/repos/ghuser/ab-
xy/zipball/5d54bdca103aa3441b92413ecd4ebe3ec951f29c" file could not be
downloaded (HTTP/2 404 ): {"message":"Not Found","documentation_url":"https:/
/docs.github.com/rest/reference/repos#download-a-repository-archive"} Now
trying to download from source - Syncing ghuser/ab-xy
(1.0.11) into cache - Upgrading symfony/polyfill-ctype (v1.24.0 => v1.25.0):
Extracting archive - Installing advanced-custom-fields/advanced-custom-
fields-pro (5.9.4): Extracting archive - Upgrading myclabs/deep-copy (1.10.2
=> 1.11.0): Extracting archive - Upgrading ghuser/menu-to-pdf (1.3.3 =>
1.3.5): Update of ghuser/menu-to-pdf failed - Installing ghuser/saferpay-
gateway-xy (1.0.11): Cloning 5d54bdca10 from cache - Installing ghuser
/woocommerce-tipping (1.2): Cloning 4d36544383 from cache - Installing
roots/wordpress (5.9.1): Extracting archive - Installing wpackagist-plugin
/acf-extended (0.8.8.7): Extracting archive - Installing wpackagist-plugin
/acf-quickedit-fields (3.1.11): Extracting archive - Installing wpackagist-
plugin/akismet (4.2.2): Extracting archive - Installing wpackagist-plugin
/contact-form-7 (5.5.6): Extracting archive - Installing wpackagist-plugin
/contact-form-7-honeypot (2.1): Extracting archive - Installing wpackagist-
plugin/filebird (4.9.8): Extracting archive - Installing wpackagist-plugin
/invisible-recaptcha (1.2.3): Extracting archive - Installing wpackagist-
plugin/log-deprecated-notices (0.4.1): Extracting archive - Installing
wpackagist-plugin/login-ip-country-restriction (6.2.2): Extracting archive -
Installing wpackagist-plugin/mailpoet (3.83.0): Extracting archive -
Installing wpackagist-plugin/scheduled-post-trigger (3.0): Extracting archive
- Installing wpackagist-plugin/simple-custom-post-order (2.5.6): Extracting
archive - Installing wpackagist-plugin/theme-translation-for-polylang
(3.2.12): Extracting archive - Installing wpackagist-plugin/woocommerce
(5.9.0): Extracting archive - Installing wpackagist-plugin/wordpress-seo
(16.9): Extracting archive [RuntimeException] The .git directory is missing
from /srv/www/xy.ch/releases/20220307193742/vendor/ghuser/menu-to-pdf,
see https://getcomposer.org/commit-deps for more information install
[--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-
run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress]
[--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a
|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix
APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ]
[--ignore-platform-reqs] [--] [<packages>]...
fatal: [ip-address]: FAILED! => {"changed": false, "stdout": "Installing dependencies from lock file\nVerifying lock file contents can be installed on current platform.\nPackage operations: 19 installs, 3 updates, 0 removals\n  - Syncing ghuser/menu-to-pdf (1.3.5) into cache\n  - Downloading ghuser/ab-xy (1.0.11)\n  - Downloading ghuser/woocommerce-tipping (1.2)\n    Failed to download ghuser/woocommerce-tipping from dist: The \"https://api.github.com/repos/ghuser/woocommerce-tipping/zipball/4d365443837dc7f2x61fea52458923d6ed57da61\" file could not be downloaded (HTTP/2 404 ):\n{\"message\":\"Not Found\",\"documentation_url\":\"https://docs.github.com/rest/reference/repos#download-a-repository-archive\"}\n    Now trying to download from source\n  - Syncing ghuser/woocommerce-tipping (1.2) into cache\n    Failed to download ghuser/ab-xy from dist: The \"https://api.github.com/repos/ghuser/ab-xy/zipball/5d54bdca103aa3441b92413ecd4ebe3ec951f29c\" file could not be downloaded (HTTP/2 404 ):\n{\"message\":\"Not Found\",\"documentation_url\":\"https://docs.github.com/rest/reference/repos#download-a-repository-archive\"}\n    Now trying to download from source\n  - Syncing ghuser/ab-xy (1.0.11) into cache\n  - Upgrading symfony/polyfill-ctype (v1.24.0 => v1.25.0): Extracting archive\n  - Installing advanced-custom-fields/advanced-custom-fields-pro (5.9.4): Extracting archive\n  - Upgrading myclabs/deep-copy (1.10.2 => 1.11.0): Extracting archive\n  - Upgrading ghuser/menu-to-pdf (1.3.3 => 1.3.5):     Update of ghuser/menu-to-pdf failed\n  - Installing ghuser/ab-xy (1.0.11): Cloning 5d54bdca10 from cache\n  - Installing ghuser/woocommerce-tipping (1.2): Cloning 4d36544383 from cache\n  - Installing roots/wordpress (5.9.1): Extracting archive\n  - Installing wpackagist-plugin/acf-extended (0.8.8.7): Extracting archive\n  - Installing wpackagist-plugin/acf-quickedit-fields (3.1.11): Extracting archive\n  - Installing wpackagist-plugin/akismet (4.2.2): Extracting archive\n  - Installing wpackagist-plugin/contact-form-7 (5.5.6): Extracting archive\n  - Installing wpackagist-plugin/contact-form-7-honeypot (2.1): Extracting archive\n  - Installing wpackagist-plugin/filebird (4.9.8): Extracting archive\n  - Installing wpackagist-plugin/invisible-recaptcha (1.2.3): Extracting archive\n  - Installing wpackagist-plugin/log-deprecated-notices (0.4.1): Extracting archive\n  - Installing wpackagist-plugin/login-ip-country-restriction (6.2.2): Extracting archive\n  - Installing wpackagist-plugin/mailpoet (3.83.0): Extracting archive\n  - Installing wpackagist-plugin/scheduled-post-trigger (3.0): Extracting archive\n  - Installing wpackagist-plugin/simple-custom-post-order (2.5.6): Extracting archive\n  - Installing wpackagist-plugin/theme-translation-for-polylang (3.2.12): Extracting archive\n  - Installing wpackagist-plugin/woocommerce (5.9.0): Extracting archive\n  - Installing wpackagist-plugin/wordpress-seo (16.9): Extracting archive\n\n                                                                                                                                                                          \n  [RuntimeException]                                                                                                                                                      \n  The .git directory is missing from /srv/www/xy.ch/releases/20220307193742/vendor/ghuser/menu-to-pdf, see https://getcomposer.org/commit-deps for more information  \n                                                                                                                                                                          \n\ninstall [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>]...\n\n", "stdout_lines": ["Installing dependencies from lock file", "Verifying lock file contents can be installed on current platform.", "Package operations: 19 installs, 3 updates, 0 removals", "  - Syncing ghuser/menu-to-pdf (1.3.5) into cache", "  - Downloading ghuser/ab-xy (1.0.11)", "  - Downloading ghuser/woocommerce-tipping (1.2)", "    Failed to download ghuser/woocommerce-tipping from dist: The \"https://api.github.com/repos/ghuser/woocommerce-tipping/zipball/4d365443837dc7f2x61fea52458923d6ed57da61\" file could not be downloaded (HTTP/2 404 ):", "{\"message\":\"Not Found\",\"documentation_url\":\"https://docs.github.com/rest/reference/repos#download-a-repository-archive\"}", "    Now trying to download from source", "  - Syncing ghuser/woocommerce-tipping (1.2) into cache", "    Failed to download ghuser/ab-xy from dist: The \"https://api.github.com/repos/ghuser/ab-xy/zipball/5d54bdca103aa3441b92413ecd4ebe3ec951f29c\" file could not be downloaded (HTTP/2 404 ):", "{\"message\":\"Not Found\",\"documentation_url\":\"https://docs.github.com/rest/reference/repos#download-a-repository-archive\"}", "    Now trying to download from source", "  - Syncing ghuser/ab-xy (1.0.11) into cache", "  - Upgrading symfony/polyfill-ctype (v1.24.0 => v1.25.0): Extracting archive", "  - Installing advanced-custom-fields/advanced-custom-fields-pro (5.9.4): Extracting archive", "  - Upgrading myclabs/deep-copy (1.10.2 => 1.11.0): Extracting archive", "  - Upgrading ghuser/menu-to-pdf (1.3.3 => 1.3.5):     Update of ghuser/menu-to-pdf failed", "  - Installing ghuser/ab-xy (1.0.11): Cloning 5d54bdca10 from cache", "  - Installing ghuser/woocommerce-tipping (1.2): Cloning 4d36544383 from cache", "  - Installing roots/wordpress (5.9.1): Extracting archive", "  - Installing wpackagist-plugin/acf-extended (0.8.8.7): Extracting archive", "  - Installing wpackagist-plugin/acf-quickedit-fields (3.1.11): Extracting archive", "  - Installing wpackagist-plugin/akismet (4.2.2): Extracting archive", "  - Installing wpackagist-plugin/contact-form-7 (5.5.6): Extracting archive", "  - Installing wpackagist-plugin/contact-form-7-honeypot (2.1): Extracting archive", "  - Installing wpackagist-plugin/filebird (4.9.8): Extracting archive", "  - Installing wpackagist-plugin/invisible-recaptcha (1.2.3): Extracting archive", "  - Installing wpackagist-plugin/log-deprecated-notices (0.4.1): Extracting archive", "  - Installing wpackagist-plugin/login-ip-country-restriction (6.2.2): Extracting archive", "  - Installing wpackagist-plugin/mailpoet (3.83.0): Extracting archive", "  - Installing wpackagist-plugin/scheduled-post-trigger (3.0): Extracting archive", "  - Installing wpackagist-plugin/simple-custom-post-order (2.5.6): Extracting archive", "  - Installing wpackagist-plugin/theme-translation-for-polylang (3.2.12): Extracting archive", "  - Installing wpackagist-plugin/woocommerce (5.9.0): Extracting archive", "  - Installing wpackagist-plugin/wordpress-seo (16.9): Extracting archive", "", "                                                                                                                                                                          ", "  [RuntimeException]                                                                                                                                                      ", "  The .git directory is missing from /srv/www/xy.ch/releases/20220307193742/vendor/ghuser/menu-to-pdf, see https://getcomposer.org/commit-deps for more information  ", "                                                                                                                                                                          ", "", "install [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>]...", ""]}

The error occurs when composer tries to install dependencies:
TASK [deploy : Install Composer dependencies]
If i remove the plugin, deployment runs successfully.

So what is happening here?
How can i find out more?

swalkinshaw · March 7, 2022, 8:18pm

Does composer install work locally on the same project?

maroluke · March 7, 2022, 9:07pm

Yes, i guess:

| ~/Projects/customer/customer.ch/site @ Users-MacBook-Pro-2 (user) 
| => composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Nothing to install, update or remove
Generating optimized autoload files
13 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

maroluke · March 21, 2022, 11:12am

Hi Scott! Have you another recommendations or ideas?

swalkinshaw · March 22, 2022, 12:38am

The only thing I can suggest is SSHing into your server and running some composer commands manually to try and debug it. Use the web user

One thing I’d definitely try is just deleting the vendor directly to make sure you get a clean composer install.

mattia · March 31, 2022, 12:13pm

@maroluke did you find a solution?
I have the exact same problem.

I updated a private plugin, it works on localhost, but I can’t deploy anymore.

TASK [deploy : Install Composer dependencies] ************************************************************************************************************************************************
System info:
  Ansible 2.9.8; Darwin
  Trellis Head
---------------------------------------------------
Loading composer repositories with package information Installing
dependencies from lock file Package operations: 9 installs, 0 updates, 0
removals - Installing advanced-custom-fields/advanced-custom-fields-pro
(5.12.1): Loading from cache - Installing mattiaband/cardinal-wp (1.5.8):
Downloading Failed to download mattiaband/cardinal-wp from dist: The
"https://api.github.com/repos/mattiaband/cardinal-
wp/zipball/35d7f5bf135392641479e824e150c765b023e32b" file could not be
downloaded (HTTP/1.1 404 Not Found) Now trying to download from source -
Installing mattiaband/cardinal-wp (1.5.8): Failed to update
git@github.com:mattiaband/cardinal-wp.git in cache, package installation for
mattiaband/cardinal-wp might fail. Cloning 35d7f5bf13 from cache
35d7f5bf135392641479e824e150c765b023e32b is gone (history was rewritten?)
[RuntimeException] Failed to execute git checkout
'35d7f5bf135392641479e824e150c765b023e32b' -- && git reset --hard
'35d7f5bf135392641479e824e150c765b023e32b' -- fatal: reference is not a tree:
35d7f5bf135392641479e824e150c765b023e32b install [--prefer-source] [--prefer-
dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-
autoloader] [--no-scripts] [--no-progress] [--no-suggest]
[-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-
authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--]
[<packages>]...
fatal: [46.101.7.190]: FAILED! => {"changed": false, "stdout": "Loading composer repositories with package information\nInstalling dependencies from lock file\nPackage operations: 9 installs, 0 updates, 0 removals\n  - Installing advanced-custom-fields/advanced-custom-fields-pro (5.12.1): Loading from cache\n  - Installing mattiaband/cardinal-wp (1.5.8): Downloading    Failed to download mattiaband/cardinal-wp from dist: The \"https://api.github.com/repos/mattiaband/cardinal-wp/zipball/35d7f5bf135392641479e824e150c765b023e32b\" file could not be downloaded (HTTP/1.1 404 Not Found)\n    Now trying to download from source\n  - Installing mattiaband/cardinal-wp (1.5.8): Failed to update git@github.com:mattiaband/cardinal-wp.git in cache, package installation for mattiaband/cardinal-wp might fail.\nCloning 35d7f5bf13 from cache\n    35d7f5bf135392641479e824e150c765b023e32b is gone (history was rewritten?)\n\n                                                                                                                                                  \n  [RuntimeException]                                                                                                                              \n  Failed to execute git checkout '35d7f5bf135392641479e824e150c765b023e32b' -- && git reset --hard '35d7f5bf135392641479e824e150c765b023e32b' --  \n                                                                                                                                                  \n  fatal: reference is not a tree: 35d7f5bf135392641479e824e150c765b023e32b                                                                        \n                                                                                                                                                  \n\ninstall [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...\n\n", "stdout_lines": ["Loading composer repositories with package information", "Installing dependencies from lock file", "Package operations: 9 installs, 0 updates, 0 removals", "  - Installing advanced-custom-fields/advanced-custom-fields-pro (5.12.1): Loading from cache", "  - Installing mattiaband/cardinal-wp (1.5.8): Downloading    Failed to download mattiaband/cardinal-wp from dist: The \"https://api.github.com/repos/mattiaband/cardinal-wp/zipball/35d7f5bf135392641479e824e150c765b023e32b\" file could not be downloaded (HTTP/1.1 404 Not Found)", "    Now trying to download from source", "  - Installing mattiaband/cardinal-wp (1.5.8): Failed to update git@github.com:mattiaband/cardinal-wp.git in cache, package installation for mattiaband/cardinal-wp might fail.", "Cloning 35d7f5bf13 from cache", "    35d7f5bf135392641479e824e150c765b023e32b is gone (history was rewritten?)", "", "                                                                                                                                                  ", "  [RuntimeException]                                                                                                                              ", "  Failed to execute git checkout '35d7f5bf135392641479e824e150c765b023e32b' -- && git reset --hard '35d7f5bf135392641479e824e150c765b023e32b' --  ", "                                                                                                                                                  ", "  fatal: reference is not a tree: 35d7f5bf135392641479e824e150c765b023e32b                                                                        ", "                                                                                                                                                  ", "", "install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...", ""]}

@swalkinshaw composer update on my local works as expected, when you say " deleting the vendor directory" do you mean on the stag/prod server or on local?

I tried to downgrade the private plugin to the previous version, and I can deploy without any errors, but when I try with the new plugin version, it works on local, but not on the staging.

do you have any other ideas/recommendations?

thanks

maroluke · March 31, 2022, 12:59pm

Hi Mattia, try composer install after you delete the vendor directory, on your production machine like Scott mentioned.

I didn’t try it out yet but i will as soon as i backup my production server. I’m sure this will help.

swalkinshaw · March 31, 2022, 6:59pm

yes that’s what I mean

maroluke · April 1, 2022, 6:36am

He meant on production i guess.

swalkinshaw · April 1, 2022, 7:06pm

Oops yes I meant delete the vendor directory on the remote server you’re deploying to

13milliseconds · April 5, 2022, 3:56pm

I was experiencing the same issue and after running composer install on the staging server, adding a Github token in the process, everything seems to deploy properly… but creating an error 500 for the whole website!

No problems whatsoever locally and the only change is updates to plugins in private repos.

Edit: Of course the guilty party was the updated plugin. WPML’s 5.xx version is throwing errors.

maroluke · April 27, 2022, 12:38pm

Hey @mattia + @13milliseconds i guess this thread could be interessting to our issue: “Help I tried to trellis up an older project and it’s not working!” and other stories

Github recently changed its ssh key rules. So the old ones, since you installed your trellis project, are not working anymore.

Instead dealing manualy with tokkens, i recommend to do it “the right way” by updating your known_hosts.yml file and re-provision your server.

Finaly it solved my issue.
The output is green (i love green these days), no errors.
I hope this helps you too.
Let me know if you have any questions.