Best practice for multiple sites on multiple servers

Hi all,

I’m wondering how to go about deploying a combination of multiple websites (not multisite) across multiple servers.

Ideally I would like the flexibility of, say, deploying one particular site in a server, a few sites in another server, and so on.

So I see that deploying multiple sites in one server is not a problem. But can different sites on different servers be accomplished with just one trellis installation? Or would I be better off with a trellis installation per site? Or a trellis installation per machine?

What do you guys reckon?

Thanks!

3 Likes

I have the same question here.

And considering each Box is around 2GB is also something to take into account.

I first started grouping most of our sites in one installation, but the problem comes when they need to be deployed on a different server.

Currently I’m moving to a one installation per remote server, but Box size and configs are making me doubt if that is the best practice.

What is going to happen when having, let’s say, 50 different installations?

Some discussions were held around this: To deploy a second site

Can anyone share his experience?

I don’t have much experience with a high volume of sites and projects. My inclination would be to use a different Trellis project for each logical group of sites. In most cases for me that would be one Trellis project for a single site for a client, another Trellis project for a separate unrelated client, etc.

Assuming “each Box” you’re referring to is a Vagrant VM, my approach would be to only keep a VM if I will be working on it in the next several days. Otherwise, I back up the DB and destroy the VM because my machine has limited space. When I need to work on a dev VM for that particular project, I vagrant up a new VM using the project’s Trellis files I’ve saved and import the DB.

2 Likes

Sounds good, @fullyint. Thanks!

I was looking at your suggestion on a previous post where you have a situation where you might have a client who owns multiple servers for different projects and wish to have only one trellis installation.

-- trellis
-- site1
-- site2

My hosts/production file will look like this.

site1 ansible_ssh_host=IP1
site2 ansible_ssh_host=IP2

[production]
site1
site2

[web]
site1
site2

And I created the hosts_folder like this:

host_vars
-- site1
---- wordpress_sites.yml
-- site2
---- wordpress_sites.yml

What should I run to provision server on IP2? Am I going the right way?

1 Like

@40Q

My hosts file is setup the same, but my wordpress_sites.yml is 1 file with all sites defined inside it. I use Ansible’s --limit to run the playbooks on specific hosts, and comment out sites which aren’t applicable to it in the wordpress_sites.yml file.

I know it’s a hack (and you have to be extra careful with your commands), but it has worked without error for me so far.

1 Like

@40Q I think it’s cool that you’re trying this out. I haven’t tried it. If you run the regular command

ansible-playbook server.yml -e env=production

this results in hosts: web:&production, meaning "run server.yml on all the hosts in my inventory that are simultaneously in the web group and the production group (c.f. Ansible docs regarding specifying “intersection of two groups”). In other words, the command will provision both site1 and site2, each on the separate and respective IPs you have assigned in your inventory.

I believe you can use the --limit option @richvida mentioned to run server.yml for only one host/site, which is one IP in your case.

ansible-playbook server.yml -e env=production --limit site2

The advantage of setting up the host_vars directory as you have done, is that you shouldn’t have to comment out inapplicable sites from worpdress_sites when you run your command because each host_vars file defines only the desired wordpress_sites just for that host.

One downside of adding host_vars is that it introduces structure that is absent from the upstream official Trellis, which could mean you’ll have to resolve conflicts that may arise if the upstream structure changes in a conflicting manner.

Another downside of adding host_vars is that you may need to deal with this issue:

I’m sure it’s possible to handle, but it may require a bit of familiarity with the Ansible’s mechanisms (read: more work) and perhaps some additional structure.

If the different projects/servers have a lot of special configs that are shared and it is particularly helpful to have them in a single Trellis project for the sake of logistics and keeping things DRY, then I may be inclined to do the integration work we’ve been discussing. Otherwise, I’d probably still be inclined to have separate Trellis projects per site/server, even though they are all for a single client.

I mean “logical group” in terms of project code and Trellis configs and/or customizations shared across sites/projects. I would not group sites/projects into a single Trellis project solely on the basis that they all belong to a single client. I don’t know which situation you face.

Again, I haven’t tried any of the above. If you trial both approaches a little bit, a preferred strategy may become clear.

3 Likes

@richvida, @fullyint, thanks for your answer.

I was able to run the setup using --limit parameter, but had some issues over the provision. I will keep working in finding a solution, but your answers opened me some new questions :slight_smile: Thanks!

I’ll also follow this advice. Once again, thanks!

1 Like

I have a need for this too, so this is what I just did for a test.

Let’s say I have two sites, and I want staging & production for each. I have these folders now:

host_vars/prod_site1/
host_vars/prod_site2/
host_vars/stage_site1/
host_vars/stage_site2/

In those folders, I made copies of the vault.yml, wordpress_sites.yml and main.yml.

My hosts/staging file looks similar to @40Q example.

I just provisioned a new staging machine like this:

ansible-playbook server.yml -e env=staging --limit stage_site1

Worked fine with no errors. Haven’t tried a deployment yet, but that’s next.

1 Like

I’m starting to think I’m “doing Trellis wrong”. I do a fresh Trellis install for each website I make and deploy it to a fresh droplet.

Personally I prefer this as its cleaner in my mind but was Trellis intended to setup dozens of sites off the same install? I’d very much dislike the ability to potentially break a bunch of other sites.

I do it the way you’re doing it. Separate everything for separate sites. It keeps things tidy and lets me focus on only breaking one thing at a time.

2 Likes

Hi all. I know this is an old thread, but this is what I do:

I keep a single trellis installation at ~/workspace/trellis/

I make a branch for each site git checkout -b dev-my-new-site and have the local bedrock installation at ~/workspace/my-new-site/

It seems to work pretty well with multiple sites that each have their own server.

I can update trellis or make global changes in the master branch and merge into the specific dev-* branches. For example, when I added valentinocossar/trellis-database-uploads-migration, I commited it in master, then merged into dev-my-new-site and updated the hosts file in dev-my-new-site.

Hope this helps someone!

Edit:

One problem with this set up is that I can only run one dev site at a time. Also, to switch sites, I have to halt and up the vagrant machine:

git checkout dev-another-site
vagrant halt && vagrant up

1 Like

I’d like to share my approach I’m testing, too, and I’d like to hear what do you think.

First, I dropped using Vagrant in favor of Docker for local development. I have a docker compose and bash script that set up all containers needed for a fresh project. I configured it in the way that a proxy container would redirect the requests to containers with wordpress (actually wordpress, web server and php). This containers are connected with a database (every project has its own, mounted on a local host). So by just running a script I set up a new project which name is based on a parent directory name. Then If I want to work on some other project, I just run the script in another project directory. The parent directory contains standard bedrock and customized trellis (for further deployment) dirs along with the docker dir. So now I’m starting working locally.

I customized the trellis a little bit by adding some bash scripts that are run before the actual deployment. When I want to deploy it to the development server, I just create a branch of master with appropriate name (let’s say I add the word dev to the branch). When this branch is pushed to the repository, the gitlab-runner on a separate server runs a script that changes the site name and everything that has to be changed in ansible playbooks, and deploys it (also installs the wordpress) to the dev server. Then I just migrate the database from local dev to the dev server. This way, when the other team members want to work on a project, they pull a repo, start a project on docker and pull a database and repeat the process of pushing after they finish.

Everything is the same for staging and production. If everything works fine on a dev server, I create the branch for staging/prod and gitlab-ci along with bash scripts and trellis deploy the whole thing to appropriate server.

I have reverse proxy with wildcard redirections for dev/staging so the sites are accessible immediately (well, the fresh set up takes approx. a minute).

The database and uploads are migrated with a plugin and the flow is local <-> dev <-> staging <-> production. Nevertheless I’d like to implement automation for that.

Final implementation would be on AWS set up with Terraform, so everything could be up and running just with a bunch of commands.

If you have some thoughts or tips, I’ll be glad if you share them.

4 Likes

Apologies for resurrecting this old thread but it’s the best indexed one and I was looking for a solution for this. I figured I’d share what worked for me.

I have my main trellis box that I use with my primary local/staging/production workflow but I took on a project where the client wanted to host the site themselves.

Rather than interrupting my existing workflow,

  1. I cloned a new trellis and created the recommended folder structure
  2. symlinked the to-be-client-hosted site between my existing Trellis and the new one (in hindsight I could have just pointed the wordpress_sites path directly to the original location).
  3. skipped provisioning the development box for the new Trellis and only used it to provision the remote server and deploy the site there.

This may not be the most elegant solution but in a time crunch it worked quite well. For all intents and purposes, the only differences between both Trellis directories are the staging and production group_vars.

1 Like

Thanks @richvida this helped me a lot.

Apparently manual deployments work fine but when I added a pipeline for auto deployments it couldn’t go through. The problem with this approach is that trellis tries to deploy multiple sites, that is if you have multiple hosts per environment, otherwise it’s fine. My pipeline used to fail every time until I added --limit <site-name> parameter to ansible-playbook command.

The other issue is that the pipeline takes about 5 minutes to pass, but I think it’s possible to add some kind of cache to solve this issue.