Now I have several ansible provisioned VMs running in production, I’m wondering how I should handle the ongoing system administration.
Let’s say I want to tweak the Nginx config for example - should I make the changes in the relevant ansible config and re-run the playbook? Currently I only run the playbook to provision and then use the admin user to ssh in and make changes manually.
What is best practice in this situation?
Ideally you should never do anything via manual SSH. This defeats the purpose of server configuration management and will lead to drift and other problems.
It’s a hard habit to get into but you should do everything in Ansible and re-provision (you can just run that single role too).
You’ll save yourself a lot of problems doing this. The most common one is that you need to scale a new server or replace an existing one. With everything in Ansible it’s just a single command to get an exact replica server. Then think of the situation where you have to somehow track down all your manual modifications and re-do them.
I knew that really. Just needed to get confirmation from the master.
To some degree this also depends on your philosophy.
Some people practice “immutable server” concept, where you don’t modify the currently running server. Any change causes a brand new server to be spun up and the old one gets killed off. (Treat your servers as cattle, not pets.)
Interesting. So is that the philosophy you follow?
I’ve been tending Linux servers by hand for years but I’m trying hard to break the habit.
In the age of Digital Ocean and cheap VMs I guess it makes sense to just spin up a new server, the only issue I can see is DNS. I suppose that’s a small price to pay.
On a related note - how do you go about keeping your Ansible roles up to date?
I have my own instance of Gitlab running on a DO droplet which I just use for storing the Ansible directory for each project (I wanted a nice private repo so I can store the passwords without encryption). I keep the site directory on Bitbucket where the guys I work with have access to everything.
Every couple of weeks or so I’ve cloned the latest version of Trellis, copied in my group_vars and hosts files and pushed them up to a fresh repo on Gitlab. Is there a better way?
Now I know that I have to edit the roles to make changes to my server configuration, it’s going to get more complicated to keep up to date. Having read what’s going on on Github I know there are a lot of things coming in this project that I want to make use of (Microcaching!).
Presently, no, I don’t follow that workflow. In my situation it’s more viable to update current servers. But there are many people/companies that do that.
[quote=“treb0r, post:5, topic:4053”]
In the age of Digital Ocean and cheap VMs I guess it makes sense to just spin up a new server, the only issue I can see is DNS. I suppose that’s a small price to pay.
[/quote]DNS will point to your load balancer and your automated provisioning scripts(like ansible) will automatically add new servers to the LB setup.
[quote=“treb0r, post:5, topic:4053”]
On a related note - how do you go about keeping your Ansible roles up to date?
[/quote]So far we don’t have much experience working with roots projects, so this is based on my overall experience. At my company we use BitBucket private repos(for us pricing of bitbucket was better than github). We decided to keep Ansible playbooks together with project files themselves, so each repo is completely self-contained. Our projects are relatively short-lived, 3-9 months, sometimes 1 year, so keeping them up-to-date is not as relevant.
I’d like to hear more about that!
[quote=“treb0r, post:7, topic:4053”]
I’d like to hear more about that!
[/quote]Take a look at this http://docs.ansible.com/guide_rolling_upgrade.html. At the very end there’s also a link to [LAMP Stack + HAProxy: Example Playbooks][1].
[1]: https://github.com/ansible/ansible-examples/tree/master/lamp_haproxy
So just to be totally clear about this, I will never again log into a VM with SSH, not even to run aptitude?
Could it be that the only reason to SSH in at all would be to peruse the logs?
I just run the playbook every day to make sure the VM is updated, yes?
Correct. You can manually log in to do lots of activities like:
But anything that really changes the machine should be done through Ansible.
Thanks for the confirmation. I’m thinking about adding a centralized loghost to handle all the logs from all the VMs in one place.
good idea. There’s some good services like Papertrail, Logentries, Loggly, etc.
Papertrail is baller. Real easy to set up.
and there is an Ansible role to install it here:
I’m going to give it a go, thanks guys.