Whitelist dev box with Fail2Ban

trellis
1

I’ve been blocked from a staging.server by fail2ban.

I have repeatedly–I think–unbanned my local IP address as returned by https://whatismyipaddress.com:

Status
|- Number of jail:	2
`- Jail list:	ssh, sshd
root@ubuntu-s-staging-example-nyc1-01:/home/admin# fail2ban-client set ssh unbanip 68.225.60.23
1
root@ubuntu-s-staging-example-nyc1-01:/home/admin# fail2ban-client set sshd unbanip 68.225.60.23
1

Then I am able to ssh in manually, but when I try to run a provisioning it gets blocked again:

TASK [python_interpreter : Get Ubuntu version] ***********************************************************************************************************
System info:
  Ansible 2.9.13; Darwin
  Trellis version (per changelog): "Remove default Vagrant SMB credentials"
---------------------------------------------------
Failed to connect to the host via ssh: ssh: connect to host 206xxx.189.xx
port 22: Connection refused
fatal: [206xxx.189.xx]: UNREACHABLE! => {"changed": false, "unreachable": true}

Not finding much in the Trellis docs yet.

Am guessing that something is, maybe going wrong here with the way trellis is connecting, but it–I believe– is working fine connection to the production server.

Does trellis add my local ip address to ip_whitelist when it initially deploys?

2

Why does fail2ban block the IP again? Are there detailed fail2ban logs, including the IP and the reason?
Maybe you are using a different IP when connecting via ansible python SSH (IPv6?)?

3

Well all I’m seeing in /var/logs/fail2ban.log is this:

2021-02-24 23:35:33,087 fail2ban.filter         [670]: INFO    [ssh] Found 68.225.60.23 - 2021-02-24 23:35:33
2021-02-24 23:35:33,193 fail2ban.filter         [670]: INFO    [sshd] Found 68.225.60.23 - 2021-02-24 23:35:33
2021-02-24 23:35:33,244 fail2ban.actions        [670]: NOTICE  [sshd] Ban 68.225.60.23
2021-02-24 23:35:33,247 fail2ban.actions        [670]: NOTICE  [ssh] Ban 68.225.60.23
2021-02-24 23:45:34,133 fail2ban.actions        [670]: NOTICE  [ssh] Unban 68.225.60.23
2021-02-24 23:45:34,133 fail2ban.actions        [670]: NOTICE  [sshd] Unban 68.225.60.23

Is there another place to look?

Also, I seem to have solved the problem by:

  1. Disabling fail2ban: sudo systemctl stop fail2ban
  2. Re-provision the fail2ban tag: trellis provision --tags fail2ban staging

I could still use some clarity on if the IP of the provisioning computer is automatically whitelisted with fail2ban.

4

Hm, why does your fail2ban actually ban that IP address?
Is ansible connecting in a way that could be interpreted as malicious, e.g. too many connection attempts in a too short time?

5

Great. There are a bunch of these:

ROOT LOGIN REFUSED FROM 68.225.60.23 port 64071 [preauth]
Connection closed by authenticating user root 68.225.60.23 port 64071 [preauth]
Connection closed by 68.225.60.23 port 49286 [preauth]

Don’t know why there would have been a root login attempted, though.

6

This only happens when ansible was used?
So ansible apparently is (mis)configured to use the root user for the SSH logins.
It tries it too often and fail2ban considers this as mailicious and (re-(bans the IP address.

7

Seems likely, but I don’t think Trellis is configured to connect as root:

sshd_permit_root_login: false

and

# Documentation: https://roots.io/trellis/docs/ssh-keys/
admin_user: admin

# Also define 'vault_users' (`group_vars/staging/vault.yml`, `group_vars/production/vault.yml`)
users:
  - name: "{{ web_user }}"
    groups:
      - "{{ web_group }}"
    keys:
      - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
      # - https://github.com/username.keys
  - name: "{{ admin_user }}"
    groups:
      - sudo
    keys:
      - "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
      # - https://github.com/username.keys

web_user: web
web_group: www-data
web_sudoers:
  - "/usr/sbin/service php7.4-fpm *"

Is there a config elsewhere I’m missing?

8

It could be the hosts/ directory in your Trellis folder. I use those files for provisioning fresh instances when they require their initial password/root login. That would be the place for forcing ansible to use a different user as Trellis usually would (admin).

9

What do you mean by that? Those files are the source for the IP addresses for the various envs, right? I always just set mine like:

[staging]
20x.189.189.xx

[web]
20x.189.189.xx
10

Run the playbook with very high verbosity (-vvvv). What SSH command ansible actually uses for establishing a connection? Also check the ansible.cfg and ~/.ssh/config.

11

Sorry, starsis. This issue is actually resolved by deactivating fail2ban on the server and running the fail2ban tasks (as noted above).

I was trying to get some clarity on how Trellis handles it.

So I can still do that if it would be useful. I was wondering if you meant you were using the hosts files in a different way than the default and something I might want to be hip to.

12

Apparently ansible tries to connect with a different, invalid user over SSH, which results in fail2ban re-banning the workstation IP. Ansible can be configured to use a specific user or password instead what is specified in Trellis. There are several ways to override the ansible connection settings.

13

Hopefully next time it happens, I’ll have the patience to dig deeper.