Roots Discourse

Netdata monitoring, opening ports?


#1

I’m trying to get netdata installed for monitoring purposes. I’ve used New Relic in the past but it requires installation from root and I have root login disabled on all Trellis instances.

I’ve got an Ansible role that gets netdata all installed and set up but I’m having issue accessing it. It’s accessed via port 19999 but I just get ERR_CONNECTION_REFUSED, which I had expected. However, I’m having issues opening the port up.

I’ve amended ferm_input_list located in group_vars/all/security.yml as follows:

ferm_input_list:
  - type: dport_accept
    dport: [http, https]
    filename: nginx_accept
  - type: dport_accept
    dport: [ssh]
    saddr: "{{ ip_whitelist }}"
  - type: dport_limit
    dport: [ssh]
    seconds: 300
    hits: 20
  - type: dport_accept
    dport: [19999]
    protocol: tcp
  - type: dport_accept
    dport: [19999]
    protocol: udp

But no luck there (either by provisioning with --tags ferm or a full provision). Any ideas?


#2

ferm_input_list is pretty simple and your additions look correct. I don’t have any experience with netdata so maybe it’s something else


#3

Hummmm.

OK, well that’s good to know. I can access netdata if I SSH in and curl 127.0.0.1:19999 but I’m guessing there’s an issue with passing an external request through nginx?


#4

If netdata requires an HTTP server then yeah you’d need to define that in Nginx. Trellis recently got support for arbitrary sites: https://roots.io/trellis/docs/nginx-includes/#sites-templates


#5

Ah, interesting. Let me take shot at that.


#6

Perhaps a little in over my head – bit stuck debugging this. Looks like the Ansible/Nginx side is all fine but 99% sure the issue is down to my config. Not expecting a ton of support, this is pretty out there! I’ll list what I’ve done so far, in case anyone is interested :grinning:

I’ve updated Trellis to include the recent Nginx sites configuration support.

I’ve added nginx-includes/netdata.conf.site.j2, which contains

# {{ ansible_managed }}

upstream backend {
    # the netdata server
    server 127.0.0.1:19999;
    keepalive 64;
}

server {
    # nginx listens to this
    listen 80;

    # the virtual host name of this
    server_name netdata.example.co.uk;

    location / {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_pass_request_headers on;
        proxy_set_header Connection "keep-alive";
        proxy_store off;
    }
}

My group_vars/production/main.yml looks like this:

mysql_root_password: "{{ vault_mysql_root_password }}" # Define this variable in group_vars/production/vault.yml

nginx_sites_confs:
  - src: no-default.conf.j2
  - src: nginx-includes/netdata.conf.site.j2

I’ve run ansible-playbook server.tml -e env=production --tags nginx-site and also tried a full provision. Both run as expected.

And as in the OP, I’ve added port 19999 to group_vars/all/security.yml.


#7

I’m not familiar with all of this, but if you have SSL enabled for the primary site, any chance the trouble accessing the netdata subdomain on port 80 is an HSTS subdomains issue?

hsts_include_subdomains - also make all subdomains be served over HTTPS (default: true)

(maybe related comment/idea of making the default false)

You’ll know better than I if hsts and https are relevant, but if so, perhaps you could have the virtual host listen on port 443 instead of 80, and maybe you’d want a redirect from http to https (example).


#8

Appreciate the reply.

I’m actually just testing this out of a non-production site, so no SSL to keep things simple.


#10

@nathobson Did you ever get this figured out?


#11

I had a similar issue, but in that case it was the hardware firewall in front of the cloud instance! I had to open the port in hardware firewall in cloud configuration panel, too.