Site doesn't redirect to https (have looked at suggested solutions)

Hi,

Last week we migrated our site from Amazon to Google Cloud using Trellis, after testing on a GC staging environment. Everything seemed to be fine, including the automatic redirect in the browser from australiascience.tv to https://australiascience.tv. I’m pretty sure I tested this after clearing out the caches in both browsers.

This morning, however, I’m finding that in both FF and Chrome, after clearing the browser cache, both our staging URL and our Production URL time out (ERR_CONNECTION_TIMED_OUT), i.e. they do not redirect and users must explicitly type https://australiascience.tv into their browsers.

I found this thread, https://discourse.roots.io/t/problems-with-com-au-domains/6680/4, and am wondering if it has something to do with our unusual TLD, “.tv”. However, the suggested fix doesn’t help, since my lib/trellis/plugins/filter/filters.py looks like this:

    # Make coding more python3-ish
    from __future__ import (absolute_import, division, print_function, unicode_literals)
    __metaclass__ = type

    import types

    from ansible import errors
    from ansible.module_utils.six import string_types

    def to_env(dict_value):
        envs = ["{0}='{1}'".format(key.upper(), str(value).replace("'","\\'")) for key, value in sorted(dict_value.items())]
        return "\n".join(envs)

    def underscore(value):
        ''' Convert dots to underscore in a string '''
        return value.replace('.', '_')

    class FilterModule(object):
        ''' Trellis jinja2 filters '''

        def filters(self):
            return {
                'to_env': to_env,
                'underscore': underscore,
            }

i.e. there is no test on the length of the hostname.

On the server, I’ve also had a look at /etc/nginx/sites-enabled/australiascience.tv.conf, but am uncertain what (if anything) to change).

Meanwhile, here is our group_vars/production/wordpress_sites.yml:

# Documentation: https://roots.io/trellis/docs/remote-server-setup/
# `wordpress_sites` options: https://roots.io/trellis/docs/wordpress-sites
# Define accompanying passwords/secrets in group_vars/production/vault.yml

# See also docs on SSL: https://roots.io/trellis/docs/ssl/
# SSL provider is https://www.geotrust.com/
wordpress_sites:
  australiascience.tv:
    site_hosts:
      - canonical: australiascience.tv
        redirects:
          - www.australiascience.tv
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    repo: git@gitlab.com:australiasciencechannel/wordpress.git # replace with your Git repo URL
    # repo_subtree_path: site # relative path to your Bedrock/WP directory in your repo
    branch: wordpress-sprint1
    multisite:
      enabled: false
    ssl:
      enabled: true
      provider: manual
      cert: ~/ssl/australiascience.tv.crt
      key: ~/ssl/australiascience.tv.key
    cache:
      enabled: true
    # We point the database to Google Cloud SQL. This db_user's password should be
    # env.db_password, as defined in vault.yml
    db_create: false
    env:
      db_name: australiascience_production
      db_user: australiascience
      db_host: 35.189.40.22

Finally, a colleague has noted that on Apache we would have made the following updates:

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://australiascience.tv/$1 [R,L]

but since this is an Nginx server we’re not sure how to replicate that configuration.

Do you have any advice for us? We would be grateful for any assistance!

Sad to hear about this.

Could you post the top several lines of your Trellis CHANGELOG.md to help us know roughly your version of Trellis?

Could you post the full content of your server’s /etc/nginx/sites-enabled/australiascience.tv.conf?

Thank you, as always, for your super-fast response!

Here is the start of CHANGELOG.md:

### HEAD
* [BREAKING] Normalize `apt` tasks ([#881](https://github.com/roots/trellis/pull/881))
* Ansible 2.4 compatibility ([#895](https://github.com/roots/trellis/pull/895))
* Default h5bp expires and cache busting to false ([#894](https://github.com/roots/trellis/pull/894))
* Deploys: Update WP theme paths for multisite subsites ([#854](https://github.com/roots/trellis/pull/854))
* Vagrant: Support DHCP ([#892](https://github.com/roots/trellis/pull/892))
* Extract Trellis::Config ([#890](https://github.com/roots/trellis/pull/890))
* Redirect directly to https canonical domain ([#889](https://github.com/roots/trellis/pull/889))
* WordPress Setup: Add Nginx `ssl_client_certificate` ([#869](https://github.com/roots/trellis/pull/869))

Here is our /etc/nginx/sites-enabled/australiascience.tv.conf:

# Ansible managed


server {
  listen [::]:443 ssl http2;
  listen 443 ssl http2;
  server_name australiascience.tv;
  
  access_log   /srv/www/australiascience.tv/logs/access.log main;
  error_log    /srv/www/australiascience.tv/logs/error.log;
  
  root  /srv/www/australiascience.tv/current/web;
  index index.php index.htm index.html;
  add_header Fastcgi-Cache $upstream_cache_status;

  # Specify a charset
  charset utf-8;

  # Set the max body size equal to PHP's max POST size.
  client_max_body_size 10g;

  # Fastcgi cache conditions
  set $skip_cache 0;
  if ($query_string != "") {
    set $skip_cache 1;
  }
  if ($request_uri ~* "/wp-admin/|/wp-json/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
    set $skip_cache 1;
  }
  if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
    set $skip_cache 1;
  }

  # SSL configuration
  include h5bp/directive-only/ssl.conf;
  include h5bp/directive-only/ssl-stapling.conf;
  ssl_dhparam /etc/nginx/ssl/dhparams.pem;
  ssl_buffer_size 1400; # 1400 bytes to fit in one MTU

  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; ";

  ssl_certificate         /etc/nginx/ssl/australiascience.tv.crt;
  ssl_certificate_key     /etc/nginx/ssl/australiascience.tv.key;

  include acme-challenge-location.conf;

  include includes.d/australiascience.tv/*.conf;

  # Prevent PHP scripts from being executed inside the uploads folder.
  location ~* /app/uploads/.*\.php$ {
    deny all;
  }
  
  location / {
    try_files $uri $uri/ /index.php?$args;
  }
  
  include h5bp/directive-only/cache-file-descriptors.conf;
  include h5bp/directive-only/extra-security.conf;
  include h5bp/directive-only/x-ua-compatible.conf;
  include h5bp/location/cross-domain-fonts.conf;
  include h5bp/location/protect-system-files.conf;
  
  location ~ \.php$ {
    try_files $uri /index.php;

    # Fastcgi cache settings
    fastcgi_cache wordpress;
    fastcgi_cache_valid 30s;
    fastcgi_cache_bypass $skip_cache;
    fastcgi_no_cache $skip_cache;

    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
    fastcgi_param DOCUMENT_ROOT $realpath_root;
    fastcgi_pass unix:/var/run/php-fpm-wordpress.sock;
  }
}

# Redirect to https
server {
  listen [::]:80;
  listen 80;
  server_name australiascience.tv;

  include acme-challenge-location.conf;

  include includes.d/australiascience.tv/*.conf;

  location / {
    return 301 https://$host$request_uri;
  }
}

# Redirect some domains
server {
  listen [::]:443 ssl http2;
  listen 443 ssl http2;
  listen [::]:80;
  listen 80;
  server_name www.australiascience.tv;

  # SSL configuration
  include h5bp/directive-only/ssl.conf;
  include h5bp/directive-only/ssl-stapling.conf;
  ssl_dhparam /etc/nginx/ssl/dhparams.pem;
  ssl_buffer_size 1400; # 1400 bytes to fit in one MTU

  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; ";

  ssl_certificate         /etc/nginx/ssl/australiascience.tv.crt;
  ssl_certificate_key     /etc/nginx/ssl/australiascience.tv.key;

  include acme-challenge-location.conf;

  include includes.d/australiascience.tv/*.conf;

  location / {
    return 301 https://australiascience.tv$request_uri;
  }
}

Thanks again. Note that we have forked Trellis so perhaps there are updates from the last month that we should have pulled.

1 Like

Thank you for posting this helpful information. I don’t think any of the missing recent Trellis updates would be a factor.

The Nginx conf looks fine. The # Redirect some domains section works. These seem to redirect fine:

https://www.australiascience.tv -> https://australiascience.tv
http://www.australiascience.tv  -> https://australiascience.tv

Of course, the # Redirect to https section is not working for some reason.

http://australiascience.tv      -> https://australiascience.tv

Maybe configs need to be applied?

The best case scenario is that the configs just haven’t taken hold for some reason and you need to apply them. For example, perhaps there was a glitch during provisioning (server.yml) and Nginx didn’t have a chance to reload at the end of the playbook. This can happen if there is an unusually severe problem, or if you submit a keyboard interrupt while the playbook is running.

:question: Did production get a complete error-free run of server.yml?

:star: Could you SSH in as root or as the admin_user and run these commands?

# test Nginx configs
sudo nginx -t

# if no errors in output above, reload Nginx
sudo service nginx reload

:star: Assuming the commands above succeed, try clearing your browsers’ caches and testing the redirect again.

Maybe competing configs?

The # Redirect to https server block has include includes.d/australiascience.tv/*.conf;

:question: Do you have any such includes and could they be interfering?
(e.g., if they have a location / {...})

:question: Similarly, do you have any other Nginx confs (e.g., in sites-enabled or elsewhere) with a server block that contains the following directives?

  • server_name australiascience.tv;
  • listen 80;

If so, Nginx could be matching these blocks instead of the desired redirects section in /etc/nginx/sites-enabled/australiascience.tv.conf and you would need to consolidate the two competing blocks into one.

:question: Any chance you have your staging site/environment also on this same server?
:star: If so, you’ll need to use a “hostname alias in hosts file” and a unique “site key” between environments. See corresponding sections here.

2 Likes

Thanks so much.

I can’t be 100% sure if provisioning ever ran with a glitch (deadline, pretty tired by that time), but I do know we had to run provisioning at least twice. Reason: we were changing over from production.australiascience.tv to australiascience.tv to reflect DNS changes after testing. Certainly on the last time I ran it, it was clean.

Interestingly, we now get:

mclarke@aussciencetv-production:~$ sudo nginx -t
[sudo] password for mclarke: 
nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/ssl/australiascience.tv.crt"
nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/ssl/australiascience.tv.crt"
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
mclarke@aussciencetv-production:~$ 
mclarke@aussciencetv-production:~$ sudo ls -l /etc/nginx/ssl
total 12
-rw-r----- 1 root root 1998 Nov  8 03:03 australiascience.tv.crt
-rw------- 1 root root 1730 Nov  8 03:03 australiascience.tv.key
-rw-r--r-- 1 root root  424 Nov  8 02:54 dhparams.pem
mclarke@aussciencetv-production:~$

Perhaps the group ‘root’ should have read access to the .key file as well?

Restarted nginx as suggested, cleared caches and tried again, but no joy.

I noticed previously (and should have mentioned – sorry) that the includes.d directory doesn’t seem to exist:

mclarke@aussciencetv-production:/etc/nginx$ ls
acme-challenge-location.conf  fastcgi_params       koi-utf     modules-available  proxy_params     sites-enabled  uwsgi_params
conf.d                        h5bp                 koi-win     modules-enabled    scgi_params      snippets       win-utf
fastcgi.conf                  h5bp-server-configs  mime.types  nginx.conf         sites-available  ssl
mclarke@aussciencetv-production:/etc/nginx$ ls sites-enabled/
australiascience.tv.conf  no-default.conf
mclarke@aussciencetv-production:/etc/nginx$ ls sites-available/
australiascience.tv.conf  default  no-default.conf
mclarke@aussciencetv-production:/etc/nginx$ 

Is it okay that they don’t exist?

Here is sites-enabled/no-default.conf:

# Ansible managed

# Drop requests for unknown hosts
#
# If no default server is defined, nginx will use the first found server.
# To prevent host header attacks, or other potential problems when an unknown
# servername is used in a request, it's recommended to drop the request
# returning 444 "no response".

server {
  listen [::]:80 default_server deferred;
  listen 80 default_server deferred;
  return 444;
}

which doesn’t include the server_name again.

Here is sites-available/default (commented-out blocks removed):

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;

        server_name _;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
}

And no, staging and production are on different instances in Google Cloud, with different IP addresses.

Should I run provisioning again, do you think?

1 Like

Also, I did a more thorough check for ‘location’:

mclarke@aussciencetv-production:/etc/nginx$ grep -r 'location' */*.conf
h5bp/basic.conf:include h5bp/location/expires.conf;
h5bp/basic.conf:include h5bp/location/cross-domain-fonts.conf;
h5bp/basic.conf:include h5bp/location/protect-system-files.conf;
h5bp-server-configs/nginx.conf:    text/vnd.rim.location.xloc
h5bp-server-configs/nginx.conf:  # for the check. It is best if you enable this in a location{} block for
sites-available/australiascience.tv.conf:  include acme-challenge-location.conf;
sites-available/australiascience.tv.conf:  location ~* /app/uploads/.*\.php$ {
sites-available/australiascience.tv.conf:  location / {
etc etc the remainder just australiascience.tv.conf

Rerunning playbooks. It shouldn’t hurt to run the entire server.yml and deploy.yml again.
:star: Please do run these again.

nginx -t warning. I doubt the nginx: [warn] "ssl_stapling" ignored, issuer certificate not found... is a factor here, but eventually you’ll want to look into it, and how ssllabs gives your domain This server's certificate chain is incomplete..

cert and key permissions

I believe the primary Nginx process is running as the root user. I think the permissions on your key and cert look correct.

includes.d The missing includes.d directory is fine. It’s actually one less thing to troubleshoot (no nginx-includes involved).

DNS issue? I’m no DNS expert, and I can’t quite think of what DNS scenario would produce this symptom, but the next step I’d take if I were in your shoes is thoroughly review your DNS, unless you’re certain it’s all in order.

IP issue? One potential related note is that right now https://production.australiascience.tv/ is still loading, and from same IP as https://australiascience.tv/ (35.197.169.180). So something’s weird because I thought the Nginx confs for this server and IP were only serving australiascience.tv and www.australiascience.tv. As a wild guess, do you have elastic IP associations or load balancers involved and that may need review?

1 Like

Thank you so much, @fullyint.

I have found the issue! Nothing to do with Trellis at all, as you suspected.

Google Cloud instances give you the option of allowing HTTP and/or HTTPS traffic, and I had enabled HTTPS only. I just spotted that, and switched it on, and of course, problem solved.

Many apologies for wasting your time!

2 Likes