500 internal server error / blank page after deployment to staging

nazeemsoeltan · July 5, 2015, 10:24pm

Hi, I recently deployed 2 new sites to staging. They don’t contain anything special, just some plugins and the default wp theme and no other customizations.

When I browse to the sites, I get a blank page, and firebug is giving me a 500 internal server error message. I turned on the Nginx debug logging and for both sites I get the following error:

    2015/07/05 18:15:50 [debug] 3438#0: *23 connect to unix:/var/run/php5-fpm-wordpress.sock, fd:17 #25
2015/07/05 18:15:50 [debug] 3438#0: *23 connected
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream connect: 0
2015/07/05 18:15:50 [debug] 3438#0: *23 posix_memalign: 00007F5D7788C7E0:128 @16
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream send request
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream send request body
2015/07/05 18:15:50 [debug] 3438#0: *23 chain writer buf fl:0 s:976
2015/07/05 18:15:50 [debug] 3438#0: *23 chain writer in: 00007F5D77886768
2015/07/05 18:15:50 [debug] 3438#0: *23 writev: 976 of 976
2015/07/05 18:15:50 [debug] 3438#0: *23 chain writer out: 0000000000000000
2015/07/05 18:15:50 [debug] 3438#0: *23 event timer add: 17: 60000:1436134610508
2015/07/05 18:15:50 [debug] 3438#0: *23 http finalize request: -4, "/index.php?" a:1, c:3
2015/07/05 18:15:50 [debug] 3438#0: *23 http request count:3 blk:0
2015/07/05 18:15:50 [debug] 3438#0: *23 http finalize request: -4, "/index.php?" a:1, c:2
2015/07/05 18:15:50 [debug] 3438#0: *23 http request count:2 blk:0
2015/07/05 18:15:50 [debug] 3438#0: *23 http run request: "/index.php?"
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream check client, write event:1, "/index.php"
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream recv(): -1 (11: Resource temporarily unavailable)
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream request: "/index.php?"
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream dummy handler
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream request: "/index.php?"
2015/07/05 18:15:50 [debug] 3438#0: *23 http upstream process header
2015/07/05 18:15:50 [debug] 3438#0: *23 malloc: 00007F5D77870B50:4096
2015/07/05 18:15:50 [debug] 3438#0: *23 recv: fd:17 104 of 4096
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 01
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 06
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 00
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 01
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 00
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 4D
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 03
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record byte: 00
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi record length: 77
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi parser: 0
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi header: "Status: 500 Internal Server Error"
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi parser: 0
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi header: "Content-type: text/html; charset=UTF-8"
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi parser: 1
2015/07/05 18:15:50 [debug] 3438#0: *23 http fastcgi header done
2015/07/05 18:15:50 [debug] 3438#0: *23 xslt filter header
2015/07/05 18:15:50 [debug] 3438#0: *23 HTTP/1.1 500 Internal Server Error^M

The weird part is that my first deployed site is working, and doesn’t have the problem described above.

I am making use of trellis.

kalenjohnson · July 6, 2015, 5:06am

My only idea is asking if you’ve provisioned the server using the 2nd project. Most things should just be skipped or “ok”, since everything should already be installed, but you do need to run the provision so that the project is actually set up in nginx and the folder structure created. Deployment doesn’t do anything more than creating a new release folder and symlinking, plus whatever hooks you may have set up.

nazeemsoeltan · July 6, 2015, 7:56am

Yes, i did run the provision. At first I forgot that, but even after provisioning, the problem remains.

kalenjohnson · July 6, 2015, 4:38pm

And there are no errors when you provisioned or when you deploy?

nazeemsoeltan · July 7, 2015, 4:30am

There are no errors indeed, I am really out of options

evanfuture · July 8, 2015, 2:38am

I continually had problems deploying, once I finally got to that stage. In a panic, I removed the entire project and rebuilt it from scratch (backing up and restoring the db and theme files of course), including the DO droplet, and it just worked. Not sure why, probably a bug in the old version that was fixed in the new, or who knows what. Its a bit like “did you try turning it off and on again?”, but worth the effort.

Being able to rebuild your project from scratch also keeps your workflow properly clean, though I ended up having to create a new git repo to avoid that tangle…

fullyint · July 8, 2015, 3:07am

Thanks for mentioning the “rebuild from scratch” strategy, @evanfuture . It may seem like a huge pain, but I see it being most effective for myself and others to approach debugging in two steps:

First get the default latest trellis up and running, and only then
Start making customizations and test along the way.

@nazeemsoeltan I’m not suggesting you skipped these steps, because it sounds like you did have things working earlier. But now that you’re stuck on this error, I’d suggest starting over with those steps, testing after each incremental change. It’s a pain, but it may be the only alternative to being stuck, feeling out of options, and just dropping it.

I’m interested in what you may find about the “500 internal server error” because I remember running into it once, but can’t manage to remember what the solution was. I vaguely remember it being something facepalm simple, at least in my case.

nazeemsoeltan · July 9, 2015, 8:31pm

Solution is provided in this topic:

[ansible-provisioning-issue-solved][1]

I turned on php errors and discovered that some salts contained spaced and were not wrapped in quotes.
[1]: Ansible Provisioning Issue - Solved

Henrique_Nunes · December 8, 2015, 7:20pm

Hi all!

Reopening this because I’m having a similar error, but it’s weirdly different.

I’m using Trellis without bedrock. So I have a typical Wordpress directory under current (don’t ask why…) i.e. index.php is in /srv/www/sitename/current/index.php

I added the machine ip to server_name so I can access by ip.

The problem is that when I requesthttp://site_ip I get a 500 error page on Chrome (not a white page).
curl:

* Hostname was NOT found in DNS cache
*   Trying <ip>...
* Connected to <ip> (<ip>) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: <ip>
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
* Server nginx is not blacklisted
< Server: nginx
< Date: Tue, 08 Dec 2015 19:11:49 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
<
* Connection #0 to host <ip> left intact

So I set up nginx log level to debug and I get something similar to the logs above. I can see it’s trying to run the right file:

fastcgi param: "SCRIPT_FILENAME: /srv/www/sitename/current/index.php"

When I request the file directly, thus: http://site_ip/index.php everything loads up fine.

So then I tried to get PHP to spit some errors. Eventually I ended up setting

php_admin_value[display_errors] = 'stderr'

on /etc/php5/fpm/pool.d/wordpress.conf. The result is that I now get a blank (white) page, and a 200 response!

curl:

* Hostname was NOT found in DNS cache
*   Trying <ip>...
* Connected to <ip> (<ip>) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: <ip>
> Accept: */*
>
< HTTP/1.1 200 OK
* Server nginx is not blacklisted
< Server: nginx
< Date: Tue, 08 Dec 2015 19:04:26 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< X-UA-Compatible: IE=Edge
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
<
* Connection #0 to host <ip> left intact

I’m completely lost here… I don’t understand why is it that requesting the file directly works, and requesting / doesn’t, when from the nginx debug log I can see that it gets the right file to PHP. I also can’t get PHP to tell me what the error is because when I do it goes away!

This is really frustrating. Any suggestions?