[HOW-TO] Use Nginx as a reverse proxy for DirectAdmin

I try to use Nginx as a reverse proxy to see if there is any improvement over Apache Worker MPM or not. I found an instruction from DirectAdmin webboard but it's quite old. So, I just tried to follow their steps and do some modification to match what I have read. Below is what I did

1. You need to compile Nginx. Nginx version 0.8.54 is the lastest stable release as of today. So, just copy and paste below code into your SSH

wget http://sysoev.ru/nginx/nginx-0.8.54.tar.gz
tar xvfz nginx-0.8.54.tar.gz
cd nginx-0.8.54
./configure --with-http_stub_status_module --with-http_gzip_static_module
make
make install

After installed, path related to Nginx is shown below : (Note that since you compile Nginx, there is no nginx.pid file there. You need to write it.)

  nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/local/nginx/sbin/nginx"
  nginx configuration prefix: "/usr/local/nginx/conf"
  nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx/logs/nginx.pid"
  nginx error log file: "/usr/local/nginx/logs/error.log"
  nginx http access log file: "/usr/local/nginx/logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

2. Edit configuration file:

nano -w /usr/local/nginx/conf/nginx.conf

Code is like below :

user  apache apache;
        
worker_processes     4; # Set it according to what your CPU have. 4 Cores = 4
worker_rlimit_nofile 8192;
    
pid /var/run/nginx.pid;
    
events {
  worker_connections 1024;
}   
    
http {
    include       mime.types;
    default_type  application/octet-stream;
      
    log_format main '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status  $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';
        
    server_tokens off;
    access_log  /var/log/nginx_access.log  main;
    error_log  /var/log/nginx_error.log debug;
        
    server_names_hash_bucket_size 64;
    sendfile on;
    tcp_nopush     on;
    tcp_nodelay    off;
    keepalive_timeout  30;
    gzip  on;
    gzip_comp_level 9;
    gzip_proxied any;
        
    proxy_buffering on;
    proxy_cache_path /usr/local/nginx/proxy_temp levels=1:2 keys_zone=one:15m inactive=7d max_size=1000m;
    proxy_buffer_size 16k;
    proxy_buffers 100 8k;
    proxy_connect_timeout      60;
    proxy_send_timeout         60;
    proxy_read_timeout         60;
    
    server {
      listen xxx.xxx.xxx.xxx:81 default rcvbuf=8192 sndbuf=16384 backlog=32000; # Real IP here
      server_name domain.name  _ ;     # "_" is for handle all hosts that are not described by server_name
      charset off;
      access_log  /var/log/nginx_host_general.access.log  main;
      location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://xxx.xxx.xxx.xxx;    # Real IP here
        client_max_body_size       16m;
        client_body_buffer_size    128k;
        proxy_buffering     on;
        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         120;
        proxy_buffer_size          16k;
        proxy_buffers              32 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
      }
    }
    # below is include so that you can have separate setting per vhost.
    include /usr/local/nginx/vhosts/*.conf;
}

In the above config, it's general reverse proxy for all domain name on DirectAdmin server. If you need to have a separate configuration like domain A to really use Nginx as a web server, not reverse proxy, then, you can just create that configuration file inside "/usr/local/nginx/vhosts/". Or you may need to change something different from global setting. Just create a new configuration file for that specific Virtualhost.

Also, insert code below if you want to monitor Nginx from Munin or anything similar :

server {
       listen 127.0.0.1:81;
       server_name localhost;
       location /nginx_status {
               stub_status on;
               access_log   off;
               allow xxx.xxx.xxx.xxx; # Real IP here
               allow 127.0.0.1;
               deny all;
       }
 }

Here, I have to set localport to use port 81 since it's the port that we use to monitor this Nginx setup.

3. Create init.d script to be able to start stop nginx service.

nano -w /etc/init.d/nginx

Copy and paste the code below :

#!/bin/sh
#
# nginx - this script starts and stops the nginx daemin
#
# chkconfig:   - 85 15 
# description:  Nginx is an HTTP(S) server, HTTP(S) reverse \
#               proxy and IMAP/POP3 proxy server
# processname: nginx
# config:      /usr/local/nginx/conf/nginx.conf
# pidfile:     /usr/local/nginx/logs/nginx.pid

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0

nginx="/usr/local/nginx/sbin/nginx"
prog=$(basename $nginx)

NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"

lockfile=/var/lock/subsys/nginx

start() {
    [ -x $nginx ] || exit 5
    [ -f $NGINX_CONF_FILE ] || exit 6
    echo -n $"Starting $prog: "
    daemon $nginx -c $NGINX_CONF_FILE
    retval=$?
    echo
    [ $retval -eq 0 ] && touch $lockfile
    return $retval
}

stop() {
    echo -n $"Stopping $prog: "
    killproc $prog -QUIT
    retval=$?
    echo
    [ $retval -eq 0 ] && rm -f $lockfile
    return $retval
}

restart() {
    configtest || return $?
    stop
    start
}

reload() {
    configtest || return $?
    echo -n $"Reloading $prog: "
    killproc $nginx -HUP
    RETVAL=$?
    echo
}

force_reload() {
    restart
}

configtest() {
  $nginx -t -c $NGINX_CONF_FILE
}

rh_status() {
    status $prog
}

rh_status_q() {
    rh_status >/dev/null 2>&1
}

case "$1" in
    start)
        rh_status_q && exit 0
        $1
        ;;
    stop)
        rh_status_q || exit 0
        $1
        ;;
    restart|configtest)
        $1
        ;;
    reload)
        rh_status_q || exit 7
        $1
        ;;
    force-reload)
        force_reload
        ;;
    status)
        rh_status
        ;;
    condrestart|try-restart)
        rh_status_q || exit 0
            ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
        exit 2
esac

As the init file is a shell script, it needs to have executable permissions.

chmod +x /etc/init.d/nginx

Now you can start, stop , reload Nginx through command below :

service nginx start
service nginx stop
service nginx restart
service nginx reload

Now we have the base script prepared, we need to add it to the default run levels:

/sbin/chkconfig nginx on

4. Once you're done, try to run the website with port 81. Type in your regular URL but end with :81 Such as ..

http://www.domainname.com:81/test/information

5. If your website loaded as usual, then, it's time to redirect all the traffic from port 80 to port 81. Just copy and paste the code below :

iptables -t nat -A PREROUTING -p tcp -s ! xxx.xxx.xxx.xxx --dport 80 -j REDIRECT --to-ports 81

Now, access your website normally, no need to add port 81.

But if your website doesn't work as it should be. You may need to run command below to disable port forwarding so that you can check your configuration again.

iptables -t nat -D PREROUTING -p tcp -s ! xxx.xxx.xxx.xxx --dport 80 -j REDIRECT --to-ports 81

Actually, I also try to serve cache static file using code below :

     location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf)$ {
         root   /home/admin/public_html;
         expires 30m;
        }

I don't know if I really need that code to make Nginx works properly or not. Still, I tried both configurations. Now.... how about load?? performance??? Is it worth to do it? ... Since I don't have any program to test it. I just base on my feeling alone. I feel that with Nginx as a reverse proxy doesn't really help much compare to Apache Worker MPM. The server load is quite the same. The only thing I found it's better is memory consumption. (~ 18 M for NginX vs ~ 200 M for Apache)

Note :
1. About nginx.pid, I found that when restart service, sometime it work, sometime it doesn't. So, check it carefully.
2. I also tested Reverse proxy cache and it seems to work pretty good. However, most of my websites are Drupal and with Reverse proxy cache, it doesn't play well together. When I update content with image, I usually end up with Repeating loop. This will only happen once I do the port forwarding. But if I work on the site with port 81, it works correctly.

Tested with below software :
1. CentOS 5.4 / 5.5 - 64 bits
2. DirectAdmin 1.37 - With Custombuild 1.2
3. Apache 2.2.17 - Worker MPM
4. Nginx 0.8.5
5. PHP 5.2.17 / 5.3.5
6. PHP-FPM in PHP 5.3.5
7. APC 3.1.7

Source :
- http://www.directadmin.com/forum/showthread.php?t=27344
- http://articles.slicehost.com/2009/2/2/centos-adding-an-nginx-init-script

 

Updated : 2011.06.04

When compile Nginx to use as HTTP server and has a reverse proxy in front of the Nginx, you may need "http_realip_module". (Comparable to mod_rpaf in Apache) So, compile with below command.

wget http://nginx.org/download/nginx-1.0.4.tar.gz
tar xvfz nginx-1.0.4.tar.gz
cd nginx-1.0.4
./configure --with-http_stub_status_module --with-http_gzip_static_module --with-http_realip_module
make
make install

 

Updated : 2012.04.24

Tested Nginx 1.2.0 and it seems to work fine. However, if you use limit_zone directive. The warning will show as below (Deprecated since version 1.1.8)

nginx: [warn] the "limit_zone" directive is deprecated, use the "limit_conn_zone" directive instead in /usr/local/nginx/conf/nginx.conf:37
nginx: [warn] the "limit_zone" directive is deprecated, use the "limit_conn_zone" directive instead in /usr/local/nginx/conf/nginx.conf:40

It still works. But you should change the config from

limit_zone name $variable size;

To

limit_conn_zone $variable zone=name:size; 

Comments

Did you tested this method with with ngnix 1.* ?

Actually, I have upgraded two of my servers from 0.8.5 to 1.0.4. (One WebServer, One Reverse Proxy Server) Both of them have DirectAdmin Control Panel installed. Both of them work fine. No need to modify any config to make it work at all. However, the one that I used as a reverse proxy. I just have it there, test it a bit but not really put it into work. Seem to work ok without any problem though.

hi BXTra .. does your config honor the logstatistics of AWStat/webalizer as well ? or are any cached hits excluded from apache-logging ?

Use it for a couple weeks now. However, I don't really know if it honors log statistic or not. I usually use Google Analytic to reduce load on my server and also to make sure I got the right statistic though. Please try it and let me know, with that setting, you can config NginX as a web server for specific site, not just reverse proxy. So, it's fun to try. :)

Hello, 3 questions please suggest 1. My DA server build php with cli mod and I don't want' to convert it to cgi mod. So can I apply follow this tutorial with no problem ? 2. Most of websites on my server is Joomla, wordpress, SMF etc. my question is if I apply this tutorial to my server what's I need to beware or what's problems which may happen ? 3. How to upgrade nginx ? Can I re "make install" without uninstall exciting nginx version ? Regards, Umpol

1. I think this will work with PHP CLI without any problem. However, you have to try it though. Let me know if it works or not :) 2. From what I tested on mostly Drupal, it works without having to do anything. You may try to play with Nginx config per domain. (VHost) Something like more caching or you can even use Nginx to serve specific domain without pass it through Apache at all. 3. No need to uninstall it. You can just following the same steps to install Nginx. Then, restart it after everything is done. So, there is no downtime on that at all.

Decrease memory in use: _Find: keepalive_timeout 30; _Change to: keepalive_timeout 2; => With 2 second, user enough to get nomal Reduce CPU usage: _Find: gzip_comp_level 9; _Change to: gzip_comp_level 5; =>Changing compress level can get performane CPU better than hight level. From 3 to 5 is good than 9

How about SSL? I try to get reverse nginx proxy working with SSL, still no luck. Maybe someone can help me out:)