Working with some clients lately to speed up their drupal webiste from 100 concurrent users to 10k on a single server.  Drupal as we all know is super bloated, as you add features, modules it gets bloated.

“Disclaimer: I am no Drupal fan nor PHP, arrrrrgh i say to both of them”

The setup mentioned below is really dependent on your needs, you might need to employ all the techniques or some, that is something you have to figure out depending on your daily requests.  We will be starting with the basics of what type of services we will need to get up and running.  For our setup we are using a single server, some of you might be jumping up and down, if you like/need to have services running on multiple servers feel free to do so.

List of components:

  • Webserver: Nginx
  • Database: PostgreSQL (you can use mysql as well, I will post a link for mysql tuning later).
  • PHP: PHP-FPM
  • PHP packages: pgsql/mysql, memcached, apc
  • Memcached

That is it for our single server setup, if your needs are bigger you might look at multiple webserver/database server with load balancers in front of them.  You can even make use of caching techniques with varnish or other servers.

To help with this there are several things you can do.  We will be discussing the most basic at this part , hardware and server setups will be for another day.

 

 

 

Drupal Setup

 

1.  Turn off ALL themes except the one you are using, most people forget the fact they have several themes enabled which hogs up memory.

2.  Next on the list is modules, you need to disable/remove any module that you dont use or dont need.  This also goes into #3 but we’ll list some of them, turn off database logging, statistics and enable syslog.

3. Log everything to syslog via drupal.

4. If you have any devel modules turn them off as well.

5. Install memcached module & enable it in drupal module listing.

6. Add the following to drupal settings.php

/*
* Memcache:
*/
$conf = array(
‘cache_inc’ => ‘./modules/memcache/memcache.inc’,
‘memcache_servers’ => array(
‘127.0.0.1:11211’ => ‘page’,
‘127.0.0.1:11212’ => ‘filter’,
‘127.0.0.1:11213’ => ‘default’,
),
‘memcache_bins’ => array(
‘cache_page’ => ‘page’,
‘cache_filter’ => ‘filter’,
‘cache’
=> ‘default’,
‘cache_menu’ => ‘default’,
‘cache_block’ => ‘default’,
‘cache_form’ => ‘default’,
‘cache_content’ => ‘default’,
‘cache_update’ => ‘default’,
‘cache_views’ => ‘default’,
),);

This will limit cache to memcached instead of the db, a lot faster and cheaper.

 

 

 

PHP Setup

 

 

2. Install apc via pecl “pecl install apc”

Modify apc.ini and add the following:

extension = apc.so
apc.enabled=1
apc.shm_size=64M
apc.shm_segments=1
apc.ttl=0
apc.user_ttl=0
apc.cache_by_default=1

 

 

 

PostgresSQL Settings

This really depends you might have other needs, the settings below are for vanilla postgresql installs, so if you have other apps I would suggest to test prior to any changes.  Also as I mentioned before you need to test what works for you, after every test run a load test to figure out if it was beneficial or not.  For postgresql changes make sure you have the necessary resources to allocate as well, you can always go in analyze the queries coming across which I will save for another post.

1. Modify postgresql.conf

a. max_connections = 500
b. shared_buffers = 512MB
c. temp_buffers = 128MB
d. work_mem = 64MB
e. maintenance_work_mem = 16MB
f. max_stack_depth = 2MB

 

 

 

 

 

Memcached Setup

1. Install memcached via your pkg manager, yum, apt, or slackware sbo =), freebsd port , gentoo emerge, etc etc.

a. pkg_manager install memcached.

b. pecl install memcache

c. add the following to your php.ini :

extension=memcache.so

[memcache]
; Data will be transferred in chunks of this size
memcache.chunk_size = 32768
memcache.hash_strategy = consistent
memcache.default_port = 0
session.save_handler = memcache
session.save_path = “unix:///tmp/memcached.socket11:0?
persistent=1&weight=1&timeout=1&retry_interval=15″

2. Run memcached instances , you can put these in a script (if you require additional memory remember to test and change accordingly).

/usr/bin/memcached -d -u root -m 512 -p 11211
/usr/bin/memcached -d -u root -m 1024 -p 11212
/usr/bin/memcached -d -u root -m 512 -p 11213

 

 

 

 Nginx & PHP-FPM Configurations

 

1. Edit nginx conf:

user
nginx;
worker_processes 8;  <—-(number of cores, not number of magical elves , i’ve seen 5000 plus in some settings, anybody bother reading these days, rant over.)
error_log /var/log/nginx/error.log crit ;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
pid
/var/run/nginx.pid;
worker_rlimit_nofile 8192;
events {
worker_connections 2048;
}
http {
include
/etc/nginx/mime.types;
default_type application/octet-stream;
log_format main ‘$remote_addr – $remote_user [$time_local] “$request” ‘
‘$status $body_bytes_sent “$http_referer” ‘
‘”$http_user_agent” “$http_x_forwarded_for”‘;
access_log /var/log/nginx/access.log main;
sendfile
on;
#tcp_nopush on;
gzip on;
gzip_comp_level 1;
gzip_proxied any;
gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss
text/javascript;
client_body_buffer_size
1k;
client_header_buffer_size
1k;
client_max_body_size
10m;
large_client_header_buffers 3 3k;
connection_pool_size
256;
request_pool_size
4k;
server_names_hash_bucket_size 128;
client_body_timeout 60;
client_header_timeout 60;
keepalive_timeout 10 10;
send_timeout
60;

# Load config files from the /etc/nginx/conf.d directory
include /etc/nginx/conf.d/*.conf;
#
# The default server
#
}

 

 

2. Modify domain settings and this to the conf, fastcig settings below:

 

 

fastcgi_connect_timeout 60;
fastcgi_send_timeout 120;
fastcgi_read_timeout 120;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;
fastcgi_split_path_info ^(.+\.php)(.*)$;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_ignore_client_abort on;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;

 

 

 

3. Modify /etc/php-fpm.d/www.conf

pm.max_children = 500
pm.start_servers = 30
pm.min_spare_servers = 5
pm.max_spare_servers = 35

 

 

And that should be it, remember install all the components first and change one thing at a time, test and write down your results and so on.  I use jmeter, its a nice little app to do testing and comes for free, here is the link: http://jakarta.apache.org/jmeter/

 

If you are confused, wondering about something feel free to message me.