Working with some clients lately to speed up their drupal webiste from 100 concurrent users to 10k on a single server. Drupal as we all know is super bloated, as you add features, modules it gets bloated.
“Disclaimer: I am no Drupal fan nor PHP, arrrrrgh i say to both of them”
The setup mentioned below is really dependent on your needs, you might need to employ all the techniques or some, that is something you have to figure out depending on your daily requests. We will be starting with the basics of what type of services we will need to get up and running. For our setup we are using a single server, some of you might be jumping up and down, if you like/need to have services running on multiple servers feel free to do so.
List of components:
- Webserver: Nginx
- Database: PostgreSQL (you can use mysql as well, I will post a link for mysql tuning later).
- PHP: PHP-FPM
- PHP packages: pgsql/mysql, memcached, apc
That is it for our single server setup, if your needs are bigger you might look at multiple webserver/database server with load balancers in front of them. You can even make use of caching techniques with varnish or other servers.
To help with this there are several things you can do. We will be discussing the most basic at this part , hardware and server setups will be for another day.
1. Turn off ALL themes except the one you are using, most people forget the fact they have several themes enabled which hogs up memory.
2. Next on the list is modules, you need to disable/remove any module that you dont use or dont need. This also goes into #3 but we’ll list some of them, turn off database logging, statistics and enable syslog.
3. Log everything to syslog via drupal.
4. If you have any devel modules turn them off as well.
5. Install memcached module & enable it in drupal module listing.
6. Add the following to drupal settings.php
$conf = array(
‘cache_inc’ => ‘./modules/memcache/memcache.inc’,
‘memcache_servers’ => array(
‘127.0.0.1:11211’ => ‘page’,
‘127.0.0.1:11212’ => ‘filter’,
‘127.0.0.1:11213’ => ‘default’,
‘memcache_bins’ => array(
‘cache_page’ => ‘page’,
‘cache_filter’ => ‘filter’,
‘cache_menu’ => ‘default’,
‘cache_block’ => ‘default’,
‘cache_form’ => ‘default’,
‘cache_content’ => ‘default’,
‘cache_update’ => ‘default’,
‘cache_views’ => ‘default’,
This will limit cache to memcached instead of the db, a lot faster and cheaper.
2. Install apc via pecl “pecl install apc”
Modify apc.ini and add the following:
extension = apc.so
This really depends you might have other needs, the settings below are for vanilla postgresql installs, so if you have other apps I would suggest to test prior to any changes. Also as I mentioned before you need to test what works for you, after every test run a load test to figure out if it was beneficial or not. For postgresql changes make sure you have the necessary resources to allocate as well, you can always go in analyze the queries coming across which I will save for another post.
1. Modify postgresql.conf
a. max_connections = 500
b. shared_buffers = 512MB
c. temp_buffers = 128MB
d. work_mem = 64MB
e. maintenance_work_mem = 16MB
f. max_stack_depth = 2MB
1. Install memcached via your pkg manager, yum, apt, or slackware sbo =), freebsd port , gentoo emerge, etc etc.
a. pkg_manager install memcached.
b. pecl install memcache
c. add the following to your php.ini :
; Data will be transferred in chunks of this size
memcache.chunk_size = 32768
memcache.hash_strategy = consistent
memcache.default_port = 0
session.save_handler = memcache
session.save_path = “unix:///tmp/memcached.socket11:0?
2. Run memcached instances , you can put these in a script (if you require additional memory remember to test and change accordingly).
/usr/bin/memcached -d -u root -m 512 -p 11211
/usr/bin/memcached -d -u root -m 1024 -p 11212
/usr/bin/memcached -d -u root -m 512 -p 11213
Nginx & PHP-FPM Configurations
1. Edit nginx conf:
worker_processes 8; <—-(number of cores, not number of magical elves , i’ve seen 5000 plus in some settings, anybody bother reading these days, rant over.)
error_log /var/log/nginx/error.log crit ;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
log_format main ‘$remote_addr – $remote_user [$time_local] “$request” ‘
‘$status $body_bytes_sent “$http_referer” ‘
access_log /var/log/nginx/access.log main;
large_client_header_buffers 3 3k;
keepalive_timeout 10 10;
# Load config files from the /etc/nginx/conf.d directory
# The default server
2. Modify domain settings and this to the conf, fastcig settings below:
fastcgi_buffers 4 256k;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
3. Modify /etc/php-fpm.d/www.conf
pm.max_children = 500
pm.start_servers = 30
pm.min_spare_servers = 5
pm.max_spare_servers = 35
And that should be it, remember install all the components first and change one thing at a time, test and write down your results and so on. I use jmeter, its a nice little app to do testing and comes for free, here is the link: http://jakarta.apache.org/jmeter/
If you are confused, wondering about something feel free to message me.