php

Saturday, October 26, 2013

View caching with Memcache and CakePHP

8:57 PM Posted by Unknown No comments
Soon after we launched TotalFilm.com we had a problem. We had far more visitors than we were expecting! It's a nice problem to have but our server was working too hard. It just couldn't cope with PHP rendering the pages using CakePHP's built in view caching. We needed to come up with a solution quickly.
We benchmarked CakePHP's view caching. It didn't seem especially slow considering the functionality it provides inside cached views. We thought that we needed certain dynamic aspects (e.g. user login for commenting) but our pages simply didn't need to be custom built for each user. We could use a cookie and a bit of Javascript to display when a user was logged in. We only needed to check if users were logged in when they actually did something. We could cache most pages in their entirety and didn't need to use PHP but unfortunately CakePHP's view caching required PHP.
Your website could well need that level of dynamism or you might not want to implement some functionality solely in Javascript. That's fine but don't press your back button in disgust just yet! You can probably use the solution described below as an emergency level of caching reserved for when you experience extreme peak loads.

What we did

We decided to use a Memcache to create our quick fix caching engine. We used the URL as the cache key as this would enable us to use the NginxHttpMemcachedModule to serve cached pages directly from Memcache completely bypassing PHP.
The following instructions assume you have already installed memcached and the PHP Memcache extension.
The files described below are also available in this tgz file.

Step One - Use custom Memcache cache engine

Unfortunately CakePHP's built in cache engine for Memcache translates slashes into underscores - presumably as they are not allowed in filenames. That was bad news as URLs have slashes in them! We also need to be able to specify different cache timeouts for different types of page. An article page which only needs to be updated in the cache when that article changes can have a very long timeout but the homepage will change frequently and requires a much shorter timeout. We therefore needed a custom Memcache cache engine. This needs to go in cake/libs/cache/viewmemcache.php.
001/**
002 * ViewMemcache storage engine for cache
003 *
004 * This is a very cut down rewrite of the memcache cache engine that comes with
005 * CakePHP. This cache engine only does the bare minimum to support chucking
006 * views into memcache.
007 */
008 
009/**
010 * ViewMemcache storage engine for cache
011 *
012 * @package       cake
013 * @subpackage    cake.cake.libs.cache
014 */
015 
016class ViewMemcacheEngine extends CacheEngine {
017 
018/**
019 * Memcache wrapper.
020 *
021 * @var Memcache
022 * @access private
023 */
024 
025  var $__Memcache = null;
026 
027/**
028 * Settings
029 *
030 *  - servers = string or array of memcache servers, default => 127.0.0.1. If an
031 *    array MemcacheEngine will use them as a pool.
032 *  - compress = boolean, default => false
033 *
034 * @var array
035 * @access public
036 */
037 
038  var $settings = array();
039 
040/**
041 * Initialize the Cache Engine
042 *
043 * Called automatically by the cache frontend
044 * To reinitialize the settings call Cache::engine('EngineName', [optional] settings = array());
045 *
046 * @param array $setting array of setting for the engine
047 * @return boolean True if the engine has been successfully initialized, false if not
048 * @access public
049 */
050 
051  function init($settings = array()) {
052    if (!class_exists('Memcache')) {
053      return false;
054    }
055    parent::init(array_merge(array(
056      'engine'=> 'ViewMemcache',
057      'prefix' => Inflector::slug(APP_DIR) . '_',
058      'servers' => array('127.0.0.1'),
059      'compress'=> false
060      ), $settings)
061    );
062 
063    if ($this->settings['compress']) {
064      $this->settings['compress'] = MEMCACHE_COMPRESSED;
065    }
066    if (!is_array($this->settings['servers'])) {
067      $this->settings['servers'] = array($this->settings['servers']);
068    }
069    if (!isset($this->__Memcache)) {
070      $return = false;
071      $this->__Memcache =& new Memcache();
072      foreach ($this->settings['servers'] as $server) {
073        $parts = explode(':', $server);
074        $host = $parts[0];
075        $port = 11211;
076        if (isset($parts[1])) {
077          $port = $parts[1];
078        }
079        if ($this->__Memcache->addServer($host, $port)) {
080          $return = true;
081        }
082      }
083      return $return;
084    }
085    return true;
086  }
087 
088/**
089 * Write data for key into cache
090 *
091 * @param string $key Identifier for the data
092 * @param mixed $value Data to be cached
093 * @param integer $duration How long to cache the data, in seconds
094 * @return boolean True if the data was succesfully cached, false on failure
095 * @access public
096 */
097 
098  function write($key, &$value, $duration) {
099    $expires = time() + $value['timeout'];
100    return $this->__Memcache->set($key, $value['data'], $this->settings['compress'], $expires);
101  }
102 
103 
104/**
105 * Read a key from the cache
106 *
107 * @param string $key Identifier for the data
108 * @return mixed The cached data, or false if the data doesn't exist, has expired, or if there was an error fetching it
109 * @access public
110 */
111 
112  function read($key) {
113    return $this->__Memcache->get($key);
114  }
115 
116 
117/**
118 * Delete a key from the cache
119 *
120 * @param string $key Identifier for the data
121 * @return boolean True if the value was succesfully deleted, false if it didn't exist or couldn't be removed
122 * @access public
123 */
124 
125  function delete($key) {
126    return $this->__Memcache->delete($key);
127  }
128 
129/**
130 * Delete all keys from the cache
131 *
132 * @return boolean True if the cache was succesfully cleared, false otherwise
133 * @access public
134 */
135  function clear() {
136    return $this->__Memcache->flush();
137  }
138 
139/**
140 * Connects to a server in connection pool
141 *
142 * @param string $host host ip address or name
143 * @param integer $port Server port
144 * @return boolean True if memcache server was connected
145 * @access public
146 */
147  function connect($host, $port = 11211) {
148    if ($this->__Memcache->getServerStatus($host, $port) === 0) {
149      if ($this->__Memcache->connect($host, $port)) {
150        return true;
151      }
152      return false;
153    }
154    return true;
155  }
156 
157/**
158 * Use the key requested rather than doing any conversion on it
159 *
160 * @param string $key key
161 * @access public
162 */
163 
164  function key($key) {
165    return $key;
166  }
167}

Step Two - Create an extra cache configuration in app/config/core.php

You will obviously need to change configuration parameters if your Memcache server is not running on the default port on localhost.
01Cache::config('view', array(
02  'engine' => 'ViewMemcache',
03  'duration'=> 3600,
04    'prefix' => '',
05    'servers' => array(
06      '127.0.0.1:11211'
07    ),
08    'compress' => false,
09));
10 
11Configure::write('ViewMemcache.timeout', 3600);

Step Three - Use our Memcache helper

The Memcache helper dumps rendered views into Memcache for us. This goes in app/views/helpers/memcache.php.
01class MemcacheHelper extends Helper
02{
03  function afterLayout()
04  {
05    $view = & ClassRegistry::getObject('view');
06 
07    if (is_object($view) && array_key_exists('docache', $view->viewVars) && $view->viewVars['docache'] === true) {
08      $timeout = Configure::read('ViewMemcache.timeout');
09 
10      if (array_key_exists('docachetimeout', $view->viewVars)) {
11        $timeout = $view->viewVars['docachetimeout'];
12      }
13 
14      if (!array_key_exists('nocachefooter', $view->viewVars)) {
15        $view->output .= "\n<!-- galeCached " . date('r') . ' -->';
16      }
17 
18      Cache::write($view->here, array( 'data' => $view->output, 'timeout' => $timeout), 'view');
19    }
20 
21    return true;
22  }
23}

Step Four - Get Nginx to check Memcache

If you aren't using Nginx you probably should be. Nginx greatly outperforms Apache HTTP Server, maybe even when serving PHP these days. I won't cover how to setup Nginx - as there are so many different configurations you can use and it's different on each platform - but the extra configuration you need is as simple as:
server {

  listen      80;
  server_name superfastwebsite;
  access_log  /opt/local/var/log/nginx/access.log;
  error_log   /opt/local/var/log/nginx/error.log;

  rewrite_log on;

  # check memcache for page existence using uri as key
  location /checkmemcache {
    internal;
    set $memcached_key $request_uri;
    memcached_connect_timeout 2000;
    memcached_read_timeout 2000;
    memcached_pass     127.0.0.1:11212;
    default_type     text/html;
    # if memcache throws one of the following errors head off to the php-cgi
    error_page 404 502  = /edpagegeneration;
  }

  # Pass the PHP scripts to FastCGI server
  # listening on 127.0.0.1:9000
  location /edpagegeneration {
    internal;
    root /data/superfastwebsite/trunk/webroot;
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_intercept_errors on; # to support 404s for PHP files not found
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
  }

  location / {
    root /data/superfastwebsite/trunk/webroot;
    index  index.php index.html;

    # all non-existing file or directory requests to index.php
    if (!-e $request_filename) {
      # redirect directly to the main php processing
      # rewrite  ^(.*)$  /index.php  last;

      # redirect to memcache page checking first which will
      # then fall through to the php-cgi 

      rewrite  ^(.*)$  /checkmemcache last;
    }
  }

  # Not found this on disk? 
  # Feed to CakePHP for further processing!
  if (!-e $request_filename) {
    rewrite ^/(.+)$ /index.php?url=$1 last;
    break;
  }

  # Pass the PHP scripts to FastCGI server
  # listening on 127.0.0.1:9000
  location ~ \.php$ {
    root /data/superfastwebsite/trunk/webroot;
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_intercept_errors on; # to support 404s for PHP files not found
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include     fastcgi_params;
  }

  # Static files.
  # Set expire headers, Turn off access log
  location ~* \favicon.ico$ {
      access_log off;
      expires 1d;
      add_header Cache-Control public;
  }

  location ~ ^/(img|js|css)/ {
      access_log off;
      expires 7d;
      add_header Cache-Control public;
  }

  # Deny access to .htaccess files,
  # git & svn repositories, etc
  location ~ /(\.ht|\.git|\.svn) {
      deny  all;
  }
}

Step Five - Get caching

Add our Memcache helper into your controller helper array:
1var $helpers = array('Memcache');
To make a page cache, add the following to the controller method:
1$this->set('docache', true);
To change the timeout from the default (in seconds):
1$this->set('docachetimeout', 3432434);
To stop the cache comment appearing (e.g. for a JSON view):
1$this->set('nocachefooter', true);

Step Six - Clear page from cache

To remove something from the cache:
1Cache::delete($key, 'view');
For now, to clear the cache simply delete the relevant cache keys - i.e. the urls - when things are updated. I will discuss cache clearing and update strategies in a future article or come and see my talk at CakeFest 2010!
Thanks to Jon Bennett for going through this tutorial and spotting a few bugs and thanks for Ed Kennedy at Future Publishing for the Nginx configuration.

0 comments:

Post a Comment