Sunday, February 10, 2013

PHP caching: shm vs. apc vs. memcache vs. mysql vs. file cache (update: fill apc from cron)

Lessons learned:
  • shm/apc are 32-60 times faster than memcached or mysql
  • shm/apc are 2 times faster than php file cache with apc
  • php file cache with apc is 15-24 times faster than memcached or mysql
  • mysql is 2 times faster than memcached when storing more than 400 bytes
  • memcached is 2 times faster than mysql when storing less than 400 bytes
  • php file cache with apc is 2-3 times faster than normal file cache
  • php file cache without apc is 8 times slower than normal file cache

Tests were made with PHP 5.3.10, MySQL 5.5.29, memcached 1.4.13, 64bit, 3.4GHz (QEMU):

shm
0.031 0.020 0.021 0.021 0.026 0.028 0.032 0.042 0.084 0.155 0.290 0.629 0.110
Total: 1.489, Avg: 0.115

apc
0.025 0.025 0.025 0.026 0.031 0.036 0.043 0.060 0.106 0.171 0.328 0.756 0.097
Total: 1.728, Avg: 0.133

memcache
3.116 3.014 3.005 3.072 3.077 3.910 3.929 4.067 4.308 10.371 15.323 25.013 3.281
Total: 85.488, Avg: 6.576

memcache socket
1.736 1.756 1.981 1.780 1.809 1.907 1.941 1.983 2.225 9.368 14.071 24.897 1.979
Total: 67.435, Avg: 5.187

memcached
2.241 2.540 2.713 2.769 2.897 3.249 4.286 5.298 7.729 10.539 16.021 28.060 2.578
Total: 90.919, Avg: 6.994

mysql myisam
3.267 3.291 3.310 3.295 3.700 3.777 3.888 4.078 4.368 6.272 6.930 9.626 3.726
Total: 59.529, Avg: 4.579

mysql memory
3.238 3.360 3.470 3.502 3.310 3.346 3.681 4.108 4.370 6.286 7.279 7.079 3.397
Total: 56.426, Avg: 4.340

file cache
0.593 0.595 0.593 0.609 0.546 0.563 0.574 0.600 0.648 0.800 1.115 1.956 0.775
Total: 9.966, Avg: 0.767

php file cache
0.177 0.176 0.175 0.180 0.187 0.188 0.195 0.210 0.236 0.318 0.479 0.901 0.228
Total: 3.650, Avg: 0.281

(10b  0.1k  0.3k  0.5k  1k    2k    4k    8k    16k   32k   64k   128k  array)
Notes: Numbers in seconds, smaller numbers are better, connection times for memcached and mysql are not counted, file cache was done on tmpfs.

Cold cache, fill apc cache from cron:

All entries in apc cache are kept in shared memory. Shared memory is available to the web server and is not shared with the command line php (php-cli). When the web server is restarted, the cache is empty and refilling the complete cache can create performance problems. To fill the apc cache with a cron job, we use a second script to dump the cache to a binary file and load it inside the web server:

// php cron.php
$data = array("cached"=>1, "hello"=>"world");
apc_store($data);
apc_bin_dumpfile(array(), array_keys($data), "/var/cache/apc_cron.bin");
// bootstrap.php, http://myserver.com/...
if (apc_fetch("cached")===false) apc_bin_loadfile("/var/cache/apc_cron.bin");

echo apc_fetch("hello"); // gives "world"

Related articles:

Here is the test script:

<?php
// memcached -m 64 -s /tmp/m.sock -a 0777 -p 0 -u memcache
// memcached -m 64 -l 127.0.0.1 -p 11211 -u memcache
set_time_limit(3600);
error_reporting(E_ALL);
ini_set("display_errors", 1);
ini_set("apc.enable_cli", 1);
mysqli_report(MYSQLI_REPORT_STRICT | MYSQLI_REPORT_ERROR);

$data = array( // test are range from 10-128,000 bytes
  "1111111110",
  str_repeat("1111111110", 10),
  str_repeat("1111111110", 30),
  str_repeat("1111111110", 50),
  str_repeat("1111111110", 100),
  str_repeat("1111111110", 200),
  str_repeat("1111111110", 400),
  str_repeat("1111111110", 800),
  str_repeat("1111111110", 1600),
  str_repeat("1111111110", 3200),
  str_repeat("1111111110", 6400),
  str_repeat("1111111110", 12800),
  array(0=>"1111111110", "id2"=>"hello world", "id3"=>"foo bar", "id4"=>42)
);

echo "shm\n";
$t = array();
foreach ($data as $key=>$val) $t[] = shm(4000+$key, $val);
echo stats($t);

echo "apc\n";
$t = array();
foreach ($data as $key=>$val) $t[] = apc((string)$key, $val);
echo stats($t);

echo "memcache\n";
$t = array();
$m = memcache_connect("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcache socket\n";
$t = array();
$m = memcache_connect("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcached\n";
$t = array();
$m = new Memcached();
$m->addServer("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);

/* not in memcached 1.x
echo "memcached socket\n";
$t = array();
$m = new Memcached();
$m->addServer("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);
*/

echo "mysql myisam\n";
$t = array();
$m = new mysqli("127.0.0.1", "root", "", "t1");
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data mediumtext) engine=myisam");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "mysql memory\n";
$t = array();
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data varchar(65500)) engine=memory");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = file_cache((string)$key, $val);
echo stats($t);

echo "php file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = php_cache((string)$key, $val);
echo stats($t);

function stats($t) {
  return "\nTotal: ".number_format(array_sum($t), 3).", ".
    "Avg: ".number_format(array_sum($t) / count($t), 3)."\n\n";
}

function format($num) {
  return number_format($num, 3);
}

function shm($id, $data) {
  if (is_array($data)) {
    $arr = true;
    $data = serialize($data);
  } else $arr = false;
  $len = strlen($data);
  $shm_id = shmop_open($id, "c", 0644, $len);
  shmop_write($shm_id, $data, 0);
  $start = microtime(true);
  if ($arr) {
    for ($i=0; $i<100000; $i++) $v = unserialize(shmop_read($shm_id, 0, $len));
  } else {
    for ($i=0; $i<100000; $i++) $v = shmop_read($shm_id, 0, $len);
  }
  echo format($end = microtime(true)-$start)." ";
  shmop_close($shm_id);
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function apc($id, $data) {
  apc_store($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = apc_fetch($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcache($m, $id, $data) {
  memcache_set($m, $id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = memcache_get($m, $id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcached($m, $id, $data) {
  $m->set($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = $m->get($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function mysql_cache($m, $id, $data) {
  $d = is_array($data) ? serialize($data) : $data;
  mysqli_query($m, "insert into t1.cache values (".$id.", '".$d."')");
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
      $v = unserialize($v[0]);
    }
  } else {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
    }
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr($v[0], 0, 10)=="1111111110");
  return $end;
}

function file_cache($id, $data) {
  file_put_contents($id, is_array($data) ? serialize($data) : $data);
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) $v = unserialize(file_get_contents($id));
  } else {
    for ($i=0; $i<100000; $i++) $v = file_get_contents($id);
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function php_cache($id, $data) {
  $id .= ".php";
  $data = is_array($data) ? var_export($data, 1) : "'".$data."'";
  file_put_contents($id, "<?php\n\$v=".$data.";");
  touch($id, time()-10); // needed for APC's file update protection
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) include($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

Labels

performance (23) benchmark (6) MySQL (5) architecture (5) coding style (5) memory usage (5) HHVM (4) C++ (3) Java (3) Javascript (3) MVC (3) SQL (3) abstraction layer (3) framework (3) maintenance (3) Go (2) Golang (2) HTML5 (2) ORM (2) PDF (2) Slim (2) Symfony (2) Zend Framework (2) Zephir (2) firewall (2) log files (2) loops (2) quality (2) real-time (2) scrum (2) streaming (2) AOP (1) Apache (1) Arrays (1) C (1) DDoS (1) Deployment (1) DoS (1) Dropbox (1) HTML to PDF (1) HipHop (1) OCR (1) OOP (1) Objects (1) PDO (1) PHP extension (1) PhantomJS (1) SPL (1) SQLite (1) Server-Sent Events (1) Silex (1) Smarty (1) SplFixedArray (1) Unicode (1) V8 (1) analytics (1) annotations (1) apc (1) archiving (1) autoloading (1) awk (1) caching (1) code quality (1) column store (1) common mistakes (1) configuration (1) controller (1) decisions (1) design patterns (1) disk space (1) dynamic routing (1) file cache (1) garbage collector (1) good developer (1) html2pdf (1) internationalization (1) invoice (1) just-in-time compiler (1) kiss (1) knockd (1) legacy code (1) legacy systems (1) logtop (1) memcache (1) memcached (1) micro framework (1) ncat (1) node.js (1) openssh (1) pfff (1) php7 (1) phpng (1) procedure models (1) ramdisk (1) recursion (1) refactoring (1) references (1) regular expressions (1) search (1) security (1) sgrep (1) shm (1) sorting (1) spatch (1) ssh (1) strange behavior (1) swig (1) template engine (1) threads (1) translation (1) ubuntu (1) ufw (1) web server (1) whois (1)