Sunday, February 10, 2013

PHP caching: shm vs. apc vs. memcache vs. mysql vs. file cache (update: fill apc from cron)

Lessons learned:
  • shm/apc are 32-60 times faster than memcached or mysql
  • shm/apc are 2 times faster than php file cache with apc
  • php file cache with apc is 15-24 times faster than memcached or mysql
  • mysql is 2 times faster than memcached when storing more than 400 bytes
  • memcached is 2 times faster than mysql when storing less than 400 bytes
  • php file cache with apc is 2-3 times faster than normal file cache
  • php file cache without apc is 8 times slower than normal file cache

Tests were made with PHP 5.3.10, MySQL 5.5.29, memcached 1.4.13, 64bit, 3.4GHz (QEMU):

shm
0.031 0.020 0.021 0.021 0.026 0.028 0.032 0.042 0.084 0.155 0.290 0.629 0.110
Total: 1.489, Avg: 0.115

apc
0.025 0.025 0.025 0.026 0.031 0.036 0.043 0.060 0.106 0.171 0.328 0.756 0.097
Total: 1.728, Avg: 0.133

memcache
3.116 3.014 3.005 3.072 3.077 3.910 3.929 4.067 4.308 10.371 15.323 25.013 3.281
Total: 85.488, Avg: 6.576

memcache socket
1.736 1.756 1.981 1.780 1.809 1.907 1.941 1.983 2.225 9.368 14.071 24.897 1.979
Total: 67.435, Avg: 5.187

memcached
2.241 2.540 2.713 2.769 2.897 3.249 4.286 5.298 7.729 10.539 16.021 28.060 2.578
Total: 90.919, Avg: 6.994

mysql myisam
3.267 3.291 3.310 3.295 3.700 3.777 3.888 4.078 4.368 6.272 6.930 9.626 3.726
Total: 59.529, Avg: 4.579

mysql memory
3.238 3.360 3.470 3.502 3.310 3.346 3.681 4.108 4.370 6.286 7.279 7.079 3.397
Total: 56.426, Avg: 4.340

file cache
0.593 0.595 0.593 0.609 0.546 0.563 0.574 0.600 0.648 0.800 1.115 1.956 0.775
Total: 9.966, Avg: 0.767

php file cache
0.177 0.176 0.175 0.180 0.187 0.188 0.195 0.210 0.236 0.318 0.479 0.901 0.228
Total: 3.650, Avg: 0.281

(10b  0.1k  0.3k  0.5k  1k    2k    4k    8k    16k   32k   64k   128k  array)
Notes: Numbers in seconds, smaller numbers are better, connection times for memcached and mysql are not counted, file cache was done on tmpfs.

Cold cache, fill apc cache from cron:

All entries in apc cache are kept in shared memory. Shared memory is available to the web server and is not shared with the command line php (php-cli). When the web server is restarted, the cache is empty and refilling the complete cache can create performance problems. To fill the apc cache with a cron job, we use a second script to dump the cache to a binary file and load it inside the web server:

// php cron.php
$data = array("cached"=>1, "hello"=>"world");
apc_store($data);
apc_bin_dumpfile(array(), array_keys($data), "/var/cache/apc_cron.bin");
// bootstrap.php, http://myserver.com/...
if (apc_fetch("cached")===false) apc_bin_loadfile("/var/cache/apc_cron.bin");

echo apc_fetch("hello"); // gives "world"

Related articles:

Here is the test script:

<?php
// memcached -m 64 -s /tmp/m.sock -a 0777 -p 0 -u memcache
// memcached -m 64 -l 127.0.0.1 -p 11211 -u memcache
set_time_limit(3600);
error_reporting(E_ALL);
ini_set("display_errors", 1);
ini_set("apc.enable_cli", 1);
mysqli_report(MYSQLI_REPORT_STRICT | MYSQLI_REPORT_ERROR);

$data = array( // test are range from 10-128,000 bytes
  "1111111110",
  str_repeat("1111111110", 10),
  str_repeat("1111111110", 30),
  str_repeat("1111111110", 50),
  str_repeat("1111111110", 100),
  str_repeat("1111111110", 200),
  str_repeat("1111111110", 400),
  str_repeat("1111111110", 800),
  str_repeat("1111111110", 1600),
  str_repeat("1111111110", 3200),
  str_repeat("1111111110", 6400),
  str_repeat("1111111110", 12800),
  array(0=>"1111111110", "id2"=>"hello world", "id3"=>"foo bar", "id4"=>42)
);

echo "shm\n";
$t = array();
foreach ($data as $key=>$val) $t[] = shm(4000+$key, $val);
echo stats($t);

echo "apc\n";
$t = array();
foreach ($data as $key=>$val) $t[] = apc((string)$key, $val);
echo stats($t);

echo "memcache\n";
$t = array();
$m = memcache_connect("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcache socket\n";
$t = array();
$m = memcache_connect("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcache($m, (string)$key, $val);
echo stats($t);

echo "memcached\n";
$t = array();
$m = new Memcached();
$m->addServer("127.0.0.1", 11211);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);

/* not in memcached 1.x
echo "memcached socket\n";
$t = array();
$m = new Memcached();
$m->addServer("unix:///tmp/m.sock", 0);
foreach ($data as $key=>$val) $t[] = memcached($m, (string)$key, $val);
echo stats($t);
*/

echo "mysql myisam\n";
$t = array();
$m = new mysqli("127.0.0.1", "root", "", "t1");
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data mediumtext) engine=myisam");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "mysql memory\n";
$t = array();
mysqli_query($m, "drop table if exists t1.cache");
mysqli_query($m, "create table t1.cache (id int primary key, data varchar(65500)) engine=memory");
foreach ($data as $key=>$val) $t[] = mysql_cache($m, $key, $val);
echo stats($t);

echo "file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = file_cache((string)$key, $val);
echo stats($t);

echo "php file cache\n";
$t = array();
foreach ($data as $key=>$val) $t[] = php_cache((string)$key, $val);
echo stats($t);

function stats($t) {
  return "\nTotal: ".number_format(array_sum($t), 3).", ".
    "Avg: ".number_format(array_sum($t) / count($t), 3)."\n\n";
}

function format($num) {
  return number_format($num, 3);
}

function shm($id, $data) {
  if (is_array($data)) {
    $arr = true;
    $data = serialize($data);
  } else $arr = false;
  $len = strlen($data);
  $shm_id = shmop_open($id, "c", 0644, $len);
  shmop_write($shm_id, $data, 0);
  $start = microtime(true);
  if ($arr) {
    for ($i=0; $i<100000; $i++) $v = unserialize(shmop_read($shm_id, 0, $len));
  } else {
    for ($i=0; $i<100000; $i++) $v = shmop_read($shm_id, 0, $len);
  }
  echo format($end = microtime(true)-$start)." ";
  shmop_close($shm_id);
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function apc($id, $data) {
  apc_store($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = apc_fetch($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcache($m, $id, $data) {
  memcache_set($m, $id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = memcache_get($m, $id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function memcached($m, $id, $data) {
  $m->set($id, $data);
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) $v = $m->get($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function mysql_cache($m, $id, $data) {
  $d = is_array($data) ? serialize($data) : $data;
  mysqli_query($m, "insert into t1.cache values (".$id.", '".$d."')");
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
      $v = unserialize($v[0]);
    }
  } else {
    for ($i=0; $i<100000; $i++) {
      $v = mysqli_query($m, "SELECT data FROM t1.cache WHERE id=".$id)->fetch_row();
    }
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr($v[0], 0, 10)=="1111111110");
  return $end;
}

function file_cache($id, $data) {
  file_put_contents($id, is_array($data) ? serialize($data) : $data);
  $start = microtime(true);
  if (is_array($data)) {
    for ($i=0; $i<100000; $i++) $v = unserialize(file_get_contents($id));
  } else {
    for ($i=0; $i<100000; $i++) $v = file_get_contents($id);
  }
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

function php_cache($id, $data) {
  $id .= ".php";
  $data = is_array($data) ? var_export($data, 1) : "'".$data."'";
  file_put_contents($id, "<?php\n\$v=".$data.";");
  touch($id, time()-10); // needed for APC's file update protection
  $start = microtime(true);
  for ($i=0; $i<100000; $i++) include($id);
  echo format($end = microtime(true)-$start)." ";
  assert(substr(is_array($v) ? $v[0] : $v, 0, 10)=="1111111110");
  return $end;
}

12 comments:

  1. Lessons learned in production systems:
    - never ever use file cache, even if it fits in memory. Smarty caching with apc versus filesystem with 300k files in 200k dirs on a busy system: loadavg went down to 3 from 40, page load times went down from 2 to 0.3 seconds
    - apc user cache is faster than raw php shm access, because locking & other admin stuff is done in C rather than in PHP (the test above is misleading because it wastes memory by storing data in fixed 4000 byte offsets without locking semaphores)

    For me, apc wins the contest in both speed and usability - as long as local caching is sufficient

    ReplyDelete
    Replies
    1. > 300k files in 200k

      Which file system did you use?
      I did some backup tests (tar+gzip) and xfs/btrfs on a SSD which were quite good.

      Delete
    2. This was on ext3. Dis i/o were quite low though, since everything fit into memory. As the server load was high and the server also felt unresponsive at times, i guess some kernel- or php cache (kernel 2.6.26, php 5.2.x) was not optimized for these numbers. As soon as we switched to using apc all servers were happy again.

      Delete
    3. ext3 is not optimized for small files. XFS is mostly used on mail servers with many small files (mail-dir format), see:
      https://oss.oracle.com/projects/btrfs/dist/documentation/benchmark.html (ext3 marked in red color)

      Delete
  2. Your title is "PHP performance and memory usage". The tests seem to only cover speed of the different solutions. I would very interested to see memory usage comparisons, as I am looking to optimize memory usage in my application.

    ReplyDelete
  3. "Nice blog.. Thanks for sharing this blog information with us…

    php

    ReplyDelete
  4. > Yac is a lockless cache, you should try to avoid or reduce the probability of multiple processes set one key

    I'm not sure if it is safe to use Yac with php-fpm?

    ReplyDelete
  5. Awesome writeup here! I was almost surprised to find it, since it seems that not too many people have tried php file caching in php yet. It turns out that the var_export version is insanely fast now in both php and hhvm, so it's highly recommended to use for arrays/objects.

    Nowhere near as in-depth as your analysis here, but I wrote a quick summary of how we use php caching to accelerate our application here: https://blog.graphiq.com/500x-faster-caching-than-redis-memcache-apc-in-php-hhvm-dcd26e8447ad#.l443zqme4

    ReplyDelete
    Replies
    1. great! You can try to skip the temporary variable to get more speed:
      file_put_contents("/tmp/$key", '<?php return ' . $val . ';');
      ...
      function cache_get($key) {
      return @include "/tmp/$key";

      Delete
    2. yea definitely makes sense like that for single variable stores. We use explicit variables in production because we also store an $expiration variable in each file.

      Delete
  6. great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
    web design Company
    web development Company
    web design Company in chennai
    web development Company in chennai
    web design Company in India
    web development Company in India

    ReplyDelete

Labels

performance (23) benchmark (6) MySQL (5) architecture (5) coding style (5) memory usage (5) HHVM (4) C++ (3) Java (3) Javascript (3) MVC (3) SQL (3) abstraction layer (3) framework (3) maintenance (3) Go (2) Golang (2) HTML5 (2) ORM (2) PDF (2) Slim (2) Symfony (2) Zend Framework (2) Zephir (2) firewall (2) log files (2) loops (2) quality (2) real-time (2) scrum (2) streaming (2) AOP (1) Apache (1) Arrays (1) C (1) DDoS (1) Deployment (1) DoS (1) Dropbox (1) HTML to PDF (1) HipHop (1) OCR (1) OOP (1) Objects (1) PDO (1) PHP extension (1) PhantomJS (1) SPL (1) SQLite (1) Server-Sent Events (1) Silex (1) Smarty (1) SplFixedArray (1) Unicode (1) V8 (1) analytics (1) annotations (1) apc (1) archiving (1) autoloading (1) awk (1) caching (1) code quality (1) column store (1) common mistakes (1) configuration (1) controller (1) decisions (1) design patterns (1) disk space (1) dynamic routing (1) file cache (1) garbage collector (1) good developer (1) html2pdf (1) internationalization (1) invoice (1) just-in-time compiler (1) kiss (1) knockd (1) legacy code (1) legacy systems (1) logtop (1) memcache (1) memcached (1) micro framework (1) ncat (1) node.js (1) openssh (1) pfff (1) php7 (1) phpng (1) procedure models (1) ramdisk (1) recursion (1) refactoring (1) references (1) regular expressions (1) search (1) security (1) sgrep (1) shm (1) sorting (1) spatch (1) ssh (1) strange behavior (1) swig (1) template engine (1) threads (1) translation (1) ubuntu (1) ufw (1) web server (1) whois (1)