When displaying images on our website, we check if the file exists with a call to file_exists(). We fall back to a dummy image if the file was missing.

However, profiling has shown that this is the slowest part of generating our pages with file_exists() taking up to 1/2 ms per file. We are only testing 40 or so files, but this still pushes 20ms onto the page load time.

Can anyone suggest a way of making this go faster? Is there a better way of testing if the file is present? If I build a cache of some kind, how should I keep it in sync.

Answer
      • 2
    • I've actually tried this once, my function took 11 times the execution time of file_exists() so my best bet is to use caching better, or come up with another method.
      • 2
    • If the slowest part in your code only adds 20ms in total load time, you should go out and treat yourself to a beer, instead of worrying about it so much you're posting a question to SO ;-)
      • 2
    • What file system are you using? - The speed of file_Exists() should mostly depend on the speed of the stat() syscall. How many files are in the directory? (Depending on the file system the number of files has an impact on the stat() speed)
      • 2
    • Oh, quoting Wikipedia... The average length of a blink is 300 to 400 Miliseconds. Not sure why, but it felt appropriate to share it with you.

file_exists() should be a very inexpensive operation. Note too that file_exists builds its own cache to help with performance.

See: http://php.net/manual/en/function.file-exists.php

  • 29
Reply Report
      • 2
    • I guess I should just accept that the performance is fine and leave it as is. I might go an break up the files into more folders though, as this will probably help things.
      • 1
    • According to the documentation caching will only occur if file_exists() returns true. So if you happen to check for in-existent files the function will check every time. You could create a symlink to the dummy image when file_exists() returns false so that subsequent calls will be cached. (this might cause other problems)

Use absolute paths! Depending on your include_path setting PHP checks all(!) these dirs if you check relative file paths! You might unset include_path temporarily before checking the existence.

realpath() does the same but I don't know if it is faster.

But file access I/O is always slow. A hard disk access IS slower than calculating something in the processor, normally.

  • 21
Reply Report

The fastest way to check existence of a local file is stream_resolve_include_path():

if (false !== stream_resolve_include_path($s3url)) { 
  //do stuff 
}

Performance results stream_resolve_include_path() vs file_exists():

Test name       Repeats         Result          Performance     
stream_resolve  10000           0.051710 sec    +0.00%
file_exists     10000           0.067452 sec    -30.44%

In test used absolute paths. Test source is here. PHP version:

PHP 5.4.23-1~dotdeb.1 (cli) (built: Dec 13 2013 21:53:21)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies

  • 18
Reply Report

We fall back to a dummy image if the file was missing

If you're just interested in falling back to this dummy image, you might want to consider letting the client negotiate with the server by means of a redirect (to the dummy image) on file-not-found.

That way you'll just have a little redirection overhead and a not-noticeable delay on the client side. At least you'll get rid of the "expensive" (which it isn't, I know) call to file_exists.

Just a thought.

  • 11
Reply Report
      • 2
    • +1 for clever. Now I'm curious about what happens if you pass jpg data back with a 404 response. This is, after all, a 404-type behavior that OP is looking for.
    • Should be rendered OK. Basically it's the same behavior for custom 404-pages; ther're rendered as XHTML if served as such. Haven't tested, though.

Benchmarks with PHP 5.6:

Existing File:

0.0012969970 : stream_resolve_include_path + include  
0.0013520717 : file_exists + include  
0.0013728141 : @include  

Invalid File:

0.0000281333 : file_exists + include  
0.0000319480 : stream_resolve_include_path + include  
0.0001471042 : @include  

Invalid Folder:

0.0000281333 : file_exists + include  
0.0000360012 : stream_resolve_include_path + include  
0.0001239776 : @include  

Code:

// microtime(true) is less accurate.
function microtime_as_num($microtime){
  $time = array_sum(explode(' ', $microtime));
  return $time;
}

function test_error_suppression_include ($file) {
  $x = 0;
  $x = @include($file);
  return $x;
}

function test_file_exists_include($file) {
  $x = 0;
  $x = file_exists($file);
  if ($x === true) {
    include $file;
  }
  return $x;
}

function test_stream_resolve_include_path_include($file) {
  $x = 0;
  $x = stream_resolve_include_path($file);
  if ($x !== false) {
    include $file;
  }
  return $x;
}

function run_test($file, $test_name) {
  echo $test_name . ":\n";
  echo str_repeat('=',strlen($test_name) + 1) . "\n";

  $results = array();
  $dec = 10000000000; // digit precision as a multiplier

  $i = 0;
  $j = 0;
  $time_start = 0;
  $time_end = 0;
  $x = -1;
  $time = 0;

  $time_start = microtime();
  $x= test_error_suppression_include($file);
  $time_end = microtime();
  $time = microtime_as_num($time_end) - microtime_as_num($time_start);

  $results[$time*$dec] = '@include';

  $i = 0;
  $j = 0;
  $time_start = 0;
  $time_end = 0;
  $x = -1;
  $time = 0;

  $time_start = microtime();
  $x= test_stream_resolve_include_path_include($file);
  $time_end = microtime();
  $time = microtime_as_num($time_end) - microtime_as_num($time_start);

  $results[$time * $dec] = 'stream_resolve_include_path + include';

  $i = 0;
  $j = 0;
  $time_start = 0;
  $time_end = 0;
  $x = -1;
  $time = 0;

  $time_start = microtime();
  $x= test_file_exists_include($file);
  $time_end = microtime();
  $time = microtime_as_num($time_end) - microtime_as_num($time_start);

  $results[$time * $dec ] = 'file_exists + include';

  ksort($results, SORT_NUMERIC);

  foreach($results as $seconds => $test) {
    echo number_format($seconds/$dec,10) . ' : ' . $test . "\n";
  }
  echo "\n\n";
}

run_test($argv[1],$argv[2]);

Command line Execution:

php test.php '/path/to/existing_but_empty_file.php' 'Existing File'  
php test.php '/path/to/non_existing_file.php' 'Invalid File'  
php test.php '/path/invalid/non_existing_file.php' 'Invalid Folder'  
  • 6
Reply Report

Create a hashing routine for sharding the files into multiple sub-directories.

filename.jpg -> 012345 -> /01/23/45.jpg

Also, you could use mod_rewrite to return your placeholder image for requests to your image directory that 404.

  • 3
Reply Report

Old question, I'm going to add an answer here. For php 5.3.8, is_file() (for an existing file) is an order of magnitude faster. For a non-existing file, the times are nearly identical. For PHP 5.1 with eaccelerator, they are a little closer.

PHP 5.3.8 w & w/o APC

time ratio (1000 iterations)
Array
(
    [3."is_file('exists')"] => 1.00x    (0.002305269241333)
    [5."is_link('exists')"] => 1.21x    (0.0027914047241211)
    [7."stream_resolve_inclu"(exists)] => 2.79x (0.0064241886138916)
    [1."file_exists('exists')"] => 13.35x   (0.030781030654907)
    [8."stream_resolve_inclu"(nonexists)] => 14.19x (0.032708406448364)
    [4."is_file('nonexists)"] => 14.23x (0.032796382904053)
    [6."is_link('nonexists)"] => 14.33x (0.033039808273315)
    [2."file_exists('nonexists)"] => 14.77x (0.034039735794067)
)

PHP 5.1 w/ eaccelerator

time ratio (1000x)
Array
(
    [3."is_file('exists')"] => 1.00x    (0.000458002090454)
    [5."is_link('exists')"] => 1.22x    (0.000559568405151)
    [6."is_link('nonexists')"] => 3.27x (0.00149989128113)
    [4."is_file('nonexists')"] => 3.36x (0.00153875350952)
    [2."file_exists('nonexists')"] => 3.92x (0.00179600715637)
    [1."file_exists('exists"] => 4.22x  (0.00193166732788)
)

There are a couple of caveats.
1) Not all "files" are files, is_file() tests for regular files, not symlinks. So on a *nix system, you can't get away with just is_file() unless you are sure that you are only dealing with regular files. For uploads, etc, this may be a fair assumption, or if the server is Windows based, which does not actually have symlinks. Otherwise, you'll have to test is_file($file) || is_link($file).

2) Performance definitely degrades for all methods if the file is missing and becomes roughly equal.

3) Biggest caveat. All the methods cache the file statistics to speed lookup, so if the file is changing regularly or quickly, deleted, reappears, deletes, then clearstatcache(); has to be run to insure that the correct file existence information is in the cache. So I tested those. I left out all the filenames and such. The important thing is that almost all the times converge, except stream_resolve_include, which is 4x as fast. Again, this server has eaccelerator on it, so YMMV.

time ratio (1000x)
Array
(
    [7."stream_resolve_inclu...;clearstatcache();"] => 1.00x    (0.0066831111907959)
    [1."file_exists(...........;clearstatcache();"] => 4.39x    (0.029333114624023)
    [3."is_file(................;clearstatcache();] => 4.55x    (0.030423402786255)
    [5."is_link(................;clearstatcache();] => 4.61x    (0.030798196792603)
    [4."is_file(................;clearstatcache();] => 4.89x    (0.032709360122681)
    [8."stream_resolve_inclu...;clearstatcache();"] => 4.90x    (0.032740354537964)
    [2."file_exists(...........;clearstatcache();"] => 4.92x    (0.032855272293091)
    [6."is_link(...............;clearstatcache();"] => 5.11x    (0.034154653549194)
)

Basically, the idea is, if you're 100% sure that it is a file, not a symlink or a directory, and in all probability, it will exist, then use is_file(). You'll see a definite gain. If the file could be a file or a symlink at any moment, then the failed is_file() 14x + is_link() 14x (is_file() || is_link()), and will end up being 2x slower overall. If the file's existence changes A LOT, then use stream_resolve_include_path().

So it depends on your usage scenario.

  • 2
Reply Report

If you want to check existence of an image file, a much faster way is to use getimagesize !

Faster locally and remotely!

if(!@GetImageSize($image_path_or_url)) // False means no imagefile
 {
 // Do something
 }
  • 1
Reply Report

I find 1/2ms per call very, very affordable. I don't think there are much faster alternatives around, as the file functions are very close to the lower layers that handle file operations.

You could however write a wrapper to file_exists() that caches results into a memcache or similar facility. That should reduce the time to next to nothing in everyday use.

  • 0
Reply Report

You could do a cronjob to periodically create a list of images and store them in DB/file/BDB/...

Every half an hour should be fine, but be sure to create an interface to reset cache in case of file addition/delete.

And then, it's also easy to run find . -mmin -30 -print0 on the shell and add new files.

  • 0
Reply Report

Warm tip !!!

This article is reproduced from Stack Exchange / Stack Overflow, please click

Trending Tags

Related Questions