mod_magnet, lighty, MySQL Proxy und überhaupt

Der von mir sehr geschätzte Kollege Jo Brunner hat mal wieder tief in die Trickkiste gegriffen, und im Rahmen seiner Xenjo Cache Registry den wertvollen Urlaub dafür verwendet (Chapeau!), sich ausführlich mit lighttpd, mod_magnet und lua zu beschäftigen. Näheres findet man auf seinem Blog


Überhaupt lohnt sich ein Blick auf all das, was Jan Kneschke (der Autor von lighttpd) so baut. Wer mit MySQL einiges an Schweinereien vorhat (zum Beispiel Sharding, Verteilung von DB-Queries auf Master & Slaves), der sollte sich im Übrigen MySQL Proxy ansehen. MySQL Proxy ist ein Query Interceptor und sitzt zwischen dem Client (also zum Beispiel dem PHP) und dem Datenbankserver. Gesteuert wird MySQL Proxy wie üblich mit lua, der kleinen wendigen Scriptsprache. Man kann also Queries on the fly umschreiben oder auf andere DB-Server „redirecten“. Ein Vorteil dabei: man muss seinen Anwendungslayer nicht anpassen, um die INSERTs/UPDATEs und DELETEs auf den Master zu richten und die SELECTs auf die Slaves – das erledigt MySQL Proxy mit einem passenden kleinen lua-Script.

Webcraft survey: lighttpd’s usage rising

According to the latest Netcraft survey statistics from 1st of February, 2007, Jan Kneschke’s little baby lighttpd is showing a steady growth, with currently serving 702712 servers which is a tremendous growth compared to January 2007 (only 172819 servers).

Who uses lighttpd? The homepage names a few:

lighttpd powers several popular Web 2.0 sites like YouTube, wikipedia and meebo. Its high speed io-infrastructure allows them to scale several times better with the same hardware than with alternative webservers.

lighttpd features a „faster FastCGI“, a COMET implementation and optimized high-speed features. Furthermore, it’s the ideal candidate for scaling out your PHP application (and getting the benefit of a fast download mechanism at no costs compared to Zend Download Server). Applications like Chorizo! or websites like sevenload (video sharing startup in Europe) are using it as their ground base for their high-speed applications.

Just give it a try. If you’re bound to Apache, just try using lighttpd as a load-balanced reverse-proxy for scaling out your Apache based applications? Or, separating your file downloads and static images into a lighttpd based powerhouse?

Make the download of large files with PHP (and lighty) very easy

As you may know, our friend Jan Kneschke is the creator of lighttpd (called „lighty“), a superb OpenSource webserver (designed for speed and easyness) where we provide exclusively consulting and implementation services for.

Some days ago I stumbled upon an old entry of Jan in lighty’s life, called „X-Sendfile“. There he explains how to speed up the delivery of (large) files with lighttpd instead of PHP (YES, lighttpd is very fast – for one customer we created an ImageServer with pure lighty that replaced a 4-server-cluster with Apache and now has 1 server with lighttpd (which is boring around at low load). The box makes 180 Mio. requests per month).

It makes use of the X-Sendfile: (upon lighttpd V1.5) or X-LIGHTTPD-send-file: (prior to V1.5) header lines. In fact, your PHP script does only the following things – for example in a Media Asset Management System:

  • check ACL for the particular user if he has the right to download the file
  • search for the original filename in the database it had when the file was uploaded
  • set the appropriate headers (the file may have a crypting filename on the disk, i.e. not the original filename as it was uploaded for)
  • output file via readfile() and the like
  • set the X-LIGHTTPD-send-file: or X-Sendfile: header
  • exit your script – let lighttpd take care about delivering the file to the client and save your PHP FCGI Binary for the next real application request

As you see, lighty will handle the delivery of the file itself. In order to support the hand-over to lighttpd you need to activate support for X-sendfile in the configuration:

fastcgi.server = {
   ".php" => {
      "" => {
          # ....
          "allow-x-send-file" => "enable"

Here’s a short example of a PHP script that lets lighttpd deliver the file:

 * lighttpd's feature of X-Sendfile explained.
 * @author Björn Schotte <>

// as this is an example, here's the static file. Usually, you may
// have something like /download.php?file_id=500 etc. This file here is 126 MB big
$file_on_harddisk = "/data/vhosts/bjoern/htdocs/acbd18db4cc2f85cedef654fccc4a4d8download.tar.gz";
$file_to_download = "download.tar.gz";

// try to fake a "ACL authentication". This is rather simple, so
// replace it with your own ACL routine
if ( !empty($_GET['authenticated']) && $_GET['authenticated'] == 1) {
        header( "Content-Disposition: attachment; filename=\"" . $file_to_download . '"' );
        Header( "X-LIGHTTPD-send-file: " . $file_on_harddisk);
} else {
    print "Sorry, you don't have permissions to download this file!<br />\n";

Happy lightning!

lighttpd’s mod_cml will change

You may have read Jo’s blog entry
Methods to reduce the load of your webserver by caching content: using lighttpd, MySQL UDF, LUA and speed everything up.
He explained there how to use lighttpd and its mod_cml together with MySQL to provide a caching system directly at the webserver, and not at the PHP level.

Now, our good ol‘ friend Jan Kneschke, author of god’s own webserver lighttpd (called „lighty“), mentioned in his blog („mod_cml is dead, long live mod_cml!“) that the name and functionality of mod_cml is subject to change. Why? mod_cml is not about caching only, it’s about deciding how to handle a request.

„Any status code can be returned, any content source will be able to be queried to assemble the content (files, memcache).

The main reason is to allow us to change the names like ‘trigger_handler’ and ‘CACHE_HIT’ to something more appropriate.“

Go and check it out!

Methods to reduce the load of your webserver by caching content: using lighttpd, MySQL UDF, LUA and speed everything up.

The method I would like to describe is based on the webserver

Lighttpd is a single process webserver written for high traffic sites.
It supports fast-cgi out of the box which makes it ideal for hosting PHP applications.
There are lots of nice modules for the daily
work like mod_access or mod_rewrite.
For more infos see the internals

There are also some benchmarks there.
Lighty´s home is always worth having a look at.