Creating hooks and scripts with mod_lua

Introduction

What is mod_lua

Stuff about what mod_lua is goes here.

What we will be discussing in this document

This document will discuss several cases where mod_lua can be used to either ease up a phase of the request processing or create more transparency in the logic behind a decision made in a phase.

Prerequisites

First and foremost, you are expected to have a basic knowledge of how the Lua programming language works. In most cases, we will try to be as pedagogical as possible and link to documents describing the functions used in the examples, but there are also many cases where it is necessary to either just assume that "it works" or do some digging yourself into what the hows and whys of various function calls.

Optimizing mod_lua for production servers

Setting a scope for Lua states

Setting the right LuaScope setting for your Lua scripts can be essential to your server's performance. By default, the scope is set to once, which means that every call to a Lua script will spawn a new Lua state that handles that script and is destroyed immediately after. This option keeps the memory footprint of mod_lua low, but also affects the processing speed of a request. If you have the memory to spare, you can set the scope to thread, which will make mod_lua spawn a Lua state that lasts the entirity of a thread's lifetime, speeding up request processing by 2-3 times. Since mod_lua will create a state for each script, this may be an expensive move, memory-wise, so to compromise between speed and memory usage, you can choose the server option to create a pool of Lua states to be used. Each request for a Lua script or a hook function will then acquire a state from the pool and release it back when it's done using it, allowing you to still gain a significant performance increase, while keeping your memory footprint low. Some examples of possible settings are:

LuaScope once
LuaScope thread
LuaScope server 5 40

As a general rule of thumb: If your server has none to low usage, use once or request, if your server has low to medium usage, use the server pool, and if it has high usage, use the thread setting. As your server's load increases, so will the number of states being actively used, and having your scope set to once/request/conn will stop being beneficial to your memory footprint.

Note: The min and max settings for the server scope denotes the minimum and maximum states to keep in a pool per server process, so keep this below your ThreadsPerChild limit.

Using code caching

By default, mod_lua stats each Lua script to determine whether a reload (and thus, a re-interpretation and re-compilation) of a script is required. This is managed through the LuaCodeCache directive. If you are running your scripts on a production server, and you do not need to update them regularly, it may be advantageous to set this directive to the forever value, which will cause mod_lua to skip the stat process and always reuse the compiled byte-code from the first access to the script, thus speeding up the processing. For Lua hooks, this can prove to increase peformance, while for scripts handled by the lua-script handler, the increase in performance may be negligible, as files httpd will stat the files regardless.

Keeping the scope clean

For maximum performance, it is generally recommended that any initialization of libraries, constants and master tables be kept outside the handle's scope:

--[[ This is good practice ]]--
require "string"
require "someLibrary"
local masterTable = {}
local constant = "Foo bar baz"

function handle(r)
    do_stuff()
end

--[[ This is bad practice ]]--
require "string"

function handle(r)
    require "someLibrary"
    local masterTable = {}
    local constant = "Foo bar baz"
    do_stuff()
end

Example 1: A basic remapping module

These first examples show how mod_lua can be used to rewrite URIs in the same way that one could do using Alias or RewriteRule, but with more clarity on how the decision-making takes place, as well as allowing for more complex decisions than would otherwise be allowed with said directives.

LuaHookTranslateName /path/too/foo.lua remap

--[[
    Simple remap example.
    This example will rewrite /foo/test.bar to the physical file
    /internal/test, somewhat like how mod_alias works.
]]--

function remap(r)
    -- Test if the URI matches our criteria
    local barFile =  r.uri:match("/foo/([a-zA-Z0-9]+)%.bar")
    if barFile then
        r.filename = "/internal/" .. barFile
    end
    return apache2.OK
end

--[[
    Advanced remap example.
    This example will evaluate some conditions, and based on that, 
    remap a file to one of two destinations, using a rewrite map.
    This is similar to mixing AliasMatch and ProxyPass, but 
    without them clashing in any way. Assuming we are on example.com, then:

    http://example.com/photos/test.png will be rewritten as /uploads/www/test.png
    http://example.com/ext/foo.html will be proxied to http://www.external.com/foo.html
    URIs that do not match, will be served by their respective default handlers
]]--

local map = {
      photos = {   
                   source = [[^/photos/(.+)\.png$]], 
                   destination = [[/uploads/www/$1.png]],
                   proxy = false
                },
      externals = {
                   source = [[^/ext/(.*)$]],
                   destination = [[http://www.external.com/$1]],
                   proxy = true
                }
}

function interpolateString(s,v)
    return s:gsub("%$(%d+)", function(a) return v[tonumber(a)] end)
end
        
function remap(r)
    -- browse through the rewrite map
    for key, entry in pairs(map) do
        -- Match source regex against URI
        local match = apache2.regex(r, entry.source, r.uri) then
        if match and match[0] then
            r.filename = interpolateString(entry.destination, match)
            -- Is this a proxied remap?
            if entry.proxy then
                r.handler = "proxy-server" -- tell mod_proxy to handle this
                r.proxyreq = apache2.PROXYREQ_REVERSE -- We'll want to do a reverse proxy
                r.filename = "proxy:" .. r.filename -- Add the proxy scheme to the destination
            end
            return apache2.OK
        end
    end
    return apache2.DECLINED
end

bla bla

Example 2: Mass virtual hosting

As with simple and advanced rewriting, you can use mod_lua for dynamically assigning a hostname to a specific document root, much like mod_vhost_alias does, but with more control over what goes where. This could be as simple as a table holding the information about which host goes into which folder, or more advanced, using a database holding the document roots of each hostname.

LuaHookTranslateName /path/too/foo.lua mass_vhost

--[[
    Simple mass vhost script
    This example will check a map for a virtual host and rewrite filename and 
    document root accordingly.
]]--

local vhosts = {
    { domain = "example.com", home = "/www/example.com" },
    { domain = "example.org", home = "/nfs/ext1/example.org" }
}

function mass_vhost(r)
    -- Match against our hostname
    for key, entry in pairs(vhosts) do
        -- match against either host or *.host:
        if apache2.strcmp_match(r.hostname, entry.domain) or
           apache2.strcmp_match(r.hostname, "*." .. entry.domain) then
            -- If it matches, rewrite filename and set document root
            local filename = r.filename:sub(r.document_root:len()+1)
            r.filename = entry.home .. filename
            apahce2.set_document_root(entry.home)
            return apache2.OK
        end
    end
    return apache2.DECLINED
end

--[[
    Advanced mass virtual hosting
    This example will query a database for vhost entries and save them for
    60 seconds before checking for updates. For best performance, such scripts
    should generally be run with LuaScope set to 'thread' or 'server'
]]--

local cached_vhosts = {}
local timeout = 60

-- Function for querying the database for saved vhost entries
function query_vhosts(r)
    local host = r.hostname
    if not cached_vhosts[host] or (cached_vhosts[host] and cached_vhosts[host].updated < os.time() - timeout) then
        local db,err = ap.dbopen(r,"mod_dbd")
        local _host = db:escape(r,host)
        local res, err = db:query(r, ("SELECT `destination` FROM `vhosts` WHERE `hostname` = '%s' LIMIT 1"):format(_host) )
        if res and #res == 1 then
            cached_vhosts[host] = { updated = os.time(), destination = res[1][1] }
        else
            cached_vhosts[host] = { updated = os.time(), destination = nil } -- don't re-query whenever there's no result, wait a while.
        end
        db:close()
    end
    if cached_vhosts[host] then 
        return cached_vhosts[host].destination
    else
        return nil
    end
end
        
function mass_vhost(r)
    -- Check whether the hostname is in our database
    local destination = query_vhosts(r)
    if destination then
        -- If found, rewrite and change document root
        local filename = r.filename:sub(r.document_root:len()+1)
        r.filename = destination .. filename
        ap.set_document_root(r,destination)
        return apache2.OK
    end
    return apache2.DECLINED
end

Example 3: A basic authorization hook

With the authorization hooks, you can add custom auth phases to your request processing, allowing you to either add new requirements that were not previously supported by httpd, or tweaking existing ones to accommodate your needs.

LuaHookAuthChecker /path/too/foo.lua check_auth

--[[ 
     A simple authentication hook that checks a table containing usernames and
     passwords of two accounts.
]]--
local accounts = {
    bob  = 'somePassword',
    jane = 'Iloveponies'
}

-- Function for parsing the Authorization header into a username and a password
function parse_auth(str)
    local user,pass = nil, nil
    if str and str:len() > 0 then
        str = apache2.base64_decode(auth):sub(7));
        user, pass = auth:match("([^:]+)%:([^:]+)")
    end
    return user, pass
end

-- The authentication hook
function check_auth(r)
    local user, pass = parse_auth(r.headers_in['Authorization'])
    local authenticated = false
    if user and pass then
        if accounts[user] and accounts[user] == pass then
            authenticated = true
            r.user = user
        end
    end
    r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"'
    if not authenticated then
        return 401
    else
        return apache2.OK
    end
end

--[[ 
     An advanced authentication checker with a database backend,
     caching account entries for 1 minute
]]--

local timeout = 60 -- Set account info to be refreshed every minute
local accounts = {}

-- Function for parsing the Authorization header into a username and a password
function parse_auth(str)
    local user,pass = nil, nil
    if str and str:len() > 0 then
        str = apache2.base64_decode(auth):sub(7));
        user, pass = auth:match("([^:]+)%:([^:]+)")
    end
    return user, pass
end

-- Function for querying the database for the account's password (stored as a salted SHA-1 hash)
function fetch_password(user)
    if not accounts[user] or (accounts[user] and accounts[user].updated < os.time() - timeout) then
        local db = apache2.dbopen(r, "mod_dbd")
        local usr = db:escape(user)
        local res, err = db:query( ("SELECT `password` FROM `accounts` WHERE `user` = '%s' LIMIT 1"):format(usr) )
        if res and #res == 1 then
            accounts[user] = { updated = os.time(), password = res[1][1] }
        else
            accounts[user] = nil
        end
        db:close()
    end
    if accounts[user] then 
        return accounts[user].password
    else
        return nil
    end
end
    
-- The authentication hook
function check_auth(r)
    local user, pass = parse_auth(r.headers_in['Authorization'])
    local authenticated = false
    if user and pass then
        pass = apache2.sha1("addSomeSalt" .. pass)
        local stored_pass = fetch_password(user)
        if stored_pass and pass == stored_pass then
            authenticated = true
            r.user = user
        end
    end
    r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"'
    if not authenticated then
        return 401
    else
        return apache2.OK
    end
end

Example 4: Authorization using LuaAuthzProvider

If you require even more advanced control over your authorization phases, you can add custom authz providers to help you manage your server. The example below shows you how you can split a single htpasswd file into groups with different permissions:

LuaAuthzProvider rights /path/to/lua/script.lua rights_handler
<Directory /www/private>
    Require rights member
</Directory>
<Directory /www/admin>
    Require rights admin
</Directory>

--[[ 
     This script has two user groups; members and admins, and whichever 
     is refered to by the "Require rights" directive is checked to see
     if the authenticated user belongs to this group.
]]--

local members = { "rbowen", "humbedooh", "igalic", "covener" }
local admins = { "humbedooh" }

function rights_handler(r, what)
    if r.user == nil then
        return apache2.AUTHZ_AUTHZ_DENIED_NO_USER
    end
    if what == "member" then
        for k, v in pairs(members) do
            if r.user == v then
                return apache2.AUTHZ_GRANTED
            end
        end
    elseif what == "admin" then
        for k, v in pairs(admins) do
            if r.user == v then
                return apache2.AUTHZ_GRANTED
            end
        end
    end
    return apache2.AUTHZ_DENIED
end

Example 5: Overlays using LuaMapHandler

Coming soon!

LuaMapHandler ^/portal/([a-z]+)/   /path/to/lua/script.lua handle_$1

Example 6: Basic Lua scripts

Also coming soon

HTTPd bindings: String manipulation

apache2.base64_encode
apache2.base64_decode
apache2.escape
apache2.unescape
apache2.escapehtml
apache2.md5
apache2.sha1
apache2.os_escape_path
apache2.escape_logitem

apache2.base64_decode( request_rec r,  string string )

Decodes a base64-encoded string

Arguments:

Return value(s):
The base64-decoded string.

Example:

local str = "This is a test"
local encoded = apache2.base64_encode(str)
local decoded = apache2.base64_decode(encoded)
        

apache2.base64_encode( request_rec r,  string string )

Encodes a string using the base64 encoding scheme.

Arguments:

Example:

local str = "This is a test"
local encoded = apache2.base64_encode(str)
local decoded = apache2.base64_decode(encoded)
        

apache2.escape( request_rec r,  string string )

url-escapes a string

Arguments:

Return value(s):
The URL-escaped string.

Example:

local str = "This is a test"
local escaped = apache2.escape(str)
print(escaped) -- prints "This+is+a+test"
        

apache2.escape_logitem( request_rec r,  string path )

Escape a string for logging

Arguments:

Return value(s):
The converted string

apache2.escapehtml( request_rec r,  string html,  boolean toasc )

Escapes HTML entities.

Arguments:

Return value(s):
The escaped HTML code.

Example:

local html = "<b>Testing!</b>"
local escaped = apache2.escapehtml(html)
r:puts(escaped) -- prints "&lt;b&gt;Testing!&lt;/b&gt;"
        

apache2.md5( request_rec r,  string string )

Computes an MD5 digest sum based on a string (binary safe)

Arguments:

Return value(s):
The MD5 digest sum of the data provided

Example:

local text = "The quick brown fox jumps over the lazy dog"
local md5 = apache2.md51(text)
r:puts(md5) -- prints out "9e107d9d372bb6826bd81d3542a419d6"
        

apache2.os_escape_path( request_rec r,  string path,  boolean partial )

convert an OS path to a URL in an OS dependent way.

Arguments:

Return value(s):
The converted URL

Example:

local path = ap_os_escape_path("C:/foo/bar.txt")
        

apache2.sha1( request_rec r,  string string )

Computes an SHA-1 digest sum based on a string (binary safe)

Arguments:

Return value(s):
The SHA-1 digest sum of the data provided

Example:

local text = "The quick brown fox jumps over the lazy dog"
local sha1 = apache2.sha1(text)
r:puts(sha1) -- prints out "2fd4e1c67a2d28fced849ee1bb76e7391b93eb12"
        

apache2.unescape( request_rec r,  string string )

unescapes an URL-escaped string

Arguments:

Return value(s):
The URL-unescaped string

Example:

local str = "This+is+a+test"
local unescaped = apache2.unescape(str)
print(unescaped) -- prints "This is a test"
        

HTTPd bindings: Request handling

apache2.requestbody
apache2.add_input_filter
apache2.get_basic_auth_pw
apache2.set_document_root
apache2.set_context_prefix
apache2.get_server_name_for_url
apache2.set_keepalive
apache2.make_etag
apache2.send_interim_response

apache2.add_input_filter( request_rec r,  string filter )

Adds an input filter to the request

Arguments:

Example:

apache2.add_input_filter(r, "SPAM_FILTER") -- Check input for spam..?
        

apache2.get_basic_auth_pw( request_rec r )

Returns the password from a basic authorization request or nil if none was supplied

Arguments:

Return value(s):
The password from a basic authorization request or nil if none was supplied

apache2.get_server_name_for_url( request_rec r )

Get the current server name from the request for the purposes of using in a URL. If the server name is an IPv6 literal address, it will be returned in URL format (e.g., "[fe80::1]").

Arguments:

apache2.make_etag( request_rec r,  boolean force_weak )

Constructs an entity tag from the resource information. If it's a real file, build in some of the file characteristics.

Arguments:

Return value(s):
The entity tag

apache2.requestbody( request_rec r,  number size,  string filename )

Reads the request body. If a filename is specified, the request body will be written to that file and the number of bytes written returned, otherwise, the full request body will be returned as a string.

Arguments:

Return value(s):
The number of bytes written if a filename was specified, otherwise it returns the entire request body as a string.

Example:

if tonumber(r.headers_in['Content-Length'] or 0) < 10000 then
    local smallfile = apache2.requestbody(r, 10000) -- fetch a small file into memory
    r:puts("I saved the uploaded file in memory")
else
    local read = apache2.requestbody(r, 0, "/path/to/tmp")
    r:puts("I saved the uploaded file in a temp directory. Total bytes written was: ", read)
end
        

apache2.send_interim_response( request_rec r,  boolean send_headers )

Sends an interim (HTTP 1xx) response immediately.

Arguments:

Example:

apache2.send_interim_response(r, false)
        

apache2.set_context_prefix( request_rec r,  string prefix,  string document )

Set context_prefix and context_document_root for a request.

Arguments:

apache2.set_document_root( request_rec r,  string root )

Sets the document root of the request.

Arguments:

Example:

-- Suppose our real document root is /var/bar, then...
if r.hostname == "www.foo.com" then
    apache2.set_document_root(r, "/www/foo") -- change document root on the fly
end
        

apache2.set_keepalive( request_rec r )

Sets the keepalive status for this request

Arguments:

Return value(s):
True if keepalive can be set, false otherwise

HTTPd bindings: Parser functions

apache2.expr
apache2.regex
apache2.strcmp_match

apache2.expr( request_rec r,  string expression )

Evaluates an ap_expr (think <If ...>) expression and returns true if the expression is true, false otherwise. A second value containing an error string is returned if the expression is invalid.

Arguments:

Return value(s):
True if the expression evaluates as true, false if the expression doesn't evaluate as true or if an error occurred. If an error occurred during parsing, a second value will be returned, containng the error string.

Example:

if apache2.expr("%{REQUEST_URI} =~ /force-gzip") then
    r:addoutputfilter("DEFLATE")
end
        

apache2.regex( request_rec r,  string expression,  string source )

Evaluates a regular expression and, if it matches the source string, captures the variables and returns the matches as a table. On error, it returns nil.

Arguments:

Return value(s):
True if the expression evaluates as true, false if the expression doesn't evaluate as true or if an error occurred. If an error occurred during parsing, a second value will be returned, containng the error string.

Example:

local matches = apache2.regex(r, [[(\S+) kitty]], "Hello kitty")
if matches and matches[1] then 
    r:puts("You said ", matches[1], " to kitty")
end
        

apache2.strcmp_match( string str,  string expexted,  boolean ignoreCase )

Determines if a string matches a pattern containing the wildcards '?' or '*'

Arguments:

Return value(s):
True if the two strings match, false otherwise.

Example:

if apache2.strcmp_match("foo.bar", "foo.*") then
    r:puts("It matches!")
end
        

HTTPd bindings: Server settings

apache2.add_version_component
apache2.mpm_query
apache2.terminate
apache2.scoreboard_process
apache2.scoreboard_worker
apache2.module_info
apache2.loaded_modules
apache2.runtime_dir_relative
apache2.server_info
apache2.state_query
apache2.custom_response
apache2.exists_config_define

apache2.add_version_component( request_rec r,  string component )

Adds a component to the server description and banner strings

Arguments:

Example:

if not apache2.banner():match("FooModule") then -- Make sure we haven't added it already
    apache2.add_version_component(r, "FooModule/1.0")
end
        

apache2.custom_response( request_rec r,  number status,  string string )

Install a custom response handler for a given status

Arguments:

Example:

apache2.custom_response(r, 404, "Not found!!")
        

apache2.exists_config_define( string name )

Checks for a definition from the server command line

Arguments:

Example:

if apache2.exists_config_define("FOO") then
    r:puts("This server was started with -DFOO")
end
        

apache2.loaded_modules( )

Returns a table containing the name (c filename) of all loaded modules

Arguments:

None

Return value(s):
A table containing the name (c filename) of all loaded modules

apache2.module_info( string c,  string file )

Returns information about a specific module (if loaded)

Arguments:

Return value(s):
The various commands available to this module as a table, or nil if the module wasn't found.

apache2.mpm_query( number i )

Queries the MPM for a specific value

Arguments:

Return value(s):
The queried value

apache2.runtime_dir_relative( request_rec r,  string file )

Returns the path of a file relative to the default runtime directory

Arguments:

Return value(s):
The path of a file relative to the default runtime directory

apache2.scoreboard_process( request_rec r,  number child )

Returns the scoreboard for a server daemon as a table

Arguments:

Return value(s):
The scoreboard for a server daemon as a table

apache2.scoreboard_worker( request_rec r,  number child,  number thread )

Returns the scoreboard for a single thread as a table

Arguments:

Return value(s):
The scoreboard for a single thread as a table

apache2.server_info( )

Returns a table with information about the server program

Arguments:

None

Return value(s):
A table with information about the server program

apache2.state_query( number field )

Query the server for some state information

Arguments:

Example:

local gen = apache2.state_query(2)
r:puts("This is generation no. " .. gen .. " of the top-level parent")
        

apache2.terminate( )

Kills off a server process. This has no other use than to show how dangerous mod_lua can be ;)

Arguments:

None

HTTPd bindings: Database connectivity

apache2.dbopen
db:query
db:do
db:close

apache2.dbopen( request_rec r,  string dbtype,  string conn_string )

Opens up a new database connection. See the DB functions for mod_pLua for more info on this.

Arguments:

Return value(s):
The database connection as a table with functions, or nil if the connection failed. If a connection failed, a second argument (string) with the error code is returned.

Example:

local db, error = apache2.dbopen(r, "mod_dbd")
if error then
    r:puts("DB error: ", error)
else
    -- DB stuff here
end
        

db:close( request_rec r )

Closes a database connection

Arguments:

Example:

local db = apache2.dbopen(r, "mod_dbd") -- open a db connection
db:close() -- close it down
        

db:do( request_rec r,  string query )

Executes a statement that doesn't return a result set

Arguments:

Return value(s):
If the statement is valid, a table of results are returned. If an error occurred, the first return value is false and the second return value is a string containing an error message.

Example:

local db = apache2.dbopen(r, "mod_dbd")
local affected = db:do("DELETE FROM `table` WHERE 1")
if affected then
    r:puts("Affected ", affected, " rows")
end
        

db:query( request_rec r,  string query )

Queries the database for information using the specified statement.

Arguments:

Return value(s):
If the statement is valid, a table of results are returned. If an error occurred, the first return value is false and the second return value is a string containing an error message.

Example:

local db = apache2.dbopen(r, "mod_dbd")
local result, error = db:query("SELECT * FROM `table` WHERE 1")
if result then
    for key, value in pairs(result)
        r:puts( ("row %s: %s\n"):format(key, table.concat(value, ", ")) )
    end
end
        

HTTPd bindings: Miscellaneous

apache2.clock
apache2.sleep

apache2.clock( )

Returns the current time in microseconds.

Arguments:

None

Return value(s):
The current time in microseconds.

apache2.sleep( number seconds )

Sleeps for a while. Floating point values can be used to sleep for less than a second.

Arguments:

Example:

r:puts("this is ")
apache2.flush(r)
apache2.sleep(0.25) -- sleep for a quarter second.
r:puts("delayed")