Digging into Apache httpd's Source 10: Request Processing — From a URL to a File and a Handler

What this post covers

This is the tenth post in the "Digging into Apache httpd's Source" series. In Part 9 we read the request line and headers off a socket, built a request_rec, and picked a virtual host from the Host header. This time we take that request_rec and follow how httpd decides "which file on disk is this URL, which settings apply to it, and who will produce the response". It is the most important junction in the whole series.

There are four questions to answer.

• Which phases does a request go through, in what order, and what gets decided in each one?
• Where, and in what order, are <Directory>, <Location>, <Files> and .htaccess from the config file combined for each request?
• Who picks the "handler", and what happens if nobody takes the request?
• How are features that "process a different URL instead", such as DirectoryIndex and ErrorDocument, implemented in the source? (Subrequests and internal redirects.)

Every line number in this post refers to that commit. Part 1 explains how to get the source at the same commit and build the Docker image used for the examples.

server/request.c is over 2,600 lines long, and the directory walk function alone is 800 of them. This post keeps only the main thread that someone seeing the code for the first time can follow. What was left out is named, with its location, under "What this post leaves out" at the end.

Words to know first

A hook is a spot the core leaves open, saying "when we get here, I will call every function registered for this point, one after another". A module registers its own function with something like ap_hook_translate_name(my_function, ...), and at that point the core calls ap_run_translate_name(r), which runs the registered functions. Function pointers, which let you pass a function around like a value, were covered in Part 2; the macros that build hooks and how their order is decided were covered in Part 6.

RUN_FIRST and RUN_ALL are the two ways of running a hook. A hook function usually returns OK (I handled it), DECLINED (not my job), or an HTTP error code (for example 404).

• RUN_FIRST: call them in order and stop at the first answer that is not DECLINED. It is "who wants to take this?", and the first hand up gets the job (APR-util apr_hooks.h lines 267–295).
• RUN_ALL: keep calling the next function while the answer is OK or DECLINED, and stop on an error. It is "everyone gets a say, but one objection ends it" (lines 222–250).

request_rec is the struct that holds everything about one request (Part 3). In this post we follow just four of its fields.

Per-directory settings (per-dir config) are the settings you write inside <Directory>, <Location>, <Files> and .htaccess. They are built separately for each section when the config file is read, and when a request arrives, the sections that match it are picked and merged. How one section becomes a C struct, and the merge function itself, were covered in Part 7. This post looks at "which sections are picked, and in what order, when a request comes in".

A subrequest is a small request created in the middle of handling a request, to find out in advance "what would happen if this URL had been requested". The client never knows about it. An internal redirect replaces the response with a different URL altogether and processes that from the start. Unlike a 301/302 redirect, which changes the address in the browser, this also happens entirely inside the server.

1. The big picture: three functions and a finish

The connection loop from Part 9 calls ap_process_async_request() (when an asynchronous MPM such as event is used) or ap_process_request() for every request it reads (modules/http/http_core.c line 159, line 212). The second one calls the first internally (line 486), so the real work is in ap_process_async_request(). Without the logging and statistics, its skeleton looks like this.

    access_status = ap_run_quick_handler(r, 0);  /* Not a look-up request */
    if (access_status == DECLINED) {
        access_status = ap_process_request_internal(r);
        if (access_status == OK) {
            access_status = ap_invoke_handler(r);
        }
    }
    /* ... */
    ap_die_r(access_status, r, HTTP_OK);

    ap_process_request_after_handler(r);
}

modules/http/http_request.c lines 447–477

The blue boxes are the stars of this post. The red box is the path taken only when there is an error.

1. The quick_handler hook — before any phase runs, it asks "does anyone want to take this whole request?". It is RUN_FIRST (server/config.c lines 172–173). It was made for caches keyed by URL, and mod_cache does indeed hook in here (modules/cache/mod_cache.c line 2605). If this returns OK, every phase below is skipped. Access control is skipped too, which is why the source comment says to use this hook "with extreme care" (lines 422–423).
2. ap_process_request_internal() — turns the URL into a file, merges the settings, checks access, and decides the content type. This is the heart of the post, from section 2 on.
3. ap_invoke_handler() — calls the handler that produces the response body. Covered in section 5.
4. ap_die_r() — despite the name, it is passed through on success as well. On OK or DONE it finishes the response with ap_finalize_request_protocol() and returns (lines 82–85). On an error code it sends an error response, and ErrorDocument lives on that path (section 6).

The final ap_process_request_after_handler() sends an EOR (End Of Request) bucket into the connection's output filters (lines 336–349). When that bucket is destroyed, the log_transaction hook that writes the access log is called (server/eor_bucket.c lines 27–46) and the request pool is destroyed. Buckets and filters are covered in Part 11, logging in Part 16.

Example 1: the call stack when the handler is reached

I checked with gdb that requests really take this path. I requested one static file, set a breakpoint on the core's default_handler(), which sends the file, and printed the call stack.

This example was run with httpd 2.5.1-dev built unchanged from the reference commit (e4af30fe6d), in a Docker container (Ubuntu 24.04). The key part of ex/u10-handler-stack/run.sh is this.

# -X: take requests in a single process instead of starting child processes (for debugging)
( sleep 2; curl -s http://127.0.0.1:8080/hello.html >/dev/null ) &
gdb -batch -q \
    -ex "set breakpoint pending on" \
    -ex "break default_handler" \
    -ex "run" \
    -ex "bt" \
    -ex "kill" \
    --args httpd -X -f /tmp/u10s.conf 2>&1 | grep -E "^#|Breakpoint 1,"

Thread 3 "httpd" hit Breakpoint 1, 0x000055beb9cfe8da in default_handler ()
#0  0x000055beb9cfe8da in default_handler ()
#1  0x000055beb9cc484d in ap_run_handler ()
#2  0x000055beb9cc53f9 in ap_invoke_handler ()
#3  0x000055beb9cb6e81 in ap_process_async_request ()
#4  0x000055beb9cafdd5 in ap_process_http_async_connection ()
#5  0x000055beb9cb0003 in ap_process_http_connection ()
#6  0x000055beb9cddc11 in ap_run_process_connection ()
#7  0x00007f05972698b2 in process_socket () from /opt/httpd/modules/mod_mpm_event.so
#8  0x00007f059726dbc7 in worker_thread () from /opt/httpd/modules/mod_mpm_event.so
#9  0x000055beb9cdaff9 in thread_start ()
#10 0x00007f05973c7b84 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#11 0x00007f0597454cc4 in clone () from /lib/x86_64-linux-gnu/libc.so.6

The addresses differ on every run. This build has no debug information, so only function names appear, without files or line numbers. Reading from the bottom up: the event MPM's worker thread (worker_thread, Part 8) → connection handling (Part 9) → ap_process_async_request() → ap_invoke_handler() → the handler hook.

Easy to get wrong: ap_process_request_internal() is not on the stack

It is the function that runs all the phases, yet it is missing from the stack. That is because it does not call the handler; it returns OK and finishes. The handler is called by ap_invoke_handler() on the next line. "The function that prepares" and "the function that runs" are separate, which is what lets the subrequests we will see later prepare without running.

2. The order of phases in ap_process_request_internal()

The comment above the function says how much weight it carries.

This is the master logic for processing requests. Do NOT duplicate this logic elsewhere, or the security model will be broken by future API changes.

(server/request.c lines 187–191) Whether it is a main request, a subrequest or an internal redirect, interpreting a URL always goes through this one function. That way there is no path that skips access control.

Each blue box is a place where a module can step in.

Here is the middle of the function, unchanged. It corresponds to the third to sixth boxes in the diagram.

    /* Same, translate_name is not suited for file subrequests */
    if (!file_req) {
        if ((access_status = walk_location_and_if(r))) {
            return access_status;
        }

        if ((access_status = ap_run_translate_name(r))) {
            return decl_die(access_status, "translate", r);
        }
    }

    /* Reset to the server default config prior to running map_to_storage
     */
    r->per_dir_config = r->server->lookup_defaults;

    if ((access_status = ap_run_map_to_storage(r))) {
        /* This request wasn't in storage (e.g. TRACE) */
        return access_status;
    }

    /* Rerun the location walk, which overrides any map_to_storage config.
     */
    if ((access_status = walk_location_and_if(r))) {
        return access_status;
    }

    if ((access_status = ap_run_post_perdir_config(r))) {
        return access_status;
    }

    /* Only on the main request! */
    if (r->main == NULL) {
        if ((access_status = ap_run_header_parser(r))) {
            return access_status;
        }
    }

server/request.c lines 277–312

Every step has the same shape: run a hook, and if the result is anything other than 0 (OK), return immediately with that value. ap_process_async_request() receives it and passes it to ap_die_r(), which builds the error response. In other words, if any phase returns 403 or 404, the request ends right there.

Here is the full list of phases. "Default" is the function the core hooks in for when no module takes the job.

I checked the kind of each hook in the source (server/request.c lines 79–108, server/config.c lines 85–86, server/protocol.c lines 2554–2557). The functions the core hooks in at the very end (APR_HOOK_REALLY_LAST) are in server/core.c lines 6164–6173.

Why the core stands at the end of every RUN_FIRST hook

The result of translate_name goes through decl_die(). If everyone returned DECLINED, this function treats it as a "configuration error" and returns 500 (lines 115–128). If nobody turns the URL into a file, there is no way forward. So for every RUN_FIRST phase where someone must answer, the core registers a function that always answers, in last place: ap_core_translate() for translate_name, core_map_to_storage() for map_to_storage, do_nothing() (which does nothing and just returns OK) for type_checker, and default_handler() for the handler. A source comment even says it would like to get rid of these do_nothings some day (line 6172).

Example 2: a module that logs every phase

To see the phases with my own eyes, I wrote a module that hooks the same function into every hook and prints r->uri, r->filename and r->handler at that moment. Every function returns DECLINED, so the server's behaviour does not change.

/* An example written for this post (not httpd's source) */
/* Hook into every request phase and print r->uri, r->filename and r->handler at each one. */
/* ... #includes omitted ... */

/* Is this the main request, a subrequest (r->main) or an internal redirect (r->prev)? */
static const char *kind(request_rec *r)
{
    if (r->main) return "sub     ";
    if (r->prev) return "redirect";
    return "main    ";
}

static void trace(request_rec *r, const char *phase)
{
    ap_log_rerror(APLOG_MARK, APLOG_NOTICE, 0, r,
                  "%s %-18s uri=%s filename=%s handler=%s",
                  kind(r), phase, r->uri,
                  r->filename ? r->filename : "(null)",
                  r->handler ? r->handler : "(null)");
}

/* Stamp out one function per phase with a macro. They all return DECLINED —
 * meaning "not my business, pass it to the next module", so the server's behaviour is unchanged. */
#define PHASE(name) \
    static int journey_##name(request_rec *r) { trace(r, #name); return DECLINED; }

PHASE(post_read_request)
PHASE(pre_translate_name)
PHASE(translate_name)
/* ... the same shape from map_to_storage to handler ... */

static void register_hooks(apr_pool_t *p)
{
    /* REALLY_FIRST so we run before other modules — each line shows the values "on entering that phase" */
    ap_hook_post_read_request(journey_post_read_request, NULL, NULL, APR_HOOK_REALLY_FIRST);
    ap_hook_quick_handler(journey_quick_handler, NULL, NULL, APR_HOOK_REALLY_FIRST);
    ap_hook_pre_translate_name(journey_pre_translate_name, NULL, NULL, APR_HOOK_REALLY_FIRST);
    ap_hook_translate_name(journey_translate_name, NULL, NULL, APR_HOOK_REALLY_FIRST);
    /* ... the remaining hooks are registered the same way ... */
}

[Shortened from the example ex/u10-request-phases/mod_journey.c]

#name turns a macro argument into a string and journey_##name glues names together; both are C preprocessor syntax (Part 2). The config has DocumentRoot /tmp/htdocs, Require all granted inside <Directory /tmp/htdocs>, and ErrorLogFormat "%M" so that the error log shows only the message. Here is the result of requesting /hello.html.

===== curl http://127.0.0.1:8080/hello.html =====
HTTP 200  body: hello
main     post_read_request  uri=/hello.html filename=(null) handler=(null)
main     quick_handler      uri=/hello.html filename=(null) handler=(null)
main     pre_translate_name uri=/hello.html filename=(null) handler=(null)
main     translate_name     uri=/hello.html filename=(null) handler=(null)
main     map_to_storage     uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     header_parser      uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     access_checker     uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     type_checker       uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     fixups             uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     insert_filter      uri=/hello.html filename=/tmp/htdocs/hello.html handler=(null)
main     handler            uri=/hello.html filename=/tmp/htdocs/hello.html handler=text/html
main     log_transaction    uri=/hello.html status=200

There are three things to read here.

• r->filename appears while passing through translate_name. My function is called first, so on the translate_name line it is still (null), and in the next phase, map_to_storage, it is /tmp/htdocs/hello.html. In between, the core's ap_core_translate() filled it in (section 3).
• check_user_id and auth_checker are missing. Where only Require all granted applies there is no need to identify the user, so those two phases are skipped. We look at this just below.
• r->handler gets its first value only in the handler phase. And that value is text/html, the same string as the Content-Type. Section 5 explains why.

Authentication and authorization: just the order

Phase 6 is really several hooks. With Satisfy All (the default) the order is as follows (lines 339–368).

1. token_checker — token authentication (for example Bearer). This hook exists only in trunk; server/request.c on the 2.4.x branch does not have it.
2. access_checker — checks that do not involve a user, such as IP address. RUN_ALL.
3. access_checker_ex — RUN_FIRST. This is where mod_authz_core decides Require. If access is granted without a user it returns OK; if a user is needed it returns DECLINED.
4. Only when step 3 returned DECLINED (or returned OK but a force_authn hook demands authentication) are check_user_id (who is it — authentication) and auth_checker (are they allowed — authorization) called, in that order.

The two phases were missing in Example 2 because Require all granted returned OK at step 3. Requesting a password-protected directory (AuthType Basic, Require valid-user) makes them appear.

===== curl -u alice:wonderland http://127.0.0.1:8080/private/secret.txt =====
HTTP 200  body: secret
main     post_read_request  uri=/private/secret.txt filename=(null) handler=(null)
main     quick_handler      uri=/private/secret.txt filename=(null) handler=(null)
main     pre_translate_name uri=/private/secret.txt filename=(null) handler=(null)
main     translate_name     uri=/private/secret.txt filename=(null) handler=(null)
main     map_to_storage     uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     header_parser      uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     access_checker     uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     check_user_id      uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     auth_checker       uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     type_checker       uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     fixups             uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     insert_filter      uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=(null)
main     handler            uri=/private/secret.txt filename=/tmp/htdocs/private/secret.txt handler=text/plain
main     log_transaction    uri=/private/secret.txt status=200

How a single Require line is decided, and how the order changes with Satisfy Any, is the subject of Part 13.

Easy to get wrong

• Authentication comes after the settings. What to require from whom is written inside <Directory> or <Location>, so it can only be decided after the settings have been merged. That is why translate_name and map_to_storage come first.
• There are three location walks. Before pre_translate_name (line 230), before translate_name (line 279) and after map_to_storage (line 299). The first two let those hooks see settings inside <Location>; the result of the first is thrown away straight after (lines 242–243, "Throw away pre_trans only merging"). Because line 290 resets r->per_dir_config to the server default, the result of the second does not survive into the final settings either. Only the third one ends up in the final settings.
• header_parser is called for the main request only (lines 307–312). It is skipped for subrequests, which we confirm in the section 6 example.

3. From URL to file: translate_name and map_to_storage

ap_core_translate() — append the URI to DocumentRoot

The heart of the core function that answers last for translate_name is a single call.

    if ((rv = apr_filepath_merge(&r->filename, ap_document_root(r), path,
                                 APR_FILEPATH_TRUENAME
 APR_FILEPATH_SECUREROOT, r->pool))
                != APR_SUCCESS) {
        ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, APLOGNO(00127)
                     "Cannot map %s to file", r->the_request);
        return HTTP_FORBIDDEN;
    }
    r->canonical_filename = r->filename;

    return OK;

server/core.c lines 5079–5089

It joins DocumentRoot (/tmp/htdocs) and the URI with its leading / removed (hello.html) using APR's path-merging function. APR_FILEPATH_SECUREROOT is a flag that makes the merge fail if the result would land outside DocumentRoot. ap_process_request_internal() has already cleaned up /../ earlier (lines 209–221), but this blocks it once more.

Where mod_alias and mod_rewrite step in

A setting like Alias /icons/ /usr/share/icons/ works by answering before the core in this phase.

    ap_hook_translate_name(translate_alias_redir,NULL,aszSucc,APR_HOOK_MIDDLE);
    ap_hook_fixups(fixup_redir,NULL,NULL,APR_HOOK_MIDDLE);

modules/mappers/mod_alias.c lines 788–789

mod_alias is MIDDLE and the core is REALLY_LAST, so mod_alias is called first. If an Alias matches, it sets r->filename and returns OK (lines 690–695). Since this is RUN_FIRST, the core's ap_core_translate() is not called at all. If a Redirect matches, it adds a Location header and returns a status such as 302; as we saw in section 2, the request ends right there and the browser gets a redirect response.

mod_rewrite hooks in with APR_HOOK_FIRST, plus the condition "before mod_proxy.c" (mod_rewrite.c lines 5765–5769). RewriteRules in the server config run in this translate_name phase, while RewriteRules inside .htaccess or <Directory> run in fixups, because the per-directory settings are only known once map_to_storage is done. When a URL is rewritten in fixups, all the phases have already been passed, so it prefixes r->filename with redirect: and sets the handler to its own (lines 5654–5655); then, in the handler phase, it starts again from the top with the internal redirect from section 6 (lines 5700–5717). How mod_rewrite processes its rules is not covered in this post.

core_map_to_storage() — two walks

The core's handler for the map_to_storage phase is short.

static int core_map_to_storage(request_rec *r)
{
    int access_status;

    if ((access_status = ap_directory_walk(r))) {
        return access_status;
    }

    if ((access_status = ap_file_walk(r))) {
        return access_status;
    }

    return OK;
}

server/core.c lines 5096–5109

As the name says, this is the phase that "maps to storage (disk)". A module that deals with something other than disk files answers here first and skips the directory walk. mod_proxy hooking in here with APR_HOOK_FIRST is an example (mod_proxy.c line 3437, Part 14).

Blue boxes are modules that can answer before the core; yellow boxes are the core, standing last. The path comes from the map_to_storage line of example 2.

4. The three walks — where settings are merged

A "walk" goes through a list of sections from the config file from the beginning, picks the ones that match this request, and merges them in turn. There are three, and each looks at a different list using a different key.

When the config file is read, each <Directory> section is pushed onto the server config's sec_dir array, and each <Location> onto the sec_url array (server/core.c lines 639–653). A <Files> section goes into sec_file of the directory config that encloses it (lines 655–665). The walk functions go through these arrays.

Yellow boxes are the starting point and the result; blue boxes are the walks that override one after another. The further down, the later it is merged, so the stronger it is.

Directory walk — going down one directory at a time

For /tmp/htdocs/docs/index.html, ap_directory_walk() goes down one level at a time — /, /tmp/, /tmp/htdocs/, /tmp/htdocs/docs/ — and does two things at each level.

1. It finds the <Directory> sections that match that depth and merges them (lines 1018–1084). After the whole config has been read, the section list is sorted by the number of path components, fewest first, with regexes at the end (server/core.c lines 709–741). That is why <Directory /> is merged before <Directory /tmp/htdocs> no matter which comes first in the config file.
2. If AllowOverride permits it, it reads that directory's .htaccess and merges it (lines 1086–1110).

Regex sections (<DirectoryMatch>, <Directory ~ ...>) are scanned and merged separately, once, after the descent is finished (lines 1332–1409). While walking it calls stat on the disk, and whatever is left after the real file is split off into r->path_info (the /extra in /script.cgi/extra).

Location walk — and the final merge

ap_location_walk() does not look at the disk; it compares the r->uri string with each section's path. All three walks end the same way: build now_merged by merging the chosen sections in turn, then merge that on top of the current r->per_dir_config once more.

    cache->dir_conf_tested = sec_ent;
    cache->dir_conf_merged = r->per_dir_config;

    /* Merge our cache->dir_conf_merged construct with the r->per_dir_configs,
     * and note the end result to (potentially) skip this step next time.
     */
    if (now_merged) {
        r->per_dir_config = ap_merge_per_dir_configs(r->pool,
                                                     r->per_dir_config,
                                                     now_merged);
    }
    cache->per_dir_result = r->per_dir_config;

    return OK;
}

server/request.c lines 1674–1688

ap_merge_per_dir_configs() is the merge function from Part 7. It calls the merge_dir_config function each module registered to build "old settings + new settings" (server/config.c line 285). Because of the order we saw in section 2 — the location walk after map_to_storage (the directory and file walks) — <Location> overrides <Directory>. That is exactly what the comment on line 299 means: "Rerun the location walk, which overrides any map_to_storage config". <If> sections are merged by ap_if_walk() right after the location walk (walk_location_and_if() lines 166–190).

The lines starting with cache are a cache that avoids redoing the merge within the same request, or when a subrequest or internal redirect ends up choosing the same sections as its parent. It is kept per request in a struct called walk_cache_t (lines 455–468). For this post it is enough to know it exists; in section 6 we see one of its effects.

Easy to get wrong: a missing file does not produce a 404 here

When ap_directory_walk() finds that a file does not exist on the way down, it just stops walking; it does not return an error (break on APR_ENOENT, lines 1228–1234). Even a request for a missing file goes through all the phases. The 404 is decided at the very end, by default_handler().

    if (r->method_number == M_GET || r->method_number == M_POST) {
        if (r->finfo.filetype == APR_NOFILE) {
            ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(00128)
                          "File does not exist: %s",
                          apr_pstrcat(r->pool, r->filename, r->path_info, NULL));
            return HTTP_NOT_FOUND;
        }

server/core.c lines 5192–5198

There is a reason for deciding this so late: even if there is no file, another module (say, a <Location> with its own handler, or mod_dir from section 6) may still take that URL. In Example 4 in section 6 we confirm this by seeing the AH00128 log line appear after the handler phase.

5. ap_invoke_handler() — picking and calling the handler

When ap_process_request_internal() returns OK, ap_invoke_handler() is called. Here is the core of it.

AP_CORE_DECLARE(int) ap_invoke_handler(request_rec *r)
{
    /* ... */
    ap_run_insert_filter(r);
    /* ... call the filters' init functions ... */
    if (!r->handler) {
        if (r->content_type && AP_REQUEST_IS_TRUSTED_CT(r)) {
            handler = r->content_type;
            /* ... cut off whatever follows ; in "text/html; charset=..." ... */
        }
        else {
            handler = AP_DEFAULT_HANDLER_NAME;
        }

        r->handler = handler;
    }

    result = ap_run_handler(r);

    r->handler = old_handler;
    /* ... */
    return result == DECLINED ? HTTP_INTERNAL_SERVER_ERROR : result;
}

server/config.c lines 389–470

1. The insert_filter hook — since we are about to produce the real response, this is the modules' last chance to insert output filters (line 405). It is a VOID hook with no return value (server/request.c line 97).
2. Deciding the handler name — if r->handler was set by SetHandler or AddHandler, it is used as is. Otherwise the Content-Type is used as the handler name (lines 420–441). That is why Example 2 printed handler=text/html. In the type_checker phase, mod_mime saw .html and set Content-Type to text/html (marking it as a trusted value, mod_mime.c line 921), and that became the handler name. If there is no content type either, the name is the empty string (AP_DEFAULT_HANDLER_NAME, include/httpd.h line 731).
3. The handler hook — RUN_FIRST (server/config.c lines 169–170). Handler modules usually start by comparing r->handler with their own name and return DECLINED if it differs. mod_status looks for "server-status" (mod_status.c lines 217–219), and mod_autoindex for its directory-listing name. A request nobody takes is caught by default_handler() at the very end, which sends the file as is. No module volunteers for a handler called text/html, so in our example /hello.html was sent by default_handler(); the stack in Example 1 is the proof.
4. If everyone returns DECLINED, the result is 500 (line 469).

The four items of the list above drawn as branches. The yellow boxes are the path the example's /hello.html actually took.

How default_handler() turns a file into a response, and how mod_mime interprets file extensions, is covered in Part 12.

Easy to get wrong: the handler name is put back afterwards

r->handler = old_handler; on line 445 clears the default handler name chosen inside this function once the handler has finished. So code that runs later, such as a logging module, may see r->handler as NULL. The fact that the handler name came from the Content-Type is only true inside ap_invoke_handler().

6. Subrequests and internal redirects

Both features "run the phases from section 2 for another URL", but their purpose and result differ.

On the left, the subrequest that only "finds out"; on the right, the internal redirect that "processes again".

Subrequests: finding out "what would happen with this URL"

ap_sub_req_lookup_uri() sets the method to GET and calls ap_sub_req_method_uri() (lines 2346–2351). make_sub_request() creates a child pool called "subrequest" under the parent's pool, places a new request_rec in it (lines 2056–2059), and records the parent in r->main (server/protocol.c line 1612). Then comes the main part.

    if (next_filter) {
        res = ap_run_quick_handler(rnew, 1);
    }
    if (res == DECLINED) {
        res = ap_process_request_internal(rnew);
    }
    if (res) {
        rnew->status = res;
    }

    return rnew;
}

server/request.c lines 2333–2344

It has the same shape as the main request in section 1, but there is no ap_invoke_handler(). It only runs the phases and returns results such as rnew->status, rnew->filename and rnew->finfo (the file information). The caller looks at the result and decides. To actually produce a response, the caller has to call ap_run_sub_req() separately (lines 2577–2594), and when it is done it calls ap_destroy_sub_req() to throw away the whole child pool. Destroying a pool removes everything in it at once, as we saw in Part 4. The second argument 1 passed to quick_handler marks it as "just looking up".

DirectoryIndex is implemented with this. In the fixups phase, if the request points to a directory, mod_dir makes a subrequest for each name listed in DirectoryIndex.

        rr = ap_sub_req_lookup_uri(name_ptr, r, r->output_filters);
        /* ... */
        if (rr->status == HTTP_OK
            && (   (rr->handler && !strcmp(rr->handler, "proxy-server"))
| rr->finfo.filetype == APR_REG)) {

            if (ap_is_HTTP_REDIRECT(d->redirect_index)) {
                apr_table_setn(r->headers_out, "Location", ap_construct_url(r->pool, rr->uri, r));
                return d->redirect_index;
            }

            ap_internal_fast_redirect(rr, r);
            return OK;
        }

modules/mappers/mod_dir.c lines 346–369

If the subrequest is 200 and points to a regular file (APR_REG), ap_internal_fast_redirect() copies the subrequest's result into the main request. This function swaps in the subrequest's r->uri, r->filename, r->finfo, r->handler and r->per_dir_config (http_request.c lines 702–712). It takes only the result without running the phases again, hence "fast".

Example 3: a directory request and mod_dir's subrequest

I requested /docs/. /tmp/htdocs/docs/index.html exists, and the config has DirectoryIndex index.html.

===== curl http://127.0.0.1:8080/docs/ =====
HTTP 200  body: docs index
main     post_read_request  uri=/docs/ filename=(null) handler=(null)
main     quick_handler      uri=/docs/ filename=(null) handler=(null)
main     pre_translate_name uri=/docs/ filename=(null) handler=(null)
main     translate_name     uri=/docs/ filename=(null) handler=(null)
main     map_to_storage     uri=/docs/ filename=/tmp/htdocs/docs/ handler=(null)
main     header_parser      uri=/docs/ filename=/tmp/htdocs/docs/ handler=(null)
main     access_checker     uri=/docs/ filename=/tmp/htdocs/docs/ handler=(null)
main     type_checker       uri=/docs/ filename=/tmp/htdocs/docs/ handler=(null)
main     fixups             uri=/docs/ filename=/tmp/htdocs/docs/ handler=(null)
sub      quick_handler(lookup) uri=/docs/index.html filename=(null) handler=(null)
sub      pre_translate_name uri=/docs/index.html filename=(null) handler=(null)
sub      translate_name     uri=/docs/index.html filename=(null) handler=(null)
sub      map_to_storage     uri=/docs/index.html filename=/tmp/htdocs/docs/index.html handler=(null)
sub      type_checker       uri=/docs/index.html filename=/tmp/htdocs/docs/index.html handler=(null)
sub      fixups             uri=/docs/index.html filename=/tmp/htdocs/docs/index.html handler=(null)
main     insert_filter      uri=/docs/index.html filename=/tmp/htdocs/docs/index.html handler=(null)
main     handler            uri=/docs/index.html filename=/tmp/htdocs/docs/index.html handler=text/html
main     log_transaction    uri=/docs/index.html status=200

What we read in the source shows up line by line.

• The subrequest is created inside the main request's fixups. My fixups function (REALLY_FIRST) prints first, and mod_dir (APR_HOOK_LAST, line 433), called after it, made the subrequest.
• The subrequest has no post_read_request, header_parser, insert_filter or handler. quick_handler was called with the lookup mark. The subrequest was not read from a socket, header_parser is called for main requests only (lines 307–312), and the handler was never run.
• The subrequest has no access_checker either. See "Easy to get wrong" below.
• After that, the main request's uri changed to /docs/index.html. That is the result copied by ap_internal_fast_redirect(). The main request does not run the phases again and goes straight to insert_filter and handler. The uri that ends up in the access log is the changed value too.

Internal redirects: starting over

ErrorDocument 404 /errors/404.html is handled inside ap_die_r(). If the custom response for an error code is a local path starting with /, it changes the method to GET and calls ap_internal_redirect().

            r->method = "GET";
            r->method_number = M_GET;
            ap_internal_redirect(custom_response, r);
            /* preserve ability to see %<m in the access log */
            r->method = original_method;
            r->method_number = original_method_number;
            return;

modules/http/http_request.c lines 199–205

The body of ap_internal_redirect() is the very same three steps as in section 1.

    access_status = ap_run_quick_handler(new, 0);  /* Not a look-up request */
    if (access_status == DECLINED) {
        access_status = ap_process_request_internal(new);
        if (access_status == OK) {
            access_status = ap_invoke_handler(new);
        }
    }
    ap_die(access_status, new);

modules/http/http_request.c lines 784–791

Before that, internal_internal_redirect() prepares the new request_rec (lines 526–687). Unlike a subrequest, it is created in the same pool as the parent (lines 537–541), the two are linked with new->prev = r and r->next = new (lines 560–561), and environment variables are passed on with REDIRECT_ prepended to their names (line 606, rename_original_env()). This is where the REDIRECT_STATUS and REDIRECT_URL you see in CGI come from. Then the post_read_request hook is called again (line 681). The comment explains that mod_setenvif and mod_unique_id need to do their work on internal redirects as well.

To stop redirects from going on forever, ap_is_recursion_limit_exceeded() counts the depth (line 532). The defaults are 10 redirects and a subrequest depth of 10 (include/http_core.h lines 134–137). The LimitInternalRecursion directive changes them.

Example 4: a missing file and ErrorDocument

/nope.html does not exist. The config has ErrorDocument 404 /errors/404.html, and LogLevel core:info makes AH00128 visible.

===== curl http://127.0.0.1:8080/nope.html =====
HTTP 404  body: custom 404 page
main     post_read_request  uri=/nope.html filename=(null) handler=(null)
main     quick_handler      uri=/nope.html filename=(null) handler=(null)
main     pre_translate_name uri=/nope.html filename=(null) handler=(null)
main     translate_name     uri=/nope.html filename=(null) handler=(null)
main     map_to_storage     uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     header_parser      uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     access_checker     uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     type_checker       uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     fixups             uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     insert_filter      uri=/nope.html filename=/tmp/htdocs/nope.html handler=(null)
main     handler            uri=/nope.html filename=/tmp/htdocs/nope.html handler=text/html
AH00128: File does not exist: /tmp/htdocs/nope.html
redirect post_read_request  uri=/errors/404.html filename=(null) handler=(null)
redirect quick_handler      uri=/errors/404.html filename=(null) handler=(null)
redirect pre_translate_name uri=/errors/404.html filename=(null) handler=(null)
redirect translate_name     uri=/errors/404.html filename=(null) handler=(null)
redirect map_to_storage     uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect header_parser      uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect access_checker     uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect type_checker       uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect fixups             uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect insert_filter      uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=(null)
redirect handler            uri=/errors/404.html filename=/tmp/htdocs/errors/404.html handler=text/html
main     log_transaction    uri=/nope.html status=404

• Even though the file is missing, the request passes through map_to_storage and type_checker (text/html, because of .html) all the way to handler. AH00128 is printed after handler. As we saw in section 4, the 404 is decided by default_handler().
• Then the redirect request runs every phase again from post_read_request. Unlike the subrequest, it has header_parser (because r->main is NULL), insert_filter and handler.
• The client got status 404 with the body custom 404 page. The address did not change.
• log_transaction runs once, and for the original request (uri=/nope.html). That is because the EOR bucket is created with the original r (http_request.c lines 348–357).

Easy to get wrong: when access checks are skipped

The subrequest in Example 3 had no access_checker, while the redirect in Example 4 did. The difference comes from this condition.

    if (r->prev && (r->prev->per_dir_config == r->per_dir_config)) {
        r->user = r->prev->user;
        r->ap_auth_type = r->prev->ap_auth_type;
    }
    else if (r->main && (r->main->per_dir_config == r->per_dir_config)) {
        r->user = r->main->user;
        r->ap_auth_type = r->main->ap_auth_type;
    }
    else {

server/request.c lines 319–327

If the merged settings are the same object (the same pointer) as the parent's (r->main or r->prev), the parent has already passed the checks, so the request just inherits the user information and skips the whole authentication and authorization block. The subrequest for index.html in the same directory got the same settings object as its parent thanks to the walk cache from section 4, so it skipped them. The redirect that went to the /errors/ directory got a freshly merged object, so it ran the checks again. Exactly when the same object comes out depends on details of the walk cache (auth_internal_per_conf and others, lines 2176–2212), which this post does not go into. What to remember is the principle: "if the settings are the same, don't check again; if the settings changed, always check again" (the comment on lines 314–318).

When a module wants to know "is this the original request the client sent?", it uses ap_is_initial_req(). It is true when there is neither r->main nor r->prev (lines 2616–2620).

What this post leaves out

• Path normalisation and %2F handling (AllowEncodedSlashes, MergeSlashes): server/request.c lines 199–275
• File subrequests (ap_sub_req_lookup_dirent, ap_sub_req_lookup_file): from lines 2353 and 2482
• Symbolic link checks (Options FollowSymLinks, resolve_symlink()): from line 552
• How the walk cache (walk_cache_t, prep_walk_cache()) works in detail: lines 455–520
• The <If> walk (ap_if_walk()): lines 1887–2032
• The post_perdir_config, dirwalk_stat and force_authn hooks: lines 79–108
• Asynchronous processing where a handler returns SUSPENDED to let go of the request for a while: http_request.c lines 455–469
• ap_send_error_response(), which builds the body of an error response: modules/http/http_protocol.c

Recap

What to look at next

We have followed a request from a URL to a file and a handler. Once the handler is chosen, the response body is produced and flows out. A handler does not write to the socket directly; it puts data into buckets and passes them down a chain of filters. Those are the things this post only named in passing as the insert_filter hook and the EOR bucket. The story continues in Part 11: Filters and Bucket Brigades — How a Response Flows Out.

Spring MVC solves the same problem. Spring Part 21's DispatcherServlet asks its list of HandlerMappings one by one in getHandler() and uses the first one that answers — the same idea as httpd's RUN_FIRST hooks (translate_name, handler). Collecting errors raised during processing in one place (processDispatchResult()) and turning them into an error response also resembles ap_die_r(). The difference is that Spring turns a URL into a Java method, while httpd turns a URL into a path on disk and the settings attached to that path. That is why httpd has, for every request, the "walks" and settings merges that Spring does not.

• Part 6: Modules and Hooks — How a Module Plugs into the Server
• Part 7: The Config File and Directives — How One Line of httpd.conf Becomes a C Struct
• Part 9: Connections and Reading a Request — From a Socket to a request_rec
• Part 12: Handlers and Static Files — How a File Becomes a Response
• Part 13: Authentication and Authorization — How a Require Line Is Decided