Digging into Apache httpd's Source 3: The Big Picture — Core, APR, MPM, Modules and the Three Big Structs

What this post covers

This is the third post in "Digging into Apache httpd's Source". In Part 1 we fetched and built the source and unfolded a map of its folders, and in Part 2 we got comfortable with the structs, function pointers and macros you need to read a large C program like httpd. This post is the big-picture map you spread out before reading the code line by line.

Open the httpd repository and you find hundreds of C files in folders such as server/, modules/, include/ and os/. At first you can't tell where the center is or what calls what. This post answers these questions:

• What layers is httpd made of, and which folder of the repository holds each layer?
• What does "the core is also a module" mean? How is a module compiled into the binary different from one you load with LoadModule?
• What are server_rec, conn_rec and request_rec, which show up everywhere in the source, and how do they point to each other?
• How are the memory pools arranged, and what path does a single request take through the server?

All line numbers in this post refer to this commit. Part 1 shows how to get the same commit and build the Docker image used for the examples.

Since this post draws a map, it follows only the main lines. Each concept points ahead to the post that reads it in detail ("we'll see this in Part N").

Terms to know first

A module is a bundle of code that adds one feature to httpd. Names start with mod_: mod_mime (decides the Content-Type from the file extension), mod_dir (DirectoryIndex), mod_proxy (reverse proxy). In the source, a module is represented by a single C struct called module.

A DSO (Dynamic Shared Object) is a library file that can be loaded while a program is running. On Linux its extension is .so. One LoadModule line in httpd.conf loads one such .so file into the server process.

APR (Apache Portable Runtime) is a C library that wraps features that differ between operating systems (memory, files, sockets, threads, loading shared libraries, and so on) behind the same function names. It is developed and installed separately from httpd. Its functions start with apr_. Features such as buckets and hooks live in its companion library, APR-util.

An MPM (Multi-Processing Module) decides how many processes and threads to create and who handles each incoming connection. event, worker and prefork are the best known. As the name says, it is a module too.

A pool is APR's way of managing memory. Instead of calling free() on each piece of memory, you take memory from a pool and later destroy the whole pool at once. httpd ties lifetimes together this way: one pool per request, one pool per connection. Part 4 covers pools in detail.

A hook is a spot the core leaves open, saying "I'll accept functions that want to step in at this point." Modules register their functions on hooks, and when that point comes the core calls the registered functions in order. Part 6 shows how hooks are registered. If C function pointers and structs feel unfamiliar, Part 2 is a good place to start.

1. Five layers on one page

Split into layers, httpd looks like this.

An arrow means "the upper box uses the lower one." The core, with the thick border, sits in the middle, defining hooks and calling them.

• Operating system. Linux, Windows and so on. The os/ folder of the httpd repository holds small bits of per-OS glue code (os/unix, os/win32, etc.).
• APR · APR-util. The floor laid on top of the OS. Basic tools such as pools, sockets, files, threads, DSO loading, buckets and hooks live here. It is a separate project outside the httpd repository.
• Core. The server/ folder. It holds 41 C files, including main.c (where main() lives), config.c (reads the config file), protocol.c (reads requests) and request.c (defines the request-processing phases). The core owns the order of "what happens when," and opens each step of that order as a hook.
• MPM. event, worker, prefork and others under server/mpm/. It creates processes and threads, accepts connections and hands them to the core.
• Modules. The modules/ folder, split into 26 subfolders such as aaa (authentication and authorization), mappers (URL to file), filters, proxy and ssl. Modules add features by registering functions on the core's hooks.

Besides these, include/ holds the headers every layer shares (httpd.h, http_config.h, etc.), and support/ holds tools such as ab and htpasswd. Part 1 has a more detailed folder map.

What APR does: hiding OS differences behind one function

One piece of code shows nicely why APR exists. When LoadModule loads a .so file, the function it ends up calling is APR's apr_dso_load().

APR_DECLARE(apr_status_t) apr_dso_load(apr_dso_handle_t **res_handle, 
                                       const char *path, apr_pool_t *pool)
{
#if defined(DSO_USE_SHL)
    shl_t os_handle = shl_load(path, BIND_IMMEDIATE, 0L);

#elif defined(DSO_USE_DYLD)
    /* ... */
#elif defined(DSO_USE_DLFCN)
    /* ... */
    os_handle = dlopen(path, flags);
#endif    
#endif /* DSO_USE_x */

dso/unix/dso.c lines 80–141 (APR 1.7.2)

#if defined(...), #elif and #endif are C preprocessor directives: before compiling, only the lines whose condition holds are kept. The same function contains code for HP-UX (shl_load), old macOS (NSLinkModule) and most Unix systems including Linux (dlopen), and only the branch for the OS being built survives. The Windows version lives in a different file entirely (dso/win32/dso.c, using LoadLibraryExW). Whatever the OS, httpd only has to call apr_dso_load().

You can also see for yourself that APR is a separate library and not part of the httpd binary. In the example container I used ldd to list the shared libraries the httpd binary depends on.

### 0. httpd 실행 파일이 기대는 공유 라이브러리 (APR 줄만)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0
	libaprutil-1.so.0 => /lib/x86_64-linux-gnu/libaprutil-1.so.0
	libapr-1.so.0 => /lib/x86_64-linux-gnu/libapr-1.so.0

All examples in this post were run in a Docker container (Ubuntu 24.04) with httpd 2.5.1-dev built unchanged from the reference commit (e4af30fe6d). Everything is in a single script, ex/u03-architecture/run.sh, and the ### lines in the output are headings the script prints. Those headings are in Korean because the script is shared with the Korean edition; the first one here reads "shared libraries the httpd binary depends on (APR lines only)," and I translate each one as we go.

libapr-1 and libaprutil-1 are APR installed from Ubuntu packages. The regular-expression library PCRE2 is separate too.

Easy to get confused: the arrows don't all point one way

The layer diagram is drawn as "upper uses lower," but between the core and the MPM or modules, calls go both ways. Modules call functions the core provides (starting with ap_), and the core calls module functions through hooks. The core doesn't know any module by name; it only knows "the functions registered on this hook." That's why you can remove or add modules without recompiling the core. The same goes for the MPM: at the end, main() merely runs a hook called ap_run_mpm(), and the config file decides which MPM is registered on it (section 3).

2. The core is a module too

One thing that surprises people reading the httpd source: even the part we call "the core" is registered with the very same module struct as every other module.

One module = one module struct

Every module has one instance of this struct, defined in include/http_config.h.

struct module_struct {
    /* ... */
    int version;
    /* ... */
    int module_index;

    /* ... */
    const char *name;
    /* ... */
    unsigned long magic;

    /* ... */
    void *(*create_dir_config) (apr_pool_t *p, char *dir);
    /* ... */
    void *(*create_server_config) (apr_pool_t *p, server_rec *s);
    /* ... */
    const command_rec *cmds;

    /* ... */
    void (*register_hooks) (apr_pool_t *p);
    /* ... */
};

include/http_config.h lines 348–424

I kept only the fields this post needs.

A field written with (* before its name, like void *(*create_dir_config) (...), is a function pointer: a slot that holds the address of a function. Each module puts its own function in the slot, and the core calls that function through the slot. It is C's way of doing what you'd do in Java by implementing an interface method (Part 2).

The core's module: core_module

The core's module struct sits at the very end of server/core.c.

AP_DECLARE_MODULE(core) = {
    MPM20_MODULE_STUFF,
    AP_PLATFORM_REWRITE_ARGS_HOOK, /* hook to run before apache parses args */
    create_core_dir_config,       /* create per-directory config structure */
    merge_core_dir_configs,       /* merge per-directory config structures */
    create_core_server_config,    /* create per-server config structure */
    merge_core_server_configs,    /* merge per-server config structures */
    core_cmds,                    /* command apr_table_t */
    register_hooks                /* register hooks */
};

server/core.c lines 6216–6225

AP_DECLARE_MODULE(core) is a macro that declares a variable named module core_module (http_config.h lines 467–469). MPM20_MODULE_STUFF is a macro that fills in the leading fields in one go: version, module_index = -1, name = __FILE__, magic and so on (http_config.h lines 499–505). After that come the functions that create directory and server configs, the directive table core_cmds (basic directives such as DocumentRoot, <Directory> and <VirtualHost>), and register_hooks.

Open the core's register_hooks and you see the core registering functions on hooks just like any other module. A few lines:

static void register_hooks(apr_pool_t *p)
{
    /* ... */
    ap_hook_create_connection(core_create_conn, NULL, NULL,
                              APR_HOOK_REALLY_LAST);
    /* ... */
    ap_hook_translate_name(ap_core_translate,NULL,NULL,APR_HOOK_REALLY_LAST);
    /* ... */
    ap_hook_handler(core_upgrade_handler,NULL,NULL,APR_HOOK_REALLY_FIRST);
    ap_hook_handler(default_handler,NULL,NULL,APR_HOOK_REALLY_LAST);
    /* ... */
}

server/core.c lines 6141–6214

• core_create_conn builds a conn_rec when a connection comes in (sections 5 and 6).
• ap_core_translate turns a URL into a file path under DocumentRoot (Part 10).
• default_handler sends out static files (Part 12).

All three are registered with APR_HOOK_REALLY_LAST, meaning "run at the very end." If another module handles the job first, its result wins; only when nobody else does does the core apply its default behavior. The core gets no special treatment; it behaves as one module that supplies the defaults.

HTTP is a module too: http_module

Even more surprising: HTTP protocol handling is not part of the core either, but a separate module. Its file isn't even under server/; it's in modules/http/.

AP_DECLARE_MODULE(http) = {
    STANDARD20_MODULE_STUFF,
    NULL,              /* create per-directory config structure */
    NULL,              /* merge per-directory config structures */
    NULL,              /* create per-server config structure */
    NULL,              /* merge per-server config structures */
    http_cmds,         /* command apr_table_t */
    register_hooks     /* register hooks */
};

modules/http/http_core.c lines 396–404

This module's register_hooks uses ap_hook_process_connection(ap_process_http_connection, ...) to register "the function that reads and handles HTTP/1.x requests when a connection arrives" (http_core.c lines 355–359). Filters such as HTTP_HEADER (writes the response headers) and CHUNK (chunked encoding) are registered by this module too. The core only knows "the hook that processes a connection"; whether that connection speaks HTTP is up to this module. That's why a module speaking a different protocol can step in on the same hook.

Other "core-like" modules

The MPM's module struct has exactly the same shape.

AP_DECLARE_MODULE(mpm_event) = {
    MPM20_MODULE_STUFF,
    NULL,                       /* hook to run before apache parses args */
    NULL,                       /* create per-directory config structure */
    NULL,                       /* merge per-directory config structures */
    NULL,                       /* create per-server config structure */
    NULL,                       /* merge per-server config structures */
    event_cmds,                 /* command apr_table_t */
    event_hooks                 /* register_hooks */
};

server/mpm/event/event.c lines 4535–4544

Inside event_hooks, ap_hook_mpm(event_run, ...) registers "the function that runs the server's main loop" on the mpm hook (event.c line 4381). Directives such as StartServers and ThreadsPerChild are in event_cmds. We read the MPM itself in Part 8.

The core, HTTP and the MPM are each one struct cut from the same template. Yellow boxes are values written into the struct; blue boxes are functions its register_hooks hangs on hooks.

Easy to get confused

• "Core" is used in two senses. Broadly, it means the whole server/ folder (the skeleton of the server: reading config, reading requests, running hooks). Narrowly, it means the single core_module in server/core.c. server/config.c and server/request.c are not modules; they are collections of functions all modules share. In this post, "the core layer" is the broad sense and core_module the narrow one.
• STANDARD20_MODULE_STUFF and MPM20_MODULE_STUFF differ by one slot. The former, for ordinary modules, also fills the trailing rewrite_args slot with NULL (http_config.h lines 489–496); the latter, for MPMs, leaves that slot open. That's why core_module and mpm_event_module have one extra line in their initializers (AP_PLATFORM_REWRITE_ARGS_HOOK or NULL). On Unix that macro is ap_mpm_rewrite_args (os/unix/os.h line 45), a small example of what the os/ folder is for.
• A module's name is a file name, not the module name. Because STANDARD20_MODULE_STUFF puts __FILE__ there, the name of core_module is "core.c". So when hook ordering refers to another module, it uses a file name like "core.c" (aszSucc[] = { "core.c", NULL } in event_hooks, event.c line 4370). It's also why the httpd -l output in section 3 shows file names.

3. Modules compiled into the binary vs. modules loaded with LoadModule

There are two ways to put a module into the server: link it statically into the binary at build time, or load a .so file while running (shared, DSO).

The lower-left branch is how one LoadModule line is processed; the right branch is the MPM check. The red box is a config error that keeps the server from starting.

A file the build generates: modules.c

The list of statically linked modules lives in a file that isn't in the repository: modules.c. When you run ./configure, build/build-modules-c.awk generates it (configure.in lines 1118–1122). I opened /usr/src/httpd/modules.c in the container that built the example image (it's a build artifact, so there's no GitHub link).

/*
 * modules.c --- automatically generated by Apache
 * configuration script.  DO NOT HAND EDIT!!!!!
 */
/* ... */
extern module core_module;
extern module so_module;
extern module http_module;
/* ... */
AP_DECLARE_DATA module *ap_prelinked_modules[] = {
  &core_module,
  &so_module,
  &http_module,
  NULL
};

Further down the same file there is also ap_preloaded_modules[], holding the same three modules. The example image was built with --enable-mods-shared=most --enable-mpms-shared=all, so every other module and MPM became a .so, and only these three are compiled into the binary. There are reasons these three remain.

• core always comes first. The awk script that writes the file starts by adding core, with the comment # the core module must come first (build/build-modules-c.awk lines 17–19).
• so can't be built as a .so. If the module that loads .so files were itself a .so, there would be no way to load it in the first place. When the build is asked for shared, it stops with mod_so can not be built as a shared DSO (modules/core/config.m4 lines 15–17), and yes is turned into static (lines 25–28).
• http is static by default. The description in modules/http/config.m4 says "This module will always be linked statically.", and its default argument is static (modules/http/config.m4 line 17). A module whose default is static isn't switched by blanket options like --enable-mods-shared=most; it changes only if you name that module explicitly ("enabled as static by default, must be explicitly changed", acinclude.m4 line 283).

At startup: the linked modules are registered first

main() calls ap_setup_prelinked_modules() before it even reads the config file (server/main.c line 574).

AP_DECLARE(const char *) ap_setup_prelinked_modules(process_rec *process)
{
    /* ... */
    total_modules = 0;
    for (m = ap_preloaded_modules; *m != NULL; m++)
        (*m)->module_index = total_modules++;

    max_modules = total_modules + DYNAMIC_MODULE_LIMIT + 1;
    conf_vector_length = max_modules;
    /* ... */

    /*
     *   Initialize chain of linked (=activate) modules
     */
    for (m = ap_prelinked_modules; *m != NULL; m++) {
        error = ap_add_module(*m, process->pconf, NULL);
        if (error) {
            return error;
        }
    }

    apr_hook_sort_all();
    /* ... */
}

server/config.c lines 763–816

It numbers the linked modules 0, 1, 2 as their module_index, and reserves DYNAMIC_MODULE_LIMIT more slots (256 by default, include/httpd.h lines 119–121) for modules that LoadModule will bring in later. Then it adds the three to the list of active modules with ap_add_module(). At this point only core, so and http are alive.

While reading the config: one LoadModule line

When the config file contains LoadModule, mod_so's load_module() is called, because the LoadModule directive itself is in mod_so's directive table (mod_so.c lines 448–450). Here are the essential lines:

    error = dso_load(cmd, &modhandle, filename, &module_file);
    /* ... */
    if (apr_dso_sym(&modsym, modhandle, modname) != APR_SUCCESS) {
        /* ... */
    }
    modp = (module*) modsym;
    modp->dynamic_load_handle = (apr_dso_handle_t *)modhandle;
    modi->modp = modp;

    /* ... */
    if (modp->magic != MODULE_MAGIC_COOKIE) {
        return apr_psprintf(cmd->pool, "API module structure '%s' in file %s "
                            /* ... */);
    }

    /* ... */
    error = ap_add_loaded_module(modp, cmd->pool, modname);

modules/core/mod_so.c lines 265–304

1. dso_load() loads the .so file into the process using apr_dso_load(), which we saw in section 1.
2. apr_dso_sym() looks inside that file for a variable named after LoadModule's first argument (for example mime_module). That variable is the module's module struct — the very variable AP_DECLARE_MODULE(mime) created.
3. It checks the magic value. This filters out the wrong kind of file, or a module compiled for a different version.
4. ap_add_loaded_module() adds it to the module list. From now on this module is treated exactly like core.

Exactly one MPM

After reading the whole config file, ap_read_config() calls ap_check_mpm() (config.c lines 2355–2360).

AP_DECLARE(const char *)ap_check_mpm(void)
{
    static const char *last_mpm_name = NULL;

    if (!_hooks.link_mpm || _hooks.link_mpm->nelts == 0)
        return "No MPM loaded.";
    else if (_hooks.link_mpm->nelts > 1)
        return "More than one MPM loaded.";

server/mpm_common.c lines 656–663

It doesn't check "is an MPM loaded?" by name, but by how many functions are registered on the mpm hook (nelts). In other words, an MPM is defined as "a module that registers a function on the mpm hook."

Example: httpd -l, -M, -V

I ran the three options in the example container. -M and -V read the default config file, /opt/httpd/conf/httpd.conf. The headings say "statically linked modules in the binary," "with the default httpd.conf, first 14 lines," and "default httpd.conf, first 9 lines."

### 1. httpd -l  (실행 파일 안에 정적으로 묶인 모듈)
Compiled in modules:
  core.c
  mod_so.c
  http_core.c

### 2. httpd -M  (기본 httpd.conf 로 실었을 때, 앞 14줄)
Loaded Modules:
 core_module (static)
 so_module (static)
 http_module (static)
 mpm_event_module (shared)
 autht_core_module (shared)
 authn_file_module (shared)
 authn_core_module (shared)
 authz_host_module (shared)
 authz_groupfile_module (shared)
 authz_user_module (shared)
 authz_core_module (shared)
 access_compat_module (shared)
 auth_basic_module (shared)
...
static: 3
shared: 24
.so files in /opt/httpd/modules: 101

### 3. httpd -V  (기본 httpd.conf, 앞 9줄)
Server version: Apache/2.5.1-dev (Unix)
Server built:   Oct 24 2026 11:41:11
Server's Module Magic Number: 20211221:35
Server loaded:  APR 1.7.2, APR-UTIL 1.6.3, PCRE 10.42 2022-12-11
Compiled using: APR 1.7.2, APR-UTIL 1.6.3, PCRE 10.42 2022-12-11
Architecture:   64-bit
Server MPM:     event
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)

• -l doesn't read the config file; it shows only the modules compiled into the binary. What it prints is each module's name, so you get file names like core.c (config.c lines 2507–2514).
• -M reads the config file, then mod_so prints the list: modules from modules.c as (static), modules loaded with LoadModule as (shared) (mod_so.c lines 378–419). 101 .so files were built, but the default httpd.conf actually loads 24. The rest are commented out as #LoadModule.
• Server MPM: event in the -V output comes from line 66 of the default httpd.conf, LoadModule mpm_event_module modules/mod_mpm_event.so (see the next item).

Easy to get confused: why -V reads the config file

httpd -V shows build information, so you'd think the config file doesn't matter. But if the MPM was built as a .so, httpd has to read the config file to learn the MPM's name. The source says so:

        case 'V':
            if (strcmp(ap_show_mpm(), "")) { /* MPM built-in? */
                show_compile_settings();
                destroy_and_exit_process(process, 0);
            }
            else {
                showcompile = 1;
                ap_run_mode = AP_SQ_RM_CONFIG_DUMP;
            }
            break;

server/main.c lines 679–688

If the MPM is compiled into the binary, it prints and exits right away; otherwise it just sets showcompile and prints after reading the config. So with a config file that loads no MPM, you get this (heading: "httpd -V / -t with a config that loads no MPM"):

### 4. MPM 을 싣지 않은 설정으로 httpd -V / -t
AH00534: httpd: Configuration error: No MPM loaded.
Server MPM:     
AH00534: httpd: Configuration error: No MPM loaded.
(exit code 1)

-V prints the error but still shows the build information, with the Server MPM: field left empty. The syntax check -t stops with the same error and exit code 1. It's worth remembering that what httpd -V reports as the MPM on a live server depends on which config file it read.

4. The three big structs: server_rec, conn_rec, request_rec

Most functions in the httpd source take one of these three as their first argument. All three are in include/httpd.h.

On a keep-alive connection (one that stays open after a response and waits for the next request — KeepAlive On in httpd.conf) several requests travel over one connection, so one conn_rec sees several request_recs come and go in turn. A server_rec lives much longer than either.

Arrows are pointer fields inside the structs. There are paths from request to connection and from connection to server, but no field going back the other way appears in this picture.

request_rec: one request

The first fields of the struct are the paths to the other structs.

struct request_rec {
    /** The pool associated with the request */
    apr_pool_t *pool;
    /** The connection to the client */
    conn_rec *connection;
    /** The virtual host for this request */
    server_rec *server;

    /** Pointer to the redirected request if this is an external redirect */
    request_rec *next;
    /** Pointer to the previous request if this is an internal redirect */
    request_rec *prev;

    /** Pointer to the main request if this is a sub-request
     * (see http_request.h) */
    request_rec *main;

include/httpd.h lines 866–881

The whole struct runs to line 1169, more than 300 lines. Here are the fields you'll meet most often on a first read.

How uri becomes filename is in Part 10, how handler picks a handler in Part 12, and filter chains in Part 11.

conn_rec: one connection

struct conn_rec {
    /** Pool associated with this connection */
    apr_pool_t *pool;
    /** Physical vhost this conn came in on */
    server_rec *base_server;
    /** used by http_vhost.c */
    void *vhost_lookup_data;

include/httpd.h lines 1203–1209

server_rec: one server (virtual host)

struct server_rec {
    /** The process this server is running in */
    process_rec *process;
    /** The next server in the list */
    server_rec *next;

include/httpd.h lines 1401–1405

One server_rec is one <VirtualHost> block in the config file. The code that handles <VirtualHost> creates a new server_rec and slots it in right after the main server.

    s->next = main_server->next;
    main_server->next = s;

server/core.c lines 3065–3066

How the two lines change the list, traced with the config from the example below (a.example at line 14, b.example at line 17). The green box is the virtual host just inserted.

Where module config lives: a void * array and module_index

module_config in server_rec and per_dir_config in request_rec hold config structs whose shape differs from module to module — mod_mime's config, mod_dir's config, and so on. The struct definitions have no such fields at all. Instead there is a void * array with one slot per module, and each module uses only slot number module_index.

#define ap_get_module_config(v,m)       \
    (((void **)(v))[(m)->module_index])
#define ap_set_module_config(v,m,val)   \
    ((((void **)(v))[(m)->module_index]) = (val))

include/http_config.h lines 553–556

void * is "a pointer that can point to anything." Each module takes out slot module_index and treats it as its own struct type. The mod_so code behind the -M output in section 3 does exactly that with ap_get_module_config(s->module_config, &so_module) (mod_so.c lines 394–395). Thanks to this, server_rec keeps the same shape no matter how many modules are loaded. How the slots are filled and merged per <Directory> is in Part 7; the downside, no type checking, is discussed in Part 2.

Example: printing the three structs for one request

To check that the three structs really are connected this way, I wrote a small module. For a request to /three, it prints a few fields of r, r->connection and r->server, the server list, and the parent relationship of the pools into the response body.

/* Example written for this post (not part of the httpd source) */
/* For one request, print a few fields of request_rec, conn_rec and server_rec,
 * how the three point to each other, and how the pools are parented. */
#include "httpd.h"
#include "http_config.h"
#include "http_protocol.h"
#include "http_main.h"     /* ap_server_conf */
#include "apr_pools.h"

static int bigthree_handler(request_rec *r)
{
    conn_rec *c = r->connection;     /* request -> connection */
    server_rec *s = r->server;       /* request -> the (virtual) server handling it */
    server_rec *v;

    if (!r->handler || strcmp(r->handler, "bigthree")) {
        return DECLINED;             /* not mine: let the next handler try */
    }
    ap_set_content_type(r, "text/plain");

    ap_rprintf(r, "[request_rec r=%pp]\n", r);
    ap_rprintf(r, "  the_request      = %s\n", r->the_request);
    ap_rprintf(r, "  uri              = %s\n", r->uri);
    ap_rprintf(r, "  hostname         = %s\n", r->hostname);
    ap_rprintf(r, "  handler          = %s\n", r->handler);

    ap_rprintf(r, "[conn_rec c=%pp]\n", c);
    ap_rprintf(r, "  client_ip        = %s\n", c->client_ip);
    ap_rprintf(r, "  local_ip         = %s\n", c->local_ip);
    ap_rprintf(r, "  keepalives       = %d\n", c->keepalives);
    ap_rprintf(r, "  base_server      = %s\n", c->base_server->server_hostname);

    ap_rprintf(r, "[server_rec s=%pp]\n", s);
    ap_rprintf(r, "  server_hostname  = %s\n", s->server_hostname);
    ap_rprintf(r, "  is_virtual       = %d\n", s->is_virtual);
    ap_rprintf(r, "  defined at       = %s:%u\n", s->defn_name, s->defn_line_number);
    ap_rprintf(r, "  process          = %s\n", s->process->short_name);

    /* The whole server list linked by s->next, starting at ap_server_conf (main server) */
    ap_rprintf(r, "[server list from ap_server_conf]\n");
    for (v = ap_server_conf; v; v = v->next) {
        ap_rprintf(r, "  %s%s\n", v->server_hostname, v == s ? "   <- r->server" : "");
    }

    /* Pool parent relationships */
    ap_rprintf(r, "[pools]\n");
    ap_rprintf(r, "  parent(r->pool) == c->pool : %s\n",
               apr_pool_parent_get(r->pool) == c->pool ? "yes" : "no");
    ap_rprintf(r, "  pconf is ancestor of r->pool : %s\n",
               apr_pool_is_ancestor(s->process->pconf, r->pool) ? "yes" : "no");
    return OK;
}

static void register_hooks(apr_pool_t *p)
{
    ap_hook_handler(bigthree_handler, NULL, NULL, APR_HOOK_MIDDLE);
}

AP_DECLARE_MODULE(bigthree) = {
    STANDARD20_MODULE_STUFF,
    NULL,                  /* create directory config */
    NULL,                  /* merge directory config */
    NULL,                  /* create server config */
    NULL,                  /* merge server config */
    NULL,                  /* directive table */
    register_hooks         /* register hooks */
};

It creates a module struct shaped just like core_module and http_module from section 2, and in register_hooks puts one function on the handler hook. I compiled the module into a .so with apxs -c and loaded it with LoadModule bigthree_module /work/.libs/mod_bigthree.so. The config file (/tmp/three.conf) sets SetHandler bigthree inside <Location /three> and defines two virtual hosts on the same port 8080, in this order:

<VirtualHost *:8080>
  ServerName a.example
</VirtualHost>
<VirtualHost *:8080>
  ServerName b.example
</VirtualHost>

a.example starts on line 14 of the config file and b.example on line 17. The main server's ServerName is localhost. I sent Host: b.example with curl and requested the same URL twice. Given two URLs in one command, curl reuses a single connection. (The first line of output is a heading: "Host: b.example, twice over the same connection.")

--- curl: Host: b.example, 같은 연결로 두 번
[request_rec r=7f212c7620a0]
  the_request      = GET /three HTTP/1.1
  uri              = /three
  hostname         = b.example
  handler          = bigthree
[conn_rec c=7f212c797350]
  client_ip        = 127.0.0.1
  local_ip         = 127.0.0.1
  keepalives       = 0
  base_server      = a.example
[server_rec s=7f212d032330]
  server_hostname  = b.example
  is_virtual       = 1
  defined at       = /tmp/three.conf:17
  process          = httpd
[server list from ap_server_conf]
  localhost
  b.example   <- r->server
  a.example
[pools]
  parent(r->pool) == c->pool : yes
  pconf is ancestor of r->pool : yes
[request_rec r=7f212c7600a0]
...
[conn_rec c=7f212c797350]
...
  keepalives       = 1
...

For the second request I kept only the lines that differ from the first and replaced the rest with .... The addresses after r=, c= and s= change on every run. Here's what to look at:

1. The two requests have different r addresses but the same c address. A new request_rec is created per request, but both point to the same conn_rec. keepalives went from 0 to 1.
2. c->base_server is a.example, while r->server is b.example. When the connection arrives, only the IP and port (*:8080) are known, so the first virtual host for that address becomes base_server; after the request headers are read, Host: b.example decides r->server. We follow this selection through server/vhost.c in Part 9.
3. The server_rec that r->server points to is the <VirtualHost> on line 17 of the config file. I checked this with defn_name and defn_line_number.
4. The request pool's parent is the connection pool. And the process's pconf is an ancestor of the request pool (section 5).

Addresses and line numbers are copied from the output above (addresses differ on every run). Blue is a request, yellow a connection, green a server.

If Host names no virtual host at all, r->server stays at c->base_server (a.example), the value ap_create_request() put there at the start (see the code in section 5). (Heading: "conn_rec and server_rec parts only.")

--- curl: Host: nosuch.example (conn_rec, server_rec 부분만)
[conn_rec c=7f212c797360]
  client_ip        = 127.0.0.1
  local_ip         = 127.0.0.1
  keepalives       = 0
  base_server      = a.example
[server_rec s=7f212d031ad0]
  server_hostname  = a.example
  is_virtual       = 1
  defined at       = /tmp/three.conf:14
  process          = httpd

Easy to get confused

• The server list is in the reverse of config-file order. Following next from ap_server_conf in the output gives localhost → b.example → a.example, although a.example came first in the config file. That's because core.c lines 3065–3066, shown above, insert each new virtual host right after the main server, not at the end of the list. Yet the one picked as base_server was a.example, the one that came first in the file. After the config has been read, server/vhost.c builds a separate per-address table and undoes the reversal along the way. The source comments say so: "They're listed off of main_s->next in the reverse order they occurred in the config file" (vhost.c lines 586–587) and "iterating backwards, so each one we see becomes the current default server" (lines 539–540). We look at that table in Part 9.
• r->server and c->base_server can differ. Code that handles a request should usually read config from r->server. With name-based virtual hosts, using c->base_server reads the wrong virtual host's settings.
• server_rec has no pool field. request_rec and conn_rec carry their own pool as the first field, but a server_rec is allocated in pconf and has no pool of its own. You can see ap_read_config() taking process->pconf and building the main server in config.c lines 2301–2307 and lines 2224–2227.
• c->id may not be a fixed number per connection. When the event MPM handles the next request on a connection, it resets c->id from the current thread number (comment: "Subsequent request on a conn, and thread number is part of ID", event.c lines 1130–1131). That's why the example uses the address of c to tell whether it's the same connection.

5. The pool hierarchy at a glance

Each of the three structs lives as long as its pool. Pools are linked as parents and children, so destroying a parent pool destroys its children too.

Every yellow box is a pool. The further down and to the right, the shorter the lifetime. The parent of the transaction pool depends on the MPM (see the table).

The code that creates the request pool is short.

AP_DECLARE(request_rec *) ap_create_request(conn_rec *conn)
{
    request_rec *r;
    apr_pool_t *p;

    apr_pool_create(&p, conn->pool);
    apr_pool_tag(p, "request");
    r = apr_pcalloc(p, sizeof(request_rec));
    AP_READ_REQUEST_ENTRY((intptr_t)r, (uintptr_t)conn);
    r->pool            = p;
    r->connection      = conn;
    r->server          = conn->base_server;

server/protocol.c lines 1058–1069

It creates the request pool as a child of the connection pool (conn->pool) and builds the request_rec inside that pool (apr_pcalloc takes zero-filled memory from a pool). So destroying the request pool makes the request_rec itself, and every string and table in it, disappear at once. You can also see r->server being set to conn->base_server at first. The place that changes it after reading the Host header is server/vhost.c line 1075 (item 2 of the example in section 4).

The request pool is destroyed when a special bucket called EOR (End Of Request) is destroyed after the response has gone all the way out (apr_pool_destroy(r->pool) in server/eor_bucket.c lines 100–112). Buckets are covered in Part 11.

The APR side of creating and destroying pools, and registering "things to do when a pool is destroyed" with apr_pool_cleanup_register(), is the topic of Part 4. How pconf is cleared on every restart and the config re-read is in Part 5.

Easy to get confused: trunk and 2.4.x differ by one pool layer for connections

In the reference commit (trunk — the development line preparing the next release), the conn_rec struct is allocated in the transaction pool, and c->pool is a separate master_conn pool created beneath it (core.c line 5516 and lines 5525–5527). In the 2.4.x branch most of us install, core_create_conn() does c->pool = ptrans;, using the transaction pool itself as the connection pool. From a module's point of view "c->pool lives as long as the connection" holds either way, but when you read 2.4.x articles or code alongside trunk, the pool hierarchy can look one level different.

The parent of the transaction pool also depends on the MPM. event and worker create it under pconf (worker.c line 673), while prefork creates it under the child process's pchild (prefork.c line 469). The diagram follows event, the MPM used in this post's examples.

6. The path a single request takes

Now let's string the layers and structs together. When a client opens a connection and sends GET /index.html, it follows this path.

The blue boxes are places where the core runs hooks; each box names the post that reads that step in detail.

Read the table top to bottom and the layers from section 1 appear. Only the first row is the MPM; in all the others the core runs a hook and core_module, http_module and other modules do the work in turn. The core itself does less than you'd expect; mostly it "runs the next hook."

ap_process_http_connection(), in the "Process the connection" row, splits into two branches depending on how the MPM works.

static int ap_process_http_connection(conn_rec *c)
{
    if (async_mpm && !c->clogging_input_filters) {
        return ap_process_http_async_connection(c);
    }
    else {
        return ap_process_http_sync_connection(c);
    }
}

modules/http/http_core.c lines 247–255

With an MPM like prefork or worker, where one thread (or process) holds a connection to the end, the sync branch handles keep-alive requests one after another in a while ((r = ap_read_request(c)) != NULL) loop (http_core.c line 197). event takes the async branch: it handles a single request and gives the connection back to the MPM. When the next request arrives, the MPM runs the process_connection hook again. That's why the arrow looping back in the diagram points at ap_run_process_connection().

Easy to get confused: event never calls ap_process_connection()

server/connection.c has a function whose name says exactly what it does:

AP_CORE_DECLARE(void) ap_process_connection(conn_rec *c, void *csd)
{
    ap_update_vhost_given_ip(c);

    ap_pre_connection(c, csd);

    if (!c->aborted) {
        ap_run_process_connection(c);
    }
}

server/connection.c lines 244–253

prefork (prefork.c line 693) and worker (worker.c line 496) call this function. event's process_socket(), however, skips it and does the same work itself: it calls ap_run_create_connection(), ap_update_vhost_given_ip() and ap_pre_connection() in turn, and later ap_run_process_connection() (event.c lines 1075–1153), because the connection has to be passed back and forth between the MPM and the core several times. This is why, when following the source, you can set a breakpoint on ap_process_connection() under event and never hit it.

What this post leaves out

Things I left out to keep the map to one page:

• The macros that declare and implement hooks (AP_DECLARE_HOOK, AP_IMPLEMENT_HOOK_RUN_FIRST, etc.) and hook sorting. include/ap_hooks.h, APR-util's include/apr_hooks.h — Part 6
• The internals of ap_add_module() and ap_add_loaded_module(), and ap_single_module_configure(). server/config.c
• The rest of process_rec (argv, short_name) and the read-the-config-twice loop in main(). server/main.c — Part 5
• conn_rec's slaves and master (secondary connections created inside one connection) and the conn_state_t state machine. include/httpd.h lines 1296–1299 and 1359–1364, server/mpm/event/event.c
• request_rec's request-body fields (read_body, remaining, etc.), plus finfo and parsed_uri
• The functions that create config arrays with conf_vector_length slots (server/config.c lines 224 and 266–313), and ap_get_module_config turning into a function instead of a macro in AP_DEBUG builds (include/http_config.h line 522)
• The structure of large modules such as modules/http2 and modules/md

Summary

What's next

Time to fold the map and start at the floor. The next post covers what this post kept calling just "pools": APR memory pools. We'll see what happens inside APR when you create parent and child pools with apr_pool_create(), take memory with apr_palloc() and destroy it all at once with apr_pool_destroy(), and we'll run an example that registers a cleanup on the request pool to catch the moment a request ends.

Digging into Apache httpd's Source 4: APR Memory Pools — Managing Memory Without free()

The Spring series spread out a big-picture map at the same point. Spring splits its modules into jars at build time and lets a single container wire beans together at run time, while httpd loads .so files at run time and treats them all as the same module struct. The mechanism for stepping into other modules' behavior differs too: Spring uses extension points such as BeanPostProcessor, httpd uses hooks. Spring's DispatcherServlet, which processes a request step by step, is in Spring Part 21; put it next to the table in section 6 and you'll see both projects implementing the same idea in different languages: split a request into phases and leave a slot to plug into at each one.