What this post covers
This is the eighth post in the "Digging into Tomcat's Source" series. In part 7 we watched CoyoteAdapter clean up the request URI and use the Mapper to decide in advance which Host, Context and Wrapper (servlet) would handle the request. It only decided — nobody has been called yet. This post follows what comes next: the path a request takes from the moment CoyoteAdapter calls the container until it reaches the servlet's front door.
We want to answer four questions.
- What is a
<Valve>in server.xml in the source, and where and in what order is it attached? - How does a request cross from Engine to Host to Context to Wrapper? Who picks the next container?
- When a valve blocks a request or something fails, who draws the error page?
- When is the access log (
AccessLogValve) written, and why does it record the wrong client address behind a proxy?
| Item | Details |
|---|---|
| Project | Apache Tomcat |
| Version | 12.0.0-M1 (main) |
| Commit | 17aa1769b0 (2026-10-23) |
| Main classes | Pipeline, Valve, core/StandardPipeline, core/Standard{Engine,Host,Context,Wrapper}Valve, valves/ValveBase, valves/ErrorReportValve, valves/AbstractAccessLogValve, valves/RemoteIpValve |
All line numbers in this post refer to this commit. How to get the same source and build the Docker image for the examples is in part 1.
The valves package alone holds about twenty valves, and the parent class of the access log valves (AbstractAccessLogValve) is 2,400 lines by itself. This post picks only the main thread that a first-time reader can follow. What I left out is listed by name and location under "What this post leaves out" at the end.
Words to know first
A container is one layer in Catalina that receives and processes requests. There are four, linked as parent and child: <Engine>, <Host> and <Context> (one web app) from server.xml, plus the Wrapper that wraps a single servlet. All four extend ContainerBase. We saw the whole hierarchy in part 2.
A valve is an object that receives a request, does something with it, and, if it wants to, hands it on to the next valve. According to the source's own note, the name comes from the valves in real plumbing that control the flow (Valve.java lines 31–32). Access logging, error pages, authentication and rewriting addresses for proxies are all done by valves.
A pipeline is a row of valves. Every container has one. So a single request goes through four pipelines in turn.
The basic valve is the valve that is always at the very end of a pipeline. The container attaches it itself when it is created, and you don't change it through configuration. Its job is mostly "pick the next container and call its pipeline".
The access log is a file that records one line per request: who asked for what, when, and what status code came back. With the default configuration it is logs/localhost_access_log.<date>.txt.
Async support is about whether a servlet may call startAsync() to set the request aside and give the thread back. Each valve declares separately "it is fine for async requests to pass through me". Async servlets themselves are covered in part 12.
1. The big picture: four containers, four pipelines
At the end of part 7, CoyoteAdapter.service() finishes mapping (postParseRequest()) and calls the container like this.
postParseSuccess = postParseRequest(req, request, res, response);
if (postParseSuccess) {
// check valves if we support async
request.setAsyncSupported(connector.getService().getContainer().getPipeline().isAsyncSupported());
// Calling the container
connector.getService().getContainer().getPipeline().getFirst().invoke(request, response);
}
java/org/apache/catalina/connector/CoyoteAdapter.java lines 342–348
getService().getContainer() is the Engine that belongs to the Service. Taking the first valve out of the Engine's pipeline and calling invoke() on it — that is the whole doorway into Catalina. From there on, valves call valves.
At the end of the Engine pipeline sits the basic valve StandardEngineValve. What it does is short.
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
// Select the Host to be used for this Request
Host host = request.getHost();
if (host == null) {
// No Host found for this request (unknown server name or no default host defined)
// Don't overwrite an existing error
if (!response.isError()) {
response.sendError(404);
}
return;
}
if (request.isAsyncSupported()) {
request.setAsyncSupported(host.getPipeline().isAsyncSupported());
}
// Ask this Host to process this request
host.getPipeline().getFirst().invoke(request, response);
}
java/org/apache/catalina/core/StandardEngineValve.java lines 53–72
request.getHost() is the Host that part 7's Mapper already wrote down. This valve doesn't choose, it just takes it out, and then calls the first valve of that Host's pipeline. The Host's basic valve does the same with the Context, and the Context's basic valve does the same with the Wrapper. So the overall shape looks like this.
Blue boxes are valves you attach through configuration; uncolored boxes are basic valves the containers attach themselves.
Each container attaches its basic valve with setBasic() in its constructor.
| Container | Basic valve | Where it is attached | What it does |
|---|---|---|---|
StandardEngine |
StandardEngineValve |
StandardEngine.java line 60 |
Takes out the Host and passes on. 404 if none |
StandardHost |
StandardHostValve |
StandardHost.java line 66 |
Takes out the Context and passes on. Swaps the class loader, request listeners, the web app's <error-page> |
StandardContext |
StandardContextValve |
StandardContext.java line 157 |
Blocks WEB-INF and META-INF, takes out the Wrapper and passes on |
StandardWrapper |
StandardWrapperValve |
StandardWrapper.java lines 86–87 |
Borrows a servlet instance, builds the filter chain and calls it |
Example 1: hanging valves on Engine, Host and Context to see the order
To check that it really goes in this order, I wrote a valve that prints > before calling the next valve and < after it returns.
// Written for this post (not part of Tomcat's source)
package lab;
// ... imports omitted ...
public class TraceValve extends ValveBase {
private String name = "trace";
private boolean dump = false;
public TraceValve() {
super(true); // declare that async requests may pass through too
}
// the Digester fills <Valve> attributes in through setters of the same name
public void setName(String name) {
this.name = name;
}
// ... setDump(), and a startInternal() that prints the pipeline at startup when dump="true" ...
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
System.out.println("> " + name + " " + describe() + " " + request.getRequestURI());
getNext().invoke(request, response);
System.out.println("< " + name + " status=" + response.getStatus());
}
// ... describe() builds a container name such as "StandardHost[localhost]" ...
}
[Shortened from the example ex/u08-valve-order/lab/TraceValve.java]
I packed this valve into a jar under $CATALINA_BASE/lib (where the common class loader looks, part 4) and added it in two places in the default server.xml. For the Context, I used the context descriptor file conf/Catalina/localhost/app.xml.
<!-- server.xml -->
<Engine name="Catalina" defaultHost="localhost">
<Valve className="lab.TraceValve" name="engine-1" dump="true"/>
<Valve className="lab.TraceValve" name="engine-2"/>
<!-- ... Realm ... -->
<Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true">
<Valve className="lab.TraceValve" name="host" dump="true"/>
<Valve className="lab.LogProbeValve"/>
<!-- ... the AccessLogValve that is already in the default config ... -->
</Host>
</Engine>
<!-- conf/Catalina/localhost/app.xml -->
<Context>
<Valve className="lab.TraceValve" name="context" dump="true"/>
<Valve className="lab.BlockValve"/>
</Context>
BlockValve is used in section 5 and LogProbeValve in section 6. The web app app has one servlet, which uses StackWalker to print the Catalina invoke()/service() methods that called it.
This example was run in a Docker container (Eclipse Temurin JDK 21) with Tomcat 12.0.0-M1-dev built as-is from the reference commit (17aa1769b0). ex/u08-valve-order/run.sh creates the configuration above, runs catalina.sh start, and sends curl requests. Here is the result for /app/hello.
===== curl /app/hello =====
HTTP 200
> engine-1 StandardEngine[Catalina] /app/hello
> engine-2 StandardEngine[Catalina] /app/hello
> host StandardHost[localhost] /app/hello
> context StandardContext[/app] /app/hello
servlet: service() reached, called through:
at org.apache.catalina.core.StandardWrapperValve.invoke:164
at org.apache.catalina.core.StandardContextValve.invoke:77
at org.apache.catalina.authenticator.AuthenticatorBase.invoke:540
at lab.BlockValve.invoke:28
at lab.TraceValve.invoke:52
at org.apache.catalina.core.StandardHostValve.invoke:115
at org.apache.catalina.valves.ErrorReportValve.invoke:82
at org.apache.catalina.valves.AbstractAccessLogValve.invoke:777
at lab.LogProbeValve.invoke:27
at lab.TraceValve.invoke:52
at org.apache.catalina.core.StandardEngineValve.invoke:71
at lab.TraceValve.invoke:52
at lab.TraceValve.invoke:52
at org.apache.catalina.connector.CoyoteAdapter.service:347
< context status=200
< host status=200
< engine-2 status=200
< engine-1 status=200
There are three things to read here.
>goes from the outside in,<from the inside out. A valve calls the next valve inside its own method, so the outer valve'sinvoke()doesn't finish until everything after it has finished. It's a wrapping structure, like the layers of an onion.- Read the stack from the bottom up. It starts at line 347 of
CoyoteAdapter.service(the code above), then the two Engine valves →StandardEngineValveline 71 → the four Host valves →StandardHostValveline 115 → the three Context valves →StandardContextValveline 77 →StandardWrapperValveline 164. The line numbers match the source lines quoted in this post. - The Host has an
ErrorReportValveI never attached, and the Context has anAuthenticatorBase(the actual class isNonLoginAuthenticator). Section 3 shows who added them and when.
StandardWrapperValve line 164 is filterChain.doFilter(...). The filters and the servlet's service() sit above it, but this example only prints Catalina's invoke/service frames, so they don't show up. That stretch is the subject of part 9.
Easy to get wrong
Pipelinehas noinvoke(). The interface description says "the valves are executed in sequence wheninvoke()is called" (Pipeline.javalines 22–24), but there is no such method in the list. Callers always usegetPipeline().getFirst().invoke(...)to call the first valve directly. The pipeline only holds the chain; it doesn't run it.- You cannot attach a valve to a Wrapper through configuration. Among the rules that read server.xml and context.xml, the only ones that attach a valve to a container are
Engine/Valve,Host/ValveandContext/Valve(a<Valve>inside<Cluster>also exists, but the cluster attaches it to the Engine or Host it belongs to;HostRuleSet.javalines 77–83;EngineRuleSetandContextRuleSetlook the same). A Wrapper is created from<servlet>in web.xml, which has no place for valves. If you want to put something in front of a single servlet, use a filter.
2. Pipeline and Valve — one linked list
Keeping only the methods used for request processing, the two interfaces look like this.
public interface Pipeline extends Contained {
// ...
Valve getBasic();
// ...
void setBasic(Valve valve);
// ...
void addValve(Valve valve);
// ...
Valve[] getValves();
// ...
Valve getFirst();
java/org/apache/catalina/Pipeline.java lines 32–108
public interface Valve {
// ...
Valve getNext();
// ...
void setNext(Valve valve);
// ...
void backgroundProcess();
// ...
void invoke(Request request, Response response) throws IOException, ServletException;
java/org/apache/catalina/Valve.java lines 34–109
The key is that a valve has getNext()/setNext(). "Who comes next" is remembered by each valve, not by the pipeline. It isn't an array or a List but a singly linked list. The implementation, StandardPipeline, uses just two fields to point at valves: first at the front and basic at the end (besides those it only has container, pointing at its own container; lines 83–95).
Where addValve() inserts a new valve shows the structure.
// Add this Valve to the set associated with this Pipeline
if (first == null) {
first = valve;
valve.setNext(basic);
} else {
Valve current = first;
while (current != null) {
if (current.getNext() == basic) {
current.setNext(valve);
valve.setNext(basic);
break;
}
current = current.getNext();
}
}
container.fireContainerEvent(Container.ADD_VALVE_EVENT, valve);
java/org/apache/catalina/core/StandardPipeline.java lines 286–302
It walks from the front looking for "the valve whose next is the basic valve" and slips the new one in between. So a new valve always lands right before the basic valve, i.e. at the end of the configured valves. If there are no valves at all, first is null and getFirst() returns the basic valve (lines 400–407). That's why, even on a Wrapper with no valves attached, getFirst().invoke() reaches StandardWrapperValve straight away.
Dotted lines are the links after addValve(C). The basic valve's next is always null.
What a valve may and may not do
The doc comment of Valve.invoke() lists a valve's contract (lines 66–99). In short:
| May | Must not |
|---|---|
| Look at and modify the request and response | Change values already used to route the request (e.g. change the virtual host from a Host pipeline) |
| Build the whole response itself and return without calling the next one | Build the whole response and also pass it to the next valve |
| Wrap the request or response and pass them on | Read the request body unless it is building the response or wrapping the request |
Call the next one with getNext().invoke() |
Change response headers or write to the output stream after getNext().invoke() returns |
| After it returns, only read the response |
The first "must not" connects to the figure in section 1. By the time the Engine's valves run, part 7's Mapper has already settled the Host, Context and Wrapper. Changing the URI in a valve does not trigger a new mapping.
ValveBase — what you inherit when writing a valve
When writing your own valve you usually extend ValveBase and write just invoke(). This class takes care of the next and container fields, Lifecycle (start and stop, part 3) and JMX registration (line 35).
Easy to get wrong: calling plain super() turns async off
ValveBase has two constructors.
public ValveBase() {
this(false);
}
// ...
public ValveBase(boolean asyncSupported) {
this.asyncSupported = asyncSupported;
}
java/org/apache/catalina/valves/ValveBase.java lines 48–60
The no-argument constructor sets asyncSupported to false. And a pipeline's async support is found by walking all its valves; if any one is false, the answer is false.
@Override
public boolean isAsyncSupported() {
Valve valve = (first != null) ? first : basic;
boolean supported = true;
while (supported && valve != null) {
supported = valve.isAsyncSupported();
valve = valve.getNext();
}
return supported;
}
java/org/apache/catalina/core/StandardPipeline.java lines 100–109
CoyoteAdapter line 345 and StandardEngineValve lines 66–68, which we saw in section 1, put this value into request.setAsyncSupported(). The Host and Context basic valves do the same. So if you attach one valve built on ValveBase() to the Engine, startAsync() is refused for every servlet on that server, even ones marked @WebServlet(asyncSupported = true). The four basic valves and the ErrorReportValve and AccessLogValve we look at in this post all pass true. That's why the example valves start with super(true) too. I confirmed this trap in the source but did not run it with an actual async request. Async continues in part 12.
Easy to get wrong: the order is simply the order you wrote
httpd's hooks let a module pick its place with an order value such as APR_HOOK_FIRST and conditions like "before that module" (httpd part 6). Tomcat valves have nothing like that. addValve() is called in the order the valves appear in the configuration file, and that order is the execution order. When you put an access log valve and an address-rewriting valve on the same container, you have to get the order right yourself (section 7).
One more thing: StandardPipeline's doc comment says it "assumes that no calls to addValve() or removeValve() are made while a request is currently being processed" (lines 42–44). It walks the linked list without locking. Valves are something you attach at startup.
3. When are valves attached — configuration, and the ones Tomcat adds itself
<Valve> in server.xml
A single <Valve className="..."> line is handled by three Digester rules we met in part 3. It creates an instance of the className class (addObjectCreate), sets the other attributes through setters of the same name (addSetProperties — the example's name="host" becomes setName("host")), and calls the parent container's addValve() (addSetNext).
digester.addObjectCreate(prefix + "Host/Valve", null, // MUST be specified in the element
"className");
digester.addSetProperties(prefix + "Host/Valve");
digester.addCallMethod(prefix + "Host/Valve/init-param", "addInitParam", 2);
digester.addCallParam(prefix + "Host/Valve/init-param/param-name", 0);
digester.addCallParam(prefix + "Host/Valve/init-param/param-value", 1);
digester.addSetNext(prefix + "Host/Valve", "addValve", "org.apache.catalina.Valve");
java/org/apache/catalina/startup/HostRuleSet.java lines 77–83
The addValve called here is ContainerBase.addValve(), which just calls pipeline.addValve(valve) (lines 918–921). Its comment calls it "a convenience method, intended for use by the digester".
Two valves Tomcat adds by itself
In example 1 there were two valves I hadn't attached. One is the Host's ErrorReportValve, which StandardHost adds when it starts.
// Set error report valve
String errorValve = getErrorReportValveClass();
if ((errorValve != null) && (!errorValve.isEmpty())) {
try {
boolean found = false;
Valve[] valves = getPipeline().getValves();
for (Valve valve : valves) {
if (errorValve.equals(valve.getClass().getName())) {
found = true;
break;
}
}
if (!found) {
Valve valve = ErrorReportValve.class.getName().equals(errorValve) ? new ErrorReportValve() :
(Valve) Class.forName(errorValve).getConstructor().newInstance();
getPipeline().addValve(valve);
}
} catch (Throwable t) {
ExceptionUtils.handleThrowable(t);
log.error(sm.getString("standardHost.invalidErrorReportValveClass", errorValve), t);
}
}
super.startInternal();
java/org/apache/catalina/core/StandardHost.java lines 785–807
The default for errorReportValveClass is org.apache.catalina.valves.ErrorReportValve (line 146). Because it is attached with addValve() when the Host starts — after server.xml has been read — it lands at the end of the configured valves, right before the basic valve. If a valve of the same class is already there it isn't added, so if you write ErrorReportValve into server.xml yourself, your position and your attributes are used. With an empty value, as in <Host errorReportValveClass="">, it isn't added at all.
The other is the Context's NonLoginAuthenticator. Even when web.xml has no <login-config>, ContextConfig inserts a dummy login config called "NONE" (lines 341–346), and, unless an authenticator valve is already configured, looks up the class for NONE in Authenticators.properties and attaches it to the pipeline (lines 395–398). The comment explains that an authenticator is needed to support HttpServletRequest.login(). Because this valve authenticates against a Realm, if there is no Realm at all ContextConfig logs the contextConfig.missingRealm error and the web app fails to start (lines 354–358). The example relies on the Realm configured on the Engine in the default server.xml. Authentication is a valve too. How this valve enforces security-constraint is the subject of part 14.
Printing the pipelines at startup
With dump="true", example 1's TraceValve prints its container's pipeline from front to back when it starts (startInternal()).
===== startup: pipelines =====
pipeline of StandardHost[localhost]: TraceValve LogProbeValve AccessLogValve ErrorReportValve StandardHostValve
pipeline of StandardContext[/app]: TraceValve BlockValve NonLoginAuthenticator StandardContextValve
pipeline of StandardEngine[Catalina]: TraceValve TraceValve StandardEngineValve
- Host: the three from server.xml (
TraceValve,LogProbeValve, and the default config'sAccessLogValve), thenErrorReportValve, then the basic valve. - Context: the two from app.xml, then
NonLoginAuthenticator, then the basic valve. - The printing order is Host → Context → Engine. You might expect the outer container first, but it isn't.
Two lines of the output above, with where each valve came from. Blue boxes were written in configuration, yellow boxes were added by Tomcat itself at startup, and gray boxes are the basic valves.
The last point comes from the order inside ContainerBase.startInternal().
// Start our child containers, if any
Container[] children = findChildren();
// ...
// Start the Valves in our pipeline (including the basic), if any
if (pipeline instanceof Lifecycle) {
((Lifecycle) pipeline).start();
}
setState(LifecycleState.STARTING);
java/org/apache/catalina/core/ContainerBase.java lines 728–759
Children are started first, and the container's own pipeline afterwards. So the Engine's valves start only after the Host has finished starting. But at that point the Host has no children (Contexts) yet. Web apps are deployed by HostConfig only when the Host fires START_EVENT via setState(STARTING) (HostConfig.java line 286; deployment is in part 10). That's why the Host's pipeline (line 756) starts before the Context's.
4. What the four basic valves do
We saw StandardEngineValve in section 1. Here are the other three.
StandardHostValve — where we cross into the web app's world
// Ask this Context to process this request. Requests that are
// already in error must have been routed here to check for
// application defined error pages so DO NOT forward them to the
// application for processing.
try {
if (!response.isErrorReportRequired()) {
context.getPipeline().getFirst().invoke(request, response);
}
} catch (Throwable t) {
ExceptionUtils.handleThrowable(t);
container.getLogger().error(sm.getString("standardHostValve.exception", request.getRequestURI()), t);
// If a new error occurred while trying to report a previous
// error allow the original error to be reported.
if (!response.isErrorReportRequired()) {
request.setAttribute(RequestDispatcher.ERROR_EXCEPTION, t);
throwable(request, response, t);
}
}
java/org/apache/catalina/core/StandardHostValve.java lines 109–126
There's a lot around the single line that calls the Context (line 115) (lines 78–169).
- It switches the thread's context class loader to the web app's (
context.bind(), line 99). From here down, the Context valves, filters and servlet run under the web app's class loader. Thefinallyblock puts it back withunbind()(line 167). Class loaders were covered in part 4. - It calls
ServletRequestListener.requestInitialized()(line 101). If a listener throws, the request ends there. - It calls the Context pipeline, and if the response is in an error state when it returns, it looks up the
<error-page>in the web app's web.xml andforwards there (lines 142–154,status()andthrowable()). Error pages defined by the web app are this valve's job. When there is no error page at all, the Tomcat default screen is drawn byErrorReportValvein section 5.
If there is no Context (no web app was mapped), it sends a 404 and returns (lines 81–88).
StandardContextValve — where WEB-INF is blocked
// Disallow any direct access to resources under WEB-INF or META-INF
MessageBytes requestPathMB = request.getRequestPathMB();
if ((requestPathMB.startsWithIgnoreCase("/META-INF/", 0)) || (requestPathMB.equalsIgnoreCase("/META-INF")) ||
(requestPathMB.startsWithIgnoreCase("/WEB-INF/", 0)) || (requestPathMB.equalsIgnoreCase("/WEB-INF"))) {
response.sendError(HttpServletResponse.SC_NOT_FOUND);
return;
}
// Select the Wrapper to be used for this Request
Wrapper wrapper = request.getWrapper();
if (wrapper == null || wrapper.isUnavailable()) {
response.sendError(HttpServletResponse.SC_NOT_FOUND);
return;
}
// Acknowledge the request
response.sendAcknowledgement(ContinueResponseTiming.IMMEDIATELY);
if (request.isAsyncSupported()) {
request.setAsyncSupported(wrapper.getPipeline().isAsyncSupported());
}
wrapper.getPipeline().getFirst().invoke(request, response);
java/org/apache/catalina/core/StandardContextValve.java lines 56–77
These few lines are why requesting /app/WEB-INF/web.xml gives a 404. If the path inside the web app (getRequestPathMB(), the part after the context path) starts with /WEB-INF/ or /META-INF/, it is blocked regardless of case. Note where this happens: in the Context's basic valve, so after every valve on the same Context (the authenticator included), just before the servlet is chosen. Next it gives a client that sent Expect: 100-continue the chance to get a 100 "go ahead" response (line 72), and hands over to the Wrapper pipeline.
StandardWrapperValve — the last valve before the servlet
Since nothing can be attached to a Wrapper pipeline through configuration, it's effectively just this basic valve. Its job is to borrow a servlet instance (wrapper.allocate(), line 113), build the filter chain (line 138), call it (line 164), and give everything back at the end (lines 85–260). The details are in part 9. For this post, remember one thing: exceptions thrown by the servlet are caught by this valve (the catch blocks at lines 169–215). The caught exception goes into the request attribute jakarta.servlet.error.exception and the status becomes (usually) 500; it is not thrown upward again (lines 293–297). The example in section 5 shows what that means.
The three subsections above in one picture. A ✗ marks a place that sends 404 and returns without calling the next pipeline.
5. Breaking the chain and reporting errors
If you don't call the next one, it ends there
There's one way for a valve to block a request: return without calling getNext().invoke(). The example's BlockValve does exactly that.
// Written for this post (not part of Tomcat's source)
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
if (request.getRequestURI().endsWith("/blocked")) {
System.out.println(" BlockValve: 403, next valve is NOT called");
response.sendError(403);
return; // getNext().invoke() is not called — the chain breaks here
}
getNext().invoke(request, response);
}
[Shortened from the example ex/u08-valve-order/lab/BlockValve.java]
/app/blocked is also mapped to the servlet. Here is the result.
===== curl /app/blocked =====
HTTP 403
> engine-1 StandardEngine[Catalina] /app/blocked
> engine-2 StandardEngine[Catalina] /app/blocked
> host StandardHost[localhost] /app/blocked
> context StandardContext[/app] /app/blocked
BlockValve: 403, next valve is NOT called
< context status=403
< host status=403
< engine-2 status=403
< engine-1 status=403
----- body (title) -----
<title>HTTP Status 403 – Forbidden</title>
(In this output and in the /app/boom and /nope outputs below, the LogProbeValve.log() lines, covered in section 6, are left out.) The servlet's lines are missing. The authenticator after BlockValve, StandardContextValve, the Wrapper, the filters and the servlet were all skipped. Yet every outer valve still printed its <. Only the path inward was cut; valves that were already entered finish the rest of their invoke() as usual.
BlockValve only called sendError(403) and wrote no body. Yet the client received HTML titled HTTP Status 403 – Forbidden. That was written by the Host's ErrorReportValve.
The inward path is cut at the red box. The outer valves still do their work on the way back.
ErrorReportValve — a valve that works on the way back
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
// Perform the request
getNext().invoke(request, response);
if (response.isCommitted()) {
// ...
return;
}
Throwable throwable = (Throwable) request.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
// ...
// One way or another, response.sendError() will have been called before
// execution reaches this point and suspended the response. Need to
// reverse that so this valve can write to the response.
response.setSuspended(false);
try {
report(request, response, throwable);
} catch (Throwable tt) {
ExceptionUtils.handleThrowable(tt);
}
}
java/org/apache/catalina/valves/ErrorReportValve.java lines 78–138
This valve calls the next valve first and looks at the response only after everything has finished. If the response has already gone to the client (isCommitted()), it can't change it, so it moves toward closing the connection and returns (lines 84–108). Otherwise report() writes the error screen. report() does nothing if the status is below 400, if any of the body has already been written, or if the error has already been reported elsewhere (lines 183–185).
Calling sendError() puts the response in a "suspended" state where later writes are thrown away. So before writing the error screen, line 131 lifts the suspension. That also means that anything BlockValve might have tried to write after sendError() would never have reached the client.
It matters that this valve sits on the Host. Whatever happened inside the Context pipeline — a valve blocked, the servlet failed, or there was no web app at all — this is a spot every request passes on its way back.
When the servlet throws
/app/boom is a path where the servlet throws an IllegalStateException.
===== curl /app/boom =====
HTTP 500
> engine-1 StandardEngine[Catalina] /app/boom
> engine-2 StandardEngine[Catalina] /app/boom
> host StandardHost[localhost] /app/boom
> context StandardContext[/app] /app/boom
servlet: throwing IllegalStateException
< context status=500
< host status=500
< engine-2 status=500
< engine-1 status=500
----- body (title) -----
<title>HTTP Status 500 – Internal Server Error</title>
TraceValve doesn't wrap getNext().invoke() in a try, yet every < was printed. The exception never reached the valves. As we saw in section 4, StandardWrapperValve caught it, set the status to 500 and put the exception in a request attribute, and ErrorReportValve read that attribute (line 110 in the code above) to draw the 500 screen. The exception's stack trace is logged by StandardWrapperValve (lines 210–215).
When there is no web app
===== curl /nope =====
HTTP 404
> engine-1 StandardEngine[Catalina] /nope
> engine-2 StandardEngine[Catalina] /nope
> host StandardHost[localhost] /nope
< host status=404
< engine-2 status=404
< engine-1 status=404
This server only has app and no root web app (ROOT), so /nope maps to no Context. StandardHostValve got null from request.getContext() and sent a 404; the Context's TraceValve was never called. Valves on a Context only see requests mapped to that web app. To do something for every request, attach to the Host or the Engine.
Easy to get wrong
- A valve's
try/catchcan't catch the servlet's exceptions. As shown above,StandardWrapperValvecatches them first. What a valve can see is the status code and the request attributejakarta.servlet.error.exception. - The web app's
<error-page>and Tomcat's default error screen are drawn by different valves. The former byStandardHostValve, the latter byErrorReportValve. If the web app forwarded to an error page that wrote a body,ErrorReportValve'sreport()hits the "already written" (getContentWritten() > 0) or "already reported" condition and does nothing.
6. AccessLogValve — the log is written after the chain is done
The default server.xml has one of these on the Host.
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
%h is the client address, %t the time, %r the request line, %s the status code and %b the number of body bytes. This pattern is the same as pattern="common" (setPattern() replaces common and combined with fixed patterns, AbstractAccessLogValve.java lines 661–670).
So does this valve's invoke() write the log? No.
@Override
public void invoke(Request request, Response response) throws IOException, ServletException {
if (tlsAttributeRequired) {
// The log pattern uses TLS attributes. Ensure these are populated
// before the request is processed because with NIO2 it is possible
// for the connection to be closed (and the TLS info lost) before
// the access log requests the TLS info. Requesting it now causes it
// to be cached in the request.
request.getAttribute(Globals.CERTIFICATES_ATTR);
}
if (cachedElements != null) {
for (CachedElement element : cachedElements) {
element.cache(request);
}
}
getNext().invoke(request, response);
}
java/org/apache/catalina/valves/AbstractAccessLogValve.java lines 762–778
It reads a few values early that might not be readable later (cache()) and passes on to the next valve. It does nothing after the call returns either. The line itself is built by a separate method, log(Request, Response, long) (lines 782–804), which belongs to the AccessLog interface. AccessLogValve is both a valve and an AccessLog (AbstractAccessLogValve line 129).
Who calls log()?
Example 1's LogProbeValve simply passes through in invoke(), and when log() is called it prints the five Tomcat methods that called it.
< context status=200
< host status=200
< engine-2 status=200
< engine-1 status=200
LogProbeValve.log() status=200 called from:
at org.apache.catalina.core.AccessLogAdapter.log:58
at org.apache.catalina.core.ContainerBase.logAccess:867
at org.apache.catalina.core.ContainerBase.logAccess:874
at org.apache.catalina.connector.CoyoteAdapter.service:409
at org.apache.coyote.http11.Http11Processor.service:404
(This is the second half of the /app/hello output.) log() was called after every valve had printed its < — that is, after the pipeline call had completely finished — from line 409 of CoyoteAdapter.service(). Here is that spot.
// Access log
if (!async && postParseSuccess) {
// Log only if processing was invoked.
// If postParseRequest() failed, it has already logged it.
Context context = request.getContext();
Host host = request.getHost();
// ...
long time = System.nanoTime() - req.getStartTimeNanos();
if (context != null) {
context.logAccess(request, response, time, false);
} else if (response.isError()) {
if (host != null) {
host.logAccess(request, response, time, false);
} else {
connector.getService().getContainer().logAccess(request, response, time, false);
}
}
}
java/org/apache/catalina/connector/CoyoteAdapter.java lines 393–417
This code is inside the finally block of service(), and the try before it has already sent out the whole response with response.finishResponse() (lines 375–376). So the access log is written after the response has gone out. That's how it knows %s (the final status code) and %b (the bytes actually sent). Async requests aren't logged here but when the async processing finishes (the !async condition).
logAccess() starts at the Context and climbs toward the parents.
@Override
public void logAccess(Request request, Response response, long time, boolean useDefault) {
boolean logged = false;
if (getAccessLog() != null) {
getAccessLog().log(request, response, time);
logged = true;
}
if (getParent() != null) {
// No need to use default logger once request/response has been logged
// once
getParent().logAccess(request, response, time, (useDefault && !logged));
}
}
java/org/apache/catalina/core/ContainerBase.java lines 861–876
The first time getAccessLog() is called, it walks its own pipeline and collects every valve that implements AccessLog (lines 878–901). Whether it finds one or several, it bundles them in an AccessLogAdapter, whose log() calls each in turn. ContainerBase.logAccess appears twice in the stack (line 874 → line 867) because the Context found no AccessLog, moved up to its parent the Host, and found it there.
A valve's invoke() only prepares; the actual writing happens after the pipeline is done, climbing back up the containers.
A request with no Context, like /nope, is still logged — through the else if (response.isError()) branch above, host.logAccess() on line 412. In the example output, log() for /nope was indeed called from CoyoteAdapter.service:412. Here is the file the default AccessLogValve left after the example.
127.0.0.1 - - [24/Oct/2026:13:41:13 +0000] "GET /app/hello HTTP/1.1" 200 6
127.0.0.1 - - [24/Oct/2026:13:41:14 +0000] "GET /app/hello HTTP/1.1" 200 6
127.0.0.1 - - [24/Oct/2026:13:41:15 +0000] "GET /app/blocked HTTP/1.1" 403 625
127.0.0.1 - - [24/Oct/2026:13:41:15 +0000] "GET /app/boom HTTP/1.1" 500 1099
127.0.0.1 - - [24/Oct/2026:13:41:16 +0000] "GET /nope HTTP/1.1" 404 689
The times differ on every run. The first line is the request run.sh sends to check that Tomcat is up. The byte counts on the 403, 500 and 404 lines (625, 1099, 689) weren't written by BlockValve or the servlet; they are the size of the error screens ErrorReportValve wrote. The numbers themselves show that the log is written after the response is out.
Easy to get wrong
- For
AccessLogValve, only "which container" matters. Whether it sits early or late within a pipeline, it writes at the same moment. But an access log on a Context only records requests mapped to that web app. - There is a path for when there is no
AccessLogat all. If the Engine has no access log,StandardEngine.logAccess()falls back to the default Host's, then that Host's ROOT web app's access log (StandardEngine.javalines 212–263). It's meant for requests that failed before mapping (whenuseDefaultcomes in astrue).
Rotating the log file by date (rotate()), buffering (buffered, default true, AccessLogValve.java line 107) and the individual pattern letters are not covered here. Logging as a whole is in part 16.
7. RemoteIpValve — a valve that changes things and puts them back
If you put a reverse proxy such as nginx or httpd in front of Tomcat, the client address Tomcat sees is always the proxy's. The proxy writes the original address into the X-Forwarded-For header (httpd part 14 shows the code where mod_proxy adds it), and RemoteIpValve is what trusts that header on the Tomcat side and changes the address for you.
It works like this. If the connecting address is a trusted proxy (internalProxies, which defaults to private ranges such as 127.0.0.0/8, 10.0.0.0/8 and 192.168.0.0/16, lines 365–367), it reads X-Forwarded-For from the right, skipping trusted proxy addresses, and takes the first address that isn't one as the client (lines 605–622). Then it changes the request's remoteAddr (line 634).
Example 2: the servlet and the access log see different addresses
I attached RemoteIpValve (all defaults) to the Engine and two AccessLogValves to the Host that differ in a single setting.
<Engine name="Catalina" defaultHost="localhost">
<Valve className="org.apache.catalina.valves.RemoteIpValve"/>
<!-- ... -->
<Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" prefix="plain" suffix=".txt" pattern="%a %h %U xff="%{X-Forwarded-For}i" %s"/>
<Valve className="org.apache.catalina.valves.AccessLogValve" prefix="attr" suffix=".txt" requestAttributesEnabled="true" pattern="%a %h %U xff="%{X-Forwarded-For}i" %s"/>
</Host>
</Engine>
%a is the client IP, %U the request path, and %{X-Forwarded-For}i the value of that request header. The web app's servlet simply echoes getRemoteAddr() and the X-Forwarded-For header. Since curl runs inside the same container, the connecting address is 127.0.0.1, and I added a header as if the request had passed through two proxies (ex/u08-remoteip-accesslog/run.sh).
===== curl -H 'X-Forwarded-For: 203.0.113.7, 10.0.0.5' /app/who =====
servlet sees remoteAddr=203.0.113.7 X-Forwarded-For=null
===== logs/plain.*.txt =====
127.0.0.1 127.0.0.1 /app/who xff="203.0.113.7, 10.0.0.5" 200
===== logs/attr.*.txt =====
203.0.113.7 203.0.113.7 /app/who xff="203.0.113.7, 10.0.0.5" 200
- The servlet sees
203.0.113.7.10.0.0.5on the right is a private address, so it was treated as a proxy and skipped. The header, fully consumed, was removed, so the servlet getsnull(lines 661–662). - For the same request, the
plainlog recorded127.0.0.1. The header also shows its original value from before it was removed. - Only the
attrlog, withrequestAttributesEnabled="true", recorded203.0.113.7.
The reason is at the end of RemoteIpValve.invoke().
try {
getNext().invoke(request, response);
} finally {
if (!request.isAsync()) {
request.setRemoteAddr(originalRemoteAddr);
request.setRemoteHost(originalRemoteHost);
// ...
}
}
java/org/apache/catalina/valves/RemoteIpValve.java lines 733–759
Once the next valve finishes, this valve puts back everything it changed (address, host, scheme, port, even the headers). But as we saw in section 6, the access log is written after the whole pipeline is done. By then the request's remoteAddr is already back to 127.0.0.1.
That's why RemoteIpValve also leaves the changed values in request attributes before restoring (org.apache.catalina.AccessLog.RemoteAddr and so on, lines 725–732). Request attributes are not restored. The access log reads those attributes only when requestAttributesEnabled is on.
if (requestAttributesEnabled) {
Object addr = request.getAttribute(REMOTE_ADDR_ATTRIBUTE);
if (addr == null) {
value = request.getRemoteAddr();
} else {
value = addr.toString();
}
} else {
value = request.getRemoteAddr();
}
java/org/apache/catalina/valves/AbstractAccessLogValve.java lines 980–989
AccessLogValve's requestAttributesEnabled defaults to false (line 555). On the other hand, RemoteIpValve's setting of the same name defaults to true (line 377), so the attributes are always left behind. To record the real client address in the access log of a Tomcat behind a proxy, give AccessLogValve requestAttributesEnabled="true".
A value changed by a valve lives only until that valve's invoke() ends. Anyone who works afterwards receives it through a request attribute.
Easy to get wrong
- Why bother restoring? It's the same idea as the contract in section 2 about not changing values already used for routing. Whatever a valve changes is visible only inside it, and the outside (earlier valves and
CoyoteAdapter) gets the original request back. What I confirmed in the source is that it restores; the source comments don't spell out the reason. - Put
RemoteIpValveearly. Valves in front of it see the proxy's address. If a valve that blocks by address (such asRemoteCIDRValve) should judge by the real client address,RemoteIpValvemust come before it. As section 2 showed, the order is simply the order you write. - It differs from 11.0.x. On the 11.0.x branch,
RemoteIpValveaccepts either a list of CIDR blocks or a regular expression forinternalProxiesandtrustedProxies. At the reference commit (main), regex support has been removed and only CIDR lists are accepted (the doc comment andsetInternalProxies(), lines 869–880). If you wrote a regex on 11.0.x, you need to rewrite it as a CIDR list for the reference commit. The other files quoted in this post (Pipeline,Valve,StandardPipeline, the four basic valves,ValveBase,ContainerBase) are identical on the 11.0.x branch.
What this post leaves out
- Valve background work (
Valve.backgroundProcess(), called periodically by the container):ContainerBase.javalines 947–955 - Valve JMX names (
type=Valve,...,seq=N):ValveBase.javalines 189–233; JMX as a whole is in part 16 - Replacing the basic valve (
setBasic()) and removing valves (removeValve()):StandardPipeline.javalines 206–372 - The HTML
ErrorReportValvedraws, theshowReportandshowServerInfosettings, and per-<Host>error pages (errorPageSupport):valves/ErrorReportValve.java - Other valves:
RemoteCIDRValveandRemoteHostValve(blocking by address or host name),StuckThreadDetectionValve(finding slow requests),SemaphoreValve(limiting concurrent requests),HealthCheckValve,JsonAccessLogValve,rewrite/RewriteValve— all injava/org/apache/catalina/valves/ - The details of forwarding to a web app's
<error-page>:StandardHostValve.javalines 182–394
Recap
| Concept | One-line summary |
|---|---|
| Pipeline | One per container. Points at valves only through first and basic; the order is a linked list kept in each valve's next |
| The doorway | CoyoteAdapter calls getFirst().invoke() on the Engine pipeline. Pipeline has no invoke() |
| Basic valve | Always at the end. Takes out the mapped Host, Context or Wrapper and calls the first valve of its pipeline |
<Valve> |
Created by the Digester and passed to addValve(). Lands right before the basic valve, so the written order is the execution order |
| Valves added automatically | ErrorReportValve when the Host starts, the authenticator (NonLoginAuthenticator) when the Context is configured |
| Breaking the chain | Return without calling getNext().invoke(). Outer valves continue on their way back |
StandardHostValve |
Switches to the web app's class loader and handles the web app's <error-page> |
StandardContextValve |
Blocks /WEB-INF/ and /META-INF/ with a 404 |
ErrorReportValve |
Calls the next one first; on return, writes the default error screen if in an error state |
| Servlet exceptions | Caught by StandardWrapperValve and turned into 500 plus a request attribute. They never reach the valves |
AccessLogValve |
invoke() only prepares. The writing happens after the response is out, via CoyoteAdapter → logAccess() |
RemoteIpValve |
Changes the address and restores it at the end. To log it, set requestAttributesEnabled="true" |
ValveBase() |
The no-arg constructor turns async support off, and one such valve turns it off for the whole pipeline |
What to look at next
The request has now reached the Wrapper's basic valve, StandardWrapperValve. This valve borrows a servlet instance, gathers the filters that match this request into an ApplicationFilterChain, and calls it. The filter chain looks like the valves but differs too: valves belong to containers, filters belong to the web app's web.xml, and a filter chain is built fresh for every request. That continues in part 9: The Filter Chain and the Servlet Call — Until service() Runs.
The earlier series covered similar things. In Spring part 23, HandlerInterceptor splits before (preHandle), after and completion into separate methods, so DispatcherServlet keeps a number to remember "how far preHandle got" and pairs them up. A valve calls getNext().invoke() wrapped inside a single method, so it needs no such bookkeeping. That's why the > and < in example 1 paired up on their own. The shape is the same as a servlet filter's chain.doFilter() (Spring part 20). The hooks in httpd part 6 are called in a loop by the core, and the return value (OK, DECLINED) decides whether to call the next one. In Tomcat that decision is made by the valve itself, by whether it calls the next one. Writing the access log after the request is completely done is the same in httpd too (the log_transaction hook in httpd part 16).
- Part 7: CoyoteAdapter and the Mapper — Which Servlet Gets the Request
- Part 3: Lifecycle and Startup — What Happens After catalina.sh start
- Part 12: Async Servlets and Non-Blocking I/O — Letting Go of the Thread
- Part 14: Authentication and Realms — How security-constraint Is Enforced
- Part 16: Embedded Tomcat, JMX and Logging — and Wrapping Up the Series