#memory-pool
Posts
-
Digging into Kafka's Source 5: Inside the Producer — From send() to Batches and the Sender Thread
KafkaProducer.send() sends nothing over the network. It serializes the record, picks a partition, appends it to a per-partition batch in the RecordAccumulator and hands back a Future; the one thread that actually sends is kafka-producer-network-thread. We follow in the source the if in ready() where linger.ms and batch.size apply, the built-in partitioner that sticks keyless records to one partition, the BufferPool that hands out batch memory and max.block.ms, retries and delivery.timeout.ms, and max.in.flight.requests.per.connection, then check them with examples that count batches with kafka-dump-log.sh and print which thread runs the callback.
-
Digging into nginx's Source 3: Memory Pools and Core Data Structures — ngx_pool_t, ngx_str_t, ngx_buf_t and ngx_chain_t
You will hardly find free() in nginx's C code either. Memory that shares a lifetime — one request, one connection — goes into a pool and is thrown away in one go. This post checks, against the real source, debug logs and gdb, how ngx_pool_t chains its blocks and uses the failed counter, the separate large list for big allocations, cleanups, the non-NUL-terminated ngx_str_t, arrays, lists, queues and hashes, and the ngx_buf_t and ngx_chain_t that carry responses.
-
Digging into Apache httpd's Source 4: APR Memory Pools — Managing Memory Without free()
You will hardly find free() in httpd's C code. Instead of handing memory back piece by piece, httpd groups everything with the same lifetime — "one request", "one connection" — into a pool and throws it away in one go. This post looks at APR pool nodes and the allocator, why apr_palloc is just a pointer bump, the order cleanups run in, where httpd creates and destroys its pools, and the two classic mistakes — all checked against the real source and real runs.