Introduction

The Wrong Memory

The objective of this book is to develop a particular way of understanding memory-safety bugs in the C programming language. Memory safety (for the purpose of this book) concerns whether accesses remain valid with respect to objects, bounds, storage, and lifetimes.

The C programming language (C) emerged from a design context in which direct manipulation of machine resources was a feature rather than an anomaly. Its relatively low-level memory model permits programmers to manipulate memory addresses directly, without generally enforcing that each resulting access remains within the bounds and lifetime of the object concerned. Many of the difficulties we now call memory-safety problems arise from these design choices.

C provides a low-level model in which memory can be manipulated through an intricate interplay of numbers, addresses, objects, storage, and lifetimes. Programmers must keep the relationships between these elements consistent, because an error in one can propagate through the others. A wrong size can produce an undersized allocation; a pointer calculation can then produce a pointer that appears valid but falls outside that allocation; a subsequent copy can overrun the object. The resulting corruption may remain hidden until much later, when its original cause is difficult to trace.

There is a simple sentence that captures a recurring pattern in many serious C memory bugs:

I calculated, addressed, allocated, copied, or freed the wrong amount or location of memory.

The first central idea of this book is: memory bugs are often chains of incorrect assumptions rather than isolated mistakes.

Memory safety bugs in C are often neatly categorized as:

integer overflow
buffer overflow
out-of-bounds access
use-after-free
double free
memory corruption

These categories are useful for describing failures, but they can hide the relationship between them. Real-world C bugs are often messy, intertwined failures.

Consider a program that receives a count, calculates a size from that count, allocates memory, walks through the resulting object, and eventually copies data into it.

There are many places where something can go wrong.

The count might be wrong.

The calculation might be wrong.

The allocation might be too small.

The pointer might be calculated from the wrong value.

The bounds might be wrong.

The copy might be too large.

The object might no longer exist.

The eventual failure might therefore occur several steps away from the original mistake.

The second central idea of this book is: we should follow the chain rather than examining each operation in isolation.

The memory chain

A useful starting point is:

integer arithmetic
    ->
allocation size
    ->
pointer arithmetic
    ->
bounds
    ->
memory corruption

This is not a rigid sequence. Real programs will skip steps, repeat them, or introduce other operations between them.

'The memory chain' is a way of thinking.

When a number is used to describe memory, we should ask where that number came from.

When an address is calculated, we should ask what object it is intended to identify.

When memory is allocated, we should ask what logical object the allocation is supposed to contain.

When a range is accessed, we should ask why the program believes that range is valid.

And when memory is freed, we should ask whether the pointer still identifies the intended object.

The answers form a chain.

From code to assumptions

C code often looks deceptively local.

For example:

size = count * element_size; // Calculate the total number of bytes needed
char *p = malloc(size); // Allocate a block of memory of that total size on the heap
char *q = p + offset; // Compute a new pointer q by adding an offset
memcpy(q, src, length); // Copy length bytes from src to the memory beginning at q

Each statement is short.

The interesting question is not whether each statement looks reasonable by itself.

The interesting question is whether the assumptions connecting them are true.

What does `count` represent?

What does `size` represent?

What object does `p` point to?

What does `offset` measure?

What is the relationship between `length` and the remaining space?

The answers to those questions turn a collection of statements into a chain of reasoning, forcing you to deduce the global invariants of the system and view the code as a logical proof that memory is allocated and addressed correctly.

A memory-safety bug is simply what happens when the programmer's assumptions in that proof no longer match the actual state of memory.

The wrong amount and the wrong location

The phrase at the heart of this book has two particularly important dimensions:

amount

and:

location.

A program can use the correct location but the wrong amount of memory.

It can use the correct amount but the wrong location.

It can get both wrong.

These cases lead to different manifestations, but the underlying question is the same:

What memory did the programmer believe this operation referred to,
and what memory did it actually refer to?

That question is often more useful than simply asking which bug category applies.

Why follow the chain?

Suppose a program crashes during a copy.

The copy may be where the problem becomes visible, but it may not be where the problem began.

Perhaps an input value was converted incorrectly.

Perhaps an arithmetic calculation overflowed.

Perhaps an allocation was therefore too small.

Perhaps a later pointer calculation still used the original logical size.

Perhaps the copy finally crossed the physical boundary of the allocation.

Looking only at the final copy tells us what went wrong at the end of the chain.

Following the chain tells us why.

That distinction matters both for debugging and for security.

It determines whether we merely patch the operation that happened to fail or repair the assumption that made the failure possible.

A way of reading C

The chapters that follow will use this idea as a recurring method of analysis.

We will not treat integer arithmetic, allocation, pointers, bounds, and memory corruption as unrelated topics.

Instead, we will follow values as they move through a program.

A value may begin as:

input

become:

count

then:

size

then:

allocation

then:

offset

then:

address

then:

memory access.

At each stage, the program makes an assumption about what that value means.

Our job is to determine whether the assumption remains true.

This approach also explains why apparently minor C details can become security vulnerabilities. A small numerical mistake is not necessarily dangerous by itself. It becomes dangerous when the resulting value is trusted to describe memory.

The question to keep asking

Throughout the book, when you encounter code that manipulates memory, return to the same statement:

I calculated, addressed, allocated, copied, or freed the wrong amount or location of memory.

Then ask:

Which one?

Where?

Based on which value?

Calculated how?

Relative to which object?

And what happens next?

Those questions are enough to begin tracing many difficult memory bugs.

The chapters that follow will fill in the technical details.

The goal is not to memorize a longer list of dangerous C constructs.

It is to develop a way of seeing the relationships between numbers, objects, addresses, and memory.

Once those relationships become visible, many apparently different C memory vulnerabilities start to look like variations of the same problem:

the program believed it was operating on one region of memory,
but the calculation, address, allocation, copy, or lifetime
described another.

That is the wrong memory.