Custom Memory Management

Designing a custom memory

We have come a long way in this chapter now. We have covered the basics of virtual memory, the stack and the heap, the new and delete expressions, memory ownership, and alignment and padding. But before we close this chapter, we will show how to customize memory management in C++. We will see how the parts we went through earlier in this chapter will come in handy when writing a custom memory allocator.

But first, what is a custom memory manager, and why do we need one?

When using new or malloc() to allocate memory, we use the built-in memory management system in C++. Most implementations of operator new use malloc(), a general-purpose memory allocator. Designing and building a general-purpose memory manager is a complicated task, and there are many people who have already spent a lot of time researching this topic. Still, there are several reasons why we might want to write a custom memory manager. Here are some examples:

• Debugging and diagnostics: We have already done this a couple of times in this chapter by overloading operator new and operator delete, just to print out some debugging information.

• Sandboxing: A custom memory manager can provide a sandbox for code that isn’t allowed to allocate unrestricted memory. The sandbox can also track memory allocations and release memory when the sandboxed code finishes executing.

• Performance: If we need dynamic memory and can’t avoid allocations, we may have to write a custom memory manager that performs better for our specific needs. Later on, we will cover some of the circumstances that we could utilize to outperform malloc().

With that said, many experienced C++ programmers have never faced a problem that actually required them to customize the standard memory manager that comes with the system. This is a good indication of how good the general-purpose memory managers actually are today, despite all the requirements they have to fulfill without any knowledge about our specific use cases. The more we know about the memory usage patterns in our application, the better the chances are that we can actually write something more efficient than malloc(). Remember the stack, for example? Allocating and deallocating memory from the stack is very fast compared to the heap, thanks to the fact that it doesn’t need to handle multiple threads and that deallocations are guaranteed to always happen in reverse order.

Building a custom memory manager usually starts with analyzing the exact memory usage patterns and implementing an arena.

Building an arena

Two frequently used terms when working with memory allocators are arena and memory pool. By arena, we means a block of contiguous memory, including a strategy for handing out parts of that memory and reclaiming it later on.

An arena could technically also be called a memory resource or an allocator, but those terms will refer to abstractions from the standard library. The custom allocator we will develop later will be implemented using the arena we create here.

Some general strategies can be used when designing an arena that will make allocations and deallocations likely to perform better than malloc() and free():

• Single-threaded: If we know that an arena will only be used from one thread, there is no need to protect data with synchronization primitives, such as locks or atomics. There is no risk that the client using the arena may be blocked by some other thread, which is important in real-time contexts.

• Fixed-size allocations: If the arena only hands out memory blocks of a fixed size, it is relatively easy to reclaim memory efficiently without memory fragmentation by using a free list.

• Limited lifetime: If we know that objects allocated from an arena only need to live during a limited and well-defined lifetime, the arena can postpone reclamation and free the memory all at once. An example could be objects created while handling a request in a server application. When the request has finished, all the memory that was handed out during the request can be reclaimed in one step. Of course, the arena needs to be big enough to handle all the allocations during the request without reclaiming memory continually; otherwise, this strategy will not work.

We will not go into further details about these strategies, but it is good to be aware of the possibilities when looking for ways to improve memory management in your program. As is often the case with optimizing software, the key is to understand the circumstances under which your program will run and to analyze the specific memory usage patterns. We do this to find ways to improve a custom memory manager compared to a general-purpose one.