Thread Safe Object Pool In C++

During the development phase of a project I was faced with a task where I had to create and destroy objects frequently. As most of you know, creating and destroying objects can be a very expensive task. At the frequency that I had to deal with object deletion and creation it was obvious that I could not achieve the performance requirements the application had to meet. To make matters worse some of the objects had a heavy constructor which dealt with expensive instructions. So I had to device a mechanism to reduce the overhead which meant creating an object pool or a memory pool. The route I had to take in this situation was an object pool since a memory pool was infeasible because of the heavy constructor.

For those who are not familiar with the object pool pattern, an object pool is a creational pattern which enables applications to re-use created objects by recycling. Contrary to the conventional method of creation and deletion, an object pool stores the used objects in a pool and re-submits them (after restoring it to its initial state) to the application when it requires. By using this pattern the overhead of creating and destroying objects can be minimized as well as making the object retrieval time constant.

Now back to the problem at hand. The application was a multi-threaded application which needed to transfer objects between threads, which implied that the objects passed had a probability of being destroyed in a different thread than it was originally created. This raised new problems. The objects had to be handled in multiple threads, which implied that a shared pool should be kept. Taking this route meant, for retrieval, locks should be used. As the number of threads increase the use of locks will degrade performance rapidly making all the additional trouble of the object pool in vain. The second option was to keep a small pool of objects for each thread. But that route also had a serious pitfall. If one thread was only producing objects and the other thread was only consuming objects, the first thread had to create objects in the conventional sense (using new or malloc) since the object pool would always be empty (only creation). The second thread would have a high number of objects in its pool because the high number of object consumptions. This is also unacceptable since the solution would not be effective in situations like this. 

As a solution thread specific pools were kept as well as a global pool, where chunk of objects would be transferred to the global pool after a limit has been exceeded, and any thread with an empty object pool could get objects from the global pool (first transferring a chunk of objects to the thread and pushing an object out from that chunk). This decision led to a couple of advantages, upon object retrieval no locks were needed since the objects were taken from the local pool and the local limits could be tuned to get the maximum performance with minimum memory consumption.

The implementation was straight forward. For the local pool static thread specific template pointers were kept to keep track of the local pool (which was simply a linked list). Static was used since multiple pools in the same thread should not create multiple linked lists. It was combined with the thread specific variable specifier to stop it from being a global variable (The combination guaranteed that multiple pools of the same type used the same linked list in the same thread). The global linked list was a static linked list. An option of passing an allocator and a deallocator was also given since that opened up the option of memory allocation for objects in different methods (VirtualAlloc, LocalAlloc, new, malloc, shared memory... etc.). A default allocator and a deallocator was assigned with the new and delete combination so that it was unnecessary to write allocators and deallocators each and every-time the pool was used.

Variables were required to keep a track of the thread pools (next, previous) I was left with a decision between keeping wrappers for the objects or enforcing the object to have the additional variables that was needed for tracking. I took the latter decision since the prior method required to pool the wrapper objects as well as keeping track of them, the additional overhead seemed overkill since most of the time it was just a matter of defining a couple of variables in the pooled object (class or the struct). So it was made a requirement that the object pooled had the following variables

_next - Should be of the pooled type

_previous - Should be of the pooled type

_in_pool - bool (to keep track if the object is already pooled/avoid pooling the object twice by mistake)

Even though the description of the internals were lengthy (hope not boring) the method in which the pool should be used is straight forward. The following sample shows how to create an object pool with a custom allocator and a deallocator and how to create and destroy objects.

#include <iostream>
#include "ObjectPool.h"

using namespace std;

struct tmp
{
    tmp* _next;
    tmp* _previous;
    bool _in_pool;
    int i;
};

struct CustomDeAllocator
{
    void operator () (void* obj, size_t size) const
    {
       tmp* t = (tmp*)obj;
       delete t;
    }
};

struct CustomAllocator
{
    void* operator () (size_t size) const
    {
       return ::operator new(size);
    }
};

int main()
{
    Utils::ObjectPool<tmp, CustomAllocator, CustomDeAllocator> pool;

    tmp* t1 = pool.Create();
    pool.Destroy(t1);

    return 0;
}

The source code for the object pool can be downloaded from the link below.

Object Pool Code