A heap or shared memory based mutex pool manager

• Download source files - 17.6 KB

Introduction

This article develops a mutex pool manager. The manager is used to manage a mutex pool with a huge mutex number. The mutex pool is allocated from the heap or shared memory based on the manager name parameter specified. An unnamed manager allocates the pool from the heap. Mutexes with an unnamed manager can synchronize threads within a single process. On the other hand, a named manager allocates the pool from the shared memory. Mutexes with a named manager can synchronize threads between processes. The manager can grow the mutex pool automatically based on runtime demand.

The mutex in the pool is implemented using the Windows API InterlockedExchange; the Windows event object is used to synchronize the threads only if there is contention.

If you have a page based memory buffer, the page number is huge, and each page needs a mutex; or if you have an in-memory hash table, the hash bucket number is huge, and each bucket needs a mutex, and the mutex pool manager may serve you very well.

How to Use the Mutex Pool Manager

The first step is to create a mutex pool manager object. The constructor of the mutex manager class is as follows:

CMutexManager::CMutexManager(DWORD dwInitCount, DWORD dwIncCount, const wchar_t *pszName)

dwInitCount specifies the initial mutex count created by the manager. dwIncCount specifies the memory allocation granularity for extending the mutex pool. As the pool grows, memory is allocated in units of dwIncCount mutexes. pszName specifies the manager name, it is limited to 128 characters. Name comparison is case sensitive; if the name matches the name of an existing named mutex manager, dwInitCount and dwIncCount are ignored because they have already been set by the process that originally created the manager. If pszName is NULL, the mutex manager is created without a name. The mutexes with a named manager can synchronize threads between processes, the mutexes with an unnamed manager can synchronize threads within a single process.

The mutex manager exposes three public methods for the applications to call, they are:

bool CMutexManager::CreateMutex(DWORD& dwIndex)

This method creates a mutex from the mutex pool, sets the mutex reference count to one, and returns the index of the mutex being created. A thread must call CloseMutex for each time that it creates the mutex.

bool CMutexManager::CreateMutex(DWORD& dwIndex)
{
    //Check if the manager is initialized

    if (m_bInitialized == false)
    {
        return false;
    }

    //Lock the manager

    CAutoLock<CSysMutex> lockAuto(m_pSysMutex);

    //Get the shared manager header

    MutexSharedManagerHeader *pSharedHeader = 
                      m_sManagerHeader.pSharedHeader;
    //Check if there is free mutex

    if (pSharedHeader->dwMutexNextFree == INVALID_MUTEX_ID) 
    {
        //Check if we can add more mutex

        if (pSharedHeader->dwMutexCount == MAX_MUTEX_COUNT)
        {
            return false;
        }
        //Determine how many new mutex to add

        DWORD dwCount = MAX_MUTEX_COUNT - pSharedHeader->dwMutexCount;
        dwCount = (dwCount >= pSharedHeader->dwIncCount) ? 
                                pSharedHeader->dwIncCount : dwCount;
        //Create one more mutex block

        if (!CreateMutexBlock(dwCount, m_strName.c_str()))
        {
            return false;
        }
    }

    //Get the free mutex ID

    dwIndex = pSharedHeader->dwMutexNextFree;
    //Get the mutex address

    Mutex *pMutex = NULL;
    while (!GetMutex(dwIndex, &pMutex))
    {
        //Create more mutex block

        if (!CreateMutexBlock(pSharedHeader->dwIncCount, 
                              m_strName.c_str()))
        {
            return false;
        }
    }
    //Set the next free mutex index

    pSharedHeader->dwMutexNextFree = pMutex->dwMutexNextFree;
    //Decrease the free mutex count

    --pSharedHeader->dwMutexFree;
    //Increase the inuse mutex count

    ++pSharedHeader->dwMutexInUse;
    //Update the new max inuse mutex count

    if (pSharedHeader->dwMutexInUse > pSharedHeader->dwMutexInUseMax)
    {
        pSharedHeader->dwMutexInUseMax = pSharedHeader->dwMutexInUse;
    }

    //Set 0 to the mutex created

    memset(pMutex, 0, sizeof(pMutex));
    //Assign the index

    pMutex->dwIndex = dwIndex;
    //Increase the mutex reference count

    pMutex->dwMutexRefCount++;

    return true;
}

bool CMutexManager::OpenMutex(DWORD dwIndex)

This method opens a mutex that has been created before, and increases the mutex reference count by one. A thread must call CloseMutex once for each time that it opens the mutex.

bool CMutexManager::OpenMutex(DWORD dwIndex)
{
    //Check if the manager is initialized

    if (m_bInitialized == false)
    {
        return false;
    }

    //Lock the manager

    CAutoLock<CSysMutex> lockAuto(m_pSysMutex);

    //Get the shared manager header

    MutexSharedManagerHeader *pSharedHeader = 
                        m_sManagerHeader.pSharedHeader;
    //Check if the mutex index is within the range

    if (dwIndex >= pSharedHeader->dwMutexCount)
    {
        return false;
    }

    Mutex *pMutex = NULL;
    //Get the mutex address

    while (!GetMutex(dwIndex, &pMutex))
    {
        //Create more mutex block

        if (!CreateMutexBlock(pSharedHeader->dwIncCount, 
                              m_strName.c_str()))
        {
            return false;
        }
    }

    //Check if the mutex has been created

    if (pMutex->dwMutexRefCount == 0)
    {
        return false;
    }
    //Increase the mutex reference count

    pMutex->dwMutexRefCount++;

    return true;
}

bool CMutexManager::CloseMutex(DWORD dwIndex)

For a mutex being created or opened before, this method decreases the mutex reference count by one. If the reference count reaches 0, the mutex manager will mark the mutex free and release it back to the mutex pool. A thread must call CloseMutex once for each time that it creates or opens the mutex.

bool CMutexManager::CloseMutex(DWORD dwIndex)
{
    //Check if the manager is initialized

    if (m_bInitialized == false)
    {
        return false;
    }

    //Lock the manager

    CAutoLock<CSysMutex> lockAuto(m_pSysMutex);

    //Get the shared manager header

    MutexSharedManagerHeader *pSharedHeader = 
                         m_sManagerHeader.pSharedHeader;
    //Check if the mutex index is within the range

    if (dwIndex >= pSharedHeader->dwMutexCount)
    {
        return false;
    }

    Mutex *pMutex = NULL;
    //Get the mutex address

    while (!GetMutex(dwIndex, &pMutex))
    {
        //Create more mutex block

        if (!CreateMutexBlock(pSharedHeader->dwIncCount, m_strName.c_str()))
        {
            return false;
        }
    }

    //Decrease the mutex reference count

    pMutex->dwMutexRefCount--;
    //If the reference count is 0, put the mutex back to the free list

    if (pMutex->dwMutexRefCount == 0)
    {
        pMutex->dwMutexNextFree = pSharedHeader->dwMutexNextFree;
        pSharedHeader->dwMutexNextFree = pMutex->dwIndex;
        //Increase the free mutex count

        ++pSharedHeader->dwMutexFree;
        //Decrease the in use mutex count

        --pSharedHeader->dwMutexInUse;
    }

    return true;
}

The helper class CMutex is provided to lock and unlock a mutex. The constructor of the class is as follows:

CMutex::CMutex(CMutexManager *pManager, DWORD dwIndex)

pManager specifies the mutex pool manager, dwIndex specifies the mutex index.

The class exposes two public methods for the applications to call, they are Lock() and Unlock().

bool CMutex::Lock()

This method waits for the ownership of the specified mutex object. The method returns when the calling thread is granted ownership. After a thread has ownership of a mutex object, it can make additional calls to Lock() without blocking its execution. This prevents a thread from deadlocking itself while waiting for a mutex that it already owns. A thread must call Unlock() once for each time that it calls Lock().

The implementation of Lock():

bool CMutex::Lock()
{
    if ((m_pMutex == NULL) || (m_pManager == NULL))
    {
        return false;
    }

    //Get the process and thread IDs

    DWORD dwProcessId = GetCurrentProcessId();
    DWORD dwThreadId  = GetCurrentThreadId();
    //The event object

    CSysEvent *pEvent = NULL;

    while (true)
    {
        //Enter the critical section

        EnterCriticalSection();

        //Check if the mutex is locked

        if (m_pMutex->bLocked == false)
        {
            //Set the lock flag

            m_pMutex->bLocked = true;
            //Increase the lock reference count

            m_pMutex->dwLockRefCount++;
            //Assign the process and thread IDs

            m_pMutex->dwProcessId = dwProcessId;
            m_pMutex->dwThreadId  = dwThreadId;
            //Free the event

            delete pEvent;
            //Leave the critical section

            LeaveCriticalSection();
            return true;
        } 
        
        //Check if the same thread calls the lock again

        if ((m_pMutex->dwProcessId == dwProcessId) &&
            (m_pMutex->dwThreadId  == dwThreadId))
        {
            //Increase the lock reference count

            m_pMutex->dwLockRefCount++;
            //Free the event

            delete pEvent;
            //Leave the critical section

            LeaveCriticalSection();
            return true;
        }

        //Increase the waiter count

        m_pMutex->dwThreadsWaiting++;
        //Create the event if it's not created before, then wait on the event

        if (pEvent == NULL)
        {
            //Get the event name

            std::wstring strName;
            if (!GetEventName(strName))
            {
                //Leave the critical section

                LeaveCriticalSection();
                return false;
            }
            //Create the event

            pEvent = new (std::nothrow) CSysEvent(false, false, 
                                           strName.c_str(), NULL);
            if (pEvent == NULL)
            {
                //Leave the critical section

                LeaveCriticalSection();
                return false;
            }
        }

        //Leave the critical section

        LeaveCriticalSection();
        //Wait on the event

        if (!pEvent->Lock(INFINITE))
        {
            delete pEvent;
            return false; 
        }
    }

    //Free the event

    delete pEvent;
    //Leave the critical section

    LeaveCriticalSection();

    return true;
}

bool CMutex::Unlock()

This method releases the ownership of the specified mutex object. A thread uses the Lock method to acquire ownership of a mutex object. To release its ownership, the thread must call Unlock() once for each time that it calls the Lock() method.

The implementation of Unlock():

bool CMutex::Unlock()
{
    if ((m_pMutex == NULL) || (m_pManager == NULL))
    {
        return false;
    }

    //Enter the critical section

    EnterCriticalSection();

    //Get the process and thread IDs

    DWORD dwProcessId = GetCurrentProcessId();
    DWORD dwThreadId  = GetCurrentThreadId();

    //Check if the thread is the owner

    if ((m_pMutex->dwProcessId != dwProcessId) ||
        (m_pMutex->dwThreadId  != dwThreadId))
    {
        LeaveCriticalSection();
        return false;
    }

    //Check if the mutex is locked

    if (m_pMutex->bLocked == false)
    {
        LeaveCriticalSection();
        return false;
    }

    //Decrease the lock reference count

    m_pMutex->dwLockRefCount--;
    //Check if the lock reference count is not 0

    if (m_pMutex->dwLockRefCount > 0)
    {
        LeaveCriticalSection();
        return true;
    }

    //Release the lock

    m_pMutex->bLocked = false;
    //Reset the process and thread IDs

    m_pMutex->dwProcessId = 0;
    m_pMutex->dwThreadId  = 0;
   
    //Check if there is a thread waitting on the lock

    if (m_pMutex->dwThreadsWaiting > 0)
    {
        //Get the event name

        std::wstring strName;
        if (!GetEventName(strName))
        {
            LeaveCriticalSection();
            return false;
        }
        //Create the event

        CSysEvent *pEvent = new (std::nothrow)CSysEvent(false, false, 
                                 strName.c_str(), NULL);
        if (pEvent == NULL)
        {
            LeaveCriticalSection();
            return false;
        }
        //Set the event state to signaled

    if (!pEvent->SetEvent ())
        {
            LeaveCriticalSection();
            delete pEvent;
            return false;
        }
        //Free the event

    delete pEvent;
        //Decrease the waiter count

        m_pMutex->dwThreadsWaiting--;
    }
    //Leave the critical section

    LeaveCriticalSection();

    return true;
}

The spin lock methods used by Lock() and Unlock() are:

bool CMutex::EnterCriticalSection()

bool CMutex::EnterCriticalSection()
{
    if (m_pMutex == 0)
    {
        return false;
    }

    //Spin and get the spin lock. Each thread only owns the spin lock 

    //for a very short of period

    while (InterlockedExchange(&(m_pMutex->lSpinLock), 1) != 0)
    {
        //Yield execution to another thread that is ready to run on 

        //the current processor.

        SwitchToThread();
    }

    return true;
}

bool CMutex::LeaveCriticalSection()

bool CMutex::LeaveCriticalSection()
{
    if (m_pMutex == NULL)
    {
        return false;
    }
    //Rlease the spin lock

    InterlockedExchange(&(m_pMutex->lSpinLock), 0);

    return true;
}

Bibliography

• Dan Chou. "A Quick and Versatile Synchronization Object" (MSDN library, Technical articles).