Chapter 11. Mutex Guide

Table of Contents

Locking Global Data
Protecting a Continuation’s Data
How to Associate a Continuation to Every HTTP Transaction
How to Add the New Continuation
How to Store Data Specific to Each HTTP Transaction
Using Locks
Special Case: Continuations Created for HTTP Transactions

Mutexes are used to lock shared data. This chapter explains how to use the mutex interface.

Mutexes

A mutex is the basic synchronization method used within Traffic Server to protect data from simultaneous access by multiple threads. A mutex acts as a lock that protects data in one program thread from being accessed by another thread.

The Traffic Server API provides two functions that attempt to access and lock the data: InkMutexLockTry and INKMutexLock. INKMutexLock is a blocking call; if you use it, you can slow Traffic Server performance (transaction processing pauses until the mutex is unlocked). It should be used only on threads created by the plugin INKContThreadCreate. Never use it on a continuation handler called back by the Cache, Net, or Event Processor. Even if the critical section is very small, do not use it. If you need to update a flag, then set a variable and/or use atomic operations. If INKMutexLock is used in any case other than the one recommended above, then the result will cause serious performance impact.

INKMutexLockTry, on the other hand, attempts to lock the mutex only if it is unlocked (i.e., not being used by another thread). It should be used in all cases other than the above mentioned INKMutexLock case. If the INKMutexLockTry attempt fails, then you can schedule a future attempt (which must be at least 10 milliseconds later).

In general, you should use INKMutexLockTry rather than INKMutexLock.

InkMutexLockTry is required if you are tying to lock Traffic Server internal or system resources (such as the network, cache, eventProcessor, HTTP state machines, and IO buffers).
InkMutexLockTry is required if you are making any blocking calls m(such as network, cache, or file IO calls).
INKMutexLock might not be necessary if you are not making blocking calls and if you are only accessing local resources.

The Traffic Server API uses the INKMutex type for a mutex. There are two typical uses of mutex. One use is for locking global data or data shared by various continuations. The other typical usage is for locking data associated with a continuation (i.e., data that might be accessed by other continuations).

Locking Global Data

The blacklist-1.c sample plugin implements a mutex that locks global data. The blacklist plugin reads its blacklisted sites from a configuration fille; file read operations are protected by a mutex created in INKPluginInit. The blacklist-1.c code uses INKMutexLockTry instead of InkMutexLock. For more detailed information, see the blacklist-1.c code - start by looking at the INKPluginInit function.

General guidelines for locking shared data are as follows:

Create a mutex for the shared data with INKMutexCreate.
Whenever you need to read or modify this data, first lock it by calling INKMutexLockTry; then read or modify the data.
When you are done with the data, unlock it with INKMutexUnlock. If you are unlocking data accessed during the processing of an HTTP transaction, then you must unlock it before calling INKHttpTxnReenable.

Protecting a Continuation’s Data

You must create a mutex to protect a continuation’s data if it might be accessed by other continuations or processes.

Create a mutex for the continuation using INKMutexCreate.
For example:
```
INKMutex mutexp;
mutexp = INKMutexCreate ();
```
When you create the continuation, specify this mutex as the continuation’s mutex.
For example:
```
INKCont contp;
contp = INKContCreate (handler, mutexp);
```

If any other functions want to access contp’s data, then it is up to them to get contp’s mutex (using, for example, INKContMutexGet) to lock it (see the sample Protocol plugin for usage).

How to Associate a Continuation to Every HTTP Transaction

There could be several reasons you'd want to create a continuation for each HTTP transaction that calls back your plugin.
Some potential scenarios are listed below.

You want to register hooks locally with the new continuation instead of registering them globally to the continuation plugin.
You want to store data specific to each HTTP transaction that you might need to reuse across various hooks.
You're using APIs (like INKHostLookup) that will call back this continuation with a certain event.

How to Add the New Continuation

A typical way of adding the new continuation is by registering the plugin continuation to be called back by HTTP transactions globally when they reach INK_HTTP_TXN_START_HOOK. Refer to the example below, which uses a transaction specific continuation called txn_contp.

   void INKPluginInit(int argc, const char *argv[]) 
     { 
        /* Plugin continuation */ 
        INKCont contp; 
        if ((contp = INKContCreate (plugin_cont_handler, NULL)) == INK_ERROR_PTR) { 
            LOG_ERROR("INKContCreate"); 
        } else { 
        if (INKHttpHookAdd (INK_HTTP_TXN_START_HOOK, contp) == INK_ERROR) { 
LOG_ERROR("INKHttpHookAdd"); 
} 
         } 
     }

In the plugin continuation handler, create the new continuation txn_contp and then register it to be called back at INK_HTTP_TXN_CLOSE_HOOK:

static int plugin_cont_handler(INKCont contp, INKEvent event, void *edata) 
     { 
        INKHttpTxn txnp = (INKHttpTxn)edata; 
       INKCont txn_contp; 

         switch (event) { 
         case INK_EVENT_HTTP_TXN_START: 
             /* Create the HTTP txn continuation */ 
             txn_contp = INKContCreate(txn_cont_handler, NULL); 

             /* Register txn_contp to be called back when txnp reaches INK_HTTP_TXN_CLOSE_HOOK */ 
             if (INKHttpTxnHookAdd (txnp, INK_HTTP_TXN_CLOSE_HOOK, txn_contp) == INK_ERROR) { 
                 LOG_ERROR("INKHttpTxnHookAdd"); 
             } 

             break; 

         default: 
             INKAssert(!"Unexpected Event"); 
             break; 
         } 

         if (INKHttpTxnReenable(txnp, INK_EVENT_HTTP_CONTINUE) == INK_ERROR) { 
             LOG_ERROR("INKHttpTxnReenable"); 
         } 

         return 0; 
     }

Remember that the txn_contp handler must destory itself when the HTTP transaction is closed. If you forget, then your plugin will have a memory leak.

static int txn_cont_handler(INKCont txn_contp, INKEvent event, void *edata) 
     { 
         INKHttpTxn txnp; 
         switch (event) { 
         case INK_EVENT_HTTP_TXN_CLOSE: 
             txnp = (INKHttpTxn) edata; 
             INKContDestroy(txn_contp); 
             break; 

         default: 
             INKAssert(!"Unexpected Event"); 
             break; 
         } 

         if (INKHttpTxnReenable(txnp, INK_EVENT_HTTP_CONTINUE) == INK_ERROR) { 
             LOG_ERROR("INKHttpTxnReenable"); 
         } 

         return 0; 
     }

How to Store Data Specific to Each HTTP Transaction

For the example above, store the data in the txn_contp data structure - this means that you'll create your own data structure. Now suppose you want to store the state of the HTTP transaction:

   typedef struct { 
         int state; 
     } ContData;

You would need to allocate the memory and initialize this structure for each HTTP txnp. You can do that in the plugin continuation handler when it is called back with INK_EVENT_HTTP_TXN_START

static int plugin_cont_handler(INKCont contp, INKEvent event, void *edata) 
     { 
        INKHttpTxn txnp = (INKHttpTxn)edata; 
       INKCont txn_contp; 
        ContData *contData; 

         switch (event) { 
         case INK_EVENT_HTTP_TXN_START: 
             /* Create the HTTP txn continuation */ 
             txn_contp = INKContCreate(txn_cont_handler, NULL); 

             /* Allocate and initialize the txn_contp data */ 
             contData = (ContData*) INKmalloc(sizeof(ContData)); 
             contData->state = 0; 
             if (INKContDataSet(txn_contp, contData) == INK_ERROR) { 
                 LOG_ERROR("INKContDataSet"); 
             } 

             /* Register txn_contp to be called back when txnp reaches INK_HTTP_TXN_CLOSE_HOOK */ 
             if (INKHttpTxnHookAdd (txnp, INK_HTTP_TXN_CLOSE_HOOK, txn_contp) == INK_ERROR) { 
                 LOG_ERROR("INKHttpTxnHookAdd"); 
             } 

             break; 

         default: 
             INKAssert(!"Unexpected Event"); 
             break; 
         } 

         if (INKHttpTxnReenable(txnp, INK_EVENT_HTTP_CONTINUE) == INK_ERROR) { 
             LOG_ERROR("INKHttpTxnReenable"); 
         } 

         return 0; 
     }
For accessing this data from anywhere, use INKContDataGet:
INKCont txn_contp; 
     ContData *contData; 

     contData = INKContDataGet(txn_contp); 
     if (contData == INK_ERROR_PTR) { 
         LOG_ERROR("INKContDataGet"); 
     } 
     contData->state = 1;
Remember to free this memory before destroying the continuation:
static int txn_cont_handler(INKCont txn_contp, INKEvent event, void *edata) 
     { 
         INKHttpTxn txnp; 
         ContData *contData; 
         switch (event) { 
         case INK_EVENT_HTTP_TXN_CLOSE: 
             txnp = (INKHttpTxn) edata; 
             contData = INKContDataGet(txn_contp); 
             if (contData == INK_ERROR_PTR) { 
                 LOG_ERROR("INKContDataGet"); 
             } else { 
                 INKfree(contData); 
             } 
             INKContDestroy(txn_contp); 
             break; 

         default: 
             INKAssert(!"Unexpected Event"); 
             break; 
         } 

         if (INKHttpTxnReenable(txnp, INK_EVENT_HTTP_CONTINUE) == INK_ERROR) { 
             LOG_ERROR("INKHttpTxnReenable"); 
         } 

         return 0; 
     }

Using Locks

You do not need to use locks when a continuation has registered itself to be called back by HTTP hooks and it only uses the HTTP APIs. In the example above, the continuation txn_contp has registered itself to be called back at HTTP hooks and it only uses the HTTP APIs. In this case only, it is safe to access data shared between txnp and txn_contp without grabbing a lock. In the example above txn_contp is created with a NULL mutex. This works because the HTTP transaction txnp is the only one that will call back txn_contp, and you are guaranteed that txn_contp will be called back only one hook at a time. After processing is finished, txn_contp will reenable txnp.

In all other cases, you should create a mutex with the continuation. In general, a lock is needed when you're using iocore APIs or any other API where txn_contp is scheduled to be called back by a processor (such as the cache processor, the DNS processor, etc.). This ensures that txn_contp is called back sequentially and not simultaneously. In other words, you need to ensure that txn_contp will not be called back by both txnp and the cache processor at the same time, since this will result in a situation wherein you're executing two pieces of code in conflict!

Special Case: Continuations Created for HTTP Transactions

If your plugin creates a new continuation for each HTTP transaction, then you probably do not have to create a new mutex for it because each HTTP transaction (INKHttpTxn object) already has its own mutex.

In the example below, it is not necessary to specify a mutex for the continuation created in txn_handler:

static void
txn_handler (INKHttpTxn txnp, INKCont contp) {
   INKCont newCont;
   ....
   newCont = INKContCreate (newCont_handler, NULL);
   //It's not necessary to create a new mutex for newCont.

   ...

   INKHttpTxnReenable (txnp, INK_EVENT_HTTP_CONTINUE);
}

static int
test_plugin (INKCont contp, INKEvent event, void *edata) {
   INKHttpTxn txnp = (INKHttpTxn) edata;

   switch (event) {
   case INK_EVENT_HTTP_READ_REQUEST_HDR:
      txn_handler (txnp, contp);
      return 0;
   default:
      break;
   }
   return 0;
}

The mutex functions are listed below:

Traffic Server Software Developers Kit