Python 3.2 threading.Lock and signals (last update: 2016-12-04, created: 2015-12-05) back to the list ↑
While coding in Python 3.4/3.5 (which I'm trying to switch to from Python 2.7) I noticed the following note in the documentation of the threading.Lock.acquire method:

Changed in version 3.2: Lock acquires can now be interrupted by signals on POSIX.

I interpreted the above line (spoiler: incorrectly) as "acquire() can now return when a POSIX signal is sent to the process" (in addition to the previous two cases of it returning when the attempt timed out or when the lock was in fact acquired.

However, while the above does make some sense in the context of the aforementioned lock acquisition method, it makes less sense when using the RAII method in form of with lock_object: statement - when entering the inner context of with one expects the lock to be owned by the current thread; if a signal would interrupt the acquisition that obviously would not be the case. So I started digging into the code.

First thing I found out, was that threading.Lock (Lib/threading.py) is just a reference to _thread.allocate_lock:

...
_allocate_lock = _thread.allocate_lock
...
# Synchronization classes

Lock = _allocate_lock

The _thread module is implemented in C (Modules/_threadmodule.c in Python 3.5.0 source code):

static struct PyModuleDef threadmodule = {
    PyModuleDef_HEAD_INIT,
    "_thread",
    thread_doc,
    -1,
    thread_methods,
    NULL,
    NULL,
    NULL,
    NULL
}; 

Since I wasn't really interested in the _thread.allocate_lock function, I went looking for __enter__ instead - it's the method invoked when then with statement is used on an object before the inner block is entered (__exit__ is called once the execution leaves the inner block / with statement):

static PyMethodDef lock_methods[] = {
    {"acquire_lock", (PyCFunction)lock_PyThread_acquire_lock, 
...
    {"acquire",      (PyCFunction)lock_PyThread_acquire_lock, 
...
    {"__enter__",    (PyCFunction)lock_PyThread_acquire_lock, 
...
};

First thing to notice is that there is an acquire_lock method which is not mentioned in the documentation - of course it's just an alias of the standard acquire method:

>>> import threading
>>> x=threading.Lock()
>>> x.acquire
<built-in method acquire of _thread.lock object at 0x7f75fde9f0f8>
>>> x.acquire_lock
<built-in method acquire_lock of _thread.lock object at 0x7f75fde9f0f8>

But that's hardly important. The important thing to note is that __enter__ is also handled by the same function, meaning that with's behavior is actually identical to that of acquire method. To go deeper one needs to look at the lock_PyThread_acquire_lock itself:

static PyObject *
lock_PyThread_acquire_lock(lockobject *self, PyObject *args, PyObject *kwds)
{
    _PyTime_t timeout;
    PyLockStatus r;

    if (lock_acquire_parse_args(args, kwds, &timeout) < 0)
        return NULL;

    r = acquire_timed(self->lock_lock, timeout);
    if (r == PY_LOCK_INTR) {
        return NULL;
    }


    if (r == PY_LOCK_ACQUIRED)
        self->locked = 1;
    return PyBool_FromLong(r == PY_LOCK_ACQUIRED);
}

The interesting thing here is if (r == PY_LOCK_INTR) { return NULL; } - "if there was a lock interruption, return NULL". The "return NULL" part here actually means "an exception has been set (raised)". So should I assume that an exception is raised if a signal was sent? If so, why doesn't the Lock.acquire() method's documentation mention any exceptions? As usual, one needs to go deeper - into the acquire_timed function:

/* Helper to acquire an interruptible lock with a timeout.  If the lock acquire
 * is interrupted, signal handlers are run, and if they raise an exception,
 * PY_LOCK_INTR is returned.  Otherwise, PY_LOCK_ACQUIRED or PY_LOCK_FAILURE
 * are returned, depending on whether the lock can be acquired withing the
 * timeout.
 */
static PyLockStatus
acquire_timed(PyThread_type_lock lock, _PyTime_t timeout)

...
    do {
...
        if (r == PY_LOCK_INTR) {
            /* Run signal handlers if we were interrupted.  Propagate
             * exceptions from signal handlers, such as KeyboardInterrupt, by
             * passing up PY_LOCK_INTR.  */
            if (Py_MakePendingCalls() < 0) {
                return PY_LOCK_INTR;
            } 
...
        }
    } while (r == PY_LOCK_INTR);  /* Retry if we were interrupted. */

    return r;
}

And that solves the riddle - PY_LOCK_INTR is ever returned by this function only if an invoked signal handler raises an exception; in such case the exception is passed. Otherwise, i.e. when a signal was handled (or ignored) with no exceptions, lock acquisition will automatically restart (and timeout time will be recalculated - I omitted this part in the snipped above).

Let's make a test to confirm the finding (Python 3.4):

>>> import signal
>>> import threading
>>> def s(ss,f): print("SIGNAL BOOM!")
... 
>>> signal.signal(signal.SIGINT, s)
<built-in function default_int_handler>
>>> x = threading.Lock()
>>> x.acquire()
True
>>> with x as k: print(k)
...  (at this point I've launched another console and run kill -2 PID)
SIGNAL BOOM!
(lock acquisition is still blocking - good)

Given the above, the cryptic Lock acquires can now be interrupted by signals on POSIX probably means that in previous versions of Python the signal handler would never be executed if a lock acquisition attempt is in progress. This, again, can be confirmed with a simple experiment (Python 2.7 this time):

>>> import signal
>>> import threading
>>> def s(ss,f): print("SIGNAL BOOM!")
... 
>>> signal.signal(signal.SIGINT, s)
<built-in function default_int_handler>
>>> 
>>> x = threading.Lock()
>>> x.acquire()
True
>>> with x as k: print(k)
...  (at this point I've launched another console and run: kill -2 PID)
(nothing happend)
^C
(right... so how do I exit...)
^Z
[1]+  Stopped                 python
> kill -9 $(jobs -p)
[1]+  Killed                  python

And that's that.
【 design & art by Xa / Gynvael Coldwind 】 【 logo font (birdman regular) by utopiafonts / Dale Harris 】