| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245 |
- :mod:`lockfile` --- Platform-independent file locking
- =====================================================
- .. module:: lockfile
- :synopsis: Platform-independent file locking
- .. moduleauthor:: Skip Montanaro <skip@pobox.com>
- .. sectionauthor:: Skip Montanaro <skip@pobox.com>
- .. note::
- This module is alpha software. It is quite possible that the API and
- implementation will change in important ways as people test it and
- provide feedback and bug fixes. In particular, if the mkdir-based
- locking scheme is sufficient for both Windows and Unix platforms, the
- link-based scheme may be deleted so that only a single locking scheme is
- used, providing cross-platform lockfile cooperation.
- .. note::
- The implementation uses the :keyword:`with` statement, both in the
- tests and in the main code, so will only work out-of-the-box with Python
- 2.5 or later. However, the use of the :keyword:`with` statement is
- minimal, so if you apply the patch in the included 2.4.diff file you can
- use it with Python 2.4. It's possible that it will work in Python 2.3
- with that patch applied as well, though the doctest code relies on APIs
- new in 2.4, so will have to be rewritten somewhat to allow testing on
- 2.3. As they say, patches welcome. ``;-)``
- The :mod:`lockfile` module exports a :class:`FileLock` class which provides
- a simple API for locking files. Unlike the Windows :func:`msvcrt.locking`
- function, the Unix :func:`fcntl.flock`, :func:`fcntl.lockf` and the
- deprecated :mod:`posixfile` module, the API is identical across both Unix
- (including Linux and Mac) and Windows platforms. The lock mechanism relies
- on the atomic nature of the :func:`link` (on Unix) and :func:`mkdir` (On
- Windows) system calls.
- .. note::
- The current implementation uses :func:`os.link` on Unix, but since that
- function is unavailable on Windows it uses :func:`os.mkdir` there. At
- this point it's not clear that using the :func:`os.mkdir` method would be
- insufficient on Unix systems. If it proves to be adequate on Unix then
- the implementation could be simplified and truly cross-platform locking
- would be possible.
- .. note::
- The current implementation doesn't provide for shared vs. exclusive
- locks. It should be possible for multiple reader processes to hold the
- lock at the same time.
- The module defines the following exceptions:
- .. exception:: Error
- This is the base class for all exceptions raised by the :class:`LockFile`
- class.
- .. exception:: LockError
- This is the base class for all exceptions raised when attempting to lock
- a file.
- .. exception:: UnlockError
- This is the base class for all exceptions raised when attempting to
- unlock a file.
- .. exception:: LockTimeout
- This exception is raised if the :func:`LockFile.acquire` method is
- called with a timeout which expires before an existing lock is released.
- .. exception:: AlreadyLocked
- This exception is raised if the :func:`LockFile.acquire` detects a
- file is already locked when in non-blocking mode.
- .. exception:: LockFailed
- This exception is raised if the :func:`LockFile.acquire` detects some
- other condition (such as a non-writable directory) which prevents it from
- creating its lock file.
- .. exception:: NotLocked
- This exception is raised if the file is not locked when
- :func:`LockFile.release` is called.
- .. exception:: NotMyLock
- This exception is raised if the file is locked by another thread or
- process when :func:`LockFile.release` is called.
- The following classes are provided:
- .. class:: LinkFileLock(path, threaded=True)
- This class uses the :func:`link(2)` system call as the basic lock
- mechanism. *path* is an object in the file system to be locked. It need
- not exist, but its directory must exist and be writable at the time the
- :func:`acquire` and :func:`release` methods are called. *threaded* is
- optional, but when set to :const:`True` locks will be distinguished
- between threads in the same process.
- .. class:: MkdirFileLock(path, threaded=True)
- This class uses the :func:`mkdir(2)` system call as the basic lock
- mechanism. The parameters have the same meaning as for the
- :class:`LinkFileLock` class.
- .. class:: SQLiteFileLock(path, threaded=True)
- This class uses the :mod:`sqlite3` module to implement the lock
- mechanism. The parameters have the same meaning as for the
- :class:`LinkFileLock` class.
- By default, the :const:`FileLock` object refers to the
- :class:`MkdirFileLock` class on Windows. On all other platforms it refers
- to the :class:`LinkFileLock` class.
- When locking a file the :class:`LinkFileLock` class creates a uniquely named
- hard link to an empty lock file. That hard link contains the hostname,
- process id, and if locks between threads are distinguished, the thread
- identifier. For example, if you want to lock access to a file named
- "README", the lock file is named "README.lock". With per-thread locks
- enabled the hard link is named HOSTNAME-THREADID-PID. With only per-process
- locks enabled the hard link is named HOSTNAME--PID.
- When using the :class:`MkdirFileLock` class the lock file is a directory.
- Referring to the example above, README.lock will be a directory and
- HOSTNAME-THREADID-PID will be an empty file within that directory.
- .. seealso::
- Module :mod:`msvcrt`
- Provides the :func:`locking` function, the standard Windows way of
- locking (parts of) a file.
- Module :mod:`posixfile`
- The deprecated (since Python 1.5) way of locking files on Posix systems.
- Module :mod:`fcntl`
- Provides the current best way to lock files on Unix systems
- (:func:`lockf` and :func:`flock`).
- Implementing Other Locking Schemes
- ----------------------------------
- There is a :class:`LockBase` base class which can be used as the foundation
- for other locking schemes. For example, if shared filesystems are not
- available, :class:`LockBase` could be subclassed to provide locking via an
- SQL database.
- FileLock Objects
- ----------------
- :class:`FileLock` objects support the :term:`context manager` protocol used
- by the statement:`with` statement. The timeout option is not supported when
- used in this fashion. While support for timeouts could be implemented,
- there is no support for handling the eventual :exc:`Timeout` exceptions
- raised by the :func:`__enter__` method, so you would have to protect the
- :keyword:`with` statement with a :keyword:`try` statement. The resulting
- construct would not be much simpler than just using a :keyword:`try` statement
- in the first place.
- :class:`FileLock` has the following user-visible methods:
- .. method:: FileLock.acquire(timeout=None)
- Lock the file associated with the :class:`FileLock` object. If the
- *timeout* is omitted or :const:`None` the caller will block until the
- file is unlocked by the object currently holding the lock. If the
- *timeout* is zero or a negative number the :exc:`AlreadyLocked` exception
- will be raised if the file is currently locked by another process or
- thread. If the *timeout* is positive, the caller will block for that
- many seconds waiting for the lock to be released. If the lock is not
- released within that period the :exc:`LockTimeout` exception will be
- raised.
- .. method:: FileLock.release()
- Unlock the file associated with the :class:`FileLock` object. If the
- file is not currently locked, the :exc:`NotLocked` exception is raised.
- If the file is locked by another thread or process the :exc:`NotMyLock`
- exception is raised.
- .. method:: is_locked()
- Return the status of the lock on the current file. If any process or
- thread (including the current one) is locking the file, :const:`True` is
- returned, otherwise :const:`False` is returned.
- .. method:: break_lock()
- If the file is currently locked, break it.
- Examples
- --------
- This example is the "hello world" for the :mod:`lockfile` module::
- lock = FileLock("/some/file/or/other")
- with lock:
- print lock.path, 'is locked.'
- To use this with Python 2.4, you can execute::
- lock = FileLock("/some/file/or/other")
- lock.acquire()
- print lock.path, 'is locked.'
- lock.release()
- If you don't want to wait forever, you might try::
- lock = FileLock("/some/file/or/other")
- while not lock.i_am_locking():
- try:
- lock.acquire(timeout=60) # wait up to 60 seconds
- except LockTimeout:
- lock.break_lock()
- lock.acquire()
- print "I locked", lock.path
- lock.release()
- Other Libraries
- ---------------
- The idea of implementing advisory locking with a standard API is not new
- with :mod:`lockfile`. There are a number of other libraries available:
- * locknix - http://pypi.python.org/pypi/locknix - Unix only
- * mx.MiscLockFile - from Marc André Lemburg, part of the mx.Base
- distribution - cross-platform.
- * Twisted - http://twistedmatrix.com/trac/browser/trunk/twisted/python/lockfile.py
- * zc.lockfile - http://pypi.python.org/pypi/zc.lockfile
- Contacting the Author
- ---------------------
- If you encounter any problems with ``lockfile``, would like help or want to
- submit a patch, contact me directly: Skip Montanaro (skip@pobox.com).
|