lockfile.rst 9.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245
  1. :mod:`lockfile` --- Platform-independent file locking
  2. =====================================================
  3. .. module:: lockfile
  4. :synopsis: Platform-independent file locking
  5. .. moduleauthor:: Skip Montanaro <skip@pobox.com>
  6. .. sectionauthor:: Skip Montanaro <skip@pobox.com>
  7. .. note::
  8. This module is alpha software. It is quite possible that the API and
  9. implementation will change in important ways as people test it and
  10. provide feedback and bug fixes. In particular, if the mkdir-based
  11. locking scheme is sufficient for both Windows and Unix platforms, the
  12. link-based scheme may be deleted so that only a single locking scheme is
  13. used, providing cross-platform lockfile cooperation.
  14. .. note::
  15. The implementation uses the :keyword:`with` statement, both in the
  16. tests and in the main code, so will only work out-of-the-box with Python
  17. 2.5 or later. However, the use of the :keyword:`with` statement is
  18. minimal, so if you apply the patch in the included 2.4.diff file you can
  19. use it with Python 2.4. It's possible that it will work in Python 2.3
  20. with that patch applied as well, though the doctest code relies on APIs
  21. new in 2.4, so will have to be rewritten somewhat to allow testing on
  22. 2.3. As they say, patches welcome. ``;-)``
  23. The :mod:`lockfile` module exports a :class:`FileLock` class which provides
  24. a simple API for locking files. Unlike the Windows :func:`msvcrt.locking`
  25. function, the Unix :func:`fcntl.flock`, :func:`fcntl.lockf` and the
  26. deprecated :mod:`posixfile` module, the API is identical across both Unix
  27. (including Linux and Mac) and Windows platforms. The lock mechanism relies
  28. on the atomic nature of the :func:`link` (on Unix) and :func:`mkdir` (On
  29. Windows) system calls.
  30. .. note::
  31. The current implementation uses :func:`os.link` on Unix, but since that
  32. function is unavailable on Windows it uses :func:`os.mkdir` there. At
  33. this point it's not clear that using the :func:`os.mkdir` method would be
  34. insufficient on Unix systems. If it proves to be adequate on Unix then
  35. the implementation could be simplified and truly cross-platform locking
  36. would be possible.
  37. .. note::
  38. The current implementation doesn't provide for shared vs. exclusive
  39. locks. It should be possible for multiple reader processes to hold the
  40. lock at the same time.
  41. The module defines the following exceptions:
  42. .. exception:: Error
  43. This is the base class for all exceptions raised by the :class:`LockFile`
  44. class.
  45. .. exception:: LockError
  46. This is the base class for all exceptions raised when attempting to lock
  47. a file.
  48. .. exception:: UnlockError
  49. This is the base class for all exceptions raised when attempting to
  50. unlock a file.
  51. .. exception:: LockTimeout
  52. This exception is raised if the :func:`LockFile.acquire` method is
  53. called with a timeout which expires before an existing lock is released.
  54. .. exception:: AlreadyLocked
  55. This exception is raised if the :func:`LockFile.acquire` detects a
  56. file is already locked when in non-blocking mode.
  57. .. exception:: LockFailed
  58. This exception is raised if the :func:`LockFile.acquire` detects some
  59. other condition (such as a non-writable directory) which prevents it from
  60. creating its lock file.
  61. .. exception:: NotLocked
  62. This exception is raised if the file is not locked when
  63. :func:`LockFile.release` is called.
  64. .. exception:: NotMyLock
  65. This exception is raised if the file is locked by another thread or
  66. process when :func:`LockFile.release` is called.
  67. The following classes are provided:
  68. .. class:: LinkFileLock(path, threaded=True)
  69. This class uses the :func:`link(2)` system call as the basic lock
  70. mechanism. *path* is an object in the file system to be locked. It need
  71. not exist, but its directory must exist and be writable at the time the
  72. :func:`acquire` and :func:`release` methods are called. *threaded* is
  73. optional, but when set to :const:`True` locks will be distinguished
  74. between threads in the same process.
  75. .. class:: MkdirFileLock(path, threaded=True)
  76. This class uses the :func:`mkdir(2)` system call as the basic lock
  77. mechanism. The parameters have the same meaning as for the
  78. :class:`LinkFileLock` class.
  79. .. class:: SQLiteFileLock(path, threaded=True)
  80. This class uses the :mod:`sqlite3` module to implement the lock
  81. mechanism. The parameters have the same meaning as for the
  82. :class:`LinkFileLock` class.
  83. By default, the :const:`FileLock` object refers to the
  84. :class:`MkdirFileLock` class on Windows. On all other platforms it refers
  85. to the :class:`LinkFileLock` class.
  86. When locking a file the :class:`LinkFileLock` class creates a uniquely named
  87. hard link to an empty lock file. That hard link contains the hostname,
  88. process id, and if locks between threads are distinguished, the thread
  89. identifier. For example, if you want to lock access to a file named
  90. "README", the lock file is named "README.lock". With per-thread locks
  91. enabled the hard link is named HOSTNAME-THREADID-PID. With only per-process
  92. locks enabled the hard link is named HOSTNAME--PID.
  93. When using the :class:`MkdirFileLock` class the lock file is a directory.
  94. Referring to the example above, README.lock will be a directory and
  95. HOSTNAME-THREADID-PID will be an empty file within that directory.
  96. .. seealso::
  97. Module :mod:`msvcrt`
  98. Provides the :func:`locking` function, the standard Windows way of
  99. locking (parts of) a file.
  100. Module :mod:`posixfile`
  101. The deprecated (since Python 1.5) way of locking files on Posix systems.
  102. Module :mod:`fcntl`
  103. Provides the current best way to lock files on Unix systems
  104. (:func:`lockf` and :func:`flock`).
  105. Implementing Other Locking Schemes
  106. ----------------------------------
  107. There is a :class:`LockBase` base class which can be used as the foundation
  108. for other locking schemes. For example, if shared filesystems are not
  109. available, :class:`LockBase` could be subclassed to provide locking via an
  110. SQL database.
  111. FileLock Objects
  112. ----------------
  113. :class:`FileLock` objects support the :term:`context manager` protocol used
  114. by the statement:`with` statement. The timeout option is not supported when
  115. used in this fashion. While support for timeouts could be implemented,
  116. there is no support for handling the eventual :exc:`Timeout` exceptions
  117. raised by the :func:`__enter__` method, so you would have to protect the
  118. :keyword:`with` statement with a :keyword:`try` statement. The resulting
  119. construct would not be much simpler than just using a :keyword:`try` statement
  120. in the first place.
  121. :class:`FileLock` has the following user-visible methods:
  122. .. method:: FileLock.acquire(timeout=None)
  123. Lock the file associated with the :class:`FileLock` object. If the
  124. *timeout* is omitted or :const:`None` the caller will block until the
  125. file is unlocked by the object currently holding the lock. If the
  126. *timeout* is zero or a negative number the :exc:`AlreadyLocked` exception
  127. will be raised if the file is currently locked by another process or
  128. thread. If the *timeout* is positive, the caller will block for that
  129. many seconds waiting for the lock to be released. If the lock is not
  130. released within that period the :exc:`LockTimeout` exception will be
  131. raised.
  132. .. method:: FileLock.release()
  133. Unlock the file associated with the :class:`FileLock` object. If the
  134. file is not currently locked, the :exc:`NotLocked` exception is raised.
  135. If the file is locked by another thread or process the :exc:`NotMyLock`
  136. exception is raised.
  137. .. method:: is_locked()
  138. Return the status of the lock on the current file. If any process or
  139. thread (including the current one) is locking the file, :const:`True` is
  140. returned, otherwise :const:`False` is returned.
  141. .. method:: break_lock()
  142. If the file is currently locked, break it.
  143. Examples
  144. --------
  145. This example is the "hello world" for the :mod:`lockfile` module::
  146. lock = FileLock("/some/file/or/other")
  147. with lock:
  148. print lock.path, 'is locked.'
  149. To use this with Python 2.4, you can execute::
  150. lock = FileLock("/some/file/or/other")
  151. lock.acquire()
  152. print lock.path, 'is locked.'
  153. lock.release()
  154. If you don't want to wait forever, you might try::
  155. lock = FileLock("/some/file/or/other")
  156. while not lock.i_am_locking():
  157. try:
  158. lock.acquire(timeout=60) # wait up to 60 seconds
  159. except LockTimeout:
  160. lock.break_lock()
  161. lock.acquire()
  162. print "I locked", lock.path
  163. lock.release()
  164. Other Libraries
  165. ---------------
  166. The idea of implementing advisory locking with a standard API is not new
  167. with :mod:`lockfile`. There are a number of other libraries available:
  168. * locknix - http://pypi.python.org/pypi/locknix - Unix only
  169. * mx.MiscLockFile - from Marc André Lemburg, part of the mx.Base
  170. distribution - cross-platform.
  171. * Twisted - http://twistedmatrix.com/trac/browser/trunk/twisted/python/lockfile.py
  172. * zc.lockfile - http://pypi.python.org/pypi/zc.lockfile
  173. Contacting the Author
  174. ---------------------
  175. If you encounter any problems with ``lockfile``, would like help or want to
  176. submit a patch, contact me directly: Skip Montanaro (skip@pobox.com).