manager-objects.txt 9.8 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303
  1. .. include:: header.txt
  2. =================
  3. Manager objects
  4. =================
  5. A manager object controls a server process which manages *shared
  6. objects*. Other processes can access the shared objects by using
  7. proxies.
  8. Manager processes will be shutdown as soon as they are garbage
  9. collected or their parent process exits. The manager classes are
  10. defined in the `processing.managers` module.
  11. BaseManager
  12. ===========
  13. `BaseManager` is the base class for all manager classes which use a
  14. server process. It does not possess any methods which create shared
  15. objects.
  16. The public methods of `BaseManager` are the following:
  17. `__init__(self, address=None, authkey=None)`
  18. Creates a manager object.
  19. Once created one should call `start()` or `serveForever()` to
  20. ensure that the manager object refers to a started manager
  21. process.
  22. The arguments to the constructor are as follows:
  23. `address`
  24. The address on which the manager process listens for
  25. new connections. If `address` is `None` then an arbitrary
  26. one is chosen.
  27. See `Listener objects <connection-ref.html#listener-objects>`_.
  28. `authkey`
  29. The authentication key which will be used to check the
  30. validity of incoming connections to the server process.
  31. If `authkey` is `None` then `currentProcess().getAuthKey()`.
  32. Otherwise `authkey` is used and it must be a string.
  33. See `Authentication keys
  34. <connection-ref.html#authentication-keys>`_.
  35. `start()`
  36. Spawn or fork a subprocess to start the manager.
  37. `serveForever()`
  38. Start the manager in the current process. See `Using a remote
  39. manager`_.
  40. `fromAddress(address, authkey)`
  41. A class method which returns a manager object referring to a
  42. pre-existing server process which is using the given address and
  43. authentication key. See `Using a remote manager`_.
  44. `shutdown()`
  45. Stop the process used by the manager. This is only available
  46. if `start()` has been used to start the server process.
  47. This can be called multiple times.
  48. `BaseManager` instances also have one read-only property:
  49. `address`
  50. The address used by the manager.
  51. The creation of managers which support arbitrary types is discussed
  52. below in `Customized managers`_.
  53. SyncManager
  54. ===========
  55. `SyncManager` is a subclass of `BaseManager` which can be used for
  56. the synchronization of processes. Objects of this type are returned
  57. by `processing.Manager()`.
  58. It also supports creation of shared lists and dictionaries. The
  59. instance methods defined by `SyncManager` are
  60. `BoundedSemaphore(value=1)`
  61. Creates a shared `threading.BoundedSemaphore` object and
  62. returns a proxy for it.
  63. `Condition(lock=None)`
  64. Creates a shared `threading.Condition` object and returns a
  65. proxy for it.
  66. If `lock` is supplied then it should be a proxy for a
  67. `threading.Lock` or `threading.RLock` object.
  68. `Event()`
  69. Creates a shared `threading.Event` object and returns a proxy
  70. for it.
  71. `Lock()`
  72. Creates a shared `threading.Lock` object and returns a proxy
  73. for it.
  74. `Namespace()`
  75. Creates a shared `Namespace` object and returns a
  76. proxy for it.
  77. See `Namespace objects`_.
  78. `Queue(maxsize=0)`
  79. Creates a shared `Queue.Queue` object and returns a proxy for
  80. it.
  81. `RLock()`
  82. Creates a shared `threading.RLock` object and returns a proxy
  83. for it.
  84. `Semaphore(value=1)`
  85. Creates a shared `threading.Semaphore` object and returns a
  86. proxy for it.
  87. `Array(typecode, sequence)`
  88. Create an array and returns a proxy for
  89. it. (`format` is ignored.)
  90. `Value(typecode, value)`
  91. Create an object with a writable `value` attribute and returns
  92. a proxy for it.
  93. `dict()`, `dict(mapping)`, `dict(sequence)`
  94. Creates a shared `dict` object and returns a proxy for it.
  95. `list()`, `list(sequence)`
  96. Creates a shared `list` object and returns a proxy for it.
  97. Namespace objects
  98. -----------------
  99. A namespace object has no public methods but does have writable
  100. attributes. Its representation shows the values of its attributes.
  101. However, when using a proxy for a namespace object, an attribute
  102. beginning with `'_'` will be an attribute of the proxy and not an
  103. attribute of the referent::
  104. >>> manager = processing.Manager()
  105. >>> Global = manager.Namespace()
  106. >>> Global.x = 10
  107. >>> Global.y = 'hello'
  108. >>> Global._z = 12.3 # this is an attribute of the proxy
  109. >>> print Global
  110. Namespace(x=10, y='hello')
  111. Customized managers
  112. ===================
  113. To create one's own manager one creates a subclass of `BaseManager`.
  114. To create a method of the subclass which will create new shared
  115. objects one uses the following function:
  116. `CreatorMethod(callable=None, proxytype=None, exposed=None, typeid=None)`
  117. Returns a function with signature `func(self, *args, **kwds)`
  118. which will create a shared object using the manager `self`
  119. and return a proxy for it.
  120. The shared objects will be created by evaluating
  121. `callable(*args, **kwds)` in the manager process.
  122. The arguments are:
  123. `callable`
  124. The callable used to create a shared object. If the
  125. manager will connect to a remote manager then this is ignored.
  126. `proxytype`
  127. The type of proxy which will be used for object returned
  128. by `callable`.
  129. If `proxytype` is `None` then each time an object is
  130. returned by `callable` either a new proxy type is created
  131. or a cached one is reused. The methods of the shared
  132. object which will be exposed via the proxy will then be
  133. determined by the `exposed` argument, see below.
  134. `exposed`
  135. Given a shared object returned by `callable`, the
  136. `exposed` argument is the list of those method names which
  137. should be exposed via |callmethod|_. [#]_ [#]_
  138. If `exposed` is `None` and `callable.__exposed__` exists then
  139. `callable.__exposed__` is used instead.
  140. If `exposed` is `None` and `callable.__exposed__` does not
  141. exist then all methods of the shared object which do not
  142. start with `'_'` will be exposed.
  143. An attempt to use |callmethod| with a method name which is
  144. not exposed will raise an exception.
  145. `typeid`
  146. If `typeid` is a string then it is used as an identifier
  147. for the callable. Otherwise, `typeid` must be `None` and
  148. a string prefixed by `callable.__name__` is used as the
  149. identifier.
  150. .. |callmethod| replace:: ``BaseProxy._callMethod()``
  151. .. _callmethod: proxy-objects.html#methods-of-baseproxy
  152. .. [#] A method here means any attribute which has a `__call__`
  153. attribute.
  154. .. [#] The method names `__repr__`, `__str__`, and `__cmp__` of a
  155. shared object are always exposed by the manager. However, instead
  156. of invoking the `__repr__()`, `__str__()`, `__cmp__()` instance
  157. methods (none of which are guaranteed to exist) they invoke the
  158. builtin functions `repr()`, `str()` and `cmp()`.
  159. Note that one should generally avoid exposing rich comparison
  160. methods like `__eq__()`, `__ne__()`, `__le__()`. To make the proxy
  161. type support comparison by value one can just expose `__cmp__()`
  162. instead (even if the referent does not have such a method).
  163. Example
  164. -------
  165. ::
  166. from processing.managers import BaseManager, CreatorMethod
  167. class FooClass(object):
  168. def bar(self):
  169. print 'BAR'
  170. def baz(self):
  171. print 'BAZ'
  172. class NewManager(BaseManager):
  173. Foo = CreatorMethod(FooClass)
  174. if __name__ == '__main__':
  175. manager = NewManager()
  176. manager.start()
  177. foo = manager.Foo()
  178. foo.bar() # prints 'BAR'
  179. foo.baz() # prints 'BAZ'
  180. manager.shutdown()
  181. See `ex_newtype.py <../examples/ex_newtype.py>`_ for more examples.
  182. Using a remote manager
  183. ======================
  184. It is possible to run a manager server on one machine and have clients
  185. use it from other machines (assuming that the firewalls involved allow
  186. it).
  187. Running the following commands creates a server for a shared queue which
  188. remote clients can use::
  189. >>> from processing.managers import BaseManager, CreatorMethod
  190. >>> import Queue
  191. >>> queue = Queue.Queue()
  192. >>> class QueueManager(BaseManager):
  193. ... get_proxy = CreatorMethod(callable=lambda:queue, typeid='get_proxy')
  194. ...
  195. >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='none')
  196. >>> m.serveForever()
  197. One client can access the server as follows::
  198. >>> from processing.managers import BaseManager, CreatorMethod
  199. >>> class QueueManager(BaseManager):
  200. ... get_proxy = CreatorMethod(typeid='get_proxy')
  201. ...
  202. >>> m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none')
  203. >>> queue = m.get_proxy()
  204. >>> queue.put('hello')
  205. Another client can also use it::
  206. >>> from processing.managers import BaseManager, CreatorMethod
  207. >>> class QueueManager(BaseManager):
  208. ... get_proxy = CreatorMethod(typeid='get_proxy')
  209. ...
  210. >>> m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none')
  211. >>> queue = m.get_proxy()
  212. >>> queue.get()
  213. 'hello'
  214. .. _Prev: connection-objects.html
  215. .. _Up: processing-ref.html
  216. .. _Next: proxy-objects.html