CHANGES.html 38 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532
  1. <?xml version="1.0" encoding="utf-8" ?>
  2. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  3. <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  4. <head>
  5. <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  6. <meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" />
  7. <title>Changelog for processing</title>
  8. <link rel="stylesheet" href="html4css1.css" type="text/css" />
  9. </head>
  10. <body>
  11. <div class="document" id="changelog-for-processing">
  12. <h1 class="title">Changelog for processing</h1>
  13. <div class="section">
  14. <h1><a id="changes-in-0-52" name="changes-in-0-52">Changes in 0.52</a></h1>
  15. <ul>
  16. <li><p class="first">On versions 0.50 and 0.51 Mac OSX <tt class="docutils literal"><span class="pre">Lock.release()</span></tt> would fail with
  17. <tt class="docutils literal"><span class="pre">OSError(errno.ENOSYS,</span> <span class="pre">&quot;[Errno</span> <span class="pre">78]</span> <span class="pre">Function</span> <span class="pre">not</span> <span class="pre">implemented&quot;)</span></tt>.
  18. This appears to be because on Mac OSX <tt class="docutils literal"><span class="pre">sem_getvalue()</span></tt> has not been
  19. implemented.</p>
  20. <p>Now <tt class="docutils literal"><span class="pre">sem_getvalue()</span></tt> is no longer needed. Unfortunately, however,
  21. on Mac OSX <tt class="docutils literal"><span class="pre">BoundedSemaphore()</span></tt> will not raise <tt class="docutils literal"><span class="pre">ValueError</span></tt> if it
  22. exceeds its initial value.</p>
  23. </li>
  24. <li><p class="first">Some changes to the code for the reduction/rebuilding of connection
  25. and socket objects so that things work the same on Windows and Unix.
  26. This should fix a couple of bugs.</p>
  27. </li>
  28. <li><p class="first">The code has been changed to consistently use &quot;camelCase&quot; for
  29. methods and (non-factory) functions. In the few cases where this
  30. has meant a change to the documented API, the old name has been
  31. retained as an alias.</p>
  32. </li>
  33. </ul>
  34. </div>
  35. <div class="section">
  36. <h1><a id="changes-in-0-51" name="changes-in-0-51">Changes in 0.51</a></h1>
  37. <ul>
  38. <li><p class="first">In 0.50 <tt class="docutils literal"><span class="pre">processing.Value()</span></tt> and <tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt>
  39. were related but had different signatures, which was rather
  40. confusing.</p>
  41. <p>Now <tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt> has been renamed
  42. <tt class="docutils literal"><span class="pre">processing.sharedctypes.RawValue()</span></tt> and
  43. <tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt> is the same as <tt class="docutils literal"><span class="pre">processing.Value()</span></tt>.</p>
  44. </li>
  45. <li><p class="first">In version 0.50 <tt class="docutils literal"><span class="pre">sendfd()</span></tt> and <tt class="docutils literal"><span class="pre">recvfd()</span></tt> apparently did not work on
  46. 64bit Linux. This has been fixed by reverting to using the CMSG_*
  47. macros as was done in 0.40.</p>
  48. <p>However, this means that systems without all the necessary CMSG_*
  49. macros (such as Solaris 8) will have to disable compilation of
  50. <tt class="docutils literal"><span class="pre">sendfd()</span></tt> and <tt class="docutils literal"><span class="pre">recvfd()</span></tt> by setting <tt class="docutils literal"><span class="pre">macros['HAVE_FD_TRANSFRER']</span> <span class="pre">=</span> <span class="pre">0</span></tt>
  51. in <tt class="docutils literal"><span class="pre">setup.py</span></tt>.</p>
  52. </li>
  53. <li><p class="first">Fixed an authentication error when using a &quot;remote&quot; manager created
  54. using <tt class="docutils literal"><span class="pre">BaseManager.from_address()</span></tt>.</p>
  55. </li>
  56. <li><p class="first">Fixed a couple of bugs which only affected Python 2.4.</p>
  57. </li>
  58. </ul>
  59. </div>
  60. <div class="section">
  61. <h1><a id="changes-in-0-50" name="changes-in-0-50">Changes in 0.50</a></h1>
  62. <ul>
  63. <li><p class="first"><tt class="docutils literal"><span class="pre">ctypes</span></tt> is now a prerequisite if you want to use shared memory --
  64. with Python 2.4 you will need to install it separately.</p>
  65. </li>
  66. <li><p class="first"><tt class="docutils literal"><span class="pre">LocalManager()</span></tt> has been removed.</p>
  67. </li>
  68. <li><p class="first">Added <tt class="docutils literal"><span class="pre">processing.Value()</span></tt> and <tt class="docutils literal"><span class="pre">processing.Array()</span></tt>
  69. which are similar to <tt class="docutils literal"><span class="pre">LocalManager.SharedValue()</span></tt> and
  70. <tt class="docutils literal"><span class="pre">LocalManager.SharedArray()</span></tt>.</p>
  71. </li>
  72. <li><p class="first">In the <tt class="docutils literal"><span class="pre">sharedctypes</span></tt> module <tt class="docutils literal"><span class="pre">new_value()</span></tt> and <tt class="docutils literal"><span class="pre">new_array()</span></tt> have
  73. been renamed <tt class="docutils literal"><span class="pre">Value()</span></tt> and <tt class="docutils literal"><span class="pre">Array()</span></tt>.</p>
  74. </li>
  75. <li><p class="first"><tt class="docutils literal"><span class="pre">Process.stop()</span></tt>, <tt class="docutils literal"><span class="pre">Process.getStoppable()</span></tt> and
  76. <tt class="docutils literal"><span class="pre">Process.setStoppable()</span></tt> have been removed. Use
  77. <tt class="docutils literal"><span class="pre">Process.terminate()</span></tt> instead.</p>
  78. </li>
  79. <li><p class="first"><tt class="docutils literal"><span class="pre">procesing.Lock</span></tt> now matches <tt class="docutils literal"><span class="pre">threading.Lock</span></tt> behaviour more
  80. closely: now a thread can release a lock it does not own, and now
  81. when a thread tries acquiring a lock it already owns a deadlock
  82. results instead of an exception.</p>
  83. </li>
  84. <li><p class="first">On Windows when the main thread is blocking on a method of <tt class="docutils literal"><span class="pre">Lock</span></tt>,
  85. <tt class="docutils literal"><span class="pre">RLock</span></tt>, <tt class="docutils literal"><span class="pre">Semaphore</span></tt>, <tt class="docutils literal"><span class="pre">BoundedSemaphore</span></tt>, <tt class="docutils literal"><span class="pre">Condition</span></tt> it will no
  86. longer ignore Ctrl-C. (The same was already true on Unix.)</p>
  87. <p>This differs from the behaviour of the equivalent objects in
  88. <tt class="docutils literal"><span class="pre">threading</span></tt> which will completely ignore Ctrl-C.</p>
  89. </li>
  90. <li><p class="first">The <tt class="docutils literal"><span class="pre">test</span></tt> sub-package has been replaced by lots of unit tests in a
  91. <tt class="docutils literal"><span class="pre">tests</span></tt> sub-package. Some of the old test files have been moved
  92. over to a new <tt class="docutils literal"><span class="pre">examples</span></tt> sub-package.</p>
  93. </li>
  94. <li><p class="first">On Windows it is now possible for a non-console python program
  95. (i.e. one using <tt class="docutils literal"><span class="pre">pythonw.exe</span></tt> instead of <tt class="docutils literal"><span class="pre">python.exe</span></tt>) to use
  96. <tt class="docutils literal"><span class="pre">processing</span></tt>.</p>
  97. <p>Previously an exception was raised when <tt class="docutils literal"><span class="pre">subprocess.py</span></tt> tried to
  98. duplicate stdin, stdout, stderr.</p>
  99. </li>
  100. <li><p class="first">Proxy objects should now be thread safe -- they now use thread local
  101. storage.</p>
  102. </li>
  103. <li><p class="first">Trying to transfer shared resources such as locks, queues etc
  104. between processes over a pipe or queue will now raise <tt class="docutils literal"><span class="pre">RuntimeError</span></tt>
  105. with a message saying that the object should only be shared between
  106. processes using inheritance.</p>
  107. <p>Previously, this worked unreliably on Windows but would fail with an
  108. unexplained <tt class="docutils literal"><span class="pre">AssertionError</span></tt> on Unix.</p>
  109. </li>
  110. <li><p class="first">The names of some of the macros used for compiling the extension
  111. have changed. See <tt class="docutils literal"><span class="pre">INSTALL.txt</span></tt> and <tt class="docutils literal"><span class="pre">setup.py</span></tt>.</p>
  112. </li>
  113. <li><p class="first">A few changes which (hopefully) make compilation possible on Solaris.</p>
  114. </li>
  115. <li><p class="first">Lots of refactoring of the code.</p>
  116. </li>
  117. <li><p class="first">Fixed reference leaks so that unit tests pass with &quot;regrtest -R::&quot;
  118. (at least on Linux).</p>
  119. </li>
  120. </ul>
  121. </div>
  122. <div class="section">
  123. <h1><a id="changes-in-0-40" name="changes-in-0-40">Changes in 0.40</a></h1>
  124. <ul>
  125. <li><p class="first">Removed <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> and <tt class="docutils literal"><span class="pre">PosixQueue</span></tt> types. Just use <tt class="docutils literal"><span class="pre">Queue</span></tt> instead.</p>
  126. </li>
  127. <li><p class="first">Previously if you forgot to use the</p>
  128. <pre class="literal-block">
  129. if __name__ == '__main__':
  130. freezeSupport()
  131. ...
  132. </pre>
  133. <p>idiom on Windows then processes could be created recursively
  134. bringing the computer to its knees. Now <tt class="docutils literal"><span class="pre">RuntimeError</span></tt> will be
  135. raised instead.</p>
  136. </li>
  137. <li><p class="first">Some refactoring of the code.</p>
  138. </li>
  139. <li><p class="first">A Unix specific bug meant that a child process might fail to start a
  140. feeder thread for a queue if its parent process had already started
  141. its own feeder thread. Fixed.</p>
  142. </li>
  143. </ul>
  144. </div>
  145. <div class="section">
  146. <h1><a id="changes-in-0-39" name="changes-in-0-39">Changes in 0.39</a></h1>
  147. <ul>
  148. <li><p class="first">One can now create one-way pipes by doing
  149. <tt class="docutils literal"><span class="pre">reader,</span> <span class="pre">writer</span> <span class="pre">=</span> <span class="pre">Pipe(duplex=False)</span></tt>.</p>
  150. </li>
  151. <li><p class="first">Rewrote code for managing shared memory maps.</p>
  152. </li>
  153. <li><p class="first">Added a <tt class="docutils literal"><span class="pre">sharedctypes</span></tt> module for creating <tt class="docutils literal"><span class="pre">ctypes</span></tt> objects allocated
  154. from shared memory. On Python 2.4 this requires the installation of
  155. <tt class="docutils literal"><span class="pre">ctypes</span></tt>.</p>
  156. <p><tt class="docutils literal"><span class="pre">ctypes</span></tt> objects are not protected by any locks so you will need to
  157. synchronize access to them (such as by using a lock). However they
  158. can be much faster to access than equivalent objects allocated using
  159. a <tt class="docutils literal"><span class="pre">LocalManager</span></tt>.</p>
  160. </li>
  161. <li><p class="first">Rearranged documentation.</p>
  162. </li>
  163. <li><p class="first">Previously the C extension caused a segfault on 64 bit machines with
  164. Python 2.5 because it used <tt class="docutils literal"><span class="pre">int</span></tt> instead of <tt class="docutils literal"><span class="pre">Py_ssize_t</span></tt> in certain
  165. places. This is now fixed. Thanks to Alexy Khrabrov for the report.</p>
  166. </li>
  167. <li><p class="first">A fix for <tt class="docutils literal"><span class="pre">Pool.terminate()</span></tt>.</p>
  168. </li>
  169. <li><p class="first">A fix for cleanup behaviour of <tt class="docutils literal"><span class="pre">Queue</span></tt>.</p>
  170. </li>
  171. </ul>
  172. </div>
  173. <div class="section">
  174. <h1><a id="changes-in-0-38" name="changes-in-0-38">Changes in 0.38</a></h1>
  175. <ul>
  176. <li><p class="first">Have revamped the queue types. Now the queue types are
  177. <tt class="docutils literal"><span class="pre">Queue</span></tt>, <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> and (on systems which support it)
  178. <tt class="docutils literal"><span class="pre">PosixQueue</span></tt>.</p>
  179. <p>Now <tt class="docutils literal"><span class="pre">Queue</span></tt> should behave just like Python's normal <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt>
  180. class except that <tt class="docutils literal"><span class="pre">qsize()</span></tt>, <tt class="docutils literal"><span class="pre">task_done()</span></tt> and <tt class="docutils literal"><span class="pre">join()</span></tt> are not
  181. implemented. In particular, if no maximum size was specified when
  182. the queue was created then <tt class="docutils literal"><span class="pre">put()</span></tt> will always succeed without
  183. blocking.</p>
  184. <p>A <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> instance is really just a pipe protected by a couple
  185. of locks. It has <tt class="docutils literal"><span class="pre">get()</span></tt>, <tt class="docutils literal"><span class="pre">put()</span></tt> and <tt class="docutils literal"><span class="pre">empty()</span></tt> methods but does
  186. not not support timeouts or non-blocking.</p>
  187. <p><tt class="docutils literal"><span class="pre">BufferedPipeQueue()</span></tt> and <tt class="docutils literal"><span class="pre">PipeQueue()</span></tt> remain as deprecated
  188. aliases of <tt class="docutils literal"><span class="pre">Queue()</span></tt> but <tt class="docutils literal"><span class="pre">BufferedPosixQueue()</span></tt> has been removed.
  189. (Not sure if we really need to keep <tt class="docutils literal"><span class="pre">PosixQueue()</span></tt>...)</p>
  190. </li>
  191. <li><p class="first">Previously the <tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> method was a little dodgy -- it
  192. could block indefinitely if <tt class="docutils literal"><span class="pre">map()</span></tt> or <tt class="docutils literal"><span class="pre">imap*()</span></tt> were used and did
  193. not try to terminate workers while they were doing a task.</p>
  194. <p>Now there are three new methods <tt class="docutils literal"><span class="pre">close()</span></tt>, <tt class="docutils literal"><span class="pre">terminate()</span></tt> and
  195. <tt class="docutils literal"><span class="pre">join()</span></tt> -- <tt class="docutils literal"><span class="pre">shutdown()</span></tt> is retained as a deprecated alias of
  196. <tt class="docutils literal"><span class="pre">terminate()</span></tt>. Thanks to Gerald John M. Manipon for feature
  197. request/suggested patch to <tt class="docutils literal"><span class="pre">shutdown()</span></tt>.</p>
  198. </li>
  199. <li><p class="first"><tt class="docutils literal"><span class="pre">Pool.imap()</span></tt> and <tt class="docutils literal"><span class="pre">Pool.imap_unordered()</span></tt> has gained a <tt class="docutils literal"><span class="pre">chunksize</span></tt>
  200. argument which allows the iterable to be submitted to the pool in
  201. chunks. Choosing <tt class="docutils literal"><span class="pre">chunksize</span></tt> appropriately makes <tt class="docutils literal"><span class="pre">Pool.imap()</span></tt>
  202. almost as fast as <tt class="docutils literal"><span class="pre">Pool.map()</span></tt> even for long iterables and cheap
  203. functions.</p>
  204. </li>
  205. <li><p class="first">Previously on Windows when the cleanup code for a <tt class="docutils literal"><span class="pre">LocalManager</span></tt>
  206. attempts to unlink the name of the file which backs the shared
  207. memory map an exception is raised if a child process still exists
  208. which has a handle open for that mmap. This is likely to happen if
  209. a daemon process inherits a <tt class="docutils literal"><span class="pre">LocalManager</span></tt> instance.</p>
  210. <p>Now the parent process will remember the filename and attempt to
  211. unlink the file name again once all the child processes have been
  212. joined or terminated. Reported by Paul Rudin.</p>
  213. </li>
  214. <li><p class="first"><tt class="docutils literal"><span class="pre">types.MethodType</span></tt> is registered with <tt class="docutils literal"><span class="pre">copy_reg</span></tt> so now instance
  215. methods and class methods should be picklable. (Unfortunately there is
  216. no obvious way of supporting the pickling of staticmethods since
  217. they are not marked with the class in which they were defined.)</p>
  218. <p>This means that on Windows it is now possible to use an instance
  219. method or class method as the target callable of a Process object.</p>
  220. </li>
  221. <li><p class="first">On Windows <tt class="docutils literal"><span class="pre">reduction.fromfd()</span></tt> now returns true instances of
  222. <tt class="docutils literal"><span class="pre">_socket.socket</span></tt>, so there is no more need for the
  223. <tt class="docutils literal"><span class="pre">_processing.falsesocket</span></tt> type.</p>
  224. </li>
  225. </ul>
  226. </div>
  227. <div class="section">
  228. <h1><a id="changes-in-0-37" name="changes-in-0-37">Changes in 0.37</a></h1>
  229. <ul class="simple">
  230. <li>Updated metadata and documentation because the project is now hosted
  231. at <tt class="docutils literal"><span class="pre">developer.berlios.de/projects/pyprocessing</span></tt>.</li>
  232. <li>The <tt class="docutils literal"><span class="pre">Pool.join()</span></tt> method has been removed. <tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> will
  233. now join the worker processes automatically.</li>
  234. <li>A pool object no longer participates in a reference cycle so
  235. <tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> should get called as soon as its reference count
  236. falls to zero.</li>
  237. <li>On Windows if <tt class="docutils literal"><span class="pre">enableLogging()</span></tt> was used at module scope then the
  238. logger used by a child process would often get two copies of the
  239. same handler. To fix this, now specifiying a handler type in
  240. <tt class="docutils literal"><span class="pre">enableLogging()</span></tt> will cause any previous handlers used by the
  241. logger to be discarded.</li>
  242. </ul>
  243. </div>
  244. <div class="section">
  245. <h1><a id="changes-in-0-36" name="changes-in-0-36">Changes in 0.36</a></h1>
  246. <ul>
  247. <li><p class="first">In recent versions on Unix the finalizers in a manager process were
  248. never given a chance to run before <tt class="docutils literal"><span class="pre">os._exit()</span></tt> was called, so old
  249. unlinked AF_UNIX sockets could accumulate in '/tmp'. Fixed.</p>
  250. </li>
  251. <li><p class="first">The shutting down of managers has been cleaned up.</p>
  252. </li>
  253. <li><p class="first">In previous versions on Windows trying to acquire a lock owned by a
  254. different thread of the current process would raise an exception.
  255. Fixed.</p>
  256. </li>
  257. <li><p class="first">In previous versions on Windows trying to use an event object for
  258. synchronization between two threads of the same process was likely
  259. to raise an exception. (This was caused by the bug described
  260. above.) Fixed.</p>
  261. </li>
  262. <li><p class="first">Previously the arguments to <tt class="docutils literal"><span class="pre">processing.Semaphore()</span></tt> and
  263. <tt class="docutils literal"><span class="pre">processing.BoundedSemaphore()</span></tt> did not have any defaults. The
  264. defaults should be 1 to match <tt class="docutils literal"><span class="pre">threading</span></tt>. Fixed.</p>
  265. </li>
  266. <li><p class="first">It should now be possible for a Windows Service created by using
  267. <tt class="docutils literal"><span class="pre">pywin32</span></tt> to spawn processes using the <tt class="docutils literal"><span class="pre">processing</span></tt> package.</p>
  268. <p>Note that <tt class="docutils literal"><span class="pre">pywin32</span></tt> apparently has a bug meaning that <tt class="docutils literal"><span class="pre">Py_Finalize()</span></tt>
  269. is never called when the service exits so functions registered with
  270. <tt class="docutils literal"><span class="pre">atexit</span></tt> never get a chance to run. Therefore it is advisable to
  271. explicitly call <tt class="docutils literal"><span class="pre">sys.exitfunc()</span></tt> or <tt class="docutils literal"><span class="pre">atexit._run_exitfuncs()</span></tt> at the
  272. end of <tt class="docutils literal"><span class="pre">ServiceFramework.DoSvcRun()</span></tt>. Otherwise child processes are
  273. liable to survive the service when it is stopped. Thanks to Charlie
  274. Hull for the report.</p>
  275. </li>
  276. <li><p class="first">Added <tt class="docutils literal"><span class="pre">getLogger()</span></tt> and <tt class="docutils literal"><span class="pre">enableLogging()</span></tt> to support logging.</p>
  277. </li>
  278. </ul>
  279. </div>
  280. <div class="section">
  281. <h1><a id="changes-in-0-35" name="changes-in-0-35">Changes in 0.35</a></h1>
  282. <ul>
  283. <li><p class="first">By default processes are no longer be stoppable using the <tt class="docutils literal"><span class="pre">stop()</span></tt>
  284. method: one must call <tt class="docutils literal"><span class="pre">setStoppable(True)</span></tt> before <tt class="docutils literal"><span class="pre">start()</span></tt> in order
  285. to use the <tt class="docutils literal"><span class="pre">stop()</span></tt> method. (Note that <tt class="docutils literal"><span class="pre">terminate()</span></tt> will work
  286. regardless of whether the process is marked as being &quot;stoppable&quot;.)</p>
  287. <p>The reason for this is that on Windows getting <tt class="docutils literal"><span class="pre">stop()</span></tt> to work
  288. involves starting a new console for the child process and installing
  289. a signal handler for the <tt class="docutils literal"><span class="pre">SIGBREAK</span></tt> signal. This unfortunately
  290. means that Ctrl-Break cannot not be used to kill all processes of
  291. the program.</p>
  292. </li>
  293. <li><p class="first">Added <tt class="docutils literal"><span class="pre">setStoppable()</span></tt> and <tt class="docutils literal"><span class="pre">getStoppable()</span></tt> methods -- see above.</p>
  294. </li>
  295. <li><p class="first">Added <tt class="docutils literal"><span class="pre">BufferedQueue</span></tt>/<tt class="docutils literal"><span class="pre">BufferedPipeQueue</span></tt>/<tt class="docutils literal"><span class="pre">BufferedPosixQueue</span></tt>.
  296. Putting an object on a buffered queue will always succeed without
  297. blocking (just like with <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> if no maximum size is
  298. specified). This makes them potentially safer than the normal queue
  299. types provided by <tt class="docutils literal"><span class="pre">processing</span></tt> which have finite capacity and may
  300. cause deadlocks if they fill.</p>
  301. <p><tt class="docutils literal"><span class="pre">test/test_worker.py</span></tt> has been updated to use <tt class="docutils literal"><span class="pre">BufferedQueue</span></tt> for
  302. the task queue instead of explicitly spawning a thread to feed tasks
  303. to the queue without risking a deadlock.</p>
  304. </li>
  305. <li><p class="first">Now when the NO_SEM_TIMED macro is set polling will be used to get
  306. around the lack of <tt class="docutils literal"><span class="pre">sem_timedwait()</span></tt>. This means that
  307. <tt class="docutils literal"><span class="pre">Condition.wait()</span></tt> and <tt class="docutils literal"><span class="pre">Queue.get()</span></tt> should now work with timeouts
  308. on Mac OS X.</p>
  309. </li>
  310. <li><p class="first">Added a <tt class="docutils literal"><span class="pre">callback</span></tt> argument to <tt class="docutils literal"><span class="pre">Pool.apply_async()</span></tt>.</p>
  311. </li>
  312. <li><p class="first">Added <tt class="docutils literal"><span class="pre">test/test_httpserverpool.py</span></tt> which runs a pool of http
  313. servers which share a single listening socket.</p>
  314. </li>
  315. <li><p class="first">Previously on Windows the process object was passed to the child
  316. process on the commandline (after pickling and hex encoding it).
  317. This caused errors when the pickled string was too large. Now if
  318. the pickled string is large then it will be passed to the child
  319. over a pipe or socket.</p>
  320. </li>
  321. <li><p class="first">Fixed bug in the iterator returned by <tt class="docutils literal"><span class="pre">Pool.imap()</span></tt>.</p>
  322. </li>
  323. <li><p class="first">Fixed bug in <tt class="docutils literal"><span class="pre">Condition.__repr__()</span></tt>.</p>
  324. </li>
  325. <li><p class="first">Fixed a handle/file descriptor leak when sockets or connections are
  326. unpickled.</p>
  327. </li>
  328. </ul>
  329. </div>
  330. <div class="section">
  331. <h1><a id="changes-in-0-34" name="changes-in-0-34">Changes in 0.34</a></h1>
  332. <ul>
  333. <li><p class="first">Although version 0.33 the C extension would compile on Mac OSX
  334. trying to import it failed with &quot;undefined symbol: _sem_timedwait&quot;.
  335. Unfortunately the <tt class="docutils literal"><span class="pre">ImportError</span></tt> exception was silently swallowed.</p>
  336. <p>This is now fixed by using the <tt class="docutils literal"><span class="pre">NO_SEM_TIMED</span></tt> macro. Unfortunately
  337. this means that some methods like <tt class="docutils literal"><span class="pre">Condition.wait()</span></tt> and
  338. <tt class="docutils literal"><span class="pre">Queue.get()</span></tt> will not work with timeouts on Mac OS X. If you
  339. really need to be able to use timeouts then you can always use the
  340. equivalent objects created with a manager. Thanks to Doug Hellmann
  341. for report and testing.</p>
  342. </li>
  343. <li><p class="first">Added a <tt class="docutils literal"><span class="pre">terminate()</span></tt> method to process objects which is more
  344. forceful than <tt class="docutils literal"><span class="pre">stop()</span></tt>.</p>
  345. </li>
  346. <li><p class="first">Fixed bug in the cleanup function registered with <tt class="docutils literal"><span class="pre">atexit</span></tt> which on
  347. Windows could cause a process which is shutting down to deadlock
  348. waiting for a manager to exit. Thanks to Dominique Wahli for report
  349. and testing.</p>
  350. </li>
  351. <li><p class="first">Added <tt class="docutils literal"><span class="pre">test/test_workers.py</span></tt> which gives an example of how to create
  352. a collection of worker processes which execute tasks from one queue
  353. and return results on another.</p>
  354. </li>
  355. <li><p class="first">Added <tt class="docutils literal"><span class="pre">processing.Pool()</span></tt> which returns a process pool object. This
  356. allows one to execute functions asynchronously. It also has a
  357. parallel implementation of the <tt class="docutils literal"><span class="pre">map()</span></tt> builtin. This is still
  358. <em>experimental</em> and undocumented --- see <tt class="docutils literal"><span class="pre">test/test_pool.py</span></tt> for
  359. example usage.</p>
  360. </li>
  361. </ul>
  362. </div>
  363. <div class="section">
  364. <h1><a id="changes-in-0-33" name="changes-in-0-33">Changes in 0.33</a></h1>
  365. <ul>
  366. <li><p class="first">Added a <tt class="docutils literal"><span class="pre">recvbytes_into()</span></tt> method for receiving byte data into
  367. objects with the writable buffer interface. Also renamed the
  368. <tt class="docutils literal"><span class="pre">_recv_string()</span></tt> and <tt class="docutils literal"><span class="pre">_send_string()</span></tt> methods of connection objects
  369. to <tt class="docutils literal"><span class="pre">recvbytes()</span></tt> and <tt class="docutils literal"><span class="pre">sendbytes()</span></tt>.</p>
  370. </li>
  371. <li><p class="first">Some optimizations for the transferring of large blocks of data
  372. using connection objects.</p>
  373. </li>
  374. <li><p class="first">On Unix <tt class="docutils literal"><span class="pre">os.sysconf()</span></tt> is now used by default to determine whether
  375. to compile in support for posix semaphores or posix message queues.</p>
  376. <p>By using the <tt class="docutils literal"><span class="pre">NO_SEM_TIMED</span></tt> and <tt class="docutils literal"><span class="pre">NO_MQ_TIMED</span></tt> macros (see
  377. <tt class="docutils literal"><span class="pre">INSTALL.txt</span></tt>) it should now also be possible to compile in
  378. (partial) semaphore or queue support on Unix systems which lack the
  379. timeout functions <tt class="docutils literal"><span class="pre">sem_timedwait()</span></tt> or <tt class="docutils literal"><span class="pre">mq_timedreceive()</span></tt> and
  380. <tt class="docutils literal"><span class="pre">mq_timesend()</span></tt>.</p>
  381. </li>
  382. <li><p class="first"><tt class="docutils literal"><span class="pre">gettimeofday()</span></tt> is now used instead of <tt class="docutils literal"><span class="pre">clock_gettime()</span></tt> making
  383. compilation of the C extension (hopefully) possible on Mac OSX. No
  384. modificaton of <tt class="docutils literal"><span class="pre">setup.py</span></tt> should be necessary. Thanks to Michele
  385. Bertoldi for report and proposed patch.</p>
  386. </li>
  387. <li><p class="first"><tt class="docutils literal"><span class="pre">cpuCount()</span></tt> function added which returns the number of CPUs
  388. in the system.</p>
  389. </li>
  390. <li><p class="first">Bugfixes to <tt class="docutils literal"><span class="pre">PosixQueue</span></tt> class.</p>
  391. </li>
  392. </ul>
  393. </div>
  394. <div class="section">
  395. <h1><a id="changes-in-0-32" name="changes-in-0-32">Changes in 0.32</a></h1>
  396. <ul class="simple">
  397. <li>Refactored and simplified <tt class="docutils literal"><span class="pre">_nonforking</span></tt> module -- info about
  398. <tt class="docutils literal"><span class="pre">sys.modules</span></tt> of parent process is no longer passed on to child
  399. process. Also <tt class="docutils literal"><span class="pre">pkgutil</span></tt> is no longer used.</li>
  400. <li>Allocated space from an mmap used by <tt class="docutils literal"><span class="pre">LocalManager</span></tt> will now be
  401. recycled.</li>
  402. <li>Better tests for <tt class="docutils literal"><span class="pre">LocalManager</span></tt>.</li>
  403. <li>Fixed bug in <tt class="docutils literal"><span class="pre">managers.py</span></tt> concerning refcounting of shared objects.
  404. Bug affects the case where the callable used to create a shared
  405. object does not return a unique object each time it is called.
  406. Thanks to Alexey Akimov for the report.</li>
  407. <li>Added a <tt class="docutils literal"><span class="pre">freezeSupport()</span></tt> function. Calling this at the appropriate
  408. point in the main module is necessary when freezing a multiprocess
  409. program to produce a Windows executable. (Has been tested with
  410. <tt class="docutils literal"><span class="pre">py2exe</span></tt>, <tt class="docutils literal"><span class="pre">PyInstaller</span></tt> and <tt class="docutils literal"><span class="pre">cx_Freeze</span></tt>.)</li>
  411. </ul>
  412. </div>
  413. <div class="section">
  414. <h1><a id="changes-in-0-31" name="changes-in-0-31">Changes in 0.31</a></h1>
  415. <ul class="simple">
  416. <li>Fixed one line bug in <tt class="docutils literal"><span class="pre">localmanager.py</span></tt> which caused shared memory maps
  417. not to be resized properly.</li>
  418. <li>Added tests for shared values/structs/arrays to <tt class="docutils literal"><span class="pre">test/test_processing</span></tt>.</li>
  419. </ul>
  420. </div>
  421. <div class="section">
  422. <h1><a id="changes-in-0-30" name="changes-in-0-30">Changes in 0.30</a></h1>
  423. <ul>
  424. <li><p class="first">Process objects now support the complete API of thread objects.</p>
  425. <p>In particular <tt class="docutils literal"><span class="pre">isAlive()</span></tt>, <tt class="docutils literal"><span class="pre">isDaemon()</span></tt>, <tt class="docutils literal"><span class="pre">setDaemon()</span></tt> have been
  426. added and <tt class="docutils literal"><span class="pre">join()</span></tt> now supports the <tt class="docutils literal"><span class="pre">timeout</span></tt> paramater.</p>
  427. <p>There are also new methods <tt class="docutils literal"><span class="pre">stop()</span></tt>, <tt class="docutils literal"><span class="pre">getPid()</span></tt> and <tt class="docutils literal"><span class="pre">getExitCode()</span></tt>.</p>
  428. </li>
  429. <li><p class="first">Implemented synchronization primitives based on the Windows mutexes
  430. and semaphores and posix named semaphores.</p>
  431. </li>
  432. <li><p class="first">Added support for sharing simple objects between processes by using
  433. a shared memory map and the <tt class="docutils literal"><span class="pre">struct</span></tt> or <tt class="docutils literal"><span class="pre">array</span></tt> modules.</p>
  434. </li>
  435. <li><p class="first">An <tt class="docutils literal"><span class="pre">activeChildren()</span></tt> function has been added to <tt class="docutils literal"><span class="pre">processing</span></tt> which
  436. returns a list of the child processes which are still alive.</p>
  437. </li>
  438. <li><p class="first">A <tt class="docutils literal"><span class="pre">Pipe()</span></tt> function has been added which returns a pair of
  439. connection objects representing the ends of a duplex connection over
  440. which picklable objects can be sent.</p>
  441. </li>
  442. <li><p class="first">socket objects etc are now picklable and can be transferred between
  443. processes. (Requires compilation of the <tt class="docutils literal"><span class="pre">_processing</span></tt> extension.)</p>
  444. </li>
  445. <li><p class="first">Subclasses of <tt class="docutils literal"><span class="pre">managers.BaseManager</span></tt> no longer automatically spawn a
  446. child process when an instance is created: the <tt class="docutils literal"><span class="pre">start()</span></tt> method must be
  447. called explicitly.</p>
  448. </li>
  449. <li><p class="first">On Windows child processes are now spawned using <tt class="docutils literal"><span class="pre">subprocess</span></tt>.</p>
  450. </li>
  451. <li><p class="first">On Windows the Python 2.5 version of <tt class="docutils literal"><span class="pre">pkgutil</span></tt> is now used for
  452. loading modules by the <tt class="docutils literal"><span class="pre">_nonforking</span></tt> module. On Python 2.4 this
  453. version of <tt class="docutils literal"><span class="pre">pkgutil</span></tt> (which uses the standard Python licence) is
  454. included in <tt class="docutils literal"><span class="pre">processing.compat</span></tt>.</p>
  455. </li>
  456. <li><p class="first">The arguments to the functions in <tt class="docutils literal"><span class="pre">processing.connection</span></tt> have
  457. changed slightly.</p>
  458. </li>
  459. <li><p class="first">Connection objects now have a <tt class="docutils literal"><span class="pre">poll()</span></tt> method which tests whether
  460. there is any data available for reading.</p>
  461. </li>
  462. <li><p class="first">The <tt class="docutils literal"><span class="pre">test/py2exedemo</span></tt> folder shows how to get <tt class="docutils literal"><span class="pre">py2exe</span></tt> to create a
  463. Windows executable from a program using the <tt class="docutils literal"><span class="pre">processing</span></tt> package.</p>
  464. </li>
  465. <li><p class="first">More tests.</p>
  466. </li>
  467. <li><p class="first">Bugfixes.</p>
  468. </li>
  469. <li><p class="first">Rearrangement of various stuff.</p>
  470. </li>
  471. </ul>
  472. </div>
  473. <div class="section">
  474. <h1><a id="changes-in-0-21" name="changes-in-0-21">Changes in 0.21</a></h1>
  475. <ul class="simple">
  476. <li>By default a proxy is now only able to access those methods of its
  477. referent which have been explicitly exposed.</li>
  478. <li>The <tt class="docutils literal"><span class="pre">connection</span></tt> sub-package now supports digest authentication.</li>
  479. <li>Process objects are now given randomly generated 'inheritable'
  480. authentication keys.</li>
  481. <li>A manager process will now only accept connections from processes
  482. using the same authentication key.</li>
  483. <li>Previously <tt class="docutils literal"><span class="pre">get_module()</span></tt> from <tt class="docutils literal"><span class="pre">_nonforking.py</span></tt> was seriously messed
  484. up (though it generally worked). It is a lot saner now.</li>
  485. <li>Python 2.4 or higher is now required.</li>
  486. </ul>
  487. </div>
  488. <div class="section">
  489. <h1><a id="changes-in-0-20" name="changes-in-0-20">Changes in 0.20</a></h1>
  490. <ul class="simple">
  491. <li>The <tt class="docutils literal"><span class="pre">doc</span></tt> folder contains HTML documentation.</li>
  492. <li><tt class="docutils literal"><span class="pre">test</span></tt> is now a subpackage. Running <tt class="docutils literal"><span class="pre">processing.test.main()</span></tt>
  493. will run test scripts using both processes and threads.</li>
  494. <li><tt class="docutils literal"><span class="pre">nonforking.py</span></tt> has been renamed <tt class="docutils literal"><span class="pre">_nonforking.py</span></tt>.
  495. <tt class="docutils literal"><span class="pre">manager.py</span></tt> has been renamed <tt class="docutils literal"><span class="pre">manager.py</span></tt>.
  496. <tt class="docutils literal"><span class="pre">connection.py</span></tt> has become a sub-package <tt class="docutils literal"><span class="pre">connection</span></tt></li>
  497. <li><tt class="docutils literal"><span class="pre">Listener</span></tt> and <tt class="docutils literal"><span class="pre">Client</span></tt> have been removed from
  498. <tt class="docutils literal"><span class="pre">processing</span></tt>, but still exist in <tt class="docutils literal"><span class="pre">processing.connection</span></tt>.</li>
  499. <li>The package is now <em>probably</em> compatible with versions of Python
  500. earlier than 2.4.</li>
  501. <li><tt class="docutils literal"><span class="pre">set</span></tt> is no longer a type supported by the default manager type.</li>
  502. <li>Many more changes.</li>
  503. </ul>
  504. </div>
  505. <div class="section">
  506. <h1><a id="changes-in-0-12" name="changes-in-0-12">Changes in 0.12</a></h1>
  507. <ul class="simple">
  508. <li>Fixed bug where the arguments to <tt class="docutils literal"><span class="pre">processing.Manager()</span></tt> were passed on
  509. to <tt class="docutils literal"><span class="pre">processing.manager.DefaultManager()</span></tt> in the wrong order.</li>
  510. <li><tt class="docutils literal"><span class="pre">processing.dummy</span></tt> is now a subpackage of <tt class="docutils literal"><span class="pre">processing</span></tt>
  511. instead of a module.</li>
  512. <li>Rearranged package so that the <tt class="docutils literal"><span class="pre">test</span></tt> folder, <tt class="docutils literal"><span class="pre">README.txt</span></tt> and
  513. <tt class="docutils literal"><span class="pre">CHANGES.txt</span></tt> are copied when the package is installed.</li>
  514. </ul>
  515. </div>
  516. <div class="section">
  517. <h1><a id="changes-in-0-11" name="changes-in-0-11">Changes in 0.11</a></h1>
  518. <ul class="simple">
  519. <li>Fixed bug on windows when the full path of <tt class="docutils literal"><span class="pre">nonforking.py</span></tt> contains a
  520. space.</li>
  521. <li>On unix there is no longer a need to make the arguments to the
  522. constructor of <tt class="docutils literal"><span class="pre">Process</span></tt> be picklable or for and instance of a
  523. subclass of <tt class="docutils literal"><span class="pre">Process</span></tt> to be picklable when you call the start method.</li>
  524. <li>On unix proxies which a child process inherits from its parent can
  525. be used by the child without any problem, so there is no longer a
  526. need to pass them as arguments to <tt class="docutils literal"><span class="pre">Process</span></tt>. (This will never be
  527. possible on windows.)</li>
  528. </ul>
  529. </div>
  530. </div>
  531. </body>
  532. </html>