queue-objects.txt 5.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124
  1. .. include:: header.txt
  2. ===============
  3. Queue objects
  4. ===============
  5. The queue type provided by `processing` is a multi-producer,
  6. multi-consumer FIFO queue modelled on the `Queue.Queue` class in the
  7. standard library.
  8. `Queue(maxsize=0)`
  9. Returns a process shared queue implemented using a pipe and a few
  10. locks/semaphores. When a process first puts an item on the queue
  11. a feeder thread is started which transfers objects from a
  12. buffer into the pipe.
  13. `Queue.Queue` implements all the methods of `Queue.Queue` except for
  14. `qsize()`, `task_done()` and `join()`.
  15. `empty()`
  16. Return `True` if the queue is empty, `False`
  17. otherwise. Because of multithreading/multiprocessing
  18. semantics, this is not reliable.
  19. `full()`
  20. Return `True` if the queue is full, `False` otherwise. Because
  21. of multithreading/multiprocessing semantics, this is not
  22. reliable.
  23. `put(item, block=True, timeout=None)`
  24. Put item into the queue. If optional args `block` is true
  25. and `timeout` is `None` (the default), block if necessary
  26. until a free slot is available. If `timeout` is a positive
  27. number, it blocks at most `timeout` seconds and raises the
  28. `Full` exception if no free slot was available within that
  29. time. Otherwise (`block` is false), put an item on the queue
  30. if a free slot is immediately available, else raise the
  31. `Full` exception (`timeout` is ignored in that case).
  32. `put_nowait(item)`, `putNoWait(item)`
  33. Equivalent to `put(item, False)`.
  34. `get(block=True, timeout=None)`
  35. Remove and return an item from the queue. If optional args
  36. `block` is true and `timeout` is `None` (the default),
  37. block if necessary until an item is available. If
  38. `timeout` is a positive number, it blocks at most
  39. `timeout` seconds and raises the `Empty` exception if no
  40. item was available within that time. Otherwise (block is
  41. false), return an item if one is immediately available,
  42. else raise the `Empty` exception (`timeout` is ignored in
  43. that case).
  44. `get_nowait()`, `getNoWait()`
  45. Equivalent to `get(False)`.
  46. `processing.Queue` has a few additional methods not found in
  47. `Queue.Queue` which are usually unnecessary:
  48. `putMany(iterable)`
  49. If the queue has infinite size then this adds all
  50. items in the iterable to the queue's buffer. So
  51. `q.putMany(X)` is a faster alternative to `for x in X:
  52. q.put(x)`. Raises an error if the queue has finite
  53. size.
  54. `close()`
  55. Indicates that no more data will be put on this queue by
  56. the current process. The background thread will quit once
  57. it has flushed all buffered data to the pipe. This is
  58. called automatically when the queue is garbage collected.
  59. `joinThread()`
  60. This joins the background thread and can only be used
  61. after `close()` has been called. This blocks until
  62. the background thread exits, ensuring that all data in
  63. the buffer has been flushed to the pipe.
  64. By default if a process is not the creator of the
  65. queue then on exit it will attempt to join the queue's
  66. background thread. The process can call
  67. `cancelJoin()` to prevent this behaviour.
  68. `cancelJoin()`
  69. Prevents the background thread from being joined
  70. automatically when the process exits. Unnecessary if
  71. the current process created the queue.
  72. .. admonition:: `Empty` and `Full`
  73. `processing` uses the usual `Queue.Empty` and `Queue.Full`
  74. exceptions to signal a timeout. They are not available in the
  75. `processing` namespace so you need to import them from `Queue`.
  76. .. warning::
  77. If a process is killed using the `terminate()` method or
  78. `os.kill()` while it is trying to use a `Queue` then the data in
  79. the queue is likely to become corrupted. This may cause any other
  80. processes to get an exception when it tries to use the queue later
  81. on.
  82. .. warning::
  83. As mentioned above, if a child process has put items on a queue
  84. (and it has not used `cancelJoin()`) then that process will not
  85. terminate until all buffered items have been flushed to the pipe.
  86. This means that if you try joining that process you may get a
  87. deadlock unless you are sure that all items which have been put on
  88. the queue have been consumed. Similarly, if the child process is
  89. non-daemonic then the parent process may hang on exit when it tries
  90. to join all it non-daemonic children.
  91. Note that a queue created using a manager does not have this issue.
  92. See `Programming Guidelines <programming-guidelines.html>`_.
  93. .. _Prev: process-objects.html
  94. .. _Up: processing-ref.html
  95. .. _Next: connection-objects.html