Update logging cookbook to show multiple worker processes using the concurrent.future...
authorVinay Sajip <vinay_sajip@yahoo.co.uk>
Mon, 22 Jul 2019 11:14:50 +0000 (12:14 +0100)
committerGitHub <noreply@github.com>
Mon, 22 Jul 2019 11:14:50 +0000 (12:14 +0100)
Doc/howto/logging-cookbook.rst

index 87ac79ef8072d6edd34fcbaf8c3a22a2da329ece..e62308192d1655ee0cb26899df0054f1df568a36 100644 (file)
@@ -948,6 +948,41 @@ This variant shows how you can e.g. apply configuration for particular loggers
 machinery in the main process (even though the logging events are generated in
 the worker processes) to direct the messages to the appropriate destinations.
 
+Using concurrent.futures.ProcessPoolExecutor
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you want to use :class:`concurrent.futures.ProcessPoolExecutor` to start
+your worker processes, you need to create the queue slightly differently.
+Instead of
+
+.. code-block:: python
+
+   queue = multiprocessing.Queue(-1)
+
+you should use
+
+.. code-block:: python
+
+   queue = multiprocessing.Manager().Queue(-1)  # also works with the examples above
+
+and you can then replace the worker creation from this::
+
+    workers = []
+    for i in range(10):
+        worker = multiprocessing.Process(target=worker_process,
+                                         args=(queue, worker_configurer))
+        workers.append(worker)
+        worker.start()
+    for w in workers:
+        w.join()
+
+to this (remembering to first import :mod:`concurrent.futures`)::
+
+    with concurrent.futures.ProcessPoolExecutor(max_workers=10) as executor:
+        for i in range(10):
+            executor.submit(worker_process, queue, worker_configurer)
+
+
 Using file rotation
 -------------------