Skip to content

Commit

Permalink
doc: document the pool parameter in decorators
Browse files Browse the repository at this point in the history
Signed-off-by: Matteo Cafasso <noxdafox@gmail.com>
  • Loading branch information
noxdafox committed Sep 30, 2024
1 parent 56eeb04 commit 6445f69
Showing 1 changed file with 34 additions and 4 deletions.
38 changes: 34 additions & 4 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py
`Concurrent Module`
-------------------

.. decorator:: concurrent.process(timeout=None, name=None, daemon=True, context=None)
.. decorator:: concurrent.process(timeout=None, name=None, daemon=True, context=None, pool=None)

Runs the decorated function in a concurrent process, taking care of the results and error management.

Expand All @@ -33,7 +33,9 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py

The *context* parameter can be used to specify the multiprocessing.context_ object used for starting the process.

.. decorator:: concurrent.thread(name=None, daemon=True)
The *pool* parameter accepts a pebble.ProcessPool_ object. If provided, the pool will be used to run the function instead of a dedicated process. The *name*, *daemon* and *context* parameters will be ignored.

.. decorator:: concurrent.thread(name=None, daemon=True, pool=None)

Runs the decorated function in a concurrent thread, taking care of the results and error management.

Expand All @@ -43,11 +45,12 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py

The *daemon* parameter switches between daemon and non-daemon threads.

The *pool* parameter accepts a pebble.ThreadPool_ object. If provided, the pool will be used to run the function instead of a dedicated thread. The *name* and *daemon* parameters will be ignored.

`Asynchronous Module`
---------------------

.. decorator:: asynchronous.process(timeout=None, name=None, daemon=True, context=None)
.. decorator:: asynchronous.process(timeout=None, name=None, daemon=True, context=None, pool=None)

Runs the decorated function in a concurrent process, taking care of the results and error management.

Expand All @@ -61,7 +64,9 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py

The *context* parameter can be used to specify the multiprocessing.context_ object used for starting the process.

.. decorator:: asynchronous.thread(name=None, daemon=True)
The *pool* parameter accepts a pebble.ProcessPool_ object. If provided, the pool will be used to run the function instead of a dedicated process. The *name*, *daemon* and *context* parameters will be ignored.

.. decorator:: asynchronous.thread(name=None, daemon=True, pool=None)

Runs the decorated function in a concurrent thread, taking care of the results and error management.

Expand All @@ -71,10 +76,13 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py

The *daemon* parameter switches between daemon and non-daemon threads.

The *pool* parameter accepts a pebble.ThreadPool_ object. If provided, the pool will be used to run the function instead of a dedicated thread. The *name* and *daemon* parameters will be ignored.


`Pebble Module`
---------------

.. _pebble.ProcessPool:
.. class:: pebble.ProcessPool(max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=None, context=None)

A Pool allows to schedule jobs into a Pool of Processes which will perform them concurrently.
Expand Down Expand Up @@ -130,6 +138,7 @@ Pebble aims to help managing threads and processes in an easier way. It wraps Py

The *join* function must be called only in the main loop. Calling it in a pebble.ProcessFuture_ callback will result in a deadlock.

.. _pebble.ThreadPool:
.. class:: pebble.ThreadPool(max_workers=multiprocessing.cpu_count(), max_tasks=0, initializer=None, initargs=None)

A ThreadPool allows to schedule jobs into a Pool of Threads which will perform them concurrently.
Expand Down Expand Up @@ -474,6 +483,27 @@ If a `timeout` is provided, it will be applied to the whole chunk and not to the

assert list(future.result()) == elements

The `concurrent` and `asynchronous` decorators accept a *pool* parameter. This is useful to control how many instances of decorated functions can be run at the same time.

::

from concurrent.futures import wait
from pebble import concurrent, ProcessPool

pool = ProcessPool(max_workers=4)

@concurrent.process(pool=pool)
def function(arg, kwarg=0):
return arg + kwarg

futures = []

# Maximum 4 executions of `function` will be executed in parallel
for _ in range(100):
futures.append(function(1, kwarg=1))

wait(futures)


Pools and AsyncIO
+++++++++++++++++
Expand Down

0 comments on commit 6445f69

Please sign in to comment.