Skip to content

Commit

Permalink
Add docs for synchronous collective operations
Browse files Browse the repository at this point in the history
  • Loading branch information
@dimitraka
dimitraka committed Feb 2, 2025
1 parent ad17f89 commit 75635f4
Show file tree
Hide file tree
Showing 9 changed files with 1,310 additions and 60 deletions.
118 changes: 107 additions & 11 deletions libs/full/collectives/include/hpx/collectives/all_gather.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,22 @@ namespace hpx { namespace collectives {
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param basename The base name identifying the all_gather operation
/// \param local_result The value to transmit to all
/// \param basename The base name identifying the all_gather operation
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param num_sites The number of participating sites (default: all
/// \param num_sites The number of participating sites (default: all
/// localities).
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// \param generation The generational counter identifying the sequence
/// number of the all_gather operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_gather operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
/// \params root_site The site that is responsible for creating the
/// \param root_site The site that is responsible for creating the
/// all_gather support object. This value is optional
/// and defaults to '0' (zero).
///
Expand All @@ -53,13 +53,13 @@ namespace hpx { namespace collectives {
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param comm A communicator object returned from \a create_communicator
/// \param local_result The value to transmit to all
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
Expand All @@ -82,10 +82,10 @@ namespace hpx { namespace collectives {
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param comm A communicator object returned from \a create_communicator
/// \param local_result The value to transmit to all
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param generation The generational counter identifying the sequence
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
Expand All @@ -105,6 +105,102 @@ namespace hpx { namespace collectives {
all_gather(communicator comm, T&& result,
generation_arg generation,
this_site_arg this_site = this_site_arg());

/// AllGather a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param basename The base name identifying the all_gather operation
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param num_sites The number of participating sites (default: all
/// localities).
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// number of the all_gather operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_gather operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
/// \param root_site The site that is responsible for creating the
/// all_gather support object. This value is optional
/// and defaults to '0' (zero).
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T>
std::vector<std::decay_t<T>> all_gather(hpx::launch::sync_policy,
char const* basename, T&& result,
num_sites_arg num_sites = num_sites_arg(),
this_site_arg this_site = this_site_arg(),
generation_arg generation = generation_arg(),
root_site_arg root_site = root_site_arg());

/// AllGather a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T>
std::vector<std::decay_t<T>> all_gather(hpx::launch::sync_policy,
communicator comm, T&& result,
this_site_arg this_site = this_site_arg(),
generation_arg generation = generation_arg());

/// AllGather a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T>
std::vector<std::decay_t<T>> all_gather(hpx::launch::sync_policy,
communicator comm, T&& result, generation_arg generation,
this_site_arg this_site = this_site_arg());
}} // namespace hpx::collectives

// clang-format on
Expand Down
108 changes: 104 additions & 4 deletions libs/full/collectives/include/hpx/collectives/all_reduce.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ namespace hpx { namespace collectives {
/// the given base name.
///
/// \param basename The base name identifying the all_reduce operation
/// \param local_result The value to transmit to all
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
Expand All @@ -34,7 +34,7 @@ namespace hpx { namespace collectives {
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \params root_site The site that is responsible for creating the
/// \param root_site The site that is responsible for creating the
/// all_reduce support object. This value is optional
/// and defaults to '0' (zero).
///
Expand All @@ -55,7 +55,7 @@ namespace hpx { namespace collectives {
/// the given base name.
///
/// \param comm A communicator object returned from \a create_communicator
/// \param local_result The value to transmit to all
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
Expand Down Expand Up @@ -86,7 +86,7 @@ namespace hpx { namespace collectives {
/// the given base name.
///
/// \param comm A communicator object returned from \a create_communicator
/// \param local_result The value to transmit to all
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
Expand All @@ -110,6 +110,106 @@ namespace hpx { namespace collectives {
all_reduce(communicator comm,
T&& result, F&& op, generation_arg generation,
this_site_arg this_site = this_site_arg());

/// AllReduce a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param basename The base name identifying the all_reduce operation
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
/// \param num_sites The number of participating sites (default: all
/// localities).
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
/// \param root_site The site that is responsible for creating the
/// all_reduce support object. This value is optional
/// and defaults to '0' (zero).
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T, typename F>
decltype(auto) all_reduce(hpx::launch::sync_policy, char const* basename,
T&& result, F&& op, num_sites_arg num_sites = num_sites_arg(),
this_site_arg this_site = this_site_arg(),
generation_arg generation = generation_arg(),
root_site_arg root_site = root_site_arg());

/// AllReduce a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T, typename F>
decltype(auto) all_reduce(hpx::launch::sync_policy, communicator comm,
T&& result, F&& op, this_site_arg this_site = this_site_arg(),
generation_arg generation = generation_arg());

/// AllReduce a set of values from different call sites
///
/// This function receives a set of values from all call sites operating on
/// the given base name.
///
/// \param policy The execution policy specifying synchronous execution.
/// \param comm A communicator object returned from \a create_communicator
/// \param result The value to transmit to all
/// participating sites from this call site.
/// \param op Reduction operation to apply to all values supplied
/// from all participating sites
/// \param generation The generational counter identifying the sequence
/// number of the all_reduce operation performed on the
/// given base name. This is optional and needs to be
/// supplied only if the all_reduce operation on the
/// given base name has to be performed more than once.
/// The generation number (if given) must be a positive
/// number greater than zero.
/// \param this_site The sequence number of this invocation (usually
/// the locality id). This value is optional and
/// defaults to whatever hpx::get_locality_id() returns.
///
/// \returns This function returns a vector with all values send by all
/// participating sites. This function executes synchronously and
/// directly returns the result.
///
template <typename T, typename F>
decltype(auto) all_reduce(hpx::launch::sync_policy, communicator comm,
T&& result, F&& op, generation_arg generation,
this_site_arg this_site = this_site_arg());
}} // namespace hpx::collectives

// clang-format on
Expand Down
Loading

0 comments on commit 75635f4

Please sign in to comment.