Auxiliary class used by class GroupCommunicator implementing its device (GPU) code paths for data passed as Array<T>. The operations are performed on the configured mfem::Device and can use GPU-aware MPI, if enabled for the device. More...

#include <communication.hpp>

Collaboration diagram for mfem::DeviceGroupCommunicator:

[legend]

Classes
struct	TypedBufferView

Public Types
enum class	Op { Sum , Min , Max }
	Reduction operation applied at the receiving dofs. More...

Public Member Functions
	DeviceGroupCommunicator (const GroupCommunicator &gc_)
	Construct a device communicator based on the GroupCommunicator gc_.

template<typename T >
void	BcastBeginTDofs (Array< T > &x_tdof) const
	Begin a group broadcast of the true-dof data x_tdof.

template<typename T >
void	BcastBeginLDofs (Array< T > &x_ldof) const
	Begin a group broadcast of the local-dof data x_ldof.

template<typename T >
void	BcastEndLDofs (Array< T > &x_ldof) const
	Finalize a group broadcast into the local-dof data x_ldof.

template<typename T >
void	ReduceBeginLDofs (const Array< T > &x_ldof) const
	Begin a group reduction of the local-dof data x_ldof.

template<typename T >
void	ReduceEndTDofs (Array< T > &x_tdof, Op op) const
	Finalize a group reduction into the true-dof data x_tdof, applying the reduction operation op.

template<typename T >
void	ReduceEndLDofs (Array< T > &x_ldof, Op op) const
	Finalize a group reduction into the local-dof data x_ldof, applying the reduction operation op.

template<typename T >
void	CopyTDofsToLDofs (const Array< T > &x_tdof, Array< T > &x_ldof) const
	Kernel: copy ltdofs from x_tdof to ldofs in x_ldof, i.e. x_ldof[ltdof_ldof[i]] = x_tdof[i].

template<typename T >
void	Prolongate (const Array< T > &x_tdof, Array< T > &x_ldof) const
	Prolongate the true-dof data x_tdof to the local-dof data x_ldof.

template<typename T >
void	ProlongateTranspose (const Array< T > &x_ldof, Array< T > &x_tdof, Op op=Op::Sum) const
	Transpose of Prolongate(): reduce the local-dof data x_ldof into the true-dof data x_tdof, applying the operation op.

template<typename T >
void	Restrict (const Array< T > &x_ldof, Array< T > &x_tdof) const
	Kernel: copy owned ldofs from x_ldof to ltdofs in x_tdof, i.e. x_tdof[i] = x_ldof[ltdof_ldof[i]].

template<typename T >
void	RestrictTranspose (const Array< T > &x_tdof, Array< T > &x_ldof) const
	Transpose of Restrict(): copy the true-dof data x_tdof into the owned local dofs of x_ldof and set the remaining (external) local dofs to zero.

Protected Types
using	buffer_max_type = int64_t

Protected Member Functions
template<typename T >
void	Exchange (const Array< T > &send_buf, const Array< int > &send_offsets, Array< T > &recv_buf, const Array< int > &recv_offsets, int tag) const

template<typename T >
void	ExchangeSharedToExternal (const Array< T > &shr_buf, Array< T > &ext_buf) const

template<typename T >
void	ExchangeExternalToShared (const Array< T > &ext_buf, Array< T > &shr_buf) const

template<typename T >
void	ReduceBeginCopy (const Array< T > &x_ldof, Array< T > &ext_buf_t) const

template<typename T >
void	ReduceEndAssembleTDofs (const Array< T > &shr_buf_t, Array< T > &x_tdof, Op op) const

template<typename T >
void	BcastBeginCopyTDofs (const Array< T > &x_tdof, Array< T > &shr_buf_t) const

template<typename T >
void	BcastBeginCopyLDofs (const Array< T > &x_ldof, Array< T > &shr_buf_t) const

template<typename T >
void	BcastEndCopy (const Array< T > &ext_buf_t, Array< T > &x_ldof) const

void	WaitAll () const

Protected Attributes
const GroupCommunicator &	gc

Array< int >	shr_ltdof

Array< int >	shr_ldof

Array< int >	ext_ldof

Array< int >	shr_buf_offsets

Array< int >	ext_buf_offsets

Array< int >	unq_ltdof

Array< int >	unq_ldof

Array< int >	unq_shr_i

Array< int >	unq_shr_j

Array< buffer_max_type >	shr_buf

Array< buffer_max_type >	ext_buf

Array< MPI_Request >	requests

int	num_requests

Friends
class	GroupCommunicator

Detailed Description

Auxiliary class used by class GroupCommunicator implementing its device (GPU) code paths for data passed as Array<T>. The operations are performed on the configured mfem::Device and can use GPU-aware MPI, if enabled for the device.

Definition at line 604 of file communication.hpp.

Member Typedef Documentation

◆ buffer_max_type

using mfem::DeviceGroupCommunicator::buffer_max_type = int64_t

protected

Definition at line 714 of file communication.hpp.

Member Enumeration Documentation

◆ Op

enum class mfem::DeviceGroupCommunicator::Op

strong

Reduction operation applied at the receiving dofs.

Enumerator
Sum
Min
Max

Definition at line 610 of file communication.hpp.

Constructor & Destructor Documentation

◆ DeviceGroupCommunicator()

mfem::DeviceGroupCommunicator::DeviceGroupCommunicator ( const GroupCommunicator & gc_ )

explicit

Construct a device communicator based on the GroupCommunicator gc_.

Definition at line 1890 of file communication.cpp.

Member Function Documentation

◆ BcastBeginCopyLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastBeginCopyLDofs	(	const Array< T > &	x_ldof,
		Array< T > &	shr_buf_t ) const

protected

Definition at line 2176 of file communication.cpp.

◆ BcastBeginCopyTDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastBeginCopyTDofs	(	const Array< T > &	x_tdof,
		Array< T > &	shr_buf_t ) const

protected

Definition at line 2167 of file communication.cpp.

◆ BcastBeginLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastBeginLDofs ( Array< T > & x_ldof ) const

Begin a group broadcast of the local-dof data x_ldof.

Definition at line 1984 of file communication.cpp.

◆ BcastBeginTDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastBeginTDofs ( Array< T > & x_tdof ) const

Begin a group broadcast of the true-dof data x_tdof.

Definition at line 1975 of file communication.cpp.

◆ BcastEndCopy()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastEndCopy	(	const Array< T > &	ext_buf_t,
		Array< T > &	x_ldof ) const

protected

Definition at line 2185 of file communication.cpp.

◆ BcastEndLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::BcastEndLDofs ( Array< T > & x_ldof ) const

Finalize a group broadcast into the local-dof data x_ldof.

Definition at line 1993 of file communication.cpp.

◆ CopyTDofsToLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::CopyTDofsToLDofs	(	const Array< T > &	x_tdof,
		Array< T > &	x_ldof ) const

Kernel: copy ltdofs from x_tdof to ldofs in x_ldof, i.e. x_ldof[ltdof_ldof[i]] = x_tdof[i].

Definition at line 2036 of file communication.cpp.

◆ Exchange()

template<typename T >

void mfem::DeviceGroupCommunicator::Exchange	(	const Array< T > &	send_buf,
		const Array< int > &	send_offsets,
		Array< T > &	recv_buf,
		const Array< int > &	recv_offsets,
		int	tag ) const

protected

Definition at line 2091 of file communication.cpp.

◆ ExchangeExternalToShared()

template<typename T >

void mfem::DeviceGroupCommunicator::ExchangeExternalToShared	(	const Array< T > &	ext_buf,
		Array< T > &	shr_buf ) const

protected

Definition at line 2132 of file communication.cpp.

◆ ExchangeSharedToExternal()

template<typename T >

void mfem::DeviceGroupCommunicator::ExchangeSharedToExternal	(	const Array< T > &	shr_buf,
		Array< T > &	ext_buf ) const

protected

Definition at line 2124 of file communication.cpp.

◆ Prolongate()

template<typename T >

void mfem::DeviceGroupCommunicator::Prolongate	(	const Array< T > &	x_tdof,
		Array< T > &	x_ldof ) const

Prolongate the true-dof data x_tdof to the local-dof data x_ldof.

Definition at line 2044 of file communication.cpp.

◆ ProlongateTranspose()

template<typename T >

void mfem::DeviceGroupCommunicator::ProlongateTranspose	(	const Array< T > &	x_ldof,
		Array< T > &	x_tdof,
		Op	op = Op::Sum ) const

Transpose of Prolongate(): reduce the local-dof data x_ldof into the true-dof data x_tdof, applying the operation op.

Definition at line 2059 of file communication.cpp.

◆ ReduceBeginCopy()

template<typename T >

void mfem::DeviceGroupCommunicator::ReduceBeginCopy	(	const Array< T > &	x_ldof,
		Array< T > &	ext_buf_t ) const

protected

Definition at line 2140 of file communication.cpp.

◆ ReduceBeginLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::ReduceBeginLDofs ( const Array< T > & x_ldof ) const

Begin a group reduction of the local-dof data x_ldof.

Definition at line 2001 of file communication.cpp.

◆ ReduceEndAssembleTDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::ReduceEndAssembleTDofs	(	const Array< T > &	shr_buf_t,
		Array< T > &	x_tdof,
		Op	op ) const

protected

Definition at line 2149 of file communication.cpp.

◆ ReduceEndLDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::ReduceEndLDofs	(	Array< T > &	x_ldof,
		Op	op ) const

Finalize a group reduction into the local-dof data x_ldof, applying the reduction operation op.

Definition at line 2018 of file communication.cpp.

◆ ReduceEndTDofs()

template<typename T >

void mfem::DeviceGroupCommunicator::ReduceEndTDofs	(	Array< T > &	x_tdof,
		Op	op ) const

Finalize a group reduction into the true-dof data x_tdof, applying the reduction operation op.

Definition at line 2010 of file communication.cpp.

◆ Restrict()

template<typename T >

void mfem::DeviceGroupCommunicator::Restrict	(	const Array< T > &	x_ldof,
		Array< T > &	x_tdof ) const

Kernel: copy owned ldofs from x_ldof to ltdofs in x_tdof, i.e. x_tdof[i] = x_ldof[ltdof_ldof[i]].

Definition at line 2075 of file communication.cpp.

◆ RestrictTranspose()

template<typename T >

void mfem::DeviceGroupCommunicator::RestrictTranspose	(	const Array< T > &	x_tdof,
		Array< T > &	x_ldof ) const

Transpose of Restrict(): copy the true-dof data x_tdof into the owned local dofs of x_ldof and set the remaining (external) local dofs to zero.

Definition at line 2083 of file communication.cpp.

◆ WaitAll()

void mfem::DeviceGroupCommunicator::WaitAll ( ) const

protected

Definition at line 2192 of file communication.cpp.

Friends And Related Symbol Documentation

◆ GroupCommunicator

friend class GroupCommunicator

friend

Definition at line 607 of file communication.hpp.

Member Data Documentation

◆ ext_buf

Array<buffer_max_type> mfem::DeviceGroupCommunicator::ext_buf

protected

Definition at line 715 of file communication.hpp.

◆ ext_buf_offsets

Array<int> mfem::DeviceGroupCommunicator::ext_buf_offsets

protected

Definition at line 711 of file communication.hpp.

◆ ext_ldof

Array<int> mfem::DeviceGroupCommunicator::ext_ldof

protected

Definition at line 710 of file communication.hpp.

◆ gc

const GroupCommunicator& mfem::DeviceGroupCommunicator::gc

protected

Definition at line 709 of file communication.hpp.

◆ num_requests

int mfem::DeviceGroupCommunicator::num_requests

mutableprotected

Definition at line 717 of file communication.hpp.

◆ requests

Array<MPI_Request> mfem::DeviceGroupCommunicator::requests

mutableprotected

Definition at line 716 of file communication.hpp.

◆ shr_buf

Array<buffer_max_type> mfem::DeviceGroupCommunicator::shr_buf

mutableprotected

Definition at line 715 of file communication.hpp.

◆ shr_buf_offsets

Array<int> mfem::DeviceGroupCommunicator::shr_buf_offsets

protected

Definition at line 711 of file communication.hpp.

◆ shr_ldof

Array<int> mfem::DeviceGroupCommunicator::shr_ldof

protected

Definition at line 710 of file communication.hpp.

◆ shr_ltdof

Array<int> mfem::DeviceGroupCommunicator::shr_ltdof

protected

Definition at line 710 of file communication.hpp.

◆ unq_ldof

Array<int> mfem::DeviceGroupCommunicator::unq_ldof

protected

Definition at line 712 of file communication.hpp.

◆ unq_ltdof

Array<int> mfem::DeviceGroupCommunicator::unq_ltdof

protected

Definition at line 712 of file communication.hpp.

◆ unq_shr_i

Array<int> mfem::DeviceGroupCommunicator::unq_shr_i

protected

Definition at line 713 of file communication.hpp.

◆ unq_shr_j

Array<int> mfem::DeviceGroupCommunicator::unq_shr_j

protected

Definition at line 713 of file communication.hpp.

The documentation for this class was generated from the following files:

general/communication.hpp
general/communication.cpp

Classes

Public Types

Public Member Functions

Protected Types

Protected Member Functions

Protected Attributes

Friends

Detailed Description

Member Typedef Documentation

◆ buffer_max_type

Member Enumeration Documentation

◆ Op

Constructor & Destructor Documentation

◆ DeviceGroupCommunicator()

Member Function Documentation

◆ BcastBeginCopyLDofs()

◆ BcastBeginCopyTDofs()

◆ BcastBeginLDofs()

◆ BcastBeginTDofs()

◆ BcastEndCopy()

◆ BcastEndLDofs()

◆ CopyTDofsToLDofs()

◆ Exchange()

◆ ExchangeExternalToShared()

◆ ExchangeSharedToExternal()

◆ Prolongate()

◆ ProlongateTranspose()

◆ ReduceBeginCopy()

◆ ReduceBeginLDofs()

◆ ReduceEndAssembleTDofs()

◆ ReduceEndLDofs()

◆ ReduceEndTDofs()

◆ Restrict()

◆ RestrictTranspose()

◆ WaitAll()

Friends And Related Symbol Documentation

◆ GroupCommunicator

Member Data Documentation

◆ ext_buf

◆ ext_buf_offsets

◆ ext_ldof

◆ gc

◆ num_requests

◆ requests

◆ shr_buf

◆ shr_buf_offsets

◆ shr_ldof

◆ shr_ltdof

◆ unq_ldof

◆ unq_ltdof

◆ unq_shr_i

◆ unq_shr_j