Communicator performing operations within groups defined by a GroupTopology with arbitrary-size data associated with each group. More...

#include <communication.hpp>

Collaboration diagram for mfem::GroupCommunicator:

[legend]

Classes
struct	OpData
	Data structure on which we define reduce operations. The data is associated with (and the operation is performed on) one group at a time. More...

Public Types
enum	Mode { byGroup , byNeighbor }
	Communication mode. More...

Public Member Functions
	GroupCommunicator (const GroupTopology &gt, Mode m=byNeighbor)
	Construct a GroupCommunicator object.

void	Create (const Array< int > &ldof_group)
	Initialize the communicator from a local-dof to group map. Finalize() is called internally.

Table &	GroupLDofTable ()
	Fill-in the returned Table reference to initialize the GroupCommunicator then call Finalize().

const Table &	GroupLDofTable () const
	Read-only access to group-ldof Table.

void	Finalize ()
	Allocate internal buffers after the GroupLDofTable is defined.

void	SetLTDofTable (const Array< int > &ldof_ltdof)
	Initialize the internal group_ltdof Table.

const GroupTopology &	GetGroupTopology ()
	Get a reference to the associated GroupTopology object.

const GroupTopology &	GetGroupTopology () const
	Get a const reference to the associated GroupTopology object.

void	GetNeighborLTDofTable (Table &nbr_ltdof) const
	Dofs to be sent to communication neighbors.

void	GetNeighborLDofTable (Table &nbr_ldof) const
	Dofs to be received from communication neighbors.

template<class T >
T *	CopyGroupToBuffer (const T ldata, T buf, int group, int layout) const
	Copy the entries corresponding to the group group from the local array ldata to the buffer buf.

template<class T >
const T *	CopyGroupFromBuffer (const T buf, T ldata, int group, int layout) const
	Copy the entries corresponding to the group group from the buffer buf to the local array ldata.

template<class T >
const T *	ReduceGroupFromBuffer (const T buf, T ldata, int group, int layout, void(*Op)(OpData< T >)) const
	Perform the reduction operation Op to the entries of group group using the values from the buffer buf and the values from the local array ldata, saving the result in the latter.

template<class T >
void	BcastBegin (T *ldata, int layout) const
	Begin a broadcast within each group where the master is the root.

template<class T >
void	BcastEnd (T *ldata, int layout) const
	Finalize a broadcast started with BcastBegin().

template<class T >
void	Bcast (T *ldata, int layout) const
	Broadcast within each group where the master is the root.

template<class T >
void	Bcast (T *ldata) const
	Broadcast within each group where the master is the root.

template<class T >
void	Bcast (Array< T > &ldata) const
	Broadcast within each group where the master is the root.

template<class T >
void	ReduceBegin (const T *ldata) const
	Begin reduction operation within each group where the master is the root.

template<class T >
void	ReduceEnd (T ldata, int layout, void(Op)(OpData< T >)) const
	Finalize reduction operation started with ReduceBegin().

template<class T >
void	Reduce (T ldata, void(Op)(OpData< T >)) const
	Reduce within each group where the master is the root.

template<class T >
void	Reduce (Array< T > &ldata, void(*Op)(OpData< T >)) const
	Reduce within each group where the master is the root.

void	PrintInfo (std::ostream &out=mfem::out) const
	Print information about the GroupCommunicator from all MPI ranks.

	~GroupCommunicator ()
	Destroy a GroupCommunicator object, deallocating internal data structures and buffers.

Static Public Member Functions
template<class T >
static void	Sum (OpData< T >)
	Reduce operation Sum, instantiated for int and double.

template<class T >
static void	Min (OpData< T >)
	Reduce operation Min, instantiated for int and double.

template<class T >
static void	Max (OpData< T >)
	Reduce operation Max, instantiated for int and double.

template<class T >
static void	BitOR (OpData< T >)
	Reduce operation bitwise OR, instantiated for int only.

Protected Attributes
const GroupTopology &	gtopo

Mode	mode

Table	group_ldof

Table	group_ltdof

int	group_buf_size

Array< char >	group_buf

MPI_Request *	requests

int	comm_lock

int	num_requests

int *	request_marker

int *	buf_offsets

Table	nbr_send_groups

Table	nbr_recv_groups

Detailed Description

Communicator performing operations within groups defined by a GroupTopology with arbitrary-size data associated with each group.

Definition at line 223 of file communication.hpp.

Member Enumeration Documentation

◆ Mode

enum mfem::GroupCommunicator::Mode

Communication mode.

Enumerator
byGroup	Communications are performed one group at a time.
byNeighbor	Communications are performed one neighbor at a time, aggregating over groups.

Definition at line 227 of file communication.hpp.

Constructor & Destructor Documentation

◆ GroupCommunicator()

mfem::GroupCommunicator::GroupCommunicator	(	const GroupTopology &	gt,
		Mode	m = byNeighbor )

Construct a GroupCommunicator object.

The object must be initialized before it can be used to perform any operations. To initialize the object, either

call Create() or
initialize the Table reference returned by GroupLDofTable() and then call Finalize().

Definition at line 356 of file communication.cpp.

◆ ~GroupCommunicator()

mfem::GroupCommunicator::~GroupCommunicator ( )

Destroy a GroupCommunicator object, deallocating internal data structures and buffers.

Definition at line 1293 of file communication.cpp.

Member Function Documentation

◆ Bcast() [1/3]

template<class T >

void mfem::GroupCommunicator::Bcast ( Array< T > & ldata ) const

inline

Broadcast within each group where the master is the root.

Definition at line 360 of file communication.hpp.

◆ Bcast() [2/3]

template<class T >

void mfem::GroupCommunicator::Bcast ( T * ldata ) const

inline

Broadcast within each group where the master is the root.

Definition at line 358 of file communication.hpp.

◆ Bcast() [3/3]

template<class T >

void mfem::GroupCommunicator::Bcast	(	T *	ldata,
		int	layout ) const

inline

Broadcast within each group where the master is the root.

The data layout can be either 0 or 1.

For a description of layout, see CopyGroupToBuffer().

Definition at line 351 of file communication.hpp.

◆ BcastBegin()

template<class T >

template void mfem::GroupCommunicator::BcastBegin< float >	(	T *	ldata,
		int	layout ) const

Begin a broadcast within each group where the master is the root.

For a description of layout, see CopyGroupToBuffer().

Definition at line 712 of file communication.cpp.

◆ BcastEnd()

template<class T >

template void mfem::GroupCommunicator::BcastEnd< float >	(	T *	ldata,
		int	layout ) const

Finalize a broadcast started with BcastBegin().

The output data layout can be:

0 - ldata is an array on all ldofs; the input layout should be either 0 or 2
1 - ldata is the same array as given to BcastBegin(); the input layout should be 1.

For more details about layout, see CopyGroupToBuffer().

Definition at line 849 of file communication.cpp.

◆ BitOR()

template<class T >

template void mfem::GroupCommunicator::BitOR< int > ( OpData< T > )

static

Reduce operation bitwise OR, instantiated for int only.

Definition at line 1160 of file communication.cpp.

◆ CopyGroupFromBuffer()

template<class T >

const T * mfem::GroupCommunicator::CopyGroupFromBuffer	(	const T *	buf,
		T *	ldata,
		int	group,
		int	layout ) const

Copy the entries corresponding to the group group from the buffer buf to the local array ldata.

For a description of layout, see CopyGroupToBuffer().

Returns: The pointer buf plus the number of elements in the group.

Definition at line 639 of file communication.cpp.

◆ CopyGroupToBuffer()

template<class T >

T * mfem::GroupCommunicator::CopyGroupToBuffer	(	const T *	ldata,
		T *	buf,
		int	group,
		int	layout ) const

Copy the entries corresponding to the group group from the local array ldata to the buffer buf.

The layout of the local array can be:

0 - ldata is an array on all ldofs: copied indices: { J[j] : I[group] <= j < I[group+1] } where I,J=group_ldof.{I,J}
1 - ldata is an array on the shared ldofs: copied indices: { j : I[group] <= j < I[group+1] } where I,J=group_ldof.{I,J}
2 - ldata is an array on the true ldofs, ltdofs: copied indices: { J[j] : I[group] <= j < I[group+1] } where I,J=group_ltdof.{I,J}.
Returns
The pointer buf plus the number of elements in the group.

Definition at line 604 of file communication.cpp.

◆ Create()

void mfem::GroupCommunicator::Create ( const Array< int > & ldof_group )

Initialize the communicator from a local-dof to group map. Finalize() is called internally.

Definition at line 368 of file communication.cpp.

◆ Finalize()

void mfem::GroupCommunicator::Finalize ( )

Allocate internal buffers after the GroupLDofTable is defined.

Definition at line 394 of file communication.cpp.

◆ GetGroupTopology() [1/2]

const GroupTopology & mfem::GroupCommunicator::GetGroupTopology ( )

inline

Get a reference to the associated GroupTopology object.

Definition at line 280 of file communication.hpp.

◆ GetGroupTopology() [2/2]

const GroupTopology & mfem::GroupCommunicator::GetGroupTopology ( ) const

inline

Get a const reference to the associated GroupTopology object.

Definition at line 283 of file communication.hpp.

◆ GetNeighborLDofTable()

void mfem::GroupCommunicator::GetNeighborLDofTable ( Table & nbr_ldof ) const

Dofs to be received from communication neighbors.

Definition at line 567 of file communication.cpp.

◆ GetNeighborLTDofTable()

void mfem::GroupCommunicator::GetNeighborLTDofTable ( Table & nbr_ltdof ) const

Dofs to be sent to communication neighbors.

Definition at line 531 of file communication.cpp.

◆ GroupLDofTable() [1/2]

Table & mfem::GroupCommunicator::GroupLDofTable ( )

inline

Fill-in the returned Table reference to initialize the GroupCommunicator then call Finalize().

Definition at line 266 of file communication.hpp.

◆ GroupLDofTable() [2/2]

const Table & mfem::GroupCommunicator::GroupLDofTable ( ) const

inline

Read-only access to group-ldof Table.

Definition at line 269 of file communication.hpp.

◆ Max()

template<class T >

template void mfem::GroupCommunicator::Max< float > ( OpData< T > )

static

Reduce operation Max, instantiated for int and double.

Definition at line 1142 of file communication.cpp.

◆ Min()

template<class T >

template void mfem::GroupCommunicator::Min< float > ( OpData< T > )

static

Reduce operation Min, instantiated for int and double.

Definition at line 1124 of file communication.cpp.

◆ PrintInfo()

void mfem::GroupCommunicator::PrintInfo ( std::ostream & out = mfem::out ) const

Print information about the GroupCommunicator from all MPI ranks.

Definition at line 1173 of file communication.cpp.

◆ Reduce() [1/2]

template<class T >

void mfem::GroupCommunicator::Reduce	(	Array< T > &	ldata,
		void(*	Op )(OpData< T >) ) const

inline

Reduce within each group where the master is the root.

Definition at line 399 of file communication.hpp.

◆ Reduce() [2/2]

template<class T >

void mfem::GroupCommunicator::Reduce	(	T *	ldata,
		void(*	Op )(OpData< T >) ) const

inline

Reduce within each group where the master is the root.

The reduce operation is given by the second argument (see below for list of the supported operations.)

Definition at line 392 of file communication.hpp.

◆ ReduceBegin()

template<class T >

template void mfem::GroupCommunicator::ReduceBegin< float > ( const T * ldata ) const

Begin reduction operation within each group where the master is the root.

The input data layout is an array on all ldofs, i.e. layout 0, see CopyGroupToBuffer().

The reduce operation will be specified when calling ReduceEnd(). This method is instantiated for int and double.

Definition at line 911 of file communication.cpp.

◆ ReduceEnd()

template<class T >

template void mfem::GroupCommunicator::ReduceEnd< float >	(	T *	ldata,
		int	layout,
		void(*	Op )(OpData< T >) ) const

Finalize reduction operation started with ReduceBegin().

The output data layout can be either 0 or 2, see CopyGroupToBuffer().

The reduce operation is given by the third argument (see below for list of the supported operations.) This method is instantiated for int and double.

Note: If the output data layout is 2, then the data from the ldata array passed to this call is used in the reduction operation, instead of the data from the ldata array passed to ReduceBegin(). Therefore, the data for master-groups has to be identical in both arrays.

Definition at line 1030 of file communication.cpp.

◆ ReduceGroupFromBuffer()

template<class T >

const T * mfem::GroupCommunicator::ReduceGroupFromBuffer	(	const T *	buf,
		T *	ldata,
		int	group,
		int	layout,
		void(*	Op )(OpData< T >) ) const

Perform the reduction operation Op to the entries of group group using the values from the buffer buf and the values from the local array ldata, saving the result in the latter.

For a description of layout, see CopyGroupToBuffer().

Returns: The pointer buf plus the number of elements in the group.

Definition at line 673 of file communication.cpp.

◆ SetLTDofTable()

void mfem::GroupCommunicator::SetLTDofTable ( const Array< int > & ldof_ltdof )

Initialize the internal group_ltdof Table.

This method must be called before performing operations that use local data layout 2, see CopyGroupToBuffer() for layout descriptions.

Definition at line 503 of file communication.cpp.

◆ Sum()

template<class T >

template void mfem::GroupCommunicator::Sum< float > ( OpData< T > )

static

Reduce operation Sum, instantiated for int and double.

Definition at line 1100 of file communication.cpp.

Member Data Documentation

◆ buf_offsets

int* mfem::GroupCommunicator::buf_offsets

protected

Definition at line 247 of file communication.hpp.

◆ comm_lock

int mfem::GroupCommunicator::comm_lock

mutableprotected

Definition at line 244 of file communication.hpp.

◆ group_buf

Array<char> mfem::GroupCommunicator::group_buf

mutableprotected

Definition at line 240 of file communication.hpp.

◆ group_buf_size

int mfem::GroupCommunicator::group_buf_size

protected

Definition at line 239 of file communication.hpp.

◆ group_ldof

Table mfem::GroupCommunicator::group_ldof

protected

Definition at line 237 of file communication.hpp.

◆ group_ltdof

Table mfem::GroupCommunicator::group_ltdof

protected

Definition at line 238 of file communication.hpp.

◆ gtopo

const GroupTopology& mfem::GroupCommunicator::gtopo

protected

Definition at line 235 of file communication.hpp.

◆ mode

Mode mfem::GroupCommunicator::mode

protected

Definition at line 236 of file communication.hpp.

◆ nbr_recv_groups

Table mfem::GroupCommunicator::nbr_recv_groups

protected

Definition at line 248 of file communication.hpp.

◆ nbr_send_groups

Table mfem::GroupCommunicator::nbr_send_groups

protected

Definition at line 248 of file communication.hpp.

◆ num_requests

int mfem::GroupCommunicator::num_requests

mutableprotected

Definition at line 245 of file communication.hpp.

◆ request_marker

int* mfem::GroupCommunicator::request_marker

protected

Definition at line 246 of file communication.hpp.

◆ requests

MPI_Request* mfem::GroupCommunicator::requests

protected

Definition at line 241 of file communication.hpp.

The documentation for this class was generated from the following files:

general/communication.hpp
general/communication.cpp

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Attributes

Detailed Description

Member Enumeration Documentation

◆ Mode

Constructor & Destructor Documentation

◆ GroupCommunicator()

◆ ~GroupCommunicator()

Member Function Documentation

◆ Bcast() [1/3]

◆ Bcast() [2/3]

◆ Bcast() [3/3]

◆ BcastBegin()

◆ BcastEnd()

◆ BitOR()

◆ CopyGroupFromBuffer()

◆ CopyGroupToBuffer()

◆ Create()

◆ Finalize()

◆ GetGroupTopology() [1/2]

◆ GetGroupTopology() [2/2]

◆ GetNeighborLDofTable()

◆ GetNeighborLTDofTable()

◆ GroupLDofTable() [1/2]

◆ GroupLDofTable() [2/2]

◆ Max()

◆ Min()

◆ PrintInfo()

◆ Reduce() [1/2]

◆ Reduce() [2/2]

◆ ReduceBegin()

◆ ReduceEnd()

◆ ReduceGroupFromBuffer()

◆ SetLTDofTable()

◆ Sum()

Member Data Documentation

◆ buf_offsets

◆ comm_lock

◆ group_buf

◆ group_buf_size

◆ group_ldof

◆ group_ltdof

◆ gtopo

◆ mode

◆ nbr_recv_groups

◆ nbr_send_groups

◆ num_requests

◆ request_marker

◆ requests