#include <deal.II/base/process_grid.h>

Public Member Functions
	ProcessGrid (MPI_Comm mpi_communicator, const unsigned int n_rows, const unsigned int n_columns)

	ProcessGrid (MPI_Comm mpi_communicator, const unsigned int n_rows_matrix, const unsigned int n_columns_matrix, const unsigned int row_block_size, const unsigned int column_block_size)

	~ProcessGrid ()

unsigned int	get_process_grid_rows () const

unsigned int	get_process_grid_columns () const

template<typename NumberType >
void	send_to_inactive (NumberType *value, const int count=1) const

bool	is_process_active () const

Private Member Functions
	ProcessGrid (MPI_Comm mpi_communicator, const std::pair< unsigned int, unsigned int > &grid_dimensions)

Private Attributes
MPI_Comm	mpi_communicator

MPI_Comm	mpi_communicator_inactive_with_root

int	blacs_context

const unsigned int	this_mpi_process

const unsigned int	n_mpi_processes

int	n_process_rows

int	n_process_columns

int	this_process_row

int	this_process_column

bool	mpi_process_is_active

Friends
template<typename NumberType >
class	::ScaLAPACKMatrix

Detailed Description

A class taking care of setting up a two-dimensional processor grid. For example an MPI communicator with 5 processes can be arranged into a 2x2 grid with the 5-th processor being inactive:

     |   0     |   1
-----| ------- |-----
0    |   P0    |  P1
     |         |
-----| ------- |-----
1    |   P2    |  P3

A shared pointer to this class is provided to ScaLAPACKMatrix matrices to perform block-cyclic distribution.

Note that this class allows to setup a process grid which has fewer MPI cores than the total number of cores in the communicator.

Currently the only place where one would use a ProcessGrid object is in connection with a ScaLAPACKMatrix object.

Author: Benjamin Brands, Denis Davydov, 2017

Definition at line 60 of file process_grid.h.

Constructor & Destructor Documentation

◆ ProcessGrid() [1/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	MPI_Comm	mpi_communicator,
		const unsigned int	n_rows,
		const unsigned int	n_columns
	)

Constructor for a process grid with n_rows and n_columns for a given mpi_communicator. The product of rows and columns should be less or equal to the total number of cores in the mpi_communicator.

Definition at line 206 of file process_grid.cc.

◆ ProcessGrid() [2/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	MPI_Comm	mpi_communicator,
		const unsigned int	n_rows_matrix,
		const unsigned int	n_columns_matrix,
		const unsigned int	row_block_size,
		const unsigned int	column_block_size
	)

Constructor for a process grid for a given mpi_communicator. In this case the process grid is heuristically chosen based on the dimensions and block-cyclic distribution of a target matrix provided in n_rows_matrix, n_columns_matrix, row_block_size and column_block_size.

The maximum number of MPI cores one can utilize is \(\min\{\frac{M}{MB}\frac{N}{NB}, Np\}\), where \(M,N\) are the matrix dimension and \(MB,NB\) are the block sizes and \(Np\) is the number of processes in the mpi_communicator. This function then creates a 2D processor grid assuming the ratio between number of process row \(p\) and columns \(q\) to be equal the ratio between matrix dimensions \(M\) and \(N\).

For example, a square matrix \(640x640\) with the block size \(32\) and the mpi_communicator with 11 cores will result in the \(3x3\) process grid.

Definition at line 193 of file process_grid.cc.

◆ ~ProcessGrid()

Utilities::MPI::ProcessGrid::~ProcessGrid ( )

Destructor.

Definition at line 217 of file process_grid.cc.

◆ ProcessGrid() [3/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	MPI_Comm	mpi_communicator,
		const std::pair< unsigned int, unsigned int > &	grid_dimensions
	)

private

A private constructor which takes grid dimensions as an std::pair.

Definition at line 103 of file process_grid.cc.

Member Function Documentation

◆ get_process_grid_rows()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_rows ( ) const

Return the number of rows in the processes grid.

◆ get_process_grid_columns()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_columns ( ) const

Return the number of columns in the processes grid.

◆ send_to_inactive()

template<typename NumberType >

template void Utilities::MPI::ProcessGrid::send_to_inactive< int >	(	NumberType *	value,
		const int	count = `1`
	)		const

Send count values stored consequently starting at value from the process with rank zero to processes which are not in the process grid.

Definition at line 229 of file process_grid.cc.

◆ is_process_active()

bool Utilities::MPI::ProcessGrid::is_process_active ( ) const

Return true if the process is active within the grid.

Friends And Related Function Documentation

◆ ::ScaLAPACKMatrix

template<typename NumberType >

friend class ::ScaLAPACKMatrix

friend

Declare class ScaLAPACK as friend to provide access to private members.

Definition at line 67 of file process_grid.h.

Member Data Documentation

◆ mpi_communicator

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator

private

An MPI communicator with all processes (active and inactive).

Definition at line 139 of file process_grid.h.

◆ mpi_communicator_inactive_with_root

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator_inactive_with_root

private

An MPI communicator with inactive processes and the process with rank zero.

Definition at line 144 of file process_grid.h.

◆ blacs_context

int Utilities::MPI::ProcessGrid::blacs_context

private

BLACS context. This is equivalent to MPI communicators and is used by ScaLAPACK.

Definition at line 150 of file process_grid.h.

◆ this_mpi_process

const unsigned int Utilities::MPI::ProcessGrid::this_mpi_process

private

Rank of this MPI process.

Definition at line 155 of file process_grid.h.

◆ n_mpi_processes

const unsigned int Utilities::MPI::ProcessGrid::n_mpi_processes

private

Total number of MPI processes.

Definition at line 160 of file process_grid.h.

◆ n_process_rows

int Utilities::MPI::ProcessGrid::n_process_rows

private

Number of rows in the processes grid.

Definition at line 165 of file process_grid.h.

◆ n_process_columns

int Utilities::MPI::ProcessGrid::n_process_columns

private

Number of columns in the processes grid.

Definition at line 170 of file process_grid.h.

◆ this_process_row

int Utilities::MPI::ProcessGrid::this_process_row

private

Row of this process in the grid.

It's negative for in-active processes.

Definition at line 177 of file process_grid.h.

◆ this_process_column

int Utilities::MPI::ProcessGrid::this_process_column

private

Column of this process in the grid.

It's negative for in-active processes.

Definition at line 184 of file process_grid.h.

◆ mpi_process_is_active

bool Utilities::MPI::ProcessGrid::mpi_process_is_active

private

A flag which is true for processes within the 2D process grid.

Definition at line 189 of file process_grid.h.

The documentation for this class was generated from the following files:

deal.II/base/process_grid.h
source/base/process_grid.cc

Public Member Functions

Private Member Functions

Private Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

◆ ProcessGrid() [1/3]

◆ ProcessGrid() [2/3]

◆ ~ProcessGrid()

◆ ProcessGrid() [3/3]

Member Function Documentation

◆ get_process_grid_rows()

◆ get_process_grid_columns()

◆ send_to_inactive()

◆ is_process_active()

Friends And Related Function Documentation

◆ ::ScaLAPACKMatrix

Member Data Documentation

◆ mpi_communicator

◆ mpi_communicator_inactive_with_root

◆ blacs_context

◆ this_mpi_process

◆ n_mpi_processes

◆ n_process_rows

◆ n_process_columns

◆ this_process_row

◆ this_process_column

◆ mpi_process_is_active