Reference documentation for deal.II version 9.3.3
\(\newcommand{\dealvcentcolon}{\mathrel{\mathop{:}}}\) \(\newcommand{\dealcoloneq}{\dealvcentcolon\mathrel{\mkern-1.2mu}=}\) \(\newcommand{\jump}[1]{\left[\!\left[ #1 \right]\!\right]}\) \(\newcommand{\average}[1]{\left\{\!\left\{ #1 \right\}\!\right\}}\)
Public Member Functions | Private Member Functions | Private Attributes | Friends | List of all members
Utilities::MPI::ProcessGrid Class Reference

#include <deal.II/base/process_grid.h>

Public Member Functions

 ProcessGrid (const MPI_Comm &mpi_communicator, const unsigned int n_rows, const unsigned int n_columns)
 
 ProcessGrid (const MPI_Comm &mpi_communicator, const unsigned int n_rows_matrix, const unsigned int n_columns_matrix, const unsigned int row_block_size, const unsigned int column_block_size)
 
 ~ProcessGrid ()
 
unsigned int get_process_grid_rows () const
 
unsigned int get_process_grid_columns () const
 
int get_this_process_row () const
 
int get_this_process_column () const
 
template<typename NumberType >
void send_to_inactive (NumberType *value, const int count=1) const
 
bool is_process_active () const
 

Private Member Functions

 ProcessGrid (const MPI_Comm &mpi_communicator, const std::pair< unsigned int, unsigned int > &grid_dimensions)
 

Private Attributes

MPI_Comm mpi_communicator
 
MPI_Comm mpi_communicator_inactive_with_root
 
int blacs_context
 
const unsigned int this_mpi_process
 
const unsigned int n_mpi_processes
 
int n_process_rows
 
int n_process_columns
 
int this_process_row
 
int this_process_column
 
bool mpi_process_is_active
 

Friends

template<typename NumberType >
class ::ScaLAPACKMatrix
 

Detailed Description

A class taking care of setting up a two-dimensional processor grid. For example an MPI communicator with 5 processes can be arranged into a 2x2 grid with the 5-th processor being inactive:

| 0 | 1
-----| ------- |-----
0 | P0 | P1
| |
-----| ------- |-----
1 | P2 | P3

A shared pointer to this class is provided to ScaLAPACKMatrix matrices to perform block-cyclic distribution.

Note that this class allows to setup a process grid which has fewer MPI cores than the total number of cores in the communicator.

Currently the only place where one would use a ProcessGrid object is in connection with a ScaLAPACKMatrix object.

Definition at line 61 of file process_grid.h.

Constructor & Destructor Documentation

◆ ProcessGrid() [1/3]

Utilities::MPI::ProcessGrid::ProcessGrid ( const MPI_Comm &  mpi_communicator,
const unsigned int  n_rows,
const unsigned int  n_columns 
)

Constructor for a process grid with n_rows and n_columns for a given mpi_communicator. The product of rows and columns should be less or equal to the total number of cores in the mpi_communicator.

Definition at line 224 of file process_grid.cc.

◆ ProcessGrid() [2/3]

Utilities::MPI::ProcessGrid::ProcessGrid ( const MPI_Comm &  mpi_communicator,
const unsigned int  n_rows_matrix,
const unsigned int  n_columns_matrix,
const unsigned int  row_block_size,
const unsigned int  column_block_size 
)

Constructor for a process grid for a given mpi_communicator. In this case the process grid is heuristically chosen based on the dimensions and block-cyclic distribution of a target matrix provided in n_rows_matrix, n_columns_matrix, row_block_size and column_block_size.

The maximum number of MPI cores one can utilize is \(\min\{\frac{M}{MB}\frac{N}{NB}, Np\}\), where \(M,N\) are the matrix dimension and \(MB,NB\) are the block sizes and \(Np\) is the number of processes in the mpi_communicator. This function then creates a 2D processor grid assuming the ratio between number of process row \(p\) and columns \(q\) to be equal the ratio between matrix dimensions \(M\) and \(N\).

For example, a square matrix \(640x640\) with the block size \(32\) and the mpi_communicator with 11 cores will result in the \(3x3\) process grid.

Definition at line 209 of file process_grid.cc.

◆ ~ProcessGrid()

Utilities::MPI::ProcessGrid::~ProcessGrid ( )

Destructor.

Definition at line 232 of file process_grid.cc.

◆ ProcessGrid() [3/3]

Utilities::MPI::ProcessGrid::ProcessGrid ( const MPI_Comm &  mpi_communicator,
const std::pair< unsigned int, unsigned int > &  grid_dimensions 
)
private

A private constructor which takes grid dimensions as an std::pair.

Definition at line 103 of file process_grid.cc.

Member Function Documentation

◆ get_process_grid_rows()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_rows ( ) const

Return the number of rows in the processes grid.

◆ get_process_grid_columns()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_columns ( ) const

Return the number of columns in the processes grid.

◆ get_this_process_row()

int Utilities::MPI::ProcessGrid::get_this_process_row ( ) const

Return row of this process in the process grid.

It's negative for inactive processes.

◆ get_this_process_column()

int Utilities::MPI::ProcessGrid::get_this_process_column ( ) const

Return column of this process in the process grid.

It's negative for inactive processes.

◆ send_to_inactive()

template<typename NumberType >
template void Utilities::MPI::ProcessGrid::send_to_inactive< int > ( NumberType *  value,
const int  count = 1 
) const

Send count values stored consequently starting at value from the process with rank zero to processes which are not in the process grid.

Definition at line 245 of file process_grid.cc.

◆ is_process_active()

bool Utilities::MPI::ProcessGrid::is_process_active ( ) const

Return true if the process is active within the grid.

Friends And Related Function Documentation

◆ ::ScaLAPACKMatrix

template<typename NumberType >
friend class ::ScaLAPACKMatrix
friend

Definition at line 66 of file process_grid.h.

Member Data Documentation

◆ mpi_communicator

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator
private

An MPI communicator with all processes (active and inactive).

Definition at line 160 of file process_grid.h.

◆ mpi_communicator_inactive_with_root

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator_inactive_with_root
private

An MPI communicator with inactive processes and the process with rank zero.

Definition at line 166 of file process_grid.h.

◆ blacs_context

int Utilities::MPI::ProcessGrid::blacs_context
private

BLACS context. This is equivalent to MPI communicators and is used by ScaLAPACK.

Definition at line 172 of file process_grid.h.

◆ this_mpi_process

const unsigned int Utilities::MPI::ProcessGrid::this_mpi_process
private

Rank of this MPI process.

Definition at line 177 of file process_grid.h.

◆ n_mpi_processes

const unsigned int Utilities::MPI::ProcessGrid::n_mpi_processes
private

Total number of MPI processes.

Definition at line 182 of file process_grid.h.

◆ n_process_rows

int Utilities::MPI::ProcessGrid::n_process_rows
private

Number of rows in the processes grid.

Definition at line 187 of file process_grid.h.

◆ n_process_columns

int Utilities::MPI::ProcessGrid::n_process_columns
private

Number of columns in the processes grid.

Definition at line 192 of file process_grid.h.

◆ this_process_row

int Utilities::MPI::ProcessGrid::this_process_row
private

Row of this process in the grid.

It's negative for in-active processes.

Definition at line 199 of file process_grid.h.

◆ this_process_column

int Utilities::MPI::ProcessGrid::this_process_column
private

Column of this process in the grid.

It's negative for in-active processes.

Definition at line 206 of file process_grid.h.

◆ mpi_process_is_active

bool Utilities::MPI::ProcessGrid::mpi_process_is_active
private

A flag which is true for processes within the 2D process grid.

Definition at line 211 of file process_grid.h.

The documentation for this class was generated from the following files:
Generated by doxygen 1.9.4