AlignedVector< T >::Deleter Class Reference

Classes
class	DeleterActionBase
class	MPISharedMemDeleterAction

Public Member Functions
	Deleter (AlignedVector< T > *owning_object)
	Deleter (AlignedVector< T > *owning_object, const bool is_shmem_root, T *aligned_shmem_pointer, MPI_Comm shmem_group_communicator, MPI_Win shmem_window)
void	operator() (T *ptr)
void	reset_owning_object (const AlignedVector< T > *new_aligned_vector_ptr)

Private Attributes
std::unique_ptr< DeleterActionBase >	deleter_action_object
const AlignedVector< T > *	owning_aligned_vector

Detailed Description

template<class T>
class AlignedVector< T >::Deleter

A class that is used as the "deleter" for a std::unique_ptr object that AlignedVector uses to store the memory used for the elements.

There are two ways the AlignedVector class can handle memory:

• Allocation via new[] in reserve() where we call posix_memalign() to obtain a chunk of memory and then do placement-new to initialize memory. If this is what we have done, then we need to call the destructors of the currently active elements by hand, and then call std::free() to return memory. In order to call the destructors of currently used elements, the deleter object needs to have access to the owning AlignedVector object to know which of the allocated elements are currently actually used.
• We have called replicate_across_communicator(), in which case the elements have been moved into a memory "window" managed by MPI. In that case, one process (the root process of an MPI communicator that ties together all processes on one node) needs to call the destructor of all elements in this shared memory space, and then all processes on that communicator need to first destroy the MPI window object, and then the MPI communicator for the shared memory node's processes. In this approach, we need to store the following data: A pointer to the owning AlignedVector object to know which elements need to be destroyed, copies of the MPI window and communicator objects, and a couple of ancillary pieces of data.

A common idiom towards using std::unique_ptr with complex de-allocation semantics is to use std::unique_ptr<T, std::function<void (T*)> and then use a lambda function to initialize the std::function deleter object. This approach is used in numerous other places in deal.II, but this has two downsides:

• std::function is relatively memory-hungry. It takes 24 bytes by itself, but then also needs to allocate memory dynamically, for example to store the capture object of a lambda function. This ends up to be quite a lot of memory given that we frequently use small Vector objects (which build on AlignedVector).
• More importantly, this breaks move operations. In a move constructor or move assignment of AlignedVector, we want to steal the memory pointed to, but we then need to install a new deleter object because the deleter needs to know about the owning object to determine which elements to call the destructor for. The problem is that we can't use the old deleter (it is a lambda function that still points to the previous owner that we are moving from) but we also don't know what deleter to install otherwise – we don't know the innards of the lambda function that was previously installed; in fact, we don't even know whether it was for regular or MPI shared-memory data management.

The way out of this is to write a custom deleter. It stores a pointer to the owning AlignedVector object. It then also stores a std::unique_ptr to an object of a class derived from a base class that implements the concrete action necessary to facilitate the "deleter" action. Based on the arguments given to the constructor of the Deleter class, the constructor then either allocates a "regular" or an MPI-based action object, and the action is facilitated by an overloaded virtual function.

In the end, this solves both of our problems:

• The deleter object only requires 16 bytes (two pointers) plus whatever dynamic memory is necessary to store the "action" objects.
• In move operations, the only thing that needs to be done is to tell the deleter about the change of owning AlignedVector object, which we can achieve via the reset_owning_object() function.

This scheme can be further optimized in the following way: In the most common case, memory is allocated via posix_memalign() and needs to destroyed via std::free(). Rather than derive an action class for this common case, dynamically allocate an object for this case, and call it, we can just special case this situation: If the pointer to the action object is nullptr, then we just execute the default action. This means that for the most common case, we do not need any dynamic memory allocation at all, and in that case the deleter object truly only uses 16 bytes: One pointer to the owning AlignedVector object, and a pointer to the action object that happens to be a nullptr. Only in the case of the MPI shared memory data management does the action pointer point somewhere, but this case is expensive anyway and so the extra dynamic memory allocation does little harm.

(One could in that case go even further: There is no really only one possible non-default action left, namely the MPI-based destruction of the shared-memory window. Instead of having the Deleter::DeleterActionBase class below, and the derived Deleter::MPISharedMemDeleterAction class, both with virtual functions, we could just get rid of the base class and make the member functions of the derived class non-virtual. We would then either store a pointer to an MPISharedMemDeleterAction object if the non-default action is requested, or a nullptr for the default action. Avoiding virtual functions would make these objects smaller by a few bytes, and the call to the delete_array() function marginally faster. That said, given that the Deleter::MPISharedMemDeleterAction object already stores all sorts of stuff and its execution is not cheap, these additional optimizations are probably not worth it. Instead, we kept the class hierarchy so that one could define other non-standard actions in the future through other derived classes.)

Definition at line 612 of file aligned_vector.h.

Constructor & Destructor Documentation

Deleter() [1/2]

template<class T >

AlignedVector< T >::Deleter::Deleter

(

AlignedVector< T > *

owning_object

)

Constructor. When this constructor is called, it installs an action that corresponds to "regular" memory allocation that needs to be handled by using std::free().

Deleter() [2/2]

template<class T >

AlignedVector< T >::Deleter::Deleter	(	AlignedVector< T > *	owning_object,
		const bool	is_shmem_root,
		T *	aligned_shmem_pointer,
		MPI_Comm	shmem_group_communicator,
		MPI_Win	shmem_window
	)

Constructor. When this constructor is called, it installs an action that corresponds to MPI-based shared memory allocation that needs to be handled by letting MPI de-allocate the shared memory window, plus destroying the MPI communicator, and doing other clean-up work.

Member Function Documentation

operator()()

template<class T >

void AlignedVector< T >::Deleter::operator()

(

T *

ptr

)

The operator called by std::unique_ptr to destroy the data it is storing. This function dispatches to the different actions that this class implements.

reset_owning_object()

template<class T >

void AlignedVector< T >::Deleter::reset_owning_object

(

const AlignedVector< T > *

new_aligned_vector_ptr

)

Reset the pointer to the owning AlignedVector object. This function is used in move operations when the pointer to the data is transferred from one AlignedVector object – i.e., the pointer itself remains unchanged, but the deleter object needs to be updated to know who the new owner now is.

Member Data Documentation

deleter_action_object

template<class T >

std::unique_ptr<DeleterActionBase> AlignedVector< T >::Deleter::deleter_action_object

private

A pointer to the object that facilitates the actual action of destroying the memory.

Definition at line 718 of file aligned_vector.h.

owning_aligned_vector

template<class T >

const AlignedVector<T>* AlignedVector< T >::Deleter::owning_aligned_vector

private

A (non-owned) pointer to the surrounding AlignedVector object that owns the memory this deleter object is responsible for deleting.

Definition at line 724 of file aligned_vector.h.

---

The documentation for this class was generated from the following file:

• include/deal.II/base/aligned_vector.h