AMReX basics (excessively basic)
WarpX is built on the Adaptive Mesh Refinement (AMR) library AMReX. This section provides a very sporadic description of the main AMReX classes and concepts relevant for WarpX, that can serve as a reminder. Please read the AMReX basics doc page, of which this section is largely inspired.
amrex::Box
: Dimension-dependent lower and upper indices defining a rectangular volume in 3D (or surface in 2D) in the index space.Box
is a lightweight meta-data class, with useful member functions.amrex::BoxArray
: Collection ofBox
on a single AMR level. The information of which MPI rank owns whichBox
in aBoxArray
is inDistributionMapping
.amrex::FArrayBox
: Fortran-ordered array of floating-pointamrex::Real
elements defined on aBox
. AFArrayBox
can represent scalar data or vector data, withncomp
components.amrex::MultiFab
: Collection of FAB (=FArrayBox
) on a single AMR level, distributed over MPI ranks. The concept of ghost cells is defined at theMultiFab
level.amrex::ParticleContainer
: A collection of particles, typically for particles of a physical species. Particles in aParticleContainer
are organized perBox
. Particles in aBox
are organized per tile (this feature is off when running on GPU). Particles within a tile are stored in several structures, each being contiguous in memory: (i) a Struct-of-Array (SoA) foramrex::ParticleReal
data such as positions, weight, momentum, etc., (ii) a Struct-of-Array (SoA) forint
data, such as ionization levels, and (iii) a Struct-of-Array (SoA) for auint64_t
unique identifier index per particle (containing a 40bit id and 24bit cpu sub-identifier as assigned at particle creation time). This id is also used to check if a particle is active/valid or marked for removal.
The simulation domain is decomposed in several Box
, and each MPI rank owns (and performs operations on) the fields and particles defined on a few of these Box
, but has the metadata of all of them. For convenience, AMReX provides iterators, to easily iterate over all FArrayBox
(or even tile-by-tile, optionally) in a MultiFab
own by the MPI rank (MFIter
), or over all particles in a ParticleContainer
on a per-box basis (ParIter
, or its derived class WarpXParIter
). These are respectively done in loops like:
// mf is a pointer to MultiFab
for ( amrex::MFIter mfi(mf, false); mfi.isValid(); ++mfi ) { ... }
and
// *this is a pointer to a ParticleContainer
for (WarpXParIter pti(*this, lev); pti.isValid(); ++pti) { ... }
When looping over FArrayBox
in a MultiFab
, the iterator provides functions to retrieve the metadata of the Box
on which the FAB
is defined (MFIter::box()
, MFIter::tilebox()
or variations) or the particles defined on this Box
(ParIter::GetParticles()
).