Skip to content

AMReX and Friends

Jump to bottom
Matt Larsen edited this page Nov 2, 2020 · 18 revisions

Using Ascent in WarpX

General WrapX build instructions: https://warpx.readthedocs.io/en/latest/building/building.html

Repos:

AMReX

PICASR: (Use as instructions suggest)

WarpX:

mkdir warpx_directory
cd warpx_directory
git clone https://github.com/ECP-WarpX/WarpX.git
git clone https://bitbucket.org/berkeleylab/picsar.git
git clone --branch development https://github.com/AMReX-Codes/amrex.git

Here is my example build (make command) that I used on my mac laptop (make sure you have mpicc in your path)

make COMP=clang USE_OMP=FALSE USE_ASCENT_INSITU=TRUE ASCENT_DIR=path_to_ascent_install

This will build an exe called something like "Bin/main3d.llvm.TPROF.MPI.ex"

For input I am using the "Plasma Wakefield example w/o mesh refinement" from:

https://warpx.readthedocs.io/en/latest/running_cpp/how_to_run.html

To actually use Ascent, here is the integration code

==== Add to Source/WarpXIO.cpp :: WarpX::WritePlotFile () after plot files are written ==

conduit::Node bp_mesh;
MultiLevelToBlueprint( finest_level+1,
                       amrex::GetVecOfConstPtrs(mf),
                       varnames,
                       Geom(),
                       t_new[0],
                       istep,
                       refRatio(),
                       bp_mesh);

///////////////////////////////////////////////////////////////////
// Render with Ascent
///////////////////////////////////////////////////////////////////

// add a scene with a pseudocolor plot
Node scenes;
scenes["s1/plots/p1/type"] = "pseudocolor";
scenes["s1/plots/p1/params/field"] = "divB";
scenes["s1/plots/p1/pipeline"] = "pl1";
// Set the output file name (ascent will add ".png")
scenes["s1/image_prefix"] = amrex::Concatenate("ascent_render_",istep[0]);

Node pipelines;
pipelines["pl1/f1/type"] = "threshold";
conduit::Node &thresh_params = pipelines["pl1/f1/params"];
thresh_params["field"] = "divB";
thresh_params["min_value"] = 25;
thresh_params["max_value"] = 1000;

// setup actions
Node actions;

Node &add_pipes = actions.append();
add_pipes["action"] = "add_pipelines";
add_pipes["pipelines"] = pipelines;

Node &add_act = actions.append();
add_act["action"] = "add_scenes";
add_act["scenes"] = scenes;

actions.append()["action"] = "execute";
actions.append()["action"] = "reset";

Ascent ascent;
Node opts;
opts["exceptions"] = "catch";
opts["mpi_comm"] = MPI_Comm_c2f(ParallelDescriptor::Communicator());
ascent.open(opts);
ascent.publish(bp_mesh);
ascent.execute(actions);
ascent.close();

// end code

Using Ascent in Nyx

Checkout Nyx:

git clone --branch development https://github.com/AMReX-Codes/amrex.git
git clone --branch development https://github.com/AMReX-Astro/Nyx.git

Example Build Command (from my Cyrus' mac, using clang)

cd Nyx/Exec/LyA/
make AMREX_HOME={PATH_TO_AMREX_ABOVE}/amrex \
     USE_OMP=FALSE COMP=clang USE_ASCENT_INSITU=TRUE \
     ASCENT_HOME={PATH_TO_ASCENT_INSTALL}

Example command 2:

make USE_ASCENT_INSITU=TRUE ASCENT_HOME=/usr/workspace/showcase/valentine/ascent-1/install/ USE_OMP=TRUE COMP=gnu USE_HEATCOOL=FALSE -j

Example Run Command: (1 mpi task)

./Nyx3d.llvm.PROF.ex inputs.rt max_step=3

Clone this wiki locally