Compilation

Manual (alternative)

Setup

  • Download WRFotron, WRFChem, make a copy for WRFMeteo without the chemistry folder, download WPS, download WPS Geography files:
cd /nobackup/${USER}
git clone https://github.com/wrfchem-leeds/WRFotron.git
git clone https://github.com/wrf-model/WRF.git WRFChem
git clone https://github.com/wrf-model/WPS.git

cp -r WRFChem WRFMeteo
cd WRFMeteo
rm -rf chem
  • Or copy these folders over from /nobackup/WRFChem:
cd /nobackup/${USER}
cp -r /nobackup/WRFChem/{WRFotron,WRFChem4.2,WRFMeteo4.2,WPS4.2,WPSGeog4} .
  • You will need to remove, or at minimum, change the module load line at the top of config.bash. The modules intel, openmpi, and WRFchem are for the CEMAC installation, and keeping these (and potentially others) can interfere with executables. These need to be removed. NCL, NCO, and conda can be used from CEMAC for manual runs, or you could have your own personal conda environments with NCL and NCO (see below). You can see the manual blueprint in the repository: config.bash.blueprint_manual.
# cemac compilation uses
module load intel/19.0.4 openmpi/3.1.4 WRFchem/4.2 ncl/6.5.0 nco/4.6.0 wrfchemconda/3.7 sge

# for manual compilation remove (at least) intel, openmpi, and WRFchem
module load ncl/6.5.0 nco/4.6.0 wrfchemconda/3.7 sge
  • The executables within pre.bash need to be copied over directly, rather than just linked which is adequate for the CEMAC method. To do this make both of the following replacements. You can see the manual blueprint in the repository: pre.bash.blueprint_manual.
# on line 21, replace:
for aFile in util geogrid ungrib link_grib.csh metgrid
# with:
for aFile in util geogrid geogrid.exe ungrib ungrib.exe link_grib.csh metgrid metgrid.exe

# and then on line 80, replace:
cp -r ${WRFdir}/run/* .
# with:
cp -r ${WRFdir}/run/* .
rm *.exe
cp -r ${WRFdir}/main/*.exe .
cp -r ${WRFmeteodir}/main/wrf.exe wrfmeteo.exe
  • All executables and preprocessors will need to have ./ before them to execute. This includes ungrib.exe, geogrid.exe, metgrid.exe, real.exe, megan_bio_emiss, wesely, exo_coldens, anthro_emiss, fire_emis, and mozbc in pre.bash. Also, wrfmeteo.exe and wrf.exe in main.bash. Also, wrf.exe in main_restart.bash. You can see the manual blueprints in the repository: pre.bash.blueprint_manual, main.bash.blueprint_manual, and main_restart.bash.blueprint_manual.
  • Add links to the preprocessor executables anthro_emis, fire_emis, and mozbc by adding the following code. You can see the manual blueprints in the repository: pre.bash.blueprint_manual.
ln -s ${WRFanthrodir}/anthro_emis . # section 4.a, line 148
ln -s ${WRFfiredir}/fire_emis .     # section 4.b, line 164
ln -s ${WRFMOZARTdir}/mozbc .         # section 6,   line 186
  • Download flex (tool for generating scanners: programs which recognize lexical patterns in text).

  • Download and compile (in serial) preprocessors:
    • anthro_emis (anthropogenic emissions preprocessor).

    • fire_emiss (fire emissions preprocessor).

    • megan (biogenic emissions preprocessor).

    • mozbc (preprocessor for lateral boundary and initial conditions).

    • wes-coldens (exocoldens and season_wesely, O2 and O3 column densities and dry deposition).

    • Check preprocessors have the correct modules and libraries linked via: ldd preprocessor.

conda deactivate
module purge
module load intel netcdf
export NETCDF=$(nc-config --prefix)
export NETCDF_DIR=$NETCDF
export FC=ifort

./make_anthro

./make_fire_emis

./make_util megan_bio_emiss

./make_mozbc

./make_util wesely

./make_util exo_coldens

Conda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
  • Run bash script, read terms, and set path:
bash Miniconda3-latest-Linux-x86_64.sh
  • Create conda environment with Python 3 (with some libraries for analysis), NCL, and NCO:
conda create -n python3_ncl_nco -c conda-forge -c oggm xarray salem xesmf numpy scipy pandas matplotlib rasterio affine ncl nco wrf-python
  • To activate/deactivate conda environment:
conda activate python3_ncl_nco
conda deactivate
  • For more information on conda, visit.
  • Create separate environments for downloading ECMWF data (requires Python 2) and ncview, which you can then load temporarily to execute these functions:
conda create -n python2_ecmwf -c conda-forge ecmwf-api-client
conda create -n ncview -c eumetsat -c conda-forge ncview libpng

Compile WPS, WRFMeteo, and WRFChem

  • Modules:
conda deactivate
module unload conda
module unload openmpi
module load intel
module load intelmpi
module load netcdf
  • Environment variables:
export FC=ifort
export NETCDF=$(nc-config --prefix)
export NETCDF_DIR=$NETCDF
export YACC='/usr/bin/yacc -d'
export FLEX_LIB_DIR='/nobackup/username/flex/lib'
export LD_LIBRARY_PATH=$FLEX_LIB_DIR:$LD_LIBRARY_PATH
export JASPERLIB=/usr/lib64
export JASPERINC=/usr/include

export WRF_EM_CORE=1 # selects the ARW core
export WRF_NMM_CORE=0 # ensures that the NMM core is deselected
export WRF_CHEM=1 # selects the WRFChem module
export WRF_KPP=1 # turns on Kinetic Pre-Processing (KPP)
export WRFIO_NCD_LARGE_FILE_SUPPORT=1 # supports large wrfout files
  • WRFChem compilation:
cd /nobackup/username/WRFChem
./clean -a
./configure
  • HPC option will be specific to your HPC architecture.
  • ARC4 = 15 = INTEL (ifort/icc) (dmpar) e.g. Distributed-Memory Parallelism MPI.
  • Compile for basic nesting: option 1.
  • Compile real (as oppose to ideal simulations).
  • Thousands of messages will appear. Compilation takes about 20-30 minutes.
./compile em_real >& log.compile
  • How do you know your compilation was successful?

    • If you have main/*.exe.
  • Check the executables have all relevant linked libraries:

ldd main/wrf.exe
  • WPS compilation (requires a successfully compiled WRF):
cd /nobackup/username/WPS
./clean -a
./configure
  • HPC option will be specific to your HPC architecture.
  • ARC4 = 17 = INTEL (ifort/icc) (serial).
  • Sometimes configure.wps can assign the incorrect path to WRFChem, check and edit if required:
gedit configure.wps
WRF_DIR="/nobackup/${USER}/WRFChem"

./compile >& log.compile
  • How do you know your compilation was successful?

    • If you have geogrid.exe, metgrid.exe, and ungrib.exe.
  • Check the executables have all relevant linked libraries:

ldd geogrid.exe
  • WRFMeteo compilation:

    • Deselect the WRFChem module
export WRF_CHEM=0

cd /nobackup/username/WRFMeteo
./clean -a
./configure
  • HPC option will be specific to your HPC architecture.
  • ARC4 = 15 = INTEL (ifort/icc) (dmpar).
  • Compile for basic nesting: option 1.
  • Compile real (as oppose to ideal simulations).
  • Thousands of messages will appear. Compilation takes about 20-30 minutes.
./compile em_real >& log.compile
  • Check have main/*.exe.
  • Check the executables have all relevant linked libraries:
ldd main/real.exe
  • If make any changes to pre-processor settings then require a fresh re-compile.
  • Also check if preprocessor requires a different module version that currently compiled with.
  • Run above environment variables to get NetCDF.
  • Add -lnetcdff to Makefile.
  • Note for wes_coldens: FC hardcoded in make_util.
  • Downloaded tools from here.
  • If need JASPER:
wget http://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/jasper-1.900.1.tar.gz
tar xvfz jasper-1.900.1.tar.gz
./configure
make
make install
export JASPERLIB=/usr/lib64 # not installed need own jasper
export JASPERINC=/usr/include
  • If need FLEX:
cd /nobackup/${USER}/flex/lib
./configure --prefix=$(pwd)/../flex
export FLEX_LIB_DIR='/nobackup/${USER}/flex/lib'