Forecasting Lab Guide to using the WRF Model at NC State - Part 1

From WolfWikis

Foreword

This User’s Guide is intended to serve as a handy reference for users of the Weather Research and Forecasting (WRF) model, particularly those running it on NC State Linux computing systems. The document is designed with local applications in mind; in other words, it is not intended to be all-inclusive, but rather to provide how-to information for the types of applications for which we use WRF, and for running it on available NC State computers. Therefore, it contains specifics about our own computing environment and software preferences, such as GEMPAK, and does not include information that can be obtained from the main WRF documentation sources, which are far more comprehensive. See http://www.wrf-model.org/index.php or http://box.mmm.ucar.edu/wrf/users/ for these sources.

This document reflects the collective efforts of several individuals working in the Meteorological Analysis and Prediction Lab (a.k.a. “Forecasting Lab”) at NC State. There was a great deal of “trail blazing” that went on in order to get the WRF system up and running on our local Linux clusters; these efforts took place after 3 of us attended the WRF Tutorial Class in Boulder in July 2004.

Any person using WRF output in a publication should acknowledge UCAR/NCAR/MMM for provision of the WRF model, and the National Science Foundation for funding these entities. Also, we don’t mean to be possessive about this manual, but it is extremely important, as a matter of ethics and scientific principle, to give credit where it is due. Therefore, I would request that those of you who utilize this manual heavily acknowledge it as a source of assistance in works that are derived from model runs facilitated by this document. There is no formal reference for it, but I would request that this be mentioned in the acknowledgement section of a publication. Thank you for your consideration.

Finally, if you have contributions, corrections, or comments, please provide these to us so that we can continue to update and upgrade the manual over time.

Finally, a special thank you to Kelly Mahoney, who has graciously volunteered to take on the thankless task of serving as the curator of the WRF Users Guide!

-Gary Lackmann
13 December 2005

NOTE: Feb 2008: This wiki is under construction, and needs to be broken up into multiple wiki pieces due to length restrictions. Currently, we have the original ("The Basics") still posted here...the second half (Part 2, "The Extras"), as well as the remaining sections on SST and namelist hints are found here: http://wikis.lib.ncsu.edu/index.php/Forecasting_Lab_Guide_to_using_the_WRF_Model_at_NC_State_-_Part_2 http://wikis.lib.ncsu.edu/index.php/Forecasting_Lab_Guide_to_using_the_WRF_Model_at_NC_State_-_Part_3

Thanks for your patience. -KMM

NOTE ABOUT EDITING WIKI: AS WE TRANSITION TO A MORE STREAMLINED ORGANIZATION, PLEASE MAKE ANY EDITS TO THE MATERIAL CONTAINED IN THIS PARTICULAR DOCUMENT TO THE FOLLOWING VERSION: http://wikis.lib.ncsu.edu/index.php/Forecasting_Lab_Guide_to_using_the_WRF_Model_at_NC_State

THANK YOU!!

The Basics

Obtaining Data

General comments: There are ftp sources for real-time gridded model data that can be used to initialize WRF, along with archived, historical datasets. For data archived in GEMPAK format, they can be put into a WRFSI-compatible format using existing programs, but this will not be discussed below at this time. Also, note that important sources of gridded model initial condition data are the global and regional reanalysis datasets that have been developed at NCAR and NCEP. Some information on these sources is included below.

1. To download model initial conditions (e.g. Eta/NAM 211/212, GFS, NARR, etc.) access the NOMADS server by going to: [for common data problems, see Random Common Problems in General Troubleshooting section.

http://nomads.ncdc.noaa.gov/data-access.html#weather_datasets

(for more recent, higher resolution datasets (e.g. NAM218, etc.), go to: http://nomads.ncdc.noaa.gov/#hires_weather_datasets)

2. Select the model gridded data of choice and click ftp4u in the table corresponding to the data source that you need. Choose the desired dates, cycles, and forecast hours, and then click “Build Collection.”

3. If the data you have requested is online (i.e. is fairly recent), you can download it by clicking “Select files for FTP.” Instructions for how to log into the server will follow after the files have been ftp’ed. If the data is not online, you will be notified by email (after filling out the necessary information) when it is available, with instructions explaining how to retrieve it.

4. In order to run WRF you will need some form of SST data. If it is not included in your IC grids, you can download 0.5deg SST data at polar.ncep.noaa.gov (cd to /pub/history/sst for archive; /pub/sst for current data). For SST data before 2001, weekly grib 1deg SST is available at http://nomad3.ncep.noaa.gov/pub/sst/weekly/. For higher resolution SST data (4 km down to 1.27 km): http://www.class.noaa.gov (see end of WRF guide for instructions on using these files for WRF).

Obtaining WRF and WRFSI

1. Downloading WRF

To download the latest version of WRF go to:

http://www.mmm.ucar.edu/wrf/users/download/get_source.html

You may have to register as a new user at the bottom of the page, after which you can download the latest version here http://www.mmm.ucar.edu/wrf/users/wrfv2/updates.html

Unzip and untar the files, and everything will be placed in the WRFV2 directory.

2. Downloading WRFSI

To download the latest version of WRFSI go to:

http://www.wrf-model.org/si/release

Unzip and untar the file in the WRFV2 directory that you just created in Step 1, and this will put all the SI files in the correct location.

3. Compiling WRF and installing WRFSI

Compiling WRF

a. To compile WRF it has to point to the netcdf libraries. On the cluster they are located in

 /opt/netcdf-3.6.0-p1

So, in your aliases.csh file you need to add the following line:

 setenv NETCDF /opt/netcdf-3.6.0-p1

You'll also need to add these lines to your aliases.csh file if you want to compile in MPI mode:

 alias mpif90     '/opt/mpich/ethernet/pgcc/bin/mpif90'
 alias mpicc      '/opt/mpich/ethernet/pgcc/bin/mpicc'

and add the following to your PATH:

 /opt/mpich/ethernet/pgcc/bin

b. Configure WRF In the WRFV2 directory type:

 configure

You will then choose from the options shown about how you want to compile WRF (single processor (1), MPI (3), etc.). (For version 2.2, if you are doing a nested run, you most likely want to select the RSL_LITE option)

This creates a file called configure.wrf which contains all the default compilation options for the platform you are working on.

c. Compile WRF

To compile in MPI mode, delete “-fast” from FCOPTIM settings in configure.wrf, and type setenv F90FLAGS in the terminal window. Also, type “add pgi” in the terminal window.

If you type

 compile 

you will see a list of choices for compiling the generic WRF model as well as options for several test cases.

To compile the model for a test case type:

 compile <testname>

If the model compiles successfully, the files

 wrf.exe
 ideal.exe

will appear in the /main directory.

To compile WRF for a case with real data type

 compile em_real

If this compiles successfully, the files

 ndown.exe
 real.exe
 wrf.exe 

will appear in the /main directory

To clean directories of all object files and executables (which you will need to do if your first compiling attempt didn’t work and you’re doing it again!), type

 clean

Typing

 clean -a 

removes all built files, including configure.wrf

Get GEOG data

Now you must obtain all of your geographical data. You can do this by doing any of the following:

1. cd into your WRFV2/wrfsi/extdata/GEOG directory and type:

 ftp aftp.fsl.noaa.gov
 anonymous
 <email address as password>
 cd divisions/frd-laps/WRFSI/Geog_Data

and obtain the following files:

• topo_30s/topo_30s.??.tar.gz (“??” Correspond to domain area…if modeling over CONUS only need NW files…if in doubt download all)
• landuse_30s/landuse_30s.??.tar.gz
• soiltype_top_30s/soiltype_top_30s.??.tar.gz
• soiltype_bot_30s/soiltype_bot_30s.??.tar.gz
• greenfrac.tar.gz
• soiltemp_1deg.tar.gz
• albedo_ncep.tar.gz
• islope.tar.gz
• maxsnowalb.tar.gz
• pctland_10m.tar.gz

Untar (tar xvfz filename) on each of the downloaded files.

2. The data can also be downloaded from: http://www.mmm.ucar.edu/wrf/users/download/get_source2.html

3. Generally, the easiest thing to do is to copy or symbolically link these files from someone else’s directory that has already downloaded these files. Or you can copy the file GEOG.tar.gz from /lockers/PAMS/Volume1/kmmahon2/wrf_files_USE and unzip and untar in your /wrfsi/extdata directory.

Install wrfsi

a. Type

 add pgi

in the terminal window [if using a version prior to 2.1, you must also type setenv LANG en_US in terminal window as well]

b. Run perl script (found in /WRFV2/wrfsi/ directory) install_wrfsi.pl to install wrfsi. (answer yes when it asks if you want to install the graphical user interface!)

c. If this worked you should have an executable in this directory called “wrf_tools”

Using WRFSI to set up your run

1. If on bigdog cluster, get on a compute node

2. From /WRFV2/wrfsi/ directory, type wrf_tools

3. When WRFSI comes up, the first thing you will do is create your domain:

a. In the “Horizontal Grid” tab, draw your approximate domain the box provided and click “update map.” (unless your domain extends to very high latitudes, you should use LCC for your projection.)

b. Enter gridspacing and the horizontal grid dimensions, clicking update map again. Repeat until you have the grid you want.

c. Choose your vertical levels (make sure your ICs have data up to the level you choose for the pressure at the top of the model)

d. After checking to make sure that everything is pointing to the right data locations…

e. Localize your domain. (This actually creates the domain grid that you have specified.)

4. The second step is the “Initial Data” step. Here you will set up your run to see the initial and boundary conditions.

a. Type the path to each data source that you are using in the corresponding space.

b. Click on the script tab at the top, and choose your data source, editing the command line to reflect the forecast length in hours (-l), the time interval between boundary condition files in hours (-t), and the cycle time to use for the model simulation (-s) in YYYYMMDDHH format. Run this script for each data source that you are using (e.g., if you are using both Eta and a separate SST field, run the script once for each one.)

c. SST Data: If you are using a separate SST field, let –l be set to 0 and –t be set to 12 and HH be 00. Also, recent experiments suggest setting –l to the forecast length.

5. Finally, you are ready to interpolate your data. This is similar to the last step: a. In the “Controls” tab, just make sure the data labels are all pointing to the right data. (Note: If using an additional SST field, type ‘SSTDATA’ into the “CONSTANTS_FULL_NAME” blank.

b. Click on the “Script” tab, and as before, modify the command line (only this time (-f) is the forecast length in hours, and the other two options are as above) [Depending on the size of your domain, running these scripts can take quite awhile so give them a chance!]

c. If this runs successfully you will have files that look like: wrf_real_input_em.d01.YYYY-MM-DD_HH:MM:SS in the /wrfsi/domains/yourdomainname/siprd directory.

Using WPS (WRF Preprocessor System )(WRFV2.2)

To download WPS, visit http://www.mmm.ucar.edu/wrf/users/download/get_source2.html

1.) Compiling WPS

Download WPS, unzip and untar it in the same directory as WRF. It will not compile correctly unless WRF has already been compiled and is in the same directory.

 tar xf WPS.tar.gz
 ls
 drwxr-xr-x  8     512 Oct 30 16:38 WPS
 -rw-r--r--  1  537490 Oct 30 16:38 WPS.tar.gz
 drwxr-xr-x 17     512 Oct 30 16:18 WRFV2

Go into the WPS directory and configure using option 3. ** Note: This option only allows regular grib format files to be used as input, with no grib2 files. However, compiling with the analogous option that does allow grib2 format does not produce the ungrib.exe executable, which is needed to complete the WPS process.

 cd WPS 
 configure
 choose option 3
 compile
 

Next, issue the compile command. It should compile for only a couple of minutes.

This should produce 3 executables:

• geogrid.exe -> localizes domain
• ungrib.exe -> extracts data from grib files
• metgrid.exe -> interpolates data to the grid

2.) Running WPS

First, unzip and untar the geographic data (geog), so that it can be utilized by WPS.

All 3 WPS executables run using information from the namelist.wps. This text file contains information about the domain grid, paths to the geographic data, the time period of the run, etc. Some variables should look familiar from the namelist that WRF runs with, although they are definitely not interchangeable. A complete description of the options in namelist is in the WPS directory in namelist.wps-all_options.

The most convenient way to set up the namelist is with the assistance of the Domain Wizard GUI, as described in the proceeding section. I’d recommend taking a look at the Domain Wizard section now. However, WPS can be run completely from the command line if the namelist is properly configured.

Each executable uses a different part of the namelist file. The &share record is the first chunk of the namelist, and has info that all the executables need, including info on the numerics, number of domains, LBCs, ect.

&share
 wrf_core = 'ARW',
 max_dom = 2,
 start_date = '2006-08-16_12:00:00','2006-08-16_12:00:00',
 end_date   = '2006-08-16_18:00:00','2006-08-16_12:00:00',
 interval_seconds = 21600
 io_form_geogrid = 2,
 opt_output_from_geogrid_path = './',
 debug_level = 0
/

a.) geogrid

The second record is the geogrid section.

&geogrid
 parent_id         =   1, 1,
 parent_grid_ratio =   1, 3,
 i_parent_start    =   1, 31
 j_parent_start    =   1, 17
 s_we              =   1, 1,
 e_we              =  74,112,
 s_sn              =   1, 1,
 e_sn              =  61,97,
 geog_data_res     = '10m','2m',
 dx = 30000,
 dy = 30000,
 map_proj = 'lambert',
 ref_lat   = 34.83
 ref_lon   = -81.03
 truelat1  =  30.0,
 truelat2  =  60.0,
 stand_lon = -98.
 geog_data_path = '/data3a/mp/gill/DATA/GEOG'
 opt_geogrid_tbl_path = 'geogrid/'
/

Set up the domain grid either by modifying this file directly or through the Domain Wizard GUI. Then run geogrid.exe.

 geogrid.exe

This should produce the files geo_em.d0N.nc, where N is the domain number.

b.) ungrib

The next step is to extract data from the grib files. The &share and &ungrib records in the namelist are used in this step. However, &share should already be set up and the &ungrib group just specifies the output format (which should be left as the default value).

&ungrib
 out_format = 'WPS'
 prefix = 'FILE'
/

However, ungrib needs some additional input: a Vtable and grib files with a specific naming convention. Before running ungrib, you’ll need to symbolically link these files in the current directory with the appropriate file names. Again, Domain Wizard can do this for you, or you can do it from command line.

First, find the correct Vtable in the ungrib/Variable_Tables directory and link it.

 ln -s ungrib/Variable_Tables/Vtable.GFS Vtable

Next, link your grib files under the filenames GRIBFILE.AAA, GRIBFILE.AAB, …, GRIBFILE.ZZZ. A csh script called link_grib.csh can link and name the files correctly.

 link_grib.csh /mmmtmp/mm5rt/data/gfs/gfs_061101_12_*

Run ungrib.

 ungrib.exe

This should produce a file of the format FILE:YYYY-MM-DD_HH for each input grib file.

c.) metgrid

Metgrid is the final step to interpolate your data onto the appropriate grid. It uses the &share and &metgrid record in the namelist.wps file. This record specifies the output format and the prefix of the files from ungrib. There is probably no need to edit these.

&metgrid
 fg_name                      = './FILE'
 io_form_metgrid              = 2, 
/

• • Note: vertical levels are specified in the last record, &mod_levs. I have yet to try modifying them, but the utility program mod_levs.exe is supposed to handle this. More info is available in the WPS users guide link at the end of this section.

Metgrid uses the output from the previous two steps. If they ran correctly, you should be ready for this last step.

 metgrid.exe

This should produce files named met_em.d0N.YYYY-MM-DD_HH:00:00, which can be used to run real.exe in the same manner as in previous WRF versions.

Additional info:
http://www.mmm.ucar.edu/wrf/users/wpsv2/wps.html
http://www.mmm.ucar.edu/wrf/users/docs/user_guide/users_guide_chap3.html

Domain Wizard

Domain Wizard (DW) is available from http://www.wrfportal.org/DomainWizard.html

1.) Launch Domain Wizard

In these procedures, the pre-compiled DW is run directly from IE on Windows by clicking on the link "Click here to run Domain Wizard" on the above webpage. On the same page, there is a tar file of DW available to compile on a Linux machine, using Java 1.4 or later (includes Java Webstart).

2.) SSH in and specify directories

After clicking on the link to launch the GUI, an ssh login window should appear to input information to ssh into whatever machine you have compiled WPS on. In this example, WPS has been compiled on bigdog.

Next, you will input 3 directories: i) the WPS directory ii) the directory where the GEOG files have been untarred and iii) the directory where files for each domain should be put. You can browse for these directories using the buttons on the right.

3.) Create a domain and namelist file

The DW GUI should now appear with 4 tabs at the top.

Now you can either open previously created domains that are stored in the domains folder you specified (using tab 2) or you can open a new one (and skip to tab 3).

Here I create a domain “test” by drawing a box in the GUI or by modifying the values on the right hand side of the GUI, which is set up almost exactly like the old WRFSI.

Now check in the domains directory you specified before, a directory named “test” should be have been created, with a JPEG of the domain you created, as well as the very important “namelist.wps”.

geogrid, ungrib, and metgrid run using the information in namelist.wps, which should be written to that file for you by the DW GUI. Of course, it’s a good idea to check the values in namelist to make sure that it reflects the information in the GUI.

4.) Finish setting up namelist and run WPS executables

Now the GUI will create some symbolic links for you to the necessary grib files. This is done in tab 4 of the GUI.

On the upper left side, specify the Vtable that you’ll use from a pull-down menu. Next, specify where and which grib files you’ll be using for LBC’s. You can browse for the directory and choose the files from a separate GUI that pops up.

Once you’ve specified the Vtable and grib files, symbolic links will be created to your domains directory (in this case, “domains/test”). The “Progress Status” output located on the lower half of the GUI should tell you what is being linked to where.

Next, choose the start and end date for your run (a calendar pops up using the smaller buttons to the right) and the frequency of the grib input files. Again, these changes should be reflected in the namelist.wps the GUI is writing to.

Now that your namelist has been created and the necessary symbolic links are in place, you’re ready to run geogrid, ungrib, and metgrib.exe

As yet, I have not been able to get these executables to run successfully from the DW GUI. However, now that everything is set up correctly, it’s easy enough to just login to bigdog (or wherever WPS is), create symbolic links to these executables inside the domains directory (here, “domains/test”) and just run the executables from command line.

Running WRF

The data is already there to run WRF for one of the idealized test cases, and they can be run by looking at the instructions here: http://www.mmm.ucar.edu/wrf/users/wrfv2/runwrf.html

To run a real case:

1. Move all of the “wrf_real_input_em.d01.YYYY-MM-DD_HH:MM:SS” files that were created in the last step of WRFSI into /test/em_real

2. Edit the namelist.input file (located in /test/em_real) to reflect your run specifications (see namelist.input hints at end of manual, or a description of the variables can be found online at: http://www.mmm.ucar.edu/wrf/users/wrfv2/wrf-namelist.html. Also, the list of available model physics packages is in the README file in the WRFV2 directory). Also, the wrfsi namelist may help you with your domain dimensions, etc. It is located at: /WRFV2/wrfsi/domains/yourdomainname/static/wrfsi.nl.

a. Single processor mode: To process the IC files, type

 real.exe

And the files wrfinput_d01 and wrfbdy_d01 will be generated.

To run the model, type

 wrf.exe

As the model runs, files called wrfout_d1_* will be created.

b. MPI mode 1. First, be sure that “nio_tasks_per_group” in the namelist.input file is equal to the number of processors you will be using.

2. real.exe can be run on a compute node from a terminal window, or using multiple processors using a script like the one found in /lockers/PAMS/Volume1/kmmahon2/wrf_files_USE/wrf_prep.csh. (See Sample Scripts for an example of wrf_prep.csh). When real.exe runs successfully, the files wrfinput_d01 and wrfbdy_d01 will be generated in the test/em_real directory.

3. Finally, to run the model, run wrf.exe. This can also be easily done from a script such as the one found in: /lockers/PAMS/Volume1/kmmahon2/wrf_files_USE/wrf_run.csh. (See Sample Scripts for an example of wrf_run.csh). As the model runs, files called wrfout_d1_* will be created.

Viewing WRF output

One way to view WRF output is by converting it into GEMPAK. This can be done using a converted developed originally by Ryan Torn (UW) and later modified by Mike Brennan (NCSU). [There is also another version of this that can be downloaded from http://speedy.aos.wisc.edu/~sgdecker/wrf2gem/index.html ]

The converter, wrf2gempak, can be (currently) copied from /lockers/PAMS/Volume1/kmmahon2/wrf_files_USE/wrf2gem/wrf2gempak

(If starting from the tar file, simply untar the wrf2gempak.tar file and type make, and the code should compile correctly. The executable file wrf2gempak will be created.) Run this file, and it will prompt you for the WRF output file and the name of the GEMPAK file you want to create. You then have the option for "bare-bones" output, which is just the basic temperature, heights, etc., or an option to output hydrometeor fields, PBL fields, LSM stuff, etc. Run the conversion on a compute node.

After that conversion is complete, open the file eta_to_pres_loop.csh. Edit the foreach loop to include the forecast hours in the WRF eta-level file you want to convert to pressure-level data. Then run the script and it should prompt you for the information you need. You should then have gempak files ready for viewing!

• • A versatile script that streamlines this process is (currently) located in:

/lockers/PAMS/Volume1/kmmahon2/wrf_files_USE/wrf2gem/wrf2gem_USE.csh Make sure you have copied the wrf2gempak executable into your directory first, though!

Other option: If you would like to have your data plotted in “SGMA” [i.e., really ELVL but called SGMA for the purpose of plotting], a script exists to do this in: /lockers/PAMS/Volume1/kmmahon2/wrf_files_USE/wrf2gem/wrf2gem_SGMA/wrf2gem_SGMA.csh

Acknowledgments

Created by the NCSU Forecasting Lab:

• Dr. Mike Brennan
• Megan Gentry
• Nicole Haglund
• Kevin Hill
• Dr. Gary Lackmann
• Kelly Mahoney

With additional contributions from:

• Matt Borkowski
• Zach Brown
• Dr. Matt Parker