Open a terminal and login to the ECS login node following practicals covered on Monday (Teleport or VDI) cd to HPCPERM folder:
ssh ecs-login cd $SCRATCH
On ECS/HPC "mars" command is installed as part of MARS client and can be called without loading any module. The solutions below provide valid MARS requests which can be passed to the mars command as:
mars < myrequestfile
# or
mars myrequestfile
Basic MARS usage
Simple retrieves
1)
Retrieve 1000,850,700 and 500 hPa temperatures from the 15th of October 2023 12 UTC Analysis > see in MARS Catalogue
Check the request output. Any "ERROR" or "WARN" messages? If not, check if the target file exists:
ls t.grb
This morning you learned how to use ecCodes. You can use it now to check content of your GRIB file:
module load ecmwf-toolbox grib_ls t.grb
How many messages does the GRIB file contain? Does it match the number of expected fields retrieved from MARS?
2)
We run the first MARS request on a login node of ECS. Because of the limitations imposed on interactive sessions (e.g., memory, cpu time) it is better to use batch jobs to run MARS request which can be large with long runtime. Yesterday, you learned how to run fractional jobs. Let's use it know to execute next MARS request. The following template script can be used for this purpose:
#!/bin/bash
#SBATCH --job-name=mars_job
#SBATCH --qos=ef
#SBATCH --time=1:00:00
#SBATCH --output=mars.%j.out
#SBATCH --error=mars.%j.out
# For larger requests downloading data from tapes
# extend walltime "--time" to 1 day or longer
set -xe
# Enable multi target option:
export MARS_MULTITARGET_STRICT_FORMAT=1
export SCRATCH
export PERM
mars <<EOF
#{paste your MARS request here}
EOF
More examples can be found on: HPC2020: example Slurm serial batch job scripts for ECS
Use the template above to retrieve wind components valid for 5 days ago at 12:00 and 18:00 from forecast made 7 days ago at 12 UTC at all 137 model levels > see in MARS Catalogue
Submit your MARS job with:
ecsbatch mars_job.sh
You can check the job status with:
watch ecsqueue -u $USER
once the MARS job is complete, please check the job's output file, read MARS messages, check for any ERROR or WARNING messages. Read "INFO" messages, which action has taken longest time:
From now on, we will use mars_job.sh template to run all MARS requests as batch jobs.
3)
Use the same batch job template (mars_job.sh) from example 2) to run MARS request retrieving the 1000 hPa geopotential from two weeks old 12 UTC forecast (> see in MARS Catalogue) steps: 12 hour to 240 hour with 12 hour intervals:
4) Fix errors in request:
retrieve,
class = od,
date = -7,
expver = 1,
levelist = 500/700/1000,
levtype = pl,
param = 130.128,
step = 83/to/93/by/1,
stream = oper,
time = 12:00:00,
type = fc,
target = "$SCRATCH/temp.pl.grb"
This request returns some errors:
mars - ERROR - 20231030.070854 - Last error is -43 mars - ERROR - 20231030.070854 - Expected 33, got 27. mars - ERROR - 20231030.070854 - Request failed
Retrieved number of fields is lower than requested. Use MARS catalogue to see which of requested values are not available in MARS. You can use eCcodes to list content of resulting GRIB file and compare it to request:
grib_ls temp.pl.grb
Fix the MARS request by removing non-existing values from the request and re-run it again.
Post-processing
1)
Retrieve the 1000, 500 and 300 hPa temperatures valid for day before yesterday at noon from the 12 UTC forecast of four days ago on an N48 Gaussian Grid > see in MARS Catalogue
Retrieve the same on a latitude/longitude grid, 1.5 by 1.5 degrees. > see in MARS Catalogue
Retrieve European domain on the same latitude-longitude grid as in the previous retrieval. Compare the size of the resulting file with the previous one. > see in MARS Catalogue
Use grib_ls to print grid type for each of the three GRIB files download in above requests. Is the resulting grid expected?
How many Latitude and Longitude point are found in European domain with 1.5 * 1.5 degrees resolution?
2)
Retrieve surface winds for previous week's 24 hour forecast on the European area converting to a 0.57 by 0.57 degrees latitude/longitude grid. > see in MARS Catalogue
What happens to the conversion?
With grib_dump you will see that the area in 10u_10v.grb and previously downloaded file from European area: t.ll.europe.grb are not identical.
Advanced MARS usage
Multi-target
Ensemble forecasts
1)
Retrieve the 2 metre temperature and 2 metre dew point temperature fields of the first 10 members of a week old 12 UTC ensemble prediction (type: perturbed forecast) for 2, 3 and 4 days in advance > see in MARS Catalogue
2)
Retrieve the same for the control forecast. > see in MARS Catalogue
Note that this time we used the MARS keyword "type" and the ecCodes key "shortName" in the name of the target file. How many target files did the requests create?
Efficiency
In one of previous' requests output find a line looking like this:
mars - INFO - ............... - The efficiency of your requests in the last 12 hours is
What does it say, what is efficiency of your MARS usage?
This score is based on number of times you access the same data tape during 12 hours. If you access the same tape more than once, the score will be lower than 100%. More accesses to the same tame - lower the score is. Conclusion: One should download all required data from a single tape file in one go to avoid queueing for the same tame multiple times!
Question: You need to download selection of surface parameters from operational analysis at 12 UTC, every day for two consecutive months September and October 2021. How many requests are you going to run?
Tip: Use MARS catalogue to see which analysis are stored together in a tape file.
List
Find the request size of for control forecast for a historical date without downloading the data:
list,
class=od,
date=2020-01-01,
expver=1,
levtype=sfc,
param=167.128,
step=1/to/96/by/1,
stream=enfo,
time=12,
type=cf
Since this only query MARS database without downloading the data and running interpolation, the request can be run interactively on the login node:
mars myrequestfile
instead of using batch system.
What is the total size of requested data?
To find which portion of requested data is available online vs on tape, print the request "cost":
list,
output=cost,
class=od,
date=2020-01-01,
expver=1,
levtype=sfc,
param=167.128,
step=1/to/96/by/1,
stream=enfo,
time=12,
type=cf
Is the data available on-line or on tapes only?
Run the same request for 7 days:
list,
output=cost,
class=od,
date=2020-01-01/to/2020-01-07,
expver=1,
levtype=sfc,
param=167.128,
step=1/to/96/by/1,
stream=enfo,
time=12,
type=cf
On how many tapes is the data distributed now?
How about entire month of such data:
list,
output=cost,
class=od,
date=2020-01-01/to/2020-01-31,
expver=1,
levtype=sfc,
param=167.128,
step=1/to/96/by/1,
stream=enfo,
time=12,
type=cf
If you need to download this data for entire January 2020, how would you proceed:
- Running one request for entire month (pros / cons)
- Running 31 daily requests (pros / cons)
?
Read, Write, Compute
Retrieve U and V wind components at 10 metre from 10 day old analysis and calculate wind speed > see in MARS catalogue
retrieve,
class = od,
expver = 1,
stream = oper,
type = analysis,
date = -10,
time = 12,
levtype = sfc,
param = 165.128, # u
grid = 1/1,
fieldset = u
retrieve,
param = 166.128, # v
fieldset = v
compute,
formula = "sqrt(u*u + v*v)",
fieldset = speed
write,
fieldset = speed,
target = "windspeed"
Additional exercise (question from Nikolaos):
This request will read previously created GRIB file, crop the European area from global domain and save it as another GRIB file:
read,
source = "windspeed",
grid = 1/1,
area = europe,
target = "europe_windspeed"
To convert previously created GRIB file to netCDF using ecCodes in a terminal on the ECS or HPC:
>module load ecmwf-toolbox >grib_to_netcdf -o windspeed.nc windspeed grib_to_netcdf: Version 2.30.2 grib_to_netcdf: Processing input file 'windspeed'. grib_to_netcdf: Found 1 GRIB field in 2 files. grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate grib_to_netcdf: Creating netCDF file 'windspeed.nc' grib_to_netcdf: NetCDF library version: 4.9.1 of Feb 9 2023 13:54:09 $ grib_to_netcdf: Creating large (64 bit) file format. grib_to_netcdf: Defining variable 'u10'. grib_to_netcdf: Done.