We are currently trying to retrieve all available CAMS from the cams-global-atmospheric-composition-forecasts dataset which match our needs (twice a day with specific variables, from 2017 to now).
To do so, we've been sending several requests every day, each aiming to retrieve one NetCDF file per day per hour we need (00:00 and 12:00). Since it's taking a long time and it seems that our requests are often QUEUED, we tried to retrieve a large period with one single request by using the start/end date keys and then extracting the information we need.
However, by doing so and by comparing to the information we got by only requesting one day+hour at a time, we noticed that the information for the same day and hour were different. For instance, here are some max differences between variables from the same day and hour, retrieved by requesting a period and extracting the date and hour in question, and then requesting this very same day and hour (we subtracted the two values of the variables to get these differences) :
Variable | Difference |
bcaod550 | 6.6 e-6 |
duaod550 | 4.8 e-5 |
omaod550 | 3.1 e-5 |
ssaod550 | 2.0 e-5 |
The precise date requested was 2019-05-01T00:00:00 and the period went from 2019-05-01T00:00:00 to 2019-05-05T00:00:00.
Since the values should be the same (after all it's the same day and hour !), we are wondering if you would be so kind as to explain these differences, and how we should proceed to obtain the correct data.
Thanks for your help 
5 Comments
Kevin Marsh
Hi Marie-France,
Can you please share examples of the CDS API scripts you were using to request the data?
Thanks,
Kevin
Marie-France LARIF
Hi Kevin,
Yes of course, here are two examples of requests we made, using the MMI on the website, and not the API. Looking at the output, the values inside the 2 CAMS file for the same date/hour differ.
Resource:cams-global-atmospheric-composition-forecasts
Request body:{
"date": "2018-08-24/2018-08-24",
"format": "netcdf",
"leadtime_hour": "0",
"model_level": [
"1",
"2",
"3",
<we ask all values from 1 to 60...>
"58",
"59",
"60"
],
"target": "@target@",
"time": "00:00",
"type": "forecast",
"variable": [
"sea_salt_aerosol_0.03-0.5um_mixing_ratio",
"sea_salt_aerosol_0.5-5um_mixing_ratio",
"sea_salt_aerosol_5-20um_mixing_ratio",
"dust_aerosol_0.03-0.55um_mixing_ratio",
"dust_aerosol_0.55-0.9um_mixing_ratio",
"dust_aerosol_0.9-20um_mixing_ratio",
"hydrophilic_organic_matter_aerosol_mixing_ratio",
"hydrophobic_organic_matter_aerosol_mixing_ratio",
"hydrophilic_black_carbon_aerosol_mixing_ratio",
"hydrophobic_black_carbon_aerosol_mixing_ratio",
"sulphate_aerosol_mixing_ratio"
]
}
Resource:cams-global-atmospheric-composition-forecasts
Request body:{
"date": "2018-08-24/2018-08-29",
"format": "netcdf",
"leadtime_hour": "0",
"model_level": [
"1",
"2",
"3",
<we ask all values from 1 to 60...>
"58",
"59",
"60"
],
"target": "@target@",
"time": [
"00:00",
"12:00"
],
"type": "forecast",
"variable": [
"sea_salt_aerosol_0.03-0.5um_mixing_ratio",
"sea_salt_aerosol_0.5-5um_mixing_ratio",
"sea_salt_aerosol_5-20um_mixing_ratio",
"dust_aerosol_0.03-0.55um_mixing_ratio",
"dust_aerosol_0.55-0.9um_mixing_ratio",
"dust_aerosol_0.9-20um_mixing_ratio",
"hydrophilic_organic_matter_aerosol_mixing_ratio",
"hydrophobic_organic_matter_aerosol_mixing_ratio",
"hydrophilic_black_carbon_aerosol_mixing_ratio",
"hydrophobic_black_carbon_aerosol_mixing_ratio",
"sulphate_aerosol_mixing_ratio"
]
}
As you can see, the only difference between the 2 requests is the date and time.
Thanks a lot for your help
Marie-France
Kevin Marsh
Hi Marie-France,
Apologies for the delay in getting back to you! I think the issue you are seeing is related to the conversion of the original GRIB files ( which has small values) to netCDF. Are you able to use GRIB files rather than netCDF?
Thanks,
Kevin
Marie-France LARIF
Hi Kevin, your answer is in line with what we suspected. No way for us to easily modify our processing chains to use GRIB files, but as the difference is tiny, we will cope with that !
Thanks again for your support
Marie-France
Kevin Marsh
Hi Marie-France,
It may be advisable to use the grib file values to check that the corresponding netCDF values are correctly represented, so that you do not get misleading results when using the netCDF versions of the data.
Hope that helps,
Kevin