Summary -Proposed Tar File Structure
Summary
- 1 tar file contains 1 variable (for a given start date, for all ensemble members);
- The number of realizations included are indicated in the tar file name (using n# - n## where # = index of first realization in that tar file and ## = number of last realization in that tar file);
- The individual tar file size limit on ECFS is 32 GB. Where this is exceeded, 2 (or more) tar files are used, with the number of ensemble members indicated in the tar file name.
- The representative figures below are derived from sample data supplied by Meteo France.
- A checksum file corresponding to each tar file should also be created and supplied with the tar file
For hindcast data
Example below based on sample Hindcast data from Meteo France (forecast data should be handled in a similar way). They are classified by the data product "Type", "Level" and "Realm"
Note: a total of 25 Ensemble members
Provider | Type | Level | Realm | Time Resolution | Tar file name | Example variable | Number of ensemble members | Estimated tar file size (GB) | Number of variables of this type/level/realm |
---|---|---|---|---|---|---|---|---|---|
Meteo France | Hindcast | surface | atmos | 6hr | lfpw_System6-v20170501_hindcast_S1993100100_atmos_6hr_surface_tas_n1-n25.tar | tas | 25 | 3.05 | 6 |
Meteo France | Hindcast | surface | atmos | day | lfpw_System6-v20170501_hindcast_S1993100100_atmos_day_surface_lwee_n1-n25.tar | lwee | 25 | 1 | 16 |
Meteo France | Hindcast | surface | land | day | lfpw_System6-v20170501_hindcast_S1993100100_land_day_surface_mrlsl_n1-n25.tar | mrlsl | 25 | 0.4 | 6 |
Meteo France | Hindcast | pressure | atmos | 12hr | lfpw_System6-v20170501_hindcast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar | va | 25 | 23.325 | 5 |
Meteo France | Hindcast | surface | ocean | 6 hr | lfpw_System6-v20170501_hindcast_S1993100100_ocean_6hr_surface_tso_n1-n25.tar | tso | 25 | 2.15 | 1 |
Meteo France | Hindcast | surface | seaIce | day | lfpw_System6-v20170501_hindcast_S1993100100_seaIce_day_surface_sic_n1-n25.tar | sic | 25 | 0.2 | 1 |
So the first tar file in the table above ( "lfpw_System6-v20170501_hindcast_S1993100100_atmos_6hr_surface_tas_n1-n25.tar" ) would contain all 25 ensemble members for the tas variable for the start date S19931001. Similar tar files would be created for tasmax, tasmin etc.
For forecast data
Note a total of 51 ensemble members; for pressure level variables where the volume of all the members would be greater than 32GB, 2 tar files should be created for each variable (see yellow boxes below)
Provider | Type | Level | Realm | Time Resolution | Tar file name | Example variable | Number of ensemble members | Estimated tar file size (GB) | Number of variables of this type/level/realm |
---|---|---|---|---|---|---|---|---|---|
Meteo France | Forecast | surface | atmos | 6hr | lfpw_System6-v20170501_forecast_S1993100100_atmos_6hr_surface_tas_n1-n51.tar | tas | 51 | 6.1 | 6 |
Meteo France | Forecast | surface | atmos | day | lfpw_System6-v20170501_forecast_S1993100100_atmos_day_surface_lwee_n1-n51.tar | lwee | 51 | 2 | 16 |
Meteo France | Forecast | surface | land | day | lfpw_System6-v20170501_forecast_S1993100100_land_day_surface_mrlsl_n1-n51.tar | mrlsl | 51 | 0.8 | 6 |
Meteo France | Forecast | pressure | atmos | 12hr | lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar | va | 25 | 23.325 | 5 |
Meteo France | Forecast | pressure | atmos | 12hr | lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n26-n51.tar | va | 26 | 23.325 | 5 |
Meteo France | Forecast | surface | ocean | 6 hr | lfpw_System6-v20170501_forecast_S1993100100_ocean_6hr_surface_tso_n1-n51.tar | tso | 51 | 4.3 | 1 |
Meteo France | Forecast | surface | seaIce | day | lfpw_System6-v20170501_forecast_S1993100100_seaIce_day_surface_sic_n1-n51.tar | sic | 51 | 0.4 | 1 |
So the forecast pressure level data for va would be sent in 2 tar files, one containing 25 members, and the other containing 26 members (lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n1-n25.tar and lfpw_System6-v20170501_forecast_S1993100100_atmos_12hr_pressure_va_n26-n51.tar ).