UserError: Restricted access to ERA5T. Please, check that that your date selection is valid.

Hello Johannes, 
I'm currently having the same issue. Did you manage to solve it?

Me too, would like to know what's going on here.

Hi,

there is currently an investigation going on regarding the way the embargo period is applied to the latest ERA5T data. 

However it is unlikely that there will be updates until sometime next week.

Hi,

we seem to have the same issue, is there any updates on this?

Dear Michela,

Thanks so much for keeping track of these issues. Is there any update on this?
Thanks,

Johannes

As today, there still seem to be a problem and I get "UserError: Restricted access to ERA5T" with the following request in 'reanalysis-era5-single-levels' :

        'product_type':'reanalysis',

        'time'    : '12:00',

        'day'     : ['01','02','03','04','05','06','07','08','09','10','11','12','13','14','15',

                     '16','17','18','19','20','21','22','23','24','25','26','27','28','29','30','31'],

        'month'   : '06',

        'year'    : '2022',

        'nocache' : '47512',

        'variable': ['sea_ice_cover','sea_surface_temperature']

We have been getting the same error when fetching wind data more recent than the 22nd of May:

I'm also getting this error for the request:

{ "area": [8, -75, -40, -35],
  "day": "04",
  "format": "netcdf",
  "month": "06",
  "product_type": "reanalysis",
  "time": ["00:00", "01:00", "02:00", "03:00", "04:00", "05:00", "06:00", "07:00", "08:00", "09:00", "10:00", "11:00",
           "12:00", "13:00", "14:00", "15:00", "16:00", "17:00", "18:00", "19:00", "20:00", "21:00", "22:00", "23:00"],
  "variable": [
    "10m_u_component_of_wind",
    "10m_v_component_of_wind",
    "2m_dewpoint_temperature",
    "2m_temperature",
    "maximum_2m_temperature_since_previous_post_processing",
    "minimum_2m_temperature_since_previous_post_processing",
    "potential_evaporation",
    "soil_temperature_level_1",
    "surface_solar_radiation_downwards",
    "surface_thermal_radiation_downwards",
    "total_cloud_cover",
    "total_precipitation" ],
  "year": "2022" }

Is there any update on the issue?

There could be more than one issue here. If your API script attempts to retrieve say 31/04/2022 or 31/06/2022 which does not exist, then while in the past this would return no data anyway (i.e. no data returned on 31/04 or 31/06), since some changes were implemented last week, it would appear that if the data does not exist, the retrieval fails over with the error UserError: Restricted access to ERA5T (misleading I would say). 

Of course for those of you who are trying to retrieve whatever ERA5 data is available, this means that your API scripts no longer work at this point in time. You could try modifying your scripts to test this and share your results here.

Dear Michela et al,

Is there any update on this issue and/or its anticipated resolution date? I'm encountering this issue with my ETL for various ERA5 datasets.

Just want to highlight that the current behaviour is also problematic when querying multiple years of data ending in the current time frame. Queries for multiple years would select months which are by definition not available, resulting in the above error. By default one can now not query multiple years which includes months in the current time frame which have not yet come to pass.

In a practical sense the new behaviour limits use to the previous year, upon which the last year's data has to be parsed separately. This is real troubling for any workflow which requires multiple years upon until fairly recent time frames.

In short, this query will work as it will query a month (January) which is available in both years.

{
  "area": [
    1,
    -1,
    -1,
    1
  ],
  "dataset_short_name": "reanalysis-era5-single-levels",
  "day": "01",
  "format": "netcdf",
  "month": "01",
  "product_type": "reanalysis",
  "target": "download.nc",
  "time": "00:00",
  "variable": "2m_temperature",
  "year": [
    "2021",
    "2022"
  ]
}

In this query I want data for the whole year of 2021 but in doing so, any data queried for 2022 will also include "unavailable" months and therefore this query fails.

{
  "area": [
    1,
    -1,
    -1,
    1
  ],
  "dataset_short_name": "reanalysis-era5-single-levels",
  "day": "01",
  "format": "netcdf",
  "month": [
    "01",
    "02",
    "03",
    "04",
    "05",
    "06",
    "07",
    "08",
    "09",
    "10",
    "11",
    "12"
  ],
  "product_type": "reanalysis",
  "target": "download.nc",
  "time": "00:00",
  "variable": "2m_temperature",
  "year": [
    "2021",
    "2022"
  ]
}

We could all get around this new problem if there was something like a "date_begin" and "date_end" set of keywords, instead of the less specific "year", "month", "day", "time" keywords.  For example, something like "date_begin": "2001-01-01 00:00".  "date_end": "2022-05-31 23:00".  Is there any such functionality already?

Hi everyone

Just to mention I am having the same issue as the other users above (getting the 'Restricted' error, error code -2) when trying to download data for 2022. Thanks ECMWF for your support.

Milana, many thanks, these are good ideas.  In the first example: Is there any limit to the number of separate dates that can be specified in one request (other than the total data size of the request)?  If not, this looks like a good approach.  The middle example would work, but sends a separate request for each date, so maybe not the best for a long date range.  In the third example: I was not aware of the "/" syntax in the date value.  I will experiment with that.  Again, thanks!

Dear users,

an issue was recently discovered with the way the 5 day embargo period was being applied to requests for ERA5 data from the CDS. This has now been resolved, however a consequence of this is that now any request which includes embargoed data will fail with an error message:

Mars server task finished in error; UserError: Restricted access to ERA5T. Please, check that your date selection is valid. For more information, visit https://climate.copernicus.eu/climate-reanalysis [mars]; Error code is -2; Request failed; Some errors reported (last error -2)

In the past, such request would have completed without the error message. Consequently, users will have to amend their requests to exclude any embargoed dates (i.e. any dates less than 5 days before the current date).
This change has been implemented in order to ensure access is as it should be, and we apologise for any inconvenience caused.

Regards

ECMWF Support

Dear Michela,

According to my tests the issue still persists, without ECMWF Support offering a solution. In short, as outlined above, if people use the CDS dataset browser to format their queries for a date range which includes multiple years (including a recent month) downloads will still fail by default. No provisions are made in the user interface, or warnings provided.

More so, the solutions which were proposed by users and DO WORK using the date: "YYYY-MM-DD/YYYY-MM-DD" syntax were forcibly removed! I consider this a huge disservice to those looking for a solution. Basically, for multi-year queries including the current year (a common instance I would think) things are broken STILL.

Please provide an actionable solutions,

Best,

Koen

Is there any update on this issue?

Hi All,

I discovered a CDS API feature that allows the user to make requests over multiple months/years without triggering this error. It's possible to specify a specific start and end date for the request, and so long as the end date is more than 5 days in the past, the request should work. Here is some example syntax using the Python cdsapi:

import cdsapi
cds = cdsapi.Client()
cds.retrieve(
    'reanalysis-era5-single-levels', {
        "area": [
            40.75,
            -74.25,
            40.5,
            -74
        ],
        "variable": "surface_runoff",
        "product_type": "reanalysis",
        "date": "2021-06-24/2022-06-24",
        'time': [
            '00:00', '01:00', '02:00',
            '03:00', '04:00', '05:00',
            '06:00', '07:00', '08:00',
            '09:00', '10:00', '11:00',
            '12:00', '13:00', '14:00',
            '15:00', '16:00', '17:00',
            '18:00', '19:00', '20:00',
            '21:00', '22:00', '23:00',
        ],
        "format": "grib"
    }, 'download.grib')

This request will retrieve the ERA5 surface runoff data from the previous year for a small area as a GRIB. Hope this helps!

Hi all,

Is there a way to download all the latest available data automatically? I think now it is difficult because the last available date/hour is unknown (for operational purposes with the API). I'm using R.

Thank you in advance!
Diego

Hi all,

The latest ERA5T available data can be calculated as below:

• Take your local time, say CEST 23:09 on 18/8/22
• Work out UTC hours, 21:09 on 18/8/22
• Use the UTC time and date and minus 5 days, in this case, 21:09 on 13/8/22
• ERA5T data is available up to 21 UTC 13/8/22

To tell the difference between ERA5 and ERA5T, you can always download data in GRIB and check the value of 'expver'.

Hope this helps.

Xiaobo

Hello all,

Are there any plans to add a check if data is ERA5 or ERA5T via the API? I realize you can check by downloading a file and reading the GRIB headers, however, knowing before downloading would be very helpful and save a lot of time. Especially when downloading large files. 

Thanks,

Travis