Hi all,

  I am recently trying to download ERA5 daily data using python script (please see the attached script for evaporation). I have successfully downloaded the data for like 20 years. But for some reason, the script doesn't work since Sep 26. The request is just queued and never completed. I also cannot find my job id on the web API job list webpage (https://apps.ecmwf.int/webmars/joblist/). My user ID is easy. The output log is as follows:

nohup: ignoring input

2021-09-28 14:56:57,676 INFO Welcome to the CDS

2021-09-28 14:56:57,676 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/tasks/services/tool/toolbox/orchestrator/workflow/clientid-17165db8fad44f37b7e19188f07c35d5

2021-09-28 14:56:57,714 INFO Request is queued.

My python version is Python 3.9.5. The platform is x86_64 GNU/Linux.

Could you please help me diagnose where the problem is?

Many thanks,

zhen

 evap.py

9 Comments

  1. Georgy Ayzel

    Hi Zhen,

    Unfortunately, I could not help you, but I also experienced a similar problem with retrieving ERA5T daily precipitation and temperature data from CDS since 28 Sept.

    I hope that that could be resolved soon as our operational runoff forecasting service depends from that data.


    All the best,

    Georgy

  2. Rob Donovan

    Since around this date, I've also not been able to download any data from the standard web page.

    This page: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land?tab=form

    The requests just go into my queue and then remains there as 'queued' and just count up the hours.

    I'm new here, and only download a few data sets, so I've no idea if this is 'normal' or something needs looking at.

  3. Milana Vuckovic

    I have been experiencing the same problem both yesterday and today trying to get ERA5-Land data. First I tried using the CDSAPI and after it was queued for a while I tried a small one using the web application.
    This morning I saw that both were in the queue for 18 hours. Today I tried again and it is still queuing for a few hours now.

    Edit: Just to clarify, I could download both data after 18 hours. Both from web and from cdsapi request. The ones I left yesterday in the loop, one went through, now it's on 2nd (3hrs in the queue).. 

    The queues for ERA5-Land seems to be comparable with the ones for ERA5-complete in the last few days, but I thought ERA5-Land is on CDS disks as it usually doesn't take this long...

  4. Rob Donovan

    FYI, if you leave them 'Queued' then eventually they do get processed. 

    I had 2 test ones, one took 20hours and the other 24.

    I also noticed, if the API query is on the reanalysis-era5-single-levels DB, then it works immediatly, but if you use the reanalysis-era5-land one, that is the one that takes a very long time.

    HTHs.

  5. Zhen Liu

    Hi Rob,

      Thanks for your positive feedback. Indeed, I just checked my datastore, the data I requested is successfully downloaded now. AMAZING~ What can I say!


    Cheers,

    zhen

  6. Cristina Vrinceanu

    Hi all,


    I have a similar issue. I see that they eventually get through, but I can't afford days for a download as I am running tests.

    I have a dataset with many satellite images that I am trying to match with the ERA5, hence right now just trying my python script.

    Did anyone hear of any issues or managed to get support?


    Cheers,

    Cristina

  7. Kevin Marsh

    Hi Cristina,

    There is currently an issue with the CDS ERA5 daily application. Please see this page in our "Announcements" forum for details

    Daily statistics calculated from ERA5 data application temporarily unavailable

    Thanks,

    Kevin


    1. Cristina Vrinceanu

      Hi Kevin,


      Thank you for your reply!

      It eventually went through. 1.8KB in 1h and 27 min, but I got my tiny subset.

      My issue was regarding the queuing process, in which a very simple request (1 date, 1 hour, very tiny area to be subsetted, 2 variables) has taken too long for repeated trials, should the code fail.


      I will try not to flood the server with too many requests, but does anyone know if there is a limit? Could not see any in the documents or perhaps I should open my eyes better.


      Thanks,

      Cristina

  8. Kevin Marsh

    Hi Cristina,

    The daily application is back on line now. You need to logon to the CDS before you can use the Daily application web interface, and this should help the performance of the application for users.

    Thanks,

    Kevin


Feedback: C3S User Satisfaction Survey - CAMS User Satisfaction Survey

Web: C3S Help and Support - CAMS Help and Support