I am using the API to download model-level data and would like to programmatically save to separate files (e.g., one for each month). When I submit a text file containing multiple requests, however, only the first one is processed. Is there a trick for submitting multiple requests simultaneously in this way, or must a different approach be taken altogether?


Thank you,

Colin Raymond

2 Comments

  1. Milana Vuckovic

    Hi,
    According to these CDS limits, one user can have only one request hitting the ECMWF archive at one time and there are total 50 slots for all the users. Seeing that you are asking about model level data which is stored on the ECMWF MARS archive, I'd guess in this case, the answer is no. 
    But it should be possible for other datasets that are on CDS disks.

    In the MARS archive where ERA5 data on model levels is stored, 1 month of ERA5 ML data is approximately on one tape, so the most optimal way to get the data is to get most you can in one request from one tape, in this case one month.
    I hope this helps (smile)
    Milana

  2. Ariel Goerl

    Hi Milana,

    Thank you for your reply. I am indeed pulling monthly ML data. Although it is possible to have only one request actually running at a time, I have found a solution to nonetheless speed up the retrieval process by simply putting multiple complete requests into a single text file (rather than combining them together in any way):

    #!/usr/bin/env python3
    import cdsapi
    c = cdsapi.Client()
    c.retrieve('reanalysis-era5-complete', {
    myrequest1
    }, 'myfile1.grib')

    #!/usr/bin/env python3
    import cdsapi
    c = cdsapi.Client()
    c.retrieve('reanalysis-era5-complete', {
    myrequest2
    }, 'myfile2.grib')

    and so on. Submitting multiple such text files (e.g. for different years), I find that the data is able to be retrieved simultaneously and prepared such that the downloads happen in immediate sequence (even though all but one state 'request queued', retrieval from the archive for all of them must be going on in the background). Quite a simple solution, but it took a fair bit of playing around to reach it!

    With best regards,

    Colin

Feedback: C3S User Satisfaction Survey - CAMS User Satisfaction Survey

Web: C3S Help and Support - CAMS Help and Support