Good afternoon,


I am trying to download data from the CDS and I always have the following error: 401 Authorization Required

I can download these data from the web page but I would really like to automate it.


The error message
python3 request_from_web.py 
2022-12-15 14:38:50,636 INFO Welcome to the CDS
2022-12-15 14:38:50,636 INFO Sending request to https://ads.atmosphere.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels
Traceback (most recent call last):
  File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 427, in _api
    result.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://ads.atmosphere.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/localadmin/request_from_web.py", line 5, in <module>
    c.retrieve(
  File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 348, in retrieve
    result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
  File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 450, in _api
    raise Exception(error)
Exception: <html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.20.2</center>
</body>
</html>


My request
import cdsapi

c = cdsapi.Client()

c.retrieve(
    'reanalysis-era5-pressure-levels',
    {
        'product_type': 'reanalysis',
        'variable': [
            'fraction_of_cloud_cover', 'geopotential', 'relative_humidity',
            'specific_humidity', 'temperature', 'u_component_of_wind',
            'v_component_of_wind', 'vertical_velocity',
        ],
        'pressure_level': [
            '300', '350', '400',
            '450', '500', '550',
            '600', '650', '700',
            '750', '775', '800',
            '825', '850', '875',
            '900', '925', '950',
            '975', '1000',
        ],
        'year': '2013',
        'month': '01',
        'day': [
            '01', '02', '03',
            '04', '05', '06',
            '07', '08', '09',
            '10', '11', '12',
            '13', '14', '15',
            '16', '17', '18',
            '19', '20', '21',
            '22', '23', '24',
            '25', '26', '27',
            '28', '29', '30',
            '31',
        ],
        'time': [
            '00:00', '01:00', '02:00',
            '03:00', '04:00', '05:00',
            '06:00', '07:00', '08:00',
            '09:00', '10:00', '11:00',
            '12:00', '13:00', '14:00',
            '15:00', '16:00', '17:00',
            '18:00', '19:00', '20:00',
            '21:00', '22:00', '23:00',
        ],
        'format': 'netcdf',
        'area': [
            48.5, 6, 46.5,
            10,
        ],
    },
    'download.nc')

My file $HOME/.cdsapirc 
url: https://cds.climate.copernicus.eu/api/v2
key: 162846:myapikey

url: https://ads.atmosphere.copernicus.eu/api/v2
key: 3250:myapikey



I've accepted the terms of Use 


I am using a Linux virtual machine behind a corporate proxy (sad)

I am using Python 3.10 and I'ive update the toolbox


Thanks for your help !

Maxime


4 Comments

  1. Hi Maxime,

    the url for ERA5 data is the CDS one not the ADS.

    Please have a look at last section in this page for details on how to handle two urls and keys: https://ads.atmosphere.copernicus.eu/api-how-to

    The most common CDS API issues are documented in the following Troubleshooting page: https://confluence.ecmwf.int/x/RKOpD

    Thanks

    1. Hello, Thanks for your rapid answer,


      I have the same problem with cds


      localadmin@lab-vm:~$ python3 request_from_web.py 
      2022-12-15 15:53:16,957 INFO Welcome to the CDS
      2022-12-15 15:53:16,957 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels
      Traceback (most recent call last):
        File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 427, in _api
          result.raise_for_status()
        File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
          raise HTTPError(http_error_msg, response=self)
      requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/home/localadmin/request_from_web.py", line 5, in <module>
          c.retrieve(
        File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 348, in retrieve
          result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
        File "/home/localadmin/.local/lib/python3.10/site-packages/cdsapi/api.py", line 450, in _api
          raise Exception(error)
      Exception: <html>
      <head><title>401 Authorization Required</title></head>
      <body>
      <center><h1>401 Authorization Required</h1></center>
      <hr><center>nginx/1.20.2</center>
      </body>
      </html>

  2. The same request and configuration is working on a normal computer (without virtual machine and proxy)

  3. Hi Maxime,

    if the same CDS  request is working ok on a computer which is not a VM/using a proxy, it suggests that the issue is with the VM/corporate proxy - maybe you can talk to the admin of those to investigate further (i.e that the required firewall ports are open)?

    Thanks,

    Kevin