We recently implemented a simple retrieval application to retrieve the Albedo product from the CDS
Here is the link to the CDS app
https://cds.climate.copernicus.eu/toolbox-editor/3086/retrieve_albedo_product_vgt
Where running the application, we encountered a memory issue (see the console log).
Traceback (most recent call last): File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 49, in handle_request result = cached(context.method, proc, context, context.args, context.kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached result = proc(context, *context.args, **context.kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 118, in __call__ return p(*args, **kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 59, in __call__ return self.proc(context, *args, **kwargs) File "/home/cds/cdsservices/services/retrieve.py", line 191, in execute time_dimensions=time_dimensions, read_attributes=read_attributes File "/opt/cdstoolbox/cdscompute/cdscompute/context.py", line 270, in call return c.call(service, *args, **kwargs).value File "/opt/cdstoolbox/cdsworkflows/cdsworkflows/future.py", line 76, in value raise self._result cdsworkflows.error.ClientError: {'traceback': 'Traceback (most recent call last): File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 49, in handle_request result = cached(context.method, proc, context, context.args, context.kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached result = proc(context, *context.args, **context.kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 118, in __call__ return p(*args, **kwargs) File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 59, in __call__ return self.proc(context, *args, **kwargs) File "/home/cds/cdsservices/services/cdm_translate.py", line 61, in execute retrieve_kwargs={'time_dimensions': time_dimensions, 'read_attributes': read_attributes} File "/opt/cdstoolbox/cdscdm/cdscdm/gridded.py", line 710, in open_dataset datasets += [xr.concat(variables[var], dim='time').sortby('time')] File "/usr/local/lib/python3.6/site-packages/xarray/core/dataset.py", line 5322, in sortby return aligned_self.isel(**indices) File "/usr/local/lib/python3.6/site-packages/xarray/core/dataset.py", line 1969, in isel var_value = var_value.isel(var_indexers) File "/usr/local/lib/python3.6/site-packages/xarray/core/variable.py", line 1118, in isel return self[key] File "/usr/local/lib/python3.6/site-packages/xarray/core/variable.py", line 767, in __getitem__ data = as_indexable(self._data)[indexer] File "/usr/local/lib/python3.6/site-packages/xarray/core/indexing.py", line 1294, in __getitem__ return array[key] MemoryError: Unable to allocate array with shape (3, 15680, 40320) and data type float32 '}
We tested the code also by adding the option grid and size in the retrieval dictionnary but that does not seem to solve the issue.
Could you please provide us a way/hints or tips to overcome this issue
Best Regards,
Iskander