Accessing data on NSIDC servers via OPeNDAP protocol

Author

Eli Holmes (NOAA)

Colab Badge JupyterHub Badge Download Badge

Overview

The National Snow and Ice Data Center servers require NASA Earthdata login authentication, but don’t require a EULA (as far as I know). However they have similar redirect issues as data that does require a EULA. The solution used for datasets that require a EULA seems to work for access.

I suggest going throuhg 3-nasa.ipynb before this tutorial.

Prerequisites

I assume you have a .netrc file at ~ (home). ~/.netrc should look just like this with your username and password.

machine urs.earthdata.nasa.gov
        login yourusername
        password yourpassword

Packages and setup

import xarray as xr
import pydap.client
# create an authenticated session
import earthaccess
earthaccess.login()
edl_session = earthaccess.get_requests_https_session()

National Snow and Ice Data Center

Their OPeNDAP server also uses NASA Earthdata login authentication. I struggled to find the right format for url that would open.

#url = "https://n5eil02u.ecs.nsidc.org/opendap/OTHR/NISE.004/2012.10.02/NISE_SSMISF17_20121002.HDFEOS"
url = "https://n5eil02u.ecs.nsidc.org/opendap/MOST/MOD10A1.006/2000.03.22/MOD10A1.A2000082.h00v08.006.2016061212345.hdf"
import pydap
from pydap.client import open_url
pydap_ds = open_url(url, session=edl_session, protocol="dap2")
# I don't know how to tell it to use dap2
ds = xr.open_dataset(url, session=edl_session, engine="pydap")
/srv/conda/envs/notebook/lib/python3.12/site-packages/pydap/handlers/dap.py:123: UserWarning: PyDAP was unable to determine the DAP protocol defaulting to DAP2 which is consider legacy and may result in slower responses. For more, see go to https://www.opendap.org/faq-page.
  _warnings.warn(
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'NDSI_Snow_Cover' has multiple fill values {np.int64(200), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'Snow_Albedo_Daily_Tile' has multiple fill values {np.int64(250), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
pydap_ds
<DatasetType with children 'NDSI_Snow_Cover', 'NDSI_Snow_Cover_Basic_QA', 'NDSI_Snow_Cover_Algorithm_Flags_QA', 'NDSI', 'Snow_Albedo_Daily_Tile', 'orbit_pnt', 'granule_pnt', 'Latitude', 'Longitude', 'YDim', 'XDim', 'MOD_Grid_Snow_500m_eos_cf_projection'>
%%time
# this works but odd errors
store = xr.backends.PydapDataStore(pydap_ds)
ds = xr.open_dataset(store)
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'NDSI_Snow_Cover' has multiple fill values {np.int64(200), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'Snow_Albedo_Daily_Tile' has multiple fill values {np.int64(250), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
CPU times: user 31.2 ms, sys: 3.93 ms, total: 35.1 ms
Wall time: 4.09 s
%%time
url="https://n5eil02u.ecs.nsidc.org/opendap/MOST/MOD10A1.006/2000.03.22/MOD10A1.A2000082.h00v08.006.2016061212345.hdf"
ds = xr.open_dataset(url, engine="pydap", session=session)
/srv/conda/envs/notebook/lib/python3.12/site-packages/pydap/handlers/dap.py:123: UserWarning: PyDAP was unable to determine the DAP protocol defaulting to DAP2 which is consider legacy and may result in slower responses. For more, see go to https://www.opendap.org/faq-page.
  _warnings.warn(
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'NDSI_Snow_Cover' has multiple fill values {np.int64(200), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
/srv/conda/envs/notebook/lib/python3.12/site-packages/xarray/conventions.py:200: SerializationWarning: variable 'Snow_Albedo_Daily_Tile' has multiple fill values {np.int64(250), np.int64(255)} defined, decoding all values to NaN.
  var = coder.decode(var, name=name)
CPU times: user 55 ms, sys: 15.3 ms, total: 70.2 ms
Wall time: 1.09 s

Conclusion

Use token-based sessions to fix redirect errors.

References