arctichoke.dataset ================== .. py:module:: arctichoke.dataset Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/arctichoke/dataset/date_type/index /autoapi/arctichoke/dataset/example_dataset/index /autoapi/arctichoke/dataset/field_mean/index /autoapi/arctichoke/dataset/get_min_max/index /autoapi/arctichoke/dataset/get_variable/index /autoapi/arctichoke/dataset/grid_type/index /autoapi/arctichoke/dataset/latlon_type/index /autoapi/arctichoke/dataset/trim_dataset/index Attributes ---------- .. autoapisummary:: arctichoke.dataset.cdo arctichoke.dataset.meta_vars arctichoke.dataset.cdo arctichoke.dataset.meta_vars Functions --------- .. autoapisummary:: arctichoke.dataset.get_date_type arctichoke.dataset.make_example_dataset arctichoke.dataset.get_field_mean arctichoke.dataset.get_variable_name arctichoke.dataset.get_min_max arctichoke.dataset.get_variable_name arctichoke.dataset.get_latlon_names arctichoke.dataset.determine_lon_type arctichoke.dataset.get_lon_type arctichoke.dataset.get_grid_type arctichoke.dataset.get_latlon_names arctichoke.dataset.get_lon_type arctichoke.dataset.trim_latlon arctichoke.dataset.trim_files Package Contents ---------------- .. py:function:: get_date_type(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the data type of the dates of a dataset. Determine the data type of the dates, the `time` dimension, of the given dataset, if applicable. :param dataset: The dataset for which to get the date type. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **date_dtype** -- The data type of the dates in the dataset as a string. :rtype: `str` .. rubric:: Examples >>> from arctichoke.verify.get_date_type import get_date_type >>> get_date_type() .. py:function:: make_example_dataset(save_as: str = None, n: int = 10, offset: (int, float) = 0, test_var_name: str = 'test_var', time_axis: (bool, int) = False, overwrite: bool = True) Create an example dataset for testing. Construct a dataset such that it is minimal in size yet contains all notable features of the datasets of HighResMIP data, and save it to a netCDF file. :param save_as: The absolute file path to which to save the example dataset. Default is `None`, which doesn't save the dataset to a file. :type save_as: `str`, `None`, optional :param n: The number of values in each dimension. Default is `10`. :type n: `int`, optional :param offset: A constant offset value to add to every value in the dataset. Default is `0`. :type offset: `int`, `float`, optional :param test_var_name: The name to give the test variable. Default is `test_var`. :type test_var_name: `str`, optional :param time_axis: Whether to include a time axis in the example dataset. If an integer is given, that will be used as the year for the datetimes. Default is `False`. :type time_axis: `bool`, `int`, optional :param overwrite: Whether to overwrite an existing file at the given filepath in `save_as`. Default is `True`. :type overwrite: `bool`, optional :returns: **example_dataset** -- A list, sorted alphabetically, of the names of the available models. :rtype: `xr.Dataset` .. rubric:: Examples >>> from arctichoke.dataset.example_dataset import make_example_dataset >>> dataset = make_example_dataset(n=3, time_axis=True) >>> dataset['test_var'].values array([[[0., 1., 2.], [3., 4., 5.], [6., 7., 8.]], [[0., 1., 2.], [3., 4., 5.], [6., 7., 8.]]]) .. py:data:: cdo .. py:function:: get_field_mean(dataset: (str, [str], xarray.DataArray, xarray.Dataset), save_as: str = None, **kwargs) Get the field mean of the dataset. Use the `cdo` function `fldmean` to take the field mean (the mean over the geographic area) of the given dataset. :param dataset: The dataset of which to take the field mean. :type dataset: `str`, list of `str`, `xarray.DataArray`, `xarray.Dataset` :param save_as: The file name to pass to `cdo.fldmean(output=save_as)`. Default is `None`, which doesn't save the dataset to a file. :type save_as: `str`, `None`, optional :param \*\*kwargs: Keyword arguments to pass to `cdo.fldmean()`. :returns: **fldmean_xr** -- A dataset of the field mean of the input data. :rtype: `xarray.Dataset` .. rubric:: Examples >>> from arctichoke.dataset.field_mean import get_field_mean >>> fldmean_xr = get_field_mean('data/NWP_cdo_CLI_areacello_Ofx_EC-Earth3P-HR_highres-future_r2i1p2f1_gn.nc') >>> fldmean_xr['areacello'].values[0] (get_field_mean) `save_as`: None array([1.3731426e+08], dtype=float32) .. py:function:: get_variable_name(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the variable name of the dataset. Opens the given dataset, checks the data variable attributes, and returns the variable name. :param dataset: The dataset for which to determine the variable name. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **var_name** -- The name of the variable :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_variable_name .. py:function:: get_min_max(dataset: (str, xarray.DataArray, xarray.Dataset), var: str = None) Get the minimum / maximum of the dataset. Opens the given dataset, finds the given variable if `xr.Dataset`, returns the minimum / maximum values of that data. :param dataset: The dataset for which to determine the minimum / maximum values. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :param var: The variable in `dataset` for which to find the minimum / maximum. Default is `None`. :type var: `str`, `None`, optional :returns: * **var_min** (`int`, `float`) -- The minimum value of the data. * **var_max** (`int`, `float`) -- The maximum value of the data. .. rubric:: Examples >>> from arctichoke.dataset.example_dataset import make_example_dataset >>> from arctichoke.dataset.get_min_max import get_min_max >>> dataset = make_example_dataset(n=3) >>> min, max = get_min_max(dataset, var='test_var') >>> print('min:',min,'max:',max) min: 0.0 max: 8.0 .. py:data:: meta_vars :value: ['time_bnds', 'vertices_latitude', 'vertices_longitude', 'latitude_bnds', 'longitude_bnds',... .. py:function:: get_variable_name(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the variable name of the dataset. Opens the given dataset, checks the data variable attributes, and returns the variable name. :param dataset: The dataset for which to determine the variable name. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **var_name** -- The name of the variable :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_variable_name .. py:function:: get_latlon_names(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the latitude and longitude variable names of the dataset. Opens the given dataset, checks the coordinates, and determines the name of the latitude and longitude variables. This will be either `lat`/`lon` or `latitude`/`longitude`. :param dataset: The dataset for which to determine the latitude and longitude names. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: * **lat_var** (`str`) -- The name of latitude variable in the dataset. This will be either `lat` or `latitude`. * **lon_var** (`str`) -- The name of longitude variable in the dataset. This will be either `lon` or `longitude`. .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_latlon_names >>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/siconc/gn/v20170928/siconc_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc') ('lat', 'lon') >>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/sithick/gn/v20170928/sithick_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc') ('latitude', 'longitude') .. py:function:: determine_lon_type(lon_min: (int, float), lon_max: (int, float)) Determine the longitude type based on the minimum and maximum values. Given the minimum and maximum longitude values of a dataset, tries to determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other. :param lon_min: The minimum longitude value. :type lon_min: `int`, `float` :param lon_max: The maximum longitude value. :type lon_max: `int`, `float` :returns: **lon_type** -- The type of longitude that the dataset has which will be `'PM_centered'`, `'IDL_centered'`, or `'other'`. :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import determine_lon_type >>> determine_lon_type(lon_min = 0, lon_max = 360) PM_centered >>> determine_lon_type(lon_min = -180, lon_max = 180) IDL_centered .. py:function:: get_lon_type(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the longitude type of the dataset. Opens the given dataset, checks the coordinates, and determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other. :param dataset: The dataset for which to determine the longitude type. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **lon_type** -- The type of longitude that the dataset has which will be `'PM_centered'`, `'IDL_centered'`, or `'other'`. :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_lon_type >>> get_lon_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc') IDL_centered .. py:data:: cdo .. py:data:: meta_vars :value: ['time_bnds', 'vertices_latitude', 'vertices_longitude', 'latitude_bnds', 'longitude_bnds',... .. py:function:: get_grid_type(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the grid type of the dataset. Opens the given datasets, checks the dimensions, and determines the type of grid: regular, irregular, or other. :param dataset: The dataset for which to determine the grid type. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **grid_type** -- The type of grid that the dataset has which will be `'regular'`, `'irregular'`, or `'other'`. :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_grid_type >>> get_grid_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc') irregular .. py:function:: get_latlon_names(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the latitude and longitude variable names of the dataset. Opens the given dataset, checks the coordinates, and determines the name of the latitude and longitude variables. This will be either `lat`/`lon` or `latitude`/`longitude`. :param dataset: The dataset for which to determine the latitude and longitude names. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: * **lat_var** (`str`) -- The name of latitude variable in the dataset. This will be either `lat` or `latitude`. * **lon_var** (`str`) -- The name of longitude variable in the dataset. This will be either `lon` or `longitude`. .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_latlon_names >>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/siconc/gn/v20170928/siconc_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc') ('lat', 'lon') >>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/sithick/gn/v20170928/sithick_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc') ('latitude', 'longitude') .. py:function:: get_lon_type(dataset: (str, xarray.DataArray, xarray.Dataset)) Get the longitude type of the dataset. Opens the given dataset, checks the coordinates, and determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other. :param dataset: The dataset for which to determine the longitude type. :type dataset: `str`, `xarray.DataArray`, `xarray.Dataset` :returns: **lon_type** -- The type of longitude that the dataset has which will be `'PM_centered'`, `'IDL_centered'`, or `'other'`. :rtype: `str` .. rubric:: Examples >>> from arctichoke.dataset.grid_type import get_lon_type >>> get_lon_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc') IDL_centered .. py:function:: trim_latlon(dataset: (str, xarray.Dataset, xarray.DataArray), map_bbox: [float, float, float, float] = sps.NWP_BBOX, precise_trim: bool = False, save_as: str = None, verbose: bool = False) Trim the given dataset. Select only the data within the given bounding box of latitude and longitude coordinates from the given dataset. :param dataset: The dataset to plot. :type dataset: `str`, `xarray.Dataset`, `xarray.DataArray` :param map_bbox: An array of coordinates defining the bounding box of the map in the following format: - [LAT_MAX, LAT_MIN, LON_MAX, LON_MIN] Default is `arctichoke.params.latlon_params.NWP_BBOX`. :type map_bbox: Array of `float`, optional :param precise_trim: Whether to precisely trim an irregular grid to the bounding box, making all values outside the bounding box null. Default is `False`. :type precise_trim: `bool`, optional :param save_as: The file name to which to save the modified dataset. Default is `None`, which doesn't save the dataset to a file. :type save_as: `str`, `None`, optional :param verbose: Whether to verbosely output information as the function executes. Default is `False`. :type verbose: `bool`, optional :returns: **xr_data_trimmed** -- The dataset trimmed to the latitude and longitude ranges provided. :rtype: `xarray.Dataset` .. rubric:: Examples >>> .. py:function:: trim_files(files_to_trim: [str], name_prefix: str = 'trim_', use_cdo_python: bool = True, overwrite: bool = False, **kwargs) Trim the specified files and save them as new files. For each given file, load the data, trim the dataset, then save that trimmed dataset as a new file in the same location with a new filename. :param files_to_trim: A list of paths of the data files to trim. :type files_to_trim: List of `str` :param name_prefix: The prefix to be prepended to each file name when saving. Default is `trim_`. :type name_prefix: `str`, optional :param use_cdo_python: Whether to tell `trim_latlon()` to use the Python implementation of `cdo` or use `subprocess` to run a shell command. Default is `True` which uses the Python implementation of `cdo`. :type use_cdo_python: `bool`, optional :param overwrite: Whether to overwrite an existing file if it exists. Default is `False`. :type overwrite: `bool`, optional :param \*\*kwargs: Keyword arguments to pass to `trim_latlon()`. :rtype: None .. rubric:: Examples >>> from arctichoke.path.find_data import list_variable_files >>> list_of_files = list_variable_files('EC-Earth3P-HR', 'siage', variant_label='r3i1p2f1') >>> from arctichoke.dataset.trim_dataset import trim_files >>> trim_files(list_of_files) (trim_files) `name_prefix`: trim_NWP_ (trim_latlon) `save_as`: /arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage/gn/v20190214/trim_NWP_siage_SImon_EC-Earth3P-HR_hist-1950_r3i1p2f1_gn_195001-195012.nc ... (trim_latlon) `save_as`: /arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage/gn/v20190214/trim_NWP_siage_SImon_EC-Earth3P-HR_hist-1950_r3i1p2f1_gn_201401-201412.nc