arctichoke.dataset

Submodules

• arctichoke.dataset.date_type
• arctichoke.dataset.example_dataset
• arctichoke.dataset.field_mean
• arctichoke.dataset.get_min_max
• arctichoke.dataset.get_variable
• arctichoke.dataset.grid_type
• arctichoke.dataset.latlon_type
• arctichoke.dataset.trim_dataset

Attributes

cdo
meta_vars
cdo
meta_vars

Functions

get_date_type(dataset)	Get the data type of the dates of a dataset.
make_example_dataset([save_as, n, offset, ...])	Create an example dataset for testing.
get_field_mean(dataset[, save_as])	Get the field mean of the dataset.
get_variable_name(dataset)	Get the variable name of the dataset.
get_min_max(dataset[, var])	Get the minimum / maximum of the dataset.
get_variable_name(dataset)	Get the variable name of the dataset.
get_latlon_names(dataset)	Get the latitude and longitude variable names of the dataset.
determine_lon_type(lon_min, lon_max)	Determine the longitude type based on the minimum and maximum values.
get_lon_type(dataset)	Get the longitude type of the dataset.
get_grid_type(dataset)	Get the grid type of the dataset.
get_latlon_names(dataset)	Get the latitude and longitude variable names of the dataset.
get_lon_type(dataset)	Get the longitude type of the dataset.
trim_latlon(dataset[, map_bbox, precise_trim, ...])	Trim the given dataset.
trim_files(files_to_trim[, name_prefix, ...])	Trim the specified files and save them as new files.

Package Contents

arctichoke.dataset.get_date_type(dataset: str, xarray.DataArray, xarray.Dataset)

Get the data type of the dates of a dataset.

Determine the data type of the dates, the time dimension, of the given dataset, if applicable.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to get the date type.

Returns:

date_dtype – The data type of the dates in the dataset as a string.

Return type:

str

Examples

>>> from arctichoke.verify.get_date_type import get_date_type
>>> get_date_type()

arctichoke.dataset.make_example_dataset(save_as: str = None, n: int = 10, offset: int, float = 0, test_var_name: str = 'test_var', time_axis: bool, int = False, overwrite: bool = True)

Create an example dataset for testing.

Construct a dataset such that it is minimal in size yet contains all notable features of the datasets of HighResMIP data, and save it to a netCDF file.

Parameters:

• save_as (str, None, optional) – The absolute file path to which to save the example dataset. Default is None, which doesn’t save the dataset to a file.

• n (int, optional) – The number of values in each dimension. Default is 10.

• offset (int, float, optional) – A constant offset value to add to every value in the dataset. Default is 0.

• test_var_name (str, optional) – The name to give the test variable. Default is test_var.

• time_axis (bool, int, optional) – Whether to include a time axis in the example dataset. If an integer is given, that will be used as the year for the datetimes. Default is False.

• overwrite (bool, optional) – Whether to overwrite an existing file at the given filepath in save_as. Default is True.

Returns:

example_dataset – A list, sorted alphabetically, of the names of the available models.

Return type:

xr.Dataset

Examples

>>> from arctichoke.dataset.example_dataset import make_example_dataset
>>> dataset = make_example_dataset(n=3, time_axis=True)
>>> dataset['test_var'].values
array([[[0., 1., 2.],
        [3., 4., 5.],
        [6., 7., 8.]],

[[0., 1., 2.],

[3., 4., 5.], [6., 7., 8.]]])

arctichoke.dataset.cdo

arctichoke.dataset.get_field_mean(dataset: str, [str], xarray.DataArray, xarray.Dataset, save_as: str = None, **kwargs)

Get the field mean of the dataset.

Use the cdo function fldmean to take the field mean (the mean over the geographic area) of the given dataset.

Parameters:

• dataset (str, list of str, xarray.DataArray, xarray.Dataset) – The dataset of which to take the field mean.

• save_as (str, None, optional) – The file name to pass to cdo.fldmean(output=save_as). Default is None, which doesn’t save the dataset to a file.

• **kwargs – Keyword arguments to pass to cdo.fldmean().

Returns:

fldmean_xr – A dataset of the field mean of the input data.

Return type:

xarray.Dataset

Examples

>>> from arctichoke.dataset.field_mean import get_field_mean
>>> fldmean_xr = get_field_mean('data/NWP_cdo_CLI_areacello_Ofx_EC-Earth3P-HR_highres-future_r2i1p2f1_gn.nc')
>>> fldmean_xr['areacello'].values[0]
(get_field_mean) `save_as`: None
array([1.3731426e+08], dtype=float32)

arctichoke.dataset.get_variable_name(dataset: str, xarray.DataArray, xarray.Dataset)

Get the variable name of the dataset.

Opens the given dataset, checks the data variable attributes, and returns the variable name.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the variable name.

Returns:

var_name – The name of the variable

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import get_variable_name

arctichoke.dataset.get_min_max(dataset: str, xarray.DataArray, xarray.Dataset, var: str = None)

Get the minimum / maximum of the dataset.

Opens the given dataset, finds the given variable if xr.Dataset, returns the minimum / maximum values of that data.

Parameters:

• dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the minimum / maximum values.

• var (str, None, optional) – The variable in dataset for which to find the minimum / maximum. Default is None.

Returns:

• var_min (int, float) – The minimum value of the data.

• var_max (int, float) – The maximum value of the data.

Examples

>>> from arctichoke.dataset.example_dataset import make_example_dataset
>>> from arctichoke.dataset.get_min_max import get_min_max
>>> dataset = make_example_dataset(n=3)
>>> min, max = get_min_max(dataset, var='test_var')
>>> print('min:',min,'max:',max)
min: 0.0 max: 8.0

arctichoke.dataset.meta_vars = ['time_bnds', 'vertices_latitude', 'vertices_longitude', 'latitude_bnds', 'longitude_bnds',...

arctichoke.dataset.get_variable_name(dataset: str, xarray.DataArray, xarray.Dataset)

Get the variable name of the dataset.

Opens the given dataset, checks the data variable attributes, and returns the variable name.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the variable name.

Returns:

var_name – The name of the variable

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import get_variable_name

arctichoke.dataset.get_latlon_names(dataset: str, xarray.DataArray, xarray.Dataset)

Get the latitude and longitude variable names of the dataset.

Opens the given dataset, checks the coordinates, and determines the name of the latitude and longitude variables. This will be either lat/lon or latitude/longitude.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the latitude and longitude names.

Returns:

• lat_var (str) – The name of latitude variable in the dataset. This will be either lat or latitude.

• lon_var (str) – The name of longitude variable in the dataset. This will be either lon or longitude.

Examples

>>> from arctichoke.dataset.grid_type import get_latlon_names
>>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/siconc/gn/v20170928/siconc_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc')
('lat', 'lon')
>>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/sithick/gn/v20170928/sithick_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc')
('latitude', 'longitude')

arctichoke.dataset.determine_lon_type(lon_min: int, float, lon_max: int, float)

Determine the longitude type based on the minimum and maximum values.

Given the minimum and maximum longitude values of a dataset, tries to determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other.

Parameters:

• lon_min (int, float) – The minimum longitude value.

• lon_max (int, float) – The maximum longitude value.

Returns:

lon_type – The type of longitude that the dataset has which will be ‘PM_centered’, ‘IDL_centered’, or ‘other’.

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import determine_lon_type
>>> determine_lon_type(lon_min = 0, lon_max = 360)
PM_centered
>>> determine_lon_type(lon_min = -180, lon_max = 180)
IDL_centered

arctichoke.dataset.get_lon_type(dataset: str, xarray.DataArray, xarray.Dataset)

Get the longitude type of the dataset.

Opens the given dataset, checks the coordinates, and determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the longitude type.

Returns:

lon_type – The type of longitude that the dataset has which will be ‘PM_centered’, ‘IDL_centered’, or ‘other’.

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import get_lon_type
>>> get_lon_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc')
IDL_centered

arctichoke.dataset.cdo

arctichoke.dataset.meta_vars = ['time_bnds', 'vertices_latitude', 'vertices_longitude', 'latitude_bnds', 'longitude_bnds',...

arctichoke.dataset.get_grid_type(dataset: str, xarray.DataArray, xarray.Dataset)

Get the grid type of the dataset.

Opens the given datasets, checks the dimensions, and determines the type of grid: regular, irregular, or other.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the grid type.

Returns:

grid_type – The type of grid that the dataset has which will be ‘regular’, ‘irregular’, or ‘other’.

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import get_grid_type
>>> get_grid_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc')
irregular

arctichoke.dataset.get_latlon_names(dataset: str, xarray.DataArray, xarray.Dataset)

Get the latitude and longitude variable names of the dataset.

Opens the given dataset, checks the coordinates, and determines the name of the latitude and longitude variables. This will be either lat/lon or latitude/longitude.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the latitude and longitude names.

Returns:

• lat_var (str) – The name of latitude variable in the dataset. This will be either lat or latitude.

• lon_var (str) – The name of longitude variable in the dataset. This will be either lon or longitude.

Examples

>>> from arctichoke.dataset.grid_type import get_latlon_names
>>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/siconc/gn/v20170928/siconc_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc')
('lat', 'lon')
>>> get_latlon_names('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-MM/hist-1950/r1i1p1f1/SImon/sithick/gn/v20170928/sithick_SImon_HadGEM3-GC31-MM_hist-1950_r1i1p1f1_gn_201401-201412.nc')
('latitude', 'longitude')

arctichoke.dataset.get_lon_type(dataset: str, xarray.DataArray, xarray.Dataset)

Get the longitude type of the dataset.

Opens the given dataset, checks the coordinates, and determines the type of longitude: Prime Meridian centered (0 to 360), International Date Line centered (-180 to 180), or other.

Parameters:

dataset (str, xarray.DataArray, xarray.Dataset) – The dataset for which to determine the longitude type.

Returns:

lon_type – The type of longitude that the dataset has which will be ‘PM_centered’, ‘IDL_centered’, or ‘other’.

Return type:

str

Examples

>>> from arctichoke.dataset.grid_type import get_lon_type
>>> get_lon_type('/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siconc/gn/v20181212/siconc_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc')
IDL_centered

arctichoke.dataset.trim_latlon(dataset: str, xarray.Dataset, xarray.DataArray, map_bbox: [float, float, float, float] = sps.NWP_BBOX, precise_trim: bool = False, save_as: str = None, verbose: bool = False)

Trim the given dataset.

Select only the data within the given bounding box of latitude and longitude coordinates from the given dataset.

Parameters:

• dataset (str, xarray.Dataset, xarray.DataArray) – The dataset to plot.

• map_bbox (Array of float, optional) –

  An array of coordinates defining the bounding box of the map in the following format:

  • [LAT_MAX, LAT_MIN, LON_MAX, LON_MIN]

  Default is arctichoke.params.latlon_params.NWP_BBOX.

• precise_trim (bool, optional) – Whether to precisely trim an irregular grid to the bounding box, making all values outside the bounding box null. Default is False.

• save_as (str, None, optional) – The file name to which to save the modified dataset. Default is None, which doesn’t save the dataset to a file.

• verbose (bool, optional) – Whether to verbosely output information as the function executes. Default is False.

Returns:

xr_data_trimmed – The dataset trimmed to the latitude and longitude ranges provided.

Return type:

xarray.Dataset

Examples

>>>

arctichoke.dataset.trim_files(files_to_trim: [str], name_prefix: str = 'trim_', use_cdo_python: bool = True, overwrite: bool = False, **kwargs)

Trim the specified files and save them as new files.

For each given file, load the data, trim the dataset, then save that trimmed dataset as a new file in the same location with a new filename.

Parameters:

• files_to_trim (List of str) – A list of paths of the data files to trim.

• name_prefix (str, optional) – The prefix to be prepended to each file name when saving. Default is trim_.

• use_cdo_python (bool, optional) – Whether to tell trim_latlon() to use the Python implementation of cdo or use subprocess to run a shell command. Default is True which uses the Python implementation of cdo.

• overwrite (bool, optional) – Whether to overwrite an existing file if it exists. Default is False.

• **kwargs – Keyword arguments to pass to trim_latlon().

Return type:

None

Examples

>>> from arctichoke.path.find_data import list_variable_files
>>> list_of_files = list_variable_files('EC-Earth3P-HR', 'siage', variant_label='r3i1p2f1')
>>> from arctichoke.dataset.trim_dataset import trim_files
>>> trim_files(list_of_files)
(trim_files) `name_prefix`: trim_NWP_
(trim_latlon) `save_as`: /arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage/gn/v20190214/trim_NWP_siage_SImon_EC-Earth3P-HR_hist-1950_r3i1p2f1_gn_195001-195012.nc
...
(trim_latlon) `save_as`: /arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage/gn/v20190214/trim_NWP_siage_SImon_EC-Earth3P-HR_hist-1950_r3i1p2f1_gn_201401-201412.nc