arctichoke.path.variable_paths

Functions

`list_available_variables`(source_id[, experiment_id, ...])	List the names of the variables available for the specified model.
`get_variable_path`(source_id, variable_id[, data_dir, ...])	Find the file path for the specified model and variable.
`list_variable_modifications`(variable_path[, ...])	List the modifications made to variable files in the specified path.

Module Contents

arctichoke.path.variable_paths.list_available_variables(source_id: str, experiment_id: str = None, list_var_mods: bool = False, **kwargs)

List the names of the variables available for the specified model.

Search the data directory that has been populated by esgpull and return a list of all the available varibles, or variable_id’s for the specified model. This assumes the esgpull convention of subdirectories: data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id.

Parameters:

source_id (str) – The name of the source ID (model) for which to find the variable paths.
experiment_id (str or None, optional) – The name of the experiment ID in which to search for available variables. If None is given, all available experiments are included. Default is None.
list_var_mods (bool, optional) – Whether to include information about modifications made to the datafiles. Default is False.
**kwargs – Keyword arguments to pass to get_model_path().

Returns:

avail_var_dict – A dictionary of experiment ID’s whose keys are dictionaries of variant labels whose keys are lists of available variables.

Return type:

dict

Examples

>>> from arctichoke.path.find_data import list_available_variables
>>> list_available_variables(source_id = 'HadGEM3-GC31-HM')
{'MOHC/HadGEM3-GC31-HM': {
    'control-1950': {
        'r1i1p1f1': ['areacello']},
    'highres-future': {
        'r1i1p1f1': ['areacello', 'siu', 'siv', 'sithick', 'siconc', 'siage'],
        'r1i3p1f1': ['siconc', 'siage', 'siv', 'sithick', 'siu']},
    'hist-1950': {
        'r1i1p1f1': ['areacello', 'siage', 'siv', 'siu', 'siconc', 'sithick'],
        'r1i3p1f1': ['siconc', 'sithick', 'siu', 'siage', 'siv']}},
'NERC/HadGEM3-GC31-HM': {
    'highres-future': {
        'r1i2p1f1': ['siv', 'siu', 'siconc', 'sithick', 'siage']},
    'hist-1950': {
        'r1i2p1f1': ['siconc', 'siu', 'sithick', 'siv', 'siage']}}}
>>> list_available_variables(source_id = 'HadGEM3-GC31-HM', experiment_id = 'hist-1950')
{'MOHC/HadGEM3-GC31-HM': {
    'hist-1950': {
        'r1i1p1f1': ['areacello', 'siage', 'siv', 'siu', 'siconc', 'sithick'],
        'r1i3p1f1': ['siconc', 'sithick', 'siu', 'siage', 'siv']}},
'NERC/HadGEM3-GC31-HM': {
    'hist-1950': {
        'r1i2p1f1': ['siconc', 'siu', 'sithick', 'siv', 'siage']}}}

arctichoke.path.variable_paths.get_variable_path(source_id: str, variable_id: str, data_dir: str = '/arctichoke_data/bergybits/data', project: str = 'CMIP6', activity_id: str = 'HighResMIP', experiment_id: str = 'hist-1950', variant_label: str = None, **kwargs)

Find the file path for the specified model and variable.

Search the data directory that has been populated by esgpull for a model’s source_id and return the file path. This assumes the esgpull convention of subdirectories: data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id.

Parameters:

source_id (str) – The name of the source ID (model) for which to find the variable path.
variable_id (str) – The name of the variable ID for which to find the variable path.
data_dir (str, optional) – The absolute file path to the data directory. The esgpull convention means this should end in /data. Default is /arctichoke_data/bergybits/data.
project (str, optional) – The name of the project in which to search for available models. Default is CMIP6.
activity_id (str, optional) – The name of the activity ID in which to search for available models. Default is HighResMIP.
experiment_id (str, optional) – The name of the experiment ID in which to search for available variables. Default is hist-1950.
variant_label (str, None, optional) – The name of the variant label (ensemble member) for which to get the variable path. If None is given, the file path for the first variant with the variable is returned. Default is None.
**kwargs – Keyword arguments to pass to get_model_path().

Returns:

variable_path – The file path to the directory for the specified variable from the specified model.

Return type:

str

Examples

>>> from arctichoke.path.find_data import get_variable_path
>>> get_variable_path(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello')
'/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello'
>>> get_variable_path(source_id = 'EC-Earth3P-HR', variable_id = 'siage')
'/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage'
/workspace/src/arctichoke/path/find_data.py:396: UserWarning: (get_variable_path) More than one file path found: ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage']
Returning first result in list.
'/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage'
>>> get_variable_path(source_id = 'EC-Earth3P-HR', variable_id = 'siage', variant_label = 'r3i1p2f1')
'/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage'

arctichoke.path.variable_paths.list_variable_modifications(variable_path: str, list_filenames: bool = False, verbose: bool = False)

List the modifications made to variable files in the specified path.

Search the given directory of data files and return a dictionary indicating whether any modifications have been made and how many files in each modification category exist. This assumes modifications are made to files by adding a prefix to the file name.

Parameters:

variable_path (str) – The filepath to the variable in question. Expected format: data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id.
list_filenames (bool, optional) – Whether to list all the file names for each modification or to give the number of files. Default is False, which gives the number of files.
verbose (bool, optional) – Whether to output additional information while the function is running. Default is False.

Returns:

var_mod_dict – A dictionary of modification prefixes, the values of which are either the number of files under that modification or a list all the files under that modification.

Return type:

dict

Examples

>>> from arctichoke.path.find_data import list_variable_modifications
>>> list_variable_modifications(variable_path = '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/AWI/AWI-CM-1-1-HR/hist-1950/r1i1p1f2/Ofx/areacello', list_filenames = True)
{'': ['areacello_Ofx_AWI-CM-1-1-HR_hist-1950_r1i1p1f2_gn.nc']}
>>> list_variable_modifications(variable_path = '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siconc')
{'': 65, 'trim_NWP_': 65}