arctichoke.path.variable_paths
==============================

.. py:module:: arctichoke.path.variable_paths


Functions
---------

.. autoapisummary::

   arctichoke.path.variable_paths.list_available_variables
   arctichoke.path.variable_paths.get_variable_path
   arctichoke.path.variable_paths.list_variable_modifications


Module Contents
---------------

.. py:function:: list_available_variables(source_id: str, experiment_id: str = None, list_var_mods: bool = False, **kwargs)

   List the names of the variables available for the specified model.

   Search the data directory that has been populated by `esgpull` and return a list of all the available varibles, or `variable_id`'s for the specified model.
   This assumes the `esgpull` convention of subdirectories: `data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id`.

   :param source_id: The name of the source ID (model) for which to find the variable paths.
   :type source_id: `str`
   :param experiment_id: The name of the experiment ID in which to search for available variables.
                         If `None` is given, all available experiments are included.
                         Default is `None`.
   :type experiment_id: `str` or `None`, optional
   :param list_var_mods: Whether to include information about modifications made to the datafiles.
                         Default is `False`.
   :type list_var_mods: `bool`, optional
   :param \*\*kwargs: Keyword arguments to pass to `get_model_path()`.

   :returns: **avail_var_dict** -- A dictionary of experiment ID's whose keys are dictionaries of variant labels whose keys are lists of available variables.
   :rtype: `dict`

   .. rubric:: Examples

   >>> from arctichoke.path.find_data import list_available_variables
   >>> list_available_variables(source_id = 'HadGEM3-GC31-HM')
   {'MOHC/HadGEM3-GC31-HM': {
       'control-1950': {
           'r1i1p1f1': ['areacello']},
       'highres-future': {
           'r1i1p1f1': ['areacello', 'siu', 'siv', 'sithick', 'siconc', 'siage'],
           'r1i3p1f1': ['siconc', 'siage', 'siv', 'sithick', 'siu']},
       'hist-1950': {
           'r1i1p1f1': ['areacello', 'siage', 'siv', 'siu', 'siconc', 'sithick'],
           'r1i3p1f1': ['siconc', 'sithick', 'siu', 'siage', 'siv']}},
   'NERC/HadGEM3-GC31-HM': {
       'highres-future': {
           'r1i2p1f1': ['siv', 'siu', 'siconc', 'sithick', 'siage']},
       'hist-1950': {
           'r1i2p1f1': ['siconc', 'siu', 'sithick', 'siv', 'siage']}}}
   >>> list_available_variables(source_id = 'HadGEM3-GC31-HM', experiment_id = 'hist-1950')
   {'MOHC/HadGEM3-GC31-HM': {
       'hist-1950': {
           'r1i1p1f1': ['areacello', 'siage', 'siv', 'siu', 'siconc', 'sithick'],
           'r1i3p1f1': ['siconc', 'sithick', 'siu', 'siage', 'siv']}},
   'NERC/HadGEM3-GC31-HM': {
       'hist-1950': {
           'r1i2p1f1': ['siconc', 'siu', 'sithick', 'siv', 'siage']}}}


.. py:function:: get_variable_path(source_id: str, variable_id: str, data_dir: str = '/arctichoke_data/bergybits/data', project: str = 'CMIP6', activity_id: str = 'HighResMIP', experiment_id: str = 'hist-1950', variant_label: str = None, **kwargs)

   Find the file path for the specified model and variable.

   Search the data directory that has been populated by `esgpull` for a model's `source_id` and return the file path.
   This assumes the `esgpull` convention of subdirectories: `data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id`.

   :param source_id: The name of the source ID (model) for which to find the variable path.
   :type source_id: `str`
   :param variable_id: The name of the variable ID for which to find the variable path.
   :type variable_id: `str`
   :param data_dir: The absolute file path to the data directory.
                    The `esgpull` convention means this should end in `/data`.
                    Default is `/arctichoke_data/bergybits/data`.
   :type data_dir: `str`, optional
   :param project: The name of the project in which to search for available models.
                   Default is `CMIP6`.
   :type project: `str`, optional
   :param activity_id: The name of the activity ID in which to search for available models.
                       Default is `HighResMIP`.
   :type activity_id: `str`, optional
   :param experiment_id: The name of the experiment ID in which to search for available variables.
                         Default is `hist-1950`.
   :type experiment_id: `str`, optional
   :param variant_label: The name of the variant label (ensemble member) for which to get the variable path.
                         If `None` is given, the file path for the first variant with the variable is returned.
                         Default is `None`.
   :type variant_label: `str`, `None`, optional
   :param \*\*kwargs: Keyword arguments to pass to `get_model_path()`.

   :returns: **variable_path** -- The file path to the directory for the specified variable from the specified model.
   :rtype: `str`

   .. rubric:: Examples

   >>> from arctichoke.path.find_data import get_variable_path
   >>> get_variable_path(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello')
   '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello'
   >>> get_variable_path(source_id = 'EC-Earth3P-HR', variable_id = 'siage')
   '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage'
   /workspace/src/arctichoke/path/find_data.py:396: UserWarning: (get_variable_path) More than one file path found: ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage']
   Returning first result in list.
   '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage'
   >>> get_variable_path(source_id = 'EC-Earth3P-HR', variable_id = 'siage', variant_label = 'r3i1p2f1')
   '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage'


.. py:function:: list_variable_modifications(variable_path: str, list_filenames: bool = False, verbose: bool = False)

   List the modifications made to variable files in the specified path.

   Search the given directory of data files and return a dictionary indicating whether any modifications have been made and how many files in each modification category exist.
   This assumes modifications are made to files by adding a prefix to the file name.

   :param variable_path: The filepath to the variable in question.
                         Expected format: `data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id`.
   :type variable_path: `str`
   :param list_filenames: Whether to list all the file names for each modification or to give the number of files.
                          Default is `False`, which gives the number of files.
   :type list_filenames: `bool`, optional
   :param verbose: Whether to output additional information while the function is running.
                   Default is `False`.
   :type verbose: `bool`, optional

   :returns: **var_mod_dict** -- A dictionary of modification prefixes, the values of which are either the number of files under that modification or a list all the files under that modification.
   :rtype: `dict`

   .. rubric:: Examples

   >>> from arctichoke.path.find_data import list_variable_modifications
   >>> list_variable_modifications(variable_path = '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/AWI/AWI-CM-1-1-HR/hist-1950/r1i1p1f2/Ofx/areacello', list_filenames = True)
   {'': ['areacello_Ofx_AWI-CM-1-1-HR_hist-1950_r1i1p1f2_gn.nc']}
   >>> list_variable_modifications(variable_path = '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siconc')
   {'': 65, 'trim_NWP_': 65}