arctichoke.path.file_lists
==========================

.. py:module:: arctichoke.path.file_lists


Functions
---------

.. autoapisummary::

   arctichoke.path.file_lists.list_variable_files
   arctichoke.path.file_lists.select_files_by_time


Module Contents
---------------

.. py:function:: list_variable_files(source_id: str, variable_id: str, with_modification: str = None, **kwargs)

   Get a list of data files for the specified variable for the specified model.

   Search for all the files for the given model and variable and return them in a list.
   This assumes the `esgpull` convention of subdirectories: `data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id/gn/version/data_file.nc`.

   :param source_id: The name of the source ID (model) for which to get the variable files.
   :type source_id: `str`
   :param variable_id: The name of the variable ID for which to get the variable files.
   :type variable_id: `str`
   :param with_modification: The prefix of a modification to the data files to find.
                             Ex: `trim_NWP_`.
                             If `None`, returns only the original files.
                             Default is `None`.
   :type with_modification: `str`, `None`, optional
   :param \*\*kwargs: Keyword arguments to pass to `get_variable_path()`.

   :returns: **data_filepaths** -- A list, sorted alphabetically, of the filepaths of the variable data files.
   :rtype: List of `str`

   .. rubric:: Examples

   >>> from arctichoke.path.find import list_variable_files
   >>> list_variable_files(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello')
   ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello/gn/v20190301/areacello_Ofx_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn.nc']
   >>> list_variable_files(source_id = 'EC-Earth3P-HR', variable_id = 'siage')
   /workspace/src/arctichoke/path/find_data.py:399: UserWarning: (get_variable_path) More than one file path found: ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage']
   Returning first result in list.
   ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage/gn/v20181212/siage_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_195001-195012.nc',
   ...
   '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage/gn/v20181212/siage_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc']


.. py:function:: select_files_by_time(data_filepaths: [str], start: (str, int), end: (str, int), test: bool = False)

   Filter the list to be only the files in the given date range.

   From the given list of datafiles, find the ones that are in between the given start and end dates and return those in a list.
   If the start and end are given as years, the returned list will include both the start and end years, if available.

   :param data_filepaths: A list, sorted alphabetically, of the filepaths of the variable data files.
   :type data_filepaths: List of `str`
   :param start: The start date of the time range, either a string in the format `YYYY-MM-DD` or an integer of a year.
   :type start: `str`, `int`
   :param end: The end date of the time range, either a string in the format `YYYY-MM-DD` or an integer of a year.
   :type end: `str`, `int`
   :param test: If `True`, the function skips verifying the `data_filepaths` exist for use in testing.
                Default is `False`.
   :type test: `bool`, optional

   :returns: **data_filepaths** -- A list, sorted alphabetically, of the filepaths in the date range.
   :rtype: List of `str`

   .. rubric:: Examples

   >>> from arctichoke.path.find import select_files_by_time
   >>> select_files_by_time(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello')
   ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello/gn/v20190301/areacello_Ofx_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn.nc']