arctichoke.path.file_lists
Functions
|
Get a list of data files for the specified variable for the specified model. |
|
Filter the list to be only the files in the given date range. |
Module Contents
- arctichoke.path.file_lists.list_variable_files(source_id: str, variable_id: str, with_modification: str = None, **kwargs)
Get a list of data files for the specified variable for the specified model.
Search for all the files for the given model and variable and return them in a list. This assumes the esgpull convention of subdirectories: data/project/activity_id/institution_id/source_id/experiment_id/variant_label/table_id/variable_id/gn/version/data_file.nc.
- Parameters:
source_id (str) – The name of the source ID (model) for which to get the variable files.
variable_id (str) – The name of the variable ID for which to get the variable files.
with_modification (str, None, optional) – The prefix of a modification to the data files to find. Ex: trim_NWP_. If None, returns only the original files. Default is None.
**kwargs – Keyword arguments to pass to get_variable_path().
- Returns:
data_filepaths – A list, sorted alphabetically, of the filepaths of the variable data files.
- Return type:
List of str
Examples
>>> from arctichoke.path.find import list_variable_files >>> list_variable_files(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello') ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello/gn/v20190301/areacello_Ofx_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn.nc'] >>> list_variable_files(source_id = 'EC-Earth3P-HR', variable_id = 'siage') /workspace/src/arctichoke/path/find_data.py:399: UserWarning: (get_variable_path) More than one file path found: ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r2i1p2f1/SImon/siage', '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r3i1p2f1/SImon/siage'] Returning first result in list. ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage/gn/v20181212/siage_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_195001-195012.nc', ... '/arctichoke_data/bergybits/data/CMIP6/HighResMIP/EC-Earth-Consortium/EC-Earth3P-HR/hist-1950/r1i1p2f1/SImon/siage/gn/v20181212/siage_SImon_EC-Earth3P-HR_hist-1950_r1i1p2f1_gn_201401-201412.nc']
- arctichoke.path.file_lists.select_files_by_time(data_filepaths: [str], start: str, int, end: str, int, test: bool = False)
Filter the list to be only the files in the given date range.
From the given list of datafiles, find the ones that are in between the given start and end dates and return those in a list. If the start and end are given as years, the returned list will include both the start and end years, if available.
- Parameters:
data_filepaths (List of str) – A list, sorted alphabetically, of the filepaths of the variable data files.
start (str, int) – The start date of the time range, either a string in the format YYYY-MM-DD or an integer of a year.
end (str, int) – The end date of the time range, either a string in the format YYYY-MM-DD or an integer of a year.
test (bool, optional) – If True, the function skips verifying the data_filepaths exist for use in testing. Default is False.
- Returns:
data_filepaths – A list, sorted alphabetically, of the filepaths in the date range.
- Return type:
List of str
Examples
>>> from arctichoke.path.find import select_files_by_time >>> select_files_by_time(source_id = 'HadGEM3-GC31-HM', variable_id = 'areacello') ['/arctichoke_data/bergybits/data/CMIP6/HighResMIP/MOHC/HadGEM3-GC31-HM/hist-1950/r1i1p1f1/Ofx/areacello/gn/v20190301/areacello_Ofx_HadGEM3-GC31-HM_hist-1950_r1i1p1f1_gn.nc']