Pi<vdZddlZddlmZmZddlmZmZddlm Z ddl m Z ddl m Z dd lmZdd lmZmZmZdd lmZdd lmZeeZGd deZ ddedeeeeefdee deee efdeeeefdeeeeff dZ ddedeeeefdee deee efdeeeeeff dZ! ddedeeeefdee deee efdeeeeefdeef dZ" ddedeedeeeeeeeeeeefffdee deee efdeeeefdeeeefdefdZ# ddedeedeeeeeeeeeeefffdee deee efdeeeefdeeeeffdZ$dS)zList and inspect datasets.N)MappingSequence)OptionalUnion)DownloadConfig) DownloadMode)StreamingDownloadManager) DatasetInfo)dataset_module_factoryget_dataset_builder_classload_dataset_builder) get_logger)VersionceZdZdS)SplitsNotFoundErrorN)__name__ __module__ __qualname__d/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/datasets/inspect.pyrr&sDrrpath data_filesdownload_config download_moderevisiontokenc bt}fd|DS)a6Get the meta information about a dataset, returned as a dict mapping config name to DatasetInfoDict. Args: path (`str`): path to the dataset repository. Can be either: - a local path to the dataset directory containing the data files, e.g. `'./dataset/squad'` - a dataset identifier on the Hugging Face Hub (list all available datasets and ids with [`huggingface_hub.list_datasets`]), e.g. `'rajpurkar/squad'`, `'nyu-mll/glue'` or``'openai/webtext'` revision (`Union[str, datasets.Version]`, *optional*): If specified, the dataset module will be loaded from the datasets repository at this version. By default: - it is set to the local version of the lib. - it will also try to load it from the main branch if it's not available at the local version of the lib. Specifying a version that is different from your local version of the lib might cause compatibility issues. download_config ([`DownloadConfig`], *optional*): Specific download configuration parameters. download_mode ([`DownloadMode`] or `str`, defaults to `REUSE_DATASET_IF_EXISTS`): Download/generate mode. data_files (`Union[Dict, List, str]`, *optional*): Defining the data_files of the dataset configuration. token (`str` or `bool`, *optional*): Optional string or boolean to use as Bearer token for remote files on the Datasets Hub. If `True`, or not specified, will get token from `"~/.huggingface"`. **config_kwargs (additional keyword arguments): Optional attributes for builder class which will override the attributes if supplied. Example: ```py >>> from datasets import get_dataset_infos >>> get_dataset_infos('cornell-movie-review-data/rotten_tomatoes') {'default': DatasetInfo(description="Movie Review Dataset. This is a dataset of containing 5,331 positive and 5,331 negative processed sentences from Rotten Tomatoes movie reviews...), ...} ``` )rrrrrrc:i|]}|td|dS))r config_namerrrrrr)get_dataset_config_info) .0r! config_kwargsrrrrrrs r z%get_dataset_infos..^sa     ,  #!+'         r)get_dataset_config_names)rrrrrrr$ config_namess``````` rget_dataset_infosr(*s}X, '# L          (    rc t|f||||d|}t|tj|}t |jp"|j d|j pdgS)aGGet the list of available config names for a particular dataset. Args: path (`str`): path to the dataset repository. Can be either: - a local path to the dataset directory containing the data files, e.g. `'./dataset/squad'` - a dataset identifier on the Hugging Face Hub (list all available datasets and ids with [`huggingface_hub.list_datasets`]), e.g. `'rajpurkar/squad'`, `'nyu-mll/glue'` or``'openai/webtext'` revision (`Union[str, datasets.Version]`, *optional*): If specified, the dataset module will be loaded from the datasets repository at this version. By default: - it is set to the local version of the lib. - it will also try to load it from the main branch if it's not available at the local version of the lib. Specifying a version that is different from your local version of the lib might cause compatibility issues. download_config ([`DownloadConfig`], *optional*): Specific download configuration parameters. download_mode ([`DownloadMode`] or `str`, defaults to `REUSE_DATASET_IF_EXISTS`): Download/generate mode. data_files (`Union[Dict, List, str]`, *optional*): Defining the data_files of the dataset configuration. **download_kwargs (additional keyword arguments): Optional attributes for [`DownloadConfig`] which will override the attributes in `download_config` if supplied, for example `token`. Example: ```py >>> from datasets import get_dataset_config_names >>> get_dataset_config_names("nyu-mll/glue") ['cola', 'sst2', 'mrpc', 'qqp', 'stsb', 'mnli', 'mnli_mismatched', 'mnli_matched', 'qnli', 'rte', 'wnli', 'ax'] ``` rrrr dataset_namer!default) r r osrbasenamelistbuilder_configskeysbuilder_kwargsgetDEFAULT_CONFIG_NAME)rrrrrdownload_kwargsdataset_module builder_clss rr&r&msh, '#   N,NIYIYZ^I_I_```K  +0022 3 3 %))-9X9e\eff8rreturnc $t|f||||d|}t|tj|}t |j}|rt|dkr|dnd} nd} |j p| S)aWGet the default config name for a particular dataset. Can return None only if the dataset has multiple configurations and no default configuration. Args: path (`str`): path to the dataset repository. Can be either: - a local path to the dataset directory containing the data files, e.g. `'./dataset/squad'` - a dataset identifier on the Hugging Face Hub (list all available datasets and ids with [`huggingface_hub.list_datasets`]), e.g. `'rajpurkar/squad'`, `'nyu-mll/glue'` or``'openai/webtext'` revision (`Union[str, datasets.Version]`, *optional*): If specified, the dataset module will be loaded from the datasets repository at this version. By default: - it is set to the local version of the lib. - it will also try to load it from the main branch if it's not available at the local version of the lib. Specifying a version that is different from your local version of the lib might cause compatibility issues. download_config ([`DownloadConfig`], *optional*): Specific download configuration parameters. download_mode ([`DownloadMode`] or `str`, defaults to `REUSE_DATASET_IF_EXISTS`): Download/generate mode. data_files (`Union[Dict, List, str]`, *optional*): Defining the data_files of the dataset configuration. **download_kwargs (additional keyword arguments): Optional attributes for [`DownloadConfig`] which will override the attributes in `download_config` if supplied, for example `token`. Returns: Optional[str]: the default config name if there is one Example: ```py >>> from datasets import get_dataset_default_config_name >>> get_dataset_default_config_name("openbookqa") 'main' ``` r*r+rrNr-) r r r.rr/r0r1r2lenr5) rrrrrr6r7r8r1default_config_names rget_dataset_default_config_namer=sZ, '#   N,NIYIYZ^I_I_```K;6;;==>>O(474H4HA4M4Moa00SW'  * A.AArr!c btf||||||d|}|j} | j|r|n t }|||_ fd|t|j|D| _n"#t$r} td| d} ~ wwxYw| S)aGet the meta information (DatasetInfo) about a dataset for a particular config Args: path (`str`): path to the dataset repository. Can be either: - a local path to the dataset directory containing the data files, e.g. `'./dataset/squad'` - a dataset identifier on the Hugging Face Hub (list all available datasets and ids with [`huggingface_hub.list_datasets`]), e.g. `'rajpurkar/squad'`, `'nyu-mll/glue'` or``'openai/webtext'` config_name (:obj:`str`, optional): Defining the name of the dataset configuration. data_files (:obj:`str` or :obj:`Sequence` or :obj:`Mapping`, optional): Path(s) to source data file(s). download_config (:class:`~download.DownloadConfig`, optional): Specific download configuration parameters. download_mode (:class:`DownloadMode` or :obj:`str`, default ``REUSE_DATASET_IF_EXISTS``): Download/generate mode. revision (:class:`~utils.Version` or :obj:`str`, optional): Version of the dataset to load. As datasets have their own git repository on the Datasets Hub, the default version "main" corresponds to their "main" branch. You can specify a different version than the default "main" by using a commit SHA or a git tag of the dataset repository. token (``str`` or :obj:`bool`, optional): Optional string or boolean to use as Bearer token for remote files on the Datasets Hub. If True, or not specified, will get token from `"~/.huggingface"`. **config_kwargs (additional keyword arguments): optional attributes for builder class which will override the attributes if supplied. )namerrrrrNc0i|]}|j|jdS))r?r,)r?)r#split_generatorrs rr%z+get_dataset_config_info..s:# $/CUY&Z&Zr) base_pathrz#   '#     G >> from datasets import get_dataset_split_names >>> get_dataset_split_names('cornell-movie-review-data/rotten_tomatoes') ['train', 'validation', 'test'] ``` )r!rrrrr)r"r0rDr2) rr!rrrrrr$rCs rget_dataset_split_namesrK's[Z #  '#     D    "" # ##r)NNNNN)NNNN)NNNNNN)%__doc__r.collections.abcrrtypingrrdownload.download_configrdownload.download_managerr #download.streaming_download_managerr rCr loadr r r utils.loggingr utils.versionrrlogger ValueErrorrstrdictr0boolr(r&r=r"rKrrrrZs ! --------""""""""444444333333IIIIII &%%%%%"""""" H       *   48048<.2(, @@ @tT3/0@n-@E,"345 @ uS'\*+ @ E$)$ % @@@@J/3048<37 ?? ?uS'\*+?n-?E,"345 ? tT3/0 ????H/3048<37 ;B;B ;BuS'\*+;Bn-;BE,"345 ;B tT3/0 ;Bc];B;B;B;B@"&_c048<.2(,77 7#7sHSM73c8TW=FX@Y;Y3ZZ[\7n- 7 E,"345 7 uS'\*+ 7 E$)$ %77777x"&_c048<.2(,7$7$ 7$#7$sHSM73c8TW=FX@Y;Y3ZZ[\7$n- 7$ E,"345 7$ uS'\*+ 7$ E$)$ %7$7$7$7$7$7$r