&`iSdZddlmZmZddlmZddlmZmZm Z ddl m Z m Z GddeZ Gdd e Zd S) zJ This module defines a dataset framework for sampling benchmark requests. )ABCabstractmethod)Path)DictListOptional) load_datasetload_from_diskc|eZdZdZdefdeededdfdZed dZ edede e fd Z dS) BenchmarkDatasetrN dataset_path random_seedreturnc"||_||_dS)a> Abstract base class for benchmark datasets. All benchmark datasets should inherit from this class and implement the required abstract methods. Args: dataset_path: The path to the dataset on disk. random_seed: The seed for the random number generator. N) _dataset_path _random_seed)selfr rs }/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/llm/_internal/batch/benchmark/dataset.py__init__zBenchmarkDataset.__init__s*'c td)z Load data from the dataset source into memory. Raises: NotImplementedError: If the method is not implemented in subclasses. z,load_data must be implemented in subclasses.NotImplementedErrorrs r load_datazBenchmarkDataset.load_data!s""PQQQr num_requestsc td)a< Sample prompts from the loaded dataset. Args: num_requests: The number of prompts to sample from the dataset. Returns: A list of sampled request dictionaries. Raises: NotImplementedError: If the method is not implemented in subclasses. z)sample must be implemented in subclasses.r)rrs rsamplezBenchmarkDataset.sample+s""MNNNrrN) __name__ __module__ __qualname__DEFAULT_RANDOM_SEEDrstrintrrrrrrrrr r s'+.((sm((  (((($RRR^R O3 O4: O O O^ O O Orr c eZdZdZ ddedededed eed df fd Zdd Zd ed e e fdZ dZ d e e fdZ de d e dzfdZxZS)ShareGPTDatasetzhImplements the ShareGPT dataset. The first human message of each conversation is used to build a prompt.,Crystalcareai/Code-feedback-sharegpt-renamedtrainNr seed hf_dataset_idhf_splittruncate_promptrct||||_||_||_||_d|_dS)a Initializes the ShareGPTDataset. Args: dataset_path: The path to the dataset on disk. seed: The seed for the random number generator. hf_dataset_id: The Hugging Face dataset ID to download if the dataset is not found on disk. hf_split: The Hugging Face split to load from the dataset. truncate_prompt: Maximum prompt length so that the prompt fits in the model's context window. N)superr_seed_hf_dataset_id _hf_split_truncate_prompt_data)rr r+r,r-r. __class__s rrzShareGPTDataset.__init__?sH$ t,,, +! /(, rcJ|j||_dSdS)z,Load data from the dataset path into memory.N)r5_load_dataset_datars rrzShareGPTDataset.load_dataZs) : 0022DJJJ  rrc|j|g}|jD]C}t||krn-||}|||D|st d|S)z'Sample prompts from the loaded dataset.Nz*ShareGPT dataset yielded no usable prompts)r5rlen_extract_promptappend ValueError)rrpromptsitem prompt_datas rrzShareGPTDataset.sample_s :  NN   J , ,D7|||++..t44K&{+++ KIJJ Jrc&t|j}td|td| |rt t |}nptd|j|jddt|j|j }| t ||S#t$r}td|d}~wwxYw) z'Load dataset from disk or Hugging Face.z Attempting to load dataset from zDataset exists on disk: z:Dataset not found on disk, downloading from Hugging Face: T)parentsexist_ok)splitz Error loading ShareGPT dataset: N)rrprintexistsr r$r2parentmkdirr r3 save_to_disk Exception RuntimeError)rpathdatasetes r _load_datasetzShareGPTDataset._load_datasetqs!D&'' 777888 888999 G{{}} 0(T33fQUQdff !!$!>>>&t':$.QQQ$$SYY///N G G GE!EEFF F Gs B"C// D9D  Dc||j}g}t|D]\}}||t dt |d|S)z:Load and process dataset data into a list of dictionaries.)r+zLoaded z samples from dataset)rOshuffler1 enumerater<rEr:)rdsdatairows rr8z"ShareGPTDataset._load_dataset_datas~    ! ! ) )tz ) : :mm  FAs KK     8D 888999 rr?c|dp|dpg}td|Dd}|r.|r|jr|d|j}d|iSdS)z Extracts the first human message of a conversation or None. The ShareGPT schema uses {"role": "human", "value": ...} for user turns. messages conversationsc3K|]P}|ddvt|ddVQdS)role>userhumanvalueN)getr$strip).0msgs r z2ShareGPTDataset._extract_prompt..sf  776??&777CGGGR(())//117777  rNprompt)r`nextrar4)rr?rXres rr;zShareGPTDataset._extract_prompts88J''J488O+D+DJ  #         &fllnn &$ 9 7$"7 78f% %tr)r)r*Nr)r r!r"__doc__r$r%rrrrrrrOr8r; __classcell__)r6s@rr(r(<s#rr L)- ---- -  - "# - ------63333 34:$GGG, DJ    DTD[rr(N)rgabcrrpathlibrtypingrrrdatasetsr r r r(r&rrrms$#######''''''''''11111111-O-O-O-O-Os-O-O-O`mmmmm&mmmmmr