*`i..ddlZddlZddlZddlZddlmZmZmZmZm Z ddl m Z ddl m Z mZmZgdZeeeee ffZeeefZeeefZ ddedeedefd Zd edeeeffd Zd ed edeefdZdededefdZdedeeeffdZ ddedeedede eeeefffdZ ddeee fdeedede eeeeee ffffdZdedeeeffdZdedefdZdS) N)DictIterableOptionalTupleUnion)Tensor)SAFE_WEIGHTS_INDEX_NAMESAFE_WEIGHTS_NAME cached_file) get_safetensors_folderget_safetensors_headermatch_param_name merge_namesget_weight_mappingsget_nested_weight_mappings#get_nested_mappings_from_state_dict*get_quantization_parameter_to_path_mappingis_quantization_parampretrained_model_name_or_path cache_dirreturnctj|rtj|St |t |d}t |t |d}|%tj|dS|%tj|dStd|d)a Given a Hugging Face stub or a local path, return the folder containing the safetensors weight files :param pretrained_model_name_or_path: local path to model or HF stub :param cache_dir: optional cache dir to search through, if none is specified the model will be searched for in the default TRANSFORMERS_CACHE :return: local folder containing model data F)r%_raise_exceptions_for_missing_entriesNrz7Could not locate safetensors weight or index file from .) ospathexistsabspathr r r split ValueError)rrsafetensors_path index_paths }/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/utils/safetensors_load.pyr r *s w~~344>w<==="%.3  %.3 J #w}}-..q11w}}Z((++  , ( , , ,  r!ct|d5}tjd|dd}||}t j|}dddn #1swxYwY|S)z Extracts the metadata from a safetensors file as JSON :param safetensors_path: path to a safetensors file :return: dictionary of metadata extracted from the safetensors file rbz4((((7<<(@BSTTW\\":>* + + +,<==F{{}} 0 0/s JJ~t , , , , W^^J ' ' j#888 %A !  % % % % % % % % % % % % % % %<(FF?$<??  !,,.. H HJC',,'?GGF3KK MsE%%E),E)F model_pathparams_to_nestreturn_unmatched_paramsct|}i}i}|D]=\}}d}|D]*} t|| } | r| |vri|| <||| | <d}+|r|s|||<>|r||fS|S)a Takes a path to a state dict saved in safetensors format and returns a nested mapping from uncompressed parameterized layer names to the file locations of each layer's compression parameters. Example of the nested mapping: layer: { bitmask: file_location, row_offsets: file_location, shape: file_location, compressed: file_location } If other parameters are found that do not match the nested parameters, they will be returned in a separate dictionary only if return_unmatched_params is True. This dictionary may be needed for cases where compressors are stacked (e.g., quantization compression followed by sparse compression). Example of the unmatched params mapping: { layer.weight_scale: file_location, layer.input_scale: file_location } This generalizes to cases where the model is split into multiple safetensors files. :param model_path: Path to the safetensors state dict, must contain either a single safetensors file or multiple files with an index. :param params_to_nest: Iterable of parameter names to nest. :param return_unmatched_params: If True, return a second dictionary containing the remaining parameters that were not matched to the params_to_nest. :return: - If return_unmatched_params is False: NestedWeightMappingType: A nested mapping of parameterized layer names to file locations of each layer's compression parameters. - If return_unmatched_params is True: Tuple[NestedWeightMappingType, WeightMappingType]: A tuple containing: - NestedWeightMappingType: A nested mapping of parameterized layer names to file locations of each layer's compression parameters. - WeightMappingType: A mapping of the remaining parameter names to their file locations that were not matched to the params_to_nest. FT)rrJr) rNrOrPweight_mappingsnested_weight_mappingsunmatched_paramsrK file_locationmatchedr3 module_paths r#rrs`**55O-3355 2 2](  J*3 ;;K &<<<:<*;7BO&{3J? " 27 2$1 S !8%'777 !!r$ state_dictci}i}|D]F}d}|D]0}t||}|r||vri||<|||||<d}1|r |s ||||<G|r||fS|S)a{ Takes a state dict and returns a nested mapping from uncompressed parameterized layer names to the value of each layer's compression parameters. Example of the nested mapping: layer: { weight_scale: ..., weight: ..., zero_point: ..., } :param state_dict: state dict of the model :param params_to_nest: Iterable of parameter names to nest. :return: Nested mapping of parameterized layer names to the value of each layer's compression parameters. If `return_unmatched_params`, then also return a dictionary mapping unused parameter names to their values FT)rFr) rXrOrPrSrTrKrVr3rWs r#rrs.   4 4(  J*3 ;;K &<<<:<*;7BLS/&{3J? " 47 4$.sO S !8%'777 !!r$ct|}i}|D]\}}t|r|||<|S)z[ Given a model path, return a mapping between a parameter and its path on disk )rrJr)rNrRmapping weight_name safe_paths r#rr&s_ **55OG"1"7"7"9"9 Y  - - #,GK   Nr$namec|drdS|drdS|drdSdS)z Checks is a parameter name is associated with a quantization parameter :param name: parameter name to check :return: True if parameter name is a quantization parameter, else False _scaleT zero_pointg_idxF)endswith)r^s r#rr4sT }}Xt }}\""t }}Wt 5r$)N)F)r,rr6r)typingrrrrrtorchrtransformers.utilsr r r __all__strNestedStateDictTypeWeightMappingTypeNestedWeightMappingTyper r rrrboolrrrrr>r$r#rms 99999999999999VVVVVVVVVV   3S&[ 112cNs$556DH''#&'3;C='''''T S T#s(^       #     *S *c *c * * * *3#3$sCx.3333r%*B"B"B"SMB""B" "E*ACT*T$U UV B"B"B"B"P%*("("S&[!("SM(""(" &94V ;L&L M MN ("("("("V 3 4S>    r$