*`i7 hddlmZddlmZmZmZmZmZddlZddl m Z ddl m Z m Z ddlmZddlmZddlmZdd lmZgd Z dd eeefd eeefd ee fdZ ddeeefde deedeeeefddffdZ ddejjded ee defdZdS))Path)Dict GeneratorOptionalTupleUnionN)BaseCompressor)CompressionFormatSparsityCompressionConfig)get_weight_mappings) safe_open) save_file)Tensor)load_compressedsave_compressedsave_compressed_modeltensors save_pathcompression_formatc |t|dkrtd|ptjj}|t jvs\|t jvsGtd|dtt jt jzt j |}| |}t||dS)a_ Save compressed tensors to disk. If tensors are not compressed, save them as is. :param tensors: dictionary of tensors to compress :param save_path: path to save compressed tensors :param compression_format: compression format used for the tensors :return: compression config, if tensors were compressed - None otherwise Nrz0No tensors or empty tensors provided to compresszUnknown compression format: z. Must be one of ) len ValueErrorr densevaluer registered_namesregistered_aliasessetload_from_registrycompressr)rrr compressorcompressed_tensorss z/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/compressors/helpers.pyrr"s#g,,!++KLLL,L/@/F/L n=???? !B!D!D D D m+= m m!."A"C"CnFgFiFi"ijj m m    23EFFJ#,,W55  ),,,,,cpur!compression_configdevicereturnc#K|!t|std||jtjjkrpt|}|D]J\}}t|d|5}| |}||fVdddn #1swxYwYKdS|j}tj ||} | ||Ed{VdS)a Load compressed tensors from disk. If tensors are not compressed, load them as is. :param compressed_tensors: path to compressed tensors. This can be a path to a file or a directory containing one or multiple safetensor files (if multiple - in the format assumed by huggingface) :param compression_config: compression config to use for decompressing tensors. :param device: device to move tensors to. If None, tensors are loaded on CPU. :param return_dict: if True, return a dictionary of decompressed tensors :return a generator that yields the name and tensor of the decompressed tensor Nz&No compressed tensors provided to loadpt) frameworkr&)config)r&)rexistsrformatr rrr itemsr get_tensorr r decompress) r!r%r&weight_mappings weight_namefile_with_weight_namefweightrr s r"rrFsw$!.@)A)A)H)H)J)J!ABBB "  $(9(?(E E E..@AA2A2G2G2I2I * * .K.0DPPP *TUk22!6)))) * * * * * * * * * * * * * * * * * 06#6 '9   (();F(KKKKKKKKKKKs B33B7 :B7 Tmodelfilenameforce_contiguousc|}|rd|D} t|||dS#t$r(}t |}|dz }t|d}~wwxYw)a Wrapper around safetensors `save_model` helper function, which allows for saving compressed model to disk. Note: The model is assumed to have a state_dict with unique entries :param model: model to save on disk :param filename: filename location to save the file :param compression_format: compression format used for the model :param force_contiguous: forcing the state_dict to be saved as contiguous tensors c>i|]\}}||S) contiguous).0kvs r" z)save_compressed_model..s&GGGDAqaGGGr#)rz_ Or use save_compressed_model(..., force_contiguous=True), read the docs for potential caveats.N) state_dictr.rrstr)r6r7rr8rAemsgs r"rros$!!##JHGGJ4D4D4F4FGGG  HASTTTTTT !ff ppoosA A<#A77A<)N)Nr$)NT)pathlibrtypingrrrrrtorchcompressed_tensors.compressorsr compressed_tensors.configr r )compressed_tensors.utils.safetensors_loadr safetensorsr safetensors.torchrr__all__rBrrnnModuleboolrr;r#r"rQs:::::::::::::: 999999RRRRRRRRIIIIII!!!!!!''''''   7;!-!- #v+ !-S$Y!-!!23!-!-!-!-L59!&L&Lc4i(&L1&L SM&LuS&[!4-. &L&L&L&LX7;!  8?!!23 r#