*`i$ddlmZmZddlmZmZmZmZmZddl Z ddl m Z ddl m Z mZddlmZddlmZddl mZdd lmZd gZGd d eeZdS) )ABCabstractmethod)Dict GeneratorOptionalTupleUnionN)SparsityCompressionConfig)QuantizationArgsQuantizationConfig) RegistryMixin)has_offloaded_params)Tensor)ModuleBaseCompressorc eZdZdZ ddeeedffdZ ddej de e de e eej ejfffdZeedee fd Zed e e efde e effd Ze dd e de deee efddffdZdede e e ejffdZdede e ejffdZdefdZde e efdejfdZdS)ra Base class representing a model compression algorithm. Each child class should implement compression_param_info, compress_weight and decompress_weight. Compressors support compressing/decompressing a full module state dict or a single quantized PyTorch leaf module. Model Load Lifecycle (run_compressed=False): - ModelCompressor.decompress() - apply_quantization_config() - BaseCompressor.decompress() Model Save Lifecycle: - ModelCompressor.compress() - BaseCompressor.compress() Module Lifecycle (run_compressed=True): - apply_quantization_config() - compressed_module = CompressedLinear(module) - initialize_module_for_quantization() - BaseCompressor.compression_param_info() - register_parameters() - compressed_module.forward() -compressed_module.decompress() :param config: config specifying compression parameters Nconfigc||_dSN)r)selfrs w/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/compressors/base.py__init__zBaseCompressor.__init__=s weight_shapequantization_argsreturnct)aY Creates a dictionary of expected shapes and dtypes for each compression parameter used by the compressor :param weight_shape: uncompressed weight shape :param quantization_args: quantization parameters for the weight :return: dictionary mapping compressed parameter names to shape and dtype NotImplementedError)rrrs rcompression_param_infoz%BaseCompressor.compression_param_infoB"###rct)zx Returns a tuple of compression parameter names introduced by the compressor during compression r)rs rcompression_param_namesz&BaseCompressor.compression_param_namesQs"###r model_statec t)z Compresses a dense state dict :param model_state: state dict of uncompressed model :param kwargs: additional arguments for compression :return: compressed state dict r)rr$kwargss rcompresszBaseCompressor.compressZr!rcpupath_to_model_or_tensorsdevicec t)a Reads a compressed state dict located at path_to_model_or_tensors and returns a generator for sequentially decompressing back to a dense state dict :param path_to_model_or_tensors: path to compressed safetensors model (directory with one or more safetensors files) or compressed tensors file :param names_to_scheme: quantization args for each quantized weight :param device: optional device to load intermediate weights into :return: compressed state dict r)rr)r*r&s r decompresszBaseCompressor.decompressis$"###rmodulect|dsdS|j}t|dsdS|j}t|dd}t|dd}t|dd}|||||S)a" Compresses a single quantized leaf PyTorch module. If the module is not quantized, this function has no effect. :param module: PyTorch module to compress :return: dictionary of compressed weight data, or None if module is not quantized quantization_schemeNweightsweight weight_scaleweight_zero_point)r1scale zero_pointr)hasattrr/r0getattrcompress_weight)rr-r/rr1r2r3s rcompress_modulezBaseCompressor.compress_module}sv455 4$8*I66 4/7400v~t<< #F,?FF##(/ $   rr1c t)z Compresses a single uncompressed weight :param weight: uncompressed weight tensor :param kwargs: additional arguments for compression r)rr1r&s rr8zBaseCompressor.compress_weights"###rcnt|j}t|rdn|}t |dsdS|j}t |dsdS|j}i}|D] \}}|||< ||| |S)a Decompresses a single compressed leaf PyTorch module. If the module is not quantized, this function has no effect. :param module: PyTorch module to decompress :return: tensor of the decompressed weight, or None if module is not quantized r(r/Nr0)compressed_datar) next parametersr*rr6r/r0named_parametersdecompress_weightto) rr- params_devicer*r/rr<name parameters rdecompress_modulez BaseCompressor.decompress_modulesV..00118 .v66IMv455 4$8*I66 4/7%6688 . .OD)$-OD ! !%%+?P&  "V** rr<c t)z Decompresses a single compressed weight :param compressed_data: dictionary of data needed for decompression :param kwargs: additional arguments for decompression :return: tensor of the decompressed weight r)rr<r&s rr@z BaseCompressor.decompress_weights"###rr)r()__name__ __module__ __qualname____doc__r r r rtorchSizerr rstrrdtyper propertyrr#rr'rr,rr9r8rEr@rrrrs1>TX57I4OP9= $ $j $$$45 $ c5U[011 2 $ $ $ $$s$$$^X$ $#v+& $ c6k  $ $ $^ $$$"%$$ 5f%tT1 2 $$$^$& f $sEL?P:Q1R    6 $ $ c5< $ $ $ $6 $#CK0 $  $ $ $ $ $ $r)abcrrtypingrrrrr rKcompressed_tensors.configr compressed_tensors.quantizationr r compressed_tensors.registryr compressed_tensors.utilsrrtorch.nnr__all__rrPrrrYs$#######:::::::::::::: ??????PPPPPPPP555555999999  l$l$l$l$l$]Cl$l$l$l$l$r