*`i*xddlZddlmZddlmZmZddlZddlmZddl m Z m Z m Z m Z mZddlmZmZmZgdZGdd eZ d ed eed efd Zd ed efdZeZee deje jddde je jZee deje jdddde je j e deje jde jdde je j Z ee deje j!dddej"ej"Z#ee deje j!dddej"ej"e deje j!dddej"ej"Z$ee dej%e j&dde dej%e j'dddZ(ee dej%e j&ddZ)ee dej%e j!dddZ*ee dej%e j!dddZ+ee dej%de j!dde dej%e j'dddZ,ee deje j-dde deje j-ddZ.ee deje j&dde deje j'dddZ/ee deje j0ddddg e deje j!dddd!Z1ee)e*e+e(e(e,e.e/e1ee e#e$d"Z2dS)#N)deepcopy)ListOptional)CompressionFormat) FP8_E4M3_DATA DynamicTypeQuantizationArgsQuantizationStrategyQuantizationType) BaseModel ConfigDictmodel_validator)QuantizationSchemepreset_name_to_schemeis_preset_schemeceZdZUdZeeed<dZee ed<dZ ee ed<dZ ee ed<dZ eeed<e d dd Zed ZdS)ra Set of QuantizationArgs defining how the weights, inputs and outputs of target list of modules should be quantized :param targets: list of modules to apply the QuantizationArgs to, can be layer names, layer types or a regular expression, typically ["Linear"] :param weights: quantization config for layer weights :param input_activations: quantization config for layer inputs :param output_activations: quantization config for layer outputs :param format: CompressionFormat for the layer targetsNweightsinput_activationsoutput_activationsformatafter)modemodelreturnc|j}|j}|j}|j}||jt jt jt jt j t j fvrE|jt jkr|j durtdtd|jd|j td||j td|tjjkrtd|ri|rg|jt jkrR|jt jkr=|j|jkr-t'jd|jd |jd t*d |S) NTzDStatic and local group-wise activation quantization is not supportedzUsing z6 strategy is not supported for activation quantizationz*Cannot apply actorder to input activationsz+Cannot apply actorder to output activationszBmixed-precision cannot be set as a format for a QuantizationSchemezXUsing GROUP strategy for both weights and input_activations with different group sizes (z vs zu) may complicate fused kernel implementations. Consider using TENSOR_GROUP strategy for both or matching group sizes.) stacklevel)rrrrstrategyr TOKENTENSORGROUP TENSOR_GROUP ATTN_HEADdynamicNotImplementedErroractorder ValueErrorrmixed_precisionvalue group_sizewarningswarn UserWarning)rinputsoutputsrrs /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/quantization/quant_scheme.pyvalidate_model_afterz'QuantizationScheme.validate_model_after8s(*-  $*$+$*$1$. 'O';'AAA$..-8 *.V_... * !MNNN  + !NOOO &6< < <T     $8$>>>#7#==="f&777 M/6/A$      forbid)extra)rrrr)__name__ __module__ __qualname____doc__rstr__annotations__rrr rrrrr2r model_configr3r1rr%s  #Y*.GX& '...48x 0188859!12999 FHSM   _'"""999#"9v:H---LLLr3rnamerrc |}|tvr9td|dttt t|}t dd|i|S)a :param name: preset quantization settings name. must exist in upper case in PRESET_SCHEMES :param targets: list of quantization targets to be passed to the Scheme :return: new QuantizationScheme for a given name with the given targets zUnknown preset scheme name z, available names: rr=)upperPRESET_SCHEMESKeyErrorlistkeysrr)r>r scheme_argss r1rr|s ::<!! >$ > > $^%8%8%:%: ; ; > >   >$/00K       r3c8|tvS)zn :param name: preset quantization settings name :return: True if the name is a preset scheme name )r@rA)r>s r1rrs ::<<> ))r3TF)num_bitstyper symmetricr%r+ scale_dtypezp_dtype)r static_minmax) rIrJrrKr%r+observerrLrM)rr )rIrJrr%rKr+rLrM)rIrJrrKr%)rIrJrrKr%rO)rIrJrr+rKr%)rIrJr+rrKr%)rIrJrrKr%block_structure)rIrJrrKr%rOr+) UNQUANTIZEDW8A16W4A16 W4A16_ASYMW8A8INT8W4A8FP8 FP8_DYNAMIC FP8_BLOCKNVFP4A16NVFP4MXFP4A16MXFP4)3r,copyrtypingrrtorchcompressed_tensors.configr*compressed_tensors.quantization.quant_argsrrr r r pydanticr r r__all__rr:rboolrdictrTFLOATr#dtyper^LOCALr_r"uint8r`raINTCHANNELr INT8_W8A8rUrVrW INT8_W4A8r!r[r\BLOCKr]rAr=r3r1rtsc!!!!!!!! 777777<;;;;;;;;;   O.O.O.O.O.O.O.O.d d3i