*`i< ddlZddlZddlmZddlmZddlmZmZm Z m Z m Z m Z m Z mZmZddlZddlZddlmZmZeddZerdd lmZgd Zd Zd ed edfdZded efdZd9ded efdZdejj dedejj fdZ!ded efdZ"deded efdZ# d:deedeed e egeffdZ$Gd d!Z% d;d"ej&d#e e'd$e'd e ej&fd%Z(d;d&Z)d'ej&d ej&fd(Z*d)ej&d*e e'd ej&fd+Z+ej,d,e-d-ed.efd/Z.ej,d0e ed-ed1e efd2Z/Gd3d4e0Z1d5ed e'fd6Z2d5ed e'fd7Z3d5ed e'fd8Z4dS)<N)wraps)MappingProxyType) TYPE_CHECKINGAnyCallableDictIterableListMappingOptionalTypeVar) AutoConfigPretrainedConfigTr)boundModelCompressor)"infer_compressor_from_model_configfix_fsdp_module_nametensor_follows_mask_structurereplace_moduleis_compressed_tensors_config getattr_chain deprecated Aliasablecombine_shards shard_tensor pack_bitmasksunpack_bitmasks patch_attr patch_attrsParameterizedDefaultDictget_num_attn_headsget_num_kv_heads get_head_dim_fsdp_wrapped_modulepretrained_model_name_or_pathreturnrcddlm}ddlm}t j|}|j|}|dS|d}|j|fi|}|j||}|S)a Given a path to a model config, extract a sparsity config if it exists and return the associated ModelCompressor :param pretrained_model_name_or_path: path to model config on disk or HF hub :return: matching compressor if config contains a sparsity config rr)CompressionConfigNformatconfig) compressed_tensors.compressorsrcompressed_tensors.configr*rfrom_pretrainedparse_sparsity_configgetload_from_registry)r'rr*r-sparsity_configr+ compressors t/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/utils/helpers.pyrrCs?>>>>>;;;;;;  '(E F FF;o;FCCOt   * *F:':6UU_UUO33F?SSSJ namecv|tdzddtzdS)z Remove FSDP wrapper prefixes from a module name Accounts for scenario where FSDP_WRAPPER_NAME is at the end of the name, as well as in the middle. :param name: name to strip :return: stripped name .)replaceFSDP_WRAPPER_NAME)r8s r6rr[s; <<)C/ 4 4 < <   r72:4maskcFttt|d\}}|d|}|dkd}t j||kstdS)a, :param tensor: tensor to check :param mask: mask structure to check for, in the format "n:m" :return: True if the tensor follows the mask structure, False otherwise. Note, some weights can incidentally be zero, so we check for atleast n zeros in each chunk of size m :r)dimT) tuplemapintsplitviewsumtorchallitem ValueError)tensorr?nm zero_countss r6rrhs S$**S//** + +DAq [[Q  FQ;###**K 9[A% & & + + - -ll 4r7model new_modulecd|vrL|ddd}|t|dzd}||}nd}|}|}t|||dS)Nr:rCrr;)rsplitlen get_submodulesetattr)rSr8rT parent_name child_nameparents r6rrs| d{{kk#q))!, #k**Q.001 $$[11   FJ +++++r7compression_configcR ddlm}t||S#t$rYdSwxYw)z Returns True if CompressedTensorsConfig is available from transformers and compression_config is an instance of CompressedTensorsConfig See: https://github.com/huggingface/transformers/pull/31704 r)CompressedTensorsConfigF)&transformers.utils.quantization_configr_ isinstance ImportError)r]r_s r6rrsMRRRRRR,.EFFF uus  &&obj chain_strct|dkr d}|d}nd|vr d}|d}nd}|d}|}|D]<}t||s|r|cSt|d|t ||}=|S)z Chain multiple getattr calls, separated by `.` :param obj: base object whose attributes are being retrieved :param chain_str: attribute names separated by `.` :param default: default value, throw error otherwise rCTrdefaultFr:z object has no attribute )rWrHhasattrAttributeErrorgetattr) rcrdargskwargs has_defaultrf attr_namesres attr_names r6rrs 4yyA~~ q' f   # %%J C&& sI&& S S$%Q%Qi%Q%QRRRc9%% Jr7 future_namemessagec4dtdtffd }|S)z Decorator to mark functions as deprecated :param new_function: Function called in place of deprecated function :param message: Deprecation message, replaces default deprecation message funcr(cnjd ddz tfd}|S)Nz6 is deprecated and will be removed in a future releasez . Please use z instead.cLtjtd|i|S)N) stacklevel)warningswarnDeprecationWarning)rjrkrsrqs r6wrappedz.deprecated..decorator..wrappeds0 M'#5! D D D D4((( (r7)__name__r)rsr{rprqs` r6 decoratorzdeprecated..decoratorsq ?=XXX &A;AAAA t ) ) ) ) )  )r7)r)rprqr}s`` r6rrs=a" r7cJeZdZdZedeeeffdZdZdZ dS)rz A mixin for enums to allow aliasing of enum members Example: >>> class MyClass(Aliasable, int, Enum): >>> ... r(ctN)NotImplementedErrorr7r6 get_aliaseszAliasable.get_aliasess!###r7ct||jrf|}|j|jkpA||j|j||j|jkS|}||j|j}|||}||kSr)ra __class__rvaluer2)selfotheraliases self_value other_values r6__eq__zAliasable.__eq__s eT^ , , -&&((G:, DJ 33;;u{EK889  &&((G TZ<}|j}| || |||z }?|S)z Combine decompressed shards along a given dimension using `narrow`. :param shards: List of decompressed shard tensors. :param dim: Dimension to combine along (default: 0). :return: Combined decompressed tensor. zThe list of shards is empty.ch|] }|j Sr)dtype).0rs r6 z!combine_shards..$s444EEK444r7rCz$All shards must have the same dtype.rc32K|]}|jVdSr)shape)rrrDs r6 z!combine_shards..*s*@@5;s+@@@@@@r7)rdevice) rNrWlistrrJrKzerosrrrcopy_)rrD shard_dtypes total_shapecombined shard_offsetr shard_sizes ` r6rrs  9788854V444L <1?@@@vay''K@@@@@@@@@K{;fQiofQiFVWWWHL##[% \:66<>> from types import SimpleNamespace >>> obj = SimpleNamespace() >>> with patch_attr(obj, "attribute", "value"): ... assert obj.attribute == "value" >>> assert not hasattr(obj, "attribute") N)objectrirYdelattr)rrr _sentineloriginal_values r6r r as IT433N D$    * * D$ / / / / / D$       * * D$ / / / / D$    s A!!(B basesvaluesc #Ktj5}t||D])\}}|t |||*dVddddS#1swxYwYdS)a Same as `patch_attr` but for a list of objects to patch Patch attribute for a list of objects with list of values. Original values are restored upon exit :param bases: objects which has the attribute to patch :param attr: name of the the attribute to patch :param values: used to replace original values. Must be same length as bases Usage: >>> from types import SimpleNamespace >>> obj1 = SimpleNamespace() >>> obj2 = SimpleNamespace() >>> with patch_attr([obj1, obj2], "attribute", ["value1", "value2"]): ... assert obj1.attribute == "value1" ... assert obj2.attribute == "value2" >>> assert not hasattr(obj1, "attribute") >>> assert not hasattr(obj2, "attribute") N) contextlib ExitStackzip enter_contextr )rrrstackrrs r6r!r!~s,    5uf-- ? ?KD%    4u = = > > > > s?A""A&)A&cheZdZdZdeegeffdZdedefdZeidde defd Z d S) r"a  Similar to `collections.DefaultDict`, but upon fetching a key which is missing, the key is passed as arguments to the `default_factory` :param default_factory: function which takes a key as input and returns the corresponding default value default_factoryc<||_ti|_dSr)rr_factory_kwargs)rrs r6__init__z!ParameterizedDefaultDict.__init__s ./33r7keyr(ct|tr|j|i|j}n|j|fi|j}|||<|Sr)rarErr)rrrs r6 __missing__z$ParameterizedDefaultDict.__missing__s\ c5 ! ! F(D(#F1EFFEE(D(EE0DEEES  r7)factory_kwargsrcht|d|5||cdddS#1swxYwYdS)a" Similar to `__getitem__`, but allows passing kwargs to factory function :param \*args: args whose tuple will value will be treated as key :param factory_kwargs: keyword arguments to pass to `default_factory` :return: dictionary entry for given key rN)r )rrrjs r6r2zParameterizedDefaultDict.gets/ @ @  :                  s '++N) r|rrrrrrrrr r2rr7r6r"r"s4#(<4444ss4D3CB3G3G    C      r7r"r-ct|dr|jSt|drt|dr|j|jzSt d|)z Get the number of attention heads used by a model :param config: model config :return: num_attention_heads of model num_attention_heads hidden_sizehead_dimzCannot determine num_attention_heads from config. Config must define either `num_attention_heads` or both `hidden_size` and `head_dim`. )rgrrrrNr,s r6r#r#syv,--  ))  ' ' GFJ,G,G !V_44      r7cTt|dr|jStd|)z Get the number of key-value attention heads used by a model :param config: model config :return: num_key_value_heads of model num_key_value_headsz\Cannot determine num_key_value_heads from config. Config must define `num_key_value_heads`. )rgrrNr,s r6r$r$sFv,-- )) /&, / /   r7ct|dr|jSt|drt|dr|j|jzSt d|)z Get the number of dimensions used by the attention heads of a model :param config: model config :return: head_dim of model rrrz}Cannot determine head_dim from config. Config must define either `head_dim` or both `hidden_size` and `num_attention_heads`. )rgrrrrNr,s r6r%r%syvz""    ' ' GF)NN)r)5rrx functoolsrtypesrtypingrrrrr r r r r rrK transformersrrrr.r__all__r=rrrrrnnModulerrrrrTensorrGrrrrcontextmanagerrr r!dictr"r#r$r%rr7r6rs\""""""                       55555555 GCz"""?>>>>>>   *+#&  0 s s    2 ,%(/ , ,%(/ , , , , S T    ss@AE#08  qc1f<%%%%%%%%@>? L'+Cy7: %,@B U\ el    #\#379# \####4  V 3 s    8 x}C#6tB / C    * - #    " ) c      r7