Pi |ddlZddlmZgdZdZdZGddeZ dd ejd ed ed ejfdZ dS)N)UniformQuantizationObserverBase)create_block_sparse_tensorcreate_semi_structured_tensorPerChannelNormObserver mask_creatorcd|dkr|dks Jdtjtj||z||zfd|z |}tj||d}tj||d}||S)N?gz*sparsity should be a value between 0 and 1)dtyperdim)torch bernoullifullrepeat_interleaveto contiguouscuda)MN blocksizesparsityr As j/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/sparsity/utils.pyrrs s??x34 / .  ANAN3Q\OOO  A 9!444A 9!444A 44;; ! ! # # ( ( * **ctjdd||zdzf}tjj|d||tj }|tj |||zz}||S)z This function returns a 1:2 sparse matrix of size (r, c). Note that this means this matrix will also be 2:4 and 4:8 sparse as well. r) num_classes) rrandintrnn functionalone_hotreshaperrint32rand)rcr choice_indicesmask sparse_weights rrrs]1a!a%1*77<<>>N ##N#BB A  EK  EJq!,,1133d:;M   E " ""rc0eZdZdZdfd ZdZdZxZS)rz` A custom observer that computes the L2 norm of each channel and stores it in a buffer. returnNc tjdtjtjdddtjtjjd|d|_| dtj gdS)NF)r qscheme reduce_range quant_min quant_maxepsr norm) super__init__rquint8per_channel_affinefinfofloat32r2averaging_constantregister_buffertensor)selfkwargs __class__s rr6zPerChannelNormObserver.__init__8s ,, EM**.      #& VU\"%5%566666rcR|dkr|S|}dt|D}|d|dc|d<|d<||}t j|d}t j|ddz}|j dkr:|j |j |j |n|xj |z c_ |S)Nrcg|]}|Sr4r4).0is r z2PerChannelNormObserver.forward..Os333q333rr ) start_dimr r) numeldetachranger permuterflattenlinalg vector_normr3resize_shapecopy_)r>x_origx new_axis_listyr3s rforwardzPerChannelNormObserver.forwardIs  <<>>Q  M MMOO43E!%%''NN333 .;B.?qAQ+ a-+ IIm $ $ M!q ) ) )|''q'11Q6 9??   ! ! I  dj ) ) ) IOOD ! ! ! ! II II rc td)Nz>PerChannelNormObserver is designed to store activations only. )NotImplementedError)r>s rcalculate_qparamsz(PerChannelNormObserver.calculate_qparams_s! L   r)r,N)__name__ __module__ __qualname____doc__r6rVrY __classcell__)r@s@rrr3se 7 7 7 7 7 7",       rrrr=rrr,cd}||zdkrtd|jd|d||z}|||}t j|ddddt||z f}t j |j|j }| d|d |j}|S) a Class for creating N:M sparsity masks. Masks will be created using the N:M ratio, where for every block of M weights, N will be pruned based on ranked weight value. Each mask will correspond to the given tensor. :param tensor: The input tensor to create a mask for :param N: The number of weights in a group to keep :param M: The size of a weight group :return: A mask tensor with the same shape as the input tensor NrzTensor of size z can't be evenly divided into z groupsr r )device)r indexvalue) rH ValueErrorrPrIabsr#rargsortintonesrascatter_)r=rrr) num_groups tensor_temprbw_bs rrres D ||~~Q Tfl T T! T T T   1$J--//%%''// A>>K M+1 - - -aaa3q1u::o >E *[&{/A B B BC <rps  JJJJJJ    + + +###,/ / / / / </ / / h  L   \ r