*`iy ddlZddlmZddlmZddlZddlmZeej dz Z gdZ ej ej dfded ejd ej d ejfd Zej ej ddfded ejd ej d eejd ejf dZded efdZej de fded ejd ej ded eejf dZdejd ejfdZdS)N)Path)Optional) safe_openzhadamards.safetensors)random_hadamard_matrixdeterministic_hadamard_matrixis_pow2cpusizedtypedevicereturncv|dkrtdttj|}|d|zkrtdt jdgg||}t |D]A}t jt j||ft j|| ff}B|S)a Construct an n-by-n Hadamard matrix, using Sylvester's construction. `n` must be a power of 2. Adapated from https://github.com/scipy/scipy/blob/v1.15.2/scipy/linalg/_special_matrices.py # noqa: E501 :param size: order of the matrix, must be a power of 2 :param dtype: data type of matrix :param device: device to construct matrix on :return: hadamard matrix of size `size` rz4Cannot construct deterministic hadamard of size <= 0z6Cannot construct deterministic hadamard of size != 2^nr r ) ValueErrorintmathlog2torchtensorrangevstackhstack)r r r rH_s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/transform/utils/hadamard.pyrr!s  qyyOPPP ty  D q$wQRRR qcU%777A4[[HH L%,1v.. a!W0E0EF G G Hgenctjdd|f||}||}|dzdz }tj|}t |S)a Produces a randomly generated Hadamard matrix. Differs from `deterministic_hadamard_matrix` in that this function supports non powers of 2 and randomization using a seeded generator Adapated from https://github.com/facebookresearch/SpinQuant/blob/main/utils/hadamard_utils.py # noqa: E501 Known matrices were retrieved from N. J. A. Sloane's Library of Hadamard Matrices http://www.neilsloane.com/hadamard/ # noqa: E501 :param size: The dimension of the hamadard matrix :param dtype: data type of matrix :param device: device to construct matrix on :param gen: Optional generator random values :return: randomly generated hadamard matrix rr)lowhighr generatorr r r)rrandinttodiag _matmul_hadU)r r r rQs rrrAsY(  !!4'SNNNA FA A A 1 A ??rnc&|dko ||dz zdkS)zt Check if a number is a power of 2 :param n: number to check :return: True iff `n` is a power of 2 rr)r*s rrr\s  q5 'a1q5kQ&'r file_pathc|jdkrtjdn|}t|dt |5}t d|Dd}|D]b}||zdkrWt||zrE|t | || ccd d d Sc d d d n #1swxYwYd S) a{ Fetch a known hadamard matrix from the given file path. The returned matrix will be of of size `k` such that `n / k` is a power of two. Return None if no such matrix exists. Note: This function reopens the safetensors file every time it is called. This is technically inefficient, but a very small runtime cost and simpler than forcing callers to manage the file open context :param n: size of known hadamard matrix :param dtype: data type to move fetched hadamard to :param device: device to move fetched hadamard to :return: a known hadamard matrix of size `n` if one exists, else None metar pt) frameworkr c34K|]}t|VdS)N)r).0keys r z*_fetch_hadamard_divisor..|s(;;3s88;;;;;;rT)reverserrN) typerr rstrsortedkeysr get_tensorr&)r*r r r- open_devicefiledivisorsdivisors r_fetch_hadamard_divisorr@fsi(*0)>)>%,u%%%FK 9S5E5E F F FT$;;tyy{{;;;TJJJ T TG7{aGAL$9$9s7||4477eF7SSSS TTTTTTTT TTTTTTTTTTTTTTTT 4sBC!C!!C%(C%Xc|d}|j}|j}t|||}|t d||d}|d|d}|}|jd|kr||jd|jddzd|jd}||j}|dddddddf|dddddddfz|dddddddf<|dddddddf|dddddddfz |dddddddf<||jd|jdd}||}}|jd|k|jd|ksJ~|d||||z}||jS)Nrr$z0Cannot construct random hadamard matrix of size rr) r r r r@rcloneviewshaper&)rAr r r hadKKinputoutputs rr(r(s! 66!99D GE XF #4v > > >D |RDRRSSS ! A GGIINN2tQ ' 'E [[]]F +a.1   5;q>5;q>Q+>5;q>RRU[))"111aaaAAA:.qqq!!!Qz1BBqqq!!!Qz"111aaaAAA:.qqq!!!Qz1BBqqq!!!QzU[^U[^R@@!5 +a.1   ;q>Q     IIaA   ! !% ( (5 0E ::ag  r)rpathlibrtypingrr safetensorsr__file__parent REPO_PATH__all__bfloat16r rr Tensorr Generatorrboolrr8r@r(r,rrrVs  !!!!!! DNN !$; ;  Q P P'5<..   ;  L  \     D'5<..%)   ; L %/ "   \ 6(s(t(((((5<..   ; L  el <ELU\r