PiddlZddlZddlZddlmZddlmZddlm Z ej e Z e ej ddlmZn"#e$re ddZYnwxYweeejddZdejd ejd ejfd Zd ejd ejd ejfdZd ejd ejdejd ejfdZdS)N) is_compiling) out_dtype)check_cpu_version) intmm_tritonzTWarning: Detected no triton, on systems without Triton certain kernels will not workTORCHAO_AUTOTUNER_ENABLEinputmat2returnctsd|vr|jjdkrYt t jjjj t j | | St t jjjj t j ||S|j|jksJd|jd|jd|jj|jjfv}|j ddzdko|j ddk}|j ddzdko|j ddk}|o| }|s|rt j |t j |t j |jjS|s|}|s(|j ddzdkr|} t t jjjj t j ||S#t$$rot j |t j|t jt j cYSwxYw)a Performs a safe integer matrix multiplication, considering different paths for torch.compile, cublas, and fallback cases. Args: input (torch.Tensor): The input tensor of shape [i, j]. mat2 (torch.Tensor): The matrix to multiply with, of shape [j, k]. Returns: torch.Tensor: The result of the matrix multiplication. Raises: AssertionError: If the tensors are not on the same device. FakeTensorcpuz3need both tensors to be on the same device but got z and r)dynamo_is_compiling__repr__devicetypertorchopsatenmmdefaultint32floatshapematmulr to is_contiguous contiguous Exceptionfloat32)rr device_cpuj_is_nonzero_multiple_of_8k_is_nonzero_multiple_of_8bad_dimensions_for_cublass h/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/kernel/intmm.py safe_int_mmr's N 0@0@ @ @ <  % % !)5; tzz|| *2EKMMM ;%, & & &^dk^^PUP\^^ ' & &4;+U\->??J"'+a.1"4"9!S AQR@R"&*Q-!"3q"8!Qtz!}q?P"A'A! . |EIIKKNN5;77u{9S9STTWW L        !      ! !  Aa       *2EKMMM    |EHHU]33TWWU]5K5KLLOO K      s4JA6K=<K=abct,tr%tjj||St ||S)a\ Performs integer matrix multiplication using intmm_triton if available and autotuner is enabled, otherwise falls back to safe_int_mm. Args: a (torch.Tensor): The first matrix to multiply. b (torch.Tensor): The second matrix to multiply. Returns: torch.Tensor: The result of the matrix multiplication. )rAUTOTUNER_ENABLErrtorchao int_matmulr')r(r)s r&r-r-[s;$4y ++Aq111 q!  scales1cr|j\}}|j\}}||dks|dksJd|dksJ|sJ|||f}|dksJt |jr2tj ||}| |j |zSt-tr&tjj|||St#||}||zS)a Performs scaled integer matrix multiplication. Args: a (torch.Tensor): The first matrix to multiply. b (torch.Tensor): The second matrix to multiply. scales1 (torch.Tensor): The scaling factors for the rows of the result. Returns: torch.Tensor: The result of the scaled matrix multiplication. Raises: AssertionError: If the dimensions of the input tensors do not match the expected shapes. rr)rsizenumelrexpanddimrrr_int_mmrdtyperr+rr,int_scaled_matmulr')r(r)r/MKNcs r&r8r8ls(" 7DAq 7DAq  Q  7==??a#7#7#7 7  Q      " """ "nnaV$$G ;;==A    ((- M!Q  ttGM""W,,$4y 221aAAAAqA w;r.)loggingosr torch._dynamorr!torch._higher_order_ops.out_dtyper torchao.utilsr getLogger__name__logger addHandler NullHandlertorchao.kernelr ImportErrorwarningboolintgetenvr+Tensorr'r-r8r.r&rOs  ======777777++++++  8 $ $%'%''(((+++++++ NN^LLL 4IBI&@!DDEEFF: u|: 5<: EL: : : : z%,5<EL"# |# #/4|# \######sAA87A8