)`iN dZddlZddlmZddlZddlmZddlmZddl m Z m Z ej dZ e d d d ejd edejfdZe d d ejd edejfdZedd ejd edejfdZe dd ejdejd edeejejffdZdS)a3 Copyright (c) 2024 by FlashInfer team. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. N)Tuple)flashinfer_api)gen_quantization_module)register_custom_opregister_fake_opcBtS)N)rbuild_and_loadk/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/flashinfer/quantization.pyget_quantization_modulers " $ $ 3 3 5 55r zflashinfer::packbitsr ) mutates_argsxbitorderreturnc |j}|tj}tj|ddzdztj|}t||||SNrdtypedevice) rtotorchboolemptysizeuint8rpackbits)rrrys r _packbitsr" sj XF UZA QVVAYY]q( FKKKA&&q(A666 Hr ctj|ddzdztj|jSr)rrrrrrrs r _fake_packbitsr%)s0 ;q A !+5;qx P P PPr bigc"t||S)a9Pack the elements of a binary-valued array into bits in a uint8 array. The semantics of this function is the same as `numpy.packbits `_. Parameters ---------- x: torch.Tensor The 1D binary-valued array to pack. bitorder: str The bit-order ("bit"/"little") of the output. Default is "big". Returns ------- y: torch.Tensor An uint8 packed array, shape ``((x.size(0) + 7) / 8),)``. Examples -------- >>> import torch >>> from flashinfer import packbits >>> x = torch.tensor([1, 0, 1, 1, 0, 0, 1, 1], dtype=torch.bool, device="cuda") >>> x_packed = packbits(x) >>> list(map(bin, x_packed.tolist())) ['0b10110011'] See Also -------- segment_packbits )r"r$s r r r .s@ Q ! !!r indptrc|dd|ddz }|dzdz}tjt||j|j}tj|d|dd<|d}|j}|tj}|tj}tj |tj |}t |||||||fS)avPack a batch of binary-valued segments into bits in a uint8 array. For each segment, the semantics of this function is the same as `numpy.packbits `_. Parameters ---------- x: torch.Tensor The 1D binary-valued array to pack, shape ``(indptr[-1],)``. indptr: torch.Tensor The index pointer of each segment in :attr:`x`, shape ``(batch_size + 1,)``. The i-th segment in :attr:`x` is ``x[indptr[i]:indptr[i+1]]``. bitorder: str The bit-order ("bit"/"little") of the output. Default is "big". Returns ------- y: torch.Tensor An uint8 packed array, shape: ``(new_indptr[-1],)``. The ``y[new_indptr[i]:new_indptr[i+1]]`` contains the packed bits ``x[indptr[i]:indptr[i+1]]``. new_indptr: torch.Tensor The new index pointer of each packed segment in :attr:`y`, shape ``(batch_size + 1,)``. It's guaranteed that ``new_indptr[i+1] - new_indptr[i] == (indptr[i+1] - indptr[i] + 7) // 8``. Examples -------- >>> import torch >>> from flashinfer import segment_packbits >>> x = torch.tensor([1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1], dtype=torch.bool, device="cuda") >>> x_packed, new_indptr = segment_packbits(x, torch.tensor([0, 4, 7, 11], device="cuda"), bitorder="big") >>> list(map(bin, x_packed.tolist())) ['0b10110000', '0b100000', '0b11010000'] >>> new_indptr tensor([0, 1, 2, 3], device='cuda:0') Note ---- ``torch.compile`` is not supported for this function because it's data dependent. See Also -------- packbits rNrrrr) rzeroslenrrcumsumitemrint32rrrsegment_packbits) rr(rseglen packed_len indptr_new output_nnzsrr!s r r0r0Qs^ABBZ&"+ %F1*"JS[[ V]SSSJ\*a00JqrrNR.%%''K XF YYu{ # #Fu{++J Ku{6BBBA..q&*hPQRRR j=r )r&)__doc__ functoolstypingrr api_loggingrjit.quantizationrutilsrrcacherTensorstrr"r%r r0r r r r>s  ''''''55555577777777 666*<<<      =< ())QelQcQelQQQ*)Q"" "" """"D;@99 |9"\9589 5< %&999999r