PiddlmZddlZedGddZedGddeZedGd d eZedGd d eZedGd deZedGddeZedGddeZ ej e eegdS)) dataclassNT)frozenceZdZdZdS) Granularityz Base class for representing the granularity of quantization. This class serves as a parent for specific granularity types used in quantization operations, such as per-tensor or per-axis quantization. N__name__ __module__ __qualname____doc__t/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/quantization/granularity.pyrr  Dr rceZdZdZdS) PerTensorz Represents per-tensor granularity in quantization. This granularity type calculates the quantization parameters based off the entire tensor. Nrr r rrrrr rceZdZUdZeed<dS)PerAxisa Represents per-axis granularity in quantization. This granularity type calculates different quantization parameters along a specified axis of the tensor. Examples: * input_tensor shape [A, B], axis 0 -> scale_shape [A, 1] * input_tensor shape [A, B], axis 1 -> scale_shape [1, B] * input_tensor shape [A, B, C], axis 1 -> scale_shape [1, B, 1] Attributes: axis (int): The axis which is kept, reduction is performed across all the other axes axisNrr r r int__annotations__r r rrr$s% IIIIIr rceZdZUdZeed<dS)PerGroupa Represents per-channel group granularity in quantization. This granularity type calculates different quantization parameters for each group of elements. For example if the input tensor is shape [8, 16], and the group size is 4, then the input tensor is reshaped to [64, 4] quantization parameters are calculated for each group of 4 elements, giving a total of 64 quantization parameters. Attributes: group_size (int): The size of each quantization group group_sizeNrr r rrr9s% OOOOOr rc"eZdZUdZdZeed<dS)PerRowa Represents row-wise granularity in quantization. Examples: * input_tensor shape [A, B], dim 0 -> scale_shape [1, B] * input_tensor shape [A, B], dim 1 -> scale_shape [A, 1] * input_tensor shape [A, B], dim -1 -> scale_shape [A, 1] * input_tensor shape [A, B, C], dim 1 -> scale_shape [A, 1, C] Attributes: dim (int): The dim which is reduced across, all other dims are kept dimN)rr r r rrrr r rrrNs)  CMMMMMr rceZdZdZdS)PerTokena: Represents per-token granularity in quantization. This granularity type calculates a different set of quantization parameters for each token, which is represented as the last dimension of the tensor. For example, if the input tensor has shape [2, 3, 4], then there are 6 tokens with 4 elements each, and we will calculate 6 sets of quantization parameters, one for each token. If the input tensor has only two dimensions, e.g. [8, 16], then this is equivalent to `PerAxis(axis=0)`, which yields 8 sets of quantization parameters. Nrr r rr r `s   Dr r c.eZdZUdZeedfed<dS)PerBlocka' Represents multidimensional per-block granularity in quantization. Example: * block_size has shape [X, Y] * input_tensor shape [A] -> scaling undefined * input_tensor shape [A, B] -> scale shape [A // X, B // Y] * input_tensor shape [A, B, C] -> scale shape [A, B // X, C // Y] * input_tensor shape [A, B, C, D] -> scale shape [A, B, C // X, D // Y], and so on Note that `PerBlock((1, Y))` is equivalent to `PerGroup(Y)` Attributes: block_size (tuple[int, ...]): The size of each quantization group . block_sizeN)rr r r tuplerrr r rr"r"ss22c3hr r") dataclassesrtorchrrrrrr r" serializationadd_safe_globalsr r rr)s""!!!!!  $         $         $k( ${( $[" $     {   $ $     {   :$$h %BCCCCCr