*`iXddlZddlmZmZmZgdZdefdZdedejdejfdZd ejdejfd Z d ejdejfd Z dS) N) BFLOAT16_DATA FP4_E2M1_DATAQuantizationArgs)maybe_convert_from_mxfp4_expgenerate_mxfp4_scalesround_to_power_2should_generatre_mxfp4_scalesargscD|jdko|jdko |jdkS)Nfloat )num_bitstype group_size)r s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/compressed_tensors/quantization/utils/mxfp4_utils.pyr r !s' =A  P$)w"6 P4?b;PPscalereturnc|j}t|rY|tjdz }d|tjz}||S|S)a Converts mxfp4 scales. Scales are powers of 2, with the exponents stored in uint8. Converts to dense dtype so that they can be applied to the weights and activations during QDQ :param scale: uint8 exponent scale :param dtype: dense dtype g@)dtyper totorchint32r )r roriginal_dtype scale_exps rrr%sb$T**(HHU[))C/ ek223xx''' Lrxc|jtjksJ|tjtj}dtjtjz dz z}dtj dzzdz tjz}tj ||z|}|tjtjS)a Round values to the closest power of 2. This is done by masking the values with BFLOAT16_SIGN_EXPONENT_MASK which essentially removes the mantissa and keeps the exponent. i.e the closest power of 2 for the input_value. E.g: 0.0825 = 1.32 (mantissa) x 2**-4 (exponent) 0.0825 ==> -4 (exponent) + 127 = 123 = 01111011 (8 bits for bfloat16) 0.0825 ==> 0.32 (mantissa) = 0101001 (7 bits for bfloat16) 0.0825 == 0b01111011_0101001 (bfloat16) 0b01111011_0101001 & 111111111_0000000 == 0b01111011_0000000 Keep the exponent + sign bit to give you the closest power of 2, 0.0625 :param x: tensor to round to closest power of 2 ) rrbfloat16viewuint16rrrmantissarexponent bitwise_and)rBFLOAT16_VAL_TO_ADDBFLOAT16_SIGN_EXPONENT_MASKblock_max_uints rrr8s" 7en $ $ $ $ u| ,,A 69O ORS ST }%) *a/  #  & !<N   U\ * * / / ? ??rcxt|}dtjtj|zdz S)a Generate mxfp4 scales. The scales require the following steps 1. Round to the closest power of 2 2. Convert to exponent Called when calculating qparams using observers. :param x: tensor to round to closest power of 2 :returns scales as exponents r)rrfloorlog2)r scale_power_2s rrrZs5%Q''M UZ 6677 7! ;;r) r*compressed_tensors.quantization.quant_argsrrr__all__r Tensorrrrrrr3s    Q(8QQQQ #(< \&@ @@@@@D