`iͥVddlZddlmZddlZddlmZddlmZmZddl m Z m Z e ddZ ej ejejejejgZejejgZdeDZed eDd Ze d e e d d dejdededededejdejfdZe e d ddejdededededejdejfdZe de e dd dejdejdejdededejdejfdZ e e dddejdejdejdededejdejfdZ!e de e dd dejdejdejdejdejdejdejfdZ"e e dddejdejdejdejdejdejdejfdZ#e d e e d!d dd"dejdededededejd#eejdejfd$Z$e e d!ddd"dejdejdejdededejd#eejdejfd%Z%e d&e e d'd dd"dejdejdejdededejd#eejdejfd(Z&e e d'ddd"dejdejdejdededejd#eejdejfd)Z'e d*e e d+d dd"dejdejdejdejdejdejd#eejdejfd,Z(e e d+ddd"d#eejdejfd-Z)e d.e e d/d dejd0ed1ed2edejde*ejejff d3Z+e d4e e d5d dejd0ed1ed2edejde*ejejff d6Z,e e d/ddejdeded2edejde*ejejff d7Z-e e d5ddejdeded2edejde*ejejff d8Z.d9Z/e d:e e d;d dejdedededejdejfd?Z0e e d;ddejdedededejdejfd@Z1e dAe e dBd dd"dejdedededejd#eejdejfdCZ2e e dBddd"dejdedededejd#eejdejfdDZ3e dEe e dFd dejdejde*ejejffdGZ4e e dFddejdejde*ejejffdHZ5e dIe e dJdKdejdejde*ejejffdLZ6e dMe e dNd dejdejde*ejejffdOZ7e e dNddejdejde*ejejffdPZ8dQZ9e dRe e dSd dejde d[e e d\d  dpdejdedededejfdjZEe e dhddejdedededejfdkZFe dle e dmd dejdejdejfdnZGe e dmddejdejdejfdoZHdS)qN)Optional)_unsqueeze_multiple)determine_qparamsvalidate_qmin_qmax)implLibraryquantized_decomposedDEFcti|]5}|tj|jtj|jf6S)torchiinfominmax.0ks x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torch/ao/quantization/fx/_decomposed.py rsB45A AEKNN./c i|]O}|ttj|jttj|jfPSr )intr finforrrs rrrsERRRqQU[^^  #ek!nn&8"9"9:RRRrc|tvrtd|t|\}}||ksJd|d|||ksJd|d|dS)NzUnsupported dtype: z9quant_min out of bound for dtype, quant_min_lower_bound: z quant_min: z9quant_max out of bound for dtype, quant_max_upper_bound: z quant_max: )_DTYPE_TO_QVALUE_BOUNDS ValueError) quant_min quant_maxdtypequant_min_lower_boundquant_max_upper_bounds r_quant_min_max_bounds_checkr"s +++6u667773J53Q00 - - - - Q"7 Q QEN Q Q . - - - - - - Q"7 Q QEN Q Q . - - - -rzxquantize_per_tensor(Tensor input, float scale, int zero_point, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_tensorCompositeExplicitAutogradinputscale zero_pointrrrreturncz|jtjtjfvr|tj}|jtjksJd|jt |||d|z }tjtj||z|z|||S)aAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead Tensorzquantize_per_tensor.tensorcN|dksJd||dksJd|t||||||SzAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Same as `quantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values >Expecting zero_point tensor to be one element, but received : 9Expecting scale tensor to be one element, but received : numelr#itemr7s rquantize_per_tensor_tensorrAms      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS         rc|jtjtjfvr|tj}|dksJd||dksJd||jtjksJd|jtj||S)Nr;r<r=r*r5)rr r,r-r.r/r?r6r7s rquantize_per_tensor_tensor_metarCs {u}en555''      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS    ;%- ' ' 'Tu{TT ( ' '  E / / //rzquantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype) -> Tensorzquantize_per_tensor.tensor2c|dksJd||dksJd|t||||||Sr:r>r7s rquantize_per_tensor_tensor2rEs      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS         rc*t||||||SN)rCr7s r quantize_per_tensor_tensor2_metarHs( +     rzdequantize_per_tensor(Tensor input, float scale, int zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_tensor out_dtyperKc|j|ksJd|d|j| tj}|tvr|||z |zSt d|)aAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per tensor quantized Tensor if combined with quantization parameters in the argument of this function (scale/zero_point) scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for input Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for input Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): dtype for input Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor Expecting input to have dtype: z , but got N,Unsupported dtype in dequantize_per_tensor: )rr r/rr.rr%r&r'rrrrKs rrIrIsJ ;%   H%HH5;HH  M  '''##j0E99OOOPPPrcJ| tj}tj||SNr5)r r/r6rOs rdequantize_per_tensor_metarRs'M  E 3 3 33rzdequantize_per_tensor.tensor(Tensor input, Tensor scale, Tensor zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensorc R|dksJd||dksJd|t|||||||SzAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Same as `dequantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values r;r<r=rJr?rIr@rOs rdequantize_per_tensor_tensorrV1s(      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS    !      rc| tj}|dksJd||dksJd||j|ks Jd||tvrtj||St d|)Nr;r<r=rMr5rN)r r/r?rrr6rrOs r!dequantize_per_tensor_tensor_metarXVsM      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS    ;%   !J5!J!J    '''Y7777OOOPPPrzdequantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensor2c |dksJd||dksJd|t|||||||SrTrUrOs rdequantize_per_tensor_tensor2rZws(      " " "]IYIYI[I[]] # " " ;;==A   SEKKMMSS    !      rc .t|||||||S)NrJ)rXrOs r"dequantize_per_tensor_tensor2_metar\s* - uj)Y   rzrchoose_qparams.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams.tensorqminqmaxepsc z|jtjtjtjfvsJd|j|t vs'Jdt d|t||tj|\}}t|||||tj |gdS)[Given an input Tensor, derive the per tensor affine quantization parameter (scale and zero_point) for target quantized Tensor from the Tensor Args: input (torch.Tensor): floating point input Tensor quant_min (int): minimum quantized value for target quantized Tensor quant_max (int): maximum quantized value for target quantized Tensor dtype (torch.dtype): dtype for target quantized Tensor Returns: scale (float): quantization parameter for the target quantized Tensor zero_point (int): quantization parameter for the target quantized Tensor CExpecting input to have dtype torch.float32/16/b16, but got dtype: $Expecting target dtype to be one of , but got: F)has_customized_qrange) rr r/r,r-rkeysraminmaxrTensorr%r]r^r_rmin_valmax_vals rchoose_qparams_tensorrls" ;       \ek[[     + + + +a/F/K/K/M/MaaZ_aa , + +tT"""}U++GW     cU#   rz|choose_qparams_symmetric.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams_symmetric.tensorc |jtjtjtjfvsJd|j|t vs'Jdt d|t||tj|\}}t|||||tj |gdtj S)rarbrcrdF)reqscheme) rr r/r,r-rrfrrgrrhper_tensor_symmetricris rchoose_qparams_symmetric_tensorrps* ;       \ek[[     + + + +a/F/K/K/M/MaaZ_aa , + +tT"""}U++GW     cU#*    rc>|jtjtjtjfvsJd|j||ksJd|d|tjdtj|jtjdtj|jfS)NrbzKExpecting quant_min to be smaller than quant_max but received min: z max: r;rdevice) rr r/r,r-emptydoublersint64r%rrr_rs rchoose_qparams_tensor_metarxs ;       \ek[[     y  &  & &# & & !  ;q U\ B B BEK U\EEE rctjdtj|jtjdtj|jfS)Nr;rr)r rtrursrvrws r$choose_qparams_symmetric_tensor_metarz$sG ;q U\ B B BEK U\EEE rctt|}d||<||d<|t |}||fS)Nr)listrangedimpermutetuple)xaxis new_axis_listys r_permute_to_axis_zeror.sUquuww((MM$M! % &&''A m rzquantize_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_channelscales zero_pointsrc|jtjtjfvr|tj}|jtjksJd|j||ksJd|t|||t||\}}dg|z}|j d|d<| |}| |}tj tj |d|z z|z||} | t|} | |S)atAffine per channel quantization for the Tensor using the same quantization parameters for each channel/axis to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_point (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r*Expecting axis to be < r;rr+)rr r,r-r.r/r~r"rshapeviewr0r1rr) r%rrrrrrpermute_axis_list new_shaperesouts rrr<sU6 {u}en555'' ;%- ' ' 'Tu{TT ( ' ' %))++   FFF    9e<<<4UDAAE eiikk!I<?IaL [[ # #F""9--K + ES6\*++k99i  C ++e-.. / /C 66%==rc|jtjtjfvr|tj}|jtjksJd|j||ksJd|t|||tj||S)Nr*rr5) rr r,r-r.r/r~r"r6)r%rrrrrrs rquantize_per_channel_metarls {u}en555'' ;%- ' ' 'Tu{TT ( ' ' %))++   FFF    9e<<<  E / / //rzdequantize_per_channel(Tensor input, Tensor scales, Tensor? zero_points, int axis, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_channelcF|j|ksJd|d|j| tj}||ksJd|t |||t ||\}}dg|z} |jd| d<|| }|||| z |z} n||z} | |} | t|} | S)aAffine per channel dequantization for the Tensor using the same quantization parameters for each channel/axis to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per channel quantized Tensor if combined with quantization parameter in the argument of this function (scales/zero_points/axis) scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_points (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for output Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): requested dtype for output Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor Expecting input to have dtype , but got dtype: Nrr;r) rr r/r~r"rrrr.rr) r%rrrrrrrKrrrrs rrrs/P ;%   NNNNN  M %))++   FFF    9e<<<4UDAAE eiikk!I<?IaL [[ # #F{'' 222f<fn &&  C ++e-.. / /C Jrc|j|ksJd|d|j| tj}||ksJd|t |||tj||S)Nrrrr5)rr r/r~r"r6)r%rrrrrrrKs rdequantize_per_channel_metars ;%   NNNNN  M %))++   FFF    9e<<<  E 3 3 33rzLchoose_qparams_per_token(Tensor input, ScalarType dtype) -> (Tensor, Tensor)choose_qparams_per_tokenc|dd}|jtjkr|}|tjkrd}d|dz zdz }ntd||d  |}tj |}||fS) Choose quantization parameters for per token quantization. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32/float16 Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor Returns: scales and zero_points, both float32 Tensors Tr~keepdimr;z/unsupported dtype in choose_qparams_per_token: gh㈵>r) absamaxrr r,floatint8 Exceptionr0div zeros_like)r%rrn_bitsrrs rrrs,YY[[  "d  3 3F |u}$$ LLNN   &1*%)  Ee E E   \\d\ # # ' ' 2 2F"6**K ; rct|jdddgz}tj|tj|jtj|tj|jfSNrr;rrr|rr rtrursrvr%rsizes rchoose_qparams_per_token_metar f  CRC ! !QC 'D ;t5< E E Eu{ EK HHH rz]_choose_qparams_per_token_asymmetric_impl(Tensor input, ScalarType dtype) -> (Tensor, Tensor))_choose_qparams_per_token_asymmetric_implCompositeImplicitAutogradcd\}}tj|dd}tj|dd}tj|tj|}tj|tj|}tjtjj}||z t||z z } | |} || z } || z } || z} || z} tj | | zdk|| z || z }tj ||| }| tj| tjfS)r)irTrrr)r aminrrrrrr/r_rr0wherer1r.float64rv)r%rr]r^rjrk min_val_neg max_val_posr_r& descaled_min descaled_maxzero_point_from_min_errorzero_point_from_max_errorr's rrr sX,JD$jB555GjB555G)GU%5g%>%>??K)GU%5g%>%>??K +em $ $ (C; &%t *<*< $> >>rzWchoose_qparams_per_token_asymmetric(Tensor input, ScalarType dtype) -> (Tensor, Tensor)#choose_qparams_per_token_asymmetricc"t||SrG)rr%rs rrrUs 5UE B BBrct|jdddgz}tj|tj|jtj|tj|jfSrrrs r(choose_qparams_per_token_asymmetric_metararrcdtjt|dd}||ks"Jd|d|||ks"Jd|d|dS)Nrz num_tokens: z scales: z zero_points: )mathprodr|rr?)r%rr num_tokenss r!_per_token_quant_qparam_dim_checkrps4 --crc233J  ' ' ';z;;FKKMM;; ( ' ' **,, , , ,EzEE1A1A1C1CEE - , , , ,rz}quantize_per_token(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_tokenct|||t||||d|z ||||}|S)aPer token quantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per token affine quantization zero_points (int32 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r+)r"rmuladdr1r0r.r%rrrrrs rrrsw6 9e<<<%eV[AAA #, [    y) $ $ E   LrcPt|||tj||SrQr"r r6rs rquantize_per_token_metars, 9e<<<  E / / //rzdequantize_per_token(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, ScalarType output_dtype) -> Tensordequantize_per_token output_dtypec@||z }||z}||S)aPer token dequantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): quantized Tensor (uint8, int8 etc.) scales (float64 torch.Tensor): quantization parameter for per token affine quantization zero_points (int64 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` )r.r%rrrrrrs rrrs)8 K E FNE 88L ! !!rcPt|||tj||SrQrrs rdequantize_per_token_metars, 9e<<<  E 6 6 66rzquantize_per_channel_group(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size) -> Tensorquantize_per_channel_groupc|dksJ||jdkr|jddkr |jd}|jd|zdksJ|dksJ|d|}tj|dksJ|dd}|dd}|d|z | || | |}|S)Nr;rrrr+) rr~reshaper isnansumrrr1clamp_r. reshape_as) r%rrrrr group_sizeto_quant input_int8s rrrs6 >>>>EKO## R(8A(=(=[_ ;r?Z '1 , , , , 99;;!    }}R,,H ;x $ $ & &! + + + + ^^B " "F%%b!,,K  S6\"" [     9 % % E E    rc|dksJ||jdkr|jddkr |jd}|jd|zdksJ|dksJtj||S)aXGroupwise quantization within each channel for an 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r;rrrr5)rr~r r6)r%rrrrrrs rquantize_per_channel_group_metars8 >>>>EKO## R(8A(=(=[_ ;r?Z '1 , , , , 99;;!      E / / //rzdequantize_per_channel_group(Tensor input, Tensor scales, Tensor? zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size, ScalarType output_dtype) -> Tensordequantize_per_channel_groupw_int8rcF|dksJ||jdkr|jddkr |jd}|jd|zdksJ|dksJ|d|}|dd}||dd} n&tjgtj|j} || | | |} | S)a!Groupwise dequantization within each channel for an 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): quantized Tensor (uint8/int8 etc.) scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` r;rrrNrr) rr~rr zerosint32rssubrrr.) rrrrrrrrw_int8_groupedzpw_dqs rrrEsD >>>>FL$$$b)9Q)>)>\"% < j (A - - - - ::<<1    ^^B 33N ^^B " "F  Q ' ' [5;v} E E E   b ! ! % %f - - 8 8 @ @ C CL Q QD Krzyfake_quant_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max) -> Tensorc:eZdZedZedZdS)FakeQuantPerChannelc.|jtjkr|tj}|jtjkr|tj}|jtjksJd|j||ksJd|t td|t t|dz|jz}t||}t||} tj |d|z z| z} tj | ||| z |z} tj | |k| |k} | | | S)Nr*rrr;r+)rr r/r.rr~r|r}ndimrr1r0 logical_andsave_for_backward) ctxr%rrrrrbroadcast_dimsunsqueeze_scalesunsqueeze_zero_pointstemprmasks rforwardzFakeQuantPerChannel.forwardso <5= ( (YYu}--F   + +%..55K{em+++ X5; X X,++eiikk!!!#JUYY[[#J#J!!!eAtnn--U4!8UZ5P5P0Q0QQ.v~FF 3K P P{5C*:$:;< Tensorzconvert_element_type.no_fusecVtjjj||SrG)r opsprimsconvert_element_typedefaultrs rr r s! 9? / 7 7u E EErc.tj||SrQrrs rconvert_element_type_metars  E / / //r)r)Irtypingrr torch._refsrtorch.ao.quantization.utilsrr torch.libraryrrquantized_decomposed_libuint8ruint16int16r_INTEGER_DTYPES float8_e5m2 float8_e4m3fn _FLOAT_DTYPESrupdater"definerhrrrr#r8rArCrErHrIrRrVrXrZr\rrlrprxrzrrrrrrrrrrrrrr/rrrrrautogradFunctionrrrr rr rrr s  ++++++MMMMMMMM'''''''' #7#95AA; EL%+u{S"E$78 9HRRMRRR    @  57RSS! <! !! !  ! ; ! \!!!TS!H 5v>> 0 < 0  0 0 0  0 ; 0 \ 0 0 0?> 0 @ :GG(,QQQ <Q <Q Q Q  Q ; Q $Q \QQQHGQ4e #(, < < |  |  ;  $ \  @ ?HH(,    $  \   IH 7  79TUU& <&"&*-&49&BG+& 5< %&&&&VU&R7 % & <&"&*-&49&BG+& 5< %&&&&  &R 7@@ <$'47>CLQK 5< %&A@& A6JJ <$'47>CLQK 5< %&KJ@  68STT, <, L,,  ,  ,  , ;, \,,,UT,^ 6??0 <0 L00  0  0  0 ;0 \000@?0._  8:UVV(,;;; <; L;%,';  ;  ;  ; ;; $; \;;;WV;| 8&AA(,444 <4 L4%,'4  4  4  4 ;4 $4 \444BA4*R   <  ;  5< %&     F   < ; 5< %&  c / (? <(? ;(? 5< %&(?(?(?  (?V] ) C <C ;C 5< %&CCC  C)   < ; 5< %&  @  46QRR# <# L## #  # ; ###SR#L 4f== 0 < 0 L 0 0 0  0 ; 0 0 0>= 0Y  68STT!& "" <" L"" "  " ; "+"""UT"B 6??!&  7 7 < 7 L 7 7 7  7 ; 7+ 7 7 7@? 7A:GG0U\0%+0%,000HG000r