§ †Pƒiáãó<—dZddlmZddlZddlmZddlmZddlm Z m Z m Z m Z ddl mZde jdejddfd ejd ejd e d ed e dedeedede fd„Zdededeefd„ZejjGd„dejj¦«¦«ZdS)z9 Utilities for scaling high precision tensors to float8. é)ÚOptionalN)ÚScalingGranularity)Útensor_already_casted_to_fp8)ÚFloat8TrainingTensorÚ GemmInputRoleÚLinearMMConfigÚhp_tensor_and_scale_to_float8)Útensor_to_scaleFÚ hp_tensorÚ float8_dtypeÚlinear_mm_configÚ reduce_amaxÚgemm_input_roleÚscaling_granularityÚ axiswise_dimÚround_scales_to_power_of_2Úreturnc óT—t|||||||¦«} t|| ||||¦«S)a6 Given a high precision tensor `hp_tensor`, scales `hp_tensor` dynamically and returns a `Float8TrainingTensor` of the result. Args: hp_tensor: the tensor to convert float8_dtype: the float8 dtype to use linear_mm_config: Defines the configuration for the scaled_mm for the 3 fwd/bwd gemms of linear reduce_amax: whether to reduce the max(abs(hp_tensor)) value across distributed ranks gemm_input_role: Defines the role of this tensor (input, weight or grad_output) in the 3 fwd/bwd gemms of linear scaling_granularity: Defines the scaling granularity axiswise_dim: if axiswise granularity is used, defines the dim to scale across round_scales_to_power_of_2: if true, round scaling factor down to the nearest power of 2. )r r ) r r r rrÚ device_meshrrrÚscales úw/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/float8/float8_scaling_utils.pyÚhp_tensor_to_float8_dynamicrsN€õ6 ØØØØØØØ"ñ ô €Eõ )ØØ ØØØØñ  ô ðócó&—|tjur|SdS)a2 Convenience function which takes in an axiswise dim which is only relevant for axiswise scaing, and a scaling type. The output is pass-through if scaling type is axiswise, and None otherwise. This is done to keep the logic from choosing the axiswise dim out of the scaling function. N)rÚAXISWISE)rrs rÚget_maybe_axiswise_dimrKs€ðÕ0Ô9Ð9Ð9ØÐØ ˆ4rcóR—eZdZdZededejfd„¦«Zed„¦«Z dS)ÚNoopFwToFloat8BwDynamiczR Forward: no-op Backward: convert to float8_e5m2 with dynamic scaling r Ú target_dtypecó"—||_||_|S©N)r r)ÚctxÚtensorr rs rÚforwardzNoopFwToFloat8BwDynamic.forwardas€ð 0ˆÔØ'ˆÔ؈ rcó¬—t|¦«r|ddfSt||j¦«}t|||j|jt j¦«}|ddfSr!)rr rr r rÚ GRAD_OUTPUT)r"ÚgradYÚ gradY_scaleÚ fp8_tensors rÚbackwardz NoopFwToFloat8BwDynamic.backwardlsg€å '¨Ñ .Ô .ð %ؘ$ Ð$Ð $Ý% e¨SÔ-=Ñ>Ô>ˆ Ý2Ø Ø Ø Ô Ø Ô Ý Ô %ñ  ô ˆ ð˜4 Ð%Ð%rN) Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú staticmethodrÚtorchÚdtyper$r*©rrrrZso€€€€€ððð ðð)ðð”kð ððñ„\ððð &ð &ñ„\ð &ð &ð &rr)r.Útypingrr0Útorchao.float8.configrÚ torchao.float8.distributed_utilsrÚ%torchao.float8.float8_training_tensorrrrr Útorchao.float8.float8_utilsr ÚINPUTÚ TENSORWISEÚTensorr1ÚboolÚintrrÚ_dynamoÚallow_in_graphÚautogradÚFunctionrr2rrúrAs¹ððððÐÐÐÐÐà € € € à4Ð4Ð4Ð4Ð4Ð4ØIÐIÐIÐIÐIÐIððððððððððððð ððððððØ%2Ô%8ØØ.@Ô.KØ"&Ø',ð+ð+ØŒ|ð+à”+ð+ð%ð+ðð +ð #ð +ð,ð+ð˜3”-ð+ð!%ð+ðð+ð+ð+ð+ð\ Øð à+ð ðˆc„]ð ð ð ð ð„Ôð&ð&ð&ð&ð&˜eœnÔ5ñ&ô&ñÔð&ð&ð&r