PiLddlmZddlZddlmZGddejZdS))AnyN)nncTeZdZdZdededdffd ZdejdejfdZxZ S) Fp32LayerNormzZ Wrapper around :class:`~torch.nn.LayerNorm` to support mixed-precision training. argskwargsreturnNc:tj|i|dS)N)super__init__)selfrr __class__s p/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/modules/layer_norm.pyr zFp32LayerNorm.__init__s%$)&)))))xc*tj||j|j|jnd|j|jnd|j}||S)z Args: x (torch.Tensor): Input tensor. Returns: torch.Tensor: The normalized output tensor having the same shape as ``x``. N) r functional layer_normfloatnormalized_shapeweightbiasepstype_as)r routputs rforwardzFp32LayerNorm.forwards})) GGII  !#';#:DK     !%!6DIOO   D H   ~~a   r) __name__ __module__ __qualname____doc__rr torchTensorr __classcell__)rs@rrrs*c*S*T******!!%,!!!!!!!!rr)typingrr!r LayerNormrrrr'sj !!!!!BL!!!!!r