Pi lddlZddlmZddlmcmZGddejZGddZdS)Nc6eZdZdZdejdejfdZdS)Lineara nn.Module used in :func:`~torchtune.modules.tied_linear.TiedLinear`, added to work with the hooks :class:`~torchtune.training._activation_offloading.NoOpManager` that ignore activation offloading context manager. Without this class, we can't add NoOp hooks, and we will offload the activation of the tied linear layer, which is slow. For more information, see how NoOpManager is called in the recipes. xweightc,tj||S)N)Flinear)selfrrs q/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/modules/tied_linear.pyforwardzLinear.forwardsx6"""N)__name__ __module__ __qualname____doc__torchTensorr r r rr sD  ##u|######r rcLeZdZdZdejfdZdejdejfdZ dS) TiedLinearaz A tied linear layer, without bias, that shares the same weight as another linear layer. This is useful for models that use tied weights, such as :func:`~torchtune.models.qwen2_0_5b`, :func:`~torchtune.models.qwen2_1_5b` and all of the :func:`~torchtune.models.gemma` and :func:`~torchtune.models.llama3_2` models. It requires as input an nn.Module, instead of the weight of the module, so it can work with FSDP. When FSDP is applied, the memory pointer to the weight is different, but the nn.Module remains the same. This is why we need to pass the nn.Module instead of the weight, if we want to keep the weights tied. Args: tied_module (nn.Module): The module whose weight is shared. Only the weight is used. The bias is ignored. Raises: AttributeError: If the provided module does not have an attribute 'weight'. tied_modulecx||_t|_t|dst ddS)NrzPProvided module does not have attribute 'weight'. Please check your tied_module.)rrr hasattrAttributeError)r rs r __init__zTiedLinear.__init__/sF&hh {H--  b   r rreturncB|||jjS)aa Args: x (torch.Tensor): Input tensor. Should have shape ``(..., in_dim)``, where ``in_dim`` is the input dimension of the tied module. Returns: torch.Tensor: The output tensor, having shape ``(..., out_dim)``, where ``out_dim`` is the output dimension of the tied module. )r rr)r rs r __call__zTiedLinear.__call__7s{{1d.5666r N) rrrrnnModulerrrrrr r rrsa$BI 7%, 75< 7 7 7 7 7 7r r) rtorch.nnrtorch.nn.functional functionalrr rrrr r r$s  # # # # #RY # # # $7$7$7$7$7$7$7$7$7$7r