Pi \ddlmZddlZddlmZddlmZmZGddejZ dS))OptionalN) linear_nf4to_nf4c eZdZdZ d dedededeejdeej f fd Z d ej d ej fd Z xZ S)FrozenNF4LinearaN A linear layer similar to ``torch.nn.Linear`` but uses a quantized NF4Tensor as its weight. This class also freezes its ``weight`` parameter and is meant to be used as the base Linear layer for modeling use cases such as QLoRA where base model parameters are frozen. Args: in_dim (int): input dimension out_dim (int): output dimension bias (bool): whether to include bias in the linear layer. Default: False device (Optional[torch.device]): device to use for the underlying weight. If ``None``, uses the default device given by `torch.get_default_device()`. dtype (Optional[torch.dtype]): dtype to use for the underlying weight. If ``None``, uses the default **quantization_kwargs: Keyword arguments to pass to `to_nf4` when quantizing the base linear weight. Examples of valid arguments are `block_size` and `scaler_block_size`, which control the granularity of weight quantization and scaler quantization respectively. This is only used if `quantize_base` is True. Default None FNin_dimout_dimbiasdevicedtypec vt||||||jd|j|jdt |jfi|}t j|jt j |ddS)N)r r r F) requires_grad) super__init__weightrequires_grad_r rtorchutils swap_tensorsnn Parameter) selfrr r r r quantization_kwargs nf4_weight __class__s ~/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/modules/low_precision/nf4_linear.pyrzFrozenNF4Linear.__init__#s tF%PPP ""5))) 9 I $ $U + + +DK??+>??     K++Je+LL     inputreturncTt||j}|j ||jz}|S)ak Runs linear operation with input tensor as given by `input`. Computation happens in higher precision, though only the nf4 weight is saved for backward for gradient computation to ensure additional memory is not used. Args: input (torch.Tensor): input tensor Returns: Tensor: output tensor )rr)rrr )rrouts rforwardzFrozenNF4Linear.forward7s0uT[999 9  /C r)FNN)__name__ __module__ __qualname____doc__intboolrrr r rTensorr" __classcell__)rs@rrrs.)-'+       &   $       (U\elrr) typingrrtorch.nnrtorchao.dtypes.nf4tensorrrLinearrrrr0s 7777777766666bi66666r