`iW5ddlmZddlmZddlZddlmcmZddlmZGddeZ Gddej Z d ej d ej d e fd ZGd dejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGdd ejZGd!d"ejZGd#d$ejZGd%d&ejZdS)')Any)BaseSparsifierN)nncjeZdZdeeefddffd ZdejdedeeefddfdZ xZ S)ImplementedSparsifierkwargsreturnNc Lt|dS)N)defaults)super__init__)selfr __class__s z/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torch/testing/_internal/common_pruning.pyr zImplementedSparsifier.__init__ s$ &)))))module tensor_namec d|jjdjd<|jd}|dddz|d<dS)Nrzlinear1.weight step_count)parametrizationsweightmaskstateget)rrrr linear_states r update_maskz!ImplementedSparsifier.update_mask sL45&q).q1z"23 %1%5%5lA%F%F%J \"""r) __name__ __module__ __qualname__dictstrrr rModuler __classcell__rs@rrr s*c3h*D******K")K#KcSVhK\`KKKKKKKKrrc<eZdZdZedejddfdZdS)MockSparseLinearz This class is a MockSparseLinear class to check convert functionality. It is the same as a normal Linear layer, except with a different type, as well as an additional from_dense method. modr c2||j|j}|S)z ) in_features out_features)clsr(linears r from_densezMockSparseLinear.from_denses$S_%'' rN)rrr __doc__ classmethodrLinearr.rrr'r'sO RY+=[rr' subset_tensorsuperset_tensorr cd}|D]M}|t|kr5tj|||s|dz }nn|t|k5dSNdS)zW Checks to see if all rows in subset tensor are present in the superset tensor rrFT)lentorchequal)r3r4irows rrows_are_subsetr;"s| A#o&&&&;sOA$677 Q #o&&&& 55 4rcHeZdZdZdfd ZdejdejfdZxZS) SimpleLinearzModel with only Linear layers without biases, some wrapped in a Sequential, some following the Sequential. Used to test basic pruned Linear-Linear fusion.r Nc lttjtjdddtjdddtjddd|_tjddd|_tjddd|_dS)NFbias )r r r Sequentialr1seqlinear1linear2rrs rr zSimpleLinear.__init__6s = Ia ' ' ' Ia ' ' ' Ia ' ' '   yAE222 yBU333 rxc||}||}||}|SN)rGrHrIrrKs rforwardzSimpleLinear.forward@4 HHQKK LLOO LLOOrr N rrr r/r r7TensorrOr$r%s@rr=r=2soVV444444%,rr=cHeZdZdZdfd ZdejdejfdZxZS) LinearBiaszModel with only Linear layers, alternating layers with biases, wrapped in a Sequential. Used to test pruned Linear-Bias-Linear fusion.r Nc Tttjtjdddtjdddtjdddtjdddtjddd|_dS) Nr?r@TrArCFrE)r r rrFr1rGrJs rr zLinearBias.__init__Ks = Ia & & & Ia ' ' ' Ia & & & Ia & & & Ia% ( ( (   rrKc0||}|SrM)rGrNs rrOzLinearBias.forwardUs HHQKKrrQrRr%s@rrUrUGsoOO      %,rrUcHeZdZdZdfd ZdejdejfdZxZS)LinearActivationzModel with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r Nc ttjtjdddtjtjdddtjtjddd|_tjddd|_tj|_ tjdd d|_ tj|_ dS) Nr?r@TrArCFrDrWrE) r r rrFr1ReLUTanhrGrHact1rIact2rJs rr zLinearActivation.__init___s = Ia & & & GII Ia ' ' ' GII Ia & & &   yAD111 GII yBU333 GII rrKc||}||}||}||}||}|SrM)rGrHr^rIr_rNs rrOzLinearActivation.forwardmR HHQKK LLOO IIaLL LLOO IIaLLrrQrRr%s@rrZrZZsoBB      %,rrZcHeZdZdZdfd ZdejdejfdZxZS)LinearActivationFunctionala,Model with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and functional activationals are called in between each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r Nc ttjtjdddtjtjdddtjtjddd|_tjddd|_tjdd d|_tjd d d|_ tj|_ dS) Nr?r@TrArCFrDrWrE) r r rrFr1r\rGrHrIlinear3r^rJs rr z#LinearActivationFunctional.__init__|s = Ia & & & GII Ia ' ' ' GII Ia & & &   yAD111 yAE222 yBU333 GII rrKc&||}||}tj|}||}tj|}||}tj|}|SrM)rGrHFrelurIrfrNs rrOz"LinearActivationFunctional.forwardsj HHQKK LLOO F1II LLOO F1II LLOO F1IIrrQrRr%s@rrcrcvsoBB      %,rrccHeZdZdZdfd ZdejdejfdZxZS) SimpleConv2dzModel with only Conv2d layers, all without bias, some in a Sequential and some following. Used to test pruned Conv2d-Conv2d fusion.r Nc Pttjtjdddddtjddddd|_tjddddd|_tjddddd|_dS) Nr rWFrA@04r r rrFConv2drGconv2d1conv2d2rJs rr zSimpleConv2d.__init__s = IaQ . . . Ib"a / / /  yRAE::: yRAE::: rrKc||}||}||}|SrMrGrsrtrNs rrOzSimpleConv2d.forwardrPrrQrRr%s@rrkrksm11;;;;;;%,rrkcHeZdZdZdfd ZdejdejfdZxZS) Conv2dBiaszModel with only Conv2d layers, some with bias, some in a Sequential and some outside. Used to test pruned Conv2d-Bias-Conv2d fusion.r Nc ttjtjdddddtjdddddtjddddd|_tjddddd|_tjdd ddd|_dS NrrmrWTrArnFrorprqrJs rr zConv2dBias.__init__s = IaQ - - - Ib"a . . . Ib"a / / /   yRAD999 yRAE::: rrKc||}||}||}|SrMrvrNs rrOzConv2dBias.forwardrPrrQrRr%s@rrxrxsm66;;;;;;%,rrxcHeZdZdZdfd ZdejdejfdZxZS)Conv2dActivationaModel with only Conv2d layers, some with bias, some in a Sequential and some following. Activation function modules in between each Sequential layer, functional activations called in-between each outside layer. Used to test pruned Conv2d-Bias-Activation-Conv2d fusion.r Nc ttjtjdddddtjtjdddddtjtjdddddtj|_tjddddd|_tjdd ddd|_ dSrz) r r rrFrrr\r]rGrsrtrJs rr zConv2dActivation.__init__s = IaQ - - - GII Ib"a . . . GII Ib"a / / / GII   yRAE::: yRAD999 rrKc||}||}tj|}||}tj|}|SrM)rGrsrhrirthardtanhrNs rrOzConv2dActivation.forwardsN HHQKK LLOO F1II LLOO JqMMrrQrRr%s@rr}r}soAA : : : : : :%,rr}cHeZdZdZdfd ZdejdejfdZxZS) Conv2dPadBiasaQModel with only Conv2d layers, all with bias and some with padding > 0, some in a Sequential and some following. Activation function modules in between each layer. Used to test that bias is propagated correctly in the special case of pruned Conv2d-Bias-(Activation)Conv2d fusion, when the second Conv2d layer has padding > 0.r Ncttjtjddddddtjtjdddddtjtjddddddtjtjddddddtjtjdddddtj |_tjdd dddd|_tj|_ tjd d dddd|_ tj|_ dS) NrrmrWT)paddingrBFrArnrorp) r r rrFrrr\r]rGrsr^rtr_rJs rr zConv2dPadBias.__init__s3 = IaQ14 8 8 8 GII Ib"a / / / GII Ib"aAD 9 9 9 GII Ib"aAD 9 9 9 GII Ib"a . . . GII   yRAqtDDD GII yRAqtDDD GII rrKc||}||}||}||}||}|SrM)rGrsr^rtr_rNs rrOzConv2dPadBias.forwardrarrQrRr%s@rrrsocc &%,rrcHeZdZdZdfd ZdejdejfdZxZS) Conv2dPoolzModel with only Conv2d layers, all with bias, some in a Sequential and some following. Activation function modules in between each layer, Pool2d modules in between each layer. Used to test pruned Conv2d-Pool2d-Conv2d fusion.r Nc ttjtjdddddtjdddtjtjdddddtjtjddd|_ tjdd ddd|_ tjddd|_ tj|_ tjd d ddd|_ tjd d ddd|_dS) NrrmrWT kernel_sizerrBrstriderrnrorp)r r rrFrr MaxPool2dr\r] AvgPool2drGrsmaxpoolaf1rtconv2d3rJs rr zConv2dPool.__init__s = IaAD A A A LQq! < < < GII Ib"!QT B B B GII LQq! < < <   yRQMMM |!QGGG 799yRQMMM yRQMMM rrKcZ||}||}||}||}||}t j|ddd}t j|}||}|SNrrr) rGrsrrrtrh avg_pool2drirrNs rrOzConv2dPool.forwards HHQKK LLOO LLOO HHQKK LLOO L!Q ? ? ? F1II LLOOrrQrRr%s@rrrss88NNNNNN  %,        rrcHeZdZdZdfd ZdejdejfdZxZS)Conv2dPoolFlattenFunctionalaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a functional Flatten followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r Nc ttjtjdddddtjdddtjtjdddddtjtjddd|_ tjddddd|_ tj|_ tjdd ddd|_ tj d |_tjd d d |_dS) NrrWTrrrr@r? )rr rA)r r rrFrrrr\r]rrGrsrrtAdaptiveAvgPool2davg_poolr1fcrJs rr z$Conv2dPoolFlattenFunctional.__init__#s  = Ia14 @ @ @ LQq! < < < GII Ia14 @ @ @ GII LQq! < < <   yA1adKKK 799yBAqtLLL ,V44 )B...rrKc\||}||}tj|ddd}||}||}||}tj|d}| |}|Sr) rGrsrh max_pool2drrtrr7flattenrrNs rrOz#Conv2dPoolFlattenFunctional.forward3s HHQKK LLOO L!Q ? ? ? HHQKK LLOO MM!   M!Q   GGAJJrrQrRr%s@rrrso@@ //////  %,        rrcHeZdZdZdfd ZdejdejfdZxZS)Conv2dPoolFlattenaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a Flatten module followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r Nc ttjtjdddddtjdddtjtjdddddtjtjddd|_ tjddddd|_ tj|_ tjdd ddd|_ tj d |_tj|_tjd d d |_dS)NrrWTrrrr@r?r)rr,rrA)r r rrFrrrr\r]rrGrsrrtrrFlattenrr1rrJs rr zConv2dPoolFlatten.__init__Es = Ia14 @ @ @ LQq! < < < GII Ia14 @ @ @ GII LQq! < < <   yA1adKKK 799yBAqtLLL ,V44 z|| )B...rrKc\||}||}tj|ddd}||}||}||}||}||}|Sr) rGrsrhrrrtrrrrNs rrOzConv2dPoolFlatten.forwardVs HHQKK LLOO L!Q ? ? ? HHQKK LLOO MM!   LLOO GGAJJrrQrRr%s@rrr?so@@ //////"  %,        rrc veZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLinearModelzCContainer module with an encoder, a recurrent module, and a linear. input_dim hidden_dim output_dim num_layersr Ncttj||||_tj|||_dSrM)r r rLSTMlstmr1r-rrrrrrs rr zLSTMLinearModel.__init__esG GIz:>> i J77 rinputcd||\}}||}||fSrM)rr-)rroutput_hiddendecodeds rrOzLSTMLinearModel.forwardls2))E**++f%%r rrr r/intr r7rStuplerOr$r%s@rrrbsMM88*-8;>8LO8 888888U\eEL%,4N.Orrc veZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLayerNormLinearModelz9Container module with an LSTM, a LayerNorm, and a linear.rrrrr Ncttj||||_tj||_tj|||_dSrM) r r rrr LayerNormnormr1r-rs rr z!LSTMLayerNormLinearModel.__init__usX GIz:>> L,, i J77 rrKc||\}}||}||}||fSrM)rrr-)rrKrs rrOz LSTMLayerNormLinearModel.forward}s<99Q<<5 IIaLL KKNN%xrrr%s@rrrrsCC88*-8;>8LO8 888888% el0J*Krr)typingrtorch.ao.pruningrr7torch.nn.functionalr functionalrhrr1r'rSboolr;r#r=rUrZrcrkrxr}rrrrrrr2rrrs7++++++ KKKKKNKKK     ry    5< %, SW     29*&ry8>29(*ry8BIDB")D        F     bi    ryr