PizddlZddlmZmZmZddlmZGddejZGddejZdS)N)nnsinpow) Parameterc,eZdZdZ dfd ZdZxZS)Snakea Implementation of a sine-based periodic activation function Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter References: - This activation function is from this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snake(256) >>> x = torch.randn(256) >>> x = a1(x) ?TFcLtt|||_||_|jr*t t j||z|_n)t t j ||z|_||j_ d|_ dS)a  Initialization. INPUT: - in_features: shape of the input - alpha: trainable parameter alpha is initialized to 1 by default, higher values = higher-frequency. alpha will be trained along with the rest of your model. & .>N) superr__init__ in_featuresalpha_logscalertorchzerosalphaones requires_gradno_div_by_zeroselfrralpha_trainabler __class__s h/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/neucodec/activations.pyr zSnake.__init__s eT##%%%&-   D"5;{#;#;e#CDDDJJ"5:k#:#:U#BCCDJ#2  )c|jdd}|jrtj|}|d||jzz t t||zdzz}|S)u Forward pass of the function. Applies the function to the input elementwise. Snake ∶= x + 1/a * sin^2 (xa) rr )r unsqueezerrexprrr)rxrs rforwardz Snake.forward3sv  $$Q''11"55   %Ie$$E  334CE NNA8N8NN Nrr TF__name__ __module__ __qualname____doc__r r" __classcell__rs@rrr s]"LQ******2       rrc,eZdZdZ dfd ZdZxZS) SnakeBetaam A modified Snake function which uses separate parameters for the magnitude of the periodic components Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude References: - This activation function is a modified version based on this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snakebeta(256) >>> x = torch.randn(256) >>> x = a1(x) r TFctt|||_||_|jrSt t j||z|_t t j||z|_ nRt t j ||z|_t t j ||z|_ ||j_ ||j _ d|_ dS)a Initialization. INPUT: - in_features: shape of the input - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude alpha is initialized to 1 by default, higher values = higher-frequency. beta is initialized to 1 by default, higher values = higher-magnitude. alpha will be trained along with the rest of your model. r N) r r,r rrrrrrbetarrrrs rr zSnakeBeta.__init__Ss i'')))&-   C"5;{#;#;e#CDDDJ!%+k":":U"BCCDII"5:k#:#:U#BCCDJ!%*["9"9E"ABBDI#2  "1 )rcz|jdd}|jdd}|jr(t j|}t j|}|d||jzz tt||zdzz}|S)u Forward pass of the function. Applies the function to the input elementwise. SnakeBeta ∶= x + 1/b * sin^2 (xa) rrr r) rrr.rrr rrr)rr!rr.s rr"zSnakeBeta.forwardqs  $$Q''11"55y""1%%//33   #Ie$$E9T??D t223s3q5y>>17M7MM Mrr#r$r*s@rr,r,As]$LQ******<       rr,) rrrrtorch.nnrModulerr,rrr3s 55555BI555p===== =====r