Pi7TddlZddlmZddlmZddlmZddlmZddl m Z Gddej Z Gd d ej Z Gd d e Zd ZddZGddej ZGddej ZGddeZdZGddej ZdS)N)List)RotaryPositionalEmbeddings) ResidualFSQTransformerBlockc \eZdZdZ d dedededeffd Zdejd ejfd Z xZ S) ISTFTa Custom implementation of ISTFT since torch.istft doesn't allow custom padding (other than `center=True`) with windowing. This is because the NOLA (Nonzero Overlap Add) check fails at the edges. See issue: https://github.com/pytorch/pytorch/issues/62323 Specifically, in the context of neural vocoding we are interested in "same" padding analogous to CNNs. The NOLA constraint is met as we trim padded samples anyway. Args: n_fft (int): Size of Fourier transform. hop_length (int): The distance between neighboring sliding window frames. win_length (int): The size of window frame and STFT filter. padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same". samen_fft hop_length win_lengthpaddingct|dvrtd||_||_||_||_tj|}| d|dS)N)centerr #Padding must be 'center' or 'same'.window) super__init__ ValueErrorrr r rtorch hann_windowregister_buffer)selfr r rrr __class__s p/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/neucodec/codec_decoder_vocos.pyrzISTFT.__init__s~  , , ,BCC C  $$":.. Xv.....specreturnc|jdkr.tj||j|j|j|jdS|jdkr|j|jz dz}ntd|dks Jd|j \}}}tj ||jd d }||jd d d d fz}|d z |jz|jz}tj j |d |fd |jfd |jf d d dd|| f}|jd |dd d} tj j | d |fd |jfd |jf || } | dksJ|| z }|S)a Compute the Inverse Short Time Fourier Transform (ISTFT) of a complex spectrogram. Args: spec (Tensor): Input complex spectrogram of shape (B, N, T), where B is the batch size, N is the number of frequency bins, and T is the number of time frames. Returns: Tensor: Reconstructed time-domain signal of shape (B, L), where L is the length of the output signal. rT)rr rzExpected a 3D tensor as inputrbackward)dimnormN) output_size kernel_sizestridergdy=)rristftr r rrrr$shapefftirfftnn functionalfoldsquareexpand transposesqueezeall) rrpadBNTifftr&y window_sqwindow_envelopes rforwardz ISTFT.forward's  <8 # #;    \V # #?T_4:CCBCC CxxzzQ ?*1aytTZQZHHdk$4-001u/$/A H  $ $ K(DO,t' %  !!Q3t8   K&&((//1b99CCAqII (-22 K(DO,t' 3  '))CH  %',,.....  rr __name__ __module__ __qualname____doc__intstrrrTensorr> __classcell__rs@rr r s  LR / / /&) /7: /EH / / / / / /7EL7U\77777777rr c6eZdZdZdejdejfdZdS) FourierHeadz'Base class for inverse fourier modules.xrc td)aJ Args: x (Tensor): Input tensor of shape (B, L, H), where B is the batch size, L is the sequence length, and H denotes the model dimension. Returns: Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal. -Subclasses must implement the forward method.NotImplementedErrorrrLs rr>zFourierHead.forwardds""QRRRrNrArBrCrDrrGr>rrrKrKasH11 S S%, S S S S S SrrKc ZeZdZdZd dedededeffd Zdejd ejfd Z xZ S) ISTFTHeada ISTFT Head module for predicting STFT complex coefficients. Args: dim (int): Hidden dimension of the model. n_fft (int): Size of Fourier transform. hop_length (int): The distance between neighboring sliding window frames, which should align with the resolution of the input features. padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same". r r$r r rct|dz}tj|||_t |||||_dS)Nr!)r r rr)rrrr.Linearoutr r*)rr$r r rout_dimrs rrzISTFTHead.__init__|s\ !)8??300J5'    rrLrc||}|dd}|dd\}}tj|}tj|d}tj|}tj|}||d|zzz}||}| d|fS)ay Forward pass of the ISTFTHead module. Args: x (Tensor): Input tensor of shape (B, L, H), where B is the batch size, L is the sequence length, and H denotes the model dimension. Returns: Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal. rr!r$gY@)maxy?) rXr3chunkrexpclipcossinr* unsqueeze)rrLx_predmagpr;Saudios rr>zISTFTHead.forwards!!!!Q''aQ''Qinnj S    IaLL IaLL 1rAv:  1 q!!6))rr?r@rIs@rrUrUps    C   s      **%,********rrUc0|tj|zSN)rsigmoid)rLs r nonlinearityrks u}Q r cHtj||ddS)Nư>T) num_groups num_channelsepsaffine)rr. GroupNorm) in_channelsros r Normalizerus* 8  KT$   rc0eZdZddddfd ZddZxZS) ResnetBlockNFi) out_channels conv_shortcut temb_channelsct||_||n|}||_||_t ||_tj ||ddd|_ |dkr%tj |||_ t ||_ tj||_tj ||ddd|_|j|jkr]|jr+tj ||ddd|_dStj ||ddd|_dSdS)Nr"r)r'r(rr)rrrtrxuse_conv_shortcutrunorm1rr.Conv1dconv1rW temb_projnorm2Dropoutdropoutconv2ry nin_shortcut)rrtrxryrrzrs rrzResnetBlock.__init__sn &&2&:{{ (!.{++ X__ 1Q%    1  "X__]LIIDN|,, x''00 X__ ,Aa%     t0 0 0% %*X__1QPQ&5&&"""%*HOO1QPQ%4%%!!! 1 0rc |}||}t|}||}|3||t|ddddddfz}||}t|}||}||}|j|jkr2|j r| |}n| |}||zSri) r}rkrrrrrrtrxr|ryr)rrLtembhs rr>zResnetBlock.forwards  JJqMM OO JJqMM  DNN<#5#566qqq!!!T47GHHA JJqMM OO LLOO JJqMM  t0 0 0% )&&q))%%a((1u rri)rArBrCrr>rHrIs@rrwrwsb """""""Hrrwc6eZdZdZdejdejfdZdS)BackbonezeBase class for the generator's backbone. It preserves the same temporal resolution across all layers.rLrc td)ai Args: x (Tensor): Input tensor of shape (B, C, L), where B is the batch size, C denotes output features, and L is the sequence length. Returns: Tensor: Output of shape (B, L, H), where B is the batch size, L is the sequence length, and H denotes the model dimension. rNrO)rrLkwargss rr>zBackbone.forwards""QRRRrNrRrSrrrrsHoo S SEL S S S S S SrrcHeZdZdZd fd Zdejdejfd ZxZS) VocosBackbonea Vocos backbone module built with ConvNeXt blocks. Supports additional conditioning with Adaptive Layer Normalization Args: input_channels (int): Number of input features channels. dim (int): Hidden dimension of the model. intermediate_dim (int): Intermediate dimension used in ConvNeXtBlock. num_layers (int): Number of ConvNeXtBlock layers. layer_scale_init_value (float, optional): Initial value for layer scaling. Defaults to `1 / num_layers`. adanorm_num_embeddings (int, optional): Number of embeddings for AdaLayerNorm. None means non-conditional model. Defaults to None.  @cd ttjdd|_d|_}d}t |||j|t |||j|g}tj||_|}t|  fdt|D}tj||_ tj d |_ t |||j|t |||j|g} tj| |_dS) Nr")r'rrg?)rtrxrzrr[c4g|]}tS))r$n_heads rotary_embedr).0_heads hidden_dimtime_rotary_embeds r z*VocosBackbone.__init__..%sB     " ZT B B B $%"l     $%"l     %  x0 rrLrc|dd}||}||}|dd}||}|dd}||}|dd}||}|S)Nrr!)r3rrrrrrQs rr>zVocosBackbone.forward>s KK1   JJqMM NN1   KK1     a  KK1   MM!   KK1    ! !! $ $r)rrrr) rArBrCrDrrrGr>rHrIs@rrrsn  313131313131j  %,        rrct|tjrMtj|jdtj|jddSdS)Ng{Gz?)stdr) isinstancer.r~init trunc_normal_weight constant_biasms r init_weightsrKs]!RY% ahD111 !&!$$$$$%%rcpeZdZ dfd Zdd ZdZdZdZdZdZ dZ dZ dZ xZ S)CodecDecoderVocosrrrr@r?F@c *t||_t|gdd|_t |||||_t||jdz|jd|_| dS)N)rrrrrrrr)r$levelsnum_quantizers)rrrrrr )r$r r r) rrr r quantizerrbackbonerUheadreset_parameters)rrrrrr vq_num_quantizersvq_dimvq_commit_weightvq_weight_initvq_full_commit_loss codebook_size codebook_dimrs rrzCodecDecoderVocos.__init__Rs $$777   &!e    /A%      rTc0|durb|ddd}||\}}|ddd}|ddd}||dfS||}||\}}||fS)NTrr!r)permuterrr)rrLvqqrs rr>zCodecDecoderVocos.forwardus :: !Q""A>>!$$DAq !Q""A !Q""Aa:  MM!  yy||1!t rcv|j|_|j|}|Sri)revalvq2embrrrLs rrzCodecDecoderVocos.vq2embs1,,.. N ! !" % %rct|j|_|j}|Sri)rrget_emb)rembss rrzCodecDecoderVocos.get_embs/,,..~%%'' rcN|dddddf}||}|Srimodelrs r inference_vqzCodecDecoderVocos.inference_vqs- tQQQzN JJqMMrch||\}}}}||}|dfSri)rr)rrLrlossperps r inference_0zCodecDecoderVocos.inference_0s4>>!,,1dD JJqMM$wrc4||}|dfSrirrQs r inferencezCodecDecoderVocos.inferences JJqMM$wrc6d}||dS)z:Remove weight normalization module from all of the layers.cr tjj|dS#t$rYdSwxYwri)rr.utilsremove_weight_normrrs r_remove_weight_normzACodecDecoderVocos.remove_weight_norm.._remove_weight_normsG 11!44444    s $( 66Napply)rrs rrz$CodecDecoderVocos.remove_weight_norms,    &'''''rc6d}||dS)z9Apply weight normalization module from all of the layers.ct|tjst|tjr&tjj|dSdSri)rr.r~ConvTranspose1drr weight_normrs r_apply_weight_normz?CodecDecoderVocos.apply_weight_norm.._apply_weight_normsR!RY'' .:a9K+L+L .**1----- . .rNr)rrs rapply_weight_normz#CodecDecoderVocos.apply_weight_norms, . . . %&&&&&rc:|tdSri)rr)rs rrz"CodecDecoderVocos.reset_parameterss <     r) rrrrrrrrFFrr)T)rArBrCrr>rrrrrrrrrHrIs@rrrQs!! ! ! ! ! ! F         ( ( ('''!!!!!!!rr)rl)rtorch.nnr.typingrtorchtune.modulesrvector_quantize_pytorchr bs_roformer5rModuler rKrUrkrurwrrrrrSrrrs 888888//////******SSSSSBISSSl S S S S S") S S S1*1*1*1*1* 1*1*1*h    99999")999x S S S S Sry S S S MMMMMHMMM`%%% ^!^!^!^!^! ^!^!^!^!^!r