`i;ddlZddlmZddlmZmZddlZddlm Z ddl m Z ddl m Z mZmZejeZ dded ed ed ed ed edeedejfdZdejdedededejf dZGdde ZdgZdS)N)Sequence)OptionalUnion)SequenceFeatureExtractor) BatchFeature)PaddingStrategy TensorTypeloggingn_freqsf_minf_maxn_mels sample_rate fft_lengthnormreturnc2||dkrtdtj|tj||z z}dt jd|dz zz}dt jd|dz zz} tj|| |dz} dd | dz zdz z} | d d| dd z } tj| d tj|d z } tjd tj}d | ddddfz| dd z }| ddddf| d dz }tj |tj ||}|9|dkr3d| d|dz| d|z z }|tj|d z}|S)aCreate a frequency bin conversion matrix (NumPy version). Args: n_freqs (int): Number of frequencies to highlight/apply f_min (float): Minimum frequency (Hz) f_max (float): Maximum frequency (Hz) n_mels (int): Number of mel filterbanks sample_rate (int): Sample rate of the audio waveform fft_length (int): FFT length norm (Optional[str]): If 'slaney', divide the triangular mel weights by the width of the mel band (area normalization). (Default: ``None``) Returns: np.ndarray: Triangular filter banks (fb matrix) of size (``n_freqs``, ``n_mels``) meaning number of frequencies to highlight/apply to x the number of filterbanks. Each column is a filterbank so that assuming there is a matrix A of size (..., ``n_freqs``), the applied result would be ``A @ create_fb_matrix_numpy(A.shape[-1], ...)``. Nslaneyz$norm must be one of None or 'slaney'dtypegF@?g@ rgg@) ValueErrornparangefloat32mathlog10linspace expand_dimszerosmaximumminimum)r r rrrrr all_freqsm_minm_maxm_ptsf_ptsf_diffslopeszero down_slopes up_slopesfbenorms /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/gemma3n/feature_extraction_gemma3n.pycreate_fb_matrixr6s> DH,,?@@@ '444 j8PQI TZuu} 566 6E TZuu} 566 6E Kufqj 1 1E REFN+c1 2E 122Yss #F ^E1 % %y!(D(D DF 8ARZ ( ( (D&CRC.(F3B3K7Kqqq!""u qrr *I D"*[)<< = =B DH,,uQ!^,uWfW~=> bnUA&&& Iarray dimensionsizestepc|jdkrtd|dkr||jdz krtd|j\}}||z |zdz}|dkrtj|d|f|jS|||f}|jd|jd|z|jdf}tjj |||S) zNA basic NumPy equivalent of PyTorch's unfold for 2D arrays along the last dim.rzFThis unfold implementation currently supports 2D arrays (batch, time).rrzFThis unfold implementation only supports unfolding the last dimension.rr)shapestrides) ndimrr=rr&rr>lib stride_tricks as_strided) r8r9r:r; batch_sizeoriginal_length num_frames output_shapeoutput_stridess r5_unfoldrH[s zQabbbB9 Q66abbb"'+J!D(T1A5JQxQ-U[AAAA D1LmA& a(84(?qAQRN 6  * *5 n * ] ]]r7c#eZdZdZddgZ d.dededededededededededededededee edee ef fd Z d!e j d"e j d#e e j e j ffd$Z d/d'ee j eeee j eeefd(eeeefd)eed*ed+eed,eeeefdeed#efd-ZxZS)0Gemma3nAudioFeatureExtractoraT An audio feature extractor Universal Speech Models https://huggingface.co/papers/2303.01037. Args: feature_size (`int`, *optional*, defaults to 128): The feature dimension of the extracted features. sampling_rate (`int`, *optional*, defaults to 16000): The sampling rate at which the audio files should be digitalized expressed in hertz (Hz). padding_value (`float`, *optional*, defaults to 0.0): Padding value used to pad the audio. Should correspond to silences. return_attention_mask (`bool`, *optional*, defaults to `True`): Whether to return the attention mask for the generated MEL spectrograms. frame_length_ms (`float`, *optional*, defaults to 32.0): The length of a frame in milliseconds. hop_length_ms (`float`, *optional*, defaults to 10.0): Length of the overlapping windows for the STFT used to obtain the Mel Frequency coefficients. min_frequency (`float`, *optional*, defaults to 125.0): The minimum frequency (in Hz) for the Mel filterbank. max_frequency (`float`, *optional*, defaults to 7600.0): The maximum frequency (in Hz) for the Mel filterbank. preemphasis (`float`, *optional*, defaults to 0.97): The preemphasis coefficient. preemphasis_htk_flavor (`bool`, *optional*, defaults to `True`): Whether to use HTK-style preemphasis. fft_overdrive (`bool`, *optional*, defaults to `True`): Whether to use FFT overdrive. dither (`float`, *optional*, defaults to 0.0): Adds dithering. In other words, adds a small Gaussian noise to each frame. E.g. use 0.0001 to add dithering with a normal distribution centered around 0.0 with standard deviation 0.0001 (assuming [-1,+1] range of raw_speech). The value 0.0 means no dithering. Dithering has similar effect as `spectrogram(mel_floor=...)`. It reduces the high log_mel_fbank values for signals with hard-zero sections, when VAD cutoff is present in the signal. input_scale_factor (`float`, *optional*, defaults to 1.0): Scaling factor applied to the input waveform. mel_floor (`float`, *optional*, defaults to 1e-05): Minimum value for Mel spectrograms to avoid log(0). per_bin_mean (`Optional[Sequence[float]]`, *optional*): Mean values for per-bin normalization. per_bin_stddev (`Optional[Sequence[float]]`, *optional*): Standard deviation values for per-bin normalization. input_featuresinput_features_mask>T@@$@@_@@ ףp= ?rh㈵>N feature_size sampling_rate padding_valuereturn_attention_maskframe_length_ms hop_length_ms min_frequency max_frequency preemphasispreemphasis_htk_flavor fft_overdriveditherinput_scale_factor mel_floor per_bin_meanper_bin_stddevc <tjd||||d|||_||_| |_| |_| |_| |_| |_tt||zdz |_ tt||zdz |_ tj|tj|_dt#jt#j|j z}|jr|dz}||_tj|j tj}ddtjdtjz|z|j z z z}|tj|_t7|jdzdz||||jd||_|/tj|dd||_nd|_|0tj|dd||_ dSd|_ dS) N)rVrWrXrYg@@rrg?r)r r rrrrr)!super__init__r\r]r^r_r`rarbintround frame_length hop_lengthrr8float64rcr"ceillog2rr r!cospiastypewindowr6rW mel_filtersreshaperdre)selfrVrWrXrYrZr[r\r]r^r_r`rarbrcrdrekwargsr hann_arangert __class__s r5riz%Gemma3nAudioFeatureExtractor.__init__s(  %''"7       +*&&<#* "4mo&E&N O OPPeMM$AF$JKKLL)2:>>>$)DId.?$@$@AAA    !OJ$i 1DDD BF1ru9{#:T=N#NOOOPmmBJ// +Oq(1,*!     # " 6 6 > >q!\ R RD   $D   %"$(>":":"B"B1a"V"VD   "&D   r7waveformattention_maskrcb|jdkrtj|d}|jdkr;||jtjj|j|jzz}|j dkr ||j z}|j dz}t|d||j }|j dkrz|jrQ|dd dfd|j z z}|dddf|j |dd d fzz }tj||gd}n.|ddd f|j |dd dfzz }n |dd df}||jz}tj||jd }tj|} tj| |j} tjtj| |j} |j | |jz } |j | |jz } | d} |d d |j t:} | | d | jdfS) rr)axisrOrr)r9r:r;.Nr)nr)r?rr%rarandomrandnr=rsrrbrlrHrmr^r_ concatenatertfftrfftrabsmatmulrulogr'rcrdresqueezebool)rwr{r|frame_size_for_unfoldframes_to_processfirst_in_frame rest_in_frameframesstftmagnitude_specmel_spec log_mel_specmel_spectrogrammasks r5_extract_spectrogramz1Gemma3nAudioFeatureExtractor._extract_spectrogramsV =A  ~hQ777H ;  $+ 0P0W0WX`Xf0g0g"ggH  "c ) )$"99H $ 1A 5$HAV]a]lmmm  c ! !* e!237!;sTEU?U!V 1#qt) >9^T-=>>vbj4>BBCC   ('$*;;L   *'$*==L&..q11000188>>%?'z9Gemma3nAudioFeatureExtractor.__call__..5s'BBB"*bT**,BBBr7rK)rrrrrY)rKrL) tensor_type) isinstancerndarraylenr=rrpadrziprKr|rrappendrsr!)rwrrrrrrrYrxis_batched_numpyis_batched_sequence is_batchedbatched_speechprepared_speechprepared_speech_maskspeechrs r5__call__z%Gemma3nAudioFeatureExtractor.__call__ sF&j"*==[#jFVBWBWZ[B[(X>>t:jYZm^`^hjr]sCtCt%<)<  0BBzBBBJJ 0Jz2:$F$F 0J//J 4*j\223J *J7 8 8!!1"7 "  ! =~?\]] . .LFD44VXtDDLFD  " "6==#<#< = = = ' ' - - - -.G[ \ \&    r7)rMrNrOTrPrQrRrSrTTTrOrrUNN)rrTrMNT)__name__ __module__ __qualname____doc__model_input_namesrjfloatrrrrirrtuplerrliststrr r rr __classcell__)rzs@r5rJrJnss))V*+@A #"&*!%#$%!'+"$'2648#B'B'B'B' B' $ B'  B'B'B'B'B'!%B'B'B'"B'B' x/!B'"!%1#B'B'B'B'B'B'H+ARZ+A+AX]^`^hjljt^tXu+A+A+A+A`6?$+,/;?04B B "*d5k4 3CT$u+EVVWB tS/12B SM B  B %SM B !sJ!78B  (~B  B B B B B B B B r7rJ)N)r"collections.abcrtypingrrnumpyr!feature_extraction_sequence_utilsrfeature_extraction_utilsrutilsr r r get_loggerrloggerrjrrrr6rHrJ__all__rgr7r5rs  $$$$$$""""""""IIIIII4444449999999999  H % %:: : : :  :  :  : 3-:Z::::z^2:^#^S^^ ^^^^&a a a a a #;a a a H * *r7