.`i*"ddlZddlZdedededefdZdejdejeeeefzd ededejf d Z ddejd ed ed edejf dZ dejde ejdejdededededeejeffdZ dS)Ntokens_per_frame num_framesqreturncZ||z}t|d|z z}|}t||S)as Compute the number of retained tokens for a given video. Method ensures that we retain all the tokens from the first frame regardless of the pruning rate. Args: tokens_per_frame: The number of tokens per frame. num_frames: The total number of frames. q: The pruning rate. Returns: The number of retained tokens. )intmax)rrr total_tokensevs_num_tokensmin_num_tokenss g/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/multimodal/evs.pycompute_retained_tokens_countrs9 $j0LQ/00N%N ~~ . .. video_embedsvideo_size_thwspatial_merge_sizec tt|\}}}||||z||z|d}||z||zz}tjj|dddf|dddfd}d|z } t jdt j |dddddddfz| gd} | d} t j | ddd } t||| } | d| } t j | tj }d|| <|| }| d}|S) aP Computes the retention mask for input video embeddings. Args: video_embeds (`torch.Tensor`): The input video embeddings of shape `(T * H * W // spatial_merge_size ^ 2, hidden_size)` video_size_thw (`torch.LongTensor` of shape `(3)`): The temporal, height and width of video. spatial_merge_size: Size reduction for rows & cols dimensions. q: (`float`): Pruning rate factor [0,1) Returns: `torch.Tensor`: The retention mask for the video embeddings of `(T * H * W // spatial_merge_size ^ 2)` shape. rN.dimrT)r descendingstable)rrr)dtype)mapr reshapesizetorchnn functionalcosine_similaritycat ones_likeviewargsortr zeros_likebool)rrrrTHWr similarity dissimilaritydissimilarity_flatorderretain_num_tokens topk_indicesretention_maskmasks rcompute_retention_maskr4&s*#~&&GAq! ''   " L //A9K4KL$66QRRW|CRCH527J NMI u|BQB111aK899 9=IqM'++B// M,"d S S SE5)a1+++,L%&8 KKKN#'N< #++M,>,>,@,@AAN   r " "D Kr?tokens_per_secondvideo_second_per_gridcJ|d}|d|z}|d|z}tj|ddd||z||z}tj|ddd|d|}tj|ddd||d} tj|gddd|||}tj||| |gd} | S)a Computes the mrope for video embeddings based on the grid dimensions. Computed mrope positions match original qwen 2.5 implementation, but positions are built for media being the first element in sequence. Args: video_size_thw: Media size (num frames, rows, cols) spatial_merge_size: Size reduction for rows & cols dimensions. tokens_per_second: Number of tokens per second. video_second_per_grid: Number of seconds per video. Returns: Tensor of shape `(T * H * W, 4)` where last dimension represents mrope positions [0:3), while the last channel contains value of llm_grid_w repeated for all positions. rrrr) raranger%expandmullongflattentensorstack) rrr6r7 llm_grid_t llm_grid_h llm_grid_wt_indexh_indexw_index positionss rcompute_mrope_for_mediarH_sg, "J"&88J"&88J L $ $ T"a[[ VB Z/ 0 0 S"%:: ; ;     Z  aQ  B + +     Z  aB  J + +     j\"" aA  J 3 3   Wgw CKKKI r input_idsmultimodal_positionsmrope_positionsnum_computed_tokensvision_start_token_idimage_token_idvideo_token_idctjtj|}|}||} ||} | | z} | } t|dkrT|r9t| dz|z n| } || fStj | }tj | d|}||krT|r9t| dz|z n| } || fS||k dd}|D]}|||k}t|rK|d}tj | d|}||k}|r||z }|}n-|||kd}d}|}n|||kd}d}|}|d|fdz}|dz|z}||j dz}|dd|z|dd||f<|d|z}tj| |dd }||zdz |dd||f<||j dz }| dz|z }||fS) au Update part of input mrope positions. Original mrope_positions are computed incorrectly, so once we prune media tokens we should reflect this in the mrope positions for the LLM. This method supports chunked prefill approach where multimodal_embeddings are passed to LLM in chunks, so input multimodal_embeddings may contain zero, some or even some part of all multimodal_embeddings for a given prompt. Each multimodal_positions has 4 extra channels (First 3 channels corresponds to original 3 mrope positions, last channel is the maximum width of the media repeated). Provided multimodal_positions do not reflect location of media position in sequence - they are computed like the media is in the 0-th position in the sequence. Method works as follows: it recomputes mrope_positions starting from the `num_computed_tokens` for `total_len_of_multimodal_embeddings` and then shifts all text tokens that goes after total_len_of_multimodal_embeddings. It also handles case when multimodal_embeddings is partial (e.g. one media is split into two prefill stages) Args: input_ids: (N,) All input tokens of the prompt (entire sequence). multimodal_positions: List of mrope positions for each media. mrope_positions: Existing mrope positions (4, N) for entire sequence. num_computed_tokens: A number of computed tokens so far. vision_start_token_id: Token indicating start of vision media. image_token_id: Image token id video_token_id: Video token id Returns: Tuple of (mrope_positions, mrope_position_delta). rrNT)as_tupler)rRrr)typingcastr LongTensorclonenumeleqlenr r item count_nonzerononzeroshapecumsumr=)rIrJrKrLrMrNrOrGN image_mask video_mask media_mask text_maskdeltatotal_mm_tokensseen_mm_tokensvision_start_indicesmm_posseen_vision_start_indiceslast_vision_start_token'seem_mm_tokens_before_last_vision_startin_the_middle_of_mediamm_embeddings_seenglobal_mm_startnext_vision_start_tokenbase local_start local_endoffset text_pos_summrope_positions_deltas rrecompute_mrope_positionsrvsl\#)+ ///11##I An--Jn--Jj(J I   A%%9B9J9JRY]]__))++a/14555QRPR%)*55O(4H5H4H)IJJN((9B9J9JRY]]__))++a/14555QRPR%%)>>GGQUGVV '?/?/%9 #6 6% ! ( ) )# 6 '@&C #6;6I333477 3!HH #& :"%LL##:+?(,??++'&'""9 ';$(;;'' #"# 5O_,-1%),>> &,q/1 .4QqSkD.@ !!![**+$|Iijj$9$>$>$@$@aHHH $06$9A$= !!!Yq[.! v|A.&]]__q014::<< + ++r)r5r5) rSrr floatrTensorrUtupler4rHlistrvrrr|s //'*//4/////,6,6$uS#s]';;66 6  \ 6666x ##& 88$888! 8  \ 8888vL,L,u|,L,%L, L,  L,  L,L, 5 S !L,L,L,L,L,L,r