`i@dZddlZddlmZddlmZmZmZmZddl Z ddl m Z ddl m Z ddl mZmZmZdd lmZdd lmZdd lmZdd lmZmZmZmZdd lmZmZddlm Z ddl!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)ddl*m+Z+m,Z,m-Z-e&j.e/Z0dTde j1de j2dee3fdZ4 dUde j5de j2de j6de3fdZ7dUdZ8ee$dGdde"Z9ee$d Gd!d"e"Z:Gd#d$e j;Z< dVd&e j;d'e j1d(e j1d)e j1d*ee j1d+e=d,e=fd-Z>Gd.d/e j;Z?Gd0d1e j;Z@Gd2d3eZAGd4d5e j;ZBGd6d7e j;ZCGd8d9e j;ZDGd:d;e j;ZEGd<d=e j;ZFGd>d?eZGGd@dAe j;ZHe$GdBdCeZIGdDdEeIZJGdFdGeIZKe$dHGdIdJeIeZLGdKdLe j;ZMe$dMGdNdOeIZNe$dPGdQdReIeZOgdSZPdS)WzPyTorch KOSMOS-2 model.N) dataclass)AnyCallableOptionalUnion)nn)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPooling!CausalLMOutputWithCrossAttentions)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuplelogging torch_int)deprecate_kwarg) Kosmos2ConfigKosmos2TextConfigKosmos2VisionConfigmaskdtypetgt_lencL|\}}||n|}|ddddddf|d|||}d|z }||tjt j|jS)z_ Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Nr?)sizeexpandto masked_filltorchboolfinfomin)r#r$r%bszsrc_len expanded_mask inverted_masks /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/kosmos2/modeling_kosmos2.py _expand_maskr5-s99;;LC ,gg'GD$)*11#q'7KKNNuUUM-'M  $ $]%5%5ej%A%A5;uCUCUCY Z ZZinput_ids_shapedevicepast_key_values_lengthc*|\}}tj||ftj|j|}tj|d|}|||dz|ddkd||}|dkr.tj tj |||||gd}|ddddddf |d|||zS)zB Make causal mask used for bi-directional self-attention. )r8rrr$r8dimN) r,fullr.r/aranger( masked_fill_viewr*catzerosr))r7r$r8r9r0r%r# mask_conds r4_make_causal_maskrF;s  #LC :w(%+e*<*<*@ P P PD TYYr]]6:::Ii9q="6"6tyy}}a"H"HH!LLL 775>>D!!y%+g/EU[abbbdhioqrrr dAAAqqq ! ( (a'DZ:Z [ [[r6c||}tj|d||z|z}||zS)a Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr=)neintr,cumsumtype_aslong) input_ids padding_idxr9r#incremental_indicess r4"create_position_ids_from_input_idsrPMsg << $ $ ( ( * *D <!444<