`i.ddlmZmZddlZddlmZddlmZddlm Z m Z ddl m Z ddl mZmZmZdd lmZmZmZmZGd d eZeGd d eZGddeeZedGddeeZgdZdS))OptionalUnionN) IJepaConfig)BaseModelOutputWithPoolingImageClassifierOutput)Unpack)TransformersKwargsauto_docstring torch_int) ViTEmbeddingsViTForImageClassificationViTModelViTPreTrainedModelc eZdZddededdffd Zdejded edejfd Z dd ejd e ej d edejfdZ xZ S)IJepaEmbeddingsFconfiguse_mask_tokenreturnNct|||`|jj}t jtjd||j |_ dS)N) super__init__ cls_tokenpatch_embeddings num_patchesnn Parametertorchrandn hidden_sizeposition_embeddings)selfrrr __class__s {/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/ijepa/modular_ijepa.pyrzIJepaEmbeddings.__init__sV 000 N+7 #%< A{FL^0_0_#`#`   embeddingsheightwidthc|jd}|jjd}tjs||kr ||kr|jS|j}|jd}||jz}||jz} t |dz} |d| | |}|dddd}tj ||| fdd }|dddd dd|}|S) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rg?rrr bicubicF)sizemode align_corners) shaper#r jit is_tracing patch_sizer reshapepermuter functional interpolateview) r$r(r)r*r num_positionspatch_pos_embeddim new_height new_widthsqrt_num_positionss r&interpolate_pos_encodingz(IJepaEmbeddings.interpolate_pos_encodings*!&q) 06q9 y##%% ,+*F*F6UZ??+ +2r"t. T_, &}c'9::)11!5GI[]`aa)11!Q1==-33 i( 4  *11!Q1==BB1b#NNr' pixel_valuesbool_masked_posr@c|j\}}}}|||}|_|jd} |j|| d} |d| } |d| z z| | zz}|r|||||z}n ||jz}||}|S)N)r@rr,?) r1r mask_tokenexpand unsqueezetype_asr@r#dropout) r$rArBr@ batch_size_r)r*r( seq_length mask_tokensmasks r&forwardzIJepaEmbeddings.forward=s (4'9$ Avu**>J\\*-- r')F)NF)__name__ __module__ __qualname__rboolrr Tensorintr@r BoolTensorrO __classcell__r%s@r&rrsaa{aDaTaaaaaa%5<%%UX%]b]i%%%%T7;). l"%"23#'   r'rcNeZdZdeejejejfddfdZdS)IJepaPreTrainedModelmodulerNct|tjtjfrtj|jjtj d|j j |jj |j_|j |jjdSdSt|tjr?|jj|jjddSt|t$rtj|jjtj d|j j |jj |j_|j"|jjdSdSdS)zInitialize the weightsg)meanstdNrD) isinstancerLinearConv2dinit trunc_normal_weightdatator float32rinitializer_rangedtypebiaszero_ LayerNormfill_rr#rE)r$r[s r& _init_weightsz"IJepaPreTrainedModel._init_weightsZs fry")4 5 5 /"$!6!6 "%%em443DKDa"7""b$%% M {&  &&((((('&  - - / K  " " $ $ $ M  $ $S ) ) ) ) )  0 0 /.0g.C.C*/225=AAK1/D//b+122  & +  ,!&,,..... / / -,r') rPrQrRrrr`rarlrnr'r&rZrZXsI/E")RY *L$M/RV//////r'rZc.eZdZddededeffd ZxZS) IJepaModelFradd_pooling_layerrct|||_t|||_dS)z add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer use_mask_token (`bool`, *optional*, defaults to `False`): Whether to use a mask token for masked image modeling. )rN)rrrrr()r$rrrrr%s r&rzIJepaModel.__init__rs<     )&PPPr')FF)rPrQrRrrSrrWrXs@r&rqrqqsf Q Q{ Qt Q]a Q Q Q Q Q Q Q Q Q Qr'rqa IJepa Model transformer with an image classification head on top (a linear layer on top of the final hidden states) e.g. for ImageNet. Note that it's possible to fine-tune IJepa on higher resolution images than the ones it has been trained on, by setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained position embeddings to the higher resolution. ) custom_introc eZdZdeffd Z d deejdeejdeejdeede e d e f d Z xZ S) IJepaForImageClassificationrct|t|d|_|dS)NF)rr)rrrqijepa post_init)r$rr%s r&rz$IJepaForImageClassification.__init__sE    %@@@  r'NrA head_masklabelsr@kwargsrc |j|f||d|}|j}||d}d} ||j|||jfi|} t | ||j|jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). )rzr@r)r<N)losslogits hidden_states attentions) rxlast_hidden_state classifierr] loss_functionrrrr) r$rArzr{r@r|outputssequence_outputrr~s r&rOz#IJepaForImageClassification.forwards/9dj / %=/ /  / /  "3!5!5!!5!rs"""""""" EEEEEEQQQQQQQQ&&&&&&BBBBBBBBBBeeeeeeeeeeeeGGGGGmGGGT/////-///0 Q Q Q Q Q%x Q Q Q    ' ' ' ' ' "68Q' '   ' T   r'