`iY+<dZddlmZddlZddlZddlZddlmZddl m Z m Z m Z m Z ddlmZmZmZmZmZmZmZmZddlmZdd lmZmZmZd d lmZd Zgd gdgZ gdgdgdggdgdgdggZ!dZ"Gddej#j$Z%Gddej#j$Z&Gddej#j$Z'Gddej#j$Z(Gddej#j$Z)Gd d!ej#j$Z*Gd"d#ej#j$Z+Gd$d%ej#j$Z,Gd&d'ej#j$Z-eGd(d)ej#j$Z.Gd*d+eZ/d,Z0d-Z1ed.e0Gd/d0e/Z2Gd1d2ej#j$Z3ed3e0Gd4d5e/eZ4ed6e0Gd7d8e/eZ5ed9e0Gd:d;e/eZ6gd<Z7dS)=zTF 2.0 LayoutLMv3 model.) annotationsN)get_tf_activation)TFBaseModelOutputTFQuestionAnsweringModelOutputTFSequenceClassifierOutputTFTokenClassifierOutput)TFPreTrainedModelTFQuestionAnsweringLossTFSequenceClassificationLossTFTokenClassificationLossget_initializerkeraskeras_serializable unpack_inputs)check_embeddings_within_bounds)add_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings)LayoutLMv3Configr)r)rr)rrr)rr) ) ))gחc4eZdZdZd fd Zd dZd d ZxZS)TFLayoutLMv3PatchEmbeddingsz$LayoutLMv3 image (patch) embeddings.configrc tjd i|t|jtjjr|jn |j|jf}tj |j ||dddt|j d|_ |j |_ |jdz|d|dzz|_||_dS) Nvalid channels_lastTproj)filters kernel_sizestridespadding data_formatuse_biaskernel_initializernamerrr)super__init__ isinstance patch_size collectionsabcIterablerlayersConv2D hidden_sizerinitializer_ranger4 input_size num_patchesr0)selfr0kwargs patch_sizes __class__s /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/layoutlmv3/modeling_tf_layoutlmv3.pyr?z$TFLayoutLMv3PatchEmbeddings.__init__As""6"""&+[_-EFF 8F  #V%67  L''&#'.v/GHH(    "-"-q0k!n{ST~6UV pixel_values tf.Tensorreturnctj|gd}||}tj|d|j|jf}|S)N)rrrrperm)tf transposer4reshaperJrG)rKrQ embeddingss rOcallz TFLayoutLMv3PatchEmbeddings.callVsQ|L|||DDD YY|,, Z R1A4CS,TUU rPNc |jrdSd|_t|ddatj|jj5|jddd|jjgddddS#1swxYwYdSdS)NTr4) builtgetattrrX name_scoper4r<buildr0 num_channelsrK input_shapes rOraz!TFLayoutLMv3PatchEmbeddings.build_s :  F 4 & & 2ty~.. N N tT4;3K LMMM N N N N N N N N N N N N N N N N N N 3 2s)A66A:=A:r0rrQrRrSrRN__name__ __module__ __qualname____doc__r?r\ra __classcell__rNs@rOr/r/>ss..*NNNNNNNNrPr/cbeZdZdZdfd ZddZdd Zdd ZddZ dddZ ddZ xZ S) TFLayoutLMv3TextEmbeddingszm LayoutLMv3 text embeddings. Same as `RobertaEmbeddings` but with added spatial (layout) embeddings. r0rc tjd i|tj|j|jt|jd|_ tj|j |jt|jd|_ tj |j d|_tj|j|_|j|_tj|j|jt|jd|_tj|j|jt|jd|_tj|j|jt|jd|_tj|j|jt|jd |_tj|j|jt|jd |_|j|_||_dS) Nword_embeddings)embeddings_initializerr<token_type_embeddings LayerNormepsilonr<position_embeddingsx_position_embeddingsy_position_embeddingsh_position_embeddingsw_position_embeddingsr=)r>r?rrE Embedding vocab_sizerGrrHrrtype_vocab_sizertLayerNormalizationlayer_norm_epsruDropouthidden_dropout_probdropout pad_token_idpadding_token_indexmax_position_embeddingsrxmax_2d_position_embeddingscoordinate_sizeryrz shape_sizer{r|max_2d_positionsr0rKr0rLrNs rOr?z#TFLayoutLMv3TextEmbeddings.__init__ms-""6"""$|55    #263K#L#L" 6   &+\%;%;  "  #263K#L#L( &<& & " 88AV]h8ii|++F,FGG #)#6 #(<#9#9  *  #263K#L#L& $:$ $  &+\%;%;  -  "#263K#L#L( &<& & " &+\%;%;  -  "#263K#L#L( &<& & " &+\%;%;  -  #263K#L#L( &<& & " &+\%;%;  -  #263K#L#L( &<& & " !' A rPbboxrRrSc , |dddddf}|dddddf}|dddddf}|dddddf}n"#t$r}td|d}~wwxYw ||}||}||} ||} n+#t$r}td|jd|d}~wwxYw|jdz } |t j|dddddf|dddddfz d| } |t j|dddddf|dddddfz d| } t j||| | | | gd }|S) Nrrrrz9Bounding box is not of shape (batch_size, seq_length, 4).z0The `bbox` coordinate values should be within 0-z range.rWaxis) IndexErrorryrzrr{rX clip_by_valuer|concat)rKrleft_position_idsupper_position_idsright_position_idslower_position_ids exceptionleft_position_embeddingsupper_position_embeddingsright_position_embeddingslower_position_embeddingsmax_position_idr{r|spatial_position_embeddingss rO%calculate_spatial_position_embeddingsz@TFLayoutLMv3TextEmbeddings.calculate_spatial_position_embeddingss1 i $QQQ1W !%aaaAg !%aaaAg !%aaaAg   i i iXYY_h h i '+'A'ABS'T'T $(,(B(BCU(V(V %(,(B(BCU(V(V %(,(B(BCU(V(V % %   a4CXaaa   /!3 $ : :  T!!!QQQ']T!!!QQQ']:A O O! ! !% : :  T!!!QQQ']T!!!QQQ']:A O O! !  ')i()))%%   ' ' ' #+*s.B x YbhGGG  ^ z,O0DEE w|j!_== rP input_idsctjtj||j|j}tj|d|z}||jz}|S)z} Replace non-padding symbols with their position numbers. Position numbers begin at padding_token_index + 1. rr)rXcast not_equalrrcumsum)rKrmaskrs rO"create_position_ids_from_input_idsz=TFLayoutLMv3TextEmbeddings.create_position_ids_from_input_idssRwr|It/GHH)/ZZyA...5 #d&>> rP inputs_embedscZ|||S||Srg)rr)rKrrs rOcreate_position_idsz.TFLayoutLMv3TextEmbeddings.create_position_idss0  >>}MM M::9EE ErPNFtf.Tensor | Nonetoken_type_idsrtrainingboolc,||||}|tj|}ntj|dd}|tj||j}|/t ||jj||}||}||z} | |} | | z } | |} | | z } | | } | | |} | S)NrWrr) rrXrzerosrrrr input_dimrtrxrrur) rKrrrrrrrdrtr[rxrs rOr\zTFLayoutLMv3TextEmbeddings.calls$  33I}MML  (9--KK(=11#2#6K  !Xk9KLLLN  *9d6J6T U U U 00;;M $ : :> J J"%:: "66|DD)) &*&P&PQU&V&V#11 ^^J// \\*x\@@ rPcV|jrdSd|_t|ddPtj|jj5|jddddn #1swxYwYt|ddPtj|jj5|jddddn #1swxYwYt|dd]tj|jj5|jdd|j j gdddn #1swxYwYt|ddPtj|j j5|j ddddn #1swxYwYt|ddPtj|j j5|j ddddn #1swxYwYt|ddPtj|j j5|j ddddn #1swxYwYt|ddPtj|jj5|jddddn #1swxYwYt|d dStj|jj5|jdddddS#1swxYwYdSdS) NTrrrtrurxryrzr{r|)r^r_rXr`rrr<rartrur0rGrxryrzr{r|rcs rOraz TFLayoutLMv3TextEmbeddings.build s :  F 4*D 1 1 =t3899 1 1$**4000 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 40$ 7 7 Ct9>?? 7 7*00666 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 4d + + 7t~233 L L$$dD$+2I%JKKK L L L L L L L L L L L L L L L 4. 5 5 At7<== 5 5(..t444 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 40$ 7 7 Ct9>?? 7 7*00666 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 40$ 7 7 Ct9>?? 7 7*00666 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 40$ 7 7 Ct9>?? 7 7*00666 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 40$ 7 7 Ct9>?? 7 7*00666 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 D CsA''A+.A+!CC C (D66D:=D:0FFFG88G<?G<2II IJ::J>J>4LL #L re)rrRrSrR)rrRrSrR)rrRrSrR)rrRrrRrSrR)NNNNNF)rrrrrrrrrrrrrSrRrg) rirjrkrlr?rrrrr\rarmrns@rOrprphs000000d'+'+'+'+R    FFFF'+!%+/)-*.#####J77777777rPrpcJeZdZdfd ZddZddd Z dd dZd!dZxZS)"TFLayoutLMv3SelfAttentionr0rc btjd i||j|jzdkr t d|jd|jd|j|_t |j|jz |_|j|jz|_tj |j|_ tj |jt|jd|_tj |jt|jd|_tj |jt|jd|_tj |j|_|j|_|j|_||_dS) NrzThe hidden size (z6) is not a multiple of the number of attention heads ()queryr;r<keyvaluer=)r>r?rGnum_attention_heads ValueErrorintattention_head_size all_head_sizemathsqrtattention_score_normaliserrrEDenserrHrrrrattention_probs_dropout_probrhas_relative_attention_biashas_spatial_attention_biasr0rs rOr?z"TFLayoutLMv3SelfAttention.__init__*s""6"""   : :a ? ?8F$688 4888  $*#= #&v'9F > > > >&%)'+33333j H H H H H H H HrPrc2eZdZdfd Zddd Zdd ZxZS)TFLayoutLMv3SelfOutputr0rc ntjdi|tj|jt |jd|_tj |j d|_ tj |j |_||_dSNdenseunitsr;r<rurv)rater=r>r?rrErrGrrHrrrrurrrr0rs rOr?zTFLayoutLMv3SelfOutput.__init__""6"""\''$Ia9b9bip(   88AV]h8ii|++1K+LL  rPFrrR input_tensorrrrSc||}|||}|||z}|SNinputs)rrrrrurKrrrs rOr\zTFLayoutLMv3SelfOutput.callH - 88  MH MM ml.JKK rPNc|jrdSd|_t|dd]tj|jj5|jdd|jjgdddn #1swxYwYt|dd`tj|j j5|j dd|jjgddddS#1swxYwYdSdSNTrru) r^r_rXr`rr<rar0rGrurcs rOrazTFLayoutLMv3SelfOutput.builds :  F 4$ ' ' 3tz// H H   $dk.E!FGGG H H H H H H H H H H H H H H H 4d + + 7t~233 L L$$dD$+2I%JKKK L L L L L L L L L L L L L L L L L L 8 7$(A44A8;A8.(C##C'*C'reFrrRrrRrrrSrRrgrirjrkr?r\rarmrns@rOrrr L L L L L L L LrPrc8eZdZdfd Z dddZddZxZS)TFLayoutLMv3Attentionr0rc tjdi|t|d|_t |d|_dS)NrKr<outputr=)r>r?rself_attentionr self_outputrs rOr?zTFLayoutLMv3Attention.__init__sP""6"""7VLLL1&xHHHrPNFrrRrrrrrrrrrSrc ||||||||}||d||} | f|ddz} | S)Nrrr)rr) rKrrrrrrr self_outputsattention_outputrs rOr\zTFLayoutLMv3Attention.callsp**      +   ++LO]U]+^^#% QRR(88rPc|jrdSd|_t|ddPtj|jj5|jddddn #1swxYwYt|ddStj|jj5|jdddddS#1swxYwYdSdS)NTrr)r^r_rXr`rr<rarrcs rOrazTFLayoutLMv3Attention.buildsX :  F 4)4 0 0 <t2788 0 0#))$/// 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 - - 9t/455 - - &&t,,, - - - - - - - - - - - - - - - - - - : 9$A''A+.A+!C  C C rerrrgrrns@rOrrs{IIIIII%)'+. - - - - - - - -rPrc0eZdZd fd Zd dZd d ZxZS) TFLayoutLMv3Intermediater0rc Dtjdi|tj|jt |jd|_t|j trt|j |_ n |j |_ ||_dS)Nrrr=)r>r?rrErintermediate_sizerrHrr@ hidden_actstrrintermediate_act_fnr0rs rOr?z!TFLayoutLMv3Intermediate.__init__s""6"""\''*vOg?h?hov(    f' - - 9'89J'K'KD $ $'-'8D $ rPrrRrSc\||}||}|S)Nr)rr%)rKrs rOr\zTFLayoutLMv3Intermediate.calls. - 88 00?? rPNc|jrdSd|_t|dd`tj|jj5|jdd|jjgddddS#1swxYwYdSdS)NTr) r^r_rXr`rr<rar0rGrcs rOrazTFLayoutLMv3Intermediate.builds :  F 4$ ' ' 3tz// H H   $dk.E!FGGG H H H H H H H H H H H H H H H H H H 4 3s(A55A9<A9re)rrRrSrRrgrrns@rOr r sm       HHHHHHHHrPr c2eZdZdfd Zddd Zdd ZxZS)TFLayoutLMv3Outputr0rc ntjdi|tj|jt |jd|_tj |j d|_ tj |j |_||_dSrrrs rOr?zTFLayoutLMv3Output.__init__rrPFrrRrrrrSc||}|||}|||z}|Srr r s rOr\zTFLayoutLMv3Output.callr rPNc|jrdSd|_t|dd]tj|jj5|jdd|jjgdddn #1swxYwYt|dd`tj|j j5|j dd|jj gddddS#1swxYwYdSdSr ) r^r_rXr`rr<rar0r"rurGrcs rOrazTFLayoutLMv3Output.build!s :  F 4$ ' ' 3tz// N N   $dk.K!LMMM N N N N N N N N N N N N N N N 4d + + 7t~233 L L$$dD$+2I%JKKK L L L L L L L L L L L L L L L L L L 8 7rrerrrgrrns@rOr)r)rrPr)c8eZdZdfd Z dddZddZxZS)TFLayoutLMv3Layerr0rc tjdi|t|d|_t |d|_t |d|_dS)N attentionr intermediaterr=)r>r?rr0r r1r) bert_outputrs rOr?zTFLayoutLMv3Layer.__init__.sd""6""".vKHHH4V.QQQ-f8DDDrPNFrrRrrrrrrrrrSrc ||||||||}|d} |dd} || } || | |} | f| z} | S)N)rrrrrrr)r0r1r2) rKrrrrrrrself_attention_outputsrrintermediate_output layer_outputs rOr\zTFLayoutLMv3Layer.call4s"&   /!"0" " 2!4(,"//0@AA''(;=MX`'aa /G+rPcr|jrdSd|_t|ddPtj|jj5|jddddn #1swxYwYt|ddPtj|jj5|jddddn #1swxYwYt|ddStj|jj5|jdddddS#1swxYwYdSdS)NTr0r1r2) r^r_rXr`r0r<rar1r2rcs rOrazTFLayoutLMv3Layer.buildNs :  F 4d + + 7t~233 + +$$T*** + + + + + + + + + + + + + + + 4 . . :t0566 . .!''--- . . . . . . . . . . . . . . . 4 - - 9t/455 - - &&t,,, - - - - - - - - - - - - - - - - - - : 9s6A''A+.A+!CC C D**D.1D.rerrrgrrns@rOr.r.-s{EEEEEE%)'+4 - - - - - - - -rPr.cbeZdZd!fd Zd"d Zd#d Zd$dZd%dZ d&d'dZd(d Z xZ S))TFLayoutLMv3Encoderr0rc tjdi||_fdtjD|_j|_j|_|jrWj|_j |_ tj j tjdd|_|jrj|_j|_tj j tjdd|_tj j tjdd|_dSdS)Nc8g|]}td|S)zlayer.r)r.).0ir0s rO z0TFLayoutLMv3Encoder.__init__..as-lllq'\a\\BBBlllrPF rel_pos_bias)rr;r:r<rel_pos_x_biasrel_pos_y_biasr=)r>r?r0rnum_hidden_layerslayerrr rel_pos_bins max_rel_posrrErrrrHr?max_rel_2d_posrel_2d_pos_binsr@rArs ` rOr?zTFLayoutLMv3Encoder.__init__^sf""6""" llllERXRjLkLklll +1+M(*0*K'  +  & 3D %1D  % 2 20#263K#L#L# !3!!D   * "("7D #)#9D "',"4"40#263K#L#L% #5##D  #(,"4"40#263K#L#L% #5##D     rPrelative_positionsrR num_bucketsr max_distancec|dz}tj|}|dz}||k}tjtj|tj|z }tj||z }||z ||z z} || z} tj| |j} tj| |dz } tj|dk|j|ztj||| zS)Nrrr) rXabsrlogrfloat32rminimumwhere) rKrHrIrJbucketsmax_exact_bucketsis_smallbuckets_log_ratiodistance_log_ratiobuckets_big_offset buckets_bigs rOrelative_position_bucketz,TFLayoutLMv3Encoder.relative_position_buckets"Q& &+,,(1,..GKK(D(DGX(XYY!Xl5F&FGG  2 2kDU6U V (*<< gk7=99 jkAo>> *Q. >>LPRPX g{Q Q   rP dense_layerkeras.layers.Denserc@tj|dtj|dz }||||}tj|||j}||}tj|gd}tj||j}|S)NrrW)depthr)rrrrr)rXrrXone_hot compute_dtyperYr) rKrYrrIrJrel_pos_matrixrrel_pos_one_hot embeddings rO _cal_pos_embz TFLayoutLMv3Encoder._cal_pos_embs 2>>>P\ceAfAfAff// \ZZ*WKtGYZZZK00 LLLL99 GIT-?@@@ rPcP||j||j|jSrg)rcr?rDrE)rKrs rO_cal_1d_pos_embz#TFLayoutLMv3Encoder._cal_1d_pos_embs&  !2L$BSUYUefffrPrc|dddddf}|dddddf}||j||j|j}||j||j|j}||z}|S)Nrr)rcr@rGrFrA)rKrposition_coord_xposition_coord_y rel_pos_x rel_pos_yrs rO_cal_2d_pos_embz#TFLayoutLMv3Encoder._cal_2d_pos_embs111a=111a=%%         %%         * rPNFTrrrrrroutput_hidden_states return_dictrrSkTFBaseModelOutput | tuple[tf.Tensor] | tuple[tf.Tensor, tf.Tensor] | tuple[tf.Tensor, tf.Tensor, tf.Tensor]c |rdnd} |rdnd} |jr||nd} |jr||nd} t |jD]A\}}|r| |fz} |||nd}|||||| | | }|d}|r | |dfz} B|r| |fz} |rt || | Std|| | fDS)Nr=)rrrrrlast_hidden_stater attentionsc3K|]}||V dSrgr=)r<rs rO z+TFLayoutLMv3Encoder.call..s1^c^o^o^o^o^orP)rrerrk enumeraterCrtuple)rKrrrrrrlrmrrall_hidden_statesall_self_attentionsrrr= layer_modulelayer_head_mask layer_outputss rOr\zTFLayoutLMv3Encoder.callsx#7@BBD$5?bb48<8Xb$&&|444^b373R\T))$///X\ (44 P POA|# I$58H$H!.7.CillO(L!%!M*!,M  P&9]1=Mnormencoderr=)r>r?r0 text_embedrpr[ visual_embedr/rrrErrrurrrrrrIrAinit_visual_bboxrr9r)rKr0rLrrNs rOr?zTFLayoutLMv3MainLayer.__init__ s""6"""   T8lSSSDO   S:6 VVVD "\<>DD8EE#&E#GGG:(H..H25H2(JJJ (K>>LLrSkeras.layers.Layerc|jjSrg)r[rr)rKs rOget_input_embeddingsz*TFLayoutLMv3MainLayer.get_input_embeddingsFs ..rPr tf.Variablec(||jj_dSrg)r[rrweight)rKrs rOset_input_embeddingsz*TFLayoutLMv3MainLayer.set_input_embeddingsIs16'...rPct)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel )NotImplementedError)rKheads_to_prunes rO _prune_headsz"TFLayoutLMv3MainLayer._prune_headsMs "!rPrtuple[int, int]max_lenrc|\}}tjd||dzz||z}tj|d}tj||dg}tjd||dzz||z}tj|d}tj|d|g}tj|ddddf|dd|ddddf|ddgd}tj|ddg}tjdd|dz |dz ggtj}tj||gd|_ dS)NrrrrWrr) rXrrrstackrZconstantrr visual_bbox) rKrrheightwidth visual_bbox_x visual_bbox_yr cls_token_boxs rOrz&TFLayoutLMv3MainLayer.init_visual_bboxTs`# Guqy$97CCuL }1===  qz:: Gvz$:GDDN }1===  6{;; h 111crc6 "M#2#$6 aaae8Lm\]\^\^N_ `   jr1g66  aGaK1%E$FbhWWW 9m[%AJJJrPrrtf.DTypectj|jd}tj||ddg}tj||}|S)Nrrrr)rXrrrr)rKrrrs rOcalculate_visual_bboxz+TFLayoutLMv3MainLayer.calculate_visual_bboxksKnT%5A>>> gkJ1+=>> gk777 rPrQrRc.||}tj|d}tj|j|ddg}tj||gd}t |dd ||jz }||}|S)Nrrrr) rrXrrrrr_rr)rKrQr[r cls_tokenss rO embed_imagez!TFLayoutLMv3MainLayer.embed_imageqs%%l33 Xj))!, WT^j!Q-?@@ Y J7a@@@  4d + + 7 $. (JYYz** rPrcLt|j}|dkrtj|d}nK|dkr-tj|d}tj|d}nt d|jdtj||j}d|z tz}|S)Nrrrrz&Wrong shape for attention_mask (shape ).g?)lenrrXrrrr_LARGE_NEGATIVE)rKrn_dimsextended_attention_masks rOget_extended_attention_maskz1TFLayoutLMv3MainLayer.get_extended_attention_masks^)** Q;;&(n^!&L&L&L # # q[[')n^!&L&L&L #&(n5LST&U&U&U # #^nFZ^^^__ _#%'*A4CU"V"V#&)@#@N"R&&rPrr"tf.Tensor | list[tf.Tensor | None]c|dg|jjzStj|}|dkr}tj|d}tj|d}tj|d}tj|d}tj||jjddddg}ng|dkrCtj|d}tj|d}tj|d}n|dkrt d|jdtj|dks Jd tj|d tj||j }|S) NrrrrWrrz!Wrong shape for head_mask (shape rzGot head_mask rank of z, but require 5.) r0rBrXrankrrrrrr_)rKrrs rO get_head_maskz#TFLayoutLMv3MainLayer.get_head_masksk  6DK99 9## Q;;yq999Iyq999Iyr:::Iyr:::IDK91aAFIIq[[yq999Iyr:::Iyr:::II q[[TTTTUU Uwy!!Q&&&(eASAS(e(e(e&&&GIt'9:: rPFrrrrrr bool | Nonerlrmrrrnc | | n |jj} | | n |jj} | | n |jj} |%t j|} | d}| d}nS|%t j|} | d}| d}n,|t j|d}nt d||j}n*||j}n ||j}n||j}n tj}||j|t j ||f|}|t j ||f|}|t j ||df|}| |||||| }d}d}|| |}t j |t j|df|}||}nt j ||gd}|jjr3|||}||}nt j ||gd}|jjs |jjrt jdt j|d|}t j|d}t j||dg}||[t jt jd||d}t j||dg}t j ||gd}n|}|||}nt j ||gd}||}||| }nv|jjs |jjr^|jjrDt jt jd||d}t j||dg}|}|jjr|}||}||}||||||| | | }|d}| s|f|ddzSt3||j|j S) NrrzEYou have to specify either input_ids or inputs_embeds or pixel_valuesrr)rrrrrrrr)rrrrrrlrmrp)r0rrlrmrXrrrronesrr[rrrrrrrrrurrrrrrrr)rKrrrrrrrrQrrlrmrrdr seq_length int_dtypeembedding_output final_bboxfinal_position_idsvisual_embeddingsvisual_attention_maskrvisual_position_idsrencoder_outputssequence_outputs rOr\zTFLayoutLMv3MainLayer.calls,2C1N--TXT_Tq$8$D $+Jj &1% > %'GZBS9T9TUV9W,X`i$j$j$j !%!6!#NQ!G!G!GJ{6 =$+:` =&(hq"(;L2M2Ma2PXa&b&b&b#&(n5Hq&Q&Q&Q#&(g.AJPQ?&S&S#(M,E#%>"(1jPY2Z2Z2Zab#c#c#cL#%7<*a#I#IL)+LBU3V]^)_)_)_&&)<& ]%:#4  #%9.>@Q-RYZ#[#[#[ #~~.>?? #||,"R"R&&y11 ,, +2/!5#'   *!, <#%(;; ; -)7&1    rPrerg)rSr)rr)r)rrrr)rrrrrf)rrRrSrR)rrrSr NNNNNNNNNNNFrrrrrrrrrrrrrrrQrrrrlrrmrrrrSrn)rirjrkr config_classr?rarrrrrrrrrr\rmrns@rOr~r~sZ#LCCCCCC*&G&G&G&GP////7777"""KKKKK.    ''''62'+!%+/+/)-&**.)-)-,0#'[ [ [ [ ][ [ [ [ [ rPr~c:eZdZdZeZdZefdZxZ S)TFLayoutLMv3PreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. layoutlmv3cvtj}tjdtjd|d<|S)N)NNrrr)r>input_signaturerX TensorSpecr)rKsigrNs rOrz+TFLayoutLMv3PreTrainedModel.input_signature\s0gg%mORXFKKKF  rP) rirjrkrlrrbase_model_prefixpropertyrrmrns@rOrrSsY $L$ XrPra This model inherits from [`TFPreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a [keras.Model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) subclass. Use it as a regular TF 2.0 Keras Model and refer to the TF 2.0 documentation for all matter related to general usage and behavior. TensorFlow models and layers in `transformers` accept two formats as input: - having all inputs as keyword arguments (like PyTorch models), or - having all inputs as a list, tuple or dict in the first positional argument. The reason the second format is supported is that Keras methods prefer this format when passing inputs to models and layers. Because of this support, when using methods like `model.fit()` things should "just work" for you - just pass your inputs and labels in any format that `model.fit()` supports! If, however, you want to use the second format outside of Keras methods like `fit()` and `predict()`, such as when creating your own layers or models with the Keras `Functional` API, there are three possibilities you can use to gather all the input Tensors in the first positional argument: - a single Tensor with `input_ids` only and nothing else: `model(input_ids)` - a list of varying length with one or several input Tensors IN THE ORDER given in the docstring: `model([input_ids, attention_mask])` or `model([input_ids, attention_mask, token_type_ids])` - a dictionary with one or several input Tensors associated to the input names given in the docstring: `model({"input_ids": input_ids, "token_type_ids": token_type_ids})` Note that when creating models and layers with [subclassing](https://keras.io/guides/making_new_layers_and_models_via_subclassing/) then you don't need to worry about any of this, as you can just pass inputs like you would to any other Python function! Parameters: config ([`LayoutLMv3Config`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~TFPreTrainedModel.from_pretrained`] method to load the model weights. a Args: input_ids (`Numpy array` or `tf.Tensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Note that `sequence_length = token_sequence_length + patch_sequence_length + 1` where `1` is for [CLS] token. See `pixel_values` for `patch_sequence_length`. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) bbox (`Numpy array` or `tf.Tensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. Note that `sequence_length = token_sequence_length + patch_sequence_length + 1` where `1` is for [CLS] token. See `pixel_values` for `patch_sequence_length`. pixel_values (`tf.Tensor` of shape `(batch_size, num_channels, height, width)`): Batch of document images. Each image is divided into patches of shape `(num_channels, config.patch_size, config.patch_size)` and the total number of patches (=`patch_sequence_length`) equals to `((height / config.patch_size) * (width / config.patch_size))`. attention_mask (`tf.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. Note that `sequence_length = token_sequence_length + patch_sequence_length + 1` where `1` is for [CLS] token. See `pixel_values` for `patch_sequence_length`. [What are attention masks?](../glossary#attention-mask) token_type_ids (`Numpy array` or `tf.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. Note that `sequence_length = token_sequence_length + patch_sequence_length + 1` where `1` is for [CLS] token. See `pixel_values` for `patch_sequence_length`. [What are token type IDs?](../glossary#token-type-ids) position_ids (`Numpy array` or `tf.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. Note that `sequence_length = token_sequence_length + patch_sequence_length + 1` where `1` is for [CLS] token. See `pixel_values` for `patch_sequence_length`. [What are position IDs?](../glossary#position-ids) head_mask (`tf.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`tf.Tensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert *input_ids* indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zdThe bare LayoutLMv3 Model transformer outputting raw hidden-states without any specific head on top.ceZdZdgZfdZeeeee e  dddZ ddZ xZ S)TFLayoutLMv3Modelrcntj|g|Ri|t|d|_dS)Nrr)r>r?r~r)rKr0rrLrNs rOr?zTFLayoutLMv3Model.__init__sB3&333F333/\JJJrP output_typerNFrrrrrrrrQrrrlrmrrrSrnc H|||||||||| | | |  } | S)a Returns: Examples: ```python >>> from transformers import AutoProcessor, TFAutoModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", apply_ocr=False) >>> model = TFAutoModel.from_pretrained("microsoft/layoutlmv3-base") >>> dataset = load_dataset("nielsr/funsd-layoutlmv3", split="train") >>> example = dataset[0] >>> image = example["image"] >>> words = example["tokens"] >>> boxes = example["bboxes"] >>> encoding = processor(image, words, boxes=boxes, return_tensors="tf") >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state ```) rrrrrrrrQrrlrmr)r)rKrrrrrrrrQrrlrmrrs rOr\zTFLayoutLMv3Model.callsHT//))%'%/!5#"   rPc|jrdSd|_t|ddStj|jj5|jdddddS#1swxYwYdSdS)NTr)r^r_rXr`rr<rarcs rOrazTFLayoutLMv3Model.build s :  F 4t , , 8t344 , ,%%d+++ , , , , , , , , , , , , , , , , , , 9 8sA((A,/A,rrrg)rirjrk"_keys_to_ignore_on_load_unexpectedr?rrLAYOUTLMV3_INPUTS_DOCSTRINGrr_CONFIG_FOR_DOCr\rarmrns@rOrrs +:):&KKKKK**+FGG+r?rrErrGrrHrclassifier_dropoutrrr num_labelsrr0)rKr0rLrrNs rOr?z'TFLayoutLMv3ClassificationHead.__init__.s""6"""\''  .v/GHH (   *0)B)NF % %TZTn |++ ,    **  .v/GHH+    rPFrrRrrrSc|||}||}|||}||}|S)Nr)rrr)rKrrrs rOr\z#TFLayoutLMv3ClassificationHead.callDsS,,v,99**W%%,,w,::--((rPNc|jrdSd|_t|dd]tj|jj5|jdd|jjgdddn #1swxYwYt|ddPtj|j j5|j ddddn #1swxYwYt|dd`tj|j j5|j dd|jjgddddS#1swxYwYdSdS)NTrrr) r^r_rXr`rr<rar0rGrrrcs rOraz$TFLayoutLMv3ClassificationHead.buildKs! :  F 4$ ' ' 3tz// H H   $dk.E!FGGG H H H H H H H H H H H H H H H 4D ) ) 5t|011 ) ) ""4((( ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 4T * * 6t}122 K K ##T41H$IJJJ K K K K K K K K K K K K K K K K K K 7 6s6(A44A8;A8.CCC(EE Erer)rrRrrrSrRrgrhrns@rOrr)s|, K K K K K K K KrPra  LayoutLMv3 Model with a sequence classification head on top (a linear layer on top of the final hidden state of the [CLS] token) e.g. for document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. ceZdZdgZdfd Zeeeee e  dddZ ddZ xZ S)%TFLayoutLMv3ForSequenceClassificationrr0rc tj|fi|||_t|d|_t |d|_dS)Nrr classifier)r>r?r0r~rrrrs rOr?z.TFLayoutLMv3ForSequenceClassification.__init__fsU**6*** /\JJJ8lSSSrPrNFrrrrrrlabelsrrrlrmrrQrrSTFSequenceClassifierOutput | tuple[tf.Tensor] | tuple[tf.Tensor, tf.Tensor] | tuple[tf.Tensor, tf.Tensor, tf.Tensor] | tuple[tf.Tensor, tf.Tensor, tf.Tensor, tf.Tensor]c^| | n |jj} ||||||||| | | | |  }|ddddddf}||| }|dn|||}| s|f|ddz}||f|zn|St |||j|jS)a Returns: Examples: ```python >>> from transformers import AutoProcessor, TFAutoModelForSequenceClassification >>> from datasets import load_dataset >>> import tensorflow as tf >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", apply_ocr=False) >>> model = TFAutoModelForSequenceClassification.from_pretrained("microsoft/layoutlmv3-base") >>> dataset = load_dataset("nielsr/funsd-layoutlmv3", split="train") >>> example = dataset[0] >>> image = example["image"] >>> words = example["tokens"] >>> boxes = example["bboxes"] >>> encoding = processor(image, words, boxes=boxes, return_tensors="tf") >>> sequence_label = tf.convert_to_tensor([1]) >>> outputs = model(**encoding, labels=sequence_label) >>> loss = outputs.loss >>> logits = outputs.logits ```N rrrrrrrlrmrrQrrrrlosslogitsrrr)r0use_return_dictrrhf_compute_lossrrrr)rKrrrrrrrrrlrmrrQrrrrrrs rOr\z*TFLayoutLMv3ForSequenceClassification.calllsh&1%