`iFddlmZddlmZmZmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZdd lmZdd lmZmZmZdd lmZdd lmZddlmZddlm Z eedGddeZ!Gdde j"Z#dZ$dBdZ%dej&de'dej&fdZ( dCde j"d ej&d!ej&d"ej&d#eej&d$e)d%e)d&eefd'Z*Gd(d)e j"Z+Gd*d+e j"Z,Gd,d-e j"Z-d.ej&d/ej&d0ej&dej&fd1Z.Gd2d3e j"Z/Gd4d5e j"Z0eGd6d7eZ1d8ej&d9e)de2ej&ej&ffd:Z3d;ej&dZ4ed?Gd@dAe1Z5d7dAgZ6dS)D) dataclass)CallableOptionalUnionN)nn pad_sequence)ACT2FN)FlashAttentionKwargs)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstring)deprecate_kwarg)can_return_tuple)AutoModelForKeypointDetection)LightGlueConfiga Base class for outputs of LightGlue keypoint matching models. Due to the nature of keypoint detection and matching, the number of keypoints is not fixed and can vary from image to image, which makes batching non-trivial. In the batch of images, the maximum number of matches is set as the dimension of the matches and matching scores. The mask tensor is used to indicate which values in the keypoints, matches, matching_scores and prune tensors are keypoint matching information. ) custom_introcLeZdZUdZdZeejed<dZ eejed<dZ eejed<dZ eejed<dZ eej ed<dZeejed<dZeeejed <dZeeejed <dS) LightGlueKeypointMatchingOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*): Loss computed during training. matches (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Index of keypoint matched in the other image. matching_scores (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Scores of predicted matches. keypoints (`torch.FloatTensor` of shape `(batch_size, num_keypoints, 2)`): Absolute (x, y) coordinates of predicted keypoints in a given image. prune (`torch.IntTensor` of shape `(batch_size, num_keypoints)`): Pruning mask indicating which keypoints are removed and at which layer. mask (`torch.BoolTensor` of shape `(batch_size, num_keypoints)`): Mask indicating which values in matches, matching_scores, keypoints and prune are keypoint matching information. hidden_states (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for the output of each stage) of shape `(batch_size, 2, num_channels, num_keypoints)` returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True` attentions (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, 2, num_heads, num_keypoints, num_keypoints)` returned when `output_attentions=True` is passed or when `config.output_attentions=True` Nlossmatchesmatching_scores keypointsprunemask hidden_states attentions)__name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrr IntTensorr!r"tupler#/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/lightglue/modeling_lightglue.pyrr's0)-D(5$ %,,,+/GXe' (///37OXe/0777-1Ix)*111'+E8EO $+++(,D(5$ %,,,8s@r/r1r1Tslllllll OT   =Ed^  uU\"E%, *D$EE F        r.r1c|ddddf}|ddddf}tj| |gdd}|S)N.rrrBrC)r(stackflatten)xx1x2rot_xs r/ rotate_halfr[esU 3!8B 319B K"b r * * * 2 22 6 6E Lr.cl|j}|}|}||}||}||zt||zz}||zt||zz}||||fS)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. dtype)r^float unsqueezer[to) qkrFrG position_ids unsqueeze_dimr^q_embedk_embeds r/apply_rotary_pos_embrhms( GE  A  A -- & &C -- & &C3w;q>>C/0G3w;q>>C/0G ::E: " "GJJUJ$;$; ;;r.r"n_repr@c|j\}}}}|dkr|S|dddddddddf|||||}||||z||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)shapeexpandreshape)r"ribatchnum_key_value_headsslenhead_dims r/ repeat_kvrrs 2?1D.E h zz!!!!QQQaaa"23::5BUW\^bdlmmM  (;e(CT8 T TTr.modulequerykeyvalueattention_maskscalingdropoutkwargsc Rt||j}t||j} tj||dd|z} |$|ddddddd|jdf} | | z} t j| dtj  |j } t j | ||j } tj| | } | dd} | | fS)Nrr rTrB)rDr^)ptrainingr)rrnum_key_value_groupsr(matmul transposerkr functionalsoftmaxfloat32rar^rzr~ contiguous) rtrurvrwrxryrzr{ key_states value_states attn_weights causal_mask attn_outputs r/eager_attention_forwardrs3 ;<s r/r7zLightGlueAttention.__init__sB  " F4F&Jd4dee $*$>&B\$\!}d* !'!9i   :T] JQWQf    i   :T] JQWQf    i   :T] JQWQf    i  & 68JQWQf    r.past_key_valuepast_key_valuesz4.58)new_nameversionNr"position_embeddingsrxencoder_hidden_statesencoder_attention_maskr{r@c |jdd}g|d|jR}|||dd} |du} | r|n|} | r|n|} || |dd} || |dd}||\}}t| | ||\} } t}|j j dkrt|j j }||| | || f|j sdn|j |jd|\}}|jg|dR}||}||fS)NrBrreagerrs)rzry)rkrqrviewrrrrhrr2_attn_implementationr r~rryrmrr)r=r"rrxrrr{ input_shape hidden_shape query_statesis_cross_attentioncurrent_statescurrent_attention_maskrrrFrGattention_interfacerrs r/rMzLightGlueAttention.forwards$)#2#. 88b8$-88 {{=1166|DDNNqRSTT 2$>2DW..-;M!a!7!7Sa[[0055lCCMMaQRSS {{>2277 EEOOPQSTUU  **HC';L*VY[^'_'_ $L*(? ; +w 6 6"9$+:Z"[ $7$7     " % $}HCC$2HL % %  % % ! \*k);;;;;;FFHH kk+.. L((r.)NNNN)r$r%r&r'rintr7rr(rOrr,rr rMrQrRs@r/rrs"GG  3      ._%0A6RRRLP158<9= *)*)|*)&eEL%,,F&GH*)!. *) ( 5 *) !) 6 *)-.*) u|Xel33 4*)*)*)SR*)*)*)*)*)r.rcHeZdZdeffd ZdejdejfdZxZS) LightGlueMLPr2cTt||_t|j|_t j|j|j|_ t j|j|j |_ t j |jd|_ dS)NT)elementwise_affine)r6r7r2r hidden_act activation_fnrr8intermediate_sizefc1rfc2 LayerNorm layer_normr<s r/r7zLightGlueMLP.__init__s  #F$569V5v7OPP9V5v7IJJ,v'?TXYYYr.r"r@c||}||}||}||}|SN)rrrr)r=r"s r/rMzLightGlueMLP.forwardsN//  66 **=99 // r. r$r%r&rr7r(rOrMrQrRs@r/rrsqZZZZZZZU\elr.rceZdZdedeffd Z d dejdejdejdee d ee d e ejee ejee ejff d Z xZ S) LightGlueTransformerLayerr2rctt|||_t ||_t|||_t ||_dSr)r6r7rself_attentionrself_mlpcross_attention cross_mlprs r/r7z"LightGlueTransformerLayer.__init__ s_ 0CC$V,, 1&)DD%f--r.F descriptorsrrxr?output_attentionsr@c2|rdnd}|rdnd}|r||fz}|j\}} } |||||\} } tj|| gd} || }||z}|r| |f}|dd| | d|| | }|B|dddd| d|dd| nd}|||||\}}tj||gd}||}||z}|r>||f}|||| | fz|z||| | fz|z}|r || fz|fz}|||fS)Nr-)rrxrrBrCrr)rrr) rkrr(catrrmfliprr)r=rrrxr?rall_hidden_statesall_attentions batch_size num_keypointsr9attention_outputself_attentionsintermediate_states output_statesself_attention_descriptorsself_attention_hidden_statesrrcross_attention_outputcross_attentionscross_intermediate_statescross_output_statescross_attention_hidden_statess r/rMz!LightGlueTransformerLayer.forwardsY#7@BBD0:d  C 1[N B 4?4E1 M>-1,?,?  ))/ -@- - )/ $i6F(GRPPP &9:: %0=%@"  P,?+O ( ' . .r1m^ T T T!WW WZ ? ? )  " "2q!Q > > C CA F F N Nz[\^_an o o o 483G3G &"7#9/ 4H4 4 0 0 %*I/IKa.bhj$k$k$k!"nn-FGG03FF  -FH[,\ )!-55j-Q_``bc./&&z=.QQST0 0   W+.@@DTCVVN-~==r.)FF) r$r%r&rrr7r(rOrrPr,rMrQrRs@r/rr s..3......05,1 H>H>\H><H> H> 'tn H> $D> H> u|XeEL&9:HU5H>H>H>H>H>H>H>r.r similarity matchability0 matchability1c|j\}}}tj|tj|ddz}tj|d}tj|ddddd}|||dz|dzfd} ||z|z| ddd|d|f<tj|d | dddddf<tj|d | dddddf<| S)z;create the log assignment matrix from logits and similarityrrrBrTrN) rkrr logsigmoidr log_softmaxrnew_fullsqueeze) rrrrnum_keypoints_0num_keypoints_1 certaintiesscores0scores1scoress r/sigmoid_log_double_softmaxr]sw4>3C0J-**=99BM--k:: % T5HQ^Qe(f(f(fjn(nn %--jAoq-Q_`` &qqq!t,&qqq!t,#n&>&>r2&F&FF  << aMBBDAJ((,,EAJ((,,66r2>>E5=D#// 5;zGW;X;X;\]]J((55 #++J!OQ qQQ %aaad+%aaad+,JWW r.c||}tj|d}|S)z0Get matchability of descriptors as a probabilityrB)rrrsigmoidr)r=rrs r/get_matchabilityz.LightGlueMatchAssignmentLayer.get_matchabilitys>((55 },,\::BB2FF r.) r$r%r&rr7r(rOrMrrQrRs@r/rrlsIIIIIII5<u| 4ELU\r.rcHeZdZdeffd ZdejdejfdZxZS)LightGlueTokenConfidenceLayerr2cttj|jd|_dSNr)r6r7rr8r9tokenr<s r/r7z&LightGlueTokenConfidenceLayer.__init__s3 Yv4a88 r.rr@c||}tj|d}|S)NrB)rdetachrrrr)r=rrs r/rMz%LightGlueTokenConfidenceLayer.forwardsG ;--//00 %%e,,44R88 r.rrRs@r/rrsj9999999 5<ELr.rc2eZdZUdZeed<dZdZdZdZ dZ dS)LightGluePreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. r2 lightglue pixel_valuesFTN) r$r%r&r'rr*base_model_prefixmain_input_namesupports_gradient_checkpointing_supports_flash_attn_supports_sdpar-r.r/rrsD #$O&+#NNNr.rr thresholdc`|j\}}}|ddddddfd}|ddddddfd}|j}|j}tj|jd|jd}tj|jd|jd} ||d|k} | |d|k} |j}| d} tj | || } tj | | d|| }| | |kz}| |d|z}tj ||d}tj ||d}tj ||g dd |dzd}tj | |g dd |dzd}||fS)z1obtain matches from a score matrix [Bx M+1 x N+1]NrBrrrr)rkmaxindicesr(arangergathervaluesexp new_tensorwhererUrrm)rr r_max0max1matches0matches1indices0indices1mutual0mutual1zeromatching_scores0matching_scores1valid0valid1rrs r/get_matches_from_scoresr#s|J1 !!!SbS#2#+  " "1 % %D !!!SbS#2#+  " "1 % %D|H|H|HN1-hoFFFtLH|HN1-hoFFFtLH(//!X666G(//!X666G ;??  D ??1  D{7D$77{7,<,C,CAx,P,PRVWW (94 5F v}}Q11 1F{68R00H{68R00Hk8X.//99!Q??GG UVXZ[[Gk#35E"FGGQQRSUVWW__`jmn`nprssO O ##r.rheightwidthctj||g|j|jd}|dz }|djdz }||ddddfz |dz }|S)a Normalize keypoints locations based on image image_shape Args: keypoints (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`): Keypoints locations in (x, y) format. height (`int`): Image height. width (`int`): Image width. Returns: Normalized keypoints locations of shape (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`). rr^NrrB.).NN)r(rrr^r r)rr$r%sizeshiftscales r/normalize_keypointsr+sr < 0@  X X XY] ^D 1HE HHRLL ! #EU3aaa<00E/4JJI r.zV LightGlue model taking images as inputs and outputting the matching of them. c4eZdZdZdeffd ZdedefdZ d$de j d e j d e e de e j e e j e j fffd Zd e j ded e j de j de j f dZd%dZde j de j dede j fdZde j d e j d e j de j de j d e j defdZdZde j de j de j de j de e j e j ff dZ d&d e j de j deded e e j de e d e e de e j e j e j e e ffd Zee d&d!e jd"e e jde e d e e dee eff d#ZxZS)'LightGlueForKeypointMatchingar LightGlue is a model matching keypoints in images by leveraging detections from a keypoint detector such as SuperPoint. It is based on the SuperGlue architecture and is designed to be lightweight and efficient. It consists of : 1. Keypoint Encoder 2. A Graph Neural Network with self and cross attention layers 3. Matching Assignment layers The correspondence ids use -1 to indicate non-matching points. Philipp Lindenberger, Paul-Edouard Sarlin and Marc Pollefeys. LightGlue: Local Feature Matching at Light Speed. In ICCV 2023. https://huggingface.co/papers/2306.13643 r2cnttjjj|_jj|_j |_ j |_ j |_ j |_ j|_|j |jkr'tj|j|j d|_ntj|_t'|_tjfdt-j D|_tjfdt-j D|_tjfdt-j dz D|_|dS)N)trust_remote_codeTr4c2g|]}t|S))r)r).0ir2s r/ z9LightGlueForKeypointMatching.__init__..s' e e e &v ; ; ; e e er.c.g|]}tSr-)rr1rr2s r/r3z9LightGlueForKeypointMatching.__init__..s" \ \ \q *6 2 2 \ \ \r.c.g|]}tSr-)rr5s r/r3z9LightGlueForKeypointMatching.__init__..s" ` ` `q *6 2 2 ` ` `r.r)r6r7r from_configkeypoint_detector_configr/keypoint_detectordescriptor_decoder_dim keypoint_detector_descriptor_dimr9num_hidden_layers num_layersfilter_thresholddepth_confidencewidth_confidencerr8input_projectionIdentityr1positional_encoder ModuleListrangetransformer_layersmatch_assignment_layerstoken_confidence post_initr<s `r/r7z%LightGlueForKeypointMatching.__init__s    !>!J  +v?W" " " 170O0f-$3 2 & 7 & 7 & 7  $"G G G$&Id.SUYUhos$t$t$tD ! !$&KMMD !">>r.keypoint_confidencesr! num_pointsc|j\}}||jdz kr||dkd}||dzd}||}d||kd|z z }||jk} n tj |tj } | S)zRevaluate whether we should stop inference based on the confidence of the keypointsrrrrBg?rCr]) rkr=rrmrNr_sumr?r(onesrP) r=rTrJr!rUrrr ratio_confidentearly_stopped_pairss r/_get_early_stopped_image_pairsz;LightGlueForKeypointMatching._get_early_stopped_image_pairs's A 1, , ,$8#C#CDAIq#Q#Q #7#?#? aQS#T#T 66{CCI!%9I%E$L$L$N$N$R$RWX$R$Y$Y\f$ffO"1D4I"I  #(*Zuz"J"J"J ""r.Nc|||}||}|j|||}t||j\}}||fSr)rGr#r>)r=rr!rJ early_stopsrrrs r/_get_keypoint_matchingz3LightGlueForKeypointMatching._get_keypoint_matching:sW  "%k2K $D:-k:;MM#:64CX#Y#Y ''r. confidencesrc^|d|jz k}|||||kz}|S)z#mask points which should be removedr)r@rN)r=r_rrJkeeps r/_get_pruning_maskz.LightGlueForKeypointMatching._get_pruning_maskBs<T223  " K4#A#A+#N#NN ND r.r prune_outputc|j\}} } |j||} ||| ||dkt jdfd||d|d|fD\} } } }}t|D]}||||fxxdz cc<d| | | |fD\} } } }| | f}t|dd}| ||||fS) z For a given layer, prune keypoints based on the confidence of the keypoints and the matchability of the descriptors. rFc3LK|]}dt|DVdS)c$g|] \}}||Sr-r-)r1tr!s r/r3zULightGlueForKeypointMatching._do_layer_keypoint_pruning...^s G G GDQtW G G Gr.N)zip)r1rpruned_keypoints_masks r/ zJLightGlueForKeypointMatching._do_layer_keypoint_pruning..]sYc c  H GC0E$F$F G G Gc c c c c c r.rc38K|]}t|dVdS)T) batch_firstNr)r1 pruned_tensors r/rjzJLightGlueForKeypointMatching._do_layer_keypoint_pruning..esKS S  D 9 9 9S S S S S S r.TrBrl padding_value) rkrGrrbrr(rrEr )r=rrr!rrcrTrJrrdescriptors_matchabilitypruned_descriptorspruned_keypoints_0pruned_keypoints_1 pruned_maskpruned_indicesr2pruned_keypointsris @r/_do_layer_keypoint_pruningz7LightGlueForKeypointMatching._do_layer_keypoint_pruningIsw', Aq#'#? #L#]#]^i#j#j $ 6 67KMegr s s 5 A A$!)U\Z_M`M` a ac c c c & ! ilDY[bcc c c _.0BKQ_z"" 4 4A N1-- . . .! 3 . . . .S S "46HJ\^i!jS S S O.0BK/0BC%n$VXYYY!#3^[R^^^r.ctjtjjd}|}|d||fD\}}d||fD\}}fd||||fD\}}}}||||fS)Nrc3:K|]}t|ddVdS)TrBrnNrr1rs r/rjzMLightGlueForKeypointMatching._concat_early_stopped_outputs..{sE3 3  T D D D3 3 3 3 3 3 r.c3:K|]}t|ddVdS)TrrnNrrzs r/rjzMLightGlueForKeypointMatching._concat_early_stopped_outputs..sE> >  T C C C> > > > > > r.c3(K|] }|V dSrr-)r1rearly_stops_indicess r/rjzMLightGlueForKeypointMatching._concat_early_stopped_outputs..sEg g  & 'g g g g g g r.)r(rUrrk)r=r}final_pruned_keypoints_indices!final_pruned_keypoints_iterationsrrids order_indicess ` r/_concat_early_stopped_outputsz:LightGlueForKeypointMatching._concat_early_stopped_outputsns$k*=>>l.4Q788+C0 1-@3 3 "$BC3 3 3 //> > *,MN> > > ::g g g g .1 g g g c"@Bc./PRY[jjjr.rrrc r|j\}fd|||fD\}}}|dddf}|dddf}|dddf}|dddf} |dddf} |dddf} tjdzd|fd|j|j} tjdzd|f|j|j} t dzD]}tj||dkd||d|| d| |d||f<tj| |dkd||d| | d| |d||f<| || |d||f<| || |d||f<| | fS)Nc3LK|]}|dzddVdS)rrBN)rm)r1rrs r/rjzJLightGlueForKeypointMatching._do_final_keypoint_pruning..sH- - 7=FNN:?Ar 2 2- - - - - - r.rrrrBr')r) rkr(fullrr^zerosrErrclamp)r=rrrrrrrrrrr _matches_matching_scoresr2rs @r/_do_final_keypoint_pruningz7LightGlueForKeypointMatching._do_final_keypoint_prunings+ A- - - - BI7TcAd- - - )/111a4=111a4=111a4=111a4=*111a40*111a40:zQ=A2gndkdqrrr ; 1_a /oNc   zQ'' F FA*/+ r!2x{'9'9!Xa[=N=NST=N=U=U'V'V++HQ8A;& '+0+ r!2x{'9'9!Xa[=N=NST=N=U=U'V'V++HQ8A;& '3C12E Q8A;. /2B12E Q8A;. / /)))r.r$r%rc  (|rdnd}|rdnd} |jddkr\|jdd} || dtj|| || || fS|j} |j\} } }} tj|| dd}|| dz|d}||| dz|nd}|| dz||j}tj | dz| }t|||}| ||| \}}|d}|j dk}|j dk}g}g}g}g}g}tj d|| | dzd}tj|}t!|jD]}|}||||}n#tj| |d f|j}|j|||||| }|\}}} |r||z}|r| | z} |r||jdz kr0|j||}!||!||| }"n tj| tj}"tj|"rT|"d(|(}#||||( \}$}%|t;|#|t;|$|t;|%|rP|t;|(|t;|(||"}t=(fd||d|d||fD\}}&}'}}|&|'f}|r"t=(fd|||!fD\}}}!tj|"rn%|r!| ||||||!|\}}}}}|r<|r:|!|||||\}}}}|"||||\}}n>||||jdz \}}tj||jz}|| d|}||||| fS)Nr-rrrBr]rrCrrPrT)rxr?r)rU)r]c3*K|] }|VdSrr-r1rr]s r/rjzALightGlueForKeypointMatching._match_image_pair..sFVV" |,VVVVVVr.c3*K|] }|VdSrr-rs r/rjzALightGlueForKeypointMatching._match_image_pair..#sFll &#K<0llllllr.)#rkrr(r new_zerosrrWrmr;rr+rSr?r@rl ones_likerEr=r(get_extended_attention_maskrXrFrHr[rPanyrEr^extendlistr,allrwrr))r=rrr$r%r!rr?rrrkrrrinitial_num_keypointsnum_points_per_pair image_indicesrR do_early_stopdo_keypoint_pruningr}rrr~rpruned_keypoints_indicespruned_keypoints_iterationsrJrextended_attention_mask layer_outputr" attentionrTrZearly_stopped_image_indicesearly_stopped_matchesearly_stopped_matching_scores keypoints_0 keypoint_1r]s) @r/_match_image_pairz.LightGlueForKeypointMatching._match_image_pairsz#7@BBD0:d ?1  " "OCRC(E""5"EI">>##E**##E**!  !2;// A,a#i Z(D(D!LLL%%j1n6KQOO FJFVt||JN,ABBB\`!))*q.:OQUQvww  Z!^FCCC ' 65AA 040I0I 9M1J1 1 - --Q/ -1 #3a7 )+&,.)#(<3HQW#X#X#X#_#_`jmn`npr#s#s &+o6N&O&O# 11R R K%**,,K*.*J*J4Q\*]*]''*/*j+b/5R[d[k*l*l*l'?42;?6%9"3 L5A 1K # F$5 $E!  #>???#**40M+N+NOOO*q6==dC[\gCh>i>ijjj9@@FabmFnAoAoppp+>?R>R*S'PUVVVV'2IaL)A,PTVc&dVVVQQMKj$ "-j 9I*fkllll!9 ; 4+lllggc02MOc9011E" 33#!03,#d Y(@$Hc  c0 c22'25#  h *,MwXg(,'F'F.% (( $G__(,'B'B;PTVZVehiVi'j'j $G_050P0PSWSb0b -,M,U,U 0- - )   -     r.rlabelsc ld}|td||n |jj}||n |jj}|jdks|ddkrtd|j\}}}} } ||dz|| | }||} | dd\} }} }| |ddd |} | |dd|j  |} ||dd}| }|dddddddf| z|dddddddf<|dddddddf| z|dddddddf<| || | | ||| \}}}}}t|||| |||| S) Nz9LightGlue is not trainable, no labels should be provided.rrzOInput must be a 5D tensor of shape (batch_size, 2, num_channels, height, width)rBr)r!rr?)rrrrr r!r"r#) ValueErrorr2rr?ndimr(rkrmr9rar;clonerr)r=rrrr?rrrchannelsr$r%keypoint_detectionsrrr!absolute_keypointsrrr r"r#s r/rMz$LightGlueForKeypointMatching.forward`s@  XYY Y1B1N--TXT_Tq$8$D $+Jj    ! !\%6%6q%9%9Q%>%>noo o1=1C. Ax#++JNHfeTT "44\BB*=bqb*A' 1k4%%j!R;;>>|LL !))*aT=bccffgstt ||J2..&__..);AAAqqq!!!QJ)G%)O111aaaA:&);AAAqqq!!!QJ)G&)P111aaaA:&EIE[E[    /!5F\F F B% /+'!    r.rNr)NNN)r$r%r&r'rr7rr_rNr(rOrrPr,rSr[r^rbrwrrrrrr) LongTensorrrrMrQrRs@r/r-r-sw  @(S(U(((( jo?? <?49L?X`aeXf? u|U5<#=>> ?????#$)L#?B#JO,#didp# ####&((((U\5<^afkfr#_\#_<#_l #_  #_ l #_$l#_#_#_#_#_Jkkk@#*#*#* #* | #* u|U\) * #*#*#*#*V(,,0/3k k <k \k  k  k u|$ k $D>k 'tnk  u|U\5<E Fk k k k Z.2,0/3 3 3 '3 )*3 $D> 3 'tn 3 u55 6 3 3 3 ^3 3 3 3 3 r.r-r)rs)7 dataclassesrtypingrrrnumpyrLr(rtorch.nn.utils.rnnr activationsr modeling_flash_attention_utilsr modeling_utilsr rprocessing_utilsrutilsrrrutils.deprecationr utils.genericrauto.modeling_autorconfiguration_lightgluerrModuler1r[rhrOrrrr_rrrrrrrrr,r#r+r-__all__r-r.r/rs("!!!!!,,,,,,,,,, ++++++!!!!!!BBBBBBFFFFFFFF&&&&&&DDDDDDDDDD000000------>>>>>>444444  : : : : :k : :  :F"<<<<< UU\ U# U%, U U U U&%% I% <% % < % U\* %  %%'(%%%%4E)E)E)E)E)E)E)E)P29"P>P>P>P>P> P>P>P>f   -2\ JO,  \    &&&&&BI&&&R     BI           $EL$U$uU\[`[gMgGh$$$$@5<SU\, j j j j j #;j j  j Z &'E Fr.