`i+ ddlZddlmZddlmZddlmZddlmZm Z m Z m Z erddl m Z  dd ejd ejd ed ed ef dZGddeZGdde Z ddZdS)N)center_to_corners_format)is_scipy_available)HungarianMatcher ImageLoss _set_aux_lossgeneralized_box_ioulinear_sum_assignment?inputstargets num_boxesalphagammac |}tj||d}||zd|z d|z zz}|d|z |zz}|dkr||zd|z d|z zz} | |z}||z S)aR Loss used in RetinaNet for dense detection: https://huggingface.co/papers/1708.02002. Args: inputs (`torch.FloatTensor` of arbitrary shape): The predictions for each example. targets (`torch.FloatTensor` with the same shape as `inputs`) A tensor storing the binary classification label for each element in the `inputs` (0 for the negative class and 1 for the positive class). num_boxes (`int`): The total number of boxes in the batch. alpha (`float`, *optional*, defaults to 0.25): Optional weighting factor in the range (0,1) to balance positive vs. negative examples. gamma (`int`, *optional*, defaults to 2): Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Returns: Loss tensor none) reductionrr)sigmoidnn functional binary_cross_entropy_with_logitssum) rrrrrprobce_lossp_tlossalpha_ts y/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/loss/loss_grounding_dino.pysigmoid_focal_lossr!s4 >>  Dm<eZdZejdZdS)GroundingDinoHungarianMatcherc|djdd\}}|ddd}|ddd}|d}tjdt ||D}||d d z }tjd |D}d } d} d| z || zzd|z dz z} | d|z | zz|dz z} | | z |z} tj ||d}tt|t| }|j |z|j | zz|j|zz}|||d }d|D}dt#||d D}d|DS)a Args: outputs (`dict`): A dictionary that contains at least these entries: * "logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits * "pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates. * "label_maps": Tuple of tensors of dim [num_classes, hidden_dim]. targets (`list[dict]`): A list of targets (len(targets) = batch_size), where each target is a dict containing: * "class_labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels * "boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates. Returns: `list[Tuple]`: A list of size `batch_size`, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) logitsNrrr pred_boxes label_mapsc0g|]\}}||dS) class_labels).0 label_maptargets r z9GroundingDinoHungarianMatcher.forward..bs(tttFWiQW &*@ Atttr"T)dimkeepdimcg|] }|d Sboxesr+r,vs r r/z9GroundingDinoHungarianMatcher.forward..gs = = =7 = = =r"r @g:0yE>)pc8g|]}t|dSr4lenr6s r r/z9GroundingDinoHungarianMatcher.forward..{s"222QQwZ222r"c>g|]\}}t||Sr+r )r,ics r r/z9GroundingDinoHungarianMatcher.forward..|s)ccc41a(1..cccr"cg|]E\}}tj|tjtj|tjfFS))dtype)torch as_tensorint64)r,r>js r r/z9GroundingDinoHungarianMatcher.forward..}sHssscgcdfg%+666QVQ\8]8]8]^sssr")shapeflattenrrBcatziprlogtcdistr r bbox_cost class_cost giou_costviewcpu enumeratesplit)selfoutputsr batch_size num_queriesout_probout_bboxr( target_bboxrrneg_cost_classpos_cost_classrNrMrO cost_matrixsizesindicess r forwardz%GroundingDinoHungarianMatcher.forwardDs/,#*("3"9"1""= K8$,,Q22::<<<(00A66\* Ytt[^_ikr[s[stttuu *..R."F"FF i = =W = = =>> e)%8a(lT>Q=V=V=X=XVW$~5G K+;;; ))A()K)KMefqMrMrsss ny04?Z3OORVR`clRll !&&z;CCGGII 22'222cc9[EVEVW\^`EaEa;b;bcccsskrssssr"N)__name__ __module__ __qualname__rBno_gradr`r+r"r r$r$Cs:U]__8t8t_8t8t8tr"r$c$eZdZdZdZdZdZdS)GroundingDinoImageLossa This class computes the losses for `GroundingDinoForObjectDetection`. The process happens in two steps: 1) we compute hungarian assignment between ground truth boxes and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and box). Args: matcher (`GroundingDinoHungarianMatcher`): Module able to compute a matching between targets and proposals. focal_alpha (`float`): Alpha parameter in focal loss. losses (`list[str]`): List of all the losses to be applied. See `get_loss` for a list of all available losses. cntj|||_||_||_dSN)rModule__init__matcher focal_alphalosses)rTrkrlrms r rjzGroundingDinoImageLoss.__init__s2 4    & r"c d}tjfdtt||D}tjdd}||}tj||jtj}||tj||<|S)z> Create one_hot based on the matching indices r&cg|]I\}\}\}}|dkr*|d|td|zn |d|JS)rr*r(r;)r,r>r._JrUs r r/zFGroundingDinoImageLoss._get_target_classes_one_hot..st   'A'ANOQRUU~&q)C 0Ea0H,I,IIIX^_mXnopXq   r"r(r)r1)devicerA) rBrHrRrI_get_source_permutation_idx zeros_likerrlongto) rTrUrr_r&r*r(idxtarget_classes_onehots ` r _get_target_classes_one_hotz2GroundingDinoImageLoss._get_target_classes_one_hots"y    +4S'5J5J+K+K     Yw|4!<<< ..w77 % 0 UZU_ ` ` `%/ %=%@%@%L%Lc"$$r"cVd|vrtdd|vrtd||||}|d}|d}tj||}tj||}|}t ||||jd}d|i} | S)z Classification loss (Binary focal loss) targets dicts must contain the key "class_labels" containing a tensor of dim [nb_target_boxes] r&z#No logits were found in the outputs text_maskz&No text_mask were found in the outputsr)rrrrrloss_ce)KeyErrorryrB masked_selectfloatr!rl) rTrUrr_rrx source_logitsr{r|rms r loss_labelsz"GroundingDinoImageLoss.loss_labelss 7 " "@AA A g % %CDD D $ @ @'SZ [ [) K( +M9EE % 34I9 U U 5 ; ; = =$ )"    W% r"N)rarbrc__doc__rjryrr+r"r rfrfsK   %%%(r"rfc t|j|j|j} gd} t | |j| } | |i}||d<||d<||d<||d<d}|jr$t||}|D] }||d<||d< ||d<| |||j rF| | ||d }| ||}d | D} |d |j |j d |j r3d  D} ||jrdi}t|jdz D]5| fd D6 |t!fdD}||fS)N)rNrMrO)labelsr5 cardinality)rkrlrmr&r'r(r{auxiliary_outputs)r&r'r(r{c i|] \}}|dz| S_encr+r,kr7s r z7GroundingDinoForObjectDetectionLoss..s"QQQtq!QZQQQr"r8)r| loss_bbox loss_giouc i|] \}}|dz| Srr+rs r rz7GroundingDinoForObjectDetectionLoss..s"IIITQ1v:qIIIr"rc(i|]\}}|dz|S)rpr+)r,rr7r>s r rz7GroundingDinoForObjectDetectionLoss.. s)#S#S#Stq!AAK#S#S#Sr"c3BK|]}|v||zVdSrhr+)r,r loss_dict weight_dicts r z6GroundingDinoForObjectDetectionLoss..s:TT1 CSCSy|k!n,CSCSCSCSTTr")r$rNrMrOrfrlrvauxiliary_lossr two_stageitemsupdatebbox_loss_coefficientgiou_loss_coefficientrangedecoder_layersr)r&rrrr'configr(r{ outputs_class outputs_coordencoder_logitsencoder_pred_boxesrkrm criterion outputs_lossr aux_outputencoder_outputs_lossencoder_loss_dictenc_weight_dictaux_weight_dictrr>rrs @@@r #GroundingDinoForObjectDetectionLossrsx,$0@FL\G0 / /F&&I LLL#L!+L!+L )L >)-GG+ 0 0J'1J| $&/J{ # #,= () ,//I  ,$,$"    &I&:FCCQQ7H7N7N7P7PQQQ*+++11K ,II[5F5F5H5HIII?+++ ,v,q011 U UA  " "#S#S#S#S{?P?P?R?R#S#S#S T T T T?+++ TTTTTiTTT T TD - --r")r r)NNNN)rBtorch.nnrimage_transformsrutilsrloss_for_object_detectionrrr r scipy.optimizer Tensorintrr!r$rfrr+r"r rs| 777777&&&&&&ffffffffffff5444444 $"$" L$" \$"$"  $"  $"$"$"$"N:t:t:t:t:t$4:t:t:tzFFFFFYFFFbF.F.F.F.F.F.r"