`i*< hddlZddlZddlZddlmZddlmZmZmZm Z GddeZ ddl m Z ddl mZmZmZdZn #e$rd ZYnwxYwdd Z ddd dd d eeeeee eeegeffffd ZejeZeGddZdS)N)Enum)AnyCallableOptionalUnionc.eZdZdZdZdZdZdZdZdZ dZ d S) RequestStatusz5Status of a generation request through its lifecycle.pending prefillingprefilling_splitsplit_pending_remainderdecodingfinishedfailedN) __name__ __module__ __qualname____doc__PENDING PREFILLINGPREFILLING_SPLITSPLIT_PENDING_REMAINDERDECODINGFINISHEDFAILEDn/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/utils/metrics.pyr r s9??GJ)7HH FFFrr )metrics)Status StatusCode get_tracerTFc&tsdSfd}|S)a Decorator that attaches a tracer to a class. This decorator should be applied to classes that need OpenTelemetry tracing. It adds a tracer attribute to the class instance that can be used by the traced decorator. Args: tracer_name_template: Optional template string for the tracer name. If provided, it should contain {module} which will be replaced with the class's full module path and {class_name} for the class name. If None, a default naming scheme will be used where: - If the module already starts with "transformers.", it will use that directly - Otherwise, it will prepend "transformers." to the module name Returns: Class decorator function c|SNr)clss rzattach_tracer..0s3rcjjtjfd}|_S)Nc|g|Ri|j}j}&|dr|d|}n d|d|}n||}t ||_dS)N transformers..)module class_name)rr startswithformatr"tracer) selfargskwargs module_namer- tracer_namer& original_inittracer_name_templates rinit_with_tracerz:attach_tracer..decorator..init_with_tracer5s M$ 0 0 0 0 0 0 0.K)J#+))/::M%0"?"?:"?"?KK"L+"L"L "L"LKK299Yc9dd $[11DKKKr)__init__ functoolswraps)r&r8r6r7s` @r decoratorz attach_tracer..decorator2sR  ' ' 2 2 2 2 2 2 ( ' 2 (  r)_has_opentelemetry)r7r<s` r attach_tracerr>s6$ . r) span_name standaloneadditional_attributesrAc4fd}||S||S)a Decorator to trace function calls with OpenTelemetry. Can be used as @traced or @traced(span_name="custom_name") Args: func: The function to trace span_name: Optional custom name for the span (defaults to function name) standalone: If True, creates a parentless span additional_attributes: Optional list of additional attributes to set on the span. Each item is a tuple of (instance_attribute_name, span_attribute_key, value_or_transform_function) where: - instance_attribute_name: Name of the attribute to get from the class instance - span_attribute_key: Key to use when setting the attribute on the span - value_or_transform_function: Either a raw value to use directly, or a function to transform the attribute value before setting it on the span Returns: Decorated function with tracing c`tsStjfd}|S)Nc |rtdrj|dnd}|du}|rt|dr|j}ntdjdj}pj}r|jn|j}||5}|dj|dj|d||rt|D]\}} t| ttttfs| '|d |t| V|d |tt| |r|D]\} } t| ttttfs| '|d | t| V|d | tt| r`|r^D][} | \} }}t|| rCt#|| }t%|r ||}n|}|||\ |i|}|cdddS#t&$rG}|t+t,j||d}~wwxYw#1swxYwYdS) N__self__rr0r*r+z function.namezfunction.modulezfunction.is_methodzargs.zkwargs.)hasattrrEr0r"rr start_spanstart_as_current_span set_attribute enumerate isinstancestrintfloatbooltypeitemsgetattrcallable Exception set_statusr r!ERRORrecord_exception)r2r3instance is_methodr0namespan_fnspaniargkeyvalue attr_configinstance_attribute_namespan_attribute_keyvalue_or_transform_functionattribute_valuetransformed_valueresulterAfuncr?r@s rwrapperz*traced..decorator..wrapperls"&nGD*,E,En$-JctAwwjnH ,I WWXx88 W!#$UDO$U$Udm$U$UVV- D+5Wf''6;WG# $""?DMBBB""#4doFFF""#7CCCL"+D//LL3%cCeT+BCCLs{ ..{q{{CHHEEEE ..{q{{CS NNKKKKR&,llnnRR U%ec3t-DEER ..E KKKK ..DKK@P@PQQQQ( VY V'<VV cn`/1CE`"8-DEEV.5h@W.X.XO'(CDDP4O4OP_4`4` 1 14O 1 ../ACTUUU!T42622F!?# # # # # # # # @!OOF:+;$<$<===))!,,,A# # # # # # # # # # s2G*K1 J KAKKKK"K)r=r:r;)rirjrAr?r@s` rr<ztraced..decoratorhsV! K   . . . . . . .   . `rr)rir?r@rAr<s ``` rtracedrkLsD85555555n | 9T??rceZdZdZdefdZdZedede ddfd Z ed e ddfd Z edd Z ed ededdfdZedede ddfdZdS)ContinuousBatchProcessorMetricsz0Metrics collection for ContinuousBatchProcessor.max_batch_tokensc<||_|dS)zInitialize metrics for continuous batch processor. Args: max_batch_tokens: Maximum number of tokens in a batch N)rn_setup_metrics)r1rns rr9z(ContinuousBatchProcessorMetrics.__init__s$ !1 rcDtstddStjd|_gd}|jddd||_|jd d d |_ |jd dd |_ gd}|jddd||_ |jddd |_ |j ddd |_|j ddd |_gd}|jddd||_|jddd |_|jd!d"d |_dS)#zIInitialize OpenTelemetry metrics and tracing if the library is available.zIOpenTelemetry is not installed. Metrics and tracing will not be recorded.Nz2transformers.generation.continuous_batch_processor) 2Kdi,i'ttft_millisecondsz#Time to first token in millisecondsms)rZ descriptionunit#explicit_bucket_boundaries_advisoryactive_requests_countz3Number of active requests currently being processedrequests)rZrrwaiting_requests_countz*Number of requests waiting to be processed) rtrvryrzr{r|r}i Ni0ui`request_latency_millisecondsz9End-to-end latency for completed requests in millisecondsdecode_prefill_ratioz3Ratio of decode tokens to prefill tokens in a batchratioprefill_tokens_processedz"Number of prefill tokens processedtokensdecode_tokens_processedz!Number of decode tokens processed) rr(rt<FPZ_brvbatch_fill_percentagez5Percentage of max_batch_tokens utilized in each batchpercentkv_cache_free_memory_bytesz/Free memory of the PagedAttentionCache in bytesbyteskv_cache_memory_bytesz0Memory usage of the PagedAttentionCache in bytes)r=loggerinfor get_metermetercreate_histogramttft_histogram create_gaugeactive_requests_gaugewaiting_requests_gaugerequest_latency_histogramdecode_prefill_ratio_gaugecreate_counterprefill_tokens_counterdecode_tokens_counterbatch_fill_percentage_histogramkv_cache_free_memory_gaugekv_cache_memory_gauge)r1 ttft_bucketslatency_bucketsbatch_fill_bucketss rrpz.ContinuousBatchProcessorMetrics._setup_metricss"  KKc d d d F&'[\\ _^^ "j99$=0< :  &*Z%<%<(M&=& & " '+j&=&=)D'>' ' #\[[)-)D)D/S0? *E* * &+/**A*A'M+B+ + ' '+j&?&?+<'@' ' # &*Z%>%>*;&?& & "RQQ/3z/J/J(O0B 0K0 0 ,+/**A*A-I+B+ + ' &*Z%<%<(J&=& & """r created_time request_idreturnNc.tsdStj|z dz} |j|td|d|dddS#t $r(}td|Yd}~dSd}~wwxYw)zRecord Time to First Token (TTFT). Args: created_time: The time the request was created request_id: The ID of the request N@@zRecorded TTFT for request : .2frzFailed to record TTFT metric: )r=timerrecordrdebugrTwarning)r1rrttft_msrhs rrecord_ttft_metricz2ContinuousBatchProcessorMetrics.record_ttft_metrics"  F9;;-7 A   & &w / / / LLSjSSGSSSS T T T T T A A A NN?A?? @ @ @ @ @ @ @ @ @ AContinuousBatchProcessorMetrics.record_kv_cache_memory_metricsBs"  F M)BBI )EK,@ @ "#S%9%9!9EAA%"22S8@AAA       M M M NNKKK L L L L L L L L L MsCC%% D/DDactive_requestswaiting_requestsc.tsdS |j||j|td|d|ddS#t $r(}td|Yd}~dSd}~wwxYw)zRecord metrics about active and waiting requests. Args: active_requests: Number of active requests waiting_requests: Number of waiting requests NzQueue metrics: z active requests, z waiting requestsz Failed to record queue metrics: )r=rrrrrrTr)r1rrrhs rrecord_queue_metricsz4ContinuousBatchProcessorMetrics.record_queue_metricsjs"  F C  & * *? ; ; ;  ' + +,< = = = LLq?qqN^qqq r r r r r C C C NNAaAA B B B B B B B B B CsAA"" B,BBc.tsdStj|z dz} |j|td|d|dddS#t $r(}td|Yd}~dSd}~wwxYw)zRecord metrics about a completed request. Args: created_time: The time the request was created request_id: The ID of the request Nrz Recorded request completion for rrrz,Failed to record request completion metric: )r=rrrrrrTr)r1rr latency_msrhs rrecord_request_completionz9ContinuousBatchProcessorMetrics.record_request_completion|s"  FikkL0F:  O  * 1 1* = = = LL\J\\*\\\\ ] ] ] ] ] O O O NNM!MM N N N N N N N N N Or)rN)rrrrrMr9rprkrNrLrlistrrrrrrrrmrmsZ::O O O b AuA#A$AAA VA$ &Cd&Ct&C&C&C V&CP %M%M%M V%MN CCC3CSWCCC VC" OeOOQUOOO VOOOrrmr%)r:loggingrenumrtypingrrrrr opentelemetryropentelemetry.tracer r!r"r= ImportErrorr>rtuplerLrk getLoggerrrrmrrrrs 111111111111     D   %%%%%%BBBBBBBBBB,,,,` U_c UUU $DsCsHcUTWZDX?X9Y/Y)Z$[\ UUUUp  8 $ $fOfOfOfOfOfOfOfOfOfOsAA  A