&`i#ddlZddlmZmZmZmZddlZddlm Z m Z ddl m Z ddl mZmZddlmZddlmZejeZe \ZZe \ZZZeGdd e ZdS) N)AnyDictListUnion) try_import_tftry_import_torch) StatsBase) safe_isnansingle_value_to_cpu)log_once) DeveloperAPIc HeZdZdZdZ ddeffd ZdefdZde ddd fd Z d e dd fd Z defd Z ddededee e e ffdZddedee dffdZdefdZdeee fffd Zdeee fdd ffd Zeddeee ffdZxZS)EmaStatsz`A Stats object that tracks the exponential average of a series of singular values (not vectors).ema{Gz? ema_coeffctj|i|tj|_|jsg|_||_dS)aInitializes a EmaStats instance. We calculate the EMA in parallel components. Also, we potentially aggregate them multiple times per reduction cycle. We therefore aggregate by taking the mean of all collected EMAs. We do this for simplicity and accept this limitation because EMAs inherently only approximate. Example to illustrate this limitation: Using an ema coefficient of 0.01: First incoming ema: [1, 2, 3, 4, 5] -> 1.1 Second incoming ema: [15] -> 15 Mean of both merged ema values: [1.1, 15] -> 8.05 True mean of all values: [1, 2, 3, 4, 5, 15] -> 5 Args: ema_coeff: The EMA coefficient to use. Defaults to 0.01. N)super__init__npnan_valueis_leaf_values_to_merge _ema_coeff)selfrargskwargs __class__s u/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/utils/metrics/stats/ema.pyrzEmaStats.__init__sG0 $)&)))f | '$&D !#returncdS)z/Returns the length of the internal values list.)rs r __len__zEmaStats.__len__6sqr!incoming_statsNc|jr Jdd|D}t|dkrdS|j||js ||_dSdS)zMerges EmaStats objects. Args: incoming_stats: The list of EmaStats objects to merge. Returns: None. The merge operation modifies self in place. zKEmaStats should only be merged at aggregation stages (root or intermediate)cg|] }|j Sr%)r).0stats r z"EmaStats.merge..Gs===ddk===r!rN)rlenrextend latest_merged)rr' all_valuess r mergezEmaStats.merge:s  Y Y X Y Y >=n=== z??a   F $$Z000| ,!+D    , ,r!valuecztr.t|r|}t|rdStr.t |tjr|}t|jr ||_dS|j |zd|j z |jzz|_dS)a Pushes a value into this Stats object. Args: value: The value to be pushed. Can be of any type. PyTorch GPU tensors are kept on GPU until reduce() or peek(). TensorFlow tensors are moved to CPU immediately. Ng?) tf is_tensornumpyr torch isinstanceTensordetachrr)rr2s r pushz EmaStats.pushRs  "",,u%% "KKMME e    F  #Zu|44 #LLNNE dk " " DKKK%'3+@DK*OO KKKr!ctj|js-tdrtd|dt |jdkr tjStrkt|jdtj rFt t|j}t|Stj|jS)z%Reduces the internal values to merge.ema_stats_merge_pushzMerging values in zv but self._value is not NaN. This leads to an inaccurate metric. Not erroring out to avoid breaking older checkpoints.r)risnanrr loggerwarningr-rrr7r8r9stacklistnanmean)rstackeds r _reduce_values_to_mergez EmaStats._reduce_values_to_mergeosx $$ 2H)I)I  NNbTbbb    t$ % % * *6M  *Z 5a 8%,GG *kk$t'<"="=>>G==)) )z$/000r!TFcompilelatest_merged_onlyc|r|jrtd|r|j|r tjStjgS|j}t |dkr tj}nt rbt|dt jrBt t|}t |}natj |}nLt|dr5t |j dkr|j}n|}n|j}t!|}|r|n|gS)aReturns the current EMA value. If value is a GPU tensor, it's converted to CPU. Args: compile: If True, the result is compiled into a single value if possible. latest_merged_only: If True, only considers the latest merged values. This parameter only works on aggregation stats (root or intermediate nodes). When enabled, peek() will only use the values from the most recent merge operation. zQlatest_merged_only can only be used on aggregation stats objects (is_leaf=False).Nrr)r ValueErrorr/rrr-r7r8r9rArBrChasattrrrrEr )rrFrGr/r2rDs r peekz EmaStats.peek~sF  $, c   $!)$6MF8O .M=!!Q&&6Z a(8%,GG6#kk$}*=*=>>G!MM'22EEJ}55EEt/00 $t,--22 KEE 88::EE #E**,uueW,r!clt|drEtCH~E$EEEEEENNc3hNNN\NNNNNr!r)loggingtypingrrrrr6rray.rllib.utils.frameworkrr"ray.rllib.utils.metrics.stats.baser #ray.rllib.utils.metrics.stats.utilsr r ray.utilr ray.util.annotationsr getLoggerrar?r7_r4rr%r!r rus-))))))))))))EEEEEEEE888888OOOOOOOO------  8 $ $    q =??2qmNmNmNmNmNymNmNmNmNmNr!