&`i(ddlmZmZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZddlmZed Gd d ZdS) )OptionalUnionN)Result) RayTaskError)ExperimentAnalysis) TuneError)Trial) PublicAPIbeta) stabilityc eZdZdZdefdZedefdZede j j fdZ dd e ed e ed ed edef dZ dde ede edejfdZdefdZdedefdZedZedZedZedede eeeffdZdedefdZ defdZ!dS) ResultGrida2A set of ``Result`` objects for interacting with Ray Tune results. You can use it to inspect the trials and obtain the best result. The constructor is a private API. This object can only be created as a result of ``Tuner.fit()``. Example: .. testcode:: import random from ray import tune def random_error_trainable(config): if random.random() < 0.5: return {"loss": 0.0} else: raise ValueError("This is an error") tuner = tune.Tuner( random_error_trainable, run_config=tune.RunConfig(name="example-experiment"), tune_config=tune.TuneConfig(num_samples=10), ) try: result_grid = tuner.fit() except ValueError: pass for i in range(len(result_grid)): result = result_grid[i] if not result.error: print(f"Trial finishes successfully with metrics" f"{result.metrics}.") else: print(f"Trial failed with error {result.error}.") .. testoutput:: :hide: ... You can also use ``result_grid`` for more advanced analysis. >>> # Get the best result based on a particular metric. >>> best_result = result_grid.get_best_result( # doctest: +SKIP ... metric="loss", mode="min") >>> # Get the best checkpoint corresponding to the best result. >>> best_checkpoint = best_result.checkpoint # doctest: +SKIP >>> # Get a dataframe for the last reported results of all of the trials >>> df = result_grid.get_dataframe() # doctest: +SKIP >>> # Get a dataframe for the minimum loss seen for each trial >>> df = result_grid.get_dataframe(metric="loss", mode="min") # doctest: +SKIP Note that trials of all statuses are included in the final result grid. If a trial is not in terminated state, its latest result and checkpoint as seen by Tune will be provided. See :doc:`/tune/examples/tune_analyze_results` for more usage examples. experiment_analysiscP|_fdjjD_dS)Nc:g|]}|S)_trial_to_result).0trialselfs h/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/tune/result_grid.py z'ResultGrid.__init__..Os4   -2D ! !% ( (   )_experiment_analysistrials_results)rrs` r__init__zResultGrid.__init__JsA%8!    6:6O6V    rreturnc|jjS)zPath pointing to the experiment directory on persistent storage. This can point to a remote storage location (e.g. S3) or to a local location (path on the head node).)rexperiment_pathrs rr zResultGrid.experiment_pathSs (88rc|jjS)zReturn the filesystem that can be used to access the experiment path. Returns: pyarrow.fs.FileSystem implementation. )r_fsr!s r filesystemzResultGrid.filesystem[s(,,rNlastTmetricmodescopefilter_nan_and_infct|jjdkr%||jjdS|s|jjst d|s|jjst d|j||||}|s*d|p |jjd}||rdnd z }t|||S) aGet the best result from all the trials run. Args: metric: Key for trial info to order on. Defaults to the metric specified in your Tuner's ``TuneConfig``. mode: One of [min, max]. Defaults to the mode specified in your Tuner's ``TuneConfig``. scope: One of [all, last, avg, last-5-avg, last-10-avg]. If `scope=last`, only look at each trial's final step for `metric`, and compare across trials based on `mode=[min,max]`. If `scope=avg`, consider the simple average over all steps for `metric` and compare across trials based on `mode=[min,max]`. If `scope=last-5-avg` or `scope=last-10-avg`, consider the simple average over the last 5 or 10 steps for `metric` and compare across trials based on `mode=[min,max]`. If `scope=all`, find each trial's min/max score for `metric` based on `mode`, and compare trials based on `mode=[min,max]`. filter_nan_and_inf: If True (default), NaN or infinite values are disregarded and these trials are never selected as the best trial. rzNo metric is provided. Either pass in a `metric` arg to `get_best_result` or specify a metric in the `TuneConfig` of your `Tuner`.z|No mode is provided. Either pass in a `mode` arg to `get_best_result` or specify a mode in the `TuneConfig` of your `Tuner`.)r&r'r(r)z*No best trial found for the given metric: z3. This means that no trial has reported this metricz~, or all values reported for this metric are NaN. To not ignore NaN values, you can set the `filter_nan_and_inf` arg to False..) lenrrrdefault_metric ValueError default_modeget_best_trial RuntimeError)rr&r'r(r) best_trial error_msgs rget_best_resultzResultGrid.get_best_resultdsA8 t(/ 0 0A 5 5(()B)I!)LMM M d7F 0   D5B 0  .==1 >   *DET6EDDD  &MM  I y)) )$$Z000r filter_metric filter_modec:|j||S)a4Return dataframe of all trials with their configs and reported results. Per default, this returns the last reported results for each trial. If ``filter_metric`` and ``filter_mode`` are set, the results from each trial are filtered for this metric and mode. For example, if ``filter_metric="some_metric"`` and ``filter_mode="max"``, for each trial, every received result is checked, and the one where ``some_metric`` is maximal is returned. Example: .. testcode:: import ray.tune def training_loop_per_worker(config): ray.tune.report({"accuracy": 0.8}) result_grid = ray.tune.Tuner( trainable=training_loop_per_worker, run_config=ray.tune.RunConfig(name="my_tune_run") ).fit() # Get last reported results per trial df = result_grid.get_dataframe() # Get best ever reported accuracy per trial df = result_grid.get_dataframe( filter_metric="accuracy", filter_mode="max" ) .. testoutput:: :hide: ... Args: filter_metric: Metric to filter best result for. filter_mode: If ``filter_metric`` is given, one of ``["min", "max"]`` to specify if we should find the minimum or maximum result. Returns: Pandas DataFrame with each trial as a row and their results as columns. )r&r')r dataframe)rr6r7s r get_dataframezResultGrid.get_dataframes*f(22 {3   rc*t|jSN)r-rr!s r__len__zResultGrid.__len__s4=!!!ric|j|S)z$Returns the i'th result in the grid.)r)rr>s r __getitem__zResultGrid.__getitem__s}Qrcd|DS)z)Returns the exceptions of errored trials.c*g|]}|j |jSr)errorrresults rrz%ResultGrid.errors..s!@@@6<@ @@@rrr!s rerrorszResultGrid.errorssA@4@@@@rcHtd|jjDS)z%Returns the number of errored trials.c<g|]}|jtjk|Sr)statusr ERRORrts rrz)ResultGrid.num_errors..s& T T T1AH .s2   8u//////rrMr!s rnum_terminatedzResultGrid.num_terminateds8  29      rrc|jtjkrdS|p|Sr<)rIr rQget_pickled_error get_error)rs r_populate_exceptionzResultGrid._populate_exceptions8 <5+ + +4&&((=EOO,=,==rc T|jj}d}|jr |jj}|j}d|D}|jj|j}t||j | ||j |jj||}|S)Nc*g|]}|j|jfSr) checkpointmetrics)rcheckpoint_results rrz/ResultGrid._trial_to_result..s4   ! )+<+D E   r)rYrZrCpath_storage_filesystemmetrics_dataframebest_checkpoints) run_metadatacheckpoint_managerlatest_checkpoint_resultrYbest_checkpoint_resultsrtrial_dataframesgettrial_idr last_resultcopyrVr\r#)rrcpmrYrcr_ metrics_dfrEs rrzResultGrid._trial_to_results 3  ' A5@J"%"=  %<    .?CCENSS !%**,,**511 $ 9 =(-    rcPd|D}d|}d|dS)Nc:g|]}|dS))indent)_reprrDs rrz'ResultGrid.__repr__..s&FFFvFLLL22FFFrz, z ResultGrid<[ z ]>)join)rall_results_reprs r__repr__zResultGrid.__repr__s;FFFFF ::&6776 06666r)NNr%T)NN)"__name__ __module__ __qualname____doc__rrpropertystrr pyarrowfs FileSystemr$rboolrr5pd DataFramer:intr=r@rFrNrR staticmethodr rrrrVrrrrrrrrsA88t /    9999X9-GJ1---X-!%"#' ?1?1 ?1sm?1 ?1 ! ?1  ?1?1?1?1F(,%)5 5 }5 c]5   5 5 5 5 n""""" S V    AAXA  X   X >5>XeI|>>\> e07#777777rr)typingrrpandasr}ryray.air.resultrray.exceptionsrray.tune.analysisrray.tune.errorrray.tune.experimentr ray.utilr rrrrrs""""""""!!!!!!''''''000000$$$$$$%%%%%% VL7L7L7L7L7L7L7L7L7L7r