&`iS1ddlZddlZddlmZddlmZmZmZddlm Z ddl m Z er ddl m Z ddlmZejeZGdd ZdS) N)deepcopy) TYPE_CHECKINGDictOptional)version) flatten_dict)Run MlflowClientceZdZdZdZd$dZ d%deedeedeed eed eed eed efd Z de de fdZ d&deedee deddfdZ d'deedee ddfdZ dedefdZd(dZd$de deefdZd$de deefdZd$d edeefd!Zd'd"eefd#ZdS))_MLflowLoggerUtilzUtil class for setting up and logging to MLflow. Use this util for any library that needs MLflow logging/tracking logic such as Ray Tune or Ray Train. c*ddl}||_d|_dS)Nr)mlflow_mlflow experiment_id)selfrs l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/air/_internal/mlflow.py__init__z_MLflowLoggerUtil.__init__s   !Nc|j}|jdt|j|}t }|j|||_||_|S)Nr)r__dict__poprr update)rmemor dict_copy copied_objects r __deepcopy__z_MLflowLoggerUtil.__deepcopy__sg, )$$$T]D11 )++ %%i000 ' rT tracking_uri registry_urirexperiment_nametracking_tokenartifact_locationcreate_experiment_if_not_existscd|r|tjd<|j||j|||ntjd}|Tddlm} |j|t d|d||_ dS#|$rYnwxYw||ntjd }|_|j | rDt d |d|j |j |_ dS|rB|r@t d ||j || |_ dS|rtd|dtd|d|d)a Sets up MLflow. Sets the Mlflow tracking uri & token, and registry URI. Also sets the MLflow experiment that the logger should use, and possibly creates new experiment if it does not exist. Args: tracking_uri: The tracking URI for the MLflow tracking server. registry_uri: The registry URI for the MLflow model registry. experiment_id: The id of an already existing MLflow experiment to use for logging. If None is passed in here and the MFLOW_EXPERIMENT_ID is not set, or the experiment with this id does not exist, ``experiment_name`` will be used instead. This argument takes precedence over ``experiment_name`` if both are passed in. experiment_name: The experiment name to use for logging. If None is passed in here, the MLFLOW_EXPERIMENT_NAME environment variable is used to determine the experiment name. If the experiment with the name already exists with MLflow, it will be reused. If not, a new experiment will be created with the provided name if ``create_experiment_if_not_exists`` is set to True. artifact_location: The location to store run artifacts. If not provided, MLFlow picks an appropriate default. Ignored if experiment already exists. tracking_token: Tracking token used to authenticate with MLflow. create_experiment_if_not_exists: Whether to create an experiment with the provided name if it does not already exist. Defaults to True. Returns: Whether setup is successful. MLFLOW_TRACKING_TOKENNMLFLOW_EXPERIMENT_IDrMlflowException)rzExperiment with provided id z( exists. Setting that as the experiment.MLFLOW_EXPERIMENT_NAME)namezExperiment with provided name zBExisting experiment not found. Creating new experiment with name: )r*r"z,Experiment with the provided experiment_id: zZ does not exist and no experiment_name provided. At least one of these has to be provided.z or experiment_name: zV does not exist. Please create an MLflow experiment and provide either its id or name.)osenvironrset_tracking_uriset_registry_urigetmlflow.exceptionsr(get_experimentloggerdebugrget_experiment_by_namecreate_experiment ValueError) rrrrr r!r"r#r(s r setup_mlflowz_MLflowLoggerUtil.setup_mlflow)szZ  A2@BJ. / %%l333 %%l333 ( M 677   $ 9 9 9 9 9 9  ++-+HHH >=>>>&3""     * O 899   &4<+N+N ,O, , & LL::::   "&!D!D""   F  >  LL;)8;;   "&!?!?$8I"@""D  F * - --- * **"*** s2AB44B<;B< dict_to_logreturnci}|D]_\}} t|}|||<#ttf$r1td||Y\wxYw|S)aParses provided dict to convert all values to float. MLflow can only log metrics that are floats. This does not apply to logging parameters or artifacts. Args: dict_to_log: The dictionary containing the metrics to log. Returns: A dictionary containing the metrics to log with all values being converted to floats, or skipped if not able to be converted. zMCannot log key {} with value {} since the value cannot be converted to float.)itemsfloatr6 TypeErrorr2r3format)rr8new_dictkeyvalues r _parse_dictz_MLflowLoggerUtil._parse_dicts%++--  JC e  %  *    ::@&e:L:L  s2?A43A4Frun_nametags set_activer cXddl}ddlm}|i}|r|||S|}t j|jt jdkr|||j |}n!|||<||j |}|S)aMStarts a new run and possibly sets it as the active run. Args: tags: Tags to set for the new run. set_active: Whether to set the new run as the active run. If an active run already exists, then that run is returned. Returns: The newly created MLflow run. rN)MLFLOW_RUN_NAME)rCrDz1.30.0rCrrD)rrD) rmlflow.utils.mlflow_tagsrG_start_active_run _get_clientrparse __version__ create_runr)rrCrDrErrGclientruns r start_runz_MLflowLoggerUtil.start_runs  <<<<<< <D  H))8$)GG G!!## =+ , , h0G0G G G##!1C$$CC%-D !##$2D4#PPC rc|j}|r|S|j||j|S)zStarts a run and sets it as the active run if one does not exist. If an active run already exists, then returns it. rH)r active_runrQr)rrCrDrSs rrJz#_MLflowLoggerUtil._start_active_runsN\,,..   |%%T-?d&   rrun_idcbddlm} |j|dS#|$rYdSwxYw)z)Check if run with the provided id exists.rr')rTTF)r0r(rget_run)rrTr(s r _run_existsz_MLflowLoggerUtil._run_existssY555555  L  / / /4   55 s %..r c|j}|j}ddlm}|||S)z@Returns an ml.tracking.MlflowClient instance to use for logging.rr )rr)rget_tracking_uriget_registry_urimlflow.trackingr )rrrr s rrKz_MLflowLoggerUtil._get_clientsO|4466 |4466 000000|LQQQQr params_to_logcNt|}|r]||rH|}|D]\}}||||dS|D]!\}}|j||"dS)aLogs the provided parameters to the run specified by run_id. If no ``run_id`` is passed in, then logs to the current active run. If there is not active run, then creates a new run and sets it as the active run. Args: params_to_log: Dictionary of parameters to log. run_id (Optional[str]): The ID of the run to log to. )rTr@rA)r@rAN)rrWrKr; log_paramr)rr\rTrOr@rAs r log_paramsz_MLflowLoggerUtil.log_paramss%]33  =d&&v.. =%%''F+1133 F F U  Cu EEEE F F,1133 = = U &&3e&<<<< = =rmetrics_to_logc|t|}||}|r^||rI|}|D]\}}|||||dS|D]"\}}|j|||#dS)a|Logs the provided metrics to the run specified by run_id. If no ``run_id`` is passed in, then logs to the current active run. If there is not active run, then creates a new run and sets it as the active run. Args: metrics_to_log: Dictionary of metrics to log. run_id (Optional[str]): The ID of the run to log to. )rTr@rAstep)r@rArbN)rrBrWrKr; log_metricr)rrbr`rTrOr@rAs r log_metricsz_MLflowLoggerUtil.log_metricss&n55)).99  Id&&v.. I%%''F,2244 R R U!!SD!QQQQ R R-2244 I I U ''Cu4'HHHH I Irdirc|rB||r-|}|||dS|j|dS)aSaves directory as artifact to the run specified by run_id. If no ``run_id`` is passed in, then saves to the current active run. If there is not active run, then creates a new run and sets it as the active run. Args: dir: Path to directory containing the files to save. run_id (Optional[str]): The ID of the run to log to. )rT local_dir)rgN)rWrK log_artifactsr)rrerTrOs rsave_artifactsz _MLflowLoggerUtil.save_artifacts/sp  6d&&v.. 6%%''F  # > > > > > L & & & 5 5 5 5 5rstatuscD|r||rm|jr'|jjj|ks-|}|||dS|j|dS)a2Terminates the run specified by run_id. If no ``run_id`` is passed in, then terminates the active run if one exists. Args: status (Optional[str]): The status to set when terminating the run. run_id (Optional[str]): The ID of the run to terminate. )rTrj)rjN)rWrrSinforTrKset_terminatedend_run)rrjrTrOs rrnz_MLflowLoggerUtil.end_run@s  0  (( 0 '')) 0 L++--29VCC%%''F  ! ! ! ? ? ? ? ? L  / / / / /r)N)NNNNNNT)NNF)NN)r9r )__name__ __module__ __qualname____doc__rrrstrboolr7rrBrQrJrWrKr_rdrirnrrr r su """    '+&*'+)-(,+/04sssmssms } s "# s ! s$C=s*.ssssjt:#'# $$3-$tn$ $  $$$$NFJ       4r}s 0000000000******-######,,,,,,  8 $ $E0E0E0E0E0E0E0E0E0E0r