&`i`ddlZddlZddlmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZmZddlmZerddlmZejeZeGd d eZGd d ZdS) N) TYPE_CHECKINGDictListOptionalTuple) TuneError)Trial)DEFAULT_METRIC) FIFOSchedulerTrialScheduler) PublicAPI)TuneControllerc deZdZdZdZ d#ded eed eed ed ed e f fd Z d eed eede fdZ ddde fdZ dZde fdZddde defdZdddddefdZddde fdZddde fdZddde defdZddde fdZdddee fd Zdefd!Zdeeeffd"ZxZS)$HyperBandSchedulera Implements the HyperBand early stopping algorithm. HyperBandScheduler early stops trials using the HyperBand optimization algorithm. It divides trials into brackets of varying sizes, and periodically early stops low-performing trials within each bracket. To use this implementation of HyperBand with Tune, all you need to do is specify the max length of time a trial can run `max_t`, the time units `time_attr`, the name of the reported objective value `metric`, and if `metric` is to be maximized or minimized (`mode`). We automatically determine reasonable values for the other HyperBand parameters based on the given values. For example, to limit trials to 10 minutes and early stop based on the `episode_mean_reward` attr, construct: ``HyperBand('time_total_s', 'episode_reward_mean', max_t=600)`` Note that Tune's stopping criteria will be applied in conjunction with HyperBand's early stopping mechanisms. See also: https://blog.ml.cmu.edu/2018/12/12/massively-parallel-hyperparameter-optimization/ Args: time_attr: The training result attr to use for comparing time. Note that you can pass in something non-temporal such as `training_iteration` as a measure of progress, the only requirement is that the attribute should increase monotonically. metric: The training result objective value attribute. Stopping procedures will use this attribute. If None but a mode was passed, the `ray.tune.result.DEFAULT_METRIC` will be used per default. mode: One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute. max_t: max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed. The scheduler will terminate trials after this time has passed. Note that this is different from the semantics of `max_t` as mentioned in the original HyperBand paper. reduction_factor: Same as `eta`. Determines how sharp the difference is between bracket space-time allocation ratios. stop_last_trials: Whether to terminate the trials after reaching max_t. Defaults to True. Ftraining_iterationNQT time_attrmetricmodemax_treduction_factorstop_last_trialscBdks Jd|r|dvs Jdt|_tt jt jt j|z dz__fd_ fd_ gg_ i_ ddd_ d_|_|_d_jd krd _njd krd _|_|_dS) NrzMax (time_attr) not valid!)minmaxz`mode` must be 'min' or 'max'!crttjj|dzz j|zzS)Nr)intnpceil_s_max_1_eta)sselfs q/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/tune/schedulers/hyperband.pyz-HyperBandScheduler.__init__..ks0RWT]a!e-DtyRS|-S%T%T!U!Uc:tj| zzSN)rr#)r$rr%s r&r'z-HyperBandScheduler.__init__..msediQB.?&?!A!Ar()bracketband_idxr?r)super__init__r#rr roundlogr" _max_t_attr_get_n0_get_r0 _hyperbands _trial_info_state _num_stopped_metric_mode _metric_op _time_attr_stop_last_trials)r%rrrrrr __class__s` ` r&r0zHyperBandScheduler.__init__Ys<qyyy6yyy  L>)))+K))) $ BHRVE]]RV)r%r$s r&rPz"HyperBandScheduler._create_bracketsLo||A Q' !3    r(cb|j|jd}t||jkS)zfChecks if the current band is filled. The size of the current band should be equal to s_max_1r,)r6r8rMr")r%rSs r&rKz#HyperBandScheduler._cur_band_filleds+ #DK $;<8}} --r(resultc |j|\}}|||||r tjSt d|d|||}t |d|d|jd| |j|S)atIf bracket is finished, all trials will be stopped. If a given trial finishes and bracket iteration is not done, the trial will be paused and resources will be given up. This scheduler will not start trials but will stop trials. The current running trial will not be handled, as the trialrunner will be given control to handle it.Processing bracket after trial z resultz for z on =) r7update_trial_statscontinue_trialr CONTINUErNdebug_process_bracketr=get)r%rDrEr]r+_actions r&on_trial_resultz"HyperBandScheduler.on_trial_results%e, ""5&111  ! !% ( ( +!* * EuEEEFFF&&@@  ? ?E ? ? ? ?!'DO!UvdS)zFair scheduling within iteration by completion percentage. List of trials not used since all trials are tracked as state of scheduler. If iteration is occupied (ie, no trials to run), then look into next iteration. cg|]}||Sr*r).0bs r& z:HyperBandScheduler.choose_trial_to_run..rs>>>a  r(c*|Sr*)completion_percentage)rs r&r'z8HyperBandScheduler.choose_trial_to_run..ss!:Q:Q:S:Sr(keyN)r6sortedcurrent_trialsrrr rsr}PENDING)r%rD hyperbandscrubbedr+rEs r&choose_trial_to_runz&HyperBandScheduler.choose_trial_to_rungs) % %I?>9>>>H!(0S0STTT % %$3355%%E  44!W%>>>66$ 7 % %tr(c d}|d|jtd|jDz }t |jD]<\}}|d|z }|D]}|r|d|z }=|S)a}This provides a progress notification for the algorithm. For each bracket, the algorithm will output a string as follows: Bracket(Max Size (n)=5, Milestone (r)=33, completed=14.6%): {PENDING: 2, RUNNING: 3, TERMINATED: 2} "Max Size" indicates the max number of pending/running experiments set according to the Hyperband algorithm. "Milestone" indicates the iterations a trial will run for before the next halving will occur. "Completed" indicates an approximate progress metric. Some brackets, like ones that are unfilled, will not reach 100%. zUsing HyperBand: z num_stopped={} total_brackets={}c34K|]}t|VdSr*rMrbands r& z2HyperBandScheduler.debug_string..s("J"J3t99"J"J"J"J"J"Jr(z Round #{}:z {})rHr9sumr6 enumerate)r%outirr+s r& debug_stringzHyperBandScheduler.debug_string|s"" 188  s"J"J9I"J"J"JJJ   !!122 4 4GAt >((++ +C 4 448??7333C 4 r(cNtd|jD|jdS)Nc34K|]}t|VdSr*rrs r&rz+HyperBandScheduler.state..s(GGdD GGGGGGr() num_brackets num_stopped)rr6r9r%s r&statezHyperBandScheduler.states4GGd6FGGGGG,   r()rNNrrT)rI __module__ __qualname____doc___supports_buffered_resultsrurrfloatboolr0rCr rUrPrKrrirer|rrrrrr __classcell__)r?s@r&rr)s**X"'. $""#!%$2$2$2 $2sm $2  $2  $2$2$2$2$2$2$2Lsm+3C= 0)G,<)GU)G)G)G)GV    .$..../8=GK4]/]:D] ]]]]~.>u 5u55553CQV*c8 tCH~        r(rc.eZdZdZ d#dedededededed efd Zd e fd Z d efdZ d efdZ d e e fdZd e d efdZd efdZdeded ee e e e ffdZd e defdZd e fdZd$dZd efdZded efdZdededefd Zd efd!Zd"S)%r[zLogical object for tracking Hyperband bracket progress. Keeps track of proper parameters as designated by HyperBand. Also keeps track of progress to ensure good scheduling. TrrWrXrYrZr$rcFi|_g|_||_|x|_|_|x|_|_||_|j|_||_ ||_ | |j|j||_ d|_ ||_d|_t!|_dS)NrF) _live_trials _all_trialsr=_n_n0_r_r0r3_cumul_rr#_halves_calculate_total_work _total_work_completed_progressrrpsetr})r%rrWrXrYrZr$rs r&r0z_Bracket.__init__s#''$((($(%   55dh!LL#$ 0"'!$r(rEc|r Jdd|j|<|j|dS)zAdd trial to bracket assuming bracket is not filled. At a later iteration, a newly added trial will be given equal opportunity to catch up.z#Cannot add trial to filled bracket!N)rJrrrLr%rEs r&rQz_Bracket.add_trialsK ;;==GG"GGG #'%  &&&&&r(r@chtfdjDS)zhChecks if all iterations have completed. TODO(rliaw): also check that `t.iterations == self._r`c3TK|]"}|jkV#dSr*)_get_result_timer)rr]r%s r&rz)_Bracket.cur_iter_done..sL    ! !& ) )T] :      r()allrvaluesrs`r&rlz_Bracket.cur_iter_donesL    +2244      r(cR|jsdS|jdko|S)NFr)rrrlrs r&rmz_Bracket.finisheds0$ 5|q 9T%7%7%9%99r(c*t|jSr*)listrrs r&rz_Bracket.current_trialssD%&&&r(c|j|}|js |jdkrdS|||jkr(t d|d|jddSdS)NrTzContinuing trial z) as it hasn't reached the time threshold z, yet.F)rrrrrrNrd)r%rEr]s r&rbz_Bracket.continue_trials"5)$ ):):4  " "6 * *T] : : LL)E))=)))   4ur(c<t|j|jkS)zChecks if bracket is filled. Only let new trials be added at current level minimizing the need to backtrack and bookkeep previous medians.)rMrrrs r&rJz_Bracket.filleds 4$%%00r(r metric_opcjdkrjs jgfSjdksJxjdzc_tt jjjz _xjjzc_ttjj _j_ tjfd}|j d|dj }}||fS)Nrrc0j|zSr*)r)rrrr%s r&r'z-_Bracket.successive_halving.. sY9J19Mf9U-Ur(r) rrrrr r!rr#rrr3rr)r%rr sorted_trialsrrs``` r&rqz_Bracket.successive_halvings  <1  T%: $b( (|a  bgdg 12233 49c$'4#34455   #U#U#U#U#U#U   "47(**-}ZxZ/HcSyr(r]c||jvsJ||dksJ||}||j|}||z }|dkr.td|||xj|z c_||j|<|j|dS)zUpdate result for trial. Called after trial has finished an iteration - will decrement iteration count. TODO(rliaw): The other alternative is to keep the trials in and make sure they're not set as pending later.rz>DDDr(c$|dS||jS)Nr)r=)r%r]s r&rz_Bracket._get_result_timeBs >1do&&r(nrc0d}|}t|dzD]~}|t|t|zz }||jz}ttj|}||jz}tt ||j|z }|S)Nrr)rangerr#r r!rr3)r%rrr$work cumulative_rrgs r&rz_Bracket._calculate_total_workGs q1u = =A CFFSVVO #D NABGAJJA NAC4+l:;;<.Zs%I%I%I1ah%I%I%Ir(c3HK|]\}}d||VdS)z{}: {}N)rH)rkvs r&rz$_Bracket.__repr__..\s4DDTQ8??1a((DDDDDDr(zBracket({}): {{{}}} ) joinrHrrr collectionsCounterrritems)r%rrcountstrial_statusess r&__repr__z_Bracket.__repr__Rs!((11"))$-88"))$*D*D*F*FGG   $%I%I8H%I%I%IJJ DDV\\^^DDD D D  &,,V^DDDr(N)T)rDr)rIrrrrurrrr0r rQrlrmrrrbrJrrqrrarxrnrrrrrr(r&r[r[sJ"&'''' '  '  ' ''''':'u'''' t    :$:::: 'U '''' E d    11111&+ tE{DK' (@..t....,+5++++2222EuEEEE't''''' s u      E# E E E E E Er(r[)rloggingtypingrrrrrnumpyr ray.tune.errorrray.tune.experimentr ray.tune.resultr #ray.tune.schedulers.trial_schedulerr r ray.util.annotationsr "ray.tune.execution.tune_controllerr getLoggerrIrNrr[rr(r&rsb==============$$$$$$%%%%%%******MMMMMMMM******BAAAAAA  8 $ $2 r r r r r r r  r j EEEEEEEEEEr(