&`iddlZddlZddlZddlZddlmZddlmZddlm Z dZ GddeZ Gdd ej Z ed kr3ejZed d d e\ZZedddddddddddddde ddZeje ejddejrd nd!iegejdd"#ejd$d%d&'ejd(d)ejd(d)d*+Zed,d-deZ e !Z"ed.d/dd0e"j#d1e"j$dd2ddSdS)3N)tune)Callback)SHOULD_CHECKPOINTcRtjdd||zdz zdz|dzzS)N皙?d)timesleepstepwidthheights /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/tune/examples/custom_checkpointing_with_callback.py evaluation_fnrs1JsOOO %$,$ $" - <<c0eZdZdZddddedefdZdZdS) SmartCheckpointCallbackaCustom callback that triggers checkpointing by updating the result dict. This callback demonstrates checkpointing logic beyond simple periodic checkpointing. It checkpoints based on performance improvements or when the loss becomes unstable. Args: checkpoint_on_improvement: Checkpoint when loss improves significantly checkpoint_on_instability: Checkpoint when loss becomes unstable Tcheckpoint_on_improvementcheckpoint_on_instabilityrrc>||_||_i|_i|_dS)N)rrbest_loss_per_trialrecent_losses_per_trial)selfrrs r__init__z SmartCheckpointCallback.__init__#s+ *C&)B&#% ')$$$rc  |j}|dtd}|dd}||jvr!td|j|<g|j|<d} d} |dkr|dzdkrd} d |} |jr5||j|d zkr!d} d |d d |j|d } ||j|<|jr|dkr|j| |t dkr dt dkr,t fd Ddz } | dkrd} d| d } nJ|j| |t dkr d| r$td|d|d| d|t<dSdS)a@Called after receiving a result from the trainable. This hook implements intelligent checkpointing logic: 1. Checkpoint when we see significant improvement 2. Checkpoint when loss becomes unstable (variance increases) 3. Always checkpoint at specific milestones (every 10 steps) mean_lossinf iterationsrF Tzmilestone at step g?zsignificant improvement: z.4fz < c3HK|]}|tdz z dzVdS)r#N)sum).0x recent_lossess r z:SmartCheckpointCallback.on_trial_result..Ys:QQaS//!339QQQQQQrrzinstability detected: variance=z)Callback requesting checkpoint for trial z at step z: N) trial_idgetfloatrrrrappendlenpopr&printr) r iterationtrialstrialresultinfor+ current_loss current_stepshould_checkpointreasonvariancer)s @ron_trial_resultz'SmartCheckpointCallback.on_trial_result.s[>zz+uU||<< zz,22  43 3 316uD $X .57D ( 2! !   r 1Q 6 6 $ 8,88F  ) Bt7ACGGG$(!r\rrr$JbckJlrrr5A(2  ) %lQ.>.> 8BM   . . .=!!A%%!!!$$$=!!Q&&QQQQ=QQQQQTUUc>>(,%MxMMMF!8BM   . . .=!!A%%!!!$$$  - gHgg|gg_egg   )-F$ % % %  - -rN)__name__ __module__ __qualname____doc__boolrr<rrrrsg  +/*. * * *$( *$( * * * *;-;-;-;-;-rrc*eZdZdZdZdZdZdZdS)OptimizationTrainablezKA simple trainable that demonstrates automatic checkpointing with callbackscHd|_|d|_|d|_dS)zInitialize the trainablerrrN)r8rr)rconfigs rsetupzOptimizationTrainable.setupos%G_ X& rct|j|j|j}|xjdz c_|j||jdS)zPerform one step of training)r rr )rr8rr)rintermediate_scores rr zOptimizationTrainable.stepusO*4+>>>>s)A&&A*-A*c:tj|d}t|d5}t j|}dddn #1swxYwY|d|_|d|_|d|_td|jdS)zALoad checkpoint - called automatically by Tune during restorationrLrNr rrzCheckpoint loaded from step ) rNrOrPrQrRloadr8rrr1)r checkpointrUrVstates rload_checkpointz%OptimizationTrainable.load_checkpoints',,z3DEE /3 ' ' !1IaLLE ! ! ! ! ! ! ! ! ! ! ! ! ! ! !!&M7^ Ho  @T->@@AAAAAsAAAN)r=r>r?r@rGr rWr]rBrrrDrDls[UU'''     ? ? ?BBBBBrrD__main__z --smoke-test store_truezFinish quickly for testing)actionhelpz<============================================================zGRay Tune Example: Smart Checkpointing with custom SHOULD_CHECKPOINT keyr!zLThis example demonstrates how to set the SHOULD_CHECKPOINT key in a callbackzBto implement intelligent checkpointing based on training progress.z Key features:zK- Callback-driven checkpointing by setting result[SHOULD_CHECKPOINT] = Truez3- Checkpoints triggered by performance improvementsz.- Milestone-based checkpointing every 10 stepsz8- Instability detection (high variance in recent losses)z4- Automatic checkpoint save/load via class trainable )sepTrsmart_checkpoint_testtraining_iterationrI)checkpoint_frequencycheckpoint_at_end)namestop callbackscheckpoint_configrmin)metricmode num_samplesr"r)rr) run_config tune_config param_spacez:Starting hyperparameter tuning with smart checkpointing...z8Watch for checkpoint messages triggered by the callback!z= ============================================================RESULTSzBest hyperparameters: zBest checkpoint: z=The checkpoints were triggered by the SmartCheckpointCallback)%argparserRrNr rayrray.tunerray.tune.resultrrr TrainablerDr=ArgumentParserparser add_argumentparse_known_argsargs_r1checkpoint_callbackTuner RunConfig smoke_testCheckpointConfig TuneConfigrandint loguniformtunerfitresultsget_best_result best_resultrFr[rBrrrs  ------ === R-R-R-R-R-hR-R-R-j)B)B)B)B)BDN)B)B)BX z $X $ & &F |2N%%''GD! EQ VL U=8B> $21"&$ DJ!4>(&T_(D"E*+3d3%&"&    $DO   "T\"c**%dob#..  #   E. EDB  iikkG))++K E5!3554K244 G       yr