.`i0&UddlZddlmZddlmZddlmZddlmZddl m Z ee Z iZ eeededzfed<iZeeededzfed <Gd dejZGd dejZdS) N)get_cached_compilation_config) init_logger)maybe_disable_graph_partition)current_platformCustomOpPluggableLayer op_registryop_registry_ootcbeZdZdZfdZedefdZeddedzfdZxZ S)ra_ Base class for pluggable layers. A PluggableLayer is a *module-composing* abstraction: it may instantiate other ``torch.nn.Module`` objects as sub-layers, and its functionality depends on these sub-layers following a generalized invocation sequence. Also, it is stateful and may hold parameters or buffers. Unlike :class:`CustomOp`, PluggableLayer does NOT provide per-platform ``forward_*`` dispatch. Instead, it supports out-of-tree (OOT) replacement of the entire layer class at instantiation time, allowing customized initialization and submodule composition. c* |j}n'#t$rtd|jddwxYw|tvr|}n6t|}td|t |t|S)NCannot instantiate 'z': its 'name' attribute was not set, possibly because it was not decorated with @PluggableLayer.register, or it's the PluggableLayer itself.z*Instantiating pluggable layer: %s using %s __name__AttributeError TypeErrorr loggerdebugstrsuper__new__)clsargskwargslayer_class_namelayer_cls_to_instantiate __class__s q/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/model_executor/custom_op.pyrzPluggableLayer.__new__%s "|     Ps|PPP    ? 2 2'* $ $'67G'H $ LL< ,--    ww7888 $/namecfd}|S)NcVtvs Jd|_|t<|SNzDuplicate op name: r rop_clsrs r decoratorz*PluggableLayer.register..decorator=;{***,H$,H,H***FK &K Mrrr&s ` rregisterzPluggableLayer.register;$      r(Ncxfd}||St|tr ||Std)Nclnj}|tvs Jd|||_|t|<|S)NzDuplicate layer name: rr r) layer_clsreg_namerrs rr&z.PluggableLayer.register_oot..decoratorKsL#/ttS\H?2224WX4W4W222%IN(1OH % r()Decorator can only be applied to classes. isinstancetyper)r_decorated_layer_clsrr&s` ` r register_ootzPluggableLayer.register_ootIsf       '  ,d 3 3 I9122 2GHH Hr(NN) r __module__ __qualname____doc__r classmethodrr+r7 __classcell__rs@rrrs  99999,C[II3:III[IIIIIr(ceZdZdZfdZddddedeffdZdZd Zd Z d Z d Z d Z dZ dZdefdZdddefdZedefdZedefdZedefdZeddedzfdZxZS)rzb Base class for custom ops. Dispatches the forward method to the appropriate backend. c* |j}n'#t$rtd|jddwxYw|tvr|}n6t|}td|t |t|S)Nr z': its 'name' attribute was not set, possibly because it was not decorated with @CustomOp.register, or it's the CustomOp base class itself.z$Instantiating custom op: %s using %sr)rrrop_nameop_cls_to_instantiaters rrzCustomOp.__new__cs lGG   Os|OOO    / ) )$' ! !$3G$< ! LL6)**    ww4555rF)enforce_enablecompile_nativerCrDct||_|||_dS)N)rD)r__init___enforce_enabledispatch_forward_forward_method)selfrCrDrs rrFzCustomOp.__init__xs@ -#44N4SSr(c|j|i|SN)rIrJrrs rforwardzCustomOp.forward}s#t#T4V444r(ct)zPyTorch-native implementation of the forward method. This method is optional. If implemented, it can be used with compilers such as torch.compile or PyTorch XLA. Also, it can be used for testing purposes. NotImplementedErrorrMs rforward_nativezCustomOp.forward_natives "!r(ctrLrPrMs r forward_cudazCustomOp.forward_cudas!!r(c|j|i|SrL)rTrMs r forward_hipzCustomOp.forward_hips t $1&111r(c|j|i|SrLrRrMs r forward_xpuzCustomOp.forward_xpu#t"D3F333r(c|j|i|SrLrXrMs r forward_cpuzCustomOp.forward_cpurZr(c|j|i|SrLrXrMs r forward_tpuzCustomOp.forward_tpus#t"D3F333r(c|j|i|SrLrXrMs r forward_ootzCustomOp.forward_ootrZr(c<t}|jp|}|r&|j|jjgn%|j|jjg|s||j |Stj r|j Stj r|jStjr|jStjr|jStjr|jS|jS)Nenable)rrGenabledenabled_custom_opsupdaterrdisabled_custom_ops maybe_compilerRris_rocmrVis_cpur\is_tpur^is_xpurYis_out_of_treer`rT)rJrDcompilation_configrds rrHzCustomOp.dispatch_forwards#;<<&8$,,..  Q  1 8 8$.:M9N O O O O  2 9 94>;N:O P P P R%%d&9.%QQ Q  # % % %# #  $ & & %# #  $ & & %# #  $ & & %# #  , . . %# #$ $r(Trbrccddlm}|s|St}|j|jkr|S|jdkr|St j|dtj ttj S)aP Compile fn if compilation enabled. Useful for CustomOp instances called from within a torch custom op, meaning the forward call is hidden from the model-level torch.compile. NOTE: this does not enable fusion across ops, so opaque custom ops should still be unwrapped wherever possible. r)CompilationModeeagerT)dynamicbackendoptions) vllm.config.compilationrprmodeNONErstorchcompilersimple_compile_backendr)rJfnrcrprns rrhzCustomOp.maybe_compiles <;;;;; I;<<  "o&: : :I  % 0 0I} $;1 7     r(returncVt}|j}t|ds9td|jt Sd|j|v}d|j|v}|r|rJd|jt s|o| S)NrzCustom op %s was not registered, which means it won't appear in the op registry. It will be enabled/disabled based on the global settings.+-zCannot enable and disable ) r custom_opshasattrr warning_oncerr default_onr)rrnrrddisableds rrdzCustomOp.enableds;<<'2 sF## )   #     &&(( ( ch..J.!sx>>Z/RRR+R+R+RRR)##%%0B(lBr(ct}|jd}|jd}||zdksJ|dk p|dkS)z Behavior controlled by `CompilationConfig.custom_ops`: On by default if 'all', off by default if 'none'. When PyTorch Inductor is used, 'none' is the default value, otherwise 'all'. noneallr)rrcount)rn count_none count_alls rrzCustomOp.default_onsh;<<'288@@ &177>> I%****>!2Y]2r(rcfd}|S)NcVtvs Jd|_|t<|Sr"r#r$s rr&z$CustomOp.register..decoratorr'r(r)r*s ` rr+zCustomOp.register r,r(Ncxfd}||St|tr ||Std)Nclnj}|tvs Jd|||_|t|<|Sr"r/)r%r1rrs rr&z(CustomOp.register_oot..decorator"sK#/ttS\H?2224T(4T4T222"FK(.OH %Mr(r2r3)r_decorated_op_clsrr&s` ` rr7zCustomOp.register_oot sj        $   )4 0 0 I9.// /GHH Hr(r8)rr9r:r;rboolrFrNrRrTrVrYr\r^r`rHrhr<rd staticmethodrrr+r7r=r>s@rrr]s 66666*27uTTT$TTTTTTT 555""""""222444 444 444 444 !%t!%!%!%!%F37    4     DCCCC[C& 3 3 3 3\ 3C[$IId III[IIIIIr()rxtorch.nnnn vllm.configr vllm.loggerrvllm.model_executor.utilsrvllm.platformsrrrr dictrr5__annotations__r Modulerrr)r(rrsZ 555555######CCCCCC++++++ X  EG T#tJ'$/?*@@@ AFFFHJc4 +d3C.DDDEJJJDIDIDIDIDIRYDIDIDINYIYIYIYIYIryYIYIYIYIYIr(