&`i|2dZddlZddlZddlZddlmZddlmZmZm Z m Z ddl m Z ddl mZeGddeZeGd d ejZeGd d eZeGd deZGddZdS)z Serialization handlers for preprocessor save/load functionality. This module implements a factory pattern to abstract different serialization formats, making it easier to add new formats and maintain existing ones. N)Enum)AnyDictOptionalUnion) cloudpickle) DeveloperAPIceZdZdZdZdZdS)HandlerFormatNamez1Enum for consistent format naming in the factory.rpickleN)__name__ __module__ __qualname____doc__ CLOUDPICKLEPICKLE/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/data/preprocessors/serialization_handlers.pyr r s;;K FFFrr c eZdZdZejdedeee ffdeee ffdZ ejdeee fde fdZ ejdeee ffdZ deee fdeee ffd Zd S) SerializationHandlerzDAbstract base class for handling preprocessor serialization formats.data PreprocessorreturncdS)zSerialize preprocessor data to the specific format. Args: data: Dictionary containing preprocessor metadata and stats Returns: Serialized data in format-specific representation Nrselfrs r serializezSerializationHandler.serializes  r serializedcdS)aUDeserialize data from the specific format. Args: serialized: Serialized data in format-specific representation Returns: For structured formats (CloudPickle/JSON/MessagePack): Dictionary containing preprocessor metadata and stats For pickle format: The actual deserialized object Nrrrs r deserializez SerializationHandler.deserialize,s  rcdS)z+Get the magic bytes/prefix for this format.Nrrs rget_magic_bytesz$SerializationHandler.get_magic_bytes9s  rc|}t|ttfr,||r|t |dS|S)z(Remove magic bytes from serialized data.N)r% isinstancestrbytes startswithlen)rrmagics rstrip_magic_bytesz&SerializationHandler.strip_magic_bytes>sY$$&& j3, / / ,J4I4I%4P4P ,c%jjll+ +rN)r rrrabcabstractmethodrrr(rr)rr"r%r-rrrrrs NN  .$sCx.89  sEz          eCJ&7  C        sEz!2    E#u*,=%U BSrrcreZdZdZdZdedeeeffde fdZ de deeeffdZ de fd Z d S) CloudPickleSerializationHandlerz-Handler for CloudPickle serialization format.sCPKL:rrrc:|jtj|zS)z2Serialize to CloudPickle format with magic prefix.)MAGIC_CLOUDPICKLErdumpsrs rrz)CloudPickleSerializationHandler.serializeLs% (9$(?(???rrc$t|tstdt|||jstd|dd||}tj|S)z$Deserialize from CloudPickle format.z4Expected bytes for CloudPickle deserialization, got z!Invalid CloudPickle magic bytes: N ) r'r) ValueErrortyper*r3r-rloads)rrcloudpickle_datas rr"z+CloudPickleSerializationHandler.deserializeRs*e,, YtJGWGWYY $$T%;<< TRCRCRRSS S11*== !1222rc|jS)N)r3r$s rr%z/CloudPickleSerializationHandler.get_magic_bytes_s %%rN) r rrrr3rrr(rr)rr"r%rrrr1r1Fs77 @.$sCx.89@ @@@@ 3e 3S#X 3 3 3 3&&&&&&&rr1c^eZdZdZdedeeeffdefdZdedefdZ defdZ d S) PickleSerializationHandlerz/Handler for legacy Pickle serialization format.rrrcttjtj|dS)z Serialize using pickle format (for backward compatibility). data is ignored, but kept for consistency ascii)base64 b64encoder r4decoders rrz$PickleSerializationHandler.serializegs- T 2 233::7CCCrrcNtjtj|S)z0Deserialize from pickle format (legacy support).)r r9r@ b64decoder!s rr"z&PickleSerializationHandler.deserializeqs |F,Z88999rcdS)Nrr$s rr%z*PickleSerializationHandler.get_magic_bytesxsrrN) r rrrrrr(rrr"r%rrrr=r=cs99D.$sCx.89D DDDD:: ::::rr=c eZdZdZejeejeiZ e dede fdZ e d de ede eeefdefd Ze d eeefdefd ZdS) SerializationHandlerFactoryz>Factory class for creating appropriate serialization handlers. format_name handler_classc||j|<dS)z%Register a new serialization handler.N) _handlers)clsrIrJs rregister_handlerz,SerializationHandlerFactory.register_handlers&3 k"""rNformat_identifierrrc |s||}||jvr>td|jdt |j|j|}|S)aGet the appropriate serialization handler for a format or serialized data. Args: format_identifier: The format to use for serialization. If None, will detect from data. data: Serialized data to detect format from (used when format_identifier is None). **kwargs: Additional keyword arguments (currently unused). Returns: SerializationHandler instance for the format Raises: ValueError: If format is not supported or cannot be detected z"Unsupported serialization format: z. Supported formats: ) detect_formatrLr7valuelistkeys)rMrOrkwargsrJs r get_handlerz'SerializationHandlerFactory.get_handlers*! 8 # 1 1$ 7 7  CM 1 1C5F5LCC&*3=+=+=+?+?&@&@CC   &78 }rrct|tr+|tjr t jSt|tr t jStd|ddd)zDetect the serialization format from the magic bytes. Args: serialized: Serialized data Returns: Format name enum Raises: ValueError: If format cannot be detected z)Cannot detect serialization format from: Nz...) r'r)r*r1r3r rr(rr7)rMrs rrQz)SerializationHandlerFactory.detect_formats j% ( ( 1Z-B-B + =. .  1%0 0 j# & & ,$+ + L 3B3 L L L   r)NN)r rrrr rr1rr=rL classmethodr8rNrrr(r)rrVrQrrrrHrH|sHH %'F "<I 3+<3T333[3:>,0#$56uS%Z()  [@ uS%Z'8 =N   [   rrH)rr.r@r enumrtypingrrrrray.cloudpicklerray.util.annotationsr r ABCrr1r=rHrrrr_s ------------''''''------(((((37(((V&&&&&&:&&&8!50G G G G G G G G G G r