.`i;ddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z e eZGdd e ZdS) )SequenceN)PreTrainedTokenizerBase)ChatCompletionRequest) DeltaMessage) init_logger)ReasoningParsercFeZdZdZdeffd ZdededeedzedzffdZ d ed ed ed e e d e e de e de dzfdZ dedefdZdedefdZd ed ede fdZd eded ede fdZd ededed ede de f dZd edeedze dzedzffdZxZS)GraniteReasoningParserz Reasoning parser for IBM Granite. IBM granite models currently use "Here is my thought process:" and "Here is my response:" to separate its thinking / response outputs. tokenizercLtj|g|Ri|d|_d|_t j|jd|jdtj|_ddg|_ddg|_ d |_ d |_ td |jD|_ dS) Nz&(?:Here's|Here is) my thought process:z(?:Here's|Here is) my response:z(.*?)z(.*)zHere's my thought process:zHere is my thought process:zHere's my response:zHere is my response::Herec34K|]}t|VdSN)len).0 think_starts {/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/reasoning/granite_reasoning_parser.py z2GraniteReasoningParser.__init__..3s;' ' !,C  ' ' ' ' ' ' )super__init__think_start_exprresponse_start_exprrecompileDOTALLreasoning_regexvalid_think_startsvalid_response_startsseq_boundary_endseq_boundary_startmaxlongest_think_start)selfr argskwargs __class__s rrzGraniteReasoningParser.__init__s4T444V444 !J#E !z% J JD,D J J JBI   ) )# '<=S%T"!$"($'' ' 040G' ' ' $ $    r model_outputrequestreturnNcl|j|}|sd|fS|d\}}|s|dfS||fS)aExtract the reasoning content & content sections, respectively. If the sequence doesn't match what we expect, i.e., the model generates something else, all content is considered non-reasoning content. Args: model_output (str): Output of the model to be parsed. request (ChatCompletionRequest): Request being processed. Returns: tuple[Optional[str], Optional[str]]: Tuple pair containing the reasoning content and non-reasoning content. Nr)rfindall)r%r)r*re_match reasoningresponse_contents rextract_reasoningz(GraniteReasoningParser.extract_reasoning7sZ'// == &% %&.qk# # #d? "***r previous_text current_text delta_textprevious_token_idscurrent_token_idsdelta_token_idsc||\}}} |s|||} n7| s||||} n|J|||| ||} | js | jsdS| S)aExtract the reasoning content / content emitted by granite models; If the sequence doesn't match what we expect, i.e., the model generates something else, all content is considered non-reasoning content. NOTE: Granite models do not use a special token to start their reasoning and response sections; instead they have token sequences, e.g., Here is my thought process: Foo Here is my response: Bar This increases the complexity of correctly handling streams, since we need to watch for specific sequences and correctly parse them without dropping content that is potentially overlapping & spanning multiple delta messages. Args: previous_text (str): Previous text outside of this delta message. current_text (str): Previous text + delta text. delta_text (str): Text to consider and parse content from. previous_token_ids (Sequence[int]): Token IDs of previous_text. current_token_ids (Sequence[int]): Token IDs of current_text. delta_token_ids (Sequence[int]): Token IDs of delta_text. Returns: Union[DeltaMessage, None] DeltaMessage with either reasoning content or content, or None. N)_get_content_sections+_get_delta_message_with_no_reasoning_bounds*_get_delta_message_with_no_response_bounds#_get_delta_message_with_both_boundscontentr/) r%r2r3r4r5r6r7r/ resp_seq_lenr= delta_messages rextract_reasoning_streamingz2GraniteReasoningParser.extract_reasoning_streamingNsF,0+E+El+S+S( <  LLjMM   KKiMM  +++ DDIw lM$ ]-D 4rtextcDtfd|jDS)zCheck if a text matches one of the possible start reasoning seqs. Args: text (str): Text to check for leading substr. Returns: bool: True if any of the possible reasoning start seqs match. c3BK|]}|VdSr startswith)rrrAs rrzDGraniteReasoningParser._is_reasoning_start_substr..sB  -8K " "4 ( (      r)anyrr%rAs `r_is_reasoning_start_substrz1GraniteReasoningParser._is_reasoning_start_substrs@    <@.sE    % %d + +      r)rFr rGs `r_is_response_start_substrz0GraniteReasoningParser._is_response_start_substrs@    "&"<      rct|t|z }||}||d|}|r|std|S|rtddStd|S)a^Parse the delta message when the current text has not yet completed its start of reasoning sequence. Args: current_text (str): The full previous + delta text. delta_text (str): Text to consider and parse content from. Returns: DeltaMessage: Message containing the parsed content. Nr/r=)rrHr)r%r3r4prev_longest_length is_substr was_substrs rr:zBGraniteReasoningParser._get_delta_message_with_no_reasoning_boundss",//#j//A33LAA 44\BVCVBV5WXX  i $   >$=== =dJ????rr/ctfd|jD}||rtddS|dt| }|||j}||j}|dkr|||dnd}|dkr||dnd} |dkr|||dnd} | rtddS|s,| rt|d|dSt|dS| r&||d|d|z}t|dSt||d|zdS)a5Parse the delta message when the current text has both reasoning content with no (response) content. NOTE that we may have overlapping tokens with the start of reasoning / start of response sequences on either side of the delta text. Args: current_text (str): The full previous + delta text. reasoning (str): reasoning content from current_text. delta_text (str): Text to consider and parse content from. Returns: DeltaMessage: Message containing the parsed content. c3BK|]}|VdSr)endswith)rrKr3s rrzTGraniteReasoningParser._get_delta_message_with_no_response_bounds..sE+ +   ! !. 1 1+ + + + + + rNrNrF)rFr rrrfindr"rL) r%r3r/r4ends_with_start_response_seqr2prev_idx delta_idxprev_was_substrdelta_continues_substrdelta_new_substrs ` rr;zAGraniteReasoningParser._get_delta_message_with_no_response_boundss.(++ + + + "&"<+ + + ( ( $   < $=== =""4S__$4"45  !&&t'>??$$T%<== 1}}  * *=+C D D D 1}}  * *< +B C C C A~~  * *:ijj+A B B B  " >$=== = C T#j).DdSSSS*dCCC C C%hii0:jyj3III)TBBB B#HII.;    rr0response_seq_lenc`|t| d}t|t||zz }|dkrd}n[t||zt|zdz } t|t|z } | | z } | dkrd} || |}t||S)aParse the delta message when the current text has both reasoning content and normal (response) content. Args: delta_text: Text to consider and parse content from. reasoning: reasoning content from current_text. response_content: response content from current_text. current_text: The full previous + delta text. response_seq_len: Len of the complete response sequence used. Returns: DeltaMessage: Message containing the parsed content. NrrN)rr) r%r4r/r0r3r\ delta_contentreasoning_end_idxdelta_reasoningstart_reasoning_idx delta_offset start_offsets rr<z:GraniteReasoningParser._get_delta_message_with_both_boundss,#C(8$9$9#9#;#;<  OOs3C/D/DGW/WX q "OOI!11C8H4I4IIAM |,,s:>L.=La (6G)GHO%!    rcd}d}d}fdt|D}|D]}|||}|%jD]}||ddkr |dz}|dz}n3|spjD]h} |t|  dzd| ddkr?|t| z } ||| } ||dzd} | t| | fccSi|r|s ||dddfSdS)aParse the text to extract the reasoning content / content if we have them. Args: current_text (str): The full previous + delta text. Returns: tuple[Optional[str], Optional[int], Optional[str]]: Tuple of len 3 containing the reasoning content, the length of the response seq (if there is one) and the non-reasoning content. rNFc0g|]\}}|jk|S)r!)ridxcharr%s r z@GraniteReasoningParser._get_content_sections..Qs5   Tt,,, ,,,rr^)NNN) enumeraterr r) r%r3current_chunk_startstart_reasoningparsed_contentdelimiter_idxscurrent_chunk_end current_chunkrrK end_reasoningr/r0s ` rr9z,GraniteReasoningParser._get_content_sections@s     &|44    "0 P P ()<=N)NOM&#'#:K$ CRC(888*;a*?.?!.C+9 $ P&*&@PPN$c.&9&9%9A%=%?%?@NSVTVSVDWWW):C> > 0 014= =r)__name__ __module__ __qualname____doc__rrstrrtupler1rintrr@boolrHrLr:r;r<r9 __classcell__)r(s@rr r sA "9      8++*?+ sTz3:% &++++.999 9 %SM 9 $C= 9"#9  9999x  s  t       c  d      @ @ @  @ @ @ @DL L L  L  L L L L \) ) )  )  )  )  ) ) ) ) V. . sTz3:sTz1 2. . . . . . . . rr )collections.abcrregexr transformersr0vllm.entrypoints.openai.chat_completion.protocolr'vllm.entrypoints.openai.engine.protocolr vllm.loggerrvllm.reasoningrrtloggerr rgrrrs%$$$$$000000A@@@@@######****** X  [ [ [ [ [ _[ [ [ [ [ r