Pi<"dZddlZddlZddlmZmZmZmZddlm Z ddl m Z dZ ej dgdZGd d ZdS) z(Implementation of punctuation processingN)ListUnionTuplePattern)str2list) Separatoru!;:,.!?¡¿—…"«»“”(){}[] _mark_index)indexmarkpositionc eZdZdZefdeeeffdZe dZ e dZ e j deeeffdZ deeeefd eeeeffd Zdeeeefd eeeeeeffd Zd ed ed eeeeeffdZedeeeefdeededed eef dZdS) PunctuationaZPreserve or remove the punctuation during phonemization Backends behave differently with punctuation: festival and espeak ignore it and remove it silently whereas segments will raise an error. The Punctuation class solves that issue by "hiding" the punctuation to the phonemization backend and restoring it afterwards. Parameters ---------- marks (str or re.Pattern) : The punctuation marks to consider for processing (either removal or preservation). If a string, each mark must be made of a single character. Default to Punctuation.default_marks(). marksc0d|_d|_||_dSN)_marks _marks_rer)selfrs j/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/phonemizer/punctuation.py__init__zPunctuation.__init__/s '+ ctS)z1Returns the default punctuation marks as a string)_DEFAULT_MARKSrr default_markszPunctuation.default_marks4s rc<|jr|jStd)z!The punctuation marks as a stringzCpunctuation initialized from regex, cannot access marks as a string)r ValueError)rs rrzPunctuation.marks9s% ; ; ^___rvaluect|tr-tjd|jzdz|_d|_dSt|tr]dt||_tjdtj |jd|_dStd)Nz((z)|\s)+z(\s*[z]+\s*)+z;punctuation marks must be defined as a string or re.Pattern) isinstancerrecompilepatternrrstrjoinsetescaper)rrs rrzPunctuation.marks@s eW % % \Z (= (IJJDNDKKK s # # \''#e**--DK Z(P4;1G1G(P(P(PQQDNNNZ[[ [rtextreturncdtdtffd t|tr |Sfd|DS)zReturns the `text` with all punctuation marks replaced by spaces The input `text` can be a string or a list and is returned with the same type and punctuation removed. r)r*c^tjjd|S)N )r"subrstrip)r)rs rauxzPunctuation.remove..auxWs%6$.#t44::<< z&Punctuation.remove..\s!+++dD +++r)r%r!)rr)r0s` @rremovezPunctuation.removeOsl =c =c = = = = = = dC  3t99 ++++d++++rct|}g}g}t|D](\}}|||\}}||z }||z })d|D|fS)aRemoves punctuation from `text`, allowing for furter restoration This method returns the text as a list of punctuated chunks, along with a list of punctuation marks for furter restoration: 'hello, my world!' -> ['hello', 'my world'], [',', '!'] cg|]}||Srr)r2r3s rr4z(Punctuation.preserve..os88848888r)r enumerate_preserve_line)rr)preserved_textpreserved_marksnumr3rs rpreservezPunctuation.preserve^sz#4.."4 % %IC--dC88KD% d "N u $OO88888/IIrr3r<cttj|j|}|s|ggfSt |dkr2|d|krgt ||dgfSg}|D]}d}||dkr*||rd}n5||dkr)||rd}| t |||g}|D]]}| |j } | d|j | dd} } | | | }^||gz|fS) z+Auxiliary method for Punctuation.preserve()rAIBEN) listr"finditerrlengroup _MarkIndex startswithendswithappendsplitr r&) rr3r<matchesrmatchr preserved_liner rMprefixsuffixs rr9zPunctuation._preserve_lineqsr{4>48899 62:  w<<1  !1!1!3!3t!;!; 3c2233 3 C CEH ""tu{{}}'E'E"'"+%%$-- *F*F% LLCAA B B B B  DJJty))E"1Xty~~eABBi'@'@FF  ! !& ) ) )DD&--rsepr/c t|}g}d}|s|rm|sK|D]D}|s+|jr$||js ||jz}||Eg}n|sO|t jd|jdd|Dg}n|d}|j|kr|d} |dd}t jd|j| j} |jrF|d|jr&|ddt|j |d<|j dkr| |dz|d<n|j dkrT||d| z|s| |jrdn|jz|dd}|dz}n|j d krA|| |s| |jrdn|jz|dz}npt|dkr|d| z|d<nN|d} |dd}| | z|dz|d<n*||d|dd}|dz}|j|m|S) azRestore punctuation in a text. This is the reverse operation of Punctuation.preserve(). It takes a list of punctuated chunks and a list of punctuation marks, as well as the separator and strip parameters used by phonemize. It returns the punctuated text as a list: ['hello', 'my world'], [',', '!'] -> ['hello, my world!'] rr-r c3$K|] }|jV dSr)r )r2ms r z&Punctuation.restore..s%D[D[PQQVD[D[D[D[D[D[rr?NrBrDr@) rwordrKrLr"r.r&r r rGr ) clsr)rrSr/punctuated_textposr3 current_markr first_words rrestorezPunctuation.restores~~4 "e4 "2 " 11D /SX/dmmCH6M6M/#ch#**40000+ " &&rvc38RWWD[D[UZD[D[D[=[=['\'\]]] %Qx %,,!8D!!""IE6#sx;;Dx;DG$4$4SX$>$>;"&q'/CMM>/":Q#,33"&a.Q%.#55'..tAw~uAuPTP]P]^a^fPgPgAumpmu/vwww#ABBx!Ag%.#55'..tU7kdmmTWT\F]F]7krrcfck/lmmm!Agt99>>'+1gnDGG)-aJ#'8D&04&7$q'&ADGG$**473338D'Ci4 "e4 "nrN)__name__ __module__ __qualname____doc__rrr%rr staticmethodrpropertyrsetterrr5rrIr=intr9 classmethodrboolr^rrrrrs  5CeCL1 \``X`  \ \5g. \ \ \\ \ ,5d3i0 ,U3S >5J , , , ,JU49c>2JuT$s)_dS]N^=^7_JJJJ&.3.S.U49d:FV;V5W....BI5d3i0IJ'III!%S III[IIIrr)rb collectionsr"typingrrrrphonemizer.utilsrphonemizer.separatorrr namedtuplerIrrrrrns/. ............%%%%%%******5 #[ #00022 }}}}}}}}}}r