*`iXdZddlmZmZddlZddlZddlZGddejZdS)z This file helps integrate xgrammar in HF transformers package by extending transformers.LogitsProcessor, which is to be fed to `model.generate()`. )ListUnionNceZdZdZdeejeejffdZde j de j de j fdZ dS) LogitsProcessora LogitsProcessor for processing logits in transformers' generate() method. Example usage ------------- .. code:: python model_name = "Qwen/Qwen2.5-0.5B-Instruct" tokenizer = AutoTokenizer.from_pretrained(model_name) config = AutoConfig.from_pretrained(model_name) # This can be larger than tokenizer.vocab_size due to paddings full_vocab_size = config.vocab_size tokenizer_info = xgr.TokenizerInfo.from_huggingface(tokenizer, vocab_size=full_vocab_size) grammar_compiler = xgr.GrammarCompiler(tokenizer_info) compiled_grammar = grammar_compiler.compile_builtin_json_grammar() xgr_logits_processor = xgr.contrib.hf.LogitsProcessor(compiled_grammar) model.generate(prompt, logits_processor=[xgr_logits_processor]) For an end-to-end example, see folder `examples/hf_transformers/`. Notes ----- - Note that this LogitsProcessor can only be used once. For each `generate()` call, instantiate a new one. - Note that this implementation may contain extra overhead. compiled_grammarcg|_t|tr|n|g|_|jdjj|_d|_d|_d|_ dS)aInitialize the LogitsProcessor. Parameters ---------- compiled_grammar : xgr.CompiledGrammar | List[xgr.CompiledGrammar] One or more grammars compiled according to the given grammar and the model's tokenizer_info. rNF) matchers isinstancelistcompiled_grammarstokenizer_info vocab_sizefull_vocab_size token_bitmask prefilled batch_size)selfrs g/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/xgrammar/contrib/hf.py__init__zLogitsProcessor.__init__+sb35 *+;T B B Z  IYHZ  $5a8GR! input_idsscoresreturnctjdkr|jd_tjdkrjnjjz_tjjks Jdfdt jD_t jjj_ |jdjkr)tdd|jddjdzj sd _ nft jD]Q}j| s0||d }j| |sJRt jD]G}j| s&j|j |H|jj}|d kr|d }t j|j |j|d kr||}|S) z Accept token sampled in the last iteration, fill in bitmask, and apply bitmask to logits. Returns: scores: Logits modified with bitmask. rz@The number of compiled grammars must be equal to the batch size.cNg|]!}tjj|"S)xgrGrammarMatcherr ).0irs r z,LogitsProcessor.__call__..Ns9BC"4#9!#<==rz;Expect input_ids.shape[0] to be LogitsProcessor.batch_size.zGot z for the former, and z for the latter.Tcudacpu)lenr shaperr rangerallocate_token_bitmaskrr RuntimeErrorr is_terminated accept_tokenfill_next_token_bitmaskdevicetypetoapply_token_bitmask_inplace)rrrr! sampled_token device_types` r__call__zLogitsProcessor.__call__<so t}   " "'oa0DOt-..22&&+do=  " D*++t>>>Q?>>GLT_G]G]DM"%!;DOTMa!b!bD  ?1  0 0Mc+cc$/cccd  ~ H!DNN4?++ H H}Q'5577H$-aL$4M=+88GGGGGt'' P PA=#1133 P a 889KQOOOm( & YYu%%F '0B0E0Efm0T0TUUU & YY{++F  rN) __name__ __module__ __qualname____doc__rrCompiledGrammarrrtorch LongTensor FloatTensorr4rrrrrs8s/BDI\D]/])^"6%"26Er@s  dddddl2dddddr