DevOps

What Actually Gets Tokenized in SFT

tok.apply_chat_template(msgs, add_generation_prompt=True) I have copied that line more times than I can count without thinking about what any of it…