The DPO Subtraction
Follow chosen and rejected sequence probabilities through a reference-relative preference loss.
From Chapter 10: From Base Model to Assistant
Try it · toy model
The DPO Subtraction
Implement10 min
Read the concepts
Follow chosen and rejected sequence probabilities through a reference-relative preference loss.
From Chapter 10: From Base Model to Assistant
Try it · toy model
The DPO Subtraction
Read the concepts