Calibration Theory · RL Algorithms and Methods · Decision Making
A blog and newsletter on The Calibration Review.
Latest research
RL Algorithms and Methods
Distributional RL With C51 and QR-DQN Compared
Two opposite designs for learning return distributions each have different strengths.
- Conformal Prediction Coverage Guarantees in PracticeMarginal coverage guarantees mask systematic failures in subgroups and changing conditions.
- Reward Model Calibration in RLHF PipelinesCalibrating reward models prevents optimization from amplifying measurement error.
- Reliability Diagrams for Probabilistic Forecast EvaluationCalibrated probabilities must match observed frequencies, and this diagram reveals when they don't.
- Offline RL Conservative Policies and Out-of-Distribution Action PenalizationConservative policies prevent overconfident decisions by penalizing actions outside training data.
Calibration Theory
View all →- Conformal Prediction Coverage Guarantees in PracticeMarginal coverage guarantees mask systematic failures in subgroups and changing conditions.
- Reliability Diagrams for Probabilistic Forecast EvaluationCalibrated probabilities must match observed frequencies, and this diagram reveals when they don't.
- Temperature Scaling vs Platt Scaling for Neural ClassifiersTemperature scaling needs one parameter; Platt scaling needs two.
- Bayesian vs Frequentist Uncertainty in Sequential Decision MakingBayesian updating handles sequential uncertainty better than frequentist methods.
RL Algorithms and Methods
View all →- Distributional RL With C51 and QR-DQN ComparedTwo opposite designs for learning return distributions each have different strengths.
- Reward Model Calibration in RLHF PipelinesCalibrating reward models prevents optimization from amplifying measurement error.
- Offline RL Conservative Policies and Out-of-Distribution Action PenalizationConservative policies prevent overconfident decisions by penalizing actions outside training data.
- Ensemble Methods for Epistemic Uncertainty in Deep RLEnsemble disagreement reveals where deep RL agents are ignorant and why it matters.