LLM Forecasting: Calibration, Scoring, and Honest Evaluation
A convincing explanation is only the beginning. Learn how to record forecasts, calculate Brier scores, and test calibration without hindsight.
Read the articleThe Signal Journal / Topic
Language models
Language models can generate plausible forecasts, but plausibility and demonstrated forecasting skill are different questions. This collection examines how to define an evaluation, record probabilities before outcomes are known, and score a series of resolved questions without selecting only the successes.
Start with the calibration guide and its small worked example. Then visit the AI topic page for the broader research workflow, including evidence retrieval and human review. A useful evaluation explains the comparison baseline, information cutoff, question selection, and treatment of missing results. These choices help readers understand what a score supports and where uncertainty remains.
Explore the related topic page or return to the complete journal.