01
When Financial Transformers Look Predictive
The model looked predictive when all assets were pooled together. A simple asset-identity baseline did even better, and performance fell to roughly random when the model had to predict changes within the same asset.
- Transformer pooled ROC-AUC
- 0.790
- Training-only static asset prior
- 0.824
- Pair-weighted within-asset ROC-AUC
- 0.492
Verified evaluation
Pooled performance did not survive within-asset testing
ROC-AUC from the frozen dissertation evaluation. Higher is better; 0.50 is chance-level discrimination.
Read: The static identity prior led the pooled comparison at 0.824, while the Transformer fell to 0.492 within assets.
Historical evaluation · measured predictions · derived ROC-AUC · 3 verified rows
