Enhancing Cross-Sectional Currency Strategies by Context-Aware Learning to Rank with Self-Attention

Transfer Ranking in Finance: Applications to Cross-Sectional Momentum with Data Scarcity

“…With conventional parameter sharing techniques, if a neural model is used to learn the source task, then a target model can be constructed by directly retaining most of its layers [70]. Motivated by the empirical superiority of context-aware LTR models using the attention mechanism over standard rankers [49,51], we utilise a hybrid approach -running the pre-trained source Transformer's encoder block 𝝃 𝑆 (•) as an additional feature extractor operating in parallel with the target Transformer's block 𝝃 𝑇 (•):…”

“…To train the source model needed for knowledge transfer, we make use of the same set of daily data relating to 30 currency pairs ∥ as per [4,51] obtained from the Bank for International Settlements (BIS) [3] spanning May-2000 to Dec-2021 which we again downsample to the weekly frequency. To measure risk aversion, we use the daily close of the VIX historical data from the Cboe Global Markets [12], where a week is labelled risk-off if it contains one or more days when the VIX is 5% higher than its 60-day moving average.…”

“…However, unlike the computer vision and NLP communities which have rapidly adopted transformerbased solutions in response to their domain-specific transfer learning problems, we note that a corresponding development in IR is surprisingly absent. Inspired by recent works [48,49] which exploits the Transformer's attention module to improve ranking performance as well as a first adaptation in the context of finance [51], we similarly make use of this mechanism and propose the Fused Encoder Networks (FEN). Central to the model's architecture is the usage of dual encoder blocks -one of which is optimised on a larger, but related, upstream dataset and then deployed in conjunction with the other encoder block that is calibrated on a smaller downstream dataset of interest.…”