Relative positional encoding for transformers with linear complexity

As part of his master’s program, Sacha Hakim was hired as an intern in the “Time Series” research area. He wrote a research thesis titled “Relative Positional Encoding in Linear Transformers,” which has not yet been published.

Reference: Sacha Hakim. 2025. Relative Positional Encoding in Linear Transformers. Master’s thesis. Master 2 in Mathematics, Vision, and Machine Learning (MVA).