Summary of two papers on Transformers and Titans by researchers at Google.
Sources :
Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin
https://arxiv.org/pdf/1706.03762
Titans: Learning to Memorize at Test Time Ali Behrouz†, Peilin Zhong†, and Vahab Mirrokni†
https://arxiv.org/pdf/2501.00663
Thes papers reference several other important works in this field. Please refer to the full paper for acomprehensive list.
Disclaimer:
Please note that parts or all this episode was generatedby AI. While the content is intended to be accurate and informative, it is recommended that you consult the original research papers for a comprehensive understanding.
