Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
[11]Reza Yazdani Aminabadi, Samyam Rajbhandari, Minjia Zhang, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Jeff Rasley, Shaden Smith, Olatunji Ruwase, and Yux... [31]Yaniv Leviathan, Matan Kalman, and Yossi Matias.Fast inference from transformers via speculative decoding, 2023. [32]Wantong Li, Madison Manley, James Read, Anki...