transformer

Understanding Transformer Attention Architectures and Their Impact on Inference Memory Requirements

Two transformer models with the same parameter count can have very different memory requirements during inference. While the total parameter…