On the Role of the Activation Function in Transformer Self-Attention: An Empirical InvestigationA Structural View of Polynomial Expressiveness and Dynamic Weight GenerationOn the Role of the Activation Function in Transformer Self-Attention: An Empirical InvestigationA Structural View of Polynomial Expressiveness and Dynamic Weight GenerationTomio Kobayashi·Sep 3